- Created Thursday, April 16th 2015 @ 07:58:16
If instead of just playing a random match, wouldnt it be more fair to play eacht match in both directions? With human players this is not possible (they would remember the cards), but the bots can be reset. So if you would just create a random list of 200 hands for both sides and 200 tables players could replay the match with the bots on the other sides of the table. Both win one: draw. Of you win both: win. I've noticed your elo system handles draws aswel (for warlight). What do you think about this idea? It would pretty much remove the chance element from the competition.
- Created Thursday, April 16th 2015 @ 23:45:18
I completely agree! Poker is a game with a particularly high variance which makes it hard to get an accurate elo measure and position in finals. As this is an AI competition, we should aim for as low luck factor as possible.
- Created Friday, April 17th 2015 @ 00:23:56
After some thought, I think your idea becomes faulty if it gives ELO for 1-1 (draw) score. If a cooler happens in an early hand (a cooler is defined as "A situation in which a player holds the second best hand, so strong considering the circumstances, that they are apt to lose the maximum with it no matter how they play it."), then the game would probably end in a draw even if a very high ranked player meets a very low ranked player, and the low ranked player will get ELO points. For example, if the players are dealt AA and KK in the first hand, they would probably go all-in in both games, and it will end in a draw no matter what. Therefore, I think only 2-0 scores should give ELO points, while draws should give +0 for both players.
- Created Friday, April 17th 2015 @ 04:19:51
I like the idea, but it would likely result in a relatively large amount of draws between bots on a similar level. I agree with Alfy that a draw should probably not give any ELO, but that would mean a large amount of games effectively not counting for anything.
I'm not sure if this is actually a problem or not, it sounds less exciting though.
- Created Friday, April 17th 2015 @ 09:18:47
It might take longer but you get an actual objective measurement of you AI strength.
- Created Friday, April 17th 2015 @ 09:34:02
I think also http://www.computerpokercompetition.org/ uses this.
I did not understand the reasoning of Alfy and Tiltless about draws not counting for ELO. If a weak player manages to draw against a strong player then of course the weak player's ELO should get up.
The "cooler" situation I don't understand at all: today if we get this situation the weak player would get 100% and an enormous boost in ELO. Example: extremely weak player meets extremely good player, and wins due to cooler, and ELO improves with about 20 (strong player's ELO reduced with 20). Even if next game the strong player wins; the weak player's ELO is reduced with almost nothing, strong player gets almost nothing. Net effect: the weak player has gained almost 20 points in a 1-1 game.
With kevin's proposal the cooler situation leads to a draw, so the weak player only wins about 10 ELO points, thus the luck factor was reduced.
Bots on similar level would result in 50% of the games ending up in draws.
A disadvantage is that games will take twice as long, so the number of played games per day (which is already low) will probably get even lower. And of course don't forget the Chris Moneymaker effect, I think it is good for theaigames.com that any bot has a chance to win the 1024 euro (for free!), hopefully more participants will enter:)
- Updated Friday, April 17th 2015 @ 15:32:23
I agree that such method of playing games reduces luck factor and can be used in the finals, but of course it can't give "an actual objective measurement of you AI strength".
There are another ways to reduce luck influence even more, such as calculating all-in chipEV difference every time players got all-in and produce results not based on who won the game, but on how many EV chips bots won (I believe these both methods are used by http://www.computerpokercompetition.org).
It could reduce the interest of beginners though, making the games look too complicated.
- Created Friday, April 17th 2015 @ 20:47:41
You are right skel, forgot about the random hands. The only way to objectively measure it is to play every possible hand both ways, but with the current timelimit of 0.5 s / move that's far from possible.