- Created Thursday, December 31st 2015 @ 23:46:52
With the finals coming up there's been a lot of activity coming up again. I was just curious what kind of general strategies you guys think are good in Heads-up play.
I don't have much experience playing poker, but from what I've read I learned heads-up play is a lot more aggressive than playing poker with 6+ people. From what I've watched in the competition I was pretty impressed by how successful "all-in" type strategies are.
Also I am curious if anyone tried out a machine learning strategy for this. Most of the papers and methods I've read online applied to Limit Texas Holdem. Not sure if the same techniques used in Limit would apply to No-Limit since No-Limit Holdem is much more complex.
- Updated Friday, January 1st 2016 @ 15:21:48
I don't have any "real" poker experience, but my Heads-up Omaha bot ended in the third place in the finals. For this competition, I ported by Omaha bot to Texas Holdem. Apart from the difference in hands (of course), the unlimitedness is the biggest difference.
Last week, I implemented a counter strategy against bots, which go all-in. The most difficult thing is that you don't get a second chance if you call on an excessive raise of your opponent and you loose: then it's game over immediately.
I have a simple counter strategy now, which works pretty well (60-65% wins) against bots (like ApocalypticAces_Texas), which always go all-in in the preflop. I optimized my fold threshold (in terms of winning probability of my hand) with a simulation program, which just simulates the all-in behavior.
I'm still wondering how to counter opponents (like MNPuck_Texas), which go all-in often, but not always. That's less straightfoward to simulate.
I also notice some bots (like BetterCallSoul), who start going all-in later in the game. Not sure why they do that, but it seems pretty successful.
I don't use real machine learning algorithms, but I do use a lot of data analysis on historic games. That's much more important in these poker challenges than for example for the Warlight challenges.
- Created Friday, January 1st 2016 @ 20:03:59
Yeah the all-in bots are surprisingly hard to counter haha. I guess the main thing is that the strategy has such high variance. My strategy tries to gauge how loose or tight an opponent is by how often they bet in a hand, and I use this to estimate their hand strength and make a decision on what to do. Haven't measured my win-rate versus all-in bots but I think this works okay.
Definitely testing my bot takes a lot of time. I've been using the poker engine and playing 100 games takes like 30 minutes for me or so. I wish I had the instant feedback of being able to play 100 games in like 10 seconds... lol.
But yeah, a good bot definitely has to be able to counter the all-in strategies.
- Updated Sunday, January 3rd 2016 @ 02:05:55
What I did to speed up test games was decrease the number of simulations, which I use to estimate my winning probability for a hand/table. My bot simulates 6,000 times, but when testing, it simulates only 200 times. That makes the probabilities a bit less accurate, but playing a game takes only about 1 second.
- Updated Sunday, January 3rd 2016 @ 16:36:16
Great idea! Yes I will try this out.
- Created Tuesday, January 5th 2016 @ 01:50:39
I think the reason same bots might go all-in a lot later in the game is because they're built to use a push/fold strategy commonly used by sit and go poker players (especially heads up).
Found this randomly on google that seems to describe it fairly well: http://www.tournamentterminator.com/tournament-strategy/sng-strategy/sitn-go-heads-up-strategy-sage/
- Created Tuesday, January 5th 2016 @ 23:05:04
Ah, interesting! Thanks for sharing!
- Updated Monday, January 11th 2016 @ 00:20:40
Wow, DaFish, that is really cool. I read about CFRM before the competition but was too afraid to attempt it haha. That is super impressive you pulled it off.
For your algorithm I was just wondering:
How did you end up defining each information set?
And how did you upload all the precomputed values to the servers? Did you store everything as strings in a file or did you use some compression techniques?
- Created Monday, January 11th 2016 @ 00:37:30
Regarding the info sets: I would like to keep that secret for the time being:). But at least preflop it has an info set for every card combination and bet sequence, with simplification that raises (except the last one) are "bundled" (ie. no difference between small raise or big raise).
As for the data: yes, I stored everything in a bunch of strings; I first zipped the data, then base64 encoded it, then split into separate strings in generated java source files. Java has unzip methods built-in which I could use to decompress at runtime . My guess is that the max limit is 2048 KB zipped; I could not upload 2166 KB and had to work very hard to bring it down to 2041 KB and that was accepted.
- Created Monday, January 11th 2016 @ 03:27:47
That's awesome DaFish, thanks so much for sharing. :)
- Created Wednesday, January 13th 2016 @ 23:00:33
That is amazing Dafish. Big congratulations on pulling off a CFR implementation. In fact, it is something I am very interested in doing myself, though I didn't have enough time for this competition. You see, I am very much a beginner at programming. It was only in the last two days of November that I uploaded my first bot to this site (for Connect four). I played around with that for a while and then started working on a holdem bot.
I was only just starting when I noticed the post informing of us of the date for final submissions for this competition. That was a big motivator to work as hard as I could to make a good bot. In the end my bot is just a very simple heuristically programmed bot probably similar to your early versions. But I am learning as much as I can about CRF and trying to understand it & thinking about how I should abstract the game state and bet sizes. I wonder if it is possible to also fit blinds and starting stacks into the abstraction (so it can play the meta game of trying to win the match instead of just trying to maximize the value from the hand). There are so many details to be worked out!
Anyway, I'm just trying to show I understand all the hard work you must have gone through to implement your bot. I really do say it is a job well done. Thank you very much for sharing your path. Your bot seems to be doing very well now. Certainly showing better than it did earlier in the tournament. Good luck the rest of the way!
- Created Thursday, January 14th 2016 @ 18:32:45
Hi Nevik, Thanks for sharing details about your bot. You must be a talented programmer, your connect four-bot is excellent and also your holdem bot does very well!
Optimizing CFR to win the match instead of maximizing the current hand: to do that in a mathematically optimal way is not so easy I think. Perhaps you can start bottom up by calculating models for many stack sizes, starting with small ones.
- Created Thursday, January 14th 2016 @ 21:15:23
Curious how many ran Monte Carlo simulations for predicted hand value versus just brute forcing it? AdsRiskBot mentioned 6000 simulations, I used brute force and it was pretty quick, even for the flop.
- Created Thursday, January 14th 2016 @ 23:36:13
Originally I brute-forced my hand value scores as well, but I switched to Monte Carlo simulations to make local testing faster.