- Created Sunday, May 22nd 2016 @ 21:11:23
It looks like a week is not going to be nearly enough to let my bot settle into its actual place on the ladder.... Last night I went to bed in 7th place and about 15 hours later, I'm in 48th!
This is the first time I was vying for a spot in the playoffs, so I think I underestimated how important consistency was. Did anyone else take consistency into account? I definitely ignored it in exchange for a higher average max score.
- Updated Sunday, May 22nd 2016 @ 21:36:10
I think that a week definitely wasn't enough time if the intention was to have everyone settle back to their proper standings. It does make it more exciting though. For all of the bots who are close to making it into finals, a 3 or 4 game win streak might be enough to push them into the top 24.
As far as consistency goes DawgBlockerson is awful. I also had the situation where I was 11th when I went to bed and 49th when I woke up, although it was a couple days ago.
- Created Sunday, May 22nd 2016 @ 21:50:55
Sorry guys for you, but that is the nature of the sports in general. I'm sure that guys who objectively are top 16-24 may not make final 24 due to randomness.
- Created Sunday, May 22nd 2016 @ 22:02:03
Yeah.... while I'm sad I'm almost certainly not going to make it to the top 24 now, I can't exactly complain. The rules have been set for a really long time, I just wish I'd considered it before I optimized for points to the exclusion of all else. Live and learn!
- Created Sunday, May 22nd 2016 @ 23:17:02
I think luck plays a big role. You can get a couple of good or bad garbage lines that can be decisive in the game. Besides that, each bot needs many, many games to settle a position in the leader board. This is the nature of the rating system. Exciting nevertheless :) My bot also floated between 8 and 38 places...
- Created Sunday, May 22nd 2016 @ 23:26:30
My biggest concern is if everyone got the same number of games this week?
I have only had one game in the past six hours and I have been watching my rank slowly creep down as everyone else plays more games. As of now I will need two wins in the next 30 mins to make top 24. Although I probably shouldn't be in top 24 anyways as I have a bug in my final version :(
- Created Monday, May 23rd 2016 @ 00:06:51
Bug may not preclude your bot from entering top 24. Doppotra also has some severe bugs (just observe some of his games) but he is top 5 currently :)
- Created Monday, May 23rd 2016 @ 00:14:29
Yeah I also noticed Artoppod's issue. Unfortunately mine occurs whenever a piece becomes close enough to the top to overflow above the 20th row. A stupid mistake on my end I made right before the lockdown deadline. I shouldn't have been messing with my pathfinding so close to the cutoff.
Good luck to those in the finals though! I will be watching and cheering for somebody to take down the overlord hogeris.
- Created Monday, May 23rd 2016 @ 09:14:09
You can't influence these big fluctuations in leaderboard rank by any strategy. These fluctuations are so big because ELO's are still very close one week after lockdown. As a result you will always loose 7 to 9 points if you loose a game. Loosing 5 games in a row (which is quite common) will make you drop 11 positions if you start at position 10. Compare that with the leaderboard just before the lockdown: loosing 40 points at position 10 would make you drop only 4 positions.
I think resetting the leaderboard after lockdown is not necessary and not desirable. It serves no purpose.
On the other hand: bots which did not make it to the top 24 because of unlucky losses during the last day, would never have made it to the last 8. Almost half of the top 24 has less than 1% probability of making it to the top 8.
- Created Tuesday, May 24th 2016 @ 04:54:43
"I think resetting the leaderboard after lockdown is not necessary and not desirable. It serves no purpose." Doesn't it serve the purpose of ranking bots by their most recent version? Otherwise, the newcomers are unfairly punished by ELO's long-term memory. Seems like a better solution is to either extend the lock in period, increase the rate of games played, use a more accurate rating system, or some combination of those.
- Created Tuesday, May 24th 2016 @ 07:57:11
BlockParty did not make it to the top 24, although it really belongs there. And last lockdown, at the Texas holdem competition, my bot was at place 30 less than 24 hours before the end of the lockdown even though that bot was clearly top 5 and a potential winner, but then I was lucky to make it to the semifinals anyway. To be fair, BlockParty is not a top six bot, and all other top 12 bots made it to the semifinals.
To me as a participant this surely is frustrating, having spent considerable time and energy. Here I think theaigames can learn a lot from how the winners were determined at the old google ai challenges or the russianaicup.ru. Personally I would like to reduce the luck factor as much as possible, and don't like the knockout scheme either too much, and would prefer round robin of best X participants, or alternatively should be based on a lot more games like best of 200.
Maybe theaigames could poll its participants what they think, and then make a wise decision?
I do like theaigames a lot, so I hope I don't sound too negative, I surely hope to be constructive...
- Updated Tuesday, May 24th 2016 @ 09:10:37
Did someone take screenshot of pre-lockdown leaderboard?
- Updated Tuesday, May 24th 2016 @ 14:56:36
@Hohol: i have the ranking on the evening before lockdown: http://pastebin.com/cFzajNXE (although i am not a participant in AI Block Battle :D )
Also i agree with @DaFish about reducing the luck factor for finals. But I think that also can be done while using a knock-out system. Edit: one of the disadvantages of a knock-out-system is that bots that are good at exploiting weaknesses of weak bots have an advantage.
- Created Tuesday, May 24th 2016 @ 23:28:59
I think there is nothing wrong with a reset. But that should do both the pairing different and the k-factor should be (way) higher (as a starter) and/or you should be done recursively.
But I think we can agree that current system is not perfect.
- Created Wednesday, May 25th 2016 @ 22:53:01
Come on, you can not find a perfect system in sports. If we would really like to have objective rankings then 6 months of games non stop would give something like that. It would not be interesting.