- Updated Monday, January 16th 2017 @ 15:12:24
Apparently the game queue is always filled with an amount of games that takes multiple days to get done. That's why it takes a long time for new bots to get ranked games going. This means bots who were better than the bots around them at the time of lockdown have a huge advantage. They are free to climb while the currrent top bots are still getting matched against eachother and are stuck around 1400 elo for this reason.
Also after the game queue is emptied the top bots will be playing against random opponents for the same duration as it took for the game queue to get emptied and because of this gain less elo for their wins than they would if they got games against the bots who were able to climb.
Depending on how many "old" games are being played it might not even be possible for the best bots to get to the top 24.
- Created Monday, January 16th 2017 @ 14:33:06
Thanks to your post, i just realized my bot was only facing strong ones for now. Let's hope something can be done about it or that the "games based on previous rankings" queue is short.
- Created Monday, January 16th 2017 @ 14:57:17
If this is indeed the case I hope that this could be fixed quicly. It would be a bit dissapointing if almost none of the top 24 bots before lockdown are in the semi-finals. Also I hope the rate of games will be increased. My bot has only played one game so far since the lockdown, not realy helping me get back in the top 10 ;)
DeveloperCreated Monday, January 16th 2017 @ 15:16:00
The old queue is almost empty, after that the rates for Tic Tac Toe are increased, so more matches are played. In the case that at the end of the week, the leaderboard is not an accurate representation of the strength of the bots, we will postpone the finals by a week.
- Created Monday, January 16th 2017 @ 16:50:08
It will be nice to crete a day in the middle of 1st round when it possible to change own bot. Some king of timeout.
- Created Tuesday, January 17th 2017 @ 22:28:32
I mentioned this in all previous competitions: resetting the ratings to 1400 serves no goal at all. It sounds fair, because all bots have the same chance, but in practice it does not work, because not enough games can be played in a week. And not resetting the game queue makes the problem even worse.
The result is what we saw in the blockbattle competition: some evident top 10 bot do not reach the semi-finals, and some much weaker bots reach the semi-finals by accident.
Organizers: for the sixth time: please learn from these comments, and don't do this anymore! Instead, just lock down the competition, and don't reset ratings.
- Created Tuesday, January 17th 2017 @ 23:12:32
@AdRiskbot, fully disagree with you. Whole two days strong bots were beating each other fail and win with almost 50/50 ratio, and, exactly because of this, they stood for that time around 1400 elo. At the moment, they rocketing to their places, as it has to be from the very beginning. In my mind, it is the only problem, even though it is a big one, the resetting of the rating is fine and right, 'cose otherwise, it would be like one of the runners in 100m sprint started right on the finish line. No one the world won't organise, watch and participate in such a sprint, right?
- Created Tuesday, January 17th 2017 @ 23:33:07
@Evedel: This week's games are not about sprinting to the top, they are just about qualifying for the semi-finals. No one in the world is interested in these games at all. The only thing that matters is who is in the top 24 next Sunday. The ones that deserve to be in the top 24 are the ones which were in the top 24 just before lockdown, right? So what's the point in resetting scores?
- Created Tuesday, January 17th 2017 @ 23:59:36
I agree with AdsRiskBot. The element of luck has IMO historically played a much too high role in the selection of the 24 semifinalists. The biggest drawback is the combination game queue mechanism + limited number of games combined + the use of Elo rating (which fluctuates a lot).
It can happen that one day before the end of the lockdown a top bot is near #1 position, so it has to play against all other top bots. If the bot is unlucky and loses a number in a row it can easily get kicked out of top 24 (this happened with my bot at Block Battle...)
This competition will be even worse as the rating differences are not so high.
I like the format of russianaicup.ru. Transformed to theaigames this could be something like:
- at the moment of lockdown, the top 200 bots are selected using the ratings at the moment of lockdown. They play a fixed number of rounds, where in each round every bot meets a random other bot, the winner gets 2 points, the loser 0, draw = 1. This is done for a couple of days.
- then 60 bots are selected, they play round robin for a number of rounds.
In russianaicup that's it, the winner is the one with most points. I think these finals are pretty exciting to watch. Compared to knockout system, more programmers will have interest in how it goes. But in theaigames spirit you could then select the top 24 bots for the knock out system.
The big advantage is that all bots play on equal terms and the luck factor is reduced: they play the same amount of games in the same conditions.
- Created Wednesday, January 18th 2017 @ 00:02:37
@AdsRoskbot, In my opinion, it's the same, "The only thing that matters is who is in the top 24 next Sunday" absolutely true and in this case (if the queue was cleaned and all play with random bots from the beginning) it would be that one, who "running faster" (winning more) then other bots over the same amount of time.
If they really deserve it, they will do it easily in one week (again, with a clear queue), because of the elo system just presents normal expectation of winning.
- Created Wednesday, January 18th 2017 @ 16:05:47
@Evedel, "If they really deserve it, they will do it easily in one week", that might be true if it does not matter who begins and if each bot will roughly play the same amount of matches. However, these are not both the case, for example since the lockdown my bot currently has 'only' played 6 matches while there are bots who have played at least 13 matches. Also, since the game is skewed in favour of the starting player it might possible for a bot to just have bad luck and face few opponents and in most of those games be the second player. With the current overall rate of games a week might simply not be enough for the elo ratings to stabilize.
DeveloperUpdated Wednesday, January 18th 2017 @ 22:23:10
We're aware that the way the competition finals are set up currently results in a sub par experience and we'll be changing this in the near future.
The double elimination tournament scheme will be replaced with a round robin tournament starting with Hack Man. Furthermore, we'll be updating the ranking algorithm before unveiling the next competition.
We'll continue to make incremental changes with each future competition based on your feedback.
DeveloperCreated Thursday, January 19th 2017 @ 08:48:14
We're also keeping a close eye on the ratings during the lockdown. If they haven't stabilized enough and the end of the week, we'll postpone the finals an extra week.
- Created Friday, January 20th 2017 @ 07:22:05
Have no idea, why you use ELO Ranking system with your tournament format. ELO good enough but it use in long time competition. And if you want to make fair placement you need to create play queue round by round. After every round 1-st place must play with 2-nd, 3rd vs 4th and etc. But you no need ELO for that, its simple swiss system. So if you use it, the placement after 28-30 matches will be reflect the relative strength of bots
- Updated Saturday, January 21st 2017 @ 19:21:45
Looks like the game queue went down?
I also agree with extending the lockdown phase. This competition has more participants than past competitions, and the games are also playing at a slower rate than in the past. It would make sense to be flexible and extend it by a week.