Page 1 of 1

How ratings work

Posted: Sat Oct 25, 2008 4:08 pm
by Charlie Reams
The general principle is that your rating should go up when you win and down when you lose. More precisely, the rating change depends on how much you win (or lose) by, the rating of your opponent and the length of the game (longer games cause larger changes.) If you're interested in the mathematical details, you can read the full Apterous ratings design document.

Re: How ratings work

Posted: Sat Oct 25, 2008 8:45 pm
by Ian Volante
Looks very well thought through.

Re: How ratings work

Posted: Sat Oct 25, 2008 9:43 pm
by Gavin Chipper
Just had a quick look at it - omega and w look very similar!

Re: How ratings work

Posted: Sun Oct 26, 2008 6:15 pm
by Michael Wallace
I think you're missing a footnote (the 1 after the thing about draws), or I'm just being blind. We all know how likely that is.

Re: How ratings work

Posted: Mon Oct 27, 2008 9:17 am
by Charlie Reams
Michael Wallace wrote:I think you're missing a footnote (the 1 after the thing about draws), or I'm just being blind. We all know how likely that is.
Hmm. Seems you can't put footnotes inside equations. It's not very important anyway.

Re: How ratings work

Posted: Mon Oct 27, 2008 1:52 pm
by Michael Wallace
Charlie Reams wrote:
Michael Wallace wrote:I think you're missing a footnote (the 1 after the thing about draws), or I'm just being blind. We all know how likely that is.
Hmm. Seems you can't put footnotes inside equations. It's not very important anyway.
It was to me, Charlie, it was to me :(

Re: How ratings work

Posted: Tue Oct 28, 2008 3:24 pm
by Martin Gardner
The ratings seem to work in a very similar way to the ISC (Internet Scrabble Club) which of course, you're a member of. In the same way, you need 11 games to have a full rating, just like on the ISC.

Re: How ratings work

Posted: Tue Oct 28, 2008 3:26 pm
by Charlie Reams
Yep, I bastardised their system. I had to add quite a bit of stuff to make it suitable but some of the formulae are identical.

Re: How ratings work

Posted: Tue Oct 28, 2008 5:44 pm
by Ian Volante
I did have a thought that in some situations, a guaranteed loss of rating for a defeat almost seems unfair. Say if Conor beat Prune by one point, surely Prune could justifiably argue for an increase (But maybe muck up the solution), although I assume the decrease would be vanishingly small at the moment.

Re: How ratings work

Posted: Tue Oct 28, 2008 6:17 pm
by Charlie Reams
Ian Volante wrote:I did have a thought that in some situations, a guaranteed loss of rating for a defeat almost seems unfair. Say if Conor beat Prune by one point, surely Prune could justifiably argue for an increase (But maybe muck up the solution), although I assume the decrease would be vanishingly small at the moment.
The problem with that is that you always get some highly rated players who are then discouraged from playing because they stand to gain so little and lose so much. With a fixed minimum gain for a win, there's always an incentive to play. The downside of that is rating inflation, but that's just the nature of the beast.

Re: How ratings work

Posted: Tue Oct 28, 2008 6:38 pm
by Ian Volante
Charlie Reams wrote:
Ian Volante wrote:I did have a thought that in some situations, a guaranteed loss of rating for a defeat almost seems unfair. Say if Conor beat Prune by one point, surely Prune could justifiably argue for an increase (But maybe muck up the solution), although I assume the decrease would be vanishingly small at the moment.
The problem with that is that you always get some highly rated players who are then discouraged from playing because they stand to gain so little and lose so much. With a fixed minimum gain for a win, there's always an incentive to play. The downside of that is rating inflation, but that's just the nature of the beast.
I missed that there was a minimum gain, thought there were just maxima for some reason.

Re: How ratings work

Posted: Tue Oct 28, 2008 8:53 pm
by Gavin Chipper
Charlie Reams wrote:
Ian Volante wrote:I did have a thought that in some situations, a guaranteed loss of rating for a defeat almost seems unfair. Say if Conor beat Prune by one point, surely Prune could justifiably argue for an increase (But maybe muck up the solution), although I assume the decrease would be vanishingly small at the moment.
The problem with that is that you always get some highly rated players who are then discouraged from playing because they stand to gain so little and lose so much. With a fixed minimum gain for a win, there's always an incentive to play. The downside of that is rating inflation, but that's just the nature of the beast.
Does that contradict Ian's point though? Fixed minimum gain for winning doesn't have to imply a loss for defeat does it?

Re: How ratings work

Posted: Thu Oct 30, 2008 5:45 pm
by Julie T
I played Kirk today, who's rating is astronomically more than mine. First time I'd played a Goatdown game, which was very interesting.
Although I obviously lost hugely, my rating didn't go down. I might not feel so nervous about playing with the greats now! :)

Re: How ratings work

Posted: Mon Nov 03, 2008 1:12 am
by Mikey Lear
I think you should add a few hundred onto everyone's rating, then multiply it by 10. That way normal people would have a rating of somewhere around a hundred instead of -100. It might be a small incentive to people who like the game but not enough to start learning the two letter extensions for LOATHED not to have a negative rating. A similar move was adopted by arcade game designers a few years after Space Invaders - everyone likes having extra 0's on their score.

Re: How ratings work

Posted: Mon Nov 03, 2008 1:28 pm
by Gavin Chipper
Mikey Lear wrote:I think you should add a few hundred onto everyone's rating, then multiply it by 10. That way normal people would have a rating of somewhere around a hundred instead of -100. It might be a small incentive to people who like the game but not enough to start learning the two letter extensions for LOATHED not to have a negative rating. A similar move was adopted by arcade game designers a few years after Space Invaders - everyone likes having extra 0's on their score.
I've never liked pointlessly high scores that can be cancelled down. You could add numbers on I suppose (starting at 600 is presumably arbitrary) but multiplying by 10 is going too far!

Re: How ratings work

Posted: Tue Nov 04, 2008 3:20 pm
by Julie T
Does your rating go down, simply by declining a challenge? Or does the Daily Duel affect it? Or others' ratings changes, regardless of whether you're playing them or not?
I'm sure I lost 10 or 20 points yesterday that I can't explain otherwise. I'd lost points by being beaten by Waldorf, but then it went down again.

Re: How ratings work

Posted: Tue Nov 04, 2008 3:32 pm
by Charlie Reams
Julie T wrote:Does your rating go down, simply by declining a challenge? Or does the Daily Duel affect it? Or others' ratings changes, regardless of whether you're playing them or not?
I'm sure I lost 10 or 20 points yesterday that I can't explain otherwise. I'd lost points by being beaten by Waldorf, but then it went down again.
Nope, your rating can only go down when you play. It's possible that this is a bug, or maybe you just misremembered. If you suspect it might be a real bug, you can jot down your rating at the end of each day and then check back the next. Or it might have just been a one-off database glitch.

Re: How ratings work

Posted: Tue Nov 04, 2008 4:30 pm
by Julie T
Charlie Reams wrote:
Julie T wrote:Does your rating go down, simply by declining a challenge? Or does the Daily Duel affect it? Or others' ratings changes, regardless of whether you're playing them or not?
I'm sure I lost 10 or 20 points yesterday that I can't explain otherwise. I'd lost points by being beaten by Waldorf, but then it went down again.
Nope, your rating can only go down when you play. It's possible that this is a bug, or maybe you just misremembered. If you suspect it might be a real bug, you can jot down your rating at the end of each day and then check back the next. Or it might have just been a one-off database glitch.
Thanks, Charlie. I'll try to remember my rating to see if it happens again. I've put it on a notepad doc. :)

Re: How ratings work

Posted: Wed Dec 17, 2008 10:42 am
by Howard Somerset
Couple of quick questions.

Does the ? against my rating mean that I haven't yet played the requisite 11 games for the full system to kick in?
Is the fact that my rating went down following a win, which contradicts what is said in the first post, also due to the fact that I've yet to play the requisite 11 games?

The rating moving the wrong way is similar to what happens on a chess site where I play regularly. People there often notice that for the first few games, their ratings often move in an unexpected direction following a game, and the description of the chess rating system seems to be very similar to the one in use here at apterous.

Re: How ratings work

Posted: Wed Dec 17, 2008 1:58 pm
by Charlie Reams
Howard Somerset wrote:Couple of quick questions.

Does the ? against my rating mean that I haven't yet played the requisite 11 games for the full system to kick in?
Is the fact that my rating went down following a win, which contradicts what is said in the first post, also due to the fact that I've yet to play the requisite 11 games?

The rating moving the wrong way is similar to what happens on a chess site where I play regularly. People there often notice that for the first few games, their ratings often move in an unexpected direction following a game, and the description of the chess rating system seems to be very similar to the one in use here at apterous.
Yes and yes :) The apterous rating system is a refinement of the Internet Scrabble Club rating system, which is a perversion of the ELO system used in chess, so your intuition is probably correct.

Re: How ratings work

Posted: Wed Dec 17, 2008 10:24 pm
by Ian Volante
Mikey Lear wrote:I think you should add a few hundred onto everyone's rating, then multiply it by 10. That way normal people would have a rating of somewhere around a hundred instead of -100. It might be a small incentive to people who like the game but not enough to start learning the two letter extensions for LOATHED not to have a negative rating. A similar move was adopted by arcade game designers a few years after Space Invaders - everyone likes having extra 0's on their score.
You still end up in a similar situation where Kirk has a rating of 15,435 while S.O. Else has a rating that tends towards 5,000, and everyone knows that 5,000 is kack.

Then again, I've just convinced myself otherwise. On balance however, I do prefer Elo ratings or similar, but positive scores I think would keep more rubbish players around. Cos that's what we want of course.

Re: How ratings work

Posted: Fri Dec 19, 2008 12:10 am
by Paul Howe
Maybe use a smaller value of beta for the conundrum attacks, as the expected victory margin would be quite a bit higher than a regular game?

Just had a 100-rounder vs Prime and it produced a net rating swing of about 200 points, so not complaining too vociferously.

Re: How ratings work

Posted: Fri Jan 02, 2009 7:23 pm
by Howard Somerset
Having previously reported my rating going down after a win, today I was soundly thrashed, by a human this time, and my rating then went up. :) I think it's better not having played 11 games yet.

Re: How ratings work

Posted: Wed Jan 07, 2009 10:27 am
by Phil Reynolds
My rating seems to have had a major boost today! :)

So does everyone else's. :(

Re: How ratings work

Posted: Wed Jan 07, 2009 12:59 pm
by Howard Somerset
Phil Reynolds wrote:My rating seems to have had a major boost today! :)

So does everyone else's. :(
Are we now being centred on 1200 instead of 600, charlie? If so, that ties in with the chess system with which I'm more familiar.

Re: How ratings work

Posted: Wed Jan 07, 2009 5:37 pm
by Charlie Reams
Howard Somerset wrote:
Phil Reynolds wrote:My rating seems to have had a major boost today! :)

So does everyone else's. :(
Are we now being centred on 1200 instead of 600, charlie? If so, that ties in with the chess system with which I'm more familiar.
Yep. People having negative ratings seemed a bit silly, so I just shifted everyone up a bit (so that Rex hovers around 2000.)

Re: How ratings work

Posted: Wed Jan 07, 2009 6:58 pm
by JimBentley
Not sure if you've noticed yet, Charlie, but the top players page and the profile pages still seem to be retaining the old ratings.

Re: How ratings work

Posted: Tue Jan 13, 2009 6:01 pm
by Howard Somerset
Just noticed that my rating is now 412, having just logged in. Last time it changed was yesterday, when I played Nick, and I seem to remember it was in the region of 1100. I've still got a ? by my rating, so are provisionals being done differently now?

Re: How ratings work

Posted: Mon Jan 19, 2009 10:39 am
by Neil Zussman
How do the ratings reflect ability on the Show?
Would a rating of 2000+ reflect a series winner, perhaps, and a 1700+ an octochamp, or am I way off the mark? Or is there no correlation at all because there are some people who find apterous much much easier/ harder than the actual show?

Re: How ratings work

Posted: Mon Jan 19, 2009 10:50 am
by Michael Wallace
Neil Zussman wrote:How do the ratings reflect ability on the Show?
Would a rating of 2000+ reflect a series winner, perhaps, and a 1700+ an octochamp, or am I way off the mark? Or is there no correlation at all because there are some people who find apterous much much easier/ harder than the actual show?
I've wondered this myself. Obviously they're a pretty vague guide (for example, your rating can be really high even if you only play crap bots 2 million times or whatever, or pretty low even if you're quite good but only ever play Kirk). I think your estimates are a bit high - looking at the list of top players I'd say 1700+ is series winner territory. As for what would indicate an octochamp, I suppose it depends on how you look at it. When I watch at home I would comfortably outscore (all qualifications about not really playing the people on the show, etc. etc. having been made) most of a series, and usually regularly outscore around half of the finalists, but I'm a bit rubbish (my rating is 1355, apparently), and certainly wouldn't be anywhere close to 100% certain I'd be an octochamp if I went on (again). Obviously my analogy isn't great, since one's apterous ability will presumably a bit better than how one would do on the show, but I might even go as far to say as people who are 1400+ would 'probably' become octochamps, and only wouldn't if they were unfortunate enough to run into a series finalist in their heats.

I'd be quite interested to see what happens when we get our first apteroussers (aptees?) going on the show, and get data on how the scores translate (although the amount of luck involved for anyone but the insanely good will make it fairly hard to infer much).

Re: How ratings work

Posted: Mon Jan 19, 2009 2:13 pm
by Charlie Reams
I think the best guide (until I do something cooler) would be to look at your average maxes per game for 15 Round games, which is on your user page. Anything above 10 is solid octochamp territory.

Re: How ratings work

Posted: Sun Feb 22, 2009 12:00 am
by Martin Gardner
Neil Zussman wrote:How do the ratings reflect ability on the Show?
Would a rating of 2000+ reflect a series winner, perhaps, and a 1700+ an octochamp, or am I way off the mark? Or is there no correlation at all because there are some people who find apterous much much easier/ harder than the actual show?
That wouldn't really suggest no correlation, just a weak(ish) one. Other than that, I don't really have an answer. However, on Apterous you can play as many games as you like, even if you lose the first one. On the real show, you lose once, it's over. Nobody's won a series *not* unbeaten since 2001 (Series 45).

Re: How ratings work

Posted: Mon Feb 23, 2009 2:28 pm
by Karen Pearson
I'm sure this has been answered somewhere before but I can't find it so apologies for repeating it.

What happens to your rating if you close a game during play? I was beating someone (who shall remain nameless) 99-0 today and that person closed down the game during the conundrum. At least, I assume he closed it down as he didn't answer my questions about it in chat and didn't take up my offer to resume (although he was there). I wondered if that meant that the game no longer counts or if the person forfeiting the game is counted as losing (but I didn't notice my rating increase).

Not that I'm too bothered about my rating, I just wanted to clarify what happens. Thanks.

Re: How ratings work

Posted: Mon Feb 23, 2009 2:50 pm
by Roxanne
it's a bit of a pain if people can close a game when they're losing and it doesn't affect their rating, but it's also a pain if your rating goes down because plusnet is rubbish and you lose your connection several times a day.

Re: How ratings work

Posted: Mon Feb 23, 2009 3:42 pm
by Charlie Reams
Yeah, what Roxanne said. I decided that the number of twats who'll just shut a game when they're losing is much smaller than the number of people who drop out of the game for legitimate reasons. Repeat offenders quickly develop a reputation as opponents to avoid (not naming names) so it catches up with them in the end.

Re: How ratings work

Posted: Mon Feb 23, 2009 4:39 pm
by Karen Pearson
Fair enough. No objections to that at all.

I don't play that often against other people but I do try to oblige if challenged by someone else. I guess I might just decline if that person offers again. Seemed rather unsporting to me but then losing at Countdown has never bothered me (fortunately, given how often I lose!).

Re: How ratings work

Posted: Mon Mar 02, 2009 10:47 pm
by Julie T
I thought that this query best fitted in here, since I'm terminally nervous about starting new threads! :lol:

Not exactly a question of earth-shattering importance, but why is my rank given as 69 on the player list http://apterous.org/top.php
but 153 overall http://apterous.org/viewuser.php?user=375

Charlie, have you, for instance, given imaginary placings to some of the greats who don't play on apterous??

Re: How ratings work

Posted: Mon Mar 02, 2009 10:50 pm
by Michael Wallace
Julie T wrote:I thought that this query best fitted in here, since I'm terminally nervous about starting new threads! :lol:

Not exactly a question of earth-shattering importance, but why is my rank given as 69 on the player list http://apterous.org/top.php
but 153 overall http://apterous.org/viewuser.php?user=375

Charlie, have you, for instance, given imaginary placings to some of the greats who don't play on apterous??
My guess is that the top players page doesn't count people who haven't played 11 games, whereas the ranking listed on your player page does (cf. the enormous increase in discrepancy between the top.php rankings and player rankings that happens when you look at people rated below 1433).

Re: How ratings work

Posted: Thu Mar 12, 2009 10:42 pm
by Dominic Colley
Michael Wallace wrote:I'd be quite interested to see what happens when we get our first apteroussers (aptees?) going on the show, and get data on how the scores translate
I'm heading to Leeds for recording on Tuesday, so I'll let you know! However, my rating (950ish) suggests I could be in the record books as the lowest scoring contestant of all time, which doesn't fill me with confidence :roll:

Re: How ratings work

Posted: Thu Mar 12, 2009 11:33 pm
by Jimmy Gough
Dominic Colley wrote:
Michael Wallace wrote:I'd be quite interested to see what happens when we get our first apteroussers (aptees?) going on the show, and get data on how the scores translate
I'm heading to Leeds for recording on Tuesday, so I'll let you know! However, my rating (950ish) suggests I could be in the record books as the lowest scoring contestant of all time, which doesn't fill me with confidence :roll:
Don't you dare take away Hazel's record :x And good luck, I very much doubt you'll be anywhere near as bad as you think, most contestants are apterous-less you have a HUGE advantage. I'm going on soon and I'm also extremely nervy.

Re: How ratings work

Posted: Fri May 29, 2009 5:38 pm
by Tom
Didn't know which thread to post this on but anyway...

Registered finally with apterous today, got a few games under my belt and really enjoyed it. Annoyingly, due to my home PC being naff I can only access from my parents one which living away I hardly get to use.

Played 6 games so far and all have been really enjoyable. I'll be using apterous hopefully when I can tonight and tomorrow and would be most keen to play a few games against some of youse experts like those in the last C of C, Kirk etc. Not really fussed about ratings etc, just a fun challenge.

Re: How ratings work

Posted: Mon Jun 22, 2009 4:56 pm
by Phil Reynolds
My rating currently stands at 923; however, at the end of a game I just played, I was told that I'd gained 600 ratings points taking my total to 2033. Was unsure which thread to post this in, as I don't know if it's a bug or just something I didn't understand.

Re: How ratings work

Posted: Mon Jun 22, 2009 5:02 pm
by Craig Beevers
Phil Reynolds wrote:My rating currently stands at 923; however, at the end of a game I just played, I was told that I'd gained 600 ratings points taking my total to 2033. Was unsure which thread to post this in, as I don't know if it's a bug or just something I didn't understand.
That'll be because you played a different format combination. You have separate ratings for various formats and here it looks like you've gone from a starting rating (1433) to a 1 game provision rating of 2033.

Re: How ratings work

Posted: Mon Jun 22, 2009 5:07 pm
by Phil Reynolds
Craig Beevers wrote:
Phil Reynolds wrote:My rating currently stands at 923; however, at the end of a game I just played, I was told that I'd gained 600 ratings points taking my total to 2033. Was unsure which thread to post this in, as I don't know if it's a bug or just something I didn't understand.
That'll be because you played a different format combination. You have separate ratings for various formats and here it looks like you've gone from a starting rating (1433) to a 1 game provision rating of 2033.
Ah right. Thanks Craig. I ought to play more often and attempt to keep up with all the changes.

Re: How ratings work

Posted: Mon Jun 22, 2009 5:33 pm
by Matt Morrison
Phil Reynolds wrote:
Craig Beevers wrote:
Phil Reynolds wrote:My rating currently stands at 923; however, at the end of a game I just played, I was told that I'd gained 600 ratings points taking my total to 2033. Was unsure which thread to post this in, as I don't know if it's a bug or just something I didn't understand.
That'll be because you played a different format combination. You have separate ratings for various formats and here it looks like you've gone from a starting rating (1433) to a 1 game provision rating of 2033.
Ah right. Thanks Craig. I ought to play more often and attempt to keep up with all the changes.
In addition to what Craig said - the ratings you see in the player list are according to the 'favourite variant' you've chosen in your user panel. Which will be Standard/ODE unless you change it.

Re: How ratings work

Posted: Wed Jun 09, 2010 4:02 pm
by Kirk Bevins
I think the Apterous World Cup affects the standard rating (although I maybe wrong)...scary times over 7 rounds.

Re: How ratings work

Posted: Wed Jun 09, 2010 4:20 pm
by Charlie Reams
Kirk Bevins wrote:I think the Apterous World Cup affects the standard rating (although I maybe wrong)...scary times over 7 rounds.
Yep. But not Pro Ranks.

Re: How ratings work

Posted: Sun Oct 31, 2010 11:18 am
by David O'Donnell
Is the rating loss/gain adjusted for time control?

Re: How ratings work

Posted: Sun Oct 31, 2010 5:17 pm
by Charlie Reams
David O'Donnell wrote:Is the rating loss/gain adjusted for time control?
Nope. It's not obvious to me how this adjustment should work. Clearly shorter games are more volatile, but what about longer games? Do we reward players more if it takes them 59 seconds to find a word? I couldn't figure it out so I decided to ignore it.

Re: How ratings work

Posted: Sun Dec 05, 2010 1:07 pm
by James Hall
Have standard rankings completely fallen out of use and been replaced by pro ranks? I can no longer find the ratings page anywhere on apterous.

Re: How ratings work

Posted: Sun Dec 05, 2010 1:19 pm
by Charlie Reams
James Hall wrote:Have standard rankings completely fallen out of use and been replaced by pro ranks? I can no longer find the ratings page anywhere on apterous.
Nope. There's no longer a list of all players and their rankings because it got cumbersome with the number of users we have now. But ratings are still in all the places they were before.

Re: How ratings work

Posted: Sun Dec 05, 2010 7:37 pm
by James Hall
Aaah OK. I just wanted to have a browse of rankings in other variants, which it seems I can't do any more - they still seem to exist though because I still get a report at the end of every game etc. Is there any way to find them out?