Which words are the most useful to know?

All discussion relevant to Countdown that is not too spoilerific. New members: come here first to introduce yourself. We don't bite, or at least rarely.
Thomas Cappleman
Series 72 Champion
Posts: 330
Joined: Fri Nov 07, 2008 9:42 pm

Re: Which words are the most useful to know?

Post by Thomas Cappleman »

You can do that in the spreadsheet using the Known column - set the first N to 1, and it'll tell you percentage of rounds you'd max.
User avatar
L'oisleatch McGraw
Devotee
Posts: 930
Joined: Sun Dec 13, 2015 2:46 am
Location: Waterford
Contact:

Re: Which words are the most useful to know?

Post by L'oisleatch McGraw »

Robert Foster wrote: Tue Mar 23, 2021 12:10 pm New version with the March dictionary update:

https://drive.google.com/drive/folders/ ... OBoD4LcS_4
Hey!
It looks like the March update is not the only change in this version.
I am noticing a few tiny differences in the order of usefulness.
e.g. In V4, ANODISERS and INSULATED were the 85th and 80th most useful nines, now they are the 69th and 72nd.

Was this new list generated with a slightly changed algorithm?
:arrow: :arrow: :arrow: S:778-ochamp
JackHurst
Series 63 Champion
Posts: 1982
Joined: Tue Jan 20, 2009 8:40 pm

Re: Which words are the most useful to know?

Post by JackHurst »

L'oisleatch McGraw wrote: Thu Apr 01, 2021 1:36 am Hey!
It looks like the March update is not the only change in this version.
I am noticing a few tiny differences in the order of usefulness.
e.g. In V4, ANODISERS and INSULATED were the 85th and 80th most useful nines, now they are the 69th and 72nd.

Was this new list generated with a slightly changed algorithm?
I'll answer on behalf of Rob from what he has told me.

The process for generating the rankings goes as follows:
- Simulate X Million Rounds
- Take the most common max as number 1 in the list
- Remove all of the rounds where word 1 appears as a max
- Take the most common max in those rounds as word 2 in the list
- Remove all rounds where word 2 appears as a max
- etc

You'll see rankings moving around every time for two reasons:
1) The simulation is different every time, so there is random behaviour
2) The dictionary is more baked into the process than you think. If GOATIER got removed, it probably wouldn't be word 2 that shoots to the top of the list. Likewise is something like AGIOTES got added, it might knock GOATIER down a few places.

There's then a follow on question of whether the process needs improving, and if so, how could it be improved. Options for improvement include
- Simulating a larger amount of rounds so the randomness evens out a bit more
- Reusing the same selections every time a new dictionary is measured

However, I'm very grateful for what we get at the moment and it has helped me a lot. I don't think improvements would make much of a difference for the extra effort.
Gavin Chipper
Post-apocalypse
Posts: 13194
Joined: Mon Jan 21, 2008 10:37 pm

Re: Which words are the most useful to know?

Post by Gavin Chipper »

JackHurst wrote: Thu Apr 01, 2021 8:00 am
The process for generating the rankings goes as follows:
- Simulate X Million Rounds
- Take the most common max as number 1 in the list
- Remove all of the rounds where word 1 appears as a max
- Take the most common max in those rounds as word 2 in the list
- Remove all rounds where word 2 appears as a max
- etc
This is quite interesting. I see this as similar to a system of proportional representation that elects candidates one at a time, rather than altogether. You elect the most supported candidate, and then once you've removed ballots appropriately (or whatever the algorithm is for the particular method) you elect the remaining candidate with the most support. And so on. There are methods that compare whole sets of candidates so are "optimal" rather than "greedy", but generally this would take too much computational power / time to be realistic. Similarly, the same could apply to Countdown words. E.g. The top two words that would max the most selections between them might not be the same as taking the top one and then the top remaining one.
JackHurst
Series 63 Champion
Posts: 1982
Joined: Tue Jan 20, 2009 8:40 pm

Re: Which words are the most useful to know?

Post by JackHurst »

From 5000 Rounds simulated, equal probability 3v 4v 5v.

Image

Code: Select all

24% maxes in top 1000
35% maxes in top 2000
42% maxes in top 3000
48% maxes in top 4000
53% maxes in top 5000
58% maxes in top 6000
61% maxes in top 7000
65% maxes in top 8000
68% maxes in top 9000
70% maxes in top 10000
72% maxes in top 11000
74% maxes in top 12000
76% maxes in top 13000
78% maxes in top 14000
79% maxes in top 15000
81% maxes in top 16000
82% maxes in top 17000
84% maxes in top 18000
85% maxes in top 19000
86% maxes in top 20000
87% maxes in top 21000
87% maxes in top 22000
88% maxes in top 23000
89% maxes in top 24000
90% maxes in top 25000
90% maxes in top 26000
91% maxes in top 27000
92% maxes in top 28000
92% maxes in top 29000
93% maxes in top 30000
93% maxes in top 31000
93% maxes in top 32000
94% maxes in top 33000
94% maxes in top 34000
95% maxes in top 35000
95% maxes in top 36000
95% maxes in top 37000
95% maxes in top 38000
96% maxes in top 39000
96% maxes in top 40000
Fiona T
Kiloposter
Posts: 1441
Joined: Mon Mar 18, 2019 12:54 pm

Re: Which words are the most useful to know?

Post by Fiona T »

Suggest that this is made a forum sticky? Lots of people still don't know it exists!
Gavin Chipper
Post-apocalypse
Posts: 13194
Joined: Mon Jan 21, 2008 10:37 pm

Re: Which words are the most useful to know?

Post by Gavin Chipper »

I suggest a new thread - Which threads are the most useful to sticky?
User avatar
L'oisleatch McGraw
Devotee
Posts: 930
Joined: Sun Dec 13, 2015 2:46 am
Location: Waterford
Contact:

Re: Which words are the most useful to know?

Post by L'oisleatch McGraw »

Gavin Chipper wrote: Fri Sep 03, 2021 4:02 pm I suggest a new thread - Which threads are the most useful to sticky?
I vote for the one about outing Gevin of a rainy Tuesday! :mrgreen:
:arrow: :arrow: :arrow: S:778-ochamp
Gavin Chipper
Post-apocalypse
Posts: 13194
Joined: Mon Jan 21, 2008 10:37 pm

Re: Which words are the most useful to know?

Post by Gavin Chipper »

L'oisleatch McGraw wrote: Fri Sep 03, 2021 7:11 pm
Gavin Chipper wrote: Fri Sep 03, 2021 4:02 pm I suggest a new thread - Which threads are the most useful to sticky?
I vote for the one about outing Gevin of a rainy Tuesday! :mrgreen:
I prefer the one about parathas.
Robert Foster
Rookie
Posts: 53
Joined: Sun Apr 17, 2011 1:42 pm
Location: Bletchley

Re: Which words are the most useful to know?

Post by Robert Foster »

September 2021 release is here! This version uses 100% organic Apterous rounds as input, for that extra crunch.

https://drive.google.com/drive/folders/ ... OBoD4LcS_4
Carl Harrison
Rookie
Posts: 71
Joined: Sun Feb 10, 2019 9:41 am

Re: Which words are the most useful to know?

Post by Carl Harrison »

Thanks for this Rob, fantastic work as always. :D

The most interesting thing for me about this new release and the different method of using actual Apterous rounds rather than simulated ones, is that the relative importance of 9's seems to have significantly increased. For example, in the top 10,000 words in the "overall" tab there are now 1916 nines, compared to 1343 in the previous version (+42%!). A similar pattern emerges whatever number of top X,000 words you use.

There was a bit of discussion on another thread previously about whether the spreadsheet previously underestimated the relative value of 9's. Maybe this new method has fixed that to some extent, but I can't easily work out why this has particularly happened with this new approach. I think the 3V/4V/5V proportions of rounds are roughly the same (the figures quoted above for the old spreadsheet were on 30/60/10, compared to the new simulations which say the proportions are 28/60/12 - so perhaps a bit of a boost for 5V nines).

Perhaps it's because of the bot-optimised picking used in Apterous (i.e. C or V chosen at end of each round to maximise available score) means 9's become more likely using this method - or are the simulations used here just from human v human games? (albeit humans may also be better at picking to maximise score than the raw simulation method used before).

I also wonder if there is any impact from moving to 25 million simulated rounds before to 5 million actual rounds now? (i.e. does the lower sample size affect things much?). It probably has done at the bottom end - every thing from 75,000+ scores a zero (i.e. never came up as the most likely max in a sample round), whereas for the previous version it was only 90,000+ which scored zero. Not really important at that end of the spreadsheet, but I wonder if this has any effect on things higher up (e.g. I can see quite a few words have moved from the top 10,000 to now right down the bottom).

What I'm really wondering is whether this new method has resulted in a more realistic better-ordered list than before (with appropriate weighting now being given to 9's), for both games on Apterous and actual TV Countdown. I suspect that's quite probably the case, but interested in any thoughts from others.
Fiona T
Kiloposter
Posts: 1441
Joined: Mon Mar 18, 2019 12:54 pm

Re: Which words are the most useful to know?

Post by Fiona T »

FWIW, I'd been using Rob's lists for a while and seen some improvement. Switched to just focussing on 9s (top 250 or so, my brain capacity is limited!) and have added about 100 to my average (very volatile) apto rating (even though I frequently miss them!).

Really has made a difference, not to mention the little endorphin rush you get when you spot a 9 you haven't got before :)
Gavin Chipper
Post-apocalypse
Posts: 13194
Joined: Mon Jan 21, 2008 10:37 pm

Re: Which words are the most useful to know?

Post by Gavin Chipper »

Carl Harrison wrote: Sun Sep 05, 2021 3:36 pm What I'm really wondering is whether this new method has resulted in a more realistic better-ordered list than before (with appropriate weighting now being given to 9's), for both games on Apterous and actual TV Countdown. I suspect that's quite probably the case, but interested in any thoughts from others.
I was thinking about whether it would be improved specifically for actual TV Countdown. Using Apterous rounds might be better for Apterous, but there is the question of what is more important. I'd say actual TV Countdown is the main thing, although a lot of people using this tool might have already been on or have no intention of going on so might consider Apterous to be more important. But on the third hand, you also have to consider CO-events. Winning e.g. COLIN is probably still considered a bigger deal than any Apterous tournament or having the best stats on Apterous.
Robert Foster
Rookie
Posts: 53
Joined: Sun Apr 17, 2011 1:42 pm
Location: Bletchley

Re: Which words are the most useful to know?

Post by Robert Foster »

Nines simply come up way more often in the apterous sample than for the randomly generated rounds I was using in my old version. For example, DERATIONS came up as a nine in 1-in-1673 apterous rounds, compared to 1-in-3476 in random rounds - more than twice as often.

The apterous sample is made up of both human and bot games, and the ratio of 3V/4V/5V rounds is almost identical to the previous version, as Carl points out. I'm pretty sure all of the discrepancy in ranking is due to the effects of smart picking of both humans and bots on apterous (avoiding nasty-looking selections, fishing for a particular letter, generally trying to increase the max length). When compared to rounds which are totally naively picked, it makes sense that you will see higher maximums in general and more 9s in particular thanks to optimal bot picking. Would be interesting to quantify this so I might try and look into it more.

As for whether the order of the list is more correct for apterous, I would hope so! (that's my main objective for making the switch). TV Countdown is less clear but I tried to show in an earlier post that you do get apterous-like selections on TV, so it's still a good yardstick. There is nowhere near enough data to create a decent usefulness list from historical TV Countdown rounds alone.

Going from 75 million rounds to 5 million rounds to run the analysis on is an unfortunate consequence of switching to apterous input. It's a trade-off between gaining 'authenticity' through using real data at the expense of precision at the tail-end of the document. I figure most people would be interested in the high- and mid-frequency words though so hopefully this isn't a big loss. Will ask Charlie for a top-up of rounds in 100 years' time maybe.

Carl also mentions that some words that were previously in the top 10,000 have now plummeted to useless - I think this effect is expected whenever you use different rounds for input. If you have a word (say TRIDENT) that despite frequently occurring as a max is outstripped in frequency by at least one other word 100% of the time it appears, it will receive a usefulness score of zero. Depending on the simulation used, it might be that TRIDENT now appears slightly more frequently than one of the words that used to barely dominate it, and so carves a niche for itself. This would be case of a word jumping from useless to very useful, and the same can happen in reverse, which is what you're describing.

Really gratifying to hear that people are actually using this thing and improving! For the people that say they are using it - how are you doing this exactly?
Carl Harrison
Rookie
Posts: 71
Joined: Sun Feb 10, 2019 9:41 am

Re: Which words are the most useful to know?

Post by Carl Harrison »

Thanks Rob - fascinating stuff, and all makes good sense.

Some of the words have moved around quite a bit, but it makes sense that when allowing for actual round-picking by bots or humans that some words might become less common (e.g. why a word like NUTRITOR is now a lot less valuable, as in practice people are typically more likely to pick another vowel with a selection like that). In fact I think a lot of people have a tendency to keep fishing for E's, and so 3 vowel rounds without E's are probably much less likely using actual data than simulated data, which is bound to have an effect on the output.

Does the apterous sample data only use "Normal" rounds, or are any of the other variants included? Obviously some variants would distort things quite a lot (e.g. junior and goatdown!) but perhaps some others wouldn't - but maybe they wouldn't add much to the sample size so wouldn't have been worth using.

All things considered I think the new methodology looks a good improvement - great to see this evolving and improving over time. Thanks Rob for the continued work on this!
Fiona T
Kiloposter
Posts: 1441
Joined: Mon Mar 18, 2019 12:54 pm

Re: Which words are the most useful to know?

Post by Fiona T »

Robert Foster wrote: Sun Sep 05, 2021 10:56 pm
Really gratifying to hear that people are actually using this thing and improving! For the people that say they are using it - how are you doing this exactly?
I've got them in Anki - tend to run through them during boring work calls :D
Thomas Cappleman
Series 72 Champion
Posts: 330
Joined: Fri Nov 07, 2008 9:42 pm

Re: Which words are the most useful to know?

Post by Thomas Cappleman »

Also using Anki - got things set so that the total words to revise a day stays within a given range, which works out as 7 9-letter and 20 other new words a day. Been doing it for about a year now, which shows up perfectly on the letters graph here: https://www.apterous.org/superstats_use ... section=10
Adam Latchford
Rookie
Posts: 45
Joined: Fri Jul 02, 2021 7:50 am

Re: Which words are the most useful to know?

Post by Adam Latchford »

After all the positive replies I think it's time I had a look at these lists , if they're good enough for all the monsters they'll be more than good enough for me!
User avatar
Rhys Benjamin
Postmaster General
Posts: 3100
Joined: Thu Sep 09, 2010 4:28 pm

Re: Which words are the most useful to know?

Post by Rhys Benjamin »

Adam Latchford wrote: Mon Sep 06, 2021 2:51 pm After all the positive replies I think it's time I had a look at these lists , if they're good enough for all the monsters they'll be more than good enough for me!
Beware the dark side. Words, endings, compounds, chemicals. For once you start down the dark path of dictionary swallowing, forever will it consume you.
The forum's resident JAILBAKER, who has SPONDERED several times...
Carl Harrison
Rookie
Posts: 71
Joined: Sun Feb 10, 2019 9:41 am

Re: Which words are the most useful to know?

Post by Carl Harrison »

Aye that is the problem - it's hard to know when to stop. How many words is "enough" for what you want to achieve? You can always think you'll improve if you just learnt some more, however far down the list you've gone. But there has to be a limit for everyone, beyond which trying to cram too many different words into your brain is counteractive as you'll start forgetting the earlier ones.

And let's be honest, playing on Apterous itself is a much more enjoyable way to improve than just spending loads of time trying to memorise words lists.
Philip A
Kiloposter
Posts: 1077
Joined: Thu Jan 07, 2021 2:56 pm

Re: Which words are the most useful to know?

Post by Philip A »

The important thing to remember is that it’s who has the longest word, not the obscurest word. So you shouldn’t get het up on bizarre words to the exclusion of everything else. So you should learn as many words as you can, but not at the expense of always striving for the longest word when playing the game. There’s no right or wrong way to learn new information - you have to go for what studying techniques work for you - but it’s a combination of studying and practising, little and often, that’s the key to success. You should also avoid over-studying and over-practising - as you practise your brain rewires neurones like building a wall. Under-do it and the wall won’t be complete, but overdo it and the mortar between the bricks won’t set and the wall will collapse. You have to strike a balance. There is no limit to learning, but it’s short term pain and long term game. It’s a mind sport.
Series 78 Runner-up
Adam Latchford
Rookie
Posts: 45
Joined: Fri Jul 02, 2021 7:50 am

Re: Which words are the most useful to know?

Post by Adam Latchford »

Philip A wrote: Wed Sep 08, 2021 3:08 pm The important thing to remember is that it’s who has the longest word, not the obscurest word. So you shouldn’t get het up on bizarre words to the exclusion of everything else. So you should learn as many words as you can, but not at the expense of always striving for the longest word when playing the game. There’s no right or wrong way to learn new information - you have to go for what studying techniques work for you - but it’s a combination of studying and practising, little and often, that’s the key to success. You should also avoid over-studying and over-practising - as you practise your brain rewires neurones like building a wall. Under-do it and the wall won’t be complete, but overdo it and the mortar between the bricks won’t set and the wall will collapse. You have to strike a balance. There is no limit to learning, but it’s short term pain and long term game. It’s a mind sport.
I'm not sure it's quite this deep...

For me personally i'd like to get to a better level than now, not become the very best, that no-one ever was. If it starts to consume me i'll take a long break out and go do other things - it's nice being able to improve without actually having to practice for something
User avatar
Rhys Benjamin
Postmaster General
Posts: 3100
Joined: Thu Sep 09, 2010 4:28 pm

Re: Which words are the most useful to know?

Post by Rhys Benjamin »

Remind me never to make a joke again.
The forum's resident JAILBAKER, who has SPONDERED several times...
Mark James
Kiloposter
Posts: 1766
Joined: Sat Oct 23, 2010 3:21 pm
Location: Dublin

Re: Which words are the most useful to know?

Post by Mark James »

Rhys Benjamin wrote: Wed Sep 08, 2021 7:46 pm Remind me never to make a joke again.
Again?
Tim Down
Acolyte
Posts: 141
Joined: Fri Feb 12, 2016 9:45 am

Re: Which words are the most useful to know?

Post by Tim Down »

Thomas Cappleman wrote: Mon Sep 06, 2021 9:15 am Also using Anki - got things set so that the total words to revise a day stays within a given range, which works out as 7 9-letter and 20 other new words a day. Been doing it for about a year now, which shows up perfectly on the letters graph here: https://www.apterous.org/superstats_use ... section=10
What do your cards in Anki consist of? Front is alphagram, back is actual word(s)? I'd rather have a random scramble but can't see a viable way to do that in Anki, which is why I've abandoned this kind of thing before.
Thomas Cappleman
Series 72 Champion
Posts: 330
Joined: Fri Nov 07, 2008 9:42 pm

Re: Which words are the most useful to know?

Post by Thomas Cappleman »

Tim Down wrote: Mon Nov 01, 2021 6:44 pm
Thomas Cappleman wrote: Mon Sep 06, 2021 9:15 am Also using Anki - got things set so that the total words to revise a day stays within a given range, which works out as 7 9-letter and 20 other new words a day. Been doing it for about a year now, which shows up perfectly on the letters graph here: https://www.apterous.org/superstats_use ... section=10
What do your cards in Anki consist of? Front is alphagram, back is actual word(s)? I'd rather have a random scramble but can't see a viable way to do that in Anki, which is why I've abandoned this kind of thing before.
Yep, exactly that. Similarly would have liked a scramble but not found a way. Does mean I occasionally write a round's letters out alphabetically to trigger my memory.
Robert Foster
Rookie
Posts: 53
Joined: Sun Apr 17, 2011 1:42 pm
Location: Bletchley

Re: Which words are the most useful to know?

Post by Robert Foster »

New word list is out, get it while it's hot and crispy:
https://drive.google.com/drive/folders/ ... OBoD4LcS_4
Robert Foster
Rookie
Posts: 53
Joined: Sun Apr 17, 2011 1:42 pm
Location: Bletchley

Re: Which words are the most useful to know?

Post by Robert Foster »

Now up-to-date with the February changes: https://drive.google.com/drive/folders/ ... OBoD4LcS_4
Dan Byrom
Acolyte
Posts: 128
Joined: Sun Apr 16, 2017 2:42 pm

Re: Which words are the most useful to know?

Post by Dan Byrom »

Hugely appreciated as always. Thanks Rob!
User avatar
L'oisleatch McGraw
Devotee
Posts: 930
Joined: Sun Dec 13, 2015 2:46 am
Location: Waterford
Contact:

Re: Which words are the most useful to know?

Post by L'oisleatch McGraw »

Question for the word study geniuses who inhabit this thread:

If someone were determined to study the most useful words in batches of 100 by word length, I have a gut feeling that the best strategy to take, for optimum game improvement, would be to do the first five batches in this order:-

1. 100 best 8s.
2. 100 best 7s.
3. Next 100 8s.
4. Next 100 7s.
5. 100 best 9s.

Does this seem about right?

Also, how low down on the priority list should the first batch of 6s be, or 5s... and would it be worth bothering with anything lower than 5?
:arrow: :arrow: :arrow: S:778-ochamp
User avatar
L'oisleatch McGraw
Devotee
Posts: 930
Joined: Sun Dec 13, 2015 2:46 am
Location: Waterford
Contact:

Re: Which words are the most useful to know?

Post by L'oisleatch McGraw »

L'oisleatch McGraw wrote: Wed Jul 20, 2022 1:28 pm First five batches in this order:-

1. 100 best 8s.
2. 100 best 7s.
3. Next 100 8s.
4. Next 100 7s.
5. 100 best 9s.

Does this seem about right?
I've done an about turn on this.
9s are too important tactically, in spite of their in game scarcity.

The 100 best 9s should probably be learned as the 3rd batch, not 5th.

EDITED to say:
Having seen Quinn's debut performance greatly enhanced by his knowledge of high probability 9s... I think given their impact and the psychological edge they give, the 100 best 9s should actually be the 2nd batch!
:arrow: :arrow: :arrow: S:778-ochamp
User avatar
L'oisleatch McGraw
Devotee
Posts: 930
Joined: Sun Dec 13, 2015 2:46 am
Location: Waterford
Contact:

Re: Which words are the most useful to know?

Post by L'oisleatch McGraw »

Yep, the 9s WILL be the 2nd batch.
But the FIRST batch will be 8s... and incidentally, it has just gone live.
:arrow: :arrow: :arrow: S:778-ochamp
User avatar
L'oisleatch McGraw
Devotee
Posts: 930
Joined: Sun Dec 13, 2015 2:46 am
Location: Waterford
Contact:

Re: Which words are the most useful to know?

Post by L'oisleatch McGraw »

I believe Rob has made a December update:- https://docs.google.com/spreadsheets/d/ ... ue&sd=true
:arrow: :arrow: :arrow: S:778-ochamp
Post Reply