Random?

All discussion relevant to Countdown that is not too spoilerific. New members: come here first to introduce yourself. We don't bite, or at least rarely.
Post Reply
Dave Preece
Devotee
Posts: 621
Joined: Thu Feb 21, 2013 11:50 pm

Random?

Post by Dave Preece »

OK, If we showed a (for want of a better name) random numbers expert the whole list of the thousands and thousands of numbers - in order - that CECIL has picked from the start of Countdown, would he or she be 100% happy that they were random?
User avatar
Callum Todd
Series 69 Champion
Posts: 1123
Joined: Tue Sep 10, 2013 3:38 pm
Location: Leeds

Re: Random?

Post by Callum Todd »

I'm not sure if that's possible to determine. The whole point of random is that each possible outcome is equally likely, and thus a biased number list is just as likely as an evenly spread number list. This means that it is just as likely that if you were to use CECIL a million times, it would produce 252, for example, a million times as it is that it would produce an even spread of numbers. So even if the list showed bias towards certain numbers, that would not mean that the numbers were not "random".
Mark Deeks wrote:Callum Todd looks like a young Ted Bundy.
User avatar
Michael Wallace
Racoonteur
Posts: 5458
Joined: Mon Jan 21, 2008 5:01 am
Location: London

Re: Random?

Post by Michael Wallace »

I am totally going to start calling myself a 'random numbers expert' rather than a statistician now.
Gavin Chipper
Post-apocalypse
Posts: 13215
Joined: Mon Jan 21, 2008 10:37 pm

Re: Random?

Post by Gavin Chipper »

Is this a numbers expert picked at random?
Dave Preece
Devotee
Posts: 621
Joined: Thu Feb 21, 2013 11:50 pm

Re: Random?

Post by Dave Preece »

Humans famously can't pick random numbers, so what you said Callum could also apply to a human list of numbers that we know not to be random enough!

Numberphile on Youtube recently had a video about a true random number generator being made from some dangerous radioactive chemical...???
Dave Preece
Devotee
Posts: 621
Joined: Thu Feb 21, 2013 11:50 pm

Re: Random?

Post by Dave Preece »

Gavin Chipper wrote:Is this a numbers expert picked at random?
No!
Dave Preece
Devotee
Posts: 621
Joined: Thu Feb 21, 2013 11:50 pm

Re: Random?

Post by Dave Preece »

Michael Wallace wrote:I am totally going to start calling myself a 'random numbers expert' rather than a statistician now.
Surely a statistician is to random numbers what a hairdresser is to trimming a beard?
User avatar
Michael Wallace
Racoonteur
Posts: 5458
Joined: Mon Jan 21, 2008 5:01 am
Location: London

Re: Random?

Post by Michael Wallace »

Dave Preece wrote:
Michael Wallace wrote:I am totally going to start calling myself a 'random numbers expert' rather than a statistician now.
Surely a statistician is to random numbers what a hairdresser is to trimming a beard?
Random numbers (and the processes that generate them) are pretty much the bedrock of everything we do. I'm not really sure what that would be in a hairdressing analogy, though.

And for what it's worth, yes, it would be easy to test whether CECIL was generating truly 'random' numbers (i.e. whenever it picks a number it has the same probability of picking any particular number, irrespective of what has come before it). You could (e.g.) look at how many times you would 'expect' to see each number (so in a sample of 1,000 numbers you'd 'expect' to see every number from 100-999 exactly once) and comparing that with how often they actually came up. If what really happened deviated too much from what you'd expect, you'd have reason to suspect them of not being random. Obviously you can't prove it definitively, but you can be pretty damn sure if you have enough data.
User avatar
Callum Todd
Series 69 Champion
Posts: 1123
Joined: Tue Sep 10, 2013 3:38 pm
Location: Leeds

Re: Random?

Post by Callum Todd »

Michael Wallace wrote:in a sample of 1,000 numbers you'd 'expect' to see every number from 100-999 exactly once
Would you? The way I understand it, that output of numbers is no more likely than any other list of 1,000 numbers.
Mark Deeks wrote:Callum Todd looks like a young Ted Bundy.
User avatar
Michael Wallace
Racoonteur
Posts: 5458
Joined: Mon Jan 21, 2008 5:01 am
Location: London

Re: Random?

Post by Michael Wallace »

Callum Todd wrote:
Michael Wallace wrote:in a sample of 1,000 numbers you'd 'expect' to see every number from 100-999 exactly once
Would you? The way I understand it, that output of numbers is no more likely than any other list of 1,000 numbers.
By 'expect' I mean if you did it lots and lots of samples of 1,000, you'd eventually average (across those samples) one of each number per sample. Hypothesis testing is about how 'extreme' your data are, and if they're too extreme you consider that evidence that your (null) hypothesis is false. For instance, if you toss a coin 100 times then sure, it's just as likely you'll get 95 heads in a row followed by 5 tails, as it is you'll get 50 heads in a row followed by 50 tails, but in one of those scenarios you'd have pretty strong evidence that the coin was biased by looking at the total number of heads. (Although you'd probably be a bit distracted by your crazily streaky coin :p)

Similarly, in the CECIL example, if you had 1,000 250s in your sample, and no other numbers, those 1,000 250s would show up as being *way* out of line with what you'd expect, so your test would probably say "wtf?", whereas if you have a few numbers that show up 2 or 3 (or a few more) times, that's much less extreme behaviour.
Gavin Chipper
Post-apocalypse
Posts: 13215
Joined: Mon Jan 21, 2008 10:37 pm

Re: Random?

Post by Gavin Chipper »

With something like CECIL (as opposed to tossing a coin) you have to worry about things like whether each number is independent of what has gone before. If you tossed a coin 100 times and got 50 heads followed by 50 tails, you might decide that the coin is fair, although of course you'd probably be looking at reasons as to why the tosses might not have been independent of the previous ones (or if someone had swapped the coin when you weren't looking). Whereas with CECIL you might just get the numbers all coming up in order (100, 101, 102 etc.) and you'd be pretty sure it wasn't random. There's various tests statisticians might apply to the data, but like scientific tests in general, they'd really only be testing for whether it fails. You'd never say "yeah, it is random", but you might say "there's no particular reason to suggest it isn't random, so it could be". Well, it's unlikely to be anyway unless it uses quantum randomness, but you could make something that does a very good impression.
User avatar
Michael Wallace
Racoonteur
Posts: 5458
Joined: Mon Jan 21, 2008 5:01 am
Location: London

Re: Random?

Post by Michael Wallace »

Gavin Chipper wrote:There's various tests statisticians might apply to the data, but like scientific tests in general, they'd really only be testing for whether it fails. You'd never say "yeah, it is random", but you might say "there's no particular reason to suggest it isn't random, so it could be".
Yeah. I mean, formally you'd define your null hypothesis (which would be that CECIL generates independent, identically distributed random variables uniformally over the set of integers [100,999], or something like that) and then see whether the data are 'consistent' with that null hypothesis. How you interpret that can get a bit philosophical (and can depend on your statistical perspective, cf. Bayesians vs. frequentists).
Dave Preece
Devotee
Posts: 621
Joined: Thu Feb 21, 2013 11:50 pm

Re: Random?

Post by Dave Preece »

Dave Preece wrote:OK, If we showed a (for want of a better name) random numbers expert the whole list of the thousands and thousands of numbers - in order - that CECIL has picked from the start of Countdown, would he or she be 100% happy that they were random?
So... Yes or no?
Dave Preece
Devotee
Posts: 621
Joined: Thu Feb 21, 2013 11:50 pm

Re: Random?

Post by Dave Preece »

Dave Preece wrote:Humans famously can't pick random numbers, so what you said Callum could also apply to a human list of numbers that we know not to be random enough!

Numberphile on Youtube recently had a video about a true random number generator being made from some dangerous radioactive chemical...???
Why did this statistician on Numberphile risk his health to use this PERFECT random number generator? Why didn't he just use a nice, safe and friendly CECIL type computer?

https://www.youtube.com/watch?v=SxP30euw3-0
User avatar
Michael Wallace
Racoonteur
Posts: 5458
Joined: Mon Jan 21, 2008 5:01 am
Location: London

Re: Random?

Post by Michael Wallace »

Dave Preece wrote:
Dave Preece wrote:OK, If we showed a (for want of a better name) random numbers expert the whole list of the thousands and thousands of numbers - in order - that CECIL has picked from the start of Countdown, would he or she be 100% happy that they were random?
So... Yes or no?
I don't know if he or she would be 100% happy, all I can say is that yes, you could test it and be relatively sure that it either was/wasn't.
User avatar
Ian Volante
Postmaster General
Posts: 3956
Joined: Wed Sep 03, 2008 8:15 pm
Location: Edinburgh
Contact:

Re: Random?

Post by Ian Volante »

Dave Preece wrote:
Dave Preece wrote:Humans famously can't pick random numbers, so what you said Callum could also apply to a human list of numbers that we know not to be random enough!

Numberphile on Youtube recently had a video about a true random number generator being made from some dangerous radioactive chemical...???
Why did this statistician on Numberphile risk his health to use this PERFECT random number generator? Why didn't he just use a nice, safe and friendly CECIL type computer?

https://www.youtube.com/watch?v=SxP30euw3-0
Computers are psuedo-random, for various complex reasons that I'm sure our random number expert Michael will be happy to explain :)

A small number of early online poker sites fell foul of this, but such problems are pretty much irrelevant for CECIS.
meles meles meles meles meles meles meles meles meles meles meles meles meles meles meles meles
Gavin Chipper
Post-apocalypse
Posts: 13215
Joined: Mon Jan 21, 2008 10:37 pm

Re: Random?

Post by Gavin Chipper »

Ian Volante wrote:
Dave Preece wrote:
Dave Preece wrote:Humans famously can't pick random numbers, so what you said Callum could also apply to a human list of numbers that we know not to be random enough!

Numberphile on Youtube recently had a video about a true random number generator being made from some dangerous radioactive chemical...???
Why did this statistician on Numberphile risk his health to use this PERFECT random number generator? Why didn't he just use a nice, safe and friendly CECIL type computer?

https://www.youtube.com/watch?v=SxP30euw3-0
Computers are psuedo-random, for various complex reasons that I'm sure our random number expert Michael will be happy to explain :)

A small number of early online poker sites fell foul of this, but such problems are pretty much irrelevant for CECIS.
The guy on the video talks about it as well. For a single random number, I think a CECIL type thing would work perfectly well. If it has an internal clock and it partly bases the number on the hundredths or thousandths of a second when you press the button, then it's as good as random because you won't be precise enough in your button press for it not to be. But for a continuous stream of numbers that a computer churns out, it's likely to take a similar amount of time for each calculation and it could end up in the same groove each time. I'm sure there are ways round this though.
Dave Preece
Devotee
Posts: 621
Joined: Thu Feb 21, 2013 11:50 pm

Re: Random?

Post by Dave Preece »

Could the endless decimal digits of pi do a better job that the date? Statistically speaking?
David Barnard
Enthusiast
Posts: 290
Joined: Fri Sep 02, 2011 1:02 am

Re: Random?

Post by David Barnard »

So let's take the rule here of most peoples perception of randomness and out of the 900 possible targets that 899 of them occurred once out of 899 and the number that was left was say 637, under your law it would mean to keep the law going 637 would HAVE to occur next, that isn't random as the number is predetermined. Probability and randomness are 2 separate things
Dave Preece
Devotee
Posts: 621
Joined: Thu Feb 21, 2013 11:50 pm

Re: Random?

Post by Dave Preece »

DEFO!!! But randomness is VERY random and nothing like what us humans expect!
User avatar
Charlie Reams
Site Admin
Posts: 9494
Joined: Fri Jan 11, 2008 2:33 pm
Location: Cambridge
Contact:

Re: Random?

Post by Charlie Reams »

David Barnard wrote:So let's take the rule here of most peoples perception of randomness and out of the 900 possible targets that 899 of them occurred once out of 899 and the number that was left was say 637, under your law it would mean to keep the law going 637 would HAVE to occur next, that isn't random as the number is predetermined. Probability and randomness are 2 separate things
Whose law?
Dan McColm
Series 71 Champion
Posts: 255
Joined: Fri Oct 22, 2010 1:23 am

Re: Random?

Post by Dan McColm »

Godwin's law.
User avatar
Michael Wallace
Racoonteur
Posts: 5458
Joined: Mon Jan 21, 2008 5:01 am
Location: London

Re: Random?

Post by Michael Wallace »

Charlie was kind enough to wang me CECIL's output (up to October 2012, or 13,364 draws), and I've done a spectacularly simple analysis as a 'first pass' of the data.

The rarest number is 100, which is presumably thanks to it no longer being possible to generate. It has come up four times, the most recent being the 3,409th number in my list (so like, 20 years ago or something?). The 'real' rarest numbers are 262 and 868, both of which have come up five times. The most common number is 609, which has come up 27 times.

A basic frequency analysis (a chi-square test, if you care) found no evidence that CECIL wasn't drawing numbers 'fairly' (p = 0.23, if you care). This test works the way I described above: it looks at the frequency of each number and compares it with how frequent that number would have been if CECIL was picking at random. If the data deviate too much from this expectation then it would be evidence for biased number generation.

There are obviously other things we might want to look at (e.g. serial independence or whatever), but not sure if I'll have the time/inclination to check them, especially as by that point you're starting to just go on a fishing expedition. If anyone can provide a reasonable a priori reason for a specific hypothesis to test I could look into it.
User avatar
Ian Volante
Postmaster General
Posts: 3956
Joined: Wed Sep 03, 2008 8:15 pm
Location: Edinburgh
Contact:

Re: Random?

Post by Ian Volante »

Michael Wallace wrote:Charlie was kind enough to wang me CECIL's output (up to October 2012, or 13,364 draws), and I've done a spectacularly simple analysis as a 'first pass' of the data.

The rarest number is 100, which is presumably thanks to it no longer being possible to generate. It has come up four times, the most recent being the 3,409th number in my list (so like, 20 years ago or something?). The 'real' rarest numbers are 262 and 868, both of which have come up five times. The most common number is 609, which has come up 27 times.

A basic frequency analysis (a chi-square test, if you care) found no evidence that CECIL wasn't drawing numbers 'fairly' (p = 0.23, if you care). This test works the way I described above: it looks at the frequency of each number and compares it with how frequent that number would have been if CECIL was picking at random. If the data deviate too much from this expectation then it would be evidence for biased number generation.

There are obviously other things we might want to look at (e.g. serial independence or whatever), but not sure if I'll have the time/inclination to check them, especially as by that point you're starting to just go on a fishing expedition. If anyone can provide a reasonable a priori reason for a specific hypothesis to test I could look into it.
Bet it fails Benford's Law.
meles meles meles meles meles meles meles meles meles meles meles meles meles meles meles meles
Dave Preece
Devotee
Posts: 621
Joined: Thu Feb 21, 2013 11:50 pm

Re: Random?

Post by Dave Preece »

Thanks Michael, that's exactly the kind of summing up I was looking for when I asked the question.

Q. Have there been far more higher numbers than lower? Say, if you looked at numbers in 100's, 200's, 300's......
User avatar
Michael Wallace
Racoonteur
Posts: 5458
Joined: Mon Jan 21, 2008 5:01 am
Location: London

Re: Random?

Post by Michael Wallace »

Ian Volante wrote:Bet it fails Benford's Law.
Haha. Very good.

Dave: nope, it's a very even spread. Using your suggesting to look at groups of 100, 11.2% of CECIL's draws have been in the range 900-999, while 11.1% have been in the range 100-199. You'd 'expect' about 11.1% of numbers to fall in each set of 100, so that looks pretty healthy. Alternatively, there have been 5,883 numbers in the range 100-499, and 5,888 in the range 600-999 (I left out the middle 100 so we could compare these two directly).

Edited to fix a rather amusing typo via s/699/999/
Last edited by Michael Wallace on Fri Nov 15, 2013 6:42 pm, edited 1 time in total.
Dave Preece
Devotee
Posts: 621
Joined: Thu Feb 21, 2013 11:50 pm

Re: Random?

Post by Dave Preece »

Interesting!
Gavin Chipper
Post-apocalypse
Posts: 13215
Joined: Mon Jan 21, 2008 10:37 pm

Re: Random?

Post by Gavin Chipper »

Michael Wallace wrote:
Gavin Chipper wrote:There's various tests statisticians might apply to the data, but like scientific tests in general, they'd really only be testing for whether it fails. You'd never say "yeah, it is random", but you might say "there's no particular reason to suggest it isn't random, so it could be".
Yeah. I mean, formally you'd define your null hypothesis (which would be that CECIL generates independent, identically distributed random variables uniformally over the set of integers [100,999], or something like that) and then see whether the data are 'consistent' with that null hypothesis. How you interpret that can get a bit philosophical (and can depend on your statistical perspective, cf. Bayesians vs. frequentists).
I see.
Post Reply