In multiple choice, using random selection increases the odds of getting all of them wrong compared to picking the same position, such as “third one”, for all the math based questions.
3 Likes
This may well be possible in Great Britain, and if so, then have at it, and good luck. It seems a fair system to cater for all abilities.
Things are rather different elsewhere, however. In Germany, for example, the paper for the Full license consists of 119 questions, 51 in the technical part, 34 in the operating practices section, and 34 in the section dealing with the German laws and regulations pertaining to amateur radio. Each of these three sections must individually be passed at 70% or more, so there is here no chance, for example, to do well at the latter two sections, and to allow those to carry a poor result in the first to gain a pass. But it’s also possible here to go from noob to full license in only one exam, if one is well prepared.
I’m not at all sure a statistician would agree with this.
If the answers are genuinely randomly distributed then it makes no difference. But if they have been “randomly” distributed by someone who thinks they should spread the answers around then you may be right.
2 Likes
A long time ago in a galaxy far, far away… I taught a Computer Systems course at the local Polytechnic. I had to produce a multiple choice exam paper for my course and ISTR 50 questions were needed. (Can’t remember now this was 1988). Anyway, there were lots of rules about the multiple choice questions. Such as I couldn’t make all the correct answers as option A. Now if you answer a paper and you get to the fourth consecutive answer and it’s another A you start to get uncomfortable. If you are a strong candidate you know you are right and keep on going. But the weaker candidates get fazed by this and start answering incorrectly to break up the run of answer A. There were many other rules to stop the questions setting out to confuse or mislead the candidates, they were being tested on the knowledge and understanding of the course. And again, I think the head of dept. ran a check on the distribution of answers to ensure it was “random enough”. But as I say, 37 years ago and my memory is hazy on the details. But it was enough to show that they took the physical details of the exams very seriously.
2 Likes
I’m pretty sure that the UK exam system shuffles the answers for each chosen question every time an exam is set, so the answers within one paper should end up randomly distributed.
1 Like
Completely agree, im in for 40 - 50% ish… on practice papers, so somewhat off!
It is possible, but its borderline! 1 slip up, and you are out! hence the need for getting to grips with some of the more grippable maths stuff!
This is fairly close to what got me through the intermediate! we had a VERY good tutor who essentially trained us to spot the wrong answers, and then take a semi educated stab at the right one!
Alan
1 Like
I just checked with my daughter, a university professor with multiple degrees in Stat. She agrees with the advantage in picking the same position in answering all mystery questions.
1 Like
As many will know, where you and/or your team believe you have made a notable scientific discovery, you need to publish your results. This allows them to be ‘peer-reviewed’ by others in your field across the world.
One established publication route is via the highly-respected journal Nature. Your article will typically include a narrative, several charts, and a summary of the vital numbers.
What is also expected is that you publish (on GitHub?) your raw data, the code you used to process it, and the full set of generated results; again for peer-review.
Long ago, you would need write all the code (in FORTRAN?) yourself, this probably included statistics functions and the like.
Today we are on a rather better wicket: A number of widely-accepted mathematical packages are used for data-processing including MATLAB, Mathcad and so forth. These are written by mathematical specialists, likely all far cleverer than you and me 
More recently, the open-source language Python, togther with its Numerical (NumPy) and Scientific (SciPy) libraries, has become an accepted choice when publishing data-processing work. This has particular benefits for peer-reviewers in less well-off countries as it is all a free download. By contrast, the above-mentioned mathematical packages are all commercial products, and rather expensive.
Given all of the above, as an exam-setter, we can use Python with reasonable confidence to generate the randomised choices needed for the ‘right answers’.
The Python to do this is notably straightforward:
Python code
from random import choice
count = 500
# generate random selections
answers = ('A','B','C','D','E')
data_store = [choice(answers) for _ in range(count)]
That’s all. The list data_store will now contain 500 uniformly-random selections from the set of possible answers. Below is a typical output (it changes ever time you run the code), in blocks of 100, to make it a little more readable.
You will notice repeated runs of sometimes 3 or 4 of the same selection. These are a feature of randomness which surprises some.
Assume we have an exam with 50 questions. We can now pick a run of 50 selections from the data below, with confidence, for our test paper.
As a student, you may believe, for example, that picking all 'A’s could be a winning exam strategy. Using a ‘run of 50’ from the data below, you can now ‘score’ how well you did. Repeat this exercise as many times as you like
73 Dave
D E D C E B E B B D E C E D E A E D C A C E A A E
A C A A A B B A A A B A B B E D C E D D A A D D E
E C D D D A E A C C C B C E D C E C B A C A B B A
C B C E A E C B A C C D A E A D B A B E B A C C EC A A C A B D A A C D B C C A E B C C B A E D A D
E B D B B B B C C C C B E B C E A B C D D D A E E
A E C E C B D C E D D C A B C E C C D E A D C B E
E C B E C E D E E D A E B D E D B E B A A A B A ED E A A E A E A C A A A C D E B A D D B D B E B E
B E D B D A E D C C D E A E B D C D B B A E A B E
D E E A B B B A D B A D A B C E B D B C E D A E D
B A C D C A A A C D E C B B B B A D D B B D B C CE A D A A A E C B B D D D E E B A B C D E A E A B
A E C C D B B E B A B D D B D A A E C B E E D B C
A B C C D C A B C B C B D B D D D A B B D C D D C
A B B A B E D E C B D B B C B A E B A D C B D C AC A C A C C B A D B C A C C C C B E A B A E B A B
E D E D B E A D E C C A E E B A A E E C E A A C C
B A E D E D C A B A A D B A C B C A E B D B A C B
A B D B B A A C A C D A E D E D D E E E D E C C A
1 Like
Do PRBNs have the same runs of one value as a true random number?
Trying to think back to uni (which decade of which century was that?) as to how pseudo-random numbers are generated. Seem to remember some sort of shift-register with various feedbacks into the various slots of the register. Now. If we’re looking for sequences of repeated values, then the maximum number of repeated values such a pseudo-random generator can create must be limited by the length of the register (or it would presumably get stuck on that value forever as the same values start to recur in the same register positions).
So surely pseudo random numbers generated by such a generator must have fewer ‘runs’ of single values (or different patterns of them) to true random numbers.
Now shoot me down for being completely wrong!
1 Like
Before somebody asks our friend AI for its take on the question of best choice strategy for multiple choice exams, how about this natural, experience-based reasoning: is it better to fight uncertainty introduced by random assignment of correct answers by uncertainty of randomly picking correct answers or by certainty of sticking with your “lucky letter”? (The number of questions is assumed to be statistically sufficient for the principles of randomness to apply.)
1 Like
Well Dave, with respect, that’s not all. The Python choice function, as with similar functions in most, if not all, programming languages (even BASIC has the RND function), does not produce truly random results. Instead, it is a deterministic random bit generator which, because it is completely determined by an initial value called a seed, produces “pseudo-random” results. The lists you have produced are interesting in themselves, but are not truly random.
True randomness can be observed in phenomena like radioactive decay, thermal noise in circuits and atmospheric noise. Indeed, some programming systems can take advantage of such randomness to produce results which tend toward true randomness.
Indeed true. And that’s fine for exam questions. But you certainly wouldn’t use for anything crypto as you know.
73 Dave
1 Like
Humans are known to be incapable of making a truly random choice from a given set of numbers or list of choices. The best we can do “randomly” is make a pseudo-random choice from a set of results produced by a pseudo-random bit generator.
The other option in our case is to study the subject, do the sums to the best of ones’ ability, and choose the answer which best fits the outcome.
1 Like
Indeed. The PRNG used by Python for a long time is known as the Mersenne Twister. This has a period of 2^19937 before the sequence repeats. The same algorithm is used by many modern programming languages and programs as it is “good enough” as the built in random generator. But it is not suitable for use as a cryptographic random number generator. It’s more than suitable for this test.
The UK government has a savings scheme Premium Bonds. You buy some government bonds and it invests it/does what it wants with the cash but it agrees to give 4% of the returns it makes back to paid in prizes. Picking bond numbers has been done electronically since it started in 1956. However, it has always used genuine noise sources for the randomness, initially gas discharge tubes and now quantum stuff. It was always done with hardware so nobody could say the programmers could program the system pay out their way. The results are checked for randomness before the allocation of prizes are published. And… you can download the winning bond numbers for each month and use them yourself as a good source of random numbers if you don’t have a suitable hardware source.
EDIT: and thanks to this thread, I checked up on something to find that one of my SSH keys was generated with a weak algorithm implementation. I need to go and update that key.
1 Like
Could we have the proof please?
1 Like
If you stick with one position of A,B,C,D then on average you have a 1 in 4 chance of being right, given enough questions. If you now introduce random selections of the four you get a 1 in 8 chance if you have enough questions.
p vs p*q. Where p and q=0.25.
The variability from one set of 50 tries to the next set of 50 tries is high. The probability of getting them all right or all wrong are both extremely low. By sticking with one letter you have a good chance of getting 8 to 10 right and maybe double that.
A professor of statistics could provide better predictions.
It would be more sensible to find a tutor to learn how to handle Ohms law and calculate the length of a dipole and then go on to solve the more common questions in the licence exam.
73
Ron
VK3AFW
1 Like
With respect, I think this is a fallacious argument. The random choices made by the question setter and the candidate are independent. For each question there are only 4 possible choices, one of which is right. If the setter distributed the correct answers randomly, it makes no difference whether the candidate chooses at random or with some “system”. Where does 1 in 8 come from? Your “p*q” is in any case 1 in 16.
This is simply the gambler’s fallacy in disguise.
As others have said, if the question setter has deliberately influenced the answer sequence and accidentally introduced some bias, there may be some marginal advantage to be gained by answering systematically.
Martyn
2 Likes
Care is needed here! The bond numbers that ERNIE spits out are truly random. But we don’t get to see those. We only see the numbers of bonds that are in the draw, i.e. bonds that have been bought and not yet redeemed. This must introduce some bias, however small. The prefix, for example, most definitely isn’t random, because older prefixes have fewer bonds remaining unredeemed.
Do NS&I even provide the winning list in the order drawn? In the old days when we looked for our winnings in the newspaper they were presented sorted. Definitely not random. [Edit: I just downloaded one of these lists.The numbers are indeed sorted. It may be possible to extract some entropy from them, but I reckon it would be a struggle.]
Martyn
1 Like
Assuming a large number of questions with 4 offered answers (A, B, C, and D, only one of them being correct) and true randomness in answer assignments, probability of a correct answer labeled A, pAA=1/4, and the same for the other three letters, i.e. pAB=1/4, pAC=1/4, and pAD=1/4.
Given that our poor hamlet is completely ignorant of the exam subject and he uses a pseudo-random generator to select his answer to each question, then probability that he selects answer labeled A, pSA=1/4, and the same for the other three letters, i.e. pSB=1/4. pSC=1/4, and pSD=1/4. Now, the probability that a randomly assigned answer label AND a randomly selected answer label match is pAX x pSX=1/4 x 1/4=1/16, for X being either A, or B, or C, or D.
On the other hand, if our hamlet sticks with a single label letter, say C, then pSC=1. while pSA=0, pSB=0 and pSD=0. Hence, the probability of correct answer is pAX x pSX=1/4 x 0=0, for X being either A, or B, or D, while it is pAC x pSC=1/4 x 1=1/4.
Obviously, sticking with a single letter answer offers greater probability of a correct answer (1/4=25%) than a random selection (1/16=6.25%).
Q.E.D.
3 Likes