Number of Iterations

From Javier


The Isthmus Partners World Cup Simulator allows to run simulations with a range of iterations of 500 - 9,999. Each iteration is the simulation of one World Cup, with the model simulating results of all games in the first round and second round, with the play-off games reflecting the outcomes of the results in the first round and previous play-off matches. Each tournament is simulated up to the final and the 'Probabilities' worksheet reflects the number of times out of 100 iterations that each team has reached a certain stage of the tournament including being champion.


If the user runs successive simulations with 500 iterations, the user can observe that the probabilities of each team winning changes, for example for 5 runs of 500 iterations (using 100% model inputs) the estimated probabilities of Brazil winning are successively 21.4%, 19.2%, 17.8%, 17.4%, 20.4%. What is the usefulness of running more iterations? Although one is tempted to say that more iterations mean more accurate estimations, the correct interpretation is rather more technical: if one compares a similar number of simulation runs using different iterations, the simulation runs with more iterations will tend to show less variance and smaller ranges (distance between maximum and minimum). This is a mouthful, so lets repeat this with an example to clarify what this means.


We complete the following:

- Group 1: 20 runs of simulations with 500 iterations
- Group 2: 20 runs of simulations with 1000 iterations
- Group 3: 20 runs of simulations with 2500 iterations
- For each run we estimate the probabilities of Spain and Brazil winning the World Cup
- Within each Group we estimate the average probability of winning for each team, the variance (standard deviation using STDEV function) of the probability of winning and the range between maximum and minimum in each Group.


Below is a table comparing the results for each Group.









Remember that each column represents the results of running the simulation exercise 20 times with the respective number of iterations. The table shows that with more iterations, the standard deviation of the distribution of estimates of winning the World Cup around the average result is tighter and the range (difference between maximum and minimum) is narrower.

What this means also is that with more iterations for the same confidence level (i.e. 90%) we can assume that the average probability is within a tighter band around the estimated average probability. For example for 2500 iterations we can say that we are 90% confident that Spain's chances of winning are between 17.00% and 19.48%, while for 500 iterations the same range will be 14.83% to 20.97%. With more iterations and more degrees of freedom (which means running more instances of the simulator) for the same level of confidence we can estimate a narrower range. There is a subtle but important difference between that statement and saying that at more iterations the estimated probability is more accurate.