The Central Limit Theorem – with Dragons

To quote Willy Wonka, “A little magic now and then is relished by the best of men [and women].” Any frequent reader of this blog will know that I am of a pragmatic nature when it comes to using statistics. For most people the Central Limit Theorem can remain in the realms of magic. I have never taught it, though at times I have waved my hands past it.

Sometimes you don’t need to know.

Students who want that sort of thing can read about it in their textbooks or look it up online. The New Zealand school curriculum does not include it, as I explained in 2012.

But – there are many curricula and introductory statistics courses that include The Central Limit Theorem, so I have chosen to blog about it, in preparation to making a video. In this post I will cover what the Central Limit does. Maybe my approach will give ideas to teachers on how they might teach it.

Sampling distribution of a mean

First let me explain what a sampling distribution is. (And let me add the term to Dr Nic’s long list of statistics terms that cause unnecessary confusion.) A sampling distribution of a mean is the distribution of the means of samples of the same size taken from the same population. The distribution of the means will be different from the distribution of values in the original population.  The Central Limit Theorem tells us useful things about the sampling distribution and its relationship to the distribution of the values in the population.

Example using dragons

We have a population of 720 dragons, and each dragon has a strength value of 1 to 8. The distribution of the strengths goes from 1 to 8 and has a population mean somewhere around 4.5. We take a sample of four dragons from the population. (Dragons are difficult to catch and measure so it will just be 4.)

We find the mean. Then we think about what other values we might have got for samples that size. In real life, that is all we can do. But to understand what is happening, we will take multiple samples using cards, and then a spreadsheet, to explore what happens.

Important aspects of the Central Limit Theorem

Aspect 1: The sampling distribution will be less spread than the population from which it is drawn.

Dragon example

What do you think is the largest value the mean strength of the four dragons will take? Theoretically you could have a sample of four dragons, each with strength of 8, giving us a sample mean of 8. But it isn’t very likely. The chances that all four values are greater than the mean are pretty small.  (It’s about a 6% chance). If there are equal numbers of dragons with each strength value, then the probability of getting all four dragons with strength 8 is 0.0002.

So already we have worked out that the distribution of the sample means is going to be less spread than the distribution of the original population.

Aspect 2: The sampling distribution will be well-modelled by a normal distribution.

Now isn’t that amazing – and really useful! And even more amazing, it doesn’t even matter what the underlying population distribution is, the sampling distribution will still (in most cases) look like a normal distribution.

If you think about it, it does make sense. I like to see practical examples – so here is one!

Dragon example

We worked out that it was really unlikely to get a sample of four dragons with a mean strength of 8. Similarly it is really unlikely to get a sample of four dragons with a mean strength of 1.
Say we assumed that the strength of dragons was uniform – there are equal numbers of dragons with each of the strengths. Then we find out all the possible combinations of strengths from samples of 4 dragons. Bearing in mind there are eight different strengths, that gives us 8 to the power of 4 or 4096 possible combinations. We can use a spreadsheet to enumerate all these equally likely combinations. Then we find the mean strength and we get this distribution.

Or we could take some samples of four dragons and see what happens. We can do this with our cards, or with a handy spreadsheet, and here is what we get.

Four samples of four dragons each

The sample mean values are 4.25, 5.25, 4.75 and 6. Even with really small samples we can see that the values of the means are clustering around some central point.

Here is what the means of 1000 samples of size 4 look like:

And hey presto – it resembles a normal distribution! By that I mean that the distribution is symmetric, with a bulge in the middle and tails in either direction. A normal distribution is useful for modelling just about anything that is the result of a large number of change effects.

The bigger the sample size and the more samples we take, the more the distribution of the means (the sampling distribution) looks like a normal distribution. The Central Limit Theorem gives mathematical explanation for this. I put this in the “magic” category unless you are planning to become a theoretical statistician.

Aspect 3: The spread of the sampling distribution is related to the spread of the population.

If you think about it, this also makes sense. If there is very little variation in the population, then the sample means will all be about the same.  On the other hand, if the population is really spread out, then the sample means will be more spread out too.

Dragon example

Say the strengths of the dragons occur equally from 1 to 5 instead of from 1 to 8. The spread of the means of teams of four dragons are going to go from 1 to 5 also, though most of the values will be near the middle.

Aspect 4: Bigger samples lead to a smaller spread in the sampling distribution.

As we increase the size of the sample, the means become less varied. We reduce the effect of one extreme value. Similarly the chance of getting all high values in our sample or all low values gets smaller and smaller. Consequently the spread of the sample means will decrease. However, the reduction is not linear. By that I mean that the effect achieved by adding one more to the sample decreases, depending on how big the sample is in the first place. Say you have a sample of size n = 4, and you increase it to n = 5, that is a 25% increase in information. If you have a sample n = 100 and increase it to size n=101, that is only a 1% increase in information.

Now here is the coolest thing! The spread of the sampling distribution is the standard deviation of the population, divided by the square root of the sample size. As we do not know the standard deviation of the population (σ), we use the standard deviation of the sample (s) to approximate it. The spread of the sampling distribution is usually called the standard error, or s.e.


Implications of the Central Limit Theorem

The properties listed above underpin most traditional statistical inference. When we find a confidence interval of a mean, we use the standard error in the formula. If we used the sample standard deviation we would be finding the values between which most of the values in the sample lie. By using the standard error, we are finding the values between which most of the sample means lie.

Sample size

The Central Limit Theorem applies best with large samples. A rule of thumb is that the sample should be 30 or more. For smaller samples we need to use the t distribution rather than the normal distribution in our testing or confidence intervals. If the sample is very small, such as less than 15, then we can still use the t-distribution if the underlying population has a normal shape. If the underlying population is not normal, and the sample is small, then other methods, such as resampling should be used, as the Central Limit Theorem does not hold.


We do not take multiple samples of the same population in real life. This simulation is just that – a pretend example to show how the Central Limit Theorem plays out. When we undergo inferential statistics we have one sample, and from that we use what we know about it to make inferences about the population from which it is drawn.

Teaching suggestion

Data cards are extremely useful tools to help understand sampling and other aspects of inference. I would suggest getting the class to take multiple small samples(n=4), using cards, and finding the means. Plot the means. Then take larger samples (n=9) and similarly plot the means. Compare the shape and spread of the distributions of the means.

The Dragonistics data cards used in this post can be purchased at The StatsLC shop.


Videos for teaching and learning statistics

It delights me that several of my statistics videos have been viewed over half a million times each. As well there is a stream of lovely comments (with the odd weird one) from happy viewers, who have found in the videos an answer to their problems.

In this post I will outline the main videos available on the Statistics Learning Centre YouTube Channel. They already belong to 24,000 playlists and lists of recommended resources in textbooks the world over. We are happy for teachers and learners to continue to link to them. Having them all in one place should make it easier for instructors to decide which ones to use in their courses.

Philosophy of the videos

Early on in my video production I wrote a series of blog posts about the videos. One was Effective multimedia teaching videos. The videos use graphics and audio to increase understanding and retention, and are mostly aimed at conceptual understanding rather than procedural understanding.

I also wrote a critique of Khan Academy videos, explaining why I felt they should be improved. Not surprisingly this ruffled a few feathers and remains my most commented on post. I would be thrilled if Khan had lifted his game, but I fear this is not the case. The Khan Academy pie chart video still uses an unacceptable example with too many and ordered categories. (January 2018)

Before setting out to make videos about confidence intervals, I critiqued the existing offerings in this post. At the time the videos were all about how to find a confidence interval, and not what it does. I suspect that may be why my video, Understanding Confidence Intervals, remains popular.

Note to instructors

You are welcome to link to our YouTube channel, and we get a tiny amount of money from people clicking on the ads. Please do NOT download the videos, as it is against YouTube rules, and deprives us of income. Note that we also have a separate pay-to-view channel, with considerably more videos, at higher resolution, with no advertising. Email us at for free trial access to these videos, with a view to providing them for your students on a subscription basis. If you have trouble with reliable internet access, we can also provide the videos as files for your network as part of the licence.

Introducing statistics

Understanding Summary Statistics 5:14 minutes

Why we need summary statistics and what each of them does. It is not about how to calculate the statistics, but what they mean. It uses the shoe example, which also appears in the PPDAC and OSEM videos.

Understanding Graphs 6:06 minutes

I briefly explains the use and interpretation of seven different types of statistical graph. They include the pictogram, bar chart, pie chart, dot plot, stem and leaf, scatterplot and time series.


Analysing and commenting on Graphical output using OSEM 7:13 minutes

This video teaches how to comment on graphs and other statistical output by using the acronym OSEM. It is especially useful for students in NCEA statistics classes in New Zealand, but many people everywhere can find OSEM awesome! We use the example of comparing the number of pairs of shoes men and women students say they own.

Variation and Sampling error 6:30 minutes

Statistical methods are necessary because of the existence of variation. Sampling error is one source of variation, and is often misunderstood. This video explains sampling error, along with natural variation, explainable variation and variation due to bias. There is an accompanying video on non-sampling error.

Sampling methods 4:54 minutes 500,000 views

This video describes five common methods of sampling in data collection – simple random, convenience, systematic, cluster and stratified. Each method has a helpful symbolic representation.

Types of data 6:20 minutes 600,000 views

The kind of graph and analysis we can do with specific data is related to the type of data it is. In this video we explain the different levels of data, with examples. This video is particularly popular at the start of courses.

Important Statistical concepts 5:34 minutes 50,000 views

This video does not receive the views it deserves, as it covers three really important ideas. Maybe I should split it up into three videos. The ideas are the difference between significance and usefulness, evidence and strength of effect, causation and association.

Other videos complementary to these, but not on YouTube are:

  • The statistical enquiry process
  • Understanding the Box Plot
  • Non-sampling error

Videos for teaching hypothesis testing

Understanding Statistical inference 6:46 minutes 40,000 views

The most difficult concept in statistics is that of inference. This video explains what statistical inference is and gives memorable examples. It is based on research around three concepts pivotal to inference – that the sample is likely to be a good representation of the population, that there is an element of uncertainty as to how well the sample represents the population, and that the way the sample is taken matters.

Understanding the p-value 4:43 minutes 500,000 views

This video explains how to use the p-value to draw conclusions from statistical output. It includes the story of Helen, making sure that the choconutties she sells have sufficient peanuts. It introduces the helpful phrase “p is low, null must go”.

Inference and evidence 3:34 minutes

This is a newer video, based on a little example I used in lectures to help students see the link between evidence and inference. Of course it involves chocolate.

Hypothesis tests 7:38 minutes 350,000 views

This entertaining video works step-by-step through a hypothesis test. Helen wishes to know whether giving away free stickers will increase her chocolate sales. This video develops the ideas from “Understanding the p-value”, giving more of the process of hypothesis testing. It is also complemented by the following video, that shows how to perform the analysis using Excel.

Two-means t-test in Excel 3:54 minutes 50,000 views

A step-by-step lesson on how to perform an independent samples t-test for difference of two means using the Data Analysis ToolPak in Excel. This is a companion video to Hypothesis tests, p-value, two means t-test.

Choosing which statistical test to use 9:33 minutes 500,000 views

I am particularly proud of this video, and the way it links the different tests together. It took a lot of work to come up with this. First it outlines a process for thinking about the data, the sample and the thing you are trying to find out. Then it works through seven tests with scenarios based around Helen and the Choconutties. This video is particularly popular near the end of the semester, for tying together the different tests and applications.


Confidence Intervals

Understanding Confidence Intervals 4:02 minutes 500,000 views

This short video gives an explanation of the concept of confidence intervals, with helpful diagrams and examples. The emphasis is on what a confidence interval is and how it is used, rather than how they are calculated or derived.

Calculating the confidence interval for a mean using a formula 5:29 minutes 200,000 views

This video carries on from “Understanding Confidence Intervals” and introduces a formula for calculating a confidence interval for a mean. It uses graphics and animation to help understanding.

There are also videos pertinent to the New Zealand curriculum using bootstrapping and informal methods to find confidence intervals.


Introduction to Probability 2:54 minutes

This video explains what probability is and why we use it. It does NOT use dice, coins or balls in urns. It is the first in a series of six videos introducing basic probability with a conceptual approach. The other five videos can be accessed through subscription.

Understanding Random Variables 5:08 minutes 90,000 views

The idea of a random variable can be surprisingly difficult. In this video we help you learn what a random variable is, and the difference between discrete and continuous random variables. It uses the example of Luke and his ice cream stand.

Understanding the Normal Distribution 7:44 minutes

In this video we explain the characteristics of the normal distribution, and why it is so useful as a model for real-life entities.

There are also two other videos about random variables, discrete and continuous.

Risk and Screening 7:54 minutes

This video explains about risk and screening, and shows how to calculate and express rates of false positives and false negatives. An imaginary disease, “Earpox” is used for the examples.

Other videos

Designing a Questionnaire 5:23 minutes 40,000 views

This was written specifically to support learning in Level 1 NCEA in the NZ school system but is relevant for anyone needing to design a questionnaire. There is a companion video on good and bad questions.

Line-fitting and regression

Scatterplots in Excel 5:17 minutes

The first step in doing a regression in Excel is to fit the line using a Scatter plot. This video shows how to do this, illustrated by the story of Helen and the effect of temperature on her sales of choconutties

Regression in Excel 6:27 minutes

This video explains Regression and how to perform regression in Excel and interpret the output. The story of Helen and her choconutties continues. This follows on from Scatterplots in Excel and Understanding the p-value.

There are three videos introducing bivariate relationships in a more conceptual way.

There are also videos covering experimental design and randomisation, time series analysis and networks. In the pipeline is a video “understanding the Central Limit Theorem.”

Supporting our endeavours

As explained in a previous post, Lessons for a budding Social Enterprise, Statistics Learning Centre is a social enterprise, with our aim to build a world of mathematicians and enable people to make intelligent use of statistics. Though we get some income from YouTube videos, it does not support the development of more videos. If you would like to help us to create further videos contact us to discuss subscriptions, sponsorship, donations and advertising possibilities. or

Dragon Trainer rich mathematical task

I love rich mathematical tasks. Here is one for all levels of schooling. What do you think?

Background to rich tasks

A rich task is an open-ended task that students can engage with at multiple levels. I use the following information from the nrich website when I am talking to teachers about rich tasks.

Some important aspects of rich mathematical tasks

Background to Dragonistics data cards

In this task we use our Dragonistics data cards, which are shown here. For a less colourful exercise you could use 24 pieces of card with numbers 1 to 8 on them.

A small sample of Dragonistics data cards

Each dragon has a strength rating of between 1 and 8, shown by the coloured dragon scales on the right-hand side of the card. The distribution of dragon strengths is not uniform, but is clustered around the middle, and depends to a certain extent on other aspects of the dragon, such as their species, gender and behaviour.

The students will already be familiar with the dragon cards, and each group of students has a set of about 24 dragonistics data cards. As there are a total of 288 dragons, each group will have a different set of dragons. Some may or may not have dragons of each strength rating.

The task

A dragon team trainer says that teams of two dragons chosen at random nearly always have a combined strength of between 7 and 11.
Is this true?
Provide evidence to support your conclusion.

Try it yourself

If you do not have any dragons of your own, make up about 20 pieces of card, with the numbers 1 to 8 on them, so you can explore the problem. Like Tracy Zager, we emphasise the necessity of exploring the maths ourselves before the children.

Possible approaches

What is great about this exercise is that you can explore it experimentally or theoretically. It has a low entry point, as encouraged on Youcubed. This is sometimes called “low floor, high ceiling”.For younger children, it is a good start to take pairs of dragons, add their strengths, and write down the answer. Then they need to work out a recording method, possibly a tally table.  You can have discussions about what it means for the dragons to be chosen randomly. You can also discuss what “nearly always” means.

Recently I used this task with a group of ten-year-olds. After they had made an attempt at solving it, I asked what they thought would be the most common team strength, and one said 9 or 10 because it is in the middle. I should have explored this idea further. What I did do, was start working out how many different combinations were possible. It is not possible to have a team of strength 1, and there is only one way to get a team of strength 2. How many ways to get strength 3? By the time we got to strength 6, they could see a pattern, that the number of combinations is one less than the total strength. So then I jumped to the other end of the distribution, asking “What is the strongest team we could possibly get?” As it happened, they did have two dragons of strength 8 in their set of dragons, so they correctly estimated the answer to be 16. So then I asked how many different ways they could get 16, and using their previous rule, they suggested 15 ways.  Then when I asked them to tell me what they were, they realised that there was only one way. From there we started working down the numbers. Unfortunately this was during a holiday programme, so I didn’t have time to pursue this further. However we will be using this exercise in our rural rich maths events.

Lessons to bring out at different levels

There are three main ways to approach this problem. The first is to experiment by randomly taking pairs of dragons, and recording their total strengths. A simple theoretical model involves thinking about all the possible outcomes and seeing what proportion of the outcomes lies between the chosen values. Then a more refined model would take into account the distributions of strengths for the given dragons.  The learners may well come up with some interesting other ways to go about this.

Extension questions

A teacher can encourage further thinking with questions such as:

Would this answer be the same for every group of dragons? Is it possible to find a set of dragons so that the only team strengths are between 5 and 11? What would happen if you had teams of three dragons. Does it make a difference if you select one team at a time, and shuffle, or divide into lots of teams and record, before shuffling? How many different team possibilities are possible? What if you only had green dragons – would this make a difference?

Show them the maths

It is important to point out the mathematical skills they are exercising as they tackle rich tasks. This task improves number skills, encourages persistence and risk-taking, develops communication skills as they are required to justify their conclusion. At higher levels it is helping to develop understanding of probability distributions, and you could also introduce or reinforce the idea of a random variable – in this case the team strength.

It would also be interesting to look at the spread for single dragons, two dragon teams and three dragon teams. With enough repetitions (and at this point a spreadsheet could be handy) the central limit theorem will start to be apparent. As you can see, there is great potential to expand this.


We need to look at ways this is also applicable in daily life, and not just for dragon trainers. The same sort of problem would occur if you had people buying different numbers of items, or different weights of suitcases. You might like to think of the combined strengths as similar to total scores in sports events. At higher levels you might discuss the concept of independence.

So rich – so many possibilities! Thoughts?

The normal distribution – three tricky bits

There are several tricky things about teaching and understanding the normal distribution, and in this post I’m going to talk about three of them. They are the idea of a model, the limitations of the normal distribution, and the idea of the probability being the area under the graph.

It’s a model!

When people hear the term distribution, they tend to think of the normal distribution. It is an appealing idea, and remarkably versatile. The normal distribution is an appropriate model for the outcome of many natural, manufacturing and human endeavours. However, it is only a model, not a rule. But sometimes the way we talk about things as “being normally distributed” can encourage incorrect thinking.

This problem can be seen in exam questions about the application of the normal distribution. They imply that the normal distribution controls the universe.

Here is are examples of question starters taken from a textbook:

  1. “The time it takes Steve to walk to school follows a normal distribution with mean 30 minutes…”.
  2. Or “The time to failure for a new component is normally distributed with a mean of…”

This terminology is too prescriptive. There is no rule that says that Steve has to time his walks to school to fit a certain distribution. Nor does a machine create components that purposefully follow a normal distribution with regard to failure time. I remember, as a student being intrigued by this idea, not really understanding the concept of a model.

When we are teaching, and at other times, it is preferable to say that things are appropriately modelled by a normal distribution. This reminds students that the normal distribution is a model. The above examples could be rewritten as

  1. “The time it takes Steve to walk to school is appropriately modelled using a normal distribution with mean 30 minutes…”.
  2. And  “The time to failure for a new component is found to have a distribution well modelled by the normal, with a mean of…”

They may seem a little clumsy, but send the important message that the normal distribution is the approximation of a random process, not the other way around.

Not everything is normal

It is also important that students do not get the idea that all distributions, or even all continuous distributions are normal. The uniform distribution and negative exponential distributions are both useful in different circumstances, and look nothing like the normal distribution. And distributions of real entities can often have many zero values, that make a distribution far from normal-looking.

The normal distribution is great for things that measure mostly around a central value, and there are increasingly fewer things as you get further from the mean in both directions. I suspect most people can understand that in many areas of life you get lots of “average” people or things, and some really good and some really bad. (Except at Lake Wobegon “where all the women are strong, all the men are good looking, and all the children are above average.”)

However the normal distribution is not useful for modelling distributions that are heavily skewed. For instance, house prices tend to have a very long tail to the right, as there are some outrageously expensive houses, even several times the value of the median. At the same time there is a clear lower bound at zero, or somewhere above it.

Inter-arrival times are not well modelled by the normal distribution, but are well modelled by a negative exponential distribution. If we want to model how long it is likely to be before the next customer arrives, we would not expect there to be as many long times as there are short times, but fewer and fewer arrivals will occur with longer gaps.

Daily rainfall is not well modelled by the normal distribution as there will be many days of zero rainfall. Amount claimed in medical insurance or any kind of insurance are not going to be well modelled by the normal distribution as there are zero claims, and also the effect of excesses. Guest stay lengths at a hotel would not be well modelled by the normal distribution. Most guests will stay one or two days, and the longer the time, the fewer people would stay that long.

Area under the graph – idea of sand

The idea of the area under the graph being the probability of an outcome’s happening in that range is conceptually challenging. I was recently introduced to the sand metaphor by Holly-Lynne  and Todd Lee. If you think about each outcome as being a grain of sand (or a pixel in a picture) then you think about how likely it is to occur, by the size of the area that encloses it. I found the metaphor very appealing, and you can read the whole paper here:

Visual representations of empirical probability distributions when using the granular density metaphor

There are other aspects of the normal distribution that can be challenging. Here is our latest video to help you to teach and learn and understand the normal distribution.

Introducing Probability

I have a guilty secret. I really love probability problems. I am so happy to be making videos about probability just now, and conditional probability and distributions and all that fun stuff. I am a little disappointed that we won’t be doing decision trees with Bayesian review, calculating EVPI. That is such fun, but I gave up teaching that some years ago.

The reason probability is fun is because it is really mathematics, and puzzles and logic. I love permutations and combinations too – there is something cool about working out how many ways something can happen.

So why should I feel guilty? Well, in all honesty I have to admit that there is very little need for most of that in a course about statistics at high-school or entry level university. When I taught statistical methods for management, we did some probability, but only from an applied viewpoint, and we never touched intersection and union signs or anything like that. We applied some distributions, but without much theoretical underpinning.

The GAISE (Guidelines for Assessment and Instruction in Statistics Education) Report says, “Teachers and students must understand that statistics and probability are not the same. Statistics uses probability, much as physics uses calculus.”

The question is, why do we teach probability – apart from the fact that it’s fun and makes a nice change from writing reports on time series and bivariate analysis, inference and experiments. The GAISE report also says, “Probability is an important part of any mathematical education. It is a part of mathematics that enriches the subject as a whole by its interactions with other uses of mathematics. Probability is an essential tool in applied mathematics and mathematical modeling. It is also an essential tool in statistics.”

The concept of probability is as important as it is misunderstood. It is vital to have an understanding of the nature of chance and variation in life, in order to be a well-informed, (or “efficient”) citizen. One area in which this is extremely important is in understanding risk and relative risk. When a person is told that their chances of dying of some rare disease have just doubled, it is important that they know that it may be because they have gone from one chance in a million to two chances in a million. Sure it has doubled, but it still is pretty trivial. An understanding of probability is also important in terms of gambling and resistance to the allures of games of chance. And more socially acceptable gambling, such as stockmarket trading, also requires an understanding of chance and variation.

The concept of probability is important, and a few rules of probability may help with understanding, but I suspect the mathematicians get carried away and create problems that are unlikely (probability close to zero) to ever occur in reality. Anything requiring a three-way Venn Diagram has moved from applied problem to logic puzzle.This is in stark contrast to the very applied data-driven approach used in teaching statistics in New Zealand.

Teaching Probability

The traditional approach to teaching probability is to start with the coin and the dice and the balls in the urns. As well as being mind-bogglingly boring and pointless, this also projects an artificial certainty about the probabilities, which is confusing when we start discussing models. If you look at the Khan Academy videos (but don’t) you will find trivial examples about coloured balls or sweets or strangely complex problems involving hitting a circular target. The traditional approach is also to teach probability as truth. “The probability of getting a boy is one-half”. What does that even mean?

I am currently reading the new Springer volume, Probabilistic Thinking, and intend to write a review and post it on this blog, if I can get through enough before my review copy expires. It is inspiring and surprisingly gripping (but I don’t think that is enough of a review to earn me a hard copy to keep.). There are many great ideas for teaching in it, that I hope to pass on in due time.

The New Zealand approach to teaching probability comes from a modelling perspective, right from the start. At level 1, the first two years of schooling, children are exploring chance situations, playing games with a chance element and describing possible outcomes. By years 5 and 6 they are assigning numeric values to the likelihood of an occurrence. They (in the curriculum) are being introduced to model estimates and experimental estimates of probability. Bearing in mind how difficult high school maths teachers are finding the new approach, I don’t have a lot of confidence that the primary teachers are equipped yet to make the philosophical changes, let alone enact them in the classroom.

Deterministic and Probabilistic models and thinking

The way we understand and make sense of variation in the world affects decisions we make.

Part of understanding variation is understanding the difference between deterministic and probabilistic (stochastic) models. The NZ curriculum specifies the following learning outcome: “Selects and uses appropriate methods to investigate probability situations including experiments, simulations, and theoretical probability, distinguishing between deterministic and probabilistic models.” This is at level 8 of the curriculum, the highest level of secondary schooling. Deterministic and probabilistic models are not familiar to all teachers of mathematics and statistics, so I’m writing about it today.


The term, model, is itself challenging. There are many ways to use the word, two of which are particularly relevant for this discussion. The first meaning is “mathematical model, as a decision-making tool”. This is the one I am familiar with from years of teaching Operations Research. The second way is “way of thinking or representing an idea”. Or something like that. It seems to come from psychology.

When teaching mathematical models in entry level operations research/management science we would spend some time clarifying what we mean by a model. I have written about this in the post, “All models are wrong.”

In a simple, concrete incarnation, a model is a representation of another object. A simple example is that of a model car or a Lego model of a house. There are aspects of the model that are the same as the original, such as the shape and ability to move or not. But many aspects of the real-life object are missing in the model. The car does not have an internal combustion engine, and the house has no soft-furnishings. (And very bumpy floors). There is little purpose for either of these models, except entertainment and the joy of creation or ownership. (You might be interested in the following video of the Lego Parisian restaurant, which I am coveting. Funny way to say Parisian!)

Many models perform useful functions. My husband works as a land-surveyor, and his work involves making models on paper or in the computer, of phenomenon on the land, and making sure that specified marks on the model correspond to the marks placed in the ground. The purpose of the model relates to ownership and making sure the sewers run in the right direction. (As a result of several years of earthquakes in Christchurch, his models are less deterministic than they used to be, and unfortunately many of our sewers ended up running the wrong way.)

Our world is full of models:

  • a map is a model of a location, which can help us get from place to place.
  • sheet music is a written model of the sound which can make a song
  • a bus timetable is a model of where buses should appear
  • a company’s financial reports are a model of one aspect of the company

Deterministic models

A deterministic model assumes certainty in all aspects. Examples of deterministic models are timetables, pricing structures, a linear programming model, the economic order quantity model, maps, accounting.

Probabilistic or stochastic models

Most models really should be stochastic or probabilistic rather than deterministic, but this is often too complicated to implement. Representing uncertainty is fraught. Some more common stochastic models are queueing models, markov chains, and most simulations.

For example when planning a school formal, there are some elements of the model that are deterministic and some that are probabilistic. The cost to hire the venue is deterministic, but the number of students who will come is probabilistic. A GPS unit uses a deterministic model to decide on the most suitable route and gives a predicted arrival time. However we know that the actual arrival time is contingent upon all sorts of aspects including road, driver, traffic and weather conditions.

Model as a way of thinking about something

The term “model” is also used to describe the way that people make sense out of their world. Some people have a more deterministic world model than others, contributed to by age, culture, religion, life experience and education. People ascribe meaning to anything from star patterns, tea leaves and moon phases to ease in finding a parking spot and not being in a certain place when a coconut falls. This is a way of turning a probabilistic world into a more deterministic and more meaningful world. Some people are happy with a probabilistic world, where things really do have a high degree of randomness. But often we are less happy when the randomness goes against us. (I find it interesting that farmers hit with bad fortune such as a snowfall or drought are happy to ask for government help, yet when there is a bumper crop, I don’t see them offering to give back some of their windfall voluntarily.)

Let us say the All Blacks win a rugby game against Australia. There are several ways we can draw meaning from this. If we are of a deterministic frame of mind, we might say that the All Blacks won because they are the best rugby team in the world.  We have assigned cause and effect to the outcome. Or we could take a more probabilistic view of it, deciding that the probability that they would win was about 70%, and that on the day they were fortunate.  Or, if we were Australian, we might say that the Australian team was far better and it was just a 1 in 100 chance that the All Blacks would win.

I developed the following scenarios for discussion in a classroom. The students can put them in order or categories according to their own criteria. After discussing their results, we could then talk about a deterministic and a probabilistic meaning for each of the scenarios.

  1. The All Blacks won the Rugby World Cup.
  2. Eri did better on a test after getting tuition.
  3. Holly was diagnosed with cancer, had a religious experience and the cancer was gone.
  4. A pet was given a homeopathic remedy and got better.
  5. Bill won $20 million in Lotto.
  6. You got five out of five right in a true/false quiz.

The regular mathematics teacher is now a long way from his or her comfort zone. The numbers have gone, along with the red tick, and there are no correct answers. This is an important aspect of understanding probability – that many things are the result of randomness. But with this idea we are pulling mathematics teachers into unfamiliar territory. Social studies, science and English teachers have had to deal with the murky area of feelings, values and ethics forever.  In terms of preparing students for a random world, I think it is territory worth spending some time in. And it might just help them find mathematics/statistics relevant!

On-line learning and teaching resources

Twenty-first century Junior Woodchuck Guidebook

I grew up reading Donald Duck comics. I love the Junior Woodchucks, and their Junior Woodchuck Guidebook. The Guidebook is a small paperback book, containing information on every conceivable subject, including geography, mythology, history, literature and the Rubaiyat of Omar Khayyam.  In our family, when we want to know something or check some piece of information, we talk about consulting the Junior Woodchuck Guidebook. (Imagine my joy when I discovered that a woodchuck is another name for a groundhog, the star of my favourite movie!) What we are referring to is the internet, the source of all possible information! Thanks to search engines, there is very little we cannot find out on the internet. And very big thanks to Wikipedia, to which I make an annual financial contribution, as should all who use it and can afford to.

You can learn just about anything on the internet. Problem is, how do you know what is good? And how do you help students find good stuff? And how do you use the internet wisely? And how can it help us as learners and teachers of statistics and operations research? These questions will take more than my usual 1000 words, so I will break it up a bit. This post is about the ways the internet can help in teaching and learning. In a later post I will talk about evaluating resources, and in particular multimedia resources.


Both the disciplines in which I am interested, statistics and operations research, apply mathematical and analytic methods to real-world problems. In statistics we are generally trying to find things out, and in operations research we are trying to make them better. Either way, the context is important. The internet enables students to find background knowledge regarding the context of the data or problem they are dealing with. It also enables instructors to write assessments and exercises that have a degree of veracity to them even if the actual raw data proves elusive. How I wish people would publish standard deviations as well as means when reporting results!


Which brings us to the second use for on-line resources. Real problems with real data are much more meaningful for students, and totally possible now that we don’t need to calculate anything by hand. Sadly, it is more difficult than first appears to find good quality raw data to analyse, but there is some available. You can see some sources in a previous post and the helpful comments.


If you are struggling to understand a concept, or to know how to teach or explain it, do a web search. I have found some great explanations, and diagrams especially, that have helped me. Or I have discovered a dearth of good diagrams, which has prompted me to make my own.


Videos can help with background knowledge, with explanations, and with inspiring students with the worth of the discipline. The problem with videos is that it takes a long time to find good ones and weed out the others. One suggestion is to enlist the help of your students. They can each watch two or three videos and decide which are the most helpful. The teacher then watches the most popular ones to check for pedagogical value. It is great when you find a site that you can trust, but even then you can’t guarantee the approach will be compatible with your own.

Social support

I particularly love Twitter, from which I get connection with other teachers and learners, and ideas and links to blogs. I belong to a Facebook group for teachers of statistics in New Zealand, and another Facebook group called “I love Operations Research”. These wax and wane in activity, and can be very helpful at times. Students and teachers can gain a lot from social networking.


There is good open-source software available, and 30-day trial versions for other software. Many schools in New Zealand use the R-based iNZight collection of programmes, which provide purpose-built means for timeseries analysis, bootstrapping and line fitting.

Answers to questions

The other day I lost the volume control off my toolbar. (Windows Vista, I’m embarrassed to admit). So I put in the search box “Lost my volume control” and was directed to a YouTube video that took me step-by-step through the convoluted process of reinstating my volume control! I was so grateful I made a donation. Just about any computer related question can be answered through a search.

Interactive demonstrations

I love these. There are two sites I have found great:

The National Library of Virtual Manipulatives, based in Utah.

NRich – It has some great ideas in the senior statistics area. From the UK.

A problem with some of these is the use of Flash, which does not play on all devices. And again – how do we decide if they are any good or not?

On-line textbooks

Why would you buy a textbook when you can get one on-line. I routinely directed my second-year statistical methods for business students to “Concepts and Applications of Inferential Statistics”. I’ve found it just the right level. Another source is Stattrek. I particularly like their short explanations of the different probability distributions.

Practice quizzes

There aren’t too many practice quizzes  around for free. Obviously, as a provider of statistical learning materials, I believe quizzes and exercises have merit for practice with immediate and focussed feedback. However, it can be very time-consuming to evaluate practice quizzes, and some just aren’t very good. On the other hand, some may argue that any time students spend learning is better than none.

Live help

There are some places that provide live, or slightly delayed help for students. I got hooked into a very fun site where you earned points by helping students. Sadly I can’t find it now, but as I was looking I found vast numbers of on-line help sites, often associated with public libraries. And there are commercial sites that provide some free help as an intro to their services. In New Zealand there is the StudyIt service, which helps students preparing for assessments in the senior high school years. At StatsLC we provide on-line help as part of our resources, and will be looking to develop this further. From time to time I get questions as a result of my YouTube videos, and enjoy answering them ,unless I am obviously doing someone’s homework! I also discovered “ShowMe” which looks like a great little iPad app, that I can use to help people more.

This has just been a quick guide to how useful the internet can be in teaching and learning. Next week I will address issues of quality and equity.

Open Letter to Khan Academy about Basic Probability

Khan academy probability videos and exercises aren’t good either

Dear Mr Khan

You have created an amazing resource that thousands of people all over the world get a lot of help from. Well done. Some of your materials are not very good, though, so I am writing this open letter in the hope that it might make some difference. Like many others, I believe that something as popular as Khan Academy will benefit from constructive criticism.

I fear that the reason that so many people like your mathematics videos so much is not because the videos are good, but because their experience in the classroom is so bad, and the curriculum is poorly thought out and encourages mechanistic thinking. This opinion is borne out by comments I have read from parents and other bloggers. The parents love you because you help their children pass tests.  (And these tests are clearly testing the type of material you are helping them to pass!) The bloggers are not so happy, because you perpetuate a type of mathematical instruction that should have disappeared by now. I can’t even imagine what the history teachers say about your content-driven delivery, but I will stick to what I know. (You can read one critique here)

Just over a year ago I wrote a balanced review of some of the Khan Academy videos about statistics. I know that statistics is difficult to explain – in fact one of the hardest subjects to teach. You can read my review here. I’ve also reviewed a selection of videos about confidence intervals, one of which was from Khan Academy. You can read the review here.

Consequently I am aware that blogging about the Khan Academy in anything other than glowing terms is an invitation for vitriol from your followers.

However, I thought it was about time I looked at the exercises that are available on KA, wondering if I should recommend them to high school teachers for their students to use for review. I decided to focus on one section, introduction to probability. I put myself in the place of a person who was struggling to understand probability at school.

Here is the verdict.

First of all the site is very nice. It shows that it has a good sized budget to use on graphics and site mechanics. It is friendly to get into. I was a bit confused that the first section in the Probability and Statistics Section is called “Independent and dependent events”. It was the first section though. The first section of this first section is called Basic Probability, so I felt I was in the right place. But then under the heading, Basic probability, it says, “Can I pick a red frog out of a bag that only contains marbles?” Now I have no trouble with humour per se, and some people find my videos pretty funny. But I am very careful to avoid confusing people with the humour. For an anxious student who is looking for help, that is a bit confusing.

I was excited to see that this section had five videos, and two sets of exercises. I was pleased about that, as I’ve wanted to try out some exercises for some time, particularly after reading the review from Fawn Nguyen on her experience with exercises on Khan Academy. (I suggest you read this – it’s pretty funny.)

So I watched the first video about probability and it was like any other KA video I’ve viewed, with primitive graphics and a stumbling repetitive narration. It was correct enough, but did not take into account any of the more recent work on understanding probability. It used coins and dice. Big yawn. It wastes a lot of time. It was ok. I do like that you have the interactive transcript so you can find your way around.

It dawned on me that nowhere do you actually talk about what probability is. You seem to assume that the students already know that. In the very start of the first video it says,

“What I want to do in this video is give you at least a basic overview of probability. Probability, a word that you’ve probably heard a lot of and you are probably just a little bit familiar with it. Hopefully this will get you a little deeper understanding.”

Later in the video there is a section on the idea of large numbers of repetitions, which is one way of understanding probability. But it really is a bit skimpy on why anyone would want to find or estimate a probability, and what the values actually mean. But it was ok.

The first video was about single instances – one toss of a coin or one roll of a die. Then the second video showed you how to answer the questions in the exercises, which involved two dice. This seemed ok, if rather a sudden jump from the first video. Sadly both of these examples perpetuate the common misconception that if there are, say, 6 alternative outcomes, they will necessarily be equally likely.


Then we get to some exercises called “Probability Space” , which is not an enormously helpful heading. But my main quest was to have a go at the exercises, so that is what I did. And that was not a good thing. The exercises were not stepped, but started right away with an example involving two dice and the phrase “at least one of”. There was meant to be a graphic to help me, but instead I had the message “scratchpad not available”. I will summarise my concerns about the exercises at the end of my letter. I clicked on a link to a video that wasn’t listed on the left, called Probability Space and got a different kind of video.

This video was better in that it had moving pictures and a script. But I have problems with gambling in videos like this. There are some cultures in which gambling is not acceptable. The other problem I have is with the term  “exact probability”, which was used several times. What do we mean by “exact probability”? How does he know it is exact? I think this sends the wrong message.

Then on to the next videos which were worked examples, entitled “Example: marbles from a bag, Example: Picking a non-blue marble, Example: Picking a yellow marble.” Now I understand that you don’t want to scare students with terminology too early, but I would have thought it helpful to call the second one, “complementary events, picking a non-blue marble”. That way if a student were having problems with complementary events in exercises from school, they could find their way here. But then I’m not sure who your audience is. Are you sure who your audience is?

The first marble video was ok, though the terminology was sloppy.

The second marble video, called “Example: picking a non-blue marble”, is glacially slow. There is a point, I guess in showing students how to draw a bag and marbles, but… Then the next example is of picking numbers at random. Why would we ever want to do this? Then we come to an example of circular targets. This involves some problem-solving regarding areas of circles, and cancelling out fractions including pi. What is this about? We are trying to teach about probablity so why have you brought in some complication involving the area of a circle?

The third marble video attempts to introduce the idea of events, but doesn’t really. By trying not to confuse with technical terms, the explanation is more confusing.

Now onto some more exercises. The Khan model is that you have to get 5 correct in a row in order to complete an exercise. I hope there is some sensible explanation for this, because it sure would drive me crazy to have to do that. (As I heard expressed on Twitter)

What are circular targets doing in with basic probability?

The first example is a circular target one.  I SO could not be bothered working out the area stuff so I used the hints to find the answer so I could move onto a more interesting example. The next example was finding the probability of a rolling a 4 from a fair six sided die. This is trivial, but would have been not a bad example to start with. Next question involve three colours of marbles, and finding the probability of not green. Then another dart-board one. Sigh. Then another dart board one. I’m never going to find out what happens if I get five right in a row if I don’t start doing these properly. Oh now – it gave me circumference. SO can’t be bothered.

And that was the end of Basic probability. I never did find out what happens if I get five correct in a row.

Venn diagrams

The next topic is called “Venn diagrams and adding probabilities “. I couldn’t resist seeing what you would do with a Venn diagram. This one nearly reduced me to tears.

As you know by now, I have an issue with gambling, so it will come as no surprise that I object to the use of playing cards in this example. It makes the assumption that students know about playing cards. You do take one and a half minutes to explain the contents of a standard pack of cards.  Maybe this is part of the curriculum, and if so, fair enough. The examples are standard – the probability of getting a Jack of Hearts etc. But then at 5:30 you start using Venn diagrams. I like Venn diagrams, but they are NOT good for what you are teaching at this level, and you actually did it wrong. I’ve put a comment in the feedback section, but don’t have great hopes that anything will change. Someone else pointed this out in the feedback two years ago, so no – it isn’t going to change.

Khan Venn diagram

This diagram is misleading, as is shown by the confusion expressed in the questions from viewers. There should be a green 3, a red 12, and a yellow 1.

Now Venn diagrams seem like a good approach in this instance, but decades of experience in teaching and communicating complex probabilities has shown that in most instances a two-way table is more helpful. The table for the Jack of Hearts problem would look like this:

Jacks Not Jacks Total
Hearts 1 12 13
Not Hearts 3 36 39
Total 4 48 52

(Any teachers reading this letter – try it! Tables are SO much easier for problem solving than Venn diagrams)

But let’s get down to principles.

The principles of instruction that KA have not followed in the examples:

  • Start easy and work up
  • Be interesting in your examples – who gives a flying fig about two dice or random numbers?
  • Make sure the hardest part of the question is the thing you are testing. This is particularly violated with the questions involving areas of circles.
  • Don’t make me so bored that I can’t face trying to get five in a row and not succeed.

My point

Yes, I do have one. Mr Khan you clearly can’t be stopped, so can you please get some real teachers with pedagogical content knowledge to go over your materials systematically and make them correct. You have some money now, and you owe it to your benefactors to GET IT RIGHT. Being flippant and amateurish is fine for amateurs but you are now a professional, and you need to be providing material that is professionally produced. I don’t care about the production values – keep the stammers and “lellows” in there if you insist. I’m very happy you don’t have background music as I can’t stand it myself. BUT… PLEASE… get some help and make your videos and exercises correct and pedagogically sound.

Dr Nic

PS – anyone else reading this letter, take a look at the following videos for mathematics.

And of course I think my own Statistics Learning Centre videos are pretty darn good as well.

Other posts about concerns about Khan:

Another Open Letter to Sal ( I particularly like the comment by Michael Paul Goldenberg)

Breaking the cycle (A comprehensive summary of the responses to criticism of Khan

The importance of being wrong

We don’t like to think we are wrong

One of the key ideas in statistics is that sometimes we will be wrong. When we report a 95% confidence interval, we will be wrong 5% of the time. Or in other words, about 1 in 20 of 95% confidence intervals will not contain the population parameter we are attempting to estimate. That is how they are defined. The thing is, we always think we are part of the 95% rather than the 5%. Mostly we will be correct, but if we do enough statistical analysis, we will almost definitely be wrong at some point. However, human nature is such that we tend to think it will be someone else. There is also a feeling of blame associated with being wrong. The feeling is that if we have somehow missed the true value with our confidence interval, it must be because we have made a mistake. However, this is not true. In fact we MUST be wrong about 5% of the time, or our interval is too big, and not really a 95% confidence interval.

The term “margin of error” appears with increasing regularity as elections approach and polling companies are keen to make money out of sooth-saying. The common meaning of the margin of error is half the width of a 95% confidence interval. So if we say the margin of error is 3%, then about one time in twenty, the true value of the proportion will actually be more than 3% away from the reported sample value.

What doesn’t help is that we seldom do know if we are correct or not. If we knew the real population value we wouldn’t be estimating it. We can contrive situations where we do know the population but pretend we don’t. If we do this in our teaching, we need to be very careful to point out that this doesn’t normally happen, but does in “classroom world” only. (Thanks to MD for this useful term.) General elections can give us an idea of being right or wrong after the event, but even then the problem of non-sampling error is conflated with sampling error. When opinion polls turn out to miss the mark, we tend to think of the cause as being due to poor sampling, or people changing their minds, or all number of imaginative explanations rather than simple, unavoidable sampling error.

So how do we teach this in such a way that it goes beyond school learning and is internalised for future use as efficient citizens?

Teaching suggestions

I have two suggestions. The first is a series of True/False statements that can be used in a number of ways. I have them as part of on-line assessment, so that the students are challenged by them regularly. They could be well used in the classroom as part of a warm-up exercise at the start of a lesson. Students can write their answers down or vote using hands.

Here are some examples of True/False statements (some of which could lead to discussion):

  1. You never know if your confidence interval contains the true population value.
  2. If you make your confidence interval wide enough you can be sure that you contain the true population value.
  3. A confidence interval tells us where we are pretty sure the sample statistic lies.
  4. It is better to have a narrow confidence interval than a wide one, as it gives us more certain information, even though it is more likely to be wrong.
  5. If your study involves twenty confidence intervals, then you know that exactly one of them will be wrong.
  6. If a confidence interval doesn’t contain the true population value, it is because it is one of the 5% that was calculated incorrectly.

You can check your answers at the end of this post.

Experiential exercise

The other teaching suggestion is for an experiential exercise. It requires a little set up time.

Make a set of cards for students with numbers on them that correspond to the point estimate of a proportion, or a score that will lead to that. (Specifications for a set of 35 cards representing the results from a proportion of 0.54 and 25 trials is given below).

Introduce the exercise as follows:
“I have a computer game, and have set the ratio of wins to losses at a certain value. Each of you has played 25 times, and the number of wins you have obtained will be on your card. It is really important that you don’t look at other people’s cards.”

Hand them out to the students. (If you have fewer than 35 in your class, it might be a good idea to make sure you include the cards with 8 and 19 in the set you use – sometimes it is ok to fudge slightly to teach a point.)
“Without getting information from anyone else, write down your best estimate of the true proportion of wins to losses in the game. Do you think you are correct? How close do you think you are to the true value?”

They will need to divide the number of wins by 25, which should not lead to any computational errors! The point is that they really can’t know how close their estimate is to the true value – and what does “correct” mean?

Then work out the margin of error for a sample of size 25, which in this case is estimated at 20%. Get the students to calculate their 95% confidence intervals, and decide if they have the interval that contains the true population value. Get them to commit one way or the other.

Now they can talk to each other about the values they have.

There are several ways you can go from here. You can tell them what the population proportion was from which the numbers were drawn (0.54). They can then see that most of them had confidence intervals that included the true value, and some didn’t. Or you can leave them wondering, which is a better lesson about real life. Or you can do one exercise where you do tell them and one where you don’t.

This is an area where probability and statistics meet. You could make a nice little binomial distribution problem out of being correct in a number of confidence intervals. There are potential problems with independence, so you need to be a bit careful with the wording. For example: Fifteen  students undertake separate statistical analyses on the topics of their choice, and construct 95% confidence intervals. What is the probability that all the confidence intervals are correct, in that they do contain the estimated population parameter? This is well modelled by a binomial distribution with n =15 and p=0.05. P(X=0)=0.46. And another interesting idea – what is the probability that two or more are incorrect? 0.17 is the answer. So there is a 17% chance that more than one of the confidence intervals does not contain the population parameter of interest.

This is an area that needs careful teaching, and I suspect that some teachers have only a sketchy understanding of the idea of confidence intervals and margins of error. It is so important to know that statistical results are meant to be wrong some of the time.

Answers: T,T,F, debatable, F,F.

Data for the 35 cards:

Number on card













Number of cards













Parts and whole

The whole may be greater than the sum of the parts, but the whole still needs those parts. A reflective teacher will think carefully about when to concentrate on the whole, and when on the parts.


If you were teaching someone golf, you wouldn’t spend days on a driving range, never going out on a course. Your student would not get the idea of what the game is, or why they need to be able to drive straight and to a desired length. Nor would it be much fun! Similarly if the person only played games of golf it would be difficult for them to develop their game. Practice driving and putting is needed.  A serious student of golf would also read and watch experts at golf.


Learning music is similar. Anyone who is serious about developing as a musician will spend a considerable amount of time developing their technique and their knowledge by practicing scales, chords and drills. But at the same time they need to be playing full pieces of music so that they feel the joy of what they are doing. As they play music, as opposed to drill, they will see how their less-interesting practice has helped them to develop their skills. However, as they practice a whole piece, they may well find a small part that is tripping them up, and focus for a while on that. If they play only the piece as a whole, it is not efficient use of time. A serious student of music will also listen to and watch great musicians, in order to develop their own understanding and knowledge.

Benefits of study of the whole and of the parts

In each of these examples we can see that there are aspects of working with the whole, and aspects of working with the parts. Study of the whole contributes perspective and meaning to study, and helps to tie things together. It helps to see where they have made progress. Study of the parts isolates areas of weakness, develops skills and saves time in practice, thus being more efficient.

It is very important for students to get an idea of the purpose of their study, and where they are going. For this reason I have written earlier about the need to see the end when starting out in a long procedure such as a regression or linear programming model.

It is also important to develop “statistical muscle memory” by repeating small parts of the exercise over and over until it is mastered. Practice helps people to learn what is general and what is specific in the different examples.

Teaching conditional probability

We are currently developing a section on probability as part of our learning materials. A fundamental understanding of probability and uncertainty are essential to a full understanding of inference. When we look at statistical evidence from data, we are holding it up against what we could reasonably expect to happen by chance, which involves a probability model. Probability lies in the more mathematical area of the study of statistics, and has some fun problem-solving aspects to it.

A popular exam question involves conditional probability. We like to use a table approach to this as it avoids many of the complications of terminology. I still remember my initial confusion over the counter-intuitive expression P(A|B) which means the probability that an object from subset B has the property of A. There are several places where students can come unstuck in Bayesian review, and the problems can take a long time. We can liken solving a conditional probability problem to a round of golf, or a long piece of music. So what we do in teaching is that first we take the students step by step through the whole problem. This includes working out what the words are saying, putting the known values into a table, calculating the unknown values in the table, and the using the table to answer the questions involving conditional probability.

Then we work on the individual steps, isolating them so that students can get sufficient practice to find out what is general and what is specific to different examples. As we do this we endeavour to provide variety such that students do not work out some heuristic based on the wording of the question, that actually stops them from understanding. An example of this is that if we use the same template each time, students will work out that the first number stated will go in a certain place in the table, and the second in another place etc. This is a short-term strategy that we need to protect them from in careful generation of questions.

As it turns out students should already have some of the necessary skills. When we review probability at the start of the unit, we get students to calculate probabilities from tables of values, including conditional probabilities. Then when they meet them again as part of the greater whole, there is a familiar ring.

Once the parts are mastered, the students can move on to a set of full questions, using each of the steps they have learned, and putting them back into the whole. Because they are fluent in the steps, it becomes more intuitive to put the whole back together, and when they meet something unusual they are better able to deal with it.

Starting a course in Operations Research/Management Science

It is interesting to contemplate what “the whole” is, with regard to any subject. In operations research we used to begin our first class, like many first classes, talking about what management science/operations research is. It was a pretty passive sort of class, and I felt it didn’t help as first-year university students had little relevant knowledge to pin the ideas on. So we changed to an approach that put them straight into the action and taught several weeks of techniques first. We started with project management and taught critical path. Then we taught identifying fixed and variable costs and break-even analysis. The next week was discounting and analysis of financial projects. Then for a softer example we looked at multi-criteria decision-making, using MCDM. It tied back to the previous week by taking a different approach to a decision regarding a landfill. Then we introduced OR/MS, and the concept of mathematical modelling. By then we could give real examples of how mathematical models could be used to inform real world problems. It was helpful to go from the concrete to the abstract. This was a much more satisfactory approach.

So the point is not that you should always start with the whole and then do the parts and then go back to the whole. The point is that a teacher needs to think carefully about the relationship between the parts and the whole, and teach in a way that is most helpful.