Why people hate statistics

This summer/Christmas break it has been my pleasure to help a young woman who is struggling with statistics, and it has prompted me to ask people who teach postgraduate statistical methods – WTF are you doing?

Louise (name changed) is a bright, hard-working young woman, who has finished an undergraduate degree at a prestigious university and is now doing a Masters degree at a different prestigious university, which is a long way from where I live and will remain nameless. I have been working through her lecture slides, past and future and attempting to develop in her some confidence that she will survive the remainder of the course, and that statistics is in fact fathomable.

Incomprehensible courses alienating research students

After each session with Louise I have come away shaking my head and wondering what this lecturer is up to. I wonder if he/she really understands statistics or is just passing on their own confusion. And the very sad thing is that I KNOW that there are hundreds of lecturers in hundreds of similar courses around the world teaching in much the same way and alienating thousands of students every year.

And they need to stop.

Here is the approach: You have approximately eight weeks, made up of four hour sessions, in which to teach your masters students everything they could possibly need to know about statistics. So you tell them everything! You use technical terms with little explanation, and you give no indication of what is important and what is background. You dive right in with no clear purpose, and you expect them to keep up.

Choosing your level

Frequently Louise would ask me to explain something and I would pause to think. I was trying to work out how deep to go. It is like when a child asks where babies come from. They may want the full details, but they may not, and you need to decide what level of answer is most appropriate. Anyone who has seen our popular YouTube videos will be aware that I encourage conceptual understanding at best, and the equivalent of a statistics drivers licence at worst. When you have eight weeks to learn everything there is to know about statistics, up to and including multiple regression, logistic regression, GLM, factor analysis, non-parametric methods and more, I believe the most you can hope for is to be able to get the computer to run the test, and then make intelligent conclusions about the output.

There was nothing in the course about data collection, data cleaning, the concept of inference or the relationship between the model and reality. My experience is that data cleaning is one of the most challenging parts of analysis, especially for novice researchers.

Use learning objectives

And maybe one of the worst problems with Louise’s course was that there were no specific learning objectives. One of my most popular posts is on the need for learning objectives. Now I am not proposing that we slavishly tell students in each class what it is they are to learn, as that can be tedious and remove the fun from more discovery style learning. What I am saying is that it is only fair to tell the students what they are supposed to be learning. This helps them to know what in the lecture is important, and what is background. They need to know whether they need to have a passing understanding of a test, or if they need to be able to run one, or if they need to know the underlying mathematics.

Take for example, the t-test. There are many ways that the t-statistic can be used, so simply referring to a test as a t-test is misleading before you even start. And starting your teaching with the statistic is not helpful. We need to start with the need! I would call it a test for the difference of two means from two groups. And I would just talk about the t statistic in passing. I would give examples of output from various scenarios, some of which reject the null, some of which don’t and maybe even one that has a p-value of 0.049 so we can talk about that. In each case we would look at how the context affects the implications of the test result. In my learning objectives I would say: Students will be able to interpret the output of a test for the difference of two means, putting the result in context. And possibly, Students will be able to identify ways in which a test for the difference of two means violates the assumptions of a t-test. Now that wasn’t hard was it?

Like driving a car

Louise likes to understand where things come from, so we did go through an overview of how various distributions have been found to model different aspects of the world well – starting with the normal distribution, and with a quick jaunt into the Central Limit Theorem. I used my Dragonistics data cards, which were invented for teaching primary school, but actually work surprisingly well at all levels! I can’t claim that Louise understands the use of the t distribution, but I hope she now believes in it. I gave her the analogy of learning to drive – that we don’t need to know what is happening under the bonnet to be a safe driver. In fact safe driving depends more on paying attention to the road conditions and human behaviour.


Louise tells me that her lecturer emphasises assumptions – that the students need to examine them all, every time they look at or perform a statistical test. Now I have no problems with this later on, but students need to have some idea of where they are going and why, before being told what luggage they can and can’t take. And my experience is that assumptions are always violated. Always. As George Box put it – “All models are wrong and some models are useful.”

It did not help that the lecturer seemed a little confused about the assumption of normality. I am not one to point the finger, as this is a tricky assumption, as the Andy Field textbook pointed out. For example, we do not require the independent variables in a multiple regression to be normally distributed as the lecturer specified. This is not even possible if we are including dummy variables. What we do need to watch out for is that the residuals are approximately modelled by a normal distribution, and if not, that we do something about it.

You may have gathered that my approach to statistics is practical rather than idealistic. Why get all hot and bothered about whether you should do a parametric or non-parametric test, when the computer package does both with ease, and you just need to check if there is any difference in the result. (I can hear some purists hyperventilating at this point!) My experience is that the results seldom differ.

What post-graduate statistical methods courses should focus on

Instructors need to concentrate on the big ideas of statistics – what is inference, why we need data, how a sample is collected matters, and the relationship between a model and the reality it is modelling. I would include the concept of correlation, and its problematic link to causation. I would talk about the difference between statistical significance and usefulness, and evidence and strength of a relationship. And I would teach students how to find the right fishing lessons! If a student is critiquing a paper that uses logistical regression, that is the time they need to read up enough about logistical regression to be able to understand what they are reading.They cannot possibly learn a useful amount about all the tests or methods that they may encounter one day.

If research students are going to be doing their own research, they need more than a one semester fly-by of techniques, and would be best to get advice from a statistician BEFORE they collect the data.

Final word

So here is my take-home message:

Stop making graduate statistical methods courses so outrageously difficult by cramming them full of advanced techniques and concepts. Instead help students to understand what statistics is about, and how powerful and wonderful it can be to find out more about the world through data.

Your word

Am I right or is my preaching of the devil? Please add your comments below.


It is so random! Or is it? The meaning of randomness

The concept of “random” is a tough one.

First there is the problem of lexical ambiguity. There are colloquial meanings for random that don’t totally tie in with the technical or domain-specific meanings for random.

Then there is the fact that people can’t actually be random.

Then there is the problem of equal chance vs displaying a long-term distribution.

And there is the problem that there are several conflicting ideas associated with the word “random”.

In this post I will look at these issues, and ask some questions about how we can better teach students about randomness and random sampling. This problem exists for many domain specific terms, that have colloquial meanings that hinder comprehension of the idea in question. You can read about more of these words, and some teaching ideas in the post, Teaching Statistical Language.

Lexical ambiguity

First there is lexical ambiguity. Lexical ambiguity is a special term meaning that the word has more than one meaning. Kaplan, Rogness and Fisher write about this in their 2014 paper “Exploiting Lexical Ambiguity to help students understand the meaning of Random.” I recently studied this paper closely in order to present the ideas and findings to a group of high school teachers. I found the concept of leveraging lexical ambiguity very interesting. As a useful intervention, Kaplan et al introduced a picture of “random zebras” to represent the colloquial meaning of random, and a picture of a hat to represent the idea of taking a random sample. I think it is a great idea to have pictures representing the different meanings, and it might be good to get students to come up with their own.

Representations of the different meanings of the word, random.

Representations of the different meanings of the word, random.

So what are the different meanings for random? I consulted some on-line dictionaries.

Different meanings

Without method

The first meaning of random describes something happening without pattern, method or conscious decision. An example is “random violence”.
Example: She dressed in a rather random faction, putting on whatever she laid her hand on in the dark.

Statistical meaning

Most on-line dictionaries also give a statistical definition, which includes that each item has an equal probability of being chosen.
Example: The students’ names were taken at random from a pile, to decide who would represent the school at the meeting.

Informal or colloquial

One meaning: Something random is either unknown, unidentified, or out of place.
Example: My father brought home some random strangers he found under a bridge.

Another colloquial meaning for random is odd and unpredictable in an amusing way.
Example: My social life is so random!

People cannot be random

There has been considerable research into why people cannot provide a sequence of random numbers that is like a truly randomly generated sequence. In our minds we like things to be shared out evenly and the series will generally have fewer runs of the same number.

Animals aren’t very random either, it seems. Yesterday I saw a whole lot of sheep in a paddock, and while they weren’t exactly lined up, there was a pretty similar distance between all the sheep.

Equal chance vs long-term distribution

In the paper quoted earlier, Kaplan et al used the following definition of random:

“We call a phenomenon random if individual outcomes are uncertain, but there is nonetheless a regular distribution of outcomes in a large number of repetitions.” From Moore (2007) The Basic Practice of Statistics.

Now to me, that does not insist that each outcome be equally likely, which matches with my idea of randomness. In my mind, random implies chance, but not equal likelihood. When creating simulation models we would generate random variates following all sorts of distributions. The outcomes would be far from even, but in the long run they would display a distribution similar to the one being modelled.

Yet the dictionaries, and the later parts of the Kaplan paper insist that randomness requires equal opportunity to be chosen. What’s a person to do?

I propose that the meaning of the adjective, “random” may depend on the noun that it is qualifying. There are random samples and random variables. There is also randomisation and randomness.

A random sample is a sample in which each object has an equal opportunity of being chosen, and each choice of object is by chance, and independent of the previous objects chosen. A random variable is one that can take a number of values, and will generally display a pattern of outcomes similar to a given distribution.

I wonder if the problem is that randomness is somehow equated with fairness. Our most familiar examples of true randomness come from gambling, with dice, cards, roulette wheels and lotto balls. In each case there is the requirement that each outcome be equally likely.

Bearing in mind the overwhelming evidence that the “statistical meaning” of randomness includes equality, I begin to think that it might not really matter if people equate randomness with equal opportunity.

However, if you think about medical or hazard risk, the story changes. Apart from known risk increasing factors associated with lifestyle, whether a person succumbs to a disease appears to be random. But the likelihood of succumbing is not equal to the likelihood of not succumbing. Similarly there is a clear random element in whether a future child has a disability known to be caused by an autorecessive gene. It is definitely random, in that there is an element of chance, and that the effects on successive children are independent. But the probability of a disability is one in four. I suppose if you look at the outcomes as being which children are affected, there is an equal chance for each child.

But then think about a “lucky dip” containing many cheap prizes and a few expensive prizes. The choice of prize is random, but there is not an even chance of getting a cheap prize or an expensive prize.

I think I have mused enough. I’m interested to know what the readers think. Whatever the conclusion is, it is clear that we need to spend some time making clear to the students what is meant by randomness, and a random sample.


Open Letter to Khan Academy about Basic Probability

Khan academy probability videos and exercises aren’t good either

Dear Mr Khan

You have created an amazing resource that thousands of people all over the world get a lot of help from. Well done. Some of your materials are not very good, though, so I am writing this open letter in the hope that it might make some difference. Like many others, I believe that something as popular as Khan Academy will benefit from constructive criticism.

I fear that the reason that so many people like your mathematics videos so much is not because the videos are good, but because their experience in the classroom is so bad, and the curriculum is poorly thought out and encourages mechanistic thinking. This opinion is borne out by comments I have read from parents and other bloggers. The parents love you because you help their children pass tests.  (And these tests are clearly testing the type of material you are helping them to pass!) The bloggers are not so happy, because you perpetuate a type of mathematical instruction that should have disappeared by now. I can’t even imagine what the history teachers say about your content-driven delivery, but I will stick to what I know. (You can read one critique here)

Just over a year ago I wrote a balanced review of some of the Khan Academy videos about statistics. I know that statistics is difficult to explain – in fact one of the hardest subjects to teach. You can read my review here. I’ve also reviewed a selection of videos about confidence intervals, one of which was from Khan Academy. You can read the review here.

Consequently I am aware that blogging about the Khan Academy in anything other than glowing terms is an invitation for vitriol from your followers.

However, I thought it was about time I looked at the exercises that are available on KA, wondering if I should recommend them to high school teachers for their students to use for review. I decided to focus on one section, introduction to probability. I put myself in the place of a person who was struggling to understand probability at school.

Here is the verdict.

First of all the site is very nice. It shows that it has a good sized budget to use on graphics and site mechanics. It is friendly to get into. I was a bit confused that the first section in the Probability and Statistics Section is called “Independent and dependent events”. It was the first section though. The first section of this first section is called Basic Probability, so I felt I was in the right place. But then under the heading, Basic probability, it says, “Can I pick a red frog out of a bag that only contains marbles?” Now I have no trouble with humour per se, and some people find my videos pretty funny. But I am very careful to avoid confusing people with the humour. For an anxious student who is looking for help, that is a bit confusing.

I was excited to see that this section had five videos, and two sets of exercises. I was pleased about that, as I’ve wanted to try out some exercises for some time, particularly after reading the review from Fawn Nguyen on her experience with exercises on Khan Academy. (I suggest you read this – it’s pretty funny.)

So I watched the first video about probability and it was like any other KA video I’ve viewed, with primitive graphics and a stumbling repetitive narration. It was correct enough, but did not take into account any of the more recent work on understanding probability. It used coins and dice. Big yawn. It wastes a lot of time. It was ok. I do like that you have the interactive transcript so you can find your way around.

It dawned on me that nowhere do you actually talk about what probability is. You seem to assume that the students already know that. In the very start of the first video it says,

“What I want to do in this video is give you at least a basic overview of probability. Probability, a word that you’ve probably heard a lot of and you are probably just a little bit familiar with it. Hopefully this will get you a little deeper understanding.”

Later in the video there is a section on the idea of large numbers of repetitions, which is one way of understanding probability. But it really is a bit skimpy on why anyone would want to find or estimate a probability, and what the values actually mean. But it was ok.

The first video was about single instances – one toss of a coin or one roll of a die. Then the second video showed you how to answer the questions in the exercises, which involved two dice. This seemed ok, if rather a sudden jump from the first video. Sadly both of these examples perpetuate the common misconception that if there are, say, 6 alternative outcomes, they will necessarily be equally likely.


Then we get to some exercises called “Probability Space” , which is not an enormously helpful heading. But my main quest was to have a go at the exercises, so that is what I did. And that was not a good thing. The exercises were not stepped, but started right away with an example involving two dice and the phrase “at least one of”. There was meant to be a graphic to help me, but instead I had the message “scratchpad not available”. I will summarise my concerns about the exercises at the end of my letter. I clicked on a link to a video that wasn’t listed on the left, called Probability Space and got a different kind of video.

This video was better in that it had moving pictures and a script. But I have problems with gambling in videos like this. There are some cultures in which gambling is not acceptable. The other problem I have is with the term  “exact probability”, which was used several times. What do we mean by “exact probability”? How does he know it is exact? I think this sends the wrong message.

Then on to the next videos which were worked examples, entitled “Example: marbles from a bag, Example: Picking a non-blue marble, Example: Picking a yellow marble.” Now I understand that you don’t want to scare students with terminology too early, but I would have thought it helpful to call the second one, “complementary events, picking a non-blue marble”. That way if a student were having problems with complementary events in exercises from school, they could find their way here. But then I’m not sure who your audience is. Are you sure who your audience is?

The first marble video was ok, though the terminology was sloppy.

The second marble video, called “Example: picking a non-blue marble”, is glacially slow. There is a point, I guess in showing students how to draw a bag and marbles, but… Then the next example is of picking numbers at random. Why would we ever want to do this? Then we come to an example of circular targets. This involves some problem-solving regarding areas of circles, and cancelling out fractions including pi. What is this about? We are trying to teach about probablity so why have you brought in some complication involving the area of a circle?

The third marble video attempts to introduce the idea of events, but doesn’t really. By trying not to confuse with technical terms, the explanation is more confusing.

Now onto some more exercises. The Khan model is that you have to get 5 correct in a row in order to complete an exercise. I hope there is some sensible explanation for this, because it sure would drive me crazy to have to do that. (As I heard expressed on Twitter)

What are circular targets doing in with basic probability?

The first example is a circular target one.  I SO could not be bothered working out the area stuff so I used the hints to find the answer so I could move onto a more interesting example. The next example was finding the probability of a rolling a 4 from a fair six sided die. This is trivial, but would have been not a bad example to start with. Next question involve three colours of marbles, and finding the probability of not green. Then another dart-board one. Sigh. Then another dart board one. I’m never going to find out what happens if I get five right in a row if I don’t start doing these properly. Oh now – it gave me circumference. SO can’t be bothered.

And that was the end of Basic probability. I never did find out what happens if I get five correct in a row.

Venn diagrams

The next topic is called “Venn diagrams and adding probabilities “. I couldn’t resist seeing what you would do with a Venn diagram. This one nearly reduced me to tears.

As you know by now, I have an issue with gambling, so it will come as no surprise that I object to the use of playing cards in this example. It makes the assumption that students know about playing cards. You do take one and a half minutes to explain the contents of a standard pack of cards.  Maybe this is part of the curriculum, and if so, fair enough. The examples are standard – the probability of getting a Jack of Hearts etc. But then at 5:30 you start using Venn diagrams. I like Venn diagrams, but they are NOT good for what you are teaching at this level, and you actually did it wrong. I’ve put a comment in the feedback section, but don’t have great hopes that anything will change. Someone else pointed this out in the feedback two years ago, so no – it isn’t going to change.

Khan Venn diagram

This diagram is misleading, as is shown by the confusion expressed in the questions from viewers. There should be a green 3, a red 12, and a yellow 1.

Now Venn diagrams seem like a good approach in this instance, but decades of experience in teaching and communicating complex probabilities has shown that in most instances a two-way table is more helpful. The table for the Jack of Hearts problem would look like this:

Jacks Not Jacks Total
Hearts 1 12 13
Not Hearts 3 36 39
Total 4 48 52

(Any teachers reading this letter – try it! Tables are SO much easier for problem solving than Venn diagrams)

But let’s get down to principles.

The principles of instruction that KA have not followed in the examples:

  • Start easy and work up
  • Be interesting in your examples – who gives a flying fig about two dice or random numbers?
  • Make sure the hardest part of the question is the thing you are testing. This is particularly violated with the questions involving areas of circles.
  • Don’t make me so bored that I can’t face trying to get five in a row and not succeed.

My point

Yes, I do have one. Mr Khan you clearly can’t be stopped, so can you please get some real teachers with pedagogical content knowledge to go over your materials systematically and make them correct. You have some money now, and you owe it to your benefactors to GET IT RIGHT. Being flippant and amateurish is fine for amateurs but you are now a professional, and you need to be providing material that is professionally produced. I don’t care about the production values – keep the stammers and “lellows” in there if you insist. I’m very happy you don’t have background music as I can’t stand it myself. BUT… PLEASE… get some help and make your videos and exercises correct and pedagogically sound.

Dr Nic

PS – anyone else reading this letter, take a look at the following videos for mathematics.

And of course I think my own Statistics Learning Centre videos are pretty darn good as well.

Other posts about concerns about Khan:

Another Open Letter to Sal ( I particularly like the comment by Michael Paul Goldenberg)

Breaking the cycle (A comprehensive summary of the responses to criticism of Khan

Make journalists learn statistics

All journalists should be required to pass a course in basic statistics before they are let loose on the unsuspecting public.

I am not talking about the kind of statistics course that mathematical statisticians are talking about. This does not involve calculus, R or anything tricky requiring a post-graduate degree. I am talking about a statistics course for citizens. And journalists. 🙂

I have thought about this for some years. My father was a journalist, and fairly innumerate unless there was a dollar sign involved. But he was of the old school, who worked their way up the ranks. These days most media people have degrees, and I am adamant that the degree should contain basic numeracy and statistics. The course I devised (which has now been taken over by the maths and stats department and will be shut down later this year, but am I bitter…?) would have been ideal. It included basic number skills, including percentages (which are harder than you think), graphing, data, chance and evidence. It required students to understand the principles behind what they were doing rather than the mechanics.

Here is what journalists should know about statistics:


One of the key concepts in statistics is that of variability and chance.  Too often a chance event is invested with unnecessary meaning. A really good example of this is the road toll. In New Zealand the road toll over the Easter break can fluctuate between 21 (in 1971) and 3 in 1998, 2002 and 2003. Then in 2012 the toll was zero, a cause of great celebration. I was happy to see one report say “There was no one reason for the zero toll this Easter, and good fortune may have played a part.” However this was a refreshing change as normally the police seem to take the credit for good news, and blame bad news on us. Rather like Economists.

With any random process you will get variability. The human mind looks for patterns and meanings even where there are none. Sadly the human mind often finds patterns and imbues meaning erroneously. Astrology is a perfect example of this – and watching Deal or No Deal is inspiring in the meaning people can find in random variation.

All journalists should have a good grasp of the concepts of variability so they stop drawing unfounded conclusions

Data Display

There are myriad examples of graphs in the media that are misleading, badly constructed, incorrectly specified, or just plain wrong. There was a wonderful one in the Herald Sun recently, which has had considerable publicity. We hope it was just an error, and nothing more sinister. But good subediting (what my father used to do, but I think ceased with the advent of the computer) would have picked this up.

There is a very nice website dedicated to this: StatsChat.   It unfortunately misquotes H.G.Wells, but has a wonderful array of examples of good and bad statistics in the media. This post gives links to all sorts of sites with bad graphs, many of which were either produced or promulgated by journalists. But not all – scientific literature also has its culprits.

Just a little aside here – why does NO-ONE ever report the standard deviation? I was writing questions involving the normal distribution for practice by students. I am a strong follower of Cobb’s view that all data should be real, so I went looking for some interesting results I could use, with a mean and standard deviation. Heck I couldn’t even find uninteresting results! The mean and the median rule supreme, and confidence intervals are getting a little look in. Percentages are often reported with a “margin of error” (does anyone understand that?). But the standard deviation is invisible. I don’t think the standard deviation is any harder to understand than the mean. (Mainly because the mean is very hard to understand!) So why is the standard deviation not mentioned?


One of the main ideas in inferential statistics is that of evidence: The data is here; do we have evidence that this is an actual effect rather than caused by random variation and sampling error? In traditional statistics this is about understanding the p-value. In resampling the idea is very similar to that of a p-value – we ask “could we have got this result by chance?” You do not have to be a mathematician to grasp this idea if it is presented in an accessible way. (See my video “Understanding the p-value” for an example.)

One very exciting addition to the New Zealand curriculum are Achievement Standards at Years 12 and 13 involving reading and understanding statistical reports. I have great hopes that as teachers embrace these standards, the level of understanding in the general population will increase, and there will be less tolerance for statistically unsound conclusions.

Another source of hope for me is “The Panel”, an afternoon radio programme hosted by Jim Mora on Radio New Zealand National. Each day different guests are invited to comment on current events in a moderately erudite and often amusing way. Sometimes they even have knowledge about the topic, and usually an expert is interviewed. It is as talkback radio really could be. I think. I’ve never listened long enough to talk-back radio to really judge as it always makes me SO ANGRY! Breathe, breathe…

I digress. I have been gratified to hear people on The Panel making worthwhile comments about sample size, sampling method, bias, association and causation. (Not usually using those exact terms, but the concepts are there.) It gives me hope that critical response to pseudo-scientific, and even scientific research is possible in the general populace. My husband thinks that should be “informed populace”, but I can dream.

It is possible for journalists to understand the important ideas of statistics without a mathematically-based and alienating course. I feel an app coming on… (Or should that be a nap?)

Hey mathematics – leave the stats alone!

Mathematicians love the elegance of mathematics

Mathematicians love mathematics. They love the elegance and the purity and the abstract nature of it all. Consequently they think there is something not quite nice about the practical real life messiness of statistics. Now this is fine, so long as they keep their prejudices away from their students! I recently met a high school maths teacher who was completely vocal about her dislike for statistics. Fortunately she doesn’t teach the final year statistics course, but she can’t avoid the sections of statistics all through the curriculum at lower levels. It hurt me to hear statistics so disliked.

Elementary school-teachers who dislike mathematics harm the good attitude formation in their pupils. They don’t like maths, and they feel uneasy doing it, and that rubs off. High school teachers are often frustrated by the attitudes with which students arrive at high school. There are moves in New Zealand to address this, through the Numeracy Project, which helps to develop skills in our Primary teachers.

What bothers me is similar. Many, if not most, of our high school teachers are pure mathematicians. Some of them allow their dislike for statistics to colour the students’ experience. Or if they don’t actively dislike statistics, they may still feel ill-at-ease, as they did not get enough background knowledge in their training. They may know the mechanisms, but have no experience of statistical analysis. I know this to be true, as I was once one of them. It is difficult to go from an exact subject like mathematics, where you find x and know when you have found it, to an art/science like statistics, where x changes depending on the context.

However I am now a born-again statistics applier. I hesitate to call myself a statistician, as I don’t use R, and I’m not exactly sure what a moment is. But I know how to do statistics in the real world. I know what you should and shouldn’t do with different data, and I know how important context is. I know that you seldom get a simple random sample, and sometimes your sample is so far from random that you blush, but soldier on anyway. I’m skeptical about Factor Analysis. And I keep learning. Every time I do a real statistical analysis I gain insights into the nature of the discipline. And I love it. Statistics is a detective game. The numbers tell a story, and it is up to us to help them reveal their secrets without so much coercion that they tell us lies to make us go away.

My wish is that pure mathematicians in high schools would accept that statistics is not mathematics and never will be. It is a mathematical science, and needs to be taught differently from mathematics.

George W. Cobb and David S. Moore wrote a paper, “Mathematics, Statistics, and Teaching”, which gives answers to questions such as “how does statistical thinking differ from mathematical thinking? and “What is the role of mathematics in statistics?”. They emphasize that beginning statistics should be taught as statistics. A beginning statistics course should use real data and automated production of graphs and analysis.

Statistics lives in the real world

This is antithetical to a pure mathematician. “Remove the maths and the graphing – or get the computer to do it, and where is the maths?”, they cry! “Exactly!”, reply the statistics teachers.

I hope there are maths (or math) teachers reading this. You can do it – you just need to accept that statistics is NOT mathematics, and learn to see the rigour and excitement in it. Embrace the messiness! Throw off the shackles of finding the one correct answer! Statistics, well-taught, will be more use to most of your students than calculus.

Operations Research and Statistics: BFF

As they say on Twitter: That silence after you tell someone you teach Operations Research (or Statistics).

Those in the OR and Statistics communities know what conversation stoppers our disciplines are. When asked what subject I teach I take a punt and respond with “Operations Research”, “Management Science” or “Statistics”. “Operations Research” is met with incomprehension, “Management Science” with miscomprehension, and “Statistics” with thinly disguised antipathy. Apart from being undervalued, what the disciplines have in common is that we do practical stuff with numbers. The pedagogies of these disciplines have much in common.

Operations Research and Management Science (which for many people are synonymous) use statistics and other mathematical analysis techniques to solve real world problems.

Operations Research/Management Science is a discipline which seeks to improve a problem situation by supplying decision makers with information and insights gained through problem analysis, often involving mathematical modelling. (Nicola Ward Petty)

A knowledge of probability and statistics forms part of the OR/MS toolkit, along with linear and non-linear programming, decision analysis, queueing, simulation, heuristics, multicriteria decision-making and operations management tools such as Critical path and inventory control. OR/MS is more than just a set of tools, however, and includes a philosophy of improvement through modelling (as stated in the definition).

Statistics focusses on extracting information from data, and provides the backbone to research in just about all human endeavours, including physics, astronomy, medicine, business, education, psychology, sport and agriculture. Statistical analysis is an essential part of the scientific method. It is often used to inform decision-making, as is OR/MS

I think it is fair to categorise both Statistics and OR/MS as decision sciences, and mathematical sciences. It is also fair to say that the average person in the street has little comprehension of either of them.

So how does this affect teachers of these disciplines? There has been considerable research into the teaching of statistics, and much less into the teaching of operations research (probably because of the number of students taking each of the subjects.) Volumes such as “The challenge of developing statistical literacy, reasoning and thinking”(2004), and “Developing students’ statistical reasoning: connecting research and teaching practice”, (2008) edited by Dani Ben-Zvi and Joan Garfield provide inspiration and guidance to statistics teachers and educational researchers.

The statistics education research literature accepts as given that there are challenges in teaching quantitative courses. Ben-Zvi & Garfield (2004) state four main challenges to success in teaching and learning statistics, which must resonate with many OR instructors. These can be paraphrased as: It can be hard to motivate students to do hard work. Many students have difficulty with the underlying mathematics, and that interferes with learning the related content. The context can mislead students who rely on experience and intuition, and students expect the focus to be on numbers, computations, formulas and one right answer. This can be summarised as

• motivation to work
• mathematical
• contexts
• inexact

Motivation to work

We accept that Statistics and OR/MS are not as inherently motivating to the majority of our students as we would like. Part of our brief is to help them understand how important the subject can be, which can be done through the use of real world examples, and preferably real-world data. What is also motivating to most humans is learning for its own sake. If students feel the joy of passing from incomprehension to comprehension to mastery, this is deeply motivating. Experiences which lead to successful learning aid student motivation.


It’s true. We use mathematics. But we are not mathematics. And when we can get the same result by avoiding the mathematics, all power to us! No one should calculate standard deviations or solve linear programs by hand any more. The ubiquitous spreadsheet has removed that necessity in all but trivial and explanatory examples. It is increasingly possible to do plausible analysis relying totally on computer packages. This way we can give students non-trivial exercises using real data. A little aside here – I personally found mathematics unappealing when there ceased to be numbers in it other than as subscripts, and promptly switched majors to Operations Research, where I my love of numbers and practical problem-solving was indulged.


One of my heroes, George Cobb, points out that statistics does not exist without the context. I would suggest the same is true of OR/MS. Remove the application area and we are back in mathematics or math programming. Context can be given in a mathematical example as an unnecessary little story to give pseudo-reality to problems that are inherently abstract. Now there is nothing wrong with abstract – it’s just that statistics and OR/MS aren’t abstract. All problems in statistics and OR/MS should have a context which is relevant and forms part of the answer to the question. Questions like “Find the expected value for the following (context-free) discrete distribution” are to be avoided. Why would we want to know the expected value? What do we do with it when we have got it? Statistics and OR need to answer questions. However, the context can also become a stumbling block when students construct incorrect knowledge based on generalisations of contexts, or allow their own intuition to over-ride what the analysis is telling them.


What I used to love about mathematics as a child was getting lots of red ticks (checkmarks not irritating insects) down beside my work. My son once gave me a handmade birthday card covered in red ticks as he knew how much I liked them. In mathematics there was one correct answer. (See my post on Re:Solutions). You had to find x, and when you found it you knew you had the right one. This is SO not true in statistics and Operations Research. Everything “depends”. I now embrace the ambiguity, whereas it felt distinctly uncomfortable at first.

By articulating these four challenges we are better equipped to face them. Let’s try to make our subjects motivating and doable, mathematically appropriate to the audience and with interesting contexts that embrace ambiguity. In this way we can better teach the Science of Better and the Science of Data.

Teaching statistical language

I received a phone call from the company that leases us our equipment. I got quite excited when the salesman told me they would waive the purchase price of a new iPad. Then I decided it was time to clarify things. “Ok,” I said, “You are using the term ‘purchase price’. To me that means the amount you pay for something when you buy it. You are telling me that if I get a new iPad on the same lease as the old iPad you will waive the purchase price. This sounds great to me, but I can’t imagine I’ve got it right.”

I didn’t have it right.


He was using the term “purchase price” to describe an extra payment at the end of the two-year term to allow me to “purchase” the two-year-old iPad. Darn.

Similar confusion arises when common terms are used in specialized ways within a discipline. Statistics and Operations Research have plenty of confusing specialised language. A Google search on “confusing statistical terms” uncovers a goldmine.

“Significant”, “Random, “Regression” and “Normal” have common meanings quite distinct from their technical meaning. “Linear Programming”, though not a term in common use, implies programming, probably in a linear fashion, which does nothing to aid comprehension. There are problems with the term “problem” and even greater problems with the word “solution.” In Operations Research a solution is nothing like an everyday solution. It doesn’t even have to be possible!

Sometimes, like my friendly salesman, we can forget how confusing these terms are. And not only are the terms confusing, but they are sticky. The students have lived their lives with one vague meaning for random, so it will take many attempts to internalize a different meaning.

In writing this post I found numerous references to this problem, and explanations of tricky terms. Here are some links:

Solutions (Hopefully feasible, but unlikely to be optimal)

Here are some thoughts on how to address the problem when teaching.

Be aware

A comprehension problem could stem from misunderstanding of language, like my communication problem with the loan clerk.

Be explicit

Say to students – “Whatever meaning you have for ‘random’, you need to put it to one side when we are talking about random in this discipline.” Get them to explain what it means to them, and explain what it means in the discipline. Find similarities and differences.

Use a modifier if it makes sense

When talking about significance in statistics, I try to call it “statistical significance”, which is a reminder that it means something different from the everyday (and newspaper) use of significant.


State the meaning alongside the term often as you can – until they are so sick of it they recite with you. For example, “This statistically significant result, meaning we have evidence that the result in the sample exists in the population, indicates that men and women do differ in their chocolate eating habits.”

Assess for it

Students learn best what is tested. Have questions about statistical language that separate out the meaning of the term from the statistical concept. A student could say that a sample contains probable bias, by being confused about the term and the application of the phenomenon and the two errors cancel out, giving them a correct answer.

Provide examples

Give examples of the terms used in the “everyday” way and in the specialized way for them to sort.

Student participation

  • Give students opportunities to use the terms in their speaking and writing.
  • Include comprehension activities as part of the class. (“But Miss, this is a Maths lesson, not an English lesson!” “Actually, Angus, this is a Statistics lesson, not a Maths lesson.”)
  • Students to identify whenever there is a conflict between everyday use and statistical use – make their own list like the one below.

Confusing terms in Statistics and Operations Research

This is not comprehensive. Get the students individually or as a class to come up with their own list, possibly humorous! (Nice list at Stats with cats)

  • Significant
  • Random
  • Normal
  • Regression
  • Representative
  • Reliable
  • Average
  • Error
  • Bias
  • Residual
  • Outlier
  • Power
  • Interaction
  • Confidence
  • Risk
  • Solution
  • Uncertainty
  • Linear programming
  • Operations Research
  • Optimal
  • Heuristic
  • And not quite in the everyday category, but annoying nonetheless : ANOVA (why is it called analysis of variance when its purpose is comparing means?)

Compared to the challenge of helping students comprehend inference, the language issue is a small one. But unless it is dealt with, it can be a barrier to the complex ideas.

And I did get the iPad. For business use of course!