Videos for teaching and learning statistics

It delights me that several of my statistics videos have been viewed over half a million times each. As well there is a stream of lovely comments (with the odd weird one) from happy viewers, who have found in the videos an answer to their problems.

In this post I will outline the main videos available on the Statistics Learning Centre YouTube Channel. They already belong to 24,000 playlists and lists of recommended resources in textbooks the world over. We are happy for teachers and learners to continue to link to them. Having them all in one place should make it easier for instructors to decide which ones to use in their courses.

Philosophy of the videos

Early on in my video production I wrote a series of blog posts about the videos. One was Effective multimedia teaching videos. The videos use graphics and audio to increase understanding and retention, and are mostly aimed at conceptual understanding rather than procedural understanding.

I also wrote a critique of Khan Academy videos, explaining why I felt they should be improved. Not surprisingly this ruffled a few feathers and remains my most commented on post. I would be thrilled if Khan had lifted his game, but I fear this is not the case. The Khan Academy pie chart video still uses an unacceptable example with too many and ordered categories. (January 2018)

Before setting out to make videos about confidence intervals, I critiqued the existing offerings in this post. At the time the videos were all about how to find a confidence interval, and not what it does. I suspect that may be why my video, Understanding Confidence Intervals, remains popular.

Note to instructors

You are welcome to link to our YouTube channel, and we get a tiny amount of money from people clicking on the ads. Please do NOT download the videos, as it is against YouTube rules, and deprives us of income. Note that we also have a separate pay-to-view channel, with considerably more videos, at higher resolution, with no advertising. Email us at for free trial access to these videos, with a view to providing them for your students on a subscription basis. If you have trouble with reliable internet access, we can also provide the videos as files for your network as part of the licence.

Introducing statistics

Understanding Summary Statistics 5:14 minutes

Why we need summary statistics and what each of them does. It is not about how to calculate the statistics, but what they mean. It uses the shoe example, which also appears in the PPDAC and OSEM videos.

Understanding Graphs 6:06 minutes

I briefly explains the use and interpretation of seven different types of statistical graph. They include the pictogram, bar chart, pie chart, dot plot, stem and leaf, scatterplot and time series.


Analysing and commenting on Graphical output using OSEM 7:13 minutes

This video teaches how to comment on graphs and other statistical output by using the acronym OSEM. It is especially useful for students in NCEA statistics classes in New Zealand, but many people everywhere can find OSEM awesome! We use the example of comparing the number of pairs of shoes men and women students say they own.

Variation and Sampling error 6:30 minutes

Statistical methods are necessary because of the existence of variation. Sampling error is one source of variation, and is often misunderstood. This video explains sampling error, along with natural variation, explainable variation and variation due to bias. There is an accompanying video on non-sampling error.

Sampling methods 4:54 minutes 500,000 views

This video describes five common methods of sampling in data collection – simple random, convenience, systematic, cluster and stratified. Each method has a helpful symbolic representation.

Types of data 6:20 minutes 600,000 views

The kind of graph and analysis we can do with specific data is related to the type of data it is. In this video we explain the different levels of data, with examples. This video is particularly popular at the start of courses.

Important Statistical concepts 5:34 minutes 50,000 views

This video does not receive the views it deserves, as it covers three really important ideas. Maybe I should split it up into three videos. The ideas are the difference between significance and usefulness, evidence and strength of effect, causation and association.

Other videos complementary to these, but not on YouTube are:

  • The statistical enquiry process
  • Understanding the Box Plot
  • Non-sampling error

Videos for teaching hypothesis testing

Understanding Statistical inference 6:46 minutes 40,000 views

The most difficult concept in statistics is that of inference. This video explains what statistical inference is and gives memorable examples. It is based on research around three concepts pivotal to inference – that the sample is likely to be a good representation of the population, that there is an element of uncertainty as to how well the sample represents the population, and that the way the sample is taken matters.

Understanding the p-value 4:43 minutes 500,000 views

This video explains how to use the p-value to draw conclusions from statistical output. It includes the story of Helen, making sure that the choconutties she sells have sufficient peanuts. It introduces the helpful phrase “p is low, null must go”.

Inference and evidence 3:34 minutes

This is a newer video, based on a little example I used in lectures to help students see the link between evidence and inference. Of course it involves chocolate.

Hypothesis tests 7:38 minutes 350,000 views

This entertaining video works step-by-step through a hypothesis test. Helen wishes to know whether giving away free stickers will increase her chocolate sales. This video develops the ideas from “Understanding the p-value”, giving more of the process of hypothesis testing. It is also complemented by the following video, that shows how to perform the analysis using Excel.

Two-means t-test in Excel 3:54 minutes 50,000 views

A step-by-step lesson on how to perform an independent samples t-test for difference of two means using the Data Analysis ToolPak in Excel. This is a companion video to Hypothesis tests, p-value, two means t-test.

Choosing which statistical test to use 9:33 minutes 500,000 views

I am particularly proud of this video, and the way it links the different tests together. It took a lot of work to come up with this. First it outlines a process for thinking about the data, the sample and the thing you are trying to find out. Then it works through seven tests with scenarios based around Helen and the Choconutties. This video is particularly popular near the end of the semester, for tying together the different tests and applications.


Confidence Intervals

Understanding Confidence Intervals 4:02 minutes 500,000 views

This short video gives an explanation of the concept of confidence intervals, with helpful diagrams and examples. The emphasis is on what a confidence interval is and how it is used, rather than how they are calculated or derived.

Calculating the confidence interval for a mean using a formula 5:29 minutes 200,000 views

This video carries on from “Understanding Confidence Intervals” and introduces a formula for calculating a confidence interval for a mean. It uses graphics and animation to help understanding.

There are also videos pertinent to the New Zealand curriculum using bootstrapping and informal methods to find confidence intervals.


Introduction to Probability 2:54 minutes

This video explains what probability is and why we use it. It does NOT use dice, coins or balls in urns. It is the first in a series of six videos introducing basic probability with a conceptual approach. The other five videos can be accessed through subscription.

Understanding Random Variables 5:08 minutes 90,000 views

The idea of a random variable can be surprisingly difficult. In this video we help you learn what a random variable is, and the difference between discrete and continuous random variables. It uses the example of Luke and his ice cream stand.

Understanding the Normal Distribution 7:44 minutes

In this video we explain the characteristics of the normal distribution, and why it is so useful as a model for real-life entities.

There are also two other videos about random variables, discrete and continuous.

Risk and Screening 7:54 minutes

This video explains about risk and screening, and shows how to calculate and express rates of false positives and false negatives. An imaginary disease, “Earpox” is used for the examples.

Other videos

Designing a Questionnaire 5:23 minutes 40,000 views

This was written specifically to support learning in Level 1 NCEA in the NZ school system but is relevant for anyone needing to design a questionnaire. There is a companion video on good and bad questions.

Line-fitting and regression

Scatterplots in Excel 5:17 minutes

The first step in doing a regression in Excel is to fit the line using a Scatter plot. This video shows how to do this, illustrated by the story of Helen and the effect of temperature on her sales of choconutties

Regression in Excel 6:27 minutes

This video explains Regression and how to perform regression in Excel and interpret the output. The story of Helen and her choconutties continues. This follows on from Scatterplots in Excel and Understanding the p-value.

There are three videos introducing bivariate relationships in a more conceptual way.

There are also videos covering experimental design and randomisation, time series analysis and networks. In the pipeline is a video “understanding the Central Limit Theorem.”

Supporting our endeavours

As explained in a previous post, Lessons for a budding Social Enterprise, Statistics Learning Centre is a social enterprise, with our aim to build a world of mathematicians and enable people to make intelligent use of statistics. Though we get some income from YouTube videos, it does not support the development of more videos. If you would like to help us to create further videos contact us to discuss subscriptions, sponsorship, donations and advertising possibilities. or


Understanding Statistical Inference

Inference is THE big idea of statistics. This is where people come unstuck. Most people can accept the use of summary descriptive statistics and graphs. They can understand why data is needed. They can see that the way a sample is taken may affect how things turn out. They often understand the need for control groups. Most statistical concepts or ideas are readily explainable. But inference is a tricky, tricky idea. Well actually – it doesn’t need to be tricky, but the way it is generally taught makes it tricky.

Procedural competence with zero understanding

I cast my mind back to my first encounter with confidence intervals and hypothesis tests. I learned how to calculate them (by hand  – yes I am that old) but had not a clue what their point was. Not a single clue. I got an A in that course. This is a common occurrence. It is possible to remain blissfully unaware of what inference is all about, while answering procedural questions in exams correctly.

But, thanks to the research and thinking of a lot of really smart and dedicated statistics teachers, we are able put a stop to that. And we must. Help us make great resourcces

We need to explicitly teach what statistical inference is. Students do not learn to understand inference by doing calculations. We need to revisit the ideas behind inference frequently. The process of hypothesis testing, is counter-intuitive and so confusing that it spills its confusion over into the concept of inference. Confidence intervals are less confusing so a better intermediate point for understanding statistical inference. But we need to start with the concept of inference.

What is statistical inference?

The idea of inference is actually not that tricky if you unbundle the concept from the application or process.

The concept of statistical inference is this –

We want to know stuff about a large group of people or things (a population). We can’t ask or test them all so we take a sample. We use what we find out from the sample to draw conclusions about the population.

That is it. Now was that so hard?

Developing understanding of statistical inference in children

I have found the paper by Makar and Rubin, presenting a “framework for thinking about informal statistical inference”, particularly helpful. In this paper they summarise studies done with children learning about inference. They suggest that “ three key principles … appeared to be essential to informal statistical inference: (1) generalization, including predictions, parameter estimates, and conclusions, that extend beyond describing the given data; (2) the use of data as evidence for those generalizations; and (3) employment of probabilistic language in describing the generalization, including informal reference to levels of certainty about the conclusions drawn.” This can be summed up as Generalisation, Data as evidence, and Probabilistic Language.

We can lead into informal inference early on in the school curriculum. The key Ideas in the NZ curriculum suggest that “ teachers should be encouraging students to read beyond the data. Eg ‘If a new student joined our class, how many children do you think would be in their family?’” In other words, though we don’t specifically use the terms population and sample, we can conversationally draw attention to what we learn from this set of data, and how that might relate to other sets of data.

Explaining directly to Adults

When teaching adults we may use a more direct approach, explaining explicitly, alongside experiential learning to understanding inference. We have just completed made a video: Understanding Inference. Within the video we have presented three basic ideas condensed from the Five Big Ideas in the very helpful book published by NCTM, “Developing Essential Understanding of Statistics, Grades 9 -12”  by Peck, Gould and Miller and Zbiek.

Ideas underlying inference

  • A sample is likely to be a good representation of the population.
  • There is an element of uncertainty as to how well the sample represents the population
  • The way the sample is taken matters.

These ideas help to provide a rationale for thinking about inference, and allow students to justify what has often been assumed or taught mathematically. In addition several memorable examples involving apples, chocolate bars and opinion polls are provided. This is available for free use on YouTube. If you wish to have access to more of our videos than are available there, do email me at


Sampling error and non-sampling error

The subject of statistics is rife with misleading terms. I have written about this before in such posts as Teaching Statistical Language and It is so random. But the terms sampling error and non-sampling error win the Dr Nic prize for counter-intuitivity and confusion generation.

Confusion abounds

To start with, the word error implies that a mistake has been made, so the term sampling error makes it sound as if we made a mistake while sampling. Well this is wrong. And the term non-sampling error (why is this even a term?) sounds as if it is the error we make from not sampling. And that is wrong too. However these terms are used extensively in the NZ statistics curriculum, so it is important that we clarify what they are about.

Fortunately the Glossary has some excellent explanations:

Sampling Error

“Sampling error is the error that arises in a data collection process as a result of taking a sample from a population rather than using the whole population.

Sampling error is one of two reasons for the difference between an estimate of a population parameter and the true, but unknown, value of the population parameter. The other reason is non-sampling error. Even if a sampling process has no non-sampling errors then estimates from different random samples (of the same size) will vary from sample to sample, and each estimate is likely to be different from the true value of the population parameter.

The sampling error for a given sample is unknown but when the sampling is random, for some estimates (for example, sample mean, sample proportion) theoretical methods may be used to measure the extent of the variation caused by sampling error.”

Non-sampling error:

“Non-sampling error is the error that arises in a data collection process as a result of factors other than taking a sample.

Non-sampling errors have the potential to cause bias in polls, surveys or samples.

There are many different types of non-sampling errors and the names used to describe them are not consistent. Examples of non-sampling errors are generally more useful than using names to describe them.

And it proceeds to give some helpful examples.

These are great definitions, and I thought about turning them into a diagram, so here it is:

Table summarising types of error.

Table summarising types of error.

And there are now two videos to go with the diagram, to help explain sampling error and non-sampling error. Here is a link to the first:

Video about sampling error

 One of my earliest posts, Sampling Error Isn’t, introduced the idea of using variation due to sampling and other variation as a way to make sense of these ideas. The sampling video above is based on this approach.

Students need lots of practice identifying potential sources of error in their own work, and in critiquing reports. In addition I have found True/False questions surprisingly effective in practising the correct use of the terms. Whatever engages the students for a time in consciously deciding which term to use, is helpful in getting them to understand and be aware of the concept. Then the odd terminology will cease to have its original confusing connotations.

Open Letter to Khan Academy about Basic Probability

Khan academy probability videos and exercises aren’t good either

Dear Mr Khan

You have created an amazing resource that thousands of people all over the world get a lot of help from. Well done. Some of your materials are not very good, though, so I am writing this open letter in the hope that it might make some difference. Like many others, I believe that something as popular as Khan Academy will benefit from constructive criticism.

I fear that the reason that so many people like your mathematics videos so much is not because the videos are good, but because their experience in the classroom is so bad, and the curriculum is poorly thought out and encourages mechanistic thinking. This opinion is borne out by comments I have read from parents and other bloggers. The parents love you because you help their children pass tests.  (And these tests are clearly testing the type of material you are helping them to pass!) The bloggers are not so happy, because you perpetuate a type of mathematical instruction that should have disappeared by now. I can’t even imagine what the history teachers say about your content-driven delivery, but I will stick to what I know. (You can read one critique here)

Just over a year ago I wrote a balanced review of some of the Khan Academy videos about statistics. I know that statistics is difficult to explain – in fact one of the hardest subjects to teach. You can read my review here. I’ve also reviewed a selection of videos about confidence intervals, one of which was from Khan Academy. You can read the review here.

Consequently I am aware that blogging about the Khan Academy in anything other than glowing terms is an invitation for vitriol from your followers.

However, I thought it was about time I looked at the exercises that are available on KA, wondering if I should recommend them to high school teachers for their students to use for review. I decided to focus on one section, introduction to probability. I put myself in the place of a person who was struggling to understand probability at school.

Here is the verdict.

First of all the site is very nice. It shows that it has a good sized budget to use on graphics and site mechanics. It is friendly to get into. I was a bit confused that the first section in the Probability and Statistics Section is called “Independent and dependent events”. It was the first section though. The first section of this first section is called Basic Probability, so I felt I was in the right place. But then under the heading, Basic probability, it says, “Can I pick a red frog out of a bag that only contains marbles?” Now I have no trouble with humour per se, and some people find my videos pretty funny. But I am very careful to avoid confusing people with the humour. For an anxious student who is looking for help, that is a bit confusing.

I was excited to see that this section had five videos, and two sets of exercises. I was pleased about that, as I’ve wanted to try out some exercises for some time, particularly after reading the review from Fawn Nguyen on her experience with exercises on Khan Academy. (I suggest you read this – it’s pretty funny.)

So I watched the first video about probability and it was like any other KA video I’ve viewed, with primitive graphics and a stumbling repetitive narration. It was correct enough, but did not take into account any of the more recent work on understanding probability. It used coins and dice. Big yawn. It wastes a lot of time. It was ok. I do like that you have the interactive transcript so you can find your way around.

It dawned on me that nowhere do you actually talk about what probability is. You seem to assume that the students already know that. In the very start of the first video it says,

“What I want to do in this video is give you at least a basic overview of probability. Probability, a word that you’ve probably heard a lot of and you are probably just a little bit familiar with it. Hopefully this will get you a little deeper understanding.”

Later in the video there is a section on the idea of large numbers of repetitions, which is one way of understanding probability. But it really is a bit skimpy on why anyone would want to find or estimate a probability, and what the values actually mean. But it was ok.

The first video was about single instances – one toss of a coin or one roll of a die. Then the second video showed you how to answer the questions in the exercises, which involved two dice. This seemed ok, if rather a sudden jump from the first video. Sadly both of these examples perpetuate the common misconception that if there are, say, 6 alternative outcomes, they will necessarily be equally likely.


Then we get to some exercises called “Probability Space” , which is not an enormously helpful heading. But my main quest was to have a go at the exercises, so that is what I did. And that was not a good thing. The exercises were not stepped, but started right away with an example involving two dice and the phrase “at least one of”. There was meant to be a graphic to help me, but instead I had the message “scratchpad not available”. I will summarise my concerns about the exercises at the end of my letter. I clicked on a link to a video that wasn’t listed on the left, called Probability Space and got a different kind of video.

This video was better in that it had moving pictures and a script. But I have problems with gambling in videos like this. There are some cultures in which gambling is not acceptable. The other problem I have is with the term  “exact probability”, which was used several times. What do we mean by “exact probability”? How does he know it is exact? I think this sends the wrong message.

Then on to the next videos which were worked examples, entitled “Example: marbles from a bag, Example: Picking a non-blue marble, Example: Picking a yellow marble.” Now I understand that you don’t want to scare students with terminology too early, but I would have thought it helpful to call the second one, “complementary events, picking a non-blue marble”. That way if a student were having problems with complementary events in exercises from school, they could find their way here. But then I’m not sure who your audience is. Are you sure who your audience is?

The first marble video was ok, though the terminology was sloppy.

The second marble video, called “Example: picking a non-blue marble”, is glacially slow. There is a point, I guess in showing students how to draw a bag and marbles, but… Then the next example is of picking numbers at random. Why would we ever want to do this? Then we come to an example of circular targets. This involves some problem-solving regarding areas of circles, and cancelling out fractions including pi. What is this about? We are trying to teach about probablity so why have you brought in some complication involving the area of a circle?

The third marble video attempts to introduce the idea of events, but doesn’t really. By trying not to confuse with technical terms, the explanation is more confusing.

Now onto some more exercises. The Khan model is that you have to get 5 correct in a row in order to complete an exercise. I hope there is some sensible explanation for this, because it sure would drive me crazy to have to do that. (As I heard expressed on Twitter)

What are circular targets doing in with basic probability?

The first example is a circular target one.  I SO could not be bothered working out the area stuff so I used the hints to find the answer so I could move onto a more interesting example. The next example was finding the probability of a rolling a 4 from a fair six sided die. This is trivial, but would have been not a bad example to start with. Next question involve three colours of marbles, and finding the probability of not green. Then another dart-board one. Sigh. Then another dart board one. I’m never going to find out what happens if I get five right in a row if I don’t start doing these properly. Oh now – it gave me circumference. SO can’t be bothered.

And that was the end of Basic probability. I never did find out what happens if I get five correct in a row.

Venn diagrams

The next topic is called “Venn diagrams and adding probabilities “. I couldn’t resist seeing what you would do with a Venn diagram. This one nearly reduced me to tears.

As you know by now, I have an issue with gambling, so it will come as no surprise that I object to the use of playing cards in this example. It makes the assumption that students know about playing cards. You do take one and a half minutes to explain the contents of a standard pack of cards.  Maybe this is part of the curriculum, and if so, fair enough. The examples are standard – the probability of getting a Jack of Hearts etc. But then at 5:30 you start using Venn diagrams. I like Venn diagrams, but they are NOT good for what you are teaching at this level, and you actually did it wrong. I’ve put a comment in the feedback section, but don’t have great hopes that anything will change. Someone else pointed this out in the feedback two years ago, so no – it isn’t going to change.

Khan Venn diagram

This diagram is misleading, as is shown by the confusion expressed in the questions from viewers. There should be a green 3, a red 12, and a yellow 1.

Now Venn diagrams seem like a good approach in this instance, but decades of experience in teaching and communicating complex probabilities has shown that in most instances a two-way table is more helpful. The table for the Jack of Hearts problem would look like this:

Jacks Not Jacks Total
Hearts 1 12 13
Not Hearts 3 36 39
Total 4 48 52

(Any teachers reading this letter – try it! Tables are SO much easier for problem solving than Venn diagrams)

But let’s get down to principles.

The principles of instruction that KA have not followed in the examples:

  • Start easy and work up
  • Be interesting in your examples – who gives a flying fig about two dice or random numbers?
  • Make sure the hardest part of the question is the thing you are testing. This is particularly violated with the questions involving areas of circles.
  • Don’t make me so bored that I can’t face trying to get five in a row and not succeed.

My point

Yes, I do have one. Mr Khan you clearly can’t be stopped, so can you please get some real teachers with pedagogical content knowledge to go over your materials systematically and make them correct. You have some money now, and you owe it to your benefactors to GET IT RIGHT. Being flippant and amateurish is fine for amateurs but you are now a professional, and you need to be providing material that is professionally produced. I don’t care about the production values – keep the stammers and “lellows” in there if you insist. I’m very happy you don’t have background music as I can’t stand it myself. BUT… PLEASE… get some help and make your videos and exercises correct and pedagogically sound.

Dr Nic

PS – anyone else reading this letter, take a look at the following videos for mathematics.

And of course I think my own Statistics Learning Centre videos are pretty darn good as well.

Other posts about concerns about Khan:

Another Open Letter to Sal ( I particularly like the comment by Michael Paul Goldenberg)

Breaking the cycle (A comprehensive summary of the responses to criticism of Khan

The flipped classroom

Back in the mid1980s I was a trainee teacher at a high school in Rotorua. My associate teacher commented that she didn’t like to give homework much of the time as the students tended to practise things wrong, thus entrenching bad habits away from her watchful gaze. She had  a very good point! Bad habits can easily be developed when practising solving equations, trigonometry, geometry.

Recently the idea of the “flipped classroom” has gained traction, particularly enabled by near universal access to internet technology in some schools or neighbourhoods. When one “flips” the classroom, the students spend their homework time learning content – watching a video or reading notes. Then the classroom time is used for putting skills to practice, interactive activities, group work, problem-solving – all active things that are better with the teacher around. Having a teacher stand at the front of the room and lecture for a large percentage of the time is not effective teaching practice.

I ws surprised at a teaching workshop to find that many of the teachers were not even aware of the concept of “flipping”. To me this is a case for Twitter as a form of professional development. To address this gap, I am writing about the flipped classroom, especially with regard to statistics and mathematics.

There are two important aspects to flipping – what the students do when they are not in class, and what students do when they are in class.

Work away from class

In theory, classroom “flipping” has always been possible. You could set students notes to read or sections of the textbook to study. In some schools and cultures this is successful, though it does presuppose a high level of literacy. Universities expect students to read, though my experience is that they avoid it if possible – unless they are taking Law, which of course means they can’t avoid it.

Technology has changed the landscape for flipping. With ready access to the internet it is feasible for video and other work to be set remotely for students. Sometimes teachers prepare the material themselves, and sometimes they may specify a YouTube video or similar to watch. This is not as easy as it may sound. As you can see from my critique of videos about confidence intervals, there is a lot of dross from which to extract the gold. And Khan Academy is no exception.

One big advantage of video over a live lecture, even if the video is merely a talking head, is that the student can control the pace and repeat parts that aren’t clear. My experience of lecturing to classes of several hundred students was that the experience was far from personal. I would set the pace to aim at the middle, as I’m sure most do. In later years I put all my lectures into short audio files with accompanying notes. Students could control the pace and repeat parts they didn’t understand. They could stop and think for a bit and do the exercises as I suggested, sometimes using Excel in parallel. They could quickly look through the notes to see if they even needed to listen to the audio. It was much more individualised.

Another advantage was that you can remove errors, stumbles, gaps and tighten up the experience. I’ve found a fifty minute lecture can be reduced to about half the time, in terms of the recording.

Despite this much more individual approach I was still expected to give lectures (that’s what lecturers do isn’t it?) until the Christchurch earthquakes made my mode of delivery expedient and we were able to stop physical lectures. The students could view the delivery of the material without coming to the university. They could then do exercises, also set up on the LMS, with instant feedback.

Work in the classroom

People tend to focus on what happens away from the classroom, when talking about flipped classroom. It is equally important to think hard about what happens in class. Having the teacher and peers there to help when working through problems in mathematics is better than being at a dead-end at home, with no one to help.  But week after week of turning up to class, working on numbered exercises from the textbooks doesn’t sound like much fun.

Taking the content delivery out of the classroom frees up the teacher for all sorts of different activities. It can be a challenge for teachers to change how they think about how to use the time. There are opportunities for more active learning, based on the grounding done on-line. In a mathematics or statistics classroom there is room for creativity and imagination. Debates, group work, competitions, games, looking for errors, peer review and peer-grading are all possibilities. If anyone thinks there is no room for imagination in the teaching of mathematics, they should take a look at the excellent blog by Fawn Nyugen, Finding ways to Nguyen students over.  I wish she had taught my sons. Or me. (Nguyen is pronounced “Win”)

I am currently working with teachers on teaching statistical report-writing. This is something that benefits from peer review and discussion. Students can work separately to write up results, and then read each other’s work. This is done in English and Social Science classes, and language classes. There is much we can learn from teachers in other disciplines.

Potential Problems

Students can also be resistant to change, and some coaching may be needed at the start of the year.

There is a big investment by teachers if they wish to create their own materials. Finding suitable materials on line can take longer than making your own. A team approach could help here, where teachers pool their resources and provide the “at home” resources and links for each other’s classes. I would be cautious not to try to do too much at once in implementing “flipped classroom”. It would probably be wise to start with just one class at a time.

Where internet access is not universal, there needs to be adaptations. It may be that the students can use school resources out of school time. Or students could take the material home on a memory stick.

Special needs

One issue to consider is the students who have special learning needs. In one Twitter discussion it was suggested that the flipped classroom is great because the student can learn the content when they have a helper (parent!) to assist. This is an admirable theory and I might have agreed had I not been on the other side. As a mother of a son with special needs, the thought of homework was often too much for me. The daily battle of life was enough without adding further challenge. In addition my son had been full-on all day and had little capacity for homework even if I had been willing. We need to avoid assuming ideal circumstances.

Try it!

Overall though, in appropriate circumstances, the concept of flipping has a lot going for it. It is always good to try new things.

If you never have a bad lesson or a failed new idea, you aren’t being daring enough!

Teaching a service course in statistics

Teaching a service course in statistics

Most students who enrol in an initial course in statistics at university level do so because they have to. I did some research on attitudes to statistics in my entry level quantitative methods course, and fewer than 1% of the students had chosen to be in that course. This is a little demoralising, if you happen to think that statistics is worthwhile and interesting.

Teaching a service course in statistics is one of the great challenges of teaching. A “Service Course” is a course in statistics for students who are majoring in some other subject, such as Marketing or Medicine or Education. For some students it is a terminating course – they will never have to look at a p-value again (they hope). For some students it is the precursor to further applied statistics such as marketing research or biological research. Having said that, statistics for citizens is important and interesting and engaging if taught that way. And we might encourage some students to carry on.

Yet the teachers and textbook writers seem to do their best to remove the joy. Statistics is a difficult subject to understand. Often the way the instructor thinks is at odds with the way the students think and learn. The mathematical nature of the subject is invested with all sorts of emotional baggage.

Here are some of the challenges of teaching a statistics service course.

Limited mathematical ability

It is important to appreciate how limited the mathematical understanding is of some of the students in service courses. In my first year quantitative methods course, I made sure my students knew basic algebra, including rearranging and solving equations. This was all done within a business context. Even elementary algebra  was quite a stumbling block to some students, for whom algebra had been a bridge too far at school. There were students in a postgrad course I taught who were not sure which was larger, out of 0.05 and 0.1, and talked about crocodiles with regard to greater than and less than signs. And these were schoolteachers! Another senior maths teacher in that group had been teaching the calculation of confidence intervals, without actually understanding what they were.

The students are not like statisticians. Methods that worked to teach statisticians and mathematicians are unlikely to work for them. I wrote about this in my post about the Golden Rule, and how it applies at a higher level for teaching.

I realised a few years ago that I am not a mathematician. I do not have the ability to think in the abstract that is part of a true mathematician. Operations Research was my thing, because I was good at mathematics, but my understanding was concrete. This has been a surprising gift for me as a teacher, as it has meant that I can understand better what the students find difficult. Formulas do not tell them anything. Calculating by hand does not lead to understanding. It is from this philosophy that I approach the production of my videos. I am particularly pleased with my recent video about confidence intervals, which explains the ideas, with nary a formula in sight, but plenty of memorable images.


One of my more constantly accessed posts is  Excel, SPSS, Minitab or R?. This consistent interest indicates that the course of software is a universal problem.  People are very quick to say how evil Excel is, and I am under no illusions as to many of the shortcomings. The main point of my post was, however, that it depends on the class you are teaching.

As I have taught mainly business students, I still hold that for them, Excel is ideal. Not so much for the statistical aspects, but because they learn to use Excel. Last Saturday the ideas for today’s posts were just forming in my mind when the phone rang, and despite my realising it was probably a telemarketer (we have caller ID on our phone) I answered it. It was a nice young woman asking me to take part in a short survey about employment opportunities for women in the Christchurch Rebuild. After I’d answered the questions, explaining that I was redundant from the university because of the earthquakes and that I had taught statistics, she realised that I had taught her. (This is a pretty common occurrence for me in our small town-city – even when I buy sushi I am served by ex-students). So I asked her about her experience in my course, and she related how she would never have taken the course, but enjoyed it and passed. I asked about Excel, and she told me that she had never realised what you could do with Excel before, and now still used it. This is not an isolated incident. When students are taught Excel as a tool, they use it as a tool, and continue to do so after the course has ended.

When business students learn using Excel, it has the appearance of relevance. They are aware that spreadsheets are used in business. It doesn’t seem like time wasted. So I stand by my choice to use Excel. However if I were still teaching at University, I would also be using iNZight. And if I taught higher levels I would continue to use SPSS, and learn more about R.


As I said in a previous post Statistics Textbooks suck out all the fun. Very few textbooks do no harm. I wonder if this site could provide a database of statistics texts and reviews. I would be happy to review textbooks and include them here. My favourite elementary textbook is, sadly, out of print. It is called “Taking the Fear out of Data Analysis”, by the fabulously named Adamantis Diamantopoulos and Bodo Schlegelmilch. It takes a practical approach, and has a warm, nurturing style. It lacks exercises. I have used extracts from it over the years. The choice of textbook, like the choice of software, is “horses for courses”, but I think there are some horses that should not be put anywhere near a course. I do wonder how many students use textbooks as anything other than a combination lucky charm and paper weight.

In comparison with the plethora of college texts of varying value, at high-school level the pickings for textbooks are thin. This probably reflects the newness of the teaching of statistics at high-school level.  A major problem with textbooks is that they are so quickly out of date, and at school level it is not practical to replace class sets too often.

Perhaps the answer is online resources, which can be updated as needed, and are flexible and give immediate feedback.  😉

Emotional baggage

I was less than gentle with a new acquaintance in the weekend. When asked about my business, I told him that I make on-line materials to help people teach and learn statistics. He proceeded to relate a story of a misplaced use of a percentage as a reason why he never takes any notice of statistics. I have tired of the “Lies, damned lies, and statistics” jibe and decided not to take it lying down. I explained that the world is a better place because of statistical analysis. Much research, including medical would not be possible in the absence of methods for statistical analysis. An understanding of the concepts of statistics is a vital part of intelligent citizenship, especially in these days of big and ubiquitous data.

I stopped at that point, but have pondered since. What is it that makes people so quick to denigrate the worth of statistics? I suspect it is ignorance and fear. They make themselves feel better about their inadequacies by devaluing the things they lack. Just a thought.

This is not an isolated instance. In fact I was so surprised when a lighthouse keeper said that statistics sounded interesting and wanted to know more, that I didn’t really know what to say next! You can read about that in a previous post. Statistics is an interesting subject – really!

But the students in a service course in statistics may well be in the rather large subset of humanity who have yet to appreciate the worth of the subject. They may even have fear and antipathy towards the subject, as I wrote about previously. Anxiety, fear and antipathy for maths, stats and OR.

People are less likely to learn if they have negative attitudes towards the subject. And when they do learn it may well be “learning to pass” rather than actual learning which is internalised.

So what?

Keep the faith! Statistics is an important subject. Keep trying new things. If you never have a bad moment in your teaching, you are not trying enough new things. And when you hear from someone whose life was changed because of your teaching, there is nothing like it!

The median outclasses the mean

The median suffers from poor marketing.

All my time at school the “average” was always calculated as the arithmetic mean, by adding up all the scores and then dividing by the number of scores. When we were taught about the median, it seemed like an inferior version of the mean. It was the thing you worked out when you weren’t smart enough to add and divide. It was used for house prices, and that was about it. Of course the mean was the superior product! Why wouldn’t you use the mean?

I’ve been preparing resources for teaching the fabulous new New Zealand curriculum, and have been brought face-to-face with my prejudices. It strikes me that the median has had very poor representation.

Public opinion of the median and mean

I put a question on Facebook and Twitter to see what people felt about the mean and the median. I briefly explained what each was, then asked which one they thought was better. Some people had no idea what I was talking about, but most felt that the mean was the superior statistic. The following are a selection of responses:

The mean, but I don’t know why.. maybe that’s just what we were taught to use when I was back in school (a long time ago!) lol

When I think of “average” I always think of the mean. I don’t know if it’s actually better though

well the median is a real pain to work out. you have to make a list of all the numbers, in order, and then count how many they are and then go to the middle. PAIN IN THE BUM. the average… well that is somewhat quicker to do, no? and i don’t see the point in the median at all. unless well no, there is just no need for it. who cares what the15th person in the class got on a test? the lowes and highes is much more interesting. As i remember it, the mode is the most commonly occuring number out of a set of numbers… i think of this as the “mode” or in English (not French), the ‘fashionable” number. oh and it stresses me how all 3 start with Ms cos that is confusing. which is why i like to use the word average.

The mean, which I’m guessing is the same as the average? When the media refer to real estate stats they always use median price, which can distort reality, we would prefer the average price. (From a real estate agent)

I don’t really think it’s a case of which is better. They’re two different things aren’t they? I think it’s usually easier to work out the average.

A number of my Facebook friends did know about statistics, and responded in favour of the median in most cases. This was an interesting comment:

“It depends. Everyone who proof read my thesis was like why on earth are you using the median – no one uses it. And most of the other similar primate studies I’ve read use the mean (except one, that was published by my associate supervisor). But my means were off their rocker, and I’m pretty sure my medians were a much better representation of reality in this case. It makes making comparisons between studies a little awkward though.

Why NOT use the median all the time?

I am hard pressed to find an instance where the mean is actually a better measure of central tendency than the median. The purpose of the mean or median (or mode) is to provide a one number summary of a set of data. The whole idea of the mean is actually quite tricky, as you can read in one of my early posts about explaining what the mean is. Generally the summary value is used to compare with another sample or population.

In my lectures I often illustrated times when the median is a better summary measure of a sample or population than the mean. This is quite common in notes and YouTube videos. Never once did I show where the mean was preferred to the median! So why were/are we so loyal to the mean, bringing out the median for special occasions and real estate?

I think there are two answers, both of them no longer valid. It is a question of legacy.

Time and ease to calculate

Despite first appearances, for anything larger than a trivial sample the mean is actually easier to calculate than the median. Putting a set of 100 values in order by hand is no easy task. (Pain in the bum, as my friend so elegantly expressed it.) Adding up scores and dividing by 100 is a walk in the park in comparison.  In the early 1980s when I learned programming (in Fortran, Pascal and Cobol), writing a sorting program was far from trivial and a large set of numbers would take a large amount of time to sort. Only in later years, as computing power has expanded, has it been possible to get a computer to calculate a median.

Formulas for confidence intervals

Means behave nicely and give nice mathematical results when manipulated. Because of this we can calculate confidence intervals using a nifty little formula and statistical tables. Until bootstrapping by computer  became do-able on a large and small scale, there was no practical way to perform inference on a number of very useful statistics, including the median and the inter-quartile range.

Conclusion: the median is better

A median is intrinsically understandable. It is the middle number when the values are put in order. End of story. – Well not quite – you do have that slightly tricky thing where the sample is even and you have to average the middle two terms, but apart from that it is easy!

A median is not affected by outliers. I learned a new term for this when I was reading up in preparation for writing this post. The term is “resistant” and I learned it from one of Mr Tarrou’s videos for AP Statistics. I found these videos after my tirade against videos on confidence intervals. Tarrou’s videos are long and a bit more mathematical than I would like. (He can’t help it – he is a maths teacher and the AP Statistics syllabus seems to have been devised by mathematical statisticians trying to put students off ever taking the subject again.) But they are GOOD. Tarrou’s videos are sound, and interesting and well put together. I will be recommending them as complementary to my own offerings. (Because I sure as heck don’t want to have to do all that icky mathsy stuff).

But I digress. The median is “resistant” because it is not at the mercy of outliers. There are lots of great examples, including in Mr Tarrou’s video. If you have a median of 5 and then add another observation of 80, the median is unlikely to stray far from the 5. However a mean is a fickle beast, and easily swayed by a flashy outlier.

The main disadvantage I can see for the median is that it can be a bit jumpy in small samples made up of discrete values. I guess if you have two well-behaved populations that are very similar and you want to see precise differences then the means might just be better – but even then you would possibly be over-interpreting small differences.

I have found it very interesting observing the behaviour of confidence intervals for the difference of two medians, compared with confidence intervals for the difference in two means. While I was preparing materials for our on-line resource, I performed nine such tests on different real data taken from students at university. The scores are very jumpy, and the differences between the medians often include exactly zero. Consequently the confidence intervals of the difference of two medians quite often have zero as their lower bound. This provides a challenge in interpretation, as I had not met this often when looking at the differences between means. However, it also illuminates the odd relationship we have with zero. Just because a confidence interval for a difference of two means is (-0.13, 3.98) and includes a zero, it is tempting to conclude that there is no significant difference. But is -0.13 really any different from zero in practical terms? The other point is that we should be leaving the confidence interval as it is, rather than stretching it into further inference.

Word on the web

I did a little surfing to see what the word on the web was.  To find out who said what, drop the entire phrase into Google. (Ah ‘tis a wonderful we live in, indeed)

  • “The mean is the one to use with symmetrically distributed data; otherwise, use the median.” Hmm – but if the data is symmetric, surely the mean = the median?
  • “An important property of the mean is that it includes every value in your data set as part of the calculation. In addition, the mean is the only measure of central tendency where the sum of the deviations of each value from the mean is always zero. “ Ok – hard to argue with that.
  • “Calculation of medians is a popular technique in summary statistics and summarizing statistical data, since it is simple to understand and easy to calculate, while also giving a measure that is more robust in the presence of outlier values than is the mean.” Totally!
  • “However, when the sample size is large and does not include outliers, the mean score usually provides a better measure of central tendency. “(Then goes on to give an example of when the median is better.)
  • “Use the median to describe the middle of a set of data that does have an outlier. Advantages of the median: Extreme values (outliers) do not affect the median as strongly as they do the mean, useful when comparing sets of data, it is unique – there is only one answer.
    Disadvantages of the median:  Not as popular as mean.(Not as popular??!)

Sorry median  – you do not win X-Factor for summary statistics. You may be more robust, and less fickle, not to mention easier to understand, but you just aren’t as popular!

I can feel a video coming on – the median has been relegated to the periphery long enough!

Update in 2018

Here is our video about different summary statistics, which also addresses the relative merits of mean and median, and why they even matter!

Good, Bad and Wrong: Videos about Confidence Intervals

Videos are useful teaching and learning resources

There is much talk about “flipped classrooms” and the wonders of Khan Academy, YouTube abounds with videos about everything…really! Even television news reports show YouTube clips. Teachers and instructors use videos in their teaching, and get their students to watch them at home, ready to build on in class time. A well put-together video can provide a different way of looking at a problem that helps a student to learn. Videos are endlessly patient and can be paused and watched at the students’ pace. (See my earlier post on multimedia for a fuller discourse on good multimedia.) The problem is: How is a teacher to know what is good and what is not? This seems to be especially difficult in an area like statistics.

I decided this week to see what was on offer and summarise for you all. To narrow it down I chose the topic of Confidence intervals. This topic is pretty universal to statistics courses, and is conceptually tricky. I wanted to see if there was a quick way of working out if a video is any good or not, without having to watch them. I was prepared to suffer so that my readers would not have to.

Videos about confidence intervals are mostly awful

And suffer I did. Not to beat around the bush – many of the videos I watched were awful. There is no other word for it. Not only were they slow, boring, mathematically based, unscripted and unedited, but many of them were just plain wrong. Back in 2008 I went looking for a video about confidence intervals for my students, and realised I had to make my own. It is still true. I do not doubt that the videos are well intentioned. Many of them may have been made (as mine were originally) for a specific class (or family member), and thus were not intended for a larger audience. Maybe those ones should come with disclaimers. “I’m not sure I really know what I am talking about – view at your own risk.”

I put “Confidence Intervals” into the YouTube search engine and examined nine of the top offerings. Mostly I went for the videos with the most views, as this would appear to be a way of filtering out poor material. (wrong again, as you will see) I also included two of my own videos.

Most these videos should not be seen by a wider audience. No – that’s not right – most of them should not be seen – by anyone. The impression they give of statistics is of a bumbling professor talking about formulas and looking up tables. Nothing in them gives a single hint about how interesting, applied and relevant the subject of statistics is. Maybe there needs to be a wikipedia approach to supposedly educational videos to provide quality control.There is just one video other than my own two that I approve of – by Keith Bower. (Biased, I know – feel free to respond.)

A possible way if you wish to find useful videos, might be to get the students to find a video they think is good, then you check that it is correct. Trying to find a good video about statistics is not a good use of your time – unless of course, you just go straight to the Statistics Learning Centre channel or Keith Bower.  🙂

If any of you gentle readers have a video you think is worth a second look, please put the link in the comments.

Brief reviews of ten videos on Confidence Intervals in no especial order except that I left the three good ones until last

I started with the videos that appeared at the top. They have paid to be first in the list, so I thought they might be good. As it turns out they are very similar to each other and from the same stable, it seems. I found them lacklustre, though not totally harmful.

1. Statistics – Confidence Intervals
Channel: EducatorVids2                Videos on the channel: 1192
17268 views  Loaded 24 Oct 2011 (32 views per day)    Duration 3:25

This video and the next one are part of some sort of course. This video seemed to start in the middle of a lecture “Now let’s go on to some examples”. The layout was utilitarian, with a talking head, and a screen showing the  working. The video, like most of them, was not scripted. The content was based on a Mathematical example with no context. I don’t really know what she was talking about. But at least it was short.

2. Statistics: Confidence Intervals (Difference in Means)
Channel                    Videos on the channel: 1914
7381 views, loaded 5 Nov 2009  (6 views per day) Duration 3:46

Very similar format to EducatorVids2. The bulk of the content was around a medical example with  7 subjects. Again it was not scripted. The  computation was tedious so that I had to fast forward.

And here is the one you have all been waiting for: Khan Academy. I should know better than to suggest that the mighty Khan is less than perfect (my previous post about KA  continues to provoke defensive vitriol.) But here goes:

3. Confidence Interval 1
Khan Academy                                  Videos on the channel: 3492
167, 213 views, loaded 28 Oct 2010 (186 views per day) Duration 14:03
246 likes 20 dislikes.

Like all Khan Academy videos (as far as I know) the format is very simple with a black screen with printed example. Again the video is not scripted, and consists of a lot of repetition as Khan doesn’t like empty air while he is writing.It is actually a lead into confidence intervals, doing a theoretical exercise involving the sampling distribution. Thus it talks about probabilities.  It would have been better to entitle it, preparation for confidence intervals, as it doesn’t actually teach about confidence intervals, and includes probability. Khan included steps to using tables to find a t value. This video was really not nice.  And it took 14 minutes! That is 14 minutes I will never have again. It is also a long time to find out that it doesn’t actually teach about confidence intervals. This video is one of the worst of the ten I viewed, and has far more views than it ought.

4. Statistics is easy: Confidence Interval
aghasemi4u                           Videos on the channel: 4 about statistics
a remarkable 296,456 views, loaded 23 March 2007, (136 per day)  Duration 5:00
186 likes and 83 dislikes

This video was simple and reasonably well put together, with nice diagrams, but only three slides in its five minutes.  The narration is unscripted and uses probability to describe the confidence interval (wrong!). There was a focus on the mathematical formula.

5. . Confidence Intervals
Madonna USI                          Videos on the channel: 22
18,403 views        Uploaded 9 Nov 2009 (15 per day) Duration 9:42
102 likes, 2 dislikes

A brief description of what confidence intervals are as well as a couple examples.Live person working on a whiteboard. Refers to a textbook. Very slow. Definition wrong – Says that we are 95% confident that the value that we found is within the range. I’m hoping this is just a slip of the tongue, but it should have been editted out.

6. How to calculate Confidence Intervals and Margin of Error
Statistics is fun                  Videos on the channel: 80
25,750 views.   Uploaded 12 July 2011 (40 per day) Duration: 6:44
145 likes, 3 dislikes

Summarised before and after, which can be tedious. Mathematically based. Slick graphics, but glacially slow in parts. Gives an example with no context. This is not statistics! Tedious. To be fair, there are lots of positive comments, and as the title says “how to calculate confidence intervals” there is no requirement to explain what they are when you get them. The channel is all about “how to calculate” and nothing about context, so I think it is a bit of a misnomer to call it “Statistics is fun”.

7. Confidence Intervals Part1 YouTube
Larry Shrewsbury Videos on the channel: 15
82,006 views. Uploaded on 12 Jul 2009 (128  per day) Duration 7:42
136 likes 53 dislikes

Part of an enterprise “Taking the Sadistics out of learning Statistics”
I found the voice irritating as it seems patronising. However some people find my accent distracting, (wot eksent?) so I can’t really be too hard on that. Very formula based, looking at the mathematics rather than the interpretation. The best part was an interesting animation – very nice way of looking at traditional confidence intervals that I hadn’t seen before.

Here are the three good videos:

8. The history, use and certain limitations of confidence intervals in statistics.
Keith Bower   Videos on the channel: 49
32,883 views. Uploaded 5 Jan 2009 (21 views per day) Duration:3:25
66 likes, 5 dislikes

Keith’s presentation isn’t visually exciting, but he is correct and clear and that goes a long way. His is just a talking head – but he is an interesting presenter and very fluent. His video has branching, such that you can click to go to another video if needed. I’ve found all his videos sound and sensible. (I got “p is low, null must go” from one of his videos.)

9. Confidence Intervals in Excel
UCMSCI              Videos on the channel
17797 views  Loaded 25 Dec 2008 (11 per day)
26 likes 0 dislikes

This was one of my earlier videos. It is scripted with visuals to help in comprehension. It takes the classical approach to confidence intervals and puts emphasis on the idea of level of confidence. Addresses the aspects that affect the width of confidence intervals. Discusses the formula for confidence intervals, and shows how to use Excel to calculate them. (I don’t think I really loaded it on Christmas day! Maybe some strange dateline thing)

10. Understanding Confidence Intervals: Statistics Help
Statistics Learning Centre   Videos on the channel: 19
550 views    Uploaded 26 March 2013  (31 per day)  6 likes 0 dislikes Duration: 4:02

This video will disappoint the mathematicians, as there are no formulas. But students love it. The point of the video is to explain what a confidence interval is, and what things affect the size of the interval. It makes use of diagrams and examples to help students understand the concepts. It is tightly scripted and edited with no wasted time. People can always pause if they need to, but it is difficult to speed up a slow presentation.

Epilogue (Obituary?)

And there you have it, folks – there is no easy way to work out which videos are going to be useful for your students without watching them all. Sorry. And if you expect me to go through this again with another topic, you clearly didn’t get the subtext.

Which comes first – problem or solution?

In teaching it can be difficult to know whether to start with a problem or a solution method. It seems more obvious to start with the problem, but sometimes it is better to introduce the possibility of the solution before posing the problem.

Mathematics teaching

A common teaching method in mathematics is to teach the theory, followed by applications. Or not followed by applications. I seem to remember learning a lot of mathematics with absolutely no application – which was fine by me, because it was fun. My husband once came home from survey school, and excitedly told me that he was using complex numbers for some sort of transformation between two irregular surfaces. Who’d have thought? I had never dreamed there could be a real-life use for the square root of -1. I just thought it was a cool idea someone thought up for the heck of it.

But yet again we come to the point that statistics and operations research are not mathematics. Without context and real-life application they cease to exist and turn into … mathematics!

Applicable mathematics

My colleague wrote a guest post about “applicable mathematics” which he separates from “applied mathematics”. Applicable maths appears when teachers make up applications to try to make mathematics seem useful. There is little to recommend about applicable maths. A form of “applicable maths” occurs in probability assessment questions where the examiner decides not to tell the examinee all the information, and the examinee has to draw Venn diagrams and use logical thinking to find out something that clearly anyone in the real world would be able to read in the data! I actually enjoy answering questions like that, and they have a point in helping students understand the underlying structure of the data. But I do not fool myself into thinking that they are anywhere near real-life. Nor are they statistics.

Which first – theory or application?

So the question is – when teaching statistics and operations research, should you start with an application or a problem or a case, and work from there to the theory? Or do students need some theory, or at least an understanding of basic principles before a case or problem can have any meaning? Or in a sequence of learning do we move back and forward between theory and application?

My first off response is that of course we should start with the data, as many books on the teaching of statistics teach us. Well actually we should start with the problem, as that really precedes the collection of the data. But then, how can we know what sorts of problems to frame if we don’t have some idea of what is possible through modelling and statistics? So should we first begin with some theory? The New Zealand Curriculum emphasises the PPDAC cycle, Problem, Plan, Data, Analysis, Conclusion. However, in order to pose the problem in the first place, we need the theory of the PPDAC cycle itself. The answer is not simple and depends on the context.

I have recently made a set of three videos explaining confidence intervals and bootstrapping. These are two very difficult topics that become simple in an instant. What I mean by that is, until you understand a confidence interval, it makes no sense, and you can see no reason why it should make sense. You go through a “liminal space” of confusion and anxiety. Then when you emerge out the other side, instantly confidence intervals make sense, and it is equally difficult to see what it was that made them confusing. This dichotomy makes teaching difficult, as the teacher needs to try to understand what made the problem confusing.

I present the idea of a confidence interval first. Then I use examples. I present the idea of bootstrapping, then give examples. I think in this instance it is helpful to delineate the theory or the idea in reasonably abstract form, interspersed with examples. I also think diagrams are immensely useful, but that’s another topic.

Critique of AtMyPace: Statistics

What prompted these thoughts about “which comes first” was a comment made about our “AtMyPace: Statistics” iOS app.

The YouTube videos used in AtMyPace:Statistics were developed to answer specific needs in a course. They generally take the format of a quick summary of the theory, followed by an example, often related to Helen and her business selling choconutties.

The iOS app, AtMyPace:Statistics was set up as a way to capitalise on the success of the YouTube videos, and we added two quizzes of ten True/false questions to complement each of the videos. We also put these same quizzes in our on-line course and found that they were surprisingly popular. In a way, they are a substitute for a textbook or notes, but require the person to commit one way or the other to an answer before reading a further explanation. We had happened on a effective way of engaging students with the material.

AtMyPace:Statistics is not designed to be a full course in statistics, but rather a tool to help students who might be struggling with concepts. We have also developed a web-based version of AtMyPace:Statistics for those who are not the happy owners of iOS devices. At present the web version is a copy of the app, but we will happily add other questions and activities when the demand arises.

I received the following critique of the AtMyPace: Statistics app:

“They are nicely done but very classical in scope. The approach is tools-oriented using a few “realistic” examples to demonstrate the tool. This could work for students who need to take exams and want accessible material.”

Very true. The material in AtMyPace:Statistics is classical in scope, as we focus on the material currently being taught in most business schools and first year statistics service courses. We are trying to make a living, and once that is happening we will set out to change the world!

The reviewer continues,

“ I think that in adult education you should reverse the order and have the training problem oriented. Take a six sigma DMAIC process as an example. The backbone is a problem scheduled to be solved. The path is DMAIC and the tools are supporting the journey. If you want to do it that way you need to tailor the problem to the audience. “

In tailored adult education it is likely that a problem-based approach will work. I would strongly recommend it.

I had an interesting discussion some time ago with a young lecturer working in a prestigious case-based MBA programme in North America. The entire MBA is taught using cases, and is popular and successful. My friend had some reservations about case-based teaching for a subject like Operations Research which has a body of skills which are needed as a foundation for analysis. Statistics would be similar. The question is making sure the students have the necessary skills and knowledge, with the ability to transfer to another setting or problem. Case-based learning is not an efficient way to accomplish this.

Criticism on Choosing the Test procedure

In another instance, David Munroe commented on our video “Choosing which statistical test to use”, which receives about 1000 views a week.  In the video I suggest a three step process involving thinking about what kind of data we have, what kind of sample, and the purpose of the analysis. The comment was:

Myself I would put purpose first. 🙂 The purpose of the analysis determines what data should be collected – and more data is not necessarily more informative. In my view it is more useful to think ‘what am I trying to achieve’ with this analysis before collecting the data (so the right data have a chance to be collected). This in contrast to: collecting the data and then going ‘now what can I get from this data?’ (although this is sometimes an appropriate research technique). I think because we’ve already collected the data any time we’re illustrating particular modelling tools or statistical tests, we reinforce the ‘collect the data first then worry about analysis’ approach – at least subconsciously.

Thanks David! Good thinking, and if I ever redo the video I may well change the order. I chose the order I did, as it seemed to go from easy to difficult. (Actually I don’t remember consciously thinking about the order – it just fell out of individual help sessions with students.)  And the diagram was developed in response to the rather artificial problems I was posing!

I’ll step back a bit and explain. One problem I have seen in teaching Statistics and Operations Research is that students fail to make connections. They also compartmentalise the different aspects and find it difficult to work out when certain procedures would be most useful. I wrote a post about this. In the statistics course I wrote a set of scenarios describing possible applications of statistical methods in a business context. The students were required to work out which technique to use in each scenario and found this remarkably difficult. They could perform a test on difference of two means quite well, but were hard-pressed to discern when the test should be used. So I made up even more questions to give them more practice, and designed my three step method for deciding on the test.  This helped.

I had not thought of it as a way to decide in a real-life situation which test to use. Surely that would be part of a much bigger process.  So my questions are rather artificial, but that doesn’t make them bad questions. Their point was to help students make linkages between different parts of the course. And for that, it works.

Bring on the criticism

I would like to finish by saying how much I appreciate criticism. It is nice when people tell me they like my materials. I feel as if I am doing something useful and helping people. I get frequent comments of this type on my YouTube site.  But when people make the effort to point out gaps and flaws in the material I am extremely grateful as it helps me to clarify my thinking and improve the approach. If nothing else, it gives me something to talk about in my blog. It is difficult producing material in a feedback vacuum.  So keep it coming!


In statistical analysis the word “significant” means that there is evidence that effect found in the sample exists in the population from which the sample was drawn. The choice of the word “significant” is unfortunate, as it is used to mean something different in common language. Reporters hear a scientist say that there is a significant effect, and tend to think big. Results gets reported as significant, meaning big, and we have effect inflation.

Where do p-values come from?

In reality, if we take a large enough sample, even a small effect will show up as significant. Because the sample is large, it is easier to detect and be sure of the existence of small effects in the population. However, this does not mean that the effect is notable or makes any difference.

Unfortunately this confusion is rife in the reporting of medical and educational research. A drug may have a statistically significant effect, which means that there is evidence that it exists in the population, but it may be to reduce incidence from 2 in a thousand to 1 in a thousand, which isn’t really much of a difference. To make matters worse a result like this can also be stated as a 50% decrease, which makes it seem even more miraculous.

This post is more about learning statistics, but those who teach it really need to be alert for this misconception. We have just posted our latest YouTube video explaining significance and usefulness, evidence and strength, association and causation.

Within the video I have tried to give memorable images, which students will hold onto, even if they don’t quite remember the reasoning. The p-value shrinking as the Evidence label grows, aims to help students understand that a small p means more evidence to reject the null. I’m also really pleased with my “p-machine”, turning the mean, standard deviation and sample size into a t statistic, which is then converted to a p-value.

There are just a few really big ideas in statistics, and these are some of them. This forms part of statistical literacy, which is important to all citizens. I hope you may find the video useful in helping students remember.