# Videos for teaching and learning statistics

It delights me that several of my statistics videos have been viewed over half a million times each. As well there is a stream of lovely comments (with the odd weird one) from happy viewers, who have found in the videos an answer to their problems.

In this post I will outline the main videos available on the Statistics Learning Centre YouTube Channel. They already belong to 24,000 playlists and lists of recommended resources in textbooks the world over. We are happy for teachers and learners to continue to link to them. Having them all in one place should make it easier for instructors to decide which ones to use in their courses.

# Philosophy of the videos

Early on in my video production I wrote a series of blog posts about the videos. One was Effective multimedia teaching videos. The videos use graphics and audio to increase understanding and retention, and are mostly aimed at conceptual understanding rather than procedural understanding.

I also wrote a critique of Khan Academy videos, explaining why I felt they should be improved. Not surprisingly this ruffled a few feathers and remains my most commented on post. I would be thrilled if Khan had lifted his game, but I fear this is not the case. The Khan Academy pie chart video still uses an unacceptable example with too many and ordered categories. (January 2018)

Before setting out to make videos about confidence intervals, I critiqued the existing offerings in this post. At the time the videos were all about how to find a confidence interval, and not what it does. I suspect that may be why my video, Understanding Confidence Intervals, remains popular.

# Introducing statistics

## Understanding Summary Statistics 5:14 minutes

Why we need summary statistics and what each of them does. It is not about how to calculate the statistics, but what they mean. It uses the shoe example, which also appears in the PPDAC and OSEM videos.

## Understanding Graphs 6:06 minutes

I briefly explains the use and interpretation of seven different types of statistical graph. They include the pictogram, bar chart, pie chart, dot plot, stem and leaf, scatterplot and time series.

## Analysing and commenting on Graphical output using OSEM 7:13 minutes

This video teaches how to comment on graphs and other statistical output by using the acronym OSEM. It is especially useful for students in NCEA statistics classes in New Zealand, but many people everywhere can find OSEM awesome! We use the example of comparing the number of pairs of shoes men and women students say they own.

## Variation and Sampling error 6:30 minutes

Statistical methods are necessary because of the existence of variation. Sampling error is one source of variation, and is often misunderstood. This video explains sampling error, along with natural variation, explainable variation and variation due to bias. There is an accompanying video on non-sampling error.

## Sampling methods 4:54 minutes 500,000 views

This video describes five common methods of sampling in data collection – simple random, convenience, systematic, cluster and stratified. Each method has a helpful symbolic representation.

## Types of data 6:20 minutes 600,000 views

The kind of graph and analysis we can do with specific data is related to the type of data it is. In this video we explain the different levels of data, with examples. This video is particularly popular at the start of courses.

## Important Statistical concepts 5:34 minutes 50,000 views

This video does not receive the views it deserves, as it covers three really important ideas. Maybe I should split it up into three videos. The ideas are the difference between significance and usefulness, evidence and strength of effect, causation and association.

Other videos complementary to these, but not on YouTube are:

• The statistical enquiry process
• Understanding the Box Plot
• Non-sampling error

# Videos for teaching hypothesis testing

## Understanding Statistical inference 6:46 minutes 40,000 views

The most difficult concept in statistics is that of inference. This video explains what statistical inference is and gives memorable examples. It is based on research around three concepts pivotal to inference – that the sample is likely to be a good representation of the population, that there is an element of uncertainty as to how well the sample represents the population, and that the way the sample is taken matters.

## Understanding the p-value 4:43 minutes 500,000 views

This video explains how to use the p-value to draw conclusions from statistical output. It includes the story of Helen, making sure that the choconutties she sells have sufficient peanuts. It introduces the helpful phrase “p is low, null must go”.

## Inference and evidence 3:34 minutes

This is a newer video, based on a little example I used in lectures to help students see the link between evidence and inference. Of course it involves chocolate.

## Hypothesis tests 7:38 minutes 350,000 views

This entertaining video works step-by-step through a hypothesis test. Helen wishes to know whether giving away free stickers will increase her chocolate sales. This video develops the ideas from “Understanding the p-value”, giving more of the process of hypothesis testing. It is also complemented by the following video, that shows how to perform the analysis using Excel.

## Two-means t-test in Excel 3:54 minutes 50,000 views

A step-by-step lesson on how to perform an independent samples t-test for difference of two means using the Data Analysis ToolPak in Excel. This is a companion video to Hypothesis tests, p-value, two means t-test.

## Choosing which statistical test to use 9:33 minutes 500,000 views

I am particularly proud of this video, and the way it links the different tests together. It took a lot of work to come up with this. First it outlines a process for thinking about the data, the sample and the thing you are trying to find out. Then it works through seven tests with scenarios based around Helen and the Choconutties. This video is particularly popular near the end of the semester, for tying together the different tests and applications.

# Confidence Intervals

## Understanding Confidence Intervals 4:02 minutes 500,000 views

This short video gives an explanation of the concept of confidence intervals, with helpful diagrams and examples. The emphasis is on what a confidence interval is and how it is used, rather than how they are calculated or derived.

## Calculating the confidence interval for a mean using a formula 5:29 minutes 200,000 views

This video carries on from “Understanding Confidence Intervals” and introduces a formula for calculating a confidence interval for a mean. It uses graphics and animation to help understanding.

There are also videos pertinent to the New Zealand curriculum using bootstrapping and informal methods to find confidence intervals.

# Probability

## Introduction to Probability 2:54 minutes

This video explains what probability is and why we use it. It does NOT use dice, coins or balls in urns. It is the first in a series of six videos introducing basic probability with a conceptual approach. The other five videos can be accessed through subscription.

## Understanding Random Variables 5:08 minutes 90,000 views

The idea of a random variable can be surprisingly difficult. In this video we help you learn what a random variable is, and the difference between discrete and continuous random variables. It uses the example of Luke and his ice cream stand.

## Understanding the Normal Distribution 7:44 minutes

In this video we explain the characteristics of the normal distribution, and why it is so useful as a model for real-life entities.

There are also two other videos about random variables, discrete and continuous.

## Risk and Screening 7:54 minutes

This video explains about risk and screening, and shows how to calculate and express rates of false positives and false negatives. An imaginary disease, “Earpox” is used for the examples.

# Other videos

## Designing a Questionnaire 5:23 minutes 40,000 views

This was written specifically to support learning in Level 1 NCEA in the NZ school system but is relevant for anyone needing to design a questionnaire. There is a companion video on good and bad questions.

# Line-fitting and regression

## Scatterplots in Excel 5:17 minutes

The first step in doing a regression in Excel is to fit the line using a Scatter plot. This video shows how to do this, illustrated by the story of Helen and the effect of temperature on her sales of choconutties

## Regression in Excel 6:27 minutes

This video explains Regression and how to perform regression in Excel and interpret the output. The story of Helen and her choconutties continues. This follows on from Scatterplots in Excel and Understanding the p-value.

There are three videos introducing bivariate relationships in a more conceptual way.

There are also videos covering experimental design and randomisation, time series analysis and networks. In the pipeline is a video “understanding the Central Limit Theorem.”

# Supporting our endeavours

As explained in a previous post, Lessons for a budding Social Enterprise, Statistics Learning Centre is a social enterprise, with our aim to build a world of mathematicians and enable people to make intelligent use of statistics. Though we get some income from YouTube videos, it does not support the development of more videos. If you would like to help us to create further videos contact us to discuss subscriptions, sponsorship, donations and advertising possibilities. info@statsLC.com or n.petty@statsLC.com.

# Political polls – why do they work – or don’t

This is written in the week before the 2017 New Zealand General Election and it is an exciting time. Many New Zealanders are finding political polls fascinating right now. We wait with bated breath for each new announcement – is our team winning this time? If it goes the way we want, we accept the result with gratitude and joy. If not, then we conclude that the polling system was at fault.

Many wonder how on earth asking 1000 people can possibly give a reading of the views of all New Zealanders. This is not a silly question. I have only occasionally been polled, so how can I believe the polls reflect my view? As a statistical communicator, I have given some thought to this. If you are a statistician or a teacher of statistics, how would you explain that inference works?

Here is my take on it.

## A bowl of seeds

Imagine you have a bowl of seeds – mustard and rocket. All the seeds are about the same size, and have been mixed up. These seeds are TINY, so several million seeds only fill up a large bowl. We will call this bowl the population. Let’s say for now that the bowl contains exactly half and half mustard and rocket, and you suspect that to be the case, but you do not know for sure.

Say you take out 10 seeds. The most likely result is that you will get 4,5 or 6 mustard seeds. There is a 65% chance, that that is what will happen. If you got any of those results, you would think that the bowl might be about half and half. You would be surprised if they were all mustard seeds. But it is possible that all ten seeds are the same. The probability of getting all mustard seeds or all rocket seeds from a bowl of half and half is about 0.002 or one chance in five hundred.

Now, if you draw out 1000 seeds, it is quite a different story. If all the 1000 seeds drawn out were mustard, you would justifiably conclude that the bowl is not half and half, and may in fact have no rocket seeds. But where do we draw the line? How likely is it to get 40% or less mustard from our 50/50 bowl? Well it is about one chance in 12,000. It is possible, but extremely unlikely – though not as unlikely as winning Lotto. We can see that the sample of 1000 seeds gives us a general idea of what is in the bowl, but we would never think it was an exact representation. If our sample was 51% mustard, we would not sensibly conclude that the seeds in the bowl were not half and half. In fact, there is only a 47% chance that we will get a sample of seeds that is between 49% and 51%.

## People are not seeds

Of course we know we are not little seeds, but people. In fact we like to think we are all special snowflakes.  (The scene from “Life of Brian” springs to mind. Brian – “You are all individuals”, crowd – “We are all individuals”, single response – “I’m not!”)

But the truth is that as a group we do act in surprisingly consistent ways. Every year as a university lecturer I tried new things to help my students to learn. And every year the pass rate was disappointingly consistent. I later devised a course that anyone could pass if they put the work in. They could keep resitting the tests until they passed. And the pass rate stayed around the same.

People do tend to act in similar ways. So if one person changes their viewpoint, there is a pretty good chance that others will have also. So long as we are aware of the limitations in precision, samples are good indicators of the populations from which they are drawn.

I have described why polls generally work. The media tends to dwell on the times that they fail, so let’s look at why that may be.

## Sampling error

Sometimes the poll may just be the one that takes an unlikely sample.  There is a one in a thousand chance that ten seeds from my bowl will all be mustard – and a one in a thousand chance that all will be rocket. It is not very likely, but it can happen. Similarly there is a teeny chance that we will get a result of less than 45% or more than 55% when we take out 1000 seeds. Not likely, but possible. This is called sampling error, and that is what the margin of error is about. Political polls in NZ generally take a sample of 1000 people, which leads to a margin of error of about 3%. What margin of error means is that we can make an interval of 3% either side of the estimate and be pretty sure that it encloses the real value from the population. So if a poll says 45% following for the Mustard Party, then we can be pretty sure that the actual following back in the population is between 42% and 48%. And what does “pretty sure” mean? It means that about one time in twenty we will get it wrong and the actual following, back in the population is outside that range. The problem is we NEVER know if this is the right one or the wrong one.  (Though I personally choose to decide that the polls that I don’t like are the wrong ones. ;))

Non-sampling error and bias

There are other problems – known as non-sampling error. I wrote a short post on it previously.

And this is where the difference between seeds and people becomes important. Some issues are:

When we take a handful of seeds from a well-mixed up bowl, every seed really does have an equal chance of being selected. But getting such a sample from the population of New Zealand is much more difficult. When landlines were in most homes, a phone poll could be a pretty representative sample. However, these days many people have only mobile phones, and which means they are less likely to be called. This would not be a problem if there were no differences politically between landline holders and others. I think most people would see that younger people are less likely to be polled than older, if landlines are used, and younger people quite possibly have different political views. Good polling companies are aware of this and use quota sampling and other methods to try to mitigate this.

The wording of the question and the order of questions can affect what people say. You can usually find out what question has been asked in a particular poll, and it should be reported as part of the report.

Unlike seeds, people do not always show their true colours. If a person is answering a poll within earshot of another family member, they may give a different answer to what they actually tick on election day. Some people are undecided, and may change their mind in the booth. Undecided voters are difficult to account for in statistics, as an undecided voter swinging between two possible coalition partners will have a different impact from a person who has not opinion or may vacillate wildly.

## When the poll is held

In a volatile political environment like the one we are experiencing in New Zealand, people can change their mind from day to day as new leaders emerge, scandals are uncovered, and even in response to reporting of political polls. The results of a poll can be affected by the day and time that the questions were asked.

## Can you believe a poll?

On balance, polls are a blunt instrument, that can give a vague idea about who people are likely to  vote for. They do work, within their limitations, but the limitations are fairly substantial. We need to be sceptical of polls, and bear in mind that the margin of error only  deals with sampling error, not all the other sources of error and bias.

And as they say – the only truly correct poll is the one on Election Day.

# Study elections in mathematics because it is important

Too often mathematics is seen as pure and apolitical.  Maths teachers may keep away from concepts that seem messy and without right and wrong answers. However, teachers of mathematics and statistics have much to offer to increase democratic power in the upcoming NZ general elections (and all future elections really). The bizarre outcomes for elections around the world recently (2016/2017 Brexit, Trump) are evidence that we need a compassionate, rational, informed populace, who is engaged in the political process, to choose who will lead our country. Knowledge is power, and when people do not understand the political process, they are less likely to vote. We need to make sure that students understand how voting, the electoral system, and political polls work. Some of our students in Year 13 will be voting this election, and students’ parents can be influenced to vote.

There are some lessons provided on the Electoral Commission site.   Sadly all the teaching resources are positioned in the social studies learning area – with none in statistics and mathematics. Similarly in the Senior Secondary guides, all the results from a search on elections were in the social studies subject area.

## Elections are mathematically and statistically interesting and relevant

In New Zealand, our MMP system throws up some very interesting mathematical processes for higher level explorations. Political polls will be constantly in the news, and provide up-to-date material for discussions about polls, sample sizes, sampling methods, sampling error etc.

## Feedback

It would be great to hear from anyone who uses these ideas. If you have developed them further, so much the better. Do share with us in the comments.

# Suggestions for lessons

These suggestions for lessons are listed more or less in increasing levels of complexity. However I have been amazed at what Year 1 children can do. It seems to me that they are more willing to tackle difficult tasks than many older children. These lessons also embrace other curriculum areas such as technology, English and social studies.

## Physical resources

Make a ballot box, make a voting paper. Talk about randomising the names on the paper. How big does the box need to be? How many ballot boxes are being made for the upcoming election? How much cardboard is needed?

Make a time series graph of poll results. Each time there is a new result, plot it on the graph over the date, and note the sample size. At higher levels you might like to put confidence intervals on either side of the plotted value. A rule of thumb is 1/square root of the sample size. For example if the sample size is 700, the margin of error is 3.7%. So if the poll reported a party gaining 34% of the vote, the confidence interval would be from 33.3% to 37.7%.

## From NZ maths  – On the Campaign Trail (CL 4)

Figure it Out, Number sense  Book 2 Level 4 – has an exercise about finding fractions, decimals, and percentages of amounts expressed as whole numbers, simple fractions, and decimals.

## From NZ maths – Whose News (CL 4)

This is a guide to running an analysis on the level of representation of different geopgraphic areas in the news. The same lesson could be used for representation of different parties or different issues.

## Graphical representations

The newspapers and online will be full of graphs and other graphical representations. Keep a collection and evaluate them for clarity and attractiveness.

## How many people will be employed on election day?

This inquiry uses a mixture of internet search, mathematical modelling, estimation and calculation.

• How many electorates are there?
• How many polling booths per electorate?
• How many people per booth?
• How long are they employed for?

## Fairness of booth provision

• Is the location of polling booths fair?
• What is the furthest distance a person might need to travel to a voting booth?
• What do people in other countries do?

## The mathematics of MMP

This link provides a thorough explanation of the system. A project could be for students to work out what it is saying and make a powerpoint presentation or short video explaining it more simply.

## Overhang and scenario modelling

Overhang occurs when a party gets more electoral MPS elected than their proportion allows. Here is a fact sheet about overhang and findings of the electoral review. Students could create scenarios to evaluate the effect of overhang and find out what is the biggest overhang possible.

## Small party provisions

How might the previous two election results have been different if there were not the 5% and coat-tailing rules?

## Gerrymandering

Different ways of assigning areas to electorates get different results. The Wikipedia article on Gerrymandering has some great examples and diagrams on how it all happens, and the history behind the name.

## Statistical analysis of age and other demographics

Statistics should be analysed in response to a problem, rather than just for the sake of it.
Suggested Scenario: A new political party is planning to appeal to young voters, under 30 years of age. They wish to find out which five electorates are the best to target. You may also wish to include turn-out statistics in your analysis.

## Statistical analysis of turn out

In the interests of better democracy, we wish to have a better voter turnout. Find out the five electorates with the best voter turnout and the worst, and come up with some ideas about why they are the best and the worst. Test out your theory/model by trying to predict the next five best and worst. Use what you find out to suggest how might we improve voter turnout.

# Graphs – beauty and truth (with apologies to Keats)

## A good graph is elegant

I really like graphs. I like the way graphs turn numbers into pictures. A good graph is elegant. It uses a few well-placed lines to communicate what would take a paragraph of text. And like a good piece of literature or art, a good graph continues to give, beyond the first reading. I love looking at my YouTube and WordPress graphs. These graphs tell me stories. The WordPress analytics tell me that when I put up a new post, I get more hits, but that everyday more than 1000 people read one of my posts. The YouTube analytics tell me stories about when people want to know about different aspects of statistics. It is currently the end of the North American school year, and the demand is for my video on Choosing which statistical test to use. Earlier in the year, the video about levels of measurement is the most popular. And not many people view videos about statistics on the 25th of December. I’m happy to report that the YouTube and WordPress graphs are good graphs.

Spreadsheets have made it possible for anyone and everyone to create graphs. I like that graphs are easier to make. Drawing graphs by hand is a laborious task and fraught with error. But sometimes my heart aches when I see a graph used badly. I suspect that this is when a graphic artist has taken control, and the search for beauty has over-ridden the need for truth.

Three graphs spurred me to write this post.

## Graph One: Bad-tasting Donut on house occupation

The first was on a website to find out about property values. I must have clicked onto something to find out about the property values in my area, and was taken to the qv website. And this is the graph that disturbed me.

Graphs named after food are seldom a good idea

Sure it is pretty – uses pretty colours and shading, and you can find out what it is saying by looking at the key – with the numbers beside it. But a pie or donut chart should not be used for data which has inherent order. The result here is that the segments are not in order. Or rather they are ordered from most frequent to least frequent, which is not intuitive. Ordinal data is best represented in a bar or column chart. To be honest, most data is best represented in a bar or column chart. My significant other suggested that bar charts aren’t as attractive as pie charts. Circles are prettier than rectangles. Circles are curvy and seem friendlier than straight lines and rectangles. So prettiness has triumphed over truth.

## Graph Two: Misleading pictogram (a tautology?)

It may be a little strong to call bad communication lack of truth. Let’s look at another example. In a way it is cheating to cite a pictogram in a post like this. Pictograms are the lowest form of graph and are so often incorrect, that finding a bad one is easier than finding a good one. In the graph below of fatalities it is difficult to work out what one little person represents.

What does one little person represent?

A quick glance, ignoring the numbers, suggests that the road toll in 2014 is just over half what it was in 2012. However, the truth, calculated from the numbers, is that the relative size is 80%. 2012 has 12 people icons, representing 280 fatalities. One icon is removed for 2013, representing a drop of 9 fatalities. 2011 has one icon fewer again, representing a drop of 2 fatalities. There is so much wrong in the reporting of road fatalities, that I will stop here. Perhaps another day…

## Graph Three: Mysterious display on Household income

And here is the other graph that perplexed me for some time. It came in the Saturday morning magazine from our newspaper, as part of an article about inequality in New Zealand. Anyone who reads my blog will be aware that my politics place me well left of centre, and I find inequality one of the great ills of the modern day. So I was keen to see what this graph would tell me. And the answer is…

See how long it takes for you to find where you appear on the graph. (Pretending you live in NZ)

I have no idea. Now, I have expertise in the promulgation of statistics, and this graph stumped me for some time. Take a good look now, before I carry on.

Graphs are the main way that statistical analysts communicate with the outside world. Graphs like these ones do us no favours, even if they are not our fault. We need to do better, and make sure that all students learn about graphs.

## Teaching suggestion – a graph a day

Here is a suggestion for teachers at all levels. Have a “graph a day” display – maybe for a month? Students can contribute graphs from the news media. Each day discuss what the graph is saying, and critique the way the graph is communicating. I have a helpful structure for reading graphs in my post: There’s more to reading graphs than meets the eye;

Here is a summary of what I’ve said and what else I could say on the topic.

• The choice of graph depends on the purpose
• The text should state the purpose of the graph
• There is not a graph for everything you wish to communicate
• Sometimes a table communicates better than a graph
• Graphs are part of the analysis as well as part of the reporting. But some graphs are better to stay hidden.
• If it takes more than a few seconds to work out what a graph is communicating it should either be dumped or have an explanation in the text
• Truth (or communication) is more important than beauty
• There is beauty in simplicity
• Be aware than many people are colour-blind, or cannot easily differentiate between different shades.

## Feedback from previous post on which graph to use

Late last year I posted four graphs of the same data and asked for people’s opinions. You can link back to the post here and see the responses: Which Graph to Use.

The interesting thing is not which graph was selected as the most popular, but rather that each graph had a considerable number of votes. My response is that it depends.  It depends on the question you are answering or the message you are sending. But yes – I agree with the crowd that Graph A is the one that best communicates the various pieces of information. I think it would be improved by ordering the categories differently. It is not very pretty, but it communicates.

I recently posted a new video on YouTube about graphs. It is a quick once-over of important types of graphs, and can help to clarify what they are about. There are examples of good graphs in there.

I have written about graphs previously and you can find them here on the Collected Works page.

I’m interested in your thoughts. And I’d love to see some beautiful and truthful graphs in the comments.

# STEM, STEM-Ed, STEAM and Statistics

STEM is a popular acronym in educational circles and is used to refer to careers and educational tasks. Though most know that the four letters stand for Science, Technology, Engineering and Mathematics, it can be difficult to pin down what exactly it means. In this post I suggest that there are two related uses of STEM as a description.

# STEM

The term, STEM, originated in the USA in the late 1990s to describe specific careers and education for these careers. There seems to be no universally agreed-upon definition of STEM. From a careers perspective, the focus is on making sure that there are enough skilled workers in the STEM areas for future development. A common (engineering?) analogy is that of a pipeline. Industrialised nations need people with STEM skills, so need to ensure there are enough people entering and staying in the pipeline in order to fill future demand. There are also identified equity issues, as STEM jobs tend to be higher-paying, and also tend to be dominated by white males. A consequence of the higher demand and pay for people with STEM skills and qualifications is that there is often a shortage of teachers in STEM subjects.

This graphic illustrates the analogy of a STEM pipeline. Are people the drips?

# STEM Education

There are multiple ways of viewing STEM Education. One category is specific STEM Projects which I refer to as STEM-Ed, and another is education in STEM subjects, as they currently exist in the school curriculum.

## STEM-Ed

It is believed that one way to encourage children and young people to continue in STEM subjects, is to embed STEM into the curriculum. There has been a move towards specific STEM-based lessons or projects, particularly at middle-school or older primary level. Pinterest is full of attractive STEM-Ed lessons based around engineering and the design process. These include tower and bridge building, making boats to carry certain weights, creating a mechanism that will protect an egg from a fall or launching projectiles a maximum or specified distance. STEM-Ed lessons use a wide range of materials, including Lego bricks, spaghetti, marshmallows, masking tape, newspaper, recycled materials – just about anything you can think of. The makerspace movement ties in with STEM-Ed.

A good STEM-Ed project is described by Anne Jolly in her post Perfect STEM lessons. Anne Jolly suggests that a “perfect” STEM lesson uses an engineering approach as a framework, applies maths and science content through authentic experiences, deals with real world issues, involves hands-on and open-ended exploration with multiple right answers for students working in teams with the teacher in a facilitator role. A STEM project should also engage students in communicating, remove the fear of failure, appeal equally to boys and girls and promote authentic assessment.

It seems that when primary/elementary school teachers talk about STEM, it is usually STEM-Ed they are referring to. Certainly material under the STEM label on Pinterest, a popular source of inspiration for teachers, tends to be STEM-Ed.

A screenshot of some STEM tasks found on Pinterest

## Education in STEM subjects

In order to encourage and enable students to continue on to STEM careers, they must study the individual subjects that make up STEM. At school level, maths, physics and chemistry are often the areas where students make decisions that limit their later opportunities in STEM areas. (Where they leak out of the pipeline?) This is where teachers of STEM subjects have a part to play. Tying their subjects to authentic, real world contexts and teaching using STEM-Ed projects can help engagement.

However, there is also a need to learn the mathematics that does not appear in a “good STEM lesson”. Current mathematics education thinking aims to enable children to become mathematicians, not just engineers. To quote Tracey Zager’s excellent book, “Becoming the math teacher you wish you’d had“, mathematicians take risks, make mistakes, are precise, rise to a challenge, ask questions, connect ideas, use intuition, reason, prove, and work together and alone. Mathematics curriculum overlaps well with STEM-Ed in the areas of measurement, geometry and statistics. Number skills are practised in context. However, to provide enough exposure to other areas of mathematics, specific STEM-Ed lessons would need to be carefully designed. I suspect that there are areas of the curriculum that are more effectively learned through other methods than STEM-Ed.

# STEAM

There is a push to add Arts to STEM, making it STEAM. The relevance of this addition depends on viewpoint. It does not seem relevant to include Arts when talking about high-shortage career paths. But at the same time, STEM jobs also require other skills, not the least being communication skills. There is a strong link between fine art and technology, through design. The inclusion of A in STEM also depends on the definition of Art. The term “Arts” can include painting, music, dance, literature, film, design and the even the humanities. Including these into STEM (as a career or subject description) seems a trifle incongruous and begs the question whether there is anything that is not included in STEAM. Physical education and foreign languages?

However, when we look at STEM-Ed, there is a rationale for the inclusion of art. Good design does have an artistic component, as is all too clear when we look at some communist-era architecture and much amateur web-design. And written and oral communication are well-developed in many STEM-Ed projects.

# Statistics

Statistics clearly has a place in both STEM and STEM-Ed. There is a demand for statisticians, and people who can use statistics in what they do. The study of the discipline of statistics gives important insights into the nature of variability in our world. STEM-Ed projects could involve collecting and analysing data in a non-trivial way, though I have not seen evidence of this. The barrier to this is the statistical understanding of the teachers creating the STEM-Ed tasks, and points to an area where statistics educators need to be involved. Another barrier can be the time taken to collect an adequate sample, clean the data and analyse it. This is why specific tasks need to be designed for this.

# Concerns about STEM and STEM-Ed

We do need to think about the focus on STEM and wonder about the philosophical underpinnings. Are we educating our students to provide workers for the industrial machine? Is this the right thing to be doing? I found a very interesting book: Philosophy of STEM Education: A Critical Investigation by Nataly Z. Chesky and Mark R. Wolfmeyer . They ask these important questions.

STEM-Ed also needs to be approached carefully. Dayle Anderson, a lecturer in science education emphasised at a Primary Maths Symposium that teachers need to keep their eye on the learning. When a project is engaging it can be seductive to think that the learning is taken care of. There are so many demands on time in a school, that STEM-Ed lessons need to be well-designed for specific learning.

# STEM, STEM-Ed and 21st Century Skills

I am quite taken with the 4 Cs of 21st Century Skills, which have been defined as Critical thinking and problem solving, Communicating, Collaborating and Creativity and innovation. These correspond well to the five Key Competencies in the New Zealand curriculum – thinking, using language, symbols, and texts, managing self, relating to others and participating and contributing. These skills are needed by people who work in STEM jobs. They need to be able to communicate and work with others.

These 21st Century skills can be developed in STEM-Ed lessons, as students are required to work together, solve problems, think, innovate and communicate their results.

# Closing thoughts

Overall I am excited about STEM and STEM-Ed. A knowledge of how the world around us works is empowering to all, whether or not they join the STEM pipeline. Making mathematics, statistics and other related subjects more relevant and desirable is always going to be a good thing. Statistics educators need to be involved make sure that statistics is a vital part of STEM-Ed.

Any suggestions on how this is best achieved? And if you are interested in STEM-Ed, please Like and Follow our Facebook page to keep up with the discussion and find out about our contribution.

# I’ve been thinking lately….

Sometimes it pays to stop and think. I have been reading a recent textbook for mathematics teachers, Dianne Siemon et al, Teaching mathematics: foundations to middle years (2011). On page 47 the authors asked me to “Take a few minutes to write down your own views about the nature of mathematics, mathematics learning and mathematics teaching.” And bearing in mind I see statistics as related to, but not enclosed by mathematics, I decided to do the same for statistics as well. So here are my thoughts:

# The nature of mathematics

Mathematicians love the elegance of mathematics

Mathematics is a way of modelling and making sense of the world. Mathematics underpins scientific and commercial endeavours as well as everyday life. Mathematics is about patterns and proofs and problem structuring and solution finding. I used to think it was all about the answer, but now I think it is more about the process. I used to think that maths was predominantly an individual endeavour, but now I can see how there is a social or community aspect as well. I fear that too often students are getting a parsimonious view of mathematics, thinking it is only about numbers, and something they have to do on their own. I find my understanding of the nature of mathematics is rapidly changing as I participate in mathematics education at different ages and stages. I have also been influenced by the work of Jo Boaler.

# To learn mathematics

My original idea of mathematics learning comes from my own successful experience of copying down notes from the board, listening to the teacher and doing the exercises in the textbook. I was not particularly fluent with my times-tables, but loved problem-solving. If I got something wrong, I was happy to try again until I nutted it out. Sometimes I even did recreational maths, like the time I enumerated all possible dice combinations in Risk to find out who had the advantage – attacker or defender. I always knew that it took practice to be good at mathematics. However I never really thought of mathematics as a social endeavour. I feel I missed out, now. From time to time I do have mathematical discussions with my colleague. It was an adventure inventing Rogo and then working out a solution method. Mathematics can be a social activity.

# To teach mathematics

When I became a maths teacher I perpetuated the method that had worked for me, as I had not been challenged to think differently. I did like the ideas of mastery learning and personalised system of instruction. This meant that learners progressed to the next step only when they had mastered the previous one. I was a successful enough teacher and enjoyed my work.

Then as a university lecturer I had to work differently, and experimented. I had a popular personalised system of instruction quantitative methods course, relying totally on students working individually, at their own pace. I am happy that many of my students were successful in an area they had previously thought out of their reach. For some students it was the only subject they passed.

# What I would do now

If I were to teach mathematics at school level again, I hope I would do things differently. I love the idea of “Number talks” and rich tasks which get students to think about different ways of doing things. I had often felt sad that there did not seem to be much opportunity to have discussions in maths, as things were either right or wrong. Now I see what fun we could have with open-ended tasks. Maths learning should be communal and loud and exciting, not solitary, quiet and routine. I have been largely constructivist in my teaching philosophy, but now I would like to try out social constructivist thinking.

# Statistics

And what about statistics? At school in the 1970s I never learned more than the summary statistics and basic probability. At uni level it was bewildering, but I managed to get an A grade in a first year paper without understanding any of the basic principles. It wasn’t until I was doing my honours year in Operations Research and was working as a tutor in Statistical methods that things stared to come together – but even then I was not at home with statistical ideas and was happy to leave them behind when I graduated.

# The nature of statistics

Statistics lives in the real world

My views now on the nature of statistics are quite different. I believe statistical thinking is related to mathematical thinking, but with less certainty and more mess. Statistics is about models of reality, based on imperfect and incomplete data. Much of statistics is a “best guess” backed up by probability theory. And statistics is SO important to empowered citizenship. There are wonderful opportunities for discussion in statistics classes. I had a fun experience recently with a bunch of Year 13 Scholarship students in the Waikato. We had collected data from the students, having asked them to interpret a bar chart and a pie chart. There were some outliers in the data and I got them to suggest what we should do about them. There were several good suggestions and I let them discuss for a while then moved on. One asked me what the answer was and I said I really couldn’t say – any one of their suggestions was valid. It was a good teaching and learning moment. Statistics is full of multiple good answers, and often no single, clearly correct, answer.

# Learning statistics

My popular Quantitative Methods for Business course was developed on the premise that learning statistics requires repeated exposure to similar analyses of multiple contexts. In the final module, students did many, many hypothesis tests, in the hope that it would gradually fall into place. That is what worked for me, and it did seem to work for many of the students. I think that is not a particularly bad way to learn statistics. But there are possibly better ways.

I do like experiential learning, and statistics is perfect for real life experiences. Perhaps the ideal way to learn statistics is by performing an investigation from start to finish, guided by a knowledgeable tutor. I say perhaps, because I have reservations about whether that is effective use of time. I wrote a blog post previously, suggesting that students need exposure to multiple examples in order to know what in the study is universal and what applies only to that particular context. So perhaps that is why students at school should be doing an investigation each year within a different context.

# The nature of understanding

This does beg the question of what it means to learn or to understand anything. I hesitate to claim full understanding. Of anything. Understanding is progressive and multi-faceted and functional. As we use a technique we understand it more, such as hypothesis testing or linear programming. Understanding is progressive. My favourite quote about understanding is from Moore and Cobb, that “Mathematical understanding is not the only understanding.” I do not understand the normal distribution because I can read the Gaussian formula. I understand it from using it, and in a different way from a person who can derive it. In this way my understanding is functional. I have no need to be able to derive the Gaussian function for what I do, and the nature and level of my understanding of the normal distribution, or multiple regression, or bootstrapping is sufficient for me, for now.

# Teaching statistics

I believe our StatsLC videos do help students to understand and learn statistics. I have put a lot of work into those explanations, and have received overwhelmingly positive feedback about the videos. However, that is no guarantee, as Khan Academy videos get almost sycophantic praise and I know that there are plenty of examples of poor pedagogy and even error in them. I have recently been reading from “Make it Stick”, which summarises theory based on experimental research on how people learn for recall and retention. I was delighted to find that the method we had happened upon in our little online quizzes was promoted as an effective method of reinforcing learning.

This has been an enlightening exercise, and I recommend it to anyone teaching in mathematics or statistics. Read the first few chapters of a contemporary text on how to teach mathematics. Dianne Siemon et al, Teaching mathematics: foundations to middle years (2011) did it for me. Then “take a few minutes to write down your own views about the nature of mathematics, mathematics learning and mathematics teaching.” To which I add my own suggestion to think about the nature of statistics or operations research. Who knows what you will find out. Maybe you could put a few of your ideas down in the comments.

# The normal distribution – three tricky bits

There are several tricky things about teaching and understanding the normal distribution, and in this post I’m going to talk about three of them. They are the idea of a model, the limitations of the normal distribution, and the idea of the probability being the area under the graph.

# It’s a model!

When people hear the term distribution, they tend to think of the normal distribution. It is an appealing idea, and remarkably versatile. The normal distribution is an appropriate model for the outcome of many natural, manufacturing and human endeavours. However, it is only a model, not a rule. But sometimes the way we talk about things as “being normally distributed” can encourage incorrect thinking.

This problem can be seen in exam questions about the application of the normal distribution. They imply that the normal distribution controls the universe.

Here is are examples of question starters taken from a textbook:

1. “The time it takes Steve to walk to school follows a normal distribution with mean 30 minutes…”.
2. Or “The time to failure for a new component is normally distributed with a mean of…”

This terminology is too prescriptive. There is no rule that says that Steve has to time his walks to school to fit a certain distribution. Nor does a machine create components that purposefully follow a normal distribution with regard to failure time. I remember, as a student being intrigued by this idea, not really understanding the concept of a model.

When we are teaching, and at other times, it is preferable to say that things are appropriately modelled by a normal distribution. This reminds students that the normal distribution is a model. The above examples could be rewritten as

1. “The time it takes Steve to walk to school is appropriately modelled using a normal distribution with mean 30 minutes…”.
2. And  “The time to failure for a new component is found to have a distribution well modelled by the normal, with a mean of…”

They may seem a little clumsy, but send the important message that the normal distribution is the approximation of a random process, not the other way around.

# Not everything is normal

It is also important that students do not get the idea that all distributions, or even all continuous distributions are normal. The uniform distribution and negative exponential distributions are both useful in different circumstances, and look nothing like the normal distribution. And distributions of real entities can often have many zero values, that make a distribution far from normal-looking.

The normal distribution is great for things that measure mostly around a central value, and there are increasingly fewer things as you get further from the mean in both directions. I suspect most people can understand that in many areas of life you get lots of “average” people or things, and some really good and some really bad. (Except at Lake Wobegon “where all the women are strong, all the men are good looking, and all the children are above average.”)

However the normal distribution is not useful for modelling distributions that are heavily skewed. For instance, house prices tend to have a very long tail to the right, as there are some outrageously expensive houses, even several times the value of the median. At the same time there is a clear lower bound at zero, or somewhere above it.

Inter-arrival times are not well modelled by the normal distribution, but are well modelled by a negative exponential distribution. If we want to model how long it is likely to be before the next customer arrives, we would not expect there to be as many long times as there are short times, but fewer and fewer arrivals will occur with longer gaps.

Daily rainfall is not well modelled by the normal distribution as there will be many days of zero rainfall. Amount claimed in medical insurance or any kind of insurance are not going to be well modelled by the normal distribution as there are zero claims, and also the effect of excesses. Guest stay lengths at a hotel would not be well modelled by the normal distribution. Most guests will stay one or two days, and the longer the time, the fewer people would stay that long.

# Area under the graph – idea of sand

The idea of the area under the graph being the probability of an outcome’s happening in that range is conceptually challenging. I was recently introduced to the sand metaphor by Holly-Lynne  and Todd Lee. If you think about each outcome as being a grain of sand (or a pixel in a picture) then you think about how likely it is to occur, by the size of the area that encloses it. I found the metaphor very appealing, and you can read the whole paper here:

Visual representations of empirical probability distributions when using the granular density metaphor

There are other aspects of the normal distribution that can be challenging. Here is our latest video to help you to teach and learn and understand the normal distribution.

# Understanding Statistical Inference

Inference is THE big idea of statistics. This is where people come unstuck. Most people can accept the use of summary descriptive statistics and graphs. They can understand why data is needed. They can see that the way a sample is taken may affect how things turn out. They often understand the need for control groups. Most statistical concepts or ideas are readily explainable. But inference is a tricky, tricky idea. Well actually – it doesn’t need to be tricky, but the way it is generally taught makes it tricky.

## Procedural competence with zero understanding

I cast my mind back to my first encounter with confidence intervals and hypothesis tests. I learned how to calculate them (by hand  – yes I am that old) but had not a clue what their point was. Not a single clue. I got an A in that course. This is a common occurrence. It is possible to remain blissfully unaware of what inference is all about, while answering procedural questions in exams correctly.

But, thanks to the research and thinking of a lot of really smart and dedicated statistics teachers, we are able put a stop to that. And we must.

We need to explicitly teach what statistical inference is. Students do not learn to understand inference by doing calculations. We need to revisit the ideas behind inference frequently. The process of hypothesis testing, is counter-intuitive and so confusing that it spills its confusion over into the concept of inference. Confidence intervals are less confusing so a better intermediate point for understanding statistical inference. But we need to start with the concept of inference.

# What is statistical inference?

The idea of inference is actually not that tricky if you unbundle the concept from the application or process.

The concept of statistical inference is this –

We want to know stuff about a large group of people or things (a population). We can’t ask or test them all so we take a sample. We use what we find out from the sample to draw conclusions about the population.

That is it. Now was that so hard?

# Developing understanding of statistical inference in children

I have found the paper by Makar and Rubin, presenting a “framework for thinking about informal statistical inference”, particularly helpful. In this paper they summarise studies done with children learning about inference. They suggest that “ three key principles … appeared to be essential to informal statistical inference: (1) generalization, including predictions, parameter estimates, and conclusions, that extend beyond describing the given data; (2) the use of data as evidence for those generalizations; and (3) employment of probabilistic language in describing the generalization, including informal reference to levels of certainty about the conclusions drawn.” This can be summed up as Generalisation, Data as evidence, and Probabilistic Language.

We can lead into informal inference early on in the school curriculum. The key Ideas in the NZ curriculum suggest that “ teachers should be encouraging students to read beyond the data. Eg ‘If a new student joined our class, how many children do you think would be in their family?’” In other words, though we don’t specifically use the terms population and sample, we can conversationally draw attention to what we learn from this set of data, and how that might relate to other sets of data.

When teaching adults we may use a more direct approach, explaining explicitly, alongside experiential learning to understanding inference. We have just completed made a video: Understanding Inference. Within the video we have presented three basic ideas condensed from the Five Big Ideas in the very helpful book published by NCTM, “Developing Essential Understanding of Statistics, Grades 9 -12”  by Peck, Gould and Miller and Zbiek.

## Ideas underlying inference

• A sample is likely to be a good representation of the population.
• There is an element of uncertainty as to how well the sample represents the population
• The way the sample is taken matters.

These ideas help to provide a rationale for thinking about inference, and allow students to justify what has often been assumed or taught mathematically. In addition several memorable examples involving apples, chocolate bars and opinion polls are provided. This is available for free use on YouTube. If you wish to have access to more of our videos than are available there, do email me at n.petty@statslc.com.

We are currently developing exciting innovative materials to help students at all levels of the curriculum to understand and enjoy statistical analysis. We would REALLY appreciate it if any readers here today would help us out by answering this survey about fast food and dessert. It will take 10 minutes at a maximum. We don’t mind what country you are from, and will do the currency conversions.  And in a few months I will let you know how we got on. and we would love you to forward it to your friends and students to fill it out also – the more the merrier! It is an example of a well-designed questionnaire, with a meaningful purpose.

# Summarising with Box and Whisker plots

In the Northern Hemisphere, it is the start of the school year, and thousands of eager students are beginning their study of statistics. I know this because this is the time of year when lots of people watch my video, Types of Data. On 23rd August the hits on the video bounced up out of their holiday slumber, just as they do every year. They gradually dwindle away until the end of January when they have a second jump in popularity, I suspect at the start of the second semester.

One of the first topics in many statistics courses is summary statistics. The greatest hits of summary statistics tend to be the mean and the standard deviation. I’ve written previously about what a difficult concept a mean is, and then another post about why the median is often preferable to the mean. In that one I promised a video. Over two years ago – oops. But we have now put these ideas into a video on summary statistics. Enjoy! In 5 minutes you can get a conceptual explanation on summary measures of position. (Also known as location or central tendency)

I was going to follow up with a video on spread and started to think about range, Interquartile range, mean absolute deviation, variance and standard deviation. So I decided instead to make a video on the wonderful boxplot, again comparing the shoe- owning habits of male and female students in a university in New Zealand.

Boxplots are great. When you combine them with dotplots as done in iNZIght and various other packages, they provide a wonderful way to get an overview of the distribution of a sample. More importantly, they provide a wonderful way to compare two samples or two groups within a sample. A distribution on its own has little meaning.

John Tukey was the first to make a box and whisker plot out of the 5-number summary way back in 1969. This was not long before I went to High School, so I never really heard about them until many years later. Drawing them by hand is less tedious than drawing a dotplot by hand, but still time consuming. We are SO lucky to have computers to make it possible to create graphs at the click of a mouse.

Sample distributions and summaries are not enormously interesting on their own, so I would suggest introducing boxplots as a way to compare two samples. Their worth then is apparent.

A colleague recently pointed out an interesting confusion and distinction. The interquartile range is the distance between the upper quartile and the lower quartile. The box in the box plot contains the middle 50% of the values in the sample. It is tempting for people to point this out and miss the point that the interquartile range is a good resistant measure of spread for the WHOLE sample. (Resistant means that it is not unduly affected by extreme values.) The range is a poor summary statistic as it is so easily affected by extreme values.

And now we come to our latest video, about the boxplot. This one is four and a half minutes long, and also uses the shoe sample as an example. I hope you and your students find it helpful. We have produced over 40 statistics videos, some of which are available for free on YouTube. If you are interested in using our videos in your teaching, do let us know and we will arrange access to the remainder of them.

# Don’t teach significance testing – Guest post

The following is a guest post by Tony Hak of Rotterdam School of Management. I know Tony would love some discussion about it in the comments. I remain undecided either way, so would like to hear arguments.

# GOOD REASONS FOR NOT TEACHING SIGNIFICANCE TESTING

It is now well understood that p-values are not informative and are not replicable. Soon null hypothesis significance testing (NHST) will be obsolete and will be replaced by the so-called “new” statistics (estimation and meta-analysis). This requires that undergraduate courses in statistics now already must teach estimation and meta-analysis as the preferred way to present and analyze empirical results. If not, then the statistical skills of the graduates from these courses will be outdated on the day these graduates leave school. But it is less evident whether or not NHST (though not preferred as an analytic tool) should still be taught. Because estimation is already routinely taught as a preparation for the teaching of NHST, the necessary reform in teaching will not require the addition of new elements in current programs but rather the removal of the current emphasis on NHST or the complete removal of the teaching of NHST from the curriculum. The current trend is to continue the teaching of NHST. In my view, however, teaching of NHST should be discontinued immediately because it is (1) ineffective and (2) dangerous, and (3) it serves no aim.

1. Ineffective: NHST is difficult to understand and it is very hard to teach it successfully

We know that even good researchers often do not appreciate the fact that NHST outcomes are subject to sampling variation and believe that a “significant” result obtained in one study almost guarantees a significant result in a replication, even one with a smaller sample size. Is it then surprising that also our students do not understand what NHST outcomes do tell us and what they do not tell us? In fact, statistics teachers know that the principles and procedures of NHST are not well understood by undergraduate students who have successfully passed their courses on NHST. Courses on NHST fail to achieve their self-stated objectives, assuming that these objectives include achieving a correct understanding of the aims, assumptions, and procedures of NHST as well as a proper interpretation of its outcomes. It is very hard indeed to find a comment on NHST in any student paper (an essay, a thesis) that is close to a correct characterization of NHST or its outcomes. There are many reasons for this failure, but obviously the most important one is that NHST a very complicated and counterintuitive procedure. It requires students and researchers to understand that a p-value is attached to an outcome (an estimate) based on its location in (or relative to) an imaginary distribution of sample outcomes around the null. Another reason, connected to their failure to understand what NHST is and does, is that students believe that NHST “corrects for chance” and hence they cannot cognitively accept that p-values themselves are subject to sampling variation (i.e. chance)

2. Dangerous: NHST thinking is addictive

One might argue that there is no harm in adding a p-value to an estimate in a research report and, hence, that there is no harm in teaching NHST, additionally to teaching estimation. However, the mixed experience with statistics reform in clinical and epidemiological research suggests that a more radical change is needed. Reports of clinical trials and of studies in clinical epidemiology now usually report estimates and confidence intervals, in addition to p-values. However, as Fidler et al. (2004) have shown, and contrary to what one would expect, authors continue to discuss their results in terms of significance. Fidler et al. therefore concluded that “editors can lead researchers to confidence intervals, but can’t make them think”. This suggests that a successful statistics reform requires a cognitive change that should be reflected in how results are interpreted in the Discussion sections of published reports.

The stickiness of dichotomous thinking can also be illustrated with the results of a more recent study of Coulson et al. (2010). They presented estimates and confidence intervals obtained in two studies to a group of researchers in psychology and medicine, and asked them to compare the results of the two studies and to interpret the difference between them. It appeared that a considerable proportion of these researchers, first, used the information about the confidence intervals to make a decision about the significance of the results (in one study) or the non-significance of the results (of the other study) and, then, drew the incorrect conclusion that the results of the two studies were in conflict. Note that no NHST information was provided and that participants were not asked in any way to “test” or to use dichotomous thinking. The results of this study suggest that NHST thinking can (and often will) be used by those who are familiar with it.

The fact that it appears to be very difficult for researchers to break the habit of thinking in terms of “testing” is, as with every addiction, a good reason for avoiding that future researchers come into contact with it in the first place and, if contact cannot be avoided, for providing them with robust resistance mechanisms. The implication for statistics teaching is that students should, first, learn estimation as the preferred way of presenting and analyzing research information and that they get introduced to NHST, if at all, only after estimation has become their routine statistical practice.

3. It serves no aim: Relevant information can be found in research reports anyway

Our experience that teaching of NHST fails its own aims consistently (because NHST is too difficult to understand) and the fact that NHST appears to be dangerous and addictive are two good reasons to immediately stop teaching NHST. But there is a seemingly strong argument for continuing to introduce students to NHST, namely that a new generation of graduates will not be able to read the (past and current) academic literature in which authors themselves routinely focus on the statistical significance of their results. It is suggested that someone who does not know NHST cannot correctly interpret outcomes of NHST practices. This argument has no value for the simple reason that it is assumed in the argument that NHST outcomes are relevant and should be interpreted. But the reason that we have the current discussion about teaching is the fact that NHST outcomes are at best uninformative (beyond the information already provided by estimation) and are at worst misleading or plain wrong. The point is all along that nothing is lost by just ignoring the information that is related to NHST in a research report and by focusing only on the information that is provided about the observed effect size and its confidence interval.

## Bibliography

Coulson, M., Healy, M., Fidler, F., & Cumming, G. (2010). Confidence Intervals Permit, But Do Not Guarantee, Better Inference than Statistical Significance Testing. Frontiers in Quantitative Psychology and Measurement, 20(1), 37-46.

Fidler, F., Thomason, N., Finch, S., & Leeman, J. (2004). Editors Can Lead Researchers to Confidence Intervals, But Can’t Make Them Think. Statistical Reform Lessons from Medicine. Psychological Science, 15(2): 119-126.

This text is a condensed version of the paper “After Statistics Reform: Should We Still Teach Significance Testing?” published in the Proceedings of ICOTS9.