Has the Numeracy Project failed?

The Numeracy Development Project has influenced the teaching of mathematics in New Zealand. It has changed the language people use to talk about mathematical understanding, introducing the terms “multiplicative thinking”, “part-whole” and “proportional reasoning” to the teacher toolkit. It has empowered some teachers to think differently about the teaching of mathematics. It has brought “number” front and centre, often crowding out algebra, geometry, measurement and statistics, which are now commonly called the strands. It has baffled a large number of parents. Has the Numeracy Development Project been a success? If not, how can we fix it?

I have been pondering about the efficacy and side-effects of the Numeracy Project in New Zealand. I have heard criticisms from Primary and Secondary teachers, and defense and explanation from advisors. I have listened to a very illuminating podcast from one of the originators of the Numeracy Project, Ian Stevens, I have had discussions with another educational developer who was there at the beginning. I even downloaded some of the “pink booklets” and began reading them, in order understand the Numeracy Project.

Then I read this article from the US organisation, National Council of Teachers of Mathematics, Strategies are not Algorithms,  and it all started to fall into place.
The authors explain that researchers analysed the way that children learn about mathematics, and the stages they generally go through. It was found that “Students who used invented strategies before they learned standard algorithms demonstrated better knowledge of base-ten number concepts and were more successful in extending their knowledge to new situations than were students who initially learned standard algorithms.” They claim that in the US “(t)he idea of “invented strategies” has been distorted to such a degree that strategies are being treated like algorithms in many textbooks and classrooms across the country.” I suspect this statement also applies in New Zealand.

Strategies taught as algorithms

Whitacre and Wessenberg refer to a paper by Carpenter et al, A Longitudinal Study of Invention and Understanding in Children’s Multidigit Addition and Subtraction. I was able to get access to read it, and found the following:
“Although we have no data regarding explicit instruction on specific invented strategies, we hypothesize that direct instruction could change the quality of children’s understanding and use of invented strategies. If these strategies were the object of direct instruction, there would be a danger that children would learn them as rote procedures in much the way that they learn standard algorithms today.” (Emphasis added)

Were they right? Are the strategies being taught as rote procedures in some New Zealand classrooms? Do we need to change the way we talk about them?

How I see the Numeracy Development Project (NDP)

The NDP started as a way to improve teacher pedagogical content knowledge to improve outcomes for students. It was intended to cover all aspects of the New Zealand Mathematics and Statistics curriculum, not just number. Ian Stevens explained: “Numeracy was never just Number. We decided that in New Zealand numeracy meant mathematics and mathematics meant numeracy.”

The Numeracy Development Project provided a model to understand progression of understanding in learning mathematics. George Box once said “All models are wrong and some models are useful.” A model of progression of understanding is useful for identifying where we are, and how to progress to where we would like to be, rather like a map. But a map is not the landscape, and children differ, circumstances change, and models in education change faster than most. I recently attended a talk by Shelley Dole, who (I think) suggested that by emphasising additive thinking in the early school years, we may undo the multiplicative and proportional thinking the students had already. If all they see is adding and subtracting, any implication towards multiplicative and proportional thinking is stifled. It is an interesting premise.
The Numeracy Project (as it is now commonly called) suggested teaching methods, strongly based around group-work and minimising the use of worksheets. Popular invented strategies for arithmetic operations were described, and the teaching of standard algorithms such as vertical alignment of numbers when adding and subtracting was de-emphasised.
An unintended outcome is that the Numeracy Project has replaced the NZ curriculum in some schools, with “Number” taking centre stage for many years. Teachers are teaching invented strategies as algorithms rather than letting students work them out for themselves. At times students are required to know all the strategies before moving on. Textbooks, worksheets and even videos based around the strategies abound, which seems anathema to the original idea.

Where now?

So where do we go from here?

To me empowerment of teachers is pivotal. Teachers need to understand and embrace the beauty of number theory, the practicality of measurement, the art and challenge of geometry, the detective possibilities in data and the power of algebra to model our world. When mathematics is seen as a way to view the world, and embedded in all our teaching, in the way literacy is, maybe then, we will see the changes we seek.

Enriching mathematics with statistics

Statistics enriches everything!

In many school systems in the world, subjects are taught separately. In primary school, children  learn reading and writing, maths and social studies at different times of the day. But more than that, many topics within subjects are also taught separately. In mathematics we often teach computational skills, geometry, measurement and statistics in separate topics throughout the school year. Textbooks tend to encourage this segmentation of the curriculum. This causes problems as students compartmentalise their learning.  They think that something learned in mathematics can’t possibly be used in Physics. They complain in mathematics if they are asked to write a sentence or a report, saying that it belongs in English.

I participated in an interesting discussion on Twitter recently about Stretch and Challenge. (Thanks #mathschat) My interpretation of “Stretch and challenge” is ways of getting students to extend their thinking beyond the original task so that they are learning more and feeling challenged. This reminds me a lot of the idea of “Low floor High Ceiling” that Jo Boaler talks about. We need tasks that are easy for students to get started on, but that do not limit students, particularly ones who have really caught onto the task and wish to keep going.

Fractions

As a statistics educator, I see applications of statistics and probability everywhere. At a workshop on proportional thinking we were each asked to represent three-quarters, having been told that our A5 piece of paper was “one”. When I saw the different representations used by the participants, I could see a graph as a great way to represent it. You could make a quick set of axes on a whiteboard, and get people to put crosses on which representation they used. The task of categorising all the representations reinforces the idea that there are many ways to show the same thing. It also gets students more aware of the different representations. Then the barchart/dotplot provides a reminder of the outcome of the task. Students who are excited about this idea could make up a little questionnaire to take home and get other family members to draw different fractions, and look at the representations, adding them to the graph back at school.

Measurement

Measurement is an area of the mathematics curriculum that is just begging to be combined with statistics. Just physically measuring an object leads to a variation in responses, which can be graphed. Getting each child to measure each object three times and take the middle value, should lead to a distribution of values with less spread. And then there is estimation. I love the example Dan Meyer uses in his Ted talk in 2010 of filling a tank with water. Students could be asked their estimate of the filling time, simply by guessing, and then use mathematical modelling to refine their estimate. Both values can be graphed and compared.

Area and Probability

Area calculations can be used nicely with probability. Children can invent games that involve tossing a coin onto a shape or shapes. The score depends on whether the coin lands within the shape, outside the shape or on a line. They can estimate what the score will be from 10 throws, simply by looking at the shape, then try it out with one lot of ten throws. Now do some area calculations. Students may have different ways of dealing with the overlap issue. Use the area calculations to improve their theoretical estimates of the probability of each outcome, and from there work out the expected value. Then do multiple trials of ten throws and see how you need to modify the model.  So much learning in one task!

Statistics obviously fits well in much topic work as well. The Olympics are looming, with all the interest and the flood of statistics they provide. Students can be given the fascinating question of which country does the best? There are so many ways to measure and to account for population. Drawing graphs gives an idea of spread and distribution.

There is so much you can do with statistics and other strands and other curriculum areas!  Statistics requires a context, and it is economical use of time if the context is something else you are teaching.

Can you tell me some ways you have incorporated statistics into other strands of mathematics or other subject areas?

Papamoa College statistics excursion to Hamilton Zoo

Pizza in the park

Pizza in the park

Last week I had a lovely experience. I visited the Hamilton Observatory and Zoo as part of a Statistics excursion with the Year 13 statistics class of Papamoa College.

The trip was organised to help students learn about where data comes from. I went along because I really love teachers and students, and it was an opportunity to experience innovation by a team of wonderful teachers.  The students travelled from Papamoa to Hamilton, stopping for pizza in Cambridge. When we got to the Hamilton Observatory, Dave welcomed us and gave an excellent talk about the stars and data. I found it fascinating to think how much data there is, and also the level of (in)accuracy of their measurements.  I then gave a short talk on the importance of statistics in terms of citizenship, and how the students can be successful in learning statistics. I talked about analysis of the Disney Princess movies and the Zika virus.

Turtle

My favourite animal of the day

The next morning we went over to the Hamilton Zoo for breakfast followed by a talk by Ken on the use of data in the Zoo. That too was fascinating, and got my brain whirring. Zoos these days are all about education and helping endangered species to survive. They have records of weights of all the animals over time, making for some very interesting data. Weights are used as an indication of health in the animals. Ken shared pictures of animals being weighed – including tricky keas and fantastically large rhinos. The Zoo also collects a wide range of other data, such as the visitor numbers, satisfaction surveys, quantity of waste and food consumption. We visited the food preparation area and heard how the diets are carefully worked out, and the food fed in such a way as to give the animals something to think about.

Papamoa stats class

Dr Nic and the teachers and students of Papamoa College give statistics two thumbs up!

Though most of my work these days is in the field of statistics education, a part of my heart still belongs to Operations Research. I saw so many ways in which OR could help with things such as diets, logistics etc. I’m not saying that they are doing anything wrong, but there is always room for improvement. Were I still teaching OR to graduate students I would be looking for a project with a zoo.

I am sure the students benefited from the experience of seeing first-hand the use of data in multiple contexts. I was glad to be able to meet with the students
and talk to many about the assignments they will be doing throughout the year. Each student has the opportunity to choose an application area for the multiple assessments. I was impressed with their level of motivation, which will lead to better learning outcomes.

Well done team at Papamoa!

 

A Statistics-centric curriculum

Calculus is the wrong summit of the pyramid.

“The mathematics curriculum that we have is based on a foundation of arithmetic and algebra. And everything we learn after that is building up towards one subject. And at top of that pyramid, it’s calculus. And I’m here to say that I think that that is the wrong summit of the pyramid … that the correct summit — that all of our students, every high school graduate should know — should be statistics: probability and statistics.”

Ted talk by Arthur Benjamin in February 2009. Watch it – it’s only 3 minutes long.

He’s right, you know.

And New Zealand would be the place to start. In New Zealand, the subject of statistics is the second most popular subject in our final year of schooling, with a cohort of 12,606. By comparison, the cohort for  English is 16,445, and calculus has a final year cohort of 8392, similar in size to Biology (9038), Chemistry (8183) and Physics (7533).

Some might argue that statistics is already the summit of our curriculum pyramid, but I would see it more as an overly large branch that threatens to unbalance the mathematics tree. I suspect many maths teachers would see it more as a parasite that threatens to suck the life out of their beloved calculus tree. The pyramid needs some reconstruction if we are really to have a statistics-centric curriculum. (Or the tree needs pruning and reshaping – I think I have too many metaphors!)

Statistics-centric curriculum

So, to use a popular phrase, what would a statistics-centric curriculum look like? And what would be the advantages and disadvantages of such a curriculum? I will deal with implementation issues later.

To start with, the base of the pyramid would look little different from the calculus-pinnacled pyramid. In the early years of schooling the emphasis would be on number skills (arithmetic), measurement and other practical and concrete aspects. There would also be a small but increased emphasis on data collection and uncertainty. This is in fact present in the NZ curriculum. Algebra would be introduced, but as a part of the curriculum, rather than the central idea. There would be much more data collection, and probability-based experimentation. Uncertainty would be embraced, rather than ignored.

In the early years of high school, probability and statistics would take a more central place in the curriculum, so that students develop important skills ready for their pinnacle course in the final two years. They would know about the statistical enquiry cycle, how to plan and collect data and write questionnaires.  They would perform their own experiments, preferably in tandem with other curriculum areas such as biology, food-tech or economics. They would understand randomness and modelling. They would be able to make critical comments about reports in the media . They would use computers to create graphs and perform analyses.

As they approach the summit, most students would focus on statistics, while those who were planning to pursue a career in engineering would also take calculus. In the final two years students would be ready to build their own probabilistic models to simulate real-world situations and solve problems. They would analyse real data and write coherent reports. They would truly understand the concept of inference, and why confidence intervals are needed, rather than calculating them by hand or deriving formulas.

There is always a trade-off. Here is my take on the skills developed in each of the curricula.

Calculus-centric curriculum

Statistics-centric curriculum

Logical thinking Communication
Abstract thinking Dealing with uncertainty and ambiguity
Problem-solving Probabilistic models
Modelling (mainly deterministic) Argumentation, deduction
Proof, induction Critical thinking
Plotting deterministic graphs from formulas Reading and creating tables and graphs from data

I actually think you also learn many of the calc-centric skills in the stats-centric curriculum, but I wanted to look even-handed.

Implementation issues

Benjamin suggests, with charming optimism, that the new focus would be “easy to implement and inexpensive.”  I have been a very interested observer in the implementation of the new statistics curriculum in New Zealand. It has not happened easily, being inexpensive has been costly, and there has been fallout. Teachers from other countries (of which there are many in mathematics teaching in NZ) have expressed amazement at how much the NZ teachers accept with only murmurs of complaint. We are a nation with a “can do” attitude, who, by virtue of small population and a one-tier government, can be very flexible. So long as we refrain from following the follies of our big siblings, the UK, US and Australia, NZ has managed to have a world-class education system. And when a new curriculum is implemented, though there is unrest and stress, there is seldom outright rebellion.

In my business, I get the joy of visiting many schools and talking with teachers of mathematics and statistics. I am fascinated by the difference between schools, which is very much a function of the head of mathematics and principal. Some have embraced the changes in focus, and are proactively developing pathways to help all students and teachers to succeed. Others are struggling to accept that statistics has a place in the mathematics curriculum, and put the teachers of statistics into a ghetto where they are punished with excessive marking demands.

The problem is that the curriculum change has been done “on the cheap”. As well as being small and nimble, NZ is not exactly rich. The curriculum change needed more advisors, more release time for teachers to develop and more computer power. These all cost. And then you have the problem of “me too” from other subjects who have had what they feel are similar changes.

And this is not really embracing a full stats-centric curriculum. Primary school teachers need training in probability and statistics if we are really to implement Benjamin’s idea fully. The cost here is much greater as there are so many more primary school teachers. It may well take a generation of students to go through the curriculum and enter back as teachers with an improved understanding.

Computers make it possible

Without computers the only statistical analysis that was possible in the classroom was trivial. Statistics was reduced to mechanistic and boring hand calculation of light-weight statistics and time-filling graph construction. With computers, graphs and analysis can be performed at the click of a mouse, making graphs a tool, rather than an endpoint. With computing power available real data can be used, and real problems can be addressed. High level thinking is needed to make sense and judgements and to avoid wrong conclusions.

Conversely, the computer has made much of calculus superfluous. With programs that can bash their way happily through millions of iterations of a heuristic algorithm, the need for analytic methods is seriously reduced. When even simple apps on an iPad can solve an algebraic equation, and Excel can use “What if” to find solutions, the need for algebra is also questionable.

Efficient citizens

In H.G. Wells’ popular but misquoted words, efficient citizenry calls for the ability to make sense of data. As the science fiction-writer that he was, he foresaw the masses of data that would be collected and available to the great unwashed. The levelling nature of the web has made everyone a potential statistician.

According to the engaging new site from the ASA, “This is statistics”, statisticians make a difference, have fun, satisfy curiosity and make money. And these days they don’t all need to be good at calculus.

Let’s start redesigning our pyramid.

Teaching Confidence Intervals

If you want your students to understand just two things about confidence intervals, what would they be?

What and what order

When making up a teaching plan for anything it is important to think about whom you are teaching, what it is you want them to learn, and what order will best achieve the most important desired outcomes. In my previous life as a university professor I mostly taught confidence intervals to business students, including MBAs. Currently I produce materials to help teach high school students. When teaching business students, I was aware that many of them had poor mathematics skills, and I did not wish that to get in the way of their understanding. High School students may well be more at home with formulas and calculations, but their understanding of the outside world is limited. Consequently the approaches for these two different students may differ.

Begin with the end in mind

I use the “all of the people, some of the time” principle when deciding on the approach to use in teaching a topic. Some of the students will understand most of the material, but most of the students will only really understand some of the material, at least the first time around. Statistics takes several attempts before you approach fluency. Generally the material students learn will be the material they get taught first, before they start to get lost. Therefore it is good to start with the important material. I wrote a post about this, suggesting starting at the very beginning is not always the best way to go. This is counter-intuitive to mathematics teachers who are often very logical and wish to take the students through from the beginning to the end.

At the start I asked this question – if you want your students to understand just two things about confidence intervals, what would they be?

To me the most important things to learn about confidence intervals are what they are and why they are needed. Learning about the formula is a long way down the list, especially in these days of computers.

The traditional approach to teaching confidence intervals

A traditional approach to teaching confidence intervals is to start with the concept of a sampling distribution, followed by calculating the confidence interval of a mean using the Z distribution. Then the t distribution is introduced. Many of the questions involve calculation by formula. Very little time is spent on what a confidence interval is and why we need them. This is the order used in many textbooks. The Khan Academy video that I reviewed in a previous post does just this.

A different approach to teaching confidence intervals

My approach is as follows:
Start with the idea of a sample and a population, and that we are using a sample to try to find out an unknown value from the population. Show our video about understanding a confidence interval. One comment on this video decried the lack of formulas. I’m not sure what formulas would satisfy the viewer, but as I was explaining what a confidence interval is, not how to get it, I had decided that formulas would not help.

The new New Zealand school curriculum follows a process to get to the use of formal confidence intervals. Previously the assessment was such that a student could pass the confidence interval section by putting values into formulas in a calculator. In the new approach, early high school students are given real data to play with, and are encouraged to suggest conclusions they might be able to draw about the population, based on the sample. Then in Year 12 they start to draw informal confidence intervals, based on the sample.
Then in Year 13, we introduce bootstrapping as an intuitively appealing way to calculate confidence intervals. Students use existing data to draw a conclusion about two medians.
In a more traditional course, you could instead use the normal-based formula for the confidence interval of a mean. We now have a video for that as well.

You could then examine the idea of the sampling distribution and the central limit theorem.

The point is that you start with getting an idea of what a confidence interval is, and then you find out how to find one, and then you start to find out the theory underpinning it. You can think of it as successive refinement. Sometimes when we see photos downloading onto a device, they start off blurry, and then gradually become clearer as we gain more information. This is a way to learn a complex idea, such as confidence intervals. We start with the big picture, and not much detail, and then gradually fill out the details of the how and how come of the calculations.

When do we teach the formulas?

Some teachers believe that the students need to know the formulas in order to understand what is going on. This is probably true for some students, but not all. There are many kinds of understanding, and I prefer a conceptual and graphical approaches. If formulas are introduced at the end of the topic, then the students who like formulas are satisfied, and the others are not alienated. Sometimes it is best to leave the vegetables until last! (This is not a comment on the students!)

For more ideas about teaching confidence intervals see other posts:
Good, bad and wrong videos about confidence intervals
Confidence Intervals: informal, traditional, bootstrap
Why teach resampling

Those who can, teach statistics

The phrase I despise more than any in popular use (and believe me there are many contenders) is “Those who can, do, and those who can’t, teach.” I like many of the sayings of George Bernard Shaw, but this one is dismissive, and ignorant and born of jealousy. To me, the ability to teach something is a step higher than being able to do it. The PhD, the highest qualification in academia, is a doctorate. The word “doctor” comes from the Latin word for teacher.

Teaching is a noble profession, on which all other noble professions rest. Teachers are generally motivated by altruism, and often go well beyond the requirements of their job-description to help students. Teachers are derided for their lack of importance, and the easiness of their job. Yet at the same time teachers are expected to undo the ills of society. Everyone “knows” what teachers should do better. Teachers are judged on their output, as if they were the only factor in the mix. Yet how many people really believe their success or failure is due only to the efforts of their teacher?

For some people, teaching comes naturally. But even then, there is the need for pedagogical content knowledge. Teaching is not a generic skill that transfers seamlessly between disciplines. You must be a thinker to be a good teacher. It is not enough to perpetuate the methods you were taught with. Reflection is a necessary part of developing as a teacher. I wrote in an earlier post, “You’re teaching it wrong”, about the process of reflection. Teachers need to know their material, and keep up-to-date with ways of teaching it. They need to be aware of ways that students will have difficulties. Teachers, by sharing ideas and research, can be part of a communal endeavour to increase both content knowledge and pedagogical content knowledge.

There is a difference between being an explainer and being a teacher. Sal Khan, maker of the Khan Academy videos, is a very good explainer. Consequently many students who view the videos are happy that elements of maths and physics that they couldn’t do, have been explained in such a way that they can solve homework problems. This is great. Explaining is an important element in teaching. My own videos aim to explain in such a way that students make sense of difficult concepts, though some videos also illustrate procedure.

Teaching is much more than explaining. Teaching includes awakening a desire to learn and providing the experiences that will help a student to learn.  In these days of ever-expanding knowledge, a content-driven approach to learning and teaching will not serve our citizens well in the long run. Students need to be empowered to seek learning, to criticize, to integrate their knowledge with their life experiences. Learning should be a transformative experience. For this to take place, the teachers need to employ a variety of learner-focussed approaches, as well as explaining.

It cracks me up, the way sugary cereals are advertised as “part of a healthy breakfast”. It isn’t exactly lying, but the healthy breakfast would do pretty well without the sugar-filled cereal. Explanations really are part of a good learning experience, but need to be complemented by discussion, participation, practice and critique.  Explanations are like porridge – healthy, but not a complete breakfast on their own.

Why statistics is so hard to teach

“I’m taking statistics in college next year, and I can’t wait!” said nobody ever!

Not many people actually want to study statistics. Fortunately many people have no choice but to study statistics, as they need it. How much nicer it would be to think that people were studying your subject because they wanted to, rather than because it is necessary for psychology/medicine/biology etc.

In New Zealand, with the changed school curriculum that gives greater focus to statistics, there is a possibility that one day students will be excited to study stats. I am impressed at the way so many teachers have embraced the changed curriculum, despite limited resources, and late changes to assessment specifications. In a few years as teachers become more familiar with and start to specialise in statistics, the change will really take hold, and the rest of the world will watch in awe.

In the meantime, though, let us look at why statistics is difficult to teach.

  1. Students generally take statistics out of necessity.
  2. Statistics is a mixture of quantitative and communication skills.
  3. It is not clear which are right and wrong answers.
  4. Statistical terminology is both vague and specific.
  5. It is difficult to get good resources, using real data in meaningful contexts.
  6. One of the basic procedures, hypothesis testing, is counter-intuitive.
  7. Because the teaching of statistics is comparatively recent, there is little developed pedagogical content knowledge. (Though this is growing)
  8. Technology is forever advancing, requiring regular updating of materials and teaching approaches.

On the other hand, statistics is also a fantastic subject to teach.

  1. Statistics is immediately applicable to life.
  2. It links in with interesting and diverse contexts, including subjects students themselves take.
  3. Studying statistics enables class discussion and debate.
  4. Statistics is necessary and does good.
  5. The study of data and chance can change the way people see the world.
  6. Technlogical advances have put the power for real statistical analysis into the hands of students.
  7. Because the teaching of statistics is new, individuals can make a difference in the way statistics is viewed and taught.

I love to teach. These days many of my students are scattered over the world, watching my videos (for free) on YouTube. It warms my heart when they thank me for making something clear, that had been confusing. I realise that my efforts are small compared to what their teacher is doing, but it is great to be a part of it.

Conceptualising Probability

The problem with probability is that it doesn’t really exist. Certainly it never exists in the past.

Probability is an invention we use to communicate our thoughts about how likely something is to happen. We have collectively agreed that 1 is a certain event and 0 is impossible. 0.5 means that there is just as much chance of something happening as not. We have some shared perception that 0.9 means that something is much more likely to happen than to not happen. Probability is also useful for when we want to do some calculations about something that isn’t certain. Often it is too hard to incorporate all uncertainty, so we assume certainty and put in some allowance for error.

Sometimes probability is used for things that happen over and over again, and in that case we feel we can check to see if our predication about how likely something is to happen was correct. The problem here is that we actually need things to happen a really big lot of times under the same circumstances in order to assess if we were correct. But when we are talking about the probability of a single event, that either will or won’t happen, we can’t test out if we were right or not afterwards, because by that time it either did or didn’t happen. The probability no longer exists.

Thus to say that there is a “true” probability somewhere in existence is rather contrived. The truth is that it either will happen or it won’t. The only way to know a true probability would be if this one event were to happen over and over and over, in the wonderful fiction of parallel universes. We could then count how many times it would turn out one way rather than another. At which point the universes would diverge!

However, for the interests of teaching about probability, there is the construct that there exists a “true probability” that something will happen.

Why think about probability?

What prompted these musings about probability was exploring the new NZ curriculum and companion documents, the Senior Secondary Guide and nzmaths.co.nz.

In Level 8 (last year of secondary school) of the senior secondary guide it says, “Selects and uses an appropriate distribution to solve a problem, demonstrating understanding of the relationship between true probability (unknown and unique to the situation), model estimates (theoretical probability) and experimental estimates.”

And at NZC level 3 (years 5 and 6 at Primary school!) in the Key ideas in Probability it talks about “Good Model, No Model and Poor Model” This statement is referred to at all levels above level 3 as well.

I decided I needed to make sense of these two conceptual frameworks: true-model-experimental and good-poor-no, and tie it to my previous conceptual framework of classical-frequency-subjective.

Here goes!

Delicious Mandarins

Let’s make this a little more concrete with an example. We need a one-off event. What is the probability that the next mandarin I eat will be delicious? It is currently mandarin season in New Zealand, and there is nothing better than a good mandarin, with the desired combination of sweet and sour, and with plenty of juice and a good texture. But, being a natural product, there is a high level of variability in the quality of mandarins, especially when they may have parted company with the tree some time ago.

There are two possible outcomes for my future event. The mandarin will be delicious or it will not. I will decide when I eat it. Some may say that there is actually a continuum of deliciousness, but for now this is not the case. I have an internal idea of deliciousness and I will know. I think back to my previous experience with mandarins. I think about a quarter are horrible, a half are nice enough and about a quarter are delicious (using the Dr Nic scale of mandarin grading). If the mandarin I eat next belongs to the same population as the ones in my memory, then I can predict that there is a 25% probability that the mandarin will be delicious.

The NZ curriculum talks about “true” probability which implies that any value I give to the probability is only a model. It may be a model based on empirical or experimental evidence. It can be based on theoretical probabilities from vast amounts of evidence, which has given us the normal distribution. The value may be only a number dredged up from my soul, which expresses the inner feeling of how likely it is that the mandarin will be delicious, based on several decades of experience in mandarin consumption.

More examples

Let us look at some more examples:

What is the probability that:

  • I will hear a bird on the way to work?
  • the flight home will be safe?
  • it will be raining when I get to Christchurch?
  • I will get a raisin in my first spoonful of muesli?
  • I will get at least one raisin in half of my spoonfuls of muesli?
  • the shower in my hotel room will be enjoyable?
  • I will get a rare Lego ® minifigure next time I buy one?

All of these events are probabilistic and have varying degrees of certainty and varying degrees of ease of modelling.

Easy to model Hard to model
Unlikely Get a rare Lego ® minifigure Raining in Christchurch
No idea Raisin in half my spoonfuls Enjoyable shower
Likely Raisin in first spoonful Bird, safe flight home

And as I construct this table I realise also that there are varying degrees of importance. Except for the flight home, none of those examples matter. I am hoping that a safe flight home has a probability extremely close to 1. I realise that there is a possibility of an incident. And it is difficult to model. But people have modelled air safety and the universal conclusion is that it is safer than driving. So I will take the probability and fly.

Conceptual Frameworks

How do we explain the different ways that probability has been described? I will now examine the three conceptual frameworks I introduced earlier, starting with the easiest.

Traditional categorisation

This is found in some form in many elementary college statistics text books. The traditional framework has three categories –classical or “a priori”, frequency or historical, and subjective.

Classical or “a priori” – I had thought of this as being “true” probability. To me, if there are three red and three white Lego® blocks in a bag and I take one out without looking, there is a 50% chance that I will get a red one. End of story. How could it be wrong? This definition is the mathematically interesting aspect of probability. It is elegant and has cool formulas and you can make up all sorts of fun examples using it. And it is the basis of gambling.

Frequency or historical – we draw on long term results of similar trials to gain information. For example we look at the rate of germination of a certain kind of seed by experiment, and that becomes a good approximation of the likelihood that any one future seed will germinate. And it also gives us a good estimate of what proportion of seeds in the future will germinate.

Subjective – We guess! We draw on our experience of previous similar events and we take a stab at it. This is not seen as a particularly good way to come up with a probability, but when we are talking about one off events, it is impossible to assess in retrospect how good the subjective probability estimate was. There is considerable research in the field of psychology about the human ability or lack thereof to attribute subjective probabilities to events.

In teaching the three part categorisation of sources of probability I had problems with the probability of rain. Where does that fit in the three categories? It uses previous experimental data to build a model, and current data to put into the model, and then a probability is produced. I decided that there is a fourth category, that I called “modelled”. But really that isn’t correct, as they are all models.

NZ curriculum terminology

So where does this all fit in the New Zealand curriculum pronouncements about probability? There are two conceptual frameworks that are used in the document, each with three categories as follows:

True, modelled, experimental

In this framework we start with the supposition that there exists somewhere in the universe a true probability distribution. We cannot know this. Our expressions of probability are only guesses at what this might be. There are two approaches we can take to estimate this “truth”. These two approaches are not independent of each other, but often intertwined.

One is a model estimate, based on theory, such as that the probability of a single outcome is the number of equally likely ways that it can occur over the number of possible outcomes. This accounts for the probability of a red brick as opposed to a white brick, drawn at random. Another example of a modelled estimate is the use of distributions such as the binomial or normal.

In addition there is the category of experimental estimate, in which we use data to draw conclusions about what it likely to happen. This is equivalent to the frequency or historical category above. Often modelled distributions use data from an experiment also. And experimental probability relies on models as well.  The main idea is that neither the modelled nor the experimental estimate of the “true” probability distribution is the true distribution, but rather a model of some sort.

Good model, poor model, no model

The other conceptual framework stated in the NZ curriculum is that of good model, poor model and no model, which relates to fitness for purpose. When it is important to have a “correct” estimate of a probability such as for building safety, gambling machines, and life insurance, then we would put effort into getting as good a model as possible. Conversely, sometimes little effort is required. Classical models are very good models, often of trivial examples such as dice games and coin tossing. Frequency models aka experimental models may or may not be good models, depending on how many observations are included, and how much the future is similar to the past. For example, a model of sales of slide rules developed before the invention of the pocket calculator will be a poor model for current sales. The ground rules have changed. And a model built on data from five observations of is unlikely to be a good model. A poor model is not fit for purpose and requires development, unless the stakes are so low that we don’t care, or the cost of better fitting is greater than the reward.

I have problems with the concept of “no model”. I presume that is the starting point, from which we develop a model or do not develop a model if it really doesn’t matter. In my examples above I include the probability that I will hear a bird on the way to work. This is not important, but rather an idle musing. I suspect I probably will hear a bird, so long as I walk and listen. But if it rains, I may not. As I am writing this in a hotel in an unfamiliar area I have no experience on which to draw. I think this comes pretty close to “no model”. I will take a guess and say the probability is 0.8. I’m pretty sure that I will hear a bird. Of course, now that I have said this, I will listen carefully, as I would feel vindicated if I hear a bird. But if I do not hear a bird, was my estimate of the probability wrong? No – I could assume that I just happened to be in the 0.2 area of my prediction. But coming back to the “no model” concept – there is now a model. I have allocated the probability of 0.8 to the likelihood of hearing a bird. This is a model. I don’t even know if it is a good model or a poor model. I will not be walking to work this way again, so I cannot even test it out for the future, and besides, my model was only for this one day, not for all days of walking to work.

So there you have it – my totally unscholarly musings on the different categorisations of probability.

What are the implications for teaching?

We need to try not to perpetuate the idea that probability is the truth. But at the same time we do not wish to make students think that probability is without merit. Probability is a very useful, and at times highly precise way of modelling and understanding the vagaries of the universe. The more teachers can use language that implies modelling rather than rules, the better. It is common, but not strictly correct to say, “This process follows a normal distribution”. As Einstein famously and enigmatically said, “God does not play dice”. Neither does God or nature use normal distribution values to determine the outcomes of natural processes. It is better to say, “this process is usefully modelled by the normal distribution.”

We can have learning experiences that help students to appreciate certainty and uncertainty and the modelling of probabilities that are not equi-probable. Thanks to the overuse of dice and coins, it is too common for people to assess things as having equal probabilities. And students need to use experiments.  First they need to appreciate that it can take a large number of observations before we can be happy that it is a “good” model. Secondly they need to use experiments to attempt to model an otherwise unknown probability distribution. What fun can be had in such a class!

But, oh mathematical ones, do not despair – the rules are still the same, it’s just the vigour with which we state them that has changed.

Comment away!

Post Script

In case anyone is interested, here are the outcomes which now have a probability of 1, as they have already occurred.

  • I will hear a bird on the way to work? Almost the minute I walked out the door!
  • the flight home will be safe? Inasmuch as I am in one piece, it was safe.
  • it will be raining when I get to Christchurch? No it wasn’t
  • I will get a raisin in my first spoonful of muesli? I did
  • I will get at least one raisin in half of my spoonfuls of muesli? I couldn’t be bothered counting.
  • the shower in my hotel room will be enjoyable? It was okay.
  • I will get a rare Lego minifigure next time I buy one? Still in the future!

The Knife-edge of Competence

I do my own video-editing using a very versatile and complex program called Adobe Premiere Pro. I have had no formal training, and get help by ringing my son, who taught me all I know and can usually rescue me with patient instructions over the phone. At times, especially in the early stages I have felt myself wobbling along the knife-edge of competence. All I needed was for something new to go wrong, or or click a button inadvertently and I would fall off the knife-edge and the whole project would disappear into a mass of binary. This was not without good reason. Premiere Pro wasn’t always stable on our computer, and at one point it took us several weeks to get our hard-drive replaced. (Apple “Time machine” saved me from despair). And sometimes I would forget to save regularly and a morning’s work was lost. (Even time-machine can’t help with that level of incompetence.)

But despite my severe limitations I have managed to edit over twenty videos that now receive due attention (and at times adulation!) on YouTube. It isn’t an easy feeling, to be teetering on the brink of disaster, real or imagined. But there was no alternative, and there is a sense of pride at having made it through with only a few scars and not too much inappropriate language.

There are some things at which I feel totally competent. I can speak to a crowd of any number of people and feel happy that they will be entertained, edified and perhaps even educated. I can analyse data using basic statistical methods. I can teach a person about inference. Performing these tasks is a joy, because I know I have the prerequisite skills and knowledge to cope with whatever happens. But on the way to getting to this point, I had to walk the knife-edge of competence.

Many teachers of statistics know too well this knife-edge. In New Zealand at present there are a large number of teachers of Year 13 statistics who are teaching about bootstrapping, when their own understanding of it is sketchy. They are teaching how to write statistical reports, when they have never written one themselves. They are assessing statements about statistics that they are not actually sure about. This is a knife-edge. They feel that any minute a student will ask them a question about the content that they cannot answer. These are not beginning teachers, but teachers with years and decades of experience in teaching mathematics and mathematical statistics. But the innovations of the curriculum have put them in an uncomfortable position. Inconsistent, tardy and even incorrect information from the qualification agency is not helping, but that is a story for another day.

In another arena there are professors and lecturers of statistics (in the antipodes we do not throw around the title “professor” with the abandon of our North American cousins) who are extremely competent at statistical mathematics and analysis but who struggle to teach in a satisfactory way. Their knife-edge concerns teaching, appropriate explanation and the generation of effective learning activities and assessments in the absence of any educational training. They fear that someone will realise one day that they don’t really know how to devise learning objectives, and provide fair assessments. I am hoping that this blog is going some way to helping these people to ask for help! Unfortunately the frequent response is avoidance behaviour, which is alarmingly supported by a system that rewards research publications rather than effective educational endeavours.

So what do you do when you are walking the knife-edge of competence?

You do the best you can.

And sometimes you fake it.

I am led to believe there is a gender-divide on this. Some people are better at hiding their incompetence than others, and just about all the people I know like that are men. I had a classmate in my honours year who was at a similar level of competence to me, but he applied for jobs I wouldn’t have contemplated. The fear of being shown up as a fake, or not knowing EXACTLY what to do at any point stopped me from venturing. He horrified me further a few years later when he set up his own company. Nearly three decades, two children and a PhD later I am not so fastidious or “nice” in the Jane Austen meaning of the word. If I think I can probably learn how to do something in time to make a reasonable fist of it and not cause actual harm, I’m likely to have a go. Hence taking my redundancy and running!

When I first lectured in statistics for management,  I did not know much beyond what I was teaching. I lived in fear that someone would ask me a question that I couldn’t answer and I would be revealed as the fake I was. Well you know, it never happened! I even taught students who were statistics majors, who did know more than I, and post-graduate students in psychology and heads of mathematics departments, and my fears were never realised. In fact the stats students told me that they finally understood the central limit theorem, thanks to my nifty little exercise using dotplots on minitab. (Which was how I had finally understood the central limit theorem – or at least the guts of it.)

I’m guessing that this is probably true for most of the mathematics teachers who are worrying. Despite their fear, they have not been challenged or called out.

The teachers’ other unease is the feeling that they are not giving the best service to their students, and the students will suffer, miss out on scholarships, decide not to get a higher education and live their lives on the street.  I may be exaggerating a little here, but certainly few of us like to give a service that is less than what we are accustomed to. We feel bad when we do something that feels substandard.

There are two things I learned in my twenty years of lecturing that may help here:

We don’t know how students perceive what we do. Every now and again I would come out of a lecture with sweat trickling down my spine because something had gone wrong. It might be that in the middle of an explanation I had had second thoughts about it, changed tack, then realised I was right in the first-place and ended up confusing myself. Or perhaps part way through a worked example it was pointed out to me that there was a numerical error in line three. To me these were bad, bad things to happen. They undermined my sense of competence. But you know, the students seldom even noticed. What felt like the worst lecture of my life, was in fact still just fine.

The other thing I learned is that we flatter ourselves when we think how much difference our knowledge may make.  Now don’t get me wrong here – teachers make an enormous difference. People who become teachers do so because we want to help people. We want to make a difference in students’ lives. We often have a sense of calling. There may be some teachers who do it because they don’t know what else to do with their degree, but I like to think that most of us teachers teach because to not teach is unthinkable. I despise, to the point of spitting as I talk, the expression “Those who can, do, and those who can’t, teach.” One day when the mood takes me I will write a whole post about the noble art of teaching and the fallacy of that dismissive statement. My next statement is so important I will give it a paragraph of its own.

A teacher who teaches from love, who truly cares about what happens to their students, even if they are struggling on the knife-edge of competence will not ruin their students’ lives through temporary incompetence in an aspect of the curriculum.

There are many ways that a teacher can have devastating effects on their students, but being, for a short time, on the knife-edge of competence, is not one of them.

Take heart, keep calm and carry on!

Is statistical enquiry a cycle?

What is the statistical enquiry cycle and why is it a cycle? Is it really a cycle?

The New Zealand curriculum for Mathematics and statistics was recently held up as an example of good practice with regard to statistics. Yay us! In New Zealand the learning of statistics starts at the beginning of schooling and is part of the curriculum right through the school years. Statistics is developed as a discipline alongside mathematics, rather than as a subset of it. There are mathematics teachers who view this as an aberration, and believe that when this particular fad is over statistics will go back where it belongs, tucked quietly behind measurement, algebra and arithmetic. But the statisticians rejoice that the rich and exciting world of real data and detective work is being opened up to early learners. The outcome for mathematics and statistics remains to be seen.

A quick look over the Australian curriculum shows ostensibly a similar emphasis with regard to content at most levels.  The big difference (at first perusal) is that the New Zealand curriculum has two strands of statistics – statistical investigation, and statistical literacy, whereas the Australian curriculum has the more mathematical approach of “Data representation and interpretation”.  Both include probability as another strand.

Data Detective Cycle

In the New Zealand curriculum, the statistical investigation strand at every level refers to the “Statistical enquiry cycle”, shown here, which is also known as the PPDAC cycle. This is a unifying theme and organising framework for teachers and learners.

The data detective poster

The data detective poster

This link takes you to a fuller explanation of the statistical enquiry cycle and its role at the different levels of the school curriculum. Note that the levels do not correspond to years. Click here to see the correspondence. The first five levels correspond to about 2 years each, whereas levels 6,7 and 8 correspond to the final three years of high school. So a child working on level 3 is generally aged about 10 or 11.

As I provide resources to support teaching and learning within the NZ curriculum I have become more aware of this framework, and have some questions and suggestions. I have made a table from which I hope to develop another diagram that students at higher levels can engage with, particularly with regard to the reporting aspects. As this is a work in progress you will have to wait!

Origins

Let’s look at the origins of the diagram and terminology. Maxine Pfannkuch (an educator) worked with Chris Wild (a statistician) to articulate what it is that statisticians do. They published their results in the international statistical review in 1999 and contributed the chapter “Towards an understanding of statistical thinking” in “The Challenge of Developing Statistical Literacy, Reasoning and Thinking”, edited by Dani Ben-Zvi and Joan Garfield. The statistical enquiry cycle has consequently been promulgated in the diagram and description referred to above. There is sound research behind this, and it makes good sense as a way of explaining what statisticians do.

Diagrams

I love diagrams. Anyone who has viewed my videos will know this. I spend a great deal of mental energy (usually while running) trying to work out ways to convey ideas in a visual way that will help people to learn, understand and remember. I also do NOT believe in the fad of learning styles, but rather I believe that all learners will gain from different presentations of concepts. I also believe that it is a useful discipline for a teacher to create different ways of expressing concepts. I am rather fussy about diagrams, however, as our Honours students would attest. I have a particular problem with arrows which mean different things in different places. If an arrow denotes passage of time in one instance it should do so in all instances, or a different style of arrow should be employed.

No way in or out

A problem I have with the PPDAC “Cycle” being a cycle is that it seems to imply that we can come in at any point and that there is no escape. If there is a logical starting point, and the link back to it is not one of process, then that should be indicated. Because the arrows are all the same style in the PPDAC diagram, it is also difficult to see a way out of the cycle. As a learner I would find it a little daunting to think that I could never escape! I am also concerned about understanding in what way does a Conclusion lead to a Problem? Surely the whole point of the word “Conclusion” is that it concludes or ends something?

To me there are at least three linkages between the Problem and the Conclusion. First of all, while in the Problem stage, we need to think about what we want to be able to say in the future Conclusion stage.  We may not know which way our conclusion will go, though we will probably have an opinion, or even a hope! (I am too post-modern in my thinking to believe in the objectivity of the researcher.) For instance we may want to be able to say – There is (or is not) evidence that women own more pairs of shoes than men. Another linkage is that when we write up our conclusion we must refer back to the original problem. And the third linkage comes from a comment Jean Thompson made on my blog about teaching time series without many computers. “Often the answer from a good statistical analysis is more questions”.  One conclusion can lead to a new problem.

I found a similar diagram online which is more sequential, starting with the problem and working vertically through the steps, with a link at the end going back to the beginning. I like this, because it does give an idea of conclusion and moving on, rather than being caught in some endless cycle. The reality for students is that they will generally do some project, which will start with a problem and end with a conclusion. Then they will move on to an unrelated project. It has also been my experience as a practitioner.

In my experience the cyclical behaviour which this diagram portrays is generally more within the cycle than over the whole cycle. For instance one may be part way through the data collection and realise that it isn’t going to work, and go back to the “Plan” stage. Some of these extra loops are suggested in my table.

Reporting

For students at a higher level who are required to write reports, it is difficult to see how the report fits in with the cycle. The “Conclusion” step includes “communication”, which could imply a report. However reports often include most of the steps, particularly when their purpose is to satisfy an assessment requirement.

Existing datasets

It is also difficult to apply the cycle in a non-cynical way to work with existing datasets. Often, in the interests of time and quality control, students are given a dataset. In reality they start, not at the Problem step, but somewhere between the Data step and the Analysis step. In their assessments they are required to read around the topic and use their imaginations to come up with the problem, look at how the data was collected, and move on from there.  This is not always the case, but it is for NCEA level 3 Bivariate Investigation, Time Series analysis and Formal Inference areas (called ‘standards’). The only area where they really do plan and collect the data is in the Experimental Design standard. Might it not be helpful to provide an adapted plan that takes into account these exigencies? Let us be explicit about it rather than coyly pretend that the data wasn’t driving everything?

In general I like the concept of the statistical enquiry cycle, and I am happy that it is providing a unifying theme to the curriculum. However, particularly at higher levels, I think it needs a bit of tweaking, taking into account the experience of teachers and learners.  If it is to hold such an important place in a curriculum that is leading the world, it deserves on-going attention.

Disclaimer

This is a blog and not an academic journal. The ideas I have contemplated need a lot more thought and background reading, but I do not have the time or the university salary to support such a luxury right now. Maybe someone else does!

Make journalists learn statistics

All journalists should be required to pass a course in basic statistics before they are let loose on the unsuspecting public.

I am not talking about the kind of statistics course that mathematical statisticians are talking about. This does not involve calculus, R or anything tricky requiring a post-graduate degree. I am talking about a statistics course for citizens. And journalists. 🙂

I have thought about this for some years. My father was a journalist, and fairly innumerate unless there was a dollar sign involved. But he was of the old school, who worked their way up the ranks. These days most media people have degrees, and I am adamant that the degree should contain basic numeracy and statistics. The course I devised (which has now been taken over by the maths and stats department and will be shut down later this year, but am I bitter…?) would have been ideal. It included basic number skills, including percentages (which are harder than you think), graphing, data, chance and evidence. It required students to understand the principles behind what they were doing rather than the mechanics.

Here is what journalists should know about statistics:

Chance

One of the key concepts in statistics is that of variability and chance.  Too often a chance event is invested with unnecessary meaning. A really good example of this is the road toll. In New Zealand the road toll over the Easter break can fluctuate between 21 (in 1971) and 3 in 1998, 2002 and 2003. Then in 2012 the toll was zero, a cause of great celebration. I was happy to see one report say “There was no one reason for the zero toll this Easter, and good fortune may have played a part.” However this was a refreshing change as normally the police seem to take the credit for good news, and blame bad news on us. Rather like Economists.

With any random process you will get variability. The human mind looks for patterns and meanings even where there are none. Sadly the human mind often finds patterns and imbues meaning erroneously. Astrology is a perfect example of this – and watching Deal or No Deal is inspiring in the meaning people can find in random variation.

All journalists should have a good grasp of the concepts of variability so they stop drawing unfounded conclusions

Data Display

There are myriad examples of graphs in the media that are misleading, badly constructed, incorrectly specified, or just plain wrong. There was a wonderful one in the Herald Sun recently, which has had considerable publicity. We hope it was just an error, and nothing more sinister. But good subediting (what my father used to do, but I think ceased with the advent of the computer) would have picked this up.

There is a very nice website dedicated to this: StatsChat.   It unfortunately misquotes H.G.Wells, but has a wonderful array of examples of good and bad statistics in the media. This post gives links to all sorts of sites with bad graphs, many of which were either produced or promulgated by journalists. But not all – scientific literature also has its culprits.

Just a little aside here – why does NO-ONE ever report the standard deviation? I was writing questions involving the normal distribution for practice by students. I am a strong follower of Cobb’s view that all data should be real, so I went looking for some interesting results I could use, with a mean and standard deviation. Heck I couldn’t even find uninteresting results! The mean and the median rule supreme, and confidence intervals are getting a little look in. Percentages are often reported with a “margin of error” (does anyone understand that?). But the standard deviation is invisible. I don’t think the standard deviation is any harder to understand than the mean. (Mainly because the mean is very hard to understand!) So why is the standard deviation not mentioned?

Evidence

One of the main ideas in inferential statistics is that of evidence: The data is here; do we have evidence that this is an actual effect rather than caused by random variation and sampling error? In traditional statistics this is about understanding the p-value. In resampling the idea is very similar to that of a p-value – we ask “could we have got this result by chance?” You do not have to be a mathematician to grasp this idea if it is presented in an accessible way. (See my video “Understanding the p-value” for an example.)

One very exciting addition to the New Zealand curriculum are Achievement Standards at Years 12 and 13 involving reading and understanding statistical reports. I have great hopes that as teachers embrace these standards, the level of understanding in the general population will increase, and there will be less tolerance for statistically unsound conclusions.

Another source of hope for me is “The Panel”, an afternoon radio programme hosted by Jim Mora on Radio New Zealand National. Each day different guests are invited to comment on current events in a moderately erudite and often amusing way. Sometimes they even have knowledge about the topic, and usually an expert is interviewed. It is as talkback radio really could be. I think. I’ve never listened long enough to talk-back radio to really judge as it always makes me SO ANGRY! Breathe, breathe…

I digress. I have been gratified to hear people on The Panel making worthwhile comments about sample size, sampling method, bias, association and causation. (Not usually using those exact terms, but the concepts are there.) It gives me hope that critical response to pseudo-scientific, and even scientific research is possible in the general populace. My husband thinks that should be “informed populace”, but I can dream.

It is possible for journalists to understand the important ideas of statistics without a mathematically-based and alienating course. I feel an app coming on… (Or should that be a nap?)