Teaching a service course in statistics

Most students who enrol in an initial course in statistics at university level do so because they have to. I did some research on attitudes to statistics in my entry level quantitative methods course, and fewer than 1% of the students had chosen to be in that course. This is a little demoralising, if you happen to think that statistics is worthwhile and interesting.

Teaching a service course in statistics is one of the great challenges of teaching. A “Service Course” is a course in statistics for students who are majoring in some other subject, such as Marketing or Medicine or Education. For some students it is a terminating course – they will never have to look at a p-value again (they hope). For some students it is the precursor to further applied statistics such as marketing research or biological research. Having said that, statistics for citizens is important and interesting and engaging if taught that way. And we might encourage some students to carry on.

Yet the teachers and textbook writers seem to do their best to remove the joy. Statistics is a difficult subject to understand. Often the way the instructor thinks is at odds with the way the students think and learn. The mathematical nature of the subject is invested with all sorts of emotional baggage.

Here are some of the challenges of teaching a statistics service course.

Limited mathematical ability

It is important to appreciate how limited the mathematical understanding is of some of the students in service courses. In my first year quantitative methods course, I made sure my students knew basic algebra, including rearranging and solving equations. This was all done within a business context. Even elementary algebra  was quite a stumbling block to some students, for whom algebra had been a bridge too far at school. There were students in a postgrad course I taught who were not sure which was larger, out of 0.05 and 0.1, and talked about crocodiles with regard to greater than and less than signs. And these were schoolteachers! Another senior maths teacher in that group had been teaching the calculation of confidence intervals, without actually understanding what they were.

The students are not like statisticians. Methods that worked to teach statisticians and mathematicians are unlikely to work for them. I wrote about this in my post about the Golden Rule, and how it applies at a higher level for teaching.

I realised a few years ago that I am not a mathematician. I do not have the ability to think in the abstract that is part of a true mathematician. Operations Research was my thing, because I was good at mathematics, but my understanding was concrete. This has been a surprising gift for me as a teacher, as it has meant that I can understand better what the students find difficult. Formulas do not tell them anything. Calculating by hand does not lead to understanding. It is from this philosophy that I approach the production of my videos. I am particularly pleased with my recent video about confidence intervals, which explains the ideas, with nary a formula in sight, but plenty of memorable images.

Software

One of my more constantly accessed posts is  Excel, SPSS, Minitab or R?. This consistent interest indicates that the course of software is a universal problem.  People are very quick to say how evil Excel is, and I am under no illusions as to many of the shortcomings. The main point of my post was, however, that it depends on the class you are teaching.

As I have taught mainly business students, I still hold that for them, Excel is ideal. Not so much for the statistical aspects, but because they learn to use Excel. Last Saturday the ideas for today’s posts were just forming in my mind when the phone rang, and despite my realising it was probably a telemarketer (we have caller ID on our phone) I answered it. It was a nice young woman asking me to take part in a short survey about employment opportunities for women in the Christchurch Rebuild. After I’d answered the questions, explaining that I was redundant from the university because of the earthquakes and that I had taught statistics, she realised that I had taught her. (This is a pretty common occurrence for me in our small town-city – even when I buy sushi I am served by ex-students). So I asked her about her experience in my course, and she related how she would never have taken the course, but enjoyed it and passed. I asked about Excel, and she told me that she had never realised what you could do with Excel before, and now still used it. This is not an isolated incident. When students are taught Excel as a tool, they use it as a tool, and continue to do so after the course has ended.

When business students learn using Excel, it has the appearance of relevance. They are aware that spreadsheets are used in business. It doesn’t seem like time wasted. So I stand by my choice to use Excel. However if I were still teaching at University, I would also be using iNZight. And if I taught higher levels I would continue to use SPSS, and learn more about R.

Textbooks

As I said in a previous post Statistics Textbooks suck out all the fun. Very few textbooks do no harm. I wonder if this site could provide a database of statistics texts and reviews. I would be happy to review textbooks and include them here. My favourite elementary textbook is, sadly, out of print. It is called “Taking the Fear out of Data Analysis”, by the fabulously named Adamantis Diamantopoulos and Bodo Schlegelmilch. It takes a practical approach, and has a warm, nurturing style. It lacks exercises. I have used extracts from it over the years. The choice of textbook, like the choice of software, is “horses for courses”, but I think there are some horses that should not be put anywhere near a course. I do wonder how many students use textbooks as anything other than a combination lucky charm and paper weight.

In comparison with the plethora of college texts of varying value, at high-school level the pickings for textbooks are thin. This probably reflects the newness of the teaching of statistics at high-school level.  A major problem with textbooks is that they are so quickly out of date, and at school level it is not practical to replace class sets too often.

Perhaps the answer is online resources, which can be updated as needed, and are flexible and give immediate feedback. 

Emotional baggage

I was less than gentle with a new acquaintance in the weekend. When asked about my business, I told him that I make on-line materials to help people teach and learn statistics. He proceeded to relate a story of a misplaced use of a percentage as a reason why he never takes any notice of statistics. I have tired of the “Lies, damned lies, and statistics” jibe and decided not to take it lying down. I explained that the world is a better place because of statistical analysis. Much research, including medical would not be possible in the absence of methods for statistical analysis. An understanding of the concepts of statistics is a vital part of intelligent citizenship, especially in these days of big and ubiquitous data.

I stopped at that point, but have pondered since. What is it that makes people so quick to denigrate the worth of statistics? I suspect it is ignorance and fear. They make themselves feel better about their inadequacies by devaluing the things they lack. Just a thought.

This is not an isolated instance. In fact I was so surprised when a lighthouse keeper said that statistics sounded interesting and wanted to know more, that I didn’t really know what to say next! You can read about that in a previous post. Statistics is an interesting subject – really!

But the students in a service course in statistics may well be in the rather large subset of humanity who have yet to appreciate the worth of the subject. They may even have fear and antipathy towards the subject, as I wrote about previously. Anxiety, fear and antipathy for maths, stats and OR.

People are less likely to learn if they have negative attitudes towards the subject. And when they do learn it may well be “learning to pass” rather than actual learning which is internalised.

So what?

Keep the faith! Statistics is an important subject. Keep trying new things. If you never have a bad moment in your teaching, you are not trying enough new things. And when you hear from someone whose life was changed because of your teaching, there is nothing like it!

Is statistical enquiry a cycle?

What is the statistical enquiry cycle and why is it a cycle? Is it really a cycle?

The New Zealand curriculum for Mathematics and statistics was recently held up as an example of good practice with regard to statistics. Yay us! In New Zealand the learning of statistics starts at the beginning of schooling and is part of the curriculum right through the school years. Statistics is developed as a discipline alongside mathematics, rather than as a subset of it. There are mathematics teachers who view this as an aberration, and believe that when this particular fad is over statistics will go back where it belongs, tucked quietly behind measurement, algebra and arithmetic. But the statisticians rejoice that the rich and exciting world of real data and detective work is being opened up to early learners. The outcome for mathematics and statistics remains to be seen.

A quick look over the Australian curriculum shows ostensibly a similar emphasis with regard to content at most levels.  The big difference (at first perusal) is that the New Zealand curriculum has two strands of statistics – statistical investigation, and statistical literacy, whereas the Australian curriculum has the more mathematical approach of “Data representation and interpretation”.  Both include probability as another strand.

Data Detective Cycle

In the New Zealand curriculum, the statistical investigation strand at every level refers to the “Statistical enquiry cycle”, shown here, which is also known as the PPDAC cycle. This is a unifying theme and organising framework for teachers and learners.

The data detective poster

This link takes you to a fuller explanation of the statistical enquiry cycle and its role at the different levels of the school curriculum. Note that the levels do not correspond to years. Click here to see the correspondence. The first five levels correspond to about 2 years each, whereas levels 6,7 and 8 correspond to the final three years of high school. So a child working on level 3 is generally aged about 10 or 11.

As I provide resources to support teaching and learning within the NZ curriculum I have become more aware of this framework, and have some questions and suggestions. I have made a table from which I hope to develop another diagram that students at higher levels can engage with, particularly with regard to the reporting aspects. As this is a work in progress you will have to wait!

Origins

Let’s look at the origins of the diagram and terminology. Maxine Pfannkuch (an educator) worked with Chris Wild (a statistician) to articulate what it is that statisticians do. They published their results in the international statistical review in 1999 and contributed the chapter “Towards an understanding of statistical thinking” in “The Challenge of Developing Statistical Literacy, Reasoning and Thinking”, edited by Dani Ben-Zvi and Joan Garfield. The statistical enquiry cycle has consequently been promulgated in the diagram and description referred to above. There is sound research behind this, and it makes good sense as a way of explaining what statisticians do.

Diagrams

I love diagrams. Anyone who has viewed my videos will know this. I spend a great deal of mental energy (usually while running) trying to work out ways to convey ideas in a visual way that will help people to learn, understand and remember. I also do NOT believe in the fad of learning styles, but rather I believe that all learners will gain from different presentations of concepts. I also believe that it is a useful discipline for a teacher to create different ways of expressing concepts. I am rather fussy about diagrams, however, as our Honours students would attest. I have a particular problem with arrows which mean different things in different places. If an arrow denotes passage of time in one instance it should do so in all instances, or a different style of arrow should be employed.

No way in or out

A problem I have with the PPDAC “Cycle” being a cycle is that it seems to imply that we can come in at any point and that there is no escape. If there is a logical starting point, and the link back to it is not one of process, then that should be indicated. Because the arrows are all the same style in the PPDAC diagram, it is also difficult to see a way out of the cycle. As a learner I would find it a little daunting to think that I could never escape! I am also concerned about understanding in what way does a Conclusion lead to a Problem? Surely the whole point of the word “Conclusion” is that it concludes or ends something?

To me there are at least three linkages between the Problem and the Conclusion. First of all, while in the Problem stage, we need to think about what we want to be able to say in the future Conclusion stage.  We may not know which way our conclusion will go, though we will probably have an opinion, or even a hope! (I am too post-modern in my thinking to believe in the objectivity of the researcher.) For instance we may want to be able to say – There is (or is not) evidence that women own more pairs of shoes than men. Another linkage is that when we write up our conclusion we must refer back to the original problem. And the third linkage comes from a comment Jean Thompson made on my blog about teaching time series without many computers. “Often the answer from a good statistical analysis is more questions”.  One conclusion can lead to a new problem.

I found a similar diagram online which is more sequential, starting with the problem and working vertically through the steps, with a link at the end going back to the beginning. I like this, because it does give an idea of conclusion and moving on, rather than being caught in some endless cycle. The reality for students is that they will generally do some project, which will start with a problem and end with a conclusion. Then they will move on to an unrelated project. It has also been my experience as a practitioner.

In my experience the cyclical behaviour which this diagram portrays is generally more within the cycle than over the whole cycle. For instance one may be part way through the data collection and realise that it isn’t going to work, and go back to the “Plan” stage. Some of these extra loops are suggested in my table.

Reporting

For students at a higher level who are required to write reports, it is difficult to see how the report fits in with the cycle. The “Conclusion” step includes “communication”, which could imply a report. However reports often include most of the steps, particularly when their purpose is to satisfy an assessment requirement.

Existing datasets

It is also difficult to apply the cycle in a non-cynical way to work with existing datasets. Often, in the interests of time and quality control, students are given a dataset. In reality they start, not at the Problem step, but somewhere between the Data step and the Analysis step. In their assessments they are required to read around the topic and use their imaginations to come up with the problem, look at how the data was collected, and move on from there.  This is not always the case, but it is for NCEA level 3 Bivariate Investigation, Time Series analysis and Formal Inference areas (called ‘standards’). The only area where they really do plan and collect the data is in the Experimental Design standard. Might it not be helpful to provide an adapted plan that takes into account these exigencies? Let us be explicit about it rather than coyly pretend that the data wasn’t driving everything?

In general I like the concept of the statistical enquiry cycle, and I am happy that it is providing a unifying theme to the curriculum. However, particularly at higher levels, I think it needs a bit of tweaking, taking into account the experience of teachers and learners.  If it is to hold such an important place in a curriculum that is leading the world, it deserves on-going attention.

Disclaimer

This is a blog and not an academic journal. The ideas I have contemplated need a lot more thought and background reading, but I do not have the time or the university salary to support such a luxury right now. Maybe someone else does!

Context – if it isn’t fun…

The role of context in statistical analysis

The wonderful advantage of teaching statistics is the real-life context within which any applicaton must exist. This can also be one of the difficulties. Statistics without context is merely the mathematics of statistics, and is sterile and theoretical.  The teaching of statistics requires real data. And real data often comes with a fairly solid back-story.

One of the interesting aspects for practicing statisticians, is that they can find out about a wide range of applications, by working in partnership with specialists. In my statistical and operations research advising I have learned about a range of subjects, including the treatment of hand injuries, children’s developmental understanding of probability, the bed occupancy in public hospitals, the educational needs of blind students, growth rates of vegetables, texted comments on service at supermarkets, killing methods of chickens, rogaine route choice, co-ordinating scientific expeditions to Antarctica and the cost of care for neonatals in intensive care. I found most of these really interesting and was keen to work with the experts on these projects. Statisticians tend to work in teams with specialists in related disciplines.

Learning a context can take time

When one is part of a long-term project, time spent learning the intricacies of the context is well spent. Without that, the meaning from the data can be lost. However, it is difficult to replicate this in the teaching of statistics, particularly in a general high school or service course. The amount of time required to become familiar with the context takes away from the time spent learning statistics. Too much time spent on one specific project or area of interest can mean that the students are unable to generalise. You need several different examples in order to know what is specific to the context and what is general to all or most contexts.

One approach is to try to have contexts with which students are already familiar. This can be enabled by collecting the data from the students themselves. The Census at School project provides international data for students to use in just this way. This is ideal, in that the context is familiar, and yet the data is “dirty” enough to provide challenges and judgment calls.

Some teachers find that this is too low-level and would prefer to use biological data, or dietary or sports data from other sources. I have some reservations about this. In New Zealand the new statistics curriculum is in its final year of introduction, and understandably there are some bedding-in issues. One I perceive is the relative importance of the context in the students’ reports. As these reports have high-stakes grades attached to them, this is an issue. I will use as an example the time series “standard”. The assessment specification states, among other things, “Using the statistical enquiry cycle to investigate time series data involves: using existing data sets, selecting a variable to investigate, selecting and using appropriate display(s), identifying features in the data and relating this to the context, finding an appropriate model, using the model to make a forecast, communicating findings in a conclusion.”

The full “standard” is given here: Investigate Time Series Data This would involve about five weeks of teaching and assessment, in parallel with four other subjects.(The final 3 years of schooling in NZ are assessed through the National Certificate of Educational Achievement (NCEA). Each year students usually take five subject areas, each of which consists of about six “achievement standards” worth between 3 and 6 credits. There is a mixture of internally and externally assessed standards.)

In this specification I see that there is a requirement for the model to be related to the context. This is a great opportunity for teachers to show how models are useful, and their limitations. I would be happy with a few sentences indicating that the student could identify a seasonal pattern and make some suggestions as to why this might relate to the context, followed by a similar analysis of the shape of the trend. However there are some teachers who are requiring students to do independent literature exploration into the area, and requiring references, while forbidding the referencing of Wikipedia.

This concerns me, and I call for robust discussion.

Statistics is not research methods any more than statistics is mathematics. Research methods and standards of evidence vary between disciplines. Clearly the evidence required in medical research will differ from that of marketing research. I do not think it is the place of the statistics teacher to be covering this. Mathematics teachers are already being stretched to teach the unfamiliar material of statistics, and I think asking them and the students to become expert in research methods is going too far.

It is also taking out all the fun.

Keep the fun

Statistics should be fun for the teacher and the students. The context needs to be accessible or you are just putting in another opportunity for antipathy and confusion. If you aren’t having fun, you aren’t doing it right. Or, more to the point, if your students aren’t having fun, you aren’t doing it right.

Some suggestions about the role of context in teaching statistics and operations research

• Use real data.
• If the context is difficult to understand, you are losing the point.
• The results should not be obvious. It is not interesting that year 12 boys weigh more than year 9 boys.
• Null results are still results. (We aren’t trying for academic publications!)
• It is okay to clean up data so you don’t confuse students before they are ready for it.
• Sometimes you should use dirty data – a bit of confusion is beneficial.
• Various contexts are better than one long project.
• Avoid the plodding parts of research methods.
• Avoid boring data. Who gives a flying fish about the relative sizes of dolphin jaws?
• Wikipedia is a great place to find out the context for most high school statistics analysis. That is where I look. It’s a great starting place for anyone.

Shibboleth, Mixolydian, Heteroscedasticity – and Kipling

All areas of human endeavour have specific language. Cricket commentators, art critics and wines buff make this very obvious.

Mixolydian

My son, who is blind, autistic and plays the piano like an angel, is studying Jazz, and I’m helping him. You can read more about this in my other blog Never Ordinary. There is a specific language around Jazz, and I’m not talking about ‘scat’. (Hmm just realised the other meaning for that word!) In the Jazz course they use words like Mixolydian, Chromatisism, Quartal Harmony…  I nod and smile. This language expresses ideas clearly and uniquely and is outside my comprehension. (Mixolydian is based on the Major scale, but with a flat 7. – clearer now?)

Trumpetty yellow, Daffodils, Narcissus

This week there was a statistics list discussion about the meaning of the term “multivariate”. As part of the ongoing discussion, someone suggested that using exact terminology exactly avoids a situation such as saying “I have yellow flowers in my garden with trumpetty bits, that come out in spring and have oniony looking bits in the ground.” This can also be said as “I have daffodils in my garden”.  However it can also be said as “I have Narcissus pseudonarcissus  in my garden”. Each of those phrases expresses the same idea, but with differing clarity or exclusiveness depending on the audience.

Shibboleth

Language can be used to exclude, as well as to inform or communicate. The term “shibboleth” comes from the book of Judges. When the Gileadites wished to find out if people crossing the river were Ephraimites, they would ask them to say the word “shibboleth”. If they said it as sibboleth, they killed them. The Old Testament can be a bit like that. The word “shibboleth” is now used to mean a code word, or knowledge that only a certain culture or group will know. Sometimes it can seem that statistical terms are used so only the initiated will be able to understand.

Virtue and Common Touch

As statisticians, operations researchers and teachers of statisticians and operations researchers we have many different opportunities to select the language we use. We must always be aware of our audience. In the poem, “If”, Kipling encourages people to be able to “…talk with crowds and keep your virtue, Or walk with Kings – nor lose the common touch,” Academics “walk with kings” when they write academic papers, using highly specialised and exclusive language. We need to make sure we do not lose the common touch. At the same time we should “keep our virtue”, and use the correct statistical term when the circumstances arise, making sure that we retain the common touch so that all understand.

Heteroscedasticity

When I use the term heteroscedasticity I am usually doing so for one of two reasons. First, that the data in question has non-constant variance, and I am explaining the concept and technical term to a client, student or colleague. Second, because I really like the word. “Heteroscedasticity” is eight syllables of tongue-twisting goodness! But, really, “non-constant variance” says exactly the same thing, has only six syllables and is easier to understand. I suspect a degree of linguistic snobbery appearing.

Communicating Statistics

Greenfield wrote a paper in 1993, which is still disappointingly relevant today. In “Communicating Statistics” (http://greenfieldresearch.co.uk/papers/Communicating%20stats.pdf) he suggests that statisticians have a great deal to offer the world, and that we aren’t doing a good job of making people aware of that. He was damning of the type of language used in academic publications, which ensure that any potentially useful results are obscured by “prolix and pseudo-objective style”.

This flows over into our consulting endeavours, where the aim should be to communicate rather than exclude. Greenfield gives the example fictionalised in this comic:

Depiction of almost true event. Click to view.

Greenfield’s parting provocative statement was to suggest that statisticians produce more cookery-books and more easy-to-use programs, and encourage their use by everybody who can benefit. These books and programs can carry the message that if they want to do better they should study more and seek the guidance of statisticians.

In closing he says “Our audience, our customers are out there. They need us, even if they do not realise it. We must change our culture, our philosophy, our public relations and our use of language to reach them.”

Greenfield Challenge

I’m not sure I want to be telling you about the Greenfield challenge, as I’m thinking of entering it, and would really like a trip to Ankara for the ENBIS conference. But in pursuit of the greater good, I am putting a link here: The Greenfield Challenge. The blurb explains:

“We would like to encourage you to report immediately whenever you’ve had dealings with non-statisticians – in whichever form (face-to-face, in writing, in form of an audio or video recording, in interactive social media … ) or context (interactions with students, educators, managers and employees of organizations in private and public sectors … ).”

Greenfield even suggests “You might even write a short story or a play.”

Still thinking about that one. I guess there is always “The Goal” to look to for an example. In the meantime I’ll stick to this work of mostly non-fiction, interspersed with opinion and anecdote.

Choose our words

When we use very specific technical terms we need to make sure that they are really necessary. Is there a simpler, and just as accurate way of saying the same thing? If our audience is statisticians, then really we can indulge in specific technical language. But if the audience includes students, non-statisticians and the general public, then we should probably use simpler terms, or at least “gloss”, or say what the word means along with its use. (There was an example of glossing right there!)

I have written earlier about the minefield of statistical terminology, particularly when the statistical word also has an everyday meaning which is not quite the same. Examples of this are “significant”, “random” and “relationship”. The post includes some suggestions for teaching statistical language.

But as well as teachers, we are also communicators, and need to get our message across in the best way possible. It is vital to determine our audience, and make sure we bring them along with us.

I contemplate the new New Zealand curriculum with excitement. Through the efforts of a group of statisticians we are able to inculcate a greater understanding of the essentials of statistics from an early age to much of the population. The role of the statisticians is to help the teachers feel at home in the world of statistics, so that they can invite their students along. These are exciting times. The rest of the world is watching.

Interpreting Scatterplots

Patterns, vocab and practice, practice, practice

An important part of statistical analysis is being able to look at graphical representation of data, extract  meaning and make comments about it, particularly related to the context. Graph interpretation is a difficult skill to teach as there is no clear algorithm, such as mathematics teachers are used to teaching, and the answers are far from clear-cut.

This post is about the challenges of teaching scatterplot interpretation, with some suggestions.

When undertaking an investigation of bivariate measurement data, a scatterplot is the graph to use. On a scatterplot we can see what shape the data seems to have, what direction the relationship goes in, how close the points are to the line, if there are clear groups and if there are unusual observations.

The problem is that when you know what to look for, spurious effects don’t get in the way, but when you don’t know what to look for, you don’t know what is spurious. This can be likened to a master chess player who can look at a game in play and see at a glance what is happening, whereas the novice sees only the individual pieces, and cannot easily tell where the action is taking place. What is needed is pattern recognition.

In addition, there is considerable room  for argument in interpreting scatterplots. What one person sees as a non-linear relationship, another person might see as a line with some unusual observations. My experience is that people tend to try for more complicated models than is sensible. A few unusual observations can affect how we see the graph. There is also a contextual content to the discussion. The nature of the individual observations, and the sample can make a big difference to the meaning drawn from the graph. For example, a scatterplot of the sodium content vs the energy content in food should not really have a strong relationship. However, if the sample of food taken is predominantly fast food, high sodium content is related to high fat content (salt on fries!) and this can appear to be a relationship. In the graph below, is there really a linear relationship, or is it just because of the choice of sample?

In a set of data about fast food, there appears to be a relationship between sodium content and energy.

Students need to be exposed to a large number of different scatterplots, Fortunately this is now possible, thanks to computers. Students should not be drawing graphs by hand.

So how do we teach this? I think about how I learned to interpret graphs, and the answer is practice, practice, practice. This is actually quite tricky for teachers to arrange, as you need to have lots of sets of data for students to look at, and you need to make sure they are giving correct answers. Practice without feedback and correction can lead to entrenched mistakes.

Because graph interpretation is about pattern recognition, we need to have patterns that students can try to match the new graphs to. It helps to have some examples that aren’t beautifully behaved. The reality of data is that quite often the nature of measurement and rounding means that the graph appears quite different from the classic scatter-plot. The following graph has a strangely ordered look to it because the x-axis variable takes only whole numbers, and the prices are nearly always close to the nearest thousand.

The asking price of used Toyota sedans against the year of manufacture.

Students also need examples of the different aspects that you would comment on in a graph, using appropriate vocabulary. Just as musicians need to label different types of scales in order to communicate with each other their musical ideas, there is a specific vocabulary for describing graphs. Unfortunately the art of describing scatterplots is not as developed as music, and at times the terms are unclear and even used in different ways by different people.

Materials produced for teacher development , available on Census @ School suggest the following things to comment on: Trend, Association, Strength, Groups and unusual observations.

The following uses the framework provided by R. Kaniuk, R. Parsonage

Trend covers the idea of whether the graph is linear or non-linear. I don’t really like the use of the word “trend” here, as to me it should be used for time-series data only. I would use the word “shape” in preference. It means a general tendency.

Association is about the direction. Is the relationship positive or negative? For example, “as the distance a car has travelled increases, the asking price tends to decrease.” The term “tends to” is very useful here.

Strength is about how close the dots are to the fitted line. In a linear model we can use correlation to quantify the strength. My experience is that students often confuse strength with slope.

Groups can appear in the data, and it is much more relevant if the appearance of groups is related to an attribute of the observations. For example in some data about food values in fast food, the dessert and salad items were quite separate from the other menu items. You can see that in the graph above of food items.

Unusual observations are a challenging feature of real-world data. Is it a mistake? Is it someone being silly, or misinterpreting a question? Is it not really from this population? Is it the result of a one-off rare occurrence (such as my redundancy payment earlier this year)? And what should you do with unusual observations? I’ve written a bit more about this in my post on dirty data. And there is uneven scatter, or heteroscedastiticity, which does not affect model definition, so much as prediction intervals.

On line practice works

An effective way to give students practice,  with timely feedback, is through on-line materials. Graphs take up a lot of room on paper, so textbooks cannot easily provide the number of examples that are needed to develop fluency. With our on-line materials we provide many examples of graphs, both standard, and not so well-behaved. Students choose from statements about the graphs. Most of the questions provide two graphs, as pattern recognition is easier to develop when looking at comparisons. For example if you give one graph and say “How strong is this relationship?”, it can be difficult to quantify. This is made easier when you ask which of two graphs has a  stronger relationship.

Students get immediate feedback in a “low-jeopardy” situation. When a tutor is working one-on-one with a student, it can be worrying to the student if they get wrong answers. The computer is infinitely patient and the student can keep trying over and over until they get their answers correct, thus reinforcing correct understanding.

This system and set of questions is part of our on-line resources for teaching Bivariate investigations, which occurs within the NZ Stats 3 course. You can find out more about our resources at www.statslc.com, and any teachers who wish to explore the materials for free should email me at n.petty(at)statslc.com.

Make journalists learn statistics

All journalists should be required to pass a course in basic statistics before they are let loose on the unsuspecting public.

I am not talking about the kind of statistics course that mathematical statisticians are talking about. This does not involve calculus, R or anything tricky requiring a post-graduate degree. I am talking about a statistics course for citizens. And journalists.

I have thought about this for some years. My father was a journalist, and fairly innumerate unless there was a dollar sign involved. But he was of the old school, who worked their way up the ranks. These days most media people have degrees, and I am adamant that the degree should contain basic numeracy and statistics. The course I devised (which has now been taken over by the maths and stats department and will be shut down later this year, but am I bitter…?) would have been ideal. It included basic number skills, including percentages (which are harder than you think), graphing, data, chance and evidence. It required students to understand the principles behind what they were doing rather than the mechanics.

Here is what journalists should know about statistics:

Chance

One of the key concepts in statistics is that of variability and chance.  Too often a chance event is invested with unnecessary meaning. A really good example of this is the road toll. In New Zealand the road toll over the Easter break can fluctuate between 21 (in 1971) and 3 in 1998, 2002 and 2003. Then in 2012 the toll was zero, a cause of great celebration. I was happy to see one report say “There was no one reason for the zero toll this Easter, and good fortune may have played a part.” However this was a refreshing change as normally the police seem to take the credit for good news, and blame bad news on us. Rather like Economists.

With any random process you will get variability. The human mind looks for patterns and meanings even where there are none. Sadly the human mind often finds patterns and imbues meaning erroneously. Astrology is a perfect example of this – and watching Deal or No Deal is inspiring in the meaning people can find in random variation.

All journalists should have a good grasp of the concepts of variability so they stop drawing unfounded conclusions

Data Display

There are myriad examples of graphs in the media that are misleading, badly constructed, incorrectly specified, or just plain wrong. There was a wonderful one in the Herald Sun recently, which has had considerable publicity. We hope it was just an error, and nothing more sinister. But good subediting (what my father used to do, but I think ceased with the advent of the computer) would have picked this up.

There is a very nice website dedicated to this: StatsChat.   It unfortunately misquotes H.G.Wells, but has a wonderful array of examples of good and bad statistics in the media. This post gives links to all sorts of sites with bad graphs, many of which were either produced or promulgated by journalists. But not all – scientific literature also has its culprits.

Just a little aside here – why does NO-ONE ever report the standard deviation? I was writing questions involving the normal distribution for practice by students. I am a strong follower of Cobb’s view that all data should be real, so I went looking for some interesting results I could use, with a mean and standard deviation. Heck I couldn’t even find uninteresting results! The mean and the median rule supreme, and confidence intervals are getting a little look in. Percentages are often reported with a “margin of error” (does anyone understand that?). But the standard deviation is invisible. I don’t think the standard deviation is any harder to understand than the mean. (Mainly because the mean is very hard to understand!) So why is the standard deviation not mentioned?

Evidence

One of the main ideas in inferential statistics is that of evidence: The data is here; do we have evidence that this is an actual effect rather than caused by random variation and sampling error? In traditional statistics this is about understanding the p-value. In resampling the idea is very similar to that of a p-value – we ask “could we have got this result by chance?” You do not have to be a mathematician to grasp this idea if it is presented in an accessible way. (See my video “Understanding the p-value” for an example.)

One very exciting addition to the New Zealand curriculum are Achievement Standards at Years 12 and 13 involving reading and understanding statistical reports. I have great hopes that as teachers embrace these standards, the level of understanding in the general population will increase, and there will be less tolerance for statistically unsound conclusions.

Another source of hope for me is “The Panel”, an afternoon radio programme hosted by Jim Mora on Radio New Zealand National. Each day different guests are invited to comment on current events in a moderately erudite and often amusing way. Sometimes they even have knowledge about the topic, and usually an expert is interviewed. It is as talkback radio really could be. I think. I’ve never listened long enough to talk-back radio to really judge as it always makes me SO ANGRY! Breathe, breathe…

I digress. I have been gratified to hear people on The Panel making worthwhile comments about sample size, sampling method, bias, association and causation. (Not usually using those exact terms, but the concepts are there.) It gives me hope that critical response to pseudo-scientific, and even scientific research is possible in the general populace. My husband thinks that should be “informed populace”, but I can dream.

It is possible for journalists to understand the important ideas of statistics without a mathematically-based and alienating course. I feel an app coming on… (Or should that be a nap?)

One year on!

I have been blogging for just under a year now, and have written over 50 posts. There have been over 30,000 hits on the blog, and some very helpful comments. I’ve had a lot of fun, and there is something exciting about thinking that other people might value my thoughts and writing. Thank you all those who have left comments or emailed me.

I spent my morning making up a summary page so that it is easier to find your way around previous posts. It is in the “Collected Works” tab above. In order to do this I had to read (or at least skim) all my previous posts. It was quite interesting really. I hope you find it easier to find what you are looking for.

Here are links to some that took my fancy today:

I am many numbers – a really interesting discussion on the role of numbers in defining who we are. Would work well in class.

Statistics and chocolate – nifty and effective teaching idea for getting across the idea of evidence with respect to probability

Teaching statistical language - how I didn’t get a free iPad

The meaning of the mean - it is trickier than you think

The Sound of Music meets Linear Programming

“Let’s start at the very beginning – a very good place to start. When you read you begin with A, B,C!” When you do statistics you begin with…probability? the mean? graphs?

Begin at the end

But really, is the beginning a very good place to start? Sometimes, we need to begin at the end. And sometimes we need to go back before the beginning. Always we need to think about where to begin, because it is seldom obvious, and copying what other teachers and textbooks have done is often a bad idea.

Linear programming

Take Linear Programming, the flagship technique of Operations Research. Most text books start with a simple two variable example, one that can be drawn on a Cartesian plane. They begin by defining the decision variables and the objective function. Next they formulate the constraints and explain the non-negativity conditions. Then finally they get around to solving the problem – often through a graphical approach, and applying it to the trivial real-life imaginary example they started with.

Here is a better approach, with Linear programming as the example:

First ensure all the class members have the prerequisite mathematical skills for what you propose to teach. If they are not good at drawing equations on a plane, you will need to teach them again, or use a different approach such as using Excel Solver. If students are not sure which way around > and < signs go, you will need to go over it. If English is their second language you will need to make sure you explain words like constraint, objective and optimum. This won’t hurt the native English speakers either.

Second think about your destination. When children learn to read, they generally know what the outcome is going to be. They will be able to look at words on a page and make sense of them. When you learn to drive, you know the outcome – you will be able to get safely from one place to another behind the wheel of a car. When we learn to bake cakes, we like to have pictures of the finished product so that we can see where we are headed. Yet somehow we try to teach as if it is a voyage of discovery with no vision of the end. Now discovery is good, if it pertains to how we get to or understand a process, but students need to know what they are learning. It also helps to have a purpose. Reading, driving and baking are all purposeful, with a clear outcome. The same should be true of linear programming (or confidence intervals or decision trees or fitted lines or just about anything else we are learning.)

You give the students an illustration of the completed LP model of the problem, preferably complex enough to be realistic. You show them how it can be useful, and give them a chance to explore the model. This is SO much easier now that we have Excel and Solver to look after the solving. Let students find out all about one model and then another and another, before you begin to show how to formulate. When people know what they are trying to produce, the reasoning behind the steps is more obvious.

Linear Regression

The same approach can be applied to teaching Linear Regression analysis. First we need to make sure that students understand what a fitted line on a graph is. Get them to interpret several fitted graphs, and use them to make predictions and write statements about the nature of the relationships modelled. Then show how to make the fitted graphs once they know why they need to.

In last week’s post I talked about histograms. Students should learn to interpret histograms and other graphs before they are required to make their own. Having to read off pie charts should help immunise them against their use.

I was in a computer lab with some students from another first year statistics course, and noticed that the first thing they were taught was how to calculate the mean and standard deviation, including the finite population correction. Was this really the most interesting way to get them introduced to the joys of data analysis and interpretation? Why start with the mean, one of the most difficult concepts in statistics?

Work backwards from the end

There is an interesting technique used for teaching skills to children with special needs. When you teach a blind child to tie shoelaces, you start at the end. You do all but the last part, and let them finish it off. This gives a sense of success and purpose. Then gradually you add the steps backwards, so that they start earlier on in the process. This also means that the part of the skill that is getting the most repetition is the new part, not the part already mastered. The same is true of memorisation. Memorise the last line first, then the last two lines etc. I suspect the same approach may well apply to more abstract skills. Maybe we should teach how to read and critique a statistical analysis, then how to write one, then finally how to do the analysis.

The spiral approach is popular, in which topics are revisited each year and built on.  I would like to incorporate principles of mastery learning along with that. Mastery learning is based on the premise that you must master a skill before moving on to the next one. This is difficult to implement in a classroom, with mixed level of ability, but is more easily enacted with the help of a Learning Management System.

New math had odd beginnings

I was born in the early 1960s and was in the first cohort of children to learn “new math(s)”, devised in the US as a reaction to the humiliation of seeing the Russians put Sputnik into space before them. Even in New Zealand we were not immune to the influence of the Cold War on education!  I loved our bright new textbooks,  which started with Set Theory – even at age 6. Every year the first page of the text book had diagrams of herds of sheep, prides of lions and other sundry collections.  I loved the Venn diagrams and the intersections – even cardinal numbers, but to this day I’m not sure how that connected with mathematics, and learning to add and subtract. And to this day I ask, “What were they thinking?” It appears that set theory is the foundation of all mathematics, and thus these mathematicians decided to start there, baffling teachers and parents alike, who were alienated by these words and symbols.

I have no doubt that the intention was to improve learning, but it seems ill-advised now. I wonder how our attempts will be viewed with the benefits of 40 years of hindsight. These days constructivism is a popular, though not universal, theory and approach to learning. The idea is that we create knowledge through adding new ideas and experiences onto our current knowledge. Sometimes that involves undoing erroneous or primitive knowledge.

Sometimes a good approach is historical – to imitate in the learner (in an accelerated form) the learning process through which mankind has progressed, preferably missing out the stupid bits. (Roman numerals are fun for some children, but pretty pointless once you realise the power of zero). It is certainly worth contemplating as an alternative approach.

This post has touched on ideas regarding the sequencing of a learning/teaching approach. There are many considerations and serious thought needs to go into where we start. Sometimes we need to start at the end.

Judgment Calls in Statistics and O.R.

The one-armed operations researcher

My mentor, Hans Daellenbach told me a story about a client asking for a one-armed Operations Researcher. The client was sick of getting answers that went, “On the one hand, the best decision would be to proceed, but on the other hand…”

People like the correct answer. They like certainty. They like to know they got it right.

I tease my husband that he has to find the best picnic spot or the best parking place, which involves us driving around considerably longer than I (or the children) were happy with. To be fair, we do end up in very nice picnic spots. However, several of the other places would have been just fine too!

In a different context I too am guilty of this – the reason I loved mathematics at school was because you knew whether you were right or wrong and could get a satisfying row of little red ticks (checkmarks) down the page. English and other arts subjects, I found too mushy as you could never get it perfect. Biology was annoying as plants were so variable, except in their ability to die. Chemistry was ok, so long as we stuck to the nice definite stuff like drawing organic molecules and balancing redox equations.

I think most mathematics teachers are mathematics teachers because they like things to be right or wrong. They like to be able to look at an answer and tell whether it is correct, or if it should get half marks for correct working. They do NOT want to mark essays, which are full of mushy judgements.

Again I am sympathetic. I once did a course in basketball refereeing. I enjoyed learning all the rules, and where to stand, and the hand signals etc, but I hated being a referee. All those decisions were just too much for me. I could never tell who had put the ball out, and was unhappy with guessing. I think I did referee two games at a church league and ended up with an angry player bashing me in the face with the ball. Looking back I think it didn’t help that I wasn’t much of a player either.

I also used to find marking exam papers very challenging, as I wanted to get it right every time. I would agonise over every mark, thinking it could be the difference between passing and failing for some poor student. However as the years went by, I realised that the odd mistake or inconsistency here or there was just usual, and within the range of error. To someone who failed by one mark, my suggestion is not to be borderline. I’m pretty sure we passed more people that we shouldn’t have, than the other way around.

Life is not deterministic

The point is, that life in general is not deterministic and certain and rule-based. This is where the great divide lies between the subject of mathematics and the practice of statistics. Generally in mathematics you can find an answer and even check that it is correct. Or you can show that there is no answer (as happened in one of our national exams in 2012!). But often in statistics there is no clear answer. Sometimes it even depends on the context. This does not sit well with some mathematics teachers.

In operations research there is an interesting tension between optimisers and people who use heuristics. Optimisers love to say that they have the optimal solution to the problem. The non-optimisers like to point out that the problem solved optimally, is so far removed from the actual problem, that all it provides is an upper or lower bound to a practical solution to the actual real-life problem situation.

Judgment calls occur all through the mathematical decision sciences. They include

• What method to use – Linear programming or heuristic search?
• Approximations – How do we model a stochastic input in a deterministic model?
• Assumptions – Is it reasonable to assume that the observations are independent?
• P-value cutoff – Does a p-value of exactly 0.05 constitute evidence against the null hypothesis?
• Sample size – Is it reasonable to draw any inferences at all from a sample of 6?
• Grouping – How do we group by age? by income?
• Data cleaning – Do we remove the outlier or leave it in?

A comment from a maths teacher on my post regarding the Central Limit Theorem included the following: “The questions that continue to irk me are i) how do you know when to make the call? ii) What are the errors involved in making such a call? I suppose that Hypothesis testing along with p-values took care of such issues and offered some form of security in accepting or rejecting such a hypothesis. I am just a little worried that objectivity is being lost, with personal interpretation being the prevailing arbiter which seems inadequate.”

These are very real concerns, and reflect the mathematical desire for correctness and security. But I propose that the security was an illusion in the first place. There has always been personal interpretation.Informal inference is a nice introduction to help us understand that. And in fact it would be a good opportunity for lively discussion in a statistics class.

With bootstrapping methods we don’t have any less information than we did using the Central Limit Theorem. We just haven’t assumed normality or independence. There was no security. There was the idea that with a 95% confidence interval, for example, we are 95% sure that we contain the true population value. I wonder how often we realised that 1 in 20 times we were just plain wrong, and in quite a few instances the population parameter would be far from the centre of the interval.

The hopeful thing about teaching statistics via bootstrapping, is that by demystifying it we may be able to inject some more healthy scepticism into the populace.

Teaching experimental design

Teaching Experimental Design – a cross-curricular opportunity

The elements that make up a statistics, operations research or quantitative methods course cover three different dimensions (and more). There are:

• techniques we wish students to master,
• concepts we wish students to internalise, and
• attitudes and emotions we wish the students to adopt.

Techniques, concepts and attitudes interact in how a student learns and perceives the subject. Sadly it is possible (and not uncommon) for students to master techniques, while staying oblivious to many of the concepts, and with an attitude of resignation or even antipathy towards the discipline.

Techniques

Often, and less than ideally, course design begins with techniques. The backbone is a list of tests, graphs and procedures that students need to master in order to pass the course. The course outline includes statements like:

• Students will be able to calculate a confidence interval for a mean.
• Students will be able to formulate a linear programming model from data.
• Students will use Excel to make correct histograms. (Good luck with this one!)

Textbooks are organised around techniques, which usually appear in a given sequence, relying on the authors’ perception of how difficult each technique is. Textbooks within a given field are remarkably similar in the techniques they cover in an introductory course.

Concepts

Concepts are more difficult to articulate. In a first course in statistics we wish students to gain an appreciation of the effects of variation. They need to understand how data from a sample differs from population data. In all of the mathematical decision sciences students struggle to understand the nature of a model. The concept of a mathematical model is far from intuitive, but essential.

Attitudes

You can’t explicitly teach attitudes. “Today class, you are going to learn to love statistics!”. These are absorbed and formed and reformed as part of the learning process, as a result of prior experiences and attitudes. I have written a post on Anxiety, fear and antipathy for maths, stats and OR, which describes the importance of perseverance, relevance, borrowed self-efficacy and love in the teaching of these subjects. Content and problem context choices can go a long way towards improving attitudes. The instructor should know whether his or her class is more interested in the projectories of gummy bears, or the more serious topics of cancer screening and crime prevention. Classes in business schools will use different examples than classes in psychology or forestry. Whatever the context, the data should be real, so that students can really engage with it.

I was both amused and a little saddened at this quote from a very good book, “Succeed – how we can reach our goals”. The author (Heidi Grant Halvorson) has described the outcomes of some interesting experiments regarding motivation. She then says, “At this point, you may be wondering if social psychologists get a particular pleasure out of asking people to do really odd things, like eating Cheerios with chopsticks, or eating raw radishes, or not laughing at Robin Williams. The short answer is yes, we do. It makes up for all those hours spent learning statistics.” Hmmm

Experimental Design

So what does this have to do with experimental design?

I have a little confession. I’ve never taught experimental design. I wish I had. I didn’t know as much then as I do now about teaching statistics, and I also taught business students. That’s my excuse, but I regret it. My reasoning was that businesses usually use observational data, not experimental data. And it’s true, except perhaps in marketing research, and process control and possibly several other areas. Oh.

George Cobb, whom I have quoted in several previous posts, proposed that experimental design is a mechanism by which students may learn important concepts. The technique is experimental design, but taught well, it is a way to convey important concepts in statistics and decision science. The pivotal concept is that of variation. If there were no variation, there would be no need for statistics or experimentation. It would be a sad, boring deterministic world. But variation exists, some of which is explainable, and some of which is natural, some of which is due to sampling and some of which is due to bad sampling or experimental practices. I have a YouTube video that explains these four sources of variation. Because variation exists, experiments need to be designed in such a way that we can uncover as best we can the explainable variation, without confounding it with the other types of variation.

The new New Zealand curriculum for Mathematics and Statistics includes experimental design at levels 2 and 3 of the National Certificate of Educational Achievement. (The last two years of Secondary School). The assessments are internal, and teachers help students set up, execute and analyse small experiments. At level two (implemented this year) the experiments generally involve two groups which are given two treatments, or a treatment and a control. The analysis involves boxplots and informal inference. Some schools used paired samples, but found the type of analysis to be limited as a result.  At level three (to be implemented in 2013) this is taken a step further, but I haven’t been able to work out what this step is from the curriculum documents. I was hoping it might be things like randomised block design, or even Taguchi methods, but I don’t think so.

Subjects for Experimentation

Bearing in mind the number of students, many of whom wish to use other members of the class, there can be issues of time and fatigue.Here are some possibilities. It would be great if other suggestions could be added as comments to this post.

Behavioural

Some teachers are reluctant to use psychological experiments as it can be a bit worrying to use our students as guinea pigs. However, this is probably the easiest option, and provided informed and parental consent is received, it should be acceptable. All sorts have been suggested such as effects of various distractions (and legal stimulants) on task completion. There are possible experiments in Physical Education (Evaluate the effectiveness of a performance enhancing programme). Or in Music – how do people respond to different music?

I’d love to see some experiments done on time taken to solve Rogo puzzles! and what the effect of route length or number choice, or size or age is.

Biology

Anything that involves growing things takes a while and can be fraught. (My own recollection of High School biology is that all my plants died.) But things like water uptake could be possible. Use sticks of celery of different lengths and see how much water they take up in a given time. Germination times or strike rates under different circumstances using cress or mustard?  Talk to the Biology teacher. There are assessment standards in NZ NCEA at levels 2 and 3 which mesh well with the statistics standards.

Technology

Baking. There are various ingredients that could have two or three levels of inclusion – making muffins with and without egg – does it affect the height? Pretty tricky to control, but fun – maybe use uniform amounts of mixture. Talk to the Food tech teacher.

Barbie bungee jumping. How does Barbie’s weight affect how far she falls. By having Barbie with and without a backpack, you get the two treatments. The bungee cords can be made out of rubber bands or elastic.

Things flying through the air from catapaults. This has been shown to work as a teaching example. There are a number of variables to alter, such as the weight of the object, the slope of the launchpad, and the person firing.

Inject statistical ideas in application areas

John Maindonald from ANU made the following comment on a previous post: “I am increasingly attracted to the idea that the place to start injecting statistical ideas is in application areas of the curriculum.  This will however work only if the teaching and learning model changes, in ways that are arguably anyway necessary in order to make effective use of those teachers who have really good and effective mathematics and statistics and computing skills.”

How exciting is that? Teachers from different discipline areas work together! There may well be logistical issues and even problems of “turf”. But wouldn’t it be great for mathematics teachers to help students with experiments and analysis in other areas of the curriculum. The students will gain from the removal of “compartments” in their learning, which will help them to integrate their knowledge. The worth of what they are doing would be obvious.

(Note for teachers in NZ. A quick look through the “assessment matrices” for other subjects uncovered a multitude of possibilities for curricular integration if the logistics and NZQA allow. )