Organising the toolbox in statistics and operations research

Don’t bury students in tools     

In our statistics courses and textbooks there is a tendency to hand our students tool after tool, wanting to teach them all they need to know. However students can feel buried under these tools and unable to decide which to use for which task. This is also true in beginning Operations Research or Management Science courses. To the instructors, it is obvious whether to use the test for paired or independent samples or whether to use multicriteria decision making or a decision tree.  But it is just another source of confusion for the student, who wants to be told what to do.

Tools for statistics and operations research

A common approach to teaching hypothesis testing in business statistics courses, if textbooks are anything to go by, is to teach several different forms of hypothesis testing, starting with the test for a mean, and test for a proportion then difference of two means, independent and paired, then difference of two proportions. Then we have tests for regression and correlation, and chi-squared test for independence. These are the seven basic statistical tests that people are likely to use or see. I would probably add ANOVA, if there is enough time. Even listed, this seems a bit confusing.

An introductory operations research course might include any number of topics including linear programming, integer programming, inventory control, queueing, simulation, decision analysis, critical path, the assignment problem, dynamic programming, systems analysis, financial modelling, inventory control…And I would hope some overall teaching about models and the OR process.

Issues with the pile of tools

Of course we need to teach the essential tools of our discipline, but there are two issues arising from this approach.

The obvious one is that students are left bewildered as to which test they should use when. Because of the way textbooks and courses are organised, students don’t usually have to decide which tool to use in a given situation. If the preceding chapter is about linear programming, then the questions will be about linear programming.

The second issue is that unless students are helped, they fail to see the connections between the techniques and are left with a fragmented view of the discipline. It is not just a question of which tool to use for which task, it is about seeing the linkages and the similarities. We want to help them have integrated knowledge.

Providing activities to help with organisation

In both my introductory courses I attempted to address this, with varying degrees of success.

In our management science course we end the year with a case of a situation with multiple needs, and the students were to identify which technique would be useful in each instance. Then the final exam has a similar question, with specific questions about over-arching concepts such as deterministic and stochastic inputs, and the purpose of the model – to optimise or inform. This is also an opportunity to address issues of ethics and worldview.

In the final section of the business statistics course we have a large bank of questions for students to work through, to give them practice in deciding which test to use. I was careful to make sure that there was more than one question related to each scenario, so that students would not learn unhelpful shortcuts, such as, if the question is about weight loss, the answer must be paired difference of two means. I also analysed the mistakes given in multichoice answers, to see where confusion was arising, sometimes due to poor wording. From this I refined the questions.

Examples of the questions for test choice in hypothesis testing

Management thinks there is a difference in productivity between the two days of the week in a certain work area. The production output of a random sample of 15 factory workers is recorded on both a Tuesday and a Friday of the same week. For each worker, the number of completed garments is counted on both days.

A restaurant manager is thinking of doing a special “girls’ night out” promotion. She suspects that groups of women together are more likely to stay for dessert than mixed adult or family groups. For the next two weeks she gets the staff to write down for each table whether they stay for dessert, and what type of group they are. She asks you to see if her suspicion is correct.

A human resources department has data on 200 past employees, including how long, in months, they stayed at the company, and the mark out of 100 they got in their recruitment assessment. They ask you to work out whether you can predict how long a person will stay, based on their test mark.

A researcher wanted to investigate whether a new diet was effective in helping with weight loss. She got 40 volunteers and got 20 to use the diet and the other 20 to eat normally. After 6 weeks the weights (in kg) before and after were recorded for each volunteer, and the difference calculated. She then looked at how the weight losses differed between the two groups.

Comment on the questions

You might notice that all the examples are in a business context. This is because this is a course for business students, and they need to know that what they are learning is relevant to their future. Questions about dolphins and pine trees are not suitable for these students. (Unless we are making money out of them!)

The master diagram

The students to work through these multiple choice questions on-line, and we offered help and coached them through questions with which they had difficulty. By taking my turn with the teaching assistants in the computer labs, I was able to understand better how the students perceived the tests, and ways to help them with this. The result is a diagram, or set of diagrams which shows the relationships between the tests, and a procedure to help them make the decision. I am a great believer in diagrams, but they need to be well thought out. Many textbooks have branching diagrams, showing a decision process for which test to use. I felt there was a more holistic way to approach it, and thought long and hard, and tried out my diagrams on students before I came up with our different approach. You can see the diagrams here by clicking on the link to the pdf which you can download: Choosing the test diagrams

The three questions which help the students to identify the most appropriate test are:

1. What level of measurement is the data – Nominal or interval/ratio?
2. How many samples do we have?
3. What is the purpose of our analysis?

I made an on-line lesson which takes the students through the steps over and over, and created the diagrams to help them. Time and again the students said how much it helped them to fit it all together. Eventually I made the following video, which is on YouTube. I suspect it must be coming up to summary time in courses in the US, as this video has recently attracted a lot of views, and positive comments.

The video is also part of our app, AtMyPace: Statistics along with two sets of questions to help students to learn about the different types of tests and how to tell them apart. You can access the same resources on-line through AtMyPace:Statistics statsLC.com.

It is important to see the subject as a whole, and not a jumbled mass of techniques and ideas, and this has really helped my students and many others through the video and app.

Best wishes for the holiday season

It is Christmas time and here in Christchurch the sun is shining and barbecues and beaches are calling. I am taking a break from the blog for the great New Zealand shut-down and will be back in the New Year.

Thank you for all the followers and especially your comments, Likes and ReTweets.

Teaching time series with limited computer access

How do you teach statistics with limited access to computers?

Last century this wasn’t really an issue, at least not in high schools, as statistics has been a peripheral part of the mathematics curriculum and the mathematics of statistics has been taught as a subset of mathematics.

But this is changing, and it looks as if the change is starting in New Zealand. The NZ school curriculum has leapt ahead of the rest of the world. Statistics is taught at all levels and at the higher levels of high school, statistics is taught as it is actually done in practice – using computers. All analysis is done by a computer package, particularly using iNZight, a purpose-built, free package. The emphasis is on understanding, concepts and critical thinking, rather than the mechanical and slow application of formulas. The rigour has moved from the calculations to the meaning. It is SO exciting!

One big concern for many teachers is access to computers. In many schools there aren’t enough computer suites to schedule the students in for their statistics classes. So how do we deal with this?

It might seem that the computers are needed every day, but in fact they aren’t. And neither is it necessary to have one computer per student.

Make them share

I’ve never had a problem when students have had to share computers. I find the people who do share a computer, learn better than those who are trying to work it out on their own. I actively encourage sharing computers in a lab.

I recently had the opportunity to be on the learning end, with computer instruction. The teacher was showing what to do at the front, and we in the class were echoing her steps on our computers. This is not ideal, as it requires everyone to be at the same pace, but as we were adults it was fine. I hadn’t brought my laptop, so I was sharing with another student. I’m pretty sure I learned more, as I got to follow what was happening on both computers, rather than trying to work it out and keep up. I was also able to help my partner, as she would lose track of what was happening when her computer wasn’t doing what it was meant to.

I have found this to be true at all levels, especially when learning a new package. Having two heads at the computer encourages discussion, which is an important element in learning. Students are also more likely to ask questions when they have already discussed a problem with another student. Pairing is so useful that some software companies get programmers to work in pairs, sharing a computer and work desk, because they have discovered that this has benefits.

Think about what we are trying to teach

I am currently developing resources for a unit in time series analysis, based on the New Zealand curriculum, and using the free software, iNZight. At first glance, you might think that the entire unit would need to be taught in a computer lab. This is definitely not the case. And because of the layout of many computer labs, in fact you are better to stay out of them for most of the unit so that students can work in groups.

I find that it is worthwhile to think about the attitudes, skills and knowledge that we wish our pupils to develop in a unit – in that order of importance! These examples are illustrative rather than exhaustive.

Attitudes – By the end of the topic all students should feel that time series analysis is interesting and relevant (and maybe even fun!).

Time series analysis is pretty straightforward at the beginner level, but can be quite exciting. Once you know the basics, and with a convenient package to speed up the mechanics, you can do some interesting detective work. I would want the students to share some of this excitement, and start to explore on their own.

Skills

Students should be able to:

• identify elements of a time series, relating them to the real life context.
• write a report on a time series analysis using correct terminology, clear enough for a non-expert reader to understand.
• use iNZight to analyse different time series.

Knowledge

• Student should be able to explain and apply the following terms correctly: time series, trend, seasonality, stationary, noise, variation

And that is about it really!

So how do we do this, with or without full computer access.

Even with unlimited computer access I would get students to work in pairs for much of the time. I would start away from the computers. First display graphs of time series to the class and get them to write down sentences about them in their pairs. Then share with the class. We should get sentences like, “It mainly goes up, and then it goes down” and “there is a pattern that repeats”. From that the teacher can introduce the ideas of trend, seasonality and noise, modelling the correct use of specialist language.

Then I would talk about the context – or maybe the context should have come first… The time series chosen should be one with an easy to identify context, such as retail sales of recreational goods, or patterns of tourist arrivals. These series are available in New Zealand at Infoshare or in iNZight format via Statslc. Other countries will have similar series available. Again get the students to write down sentences, this time relating them to the context.

Homework could be to find a graph of a time series on-line or in a magazine. Or to make a list of things that might show seasonality.

Next I would get the students onto the computers in pairs. They should have a worksheet like the one here, so that they can work step-by-step through the package at their own pace in pairs. At some time during the class they could swap roles, if one has been instructing and the other operating.

The data set here RetailNZTS4 has four series in it, which show different behaviours. Students should see if they can get all the graphs they need for further analysis.

Four time series compared using iNZight software

The next class is away from the computers again. Here they are writing sentences about the graphs. They should do this alone, and in pairs, and compare in groups. It would be good to have a computer or two available for students to take turns to get any graphs they might find they need. When people are in front of a computer it tends to dominate their thinking and they can produce far too much output with very little thought. Moving away from the computer encourages a more reflective approach.

Then start on another data set. I would use the one about accommodation, AccRegNZTS13 comparing the seasonal patterns of occupancy in different regions of New Zealand. If there are enough computers, the students can spend one day creating the graphs and exploring, then the next day writing it up. Maybe different groups could take different regions, and find out why the pattern is the way it is for that region, then report back to the class.

Then the teacher may like to give some of the mathematical background to how a computer package would go about producing the output.RetailNZTS4

The learning is in the writing and the talking.

The point I’m trying to make is that you actually need to move away from the computers quite often. If you are REALLY stuck for computers you could even print off (and laminate?) the outputs from the different time series, so that the students can study and discuss them. Number or name them for easy reference, and have question sheets to go with them.The computer is only the tool, and with a bit of creativity, we can still teach the important attitudes, skills and knowledge with limited computer access.

I am aware as I am writing this that it is some time since I taught a class of high school students. I would be thrilled to hear comments from the “chalk-face” as to how realistic you think this is! And of course other suggestions will be welcome for teaching a computer-rich subject in a computer-poor environment.

Having said that, one of my experiences as a trainee teacher was having to teach my first lesson to a class at Rotorua Lakes High School during a powercut – which meant no computers and no OHP. We did desk-checking (how you can use pen and paper to look for bugs in code) and it went surprisingly well.

One year on!

I have been blogging for just under a year now, and have written over 50 posts. There have been over 30,000 hits on the blog, and some very helpful comments. I’ve had a lot of fun, and there is something exciting about thinking that other people might value my thoughts and writing. Thank you all those who have left comments or emailed me.

I spent my morning making up a summary page so that it is easier to find your way around previous posts. It is in the “Collected Works” tab above. In order to do this I had to read (or at least skim) all my previous posts. It was quite interesting really. I hope you find it easier to find what you are looking for.

Here are links to some that took my fancy today:

I am many numbers – a really interesting discussion on the role of numbers in defining who we are. Would work well in class.

Statistics and chocolate – nifty and effective teaching idea for getting across the idea of evidence with respect to probability

Teaching statistical language - how I didn’t get a free iPad

The meaning of the mean - it is trickier than you think

Careers advice in Mathematical Sciences

Mathematics teachers as Careers advisors

What can you do if you are good at mathematics? Become a maths teacher, of course!

I wonder how many of our students are aware of what wonderful and exciting career opportunities are out there for the mathematically competent, including being a mathematics teacher.

I also wonder how many teachers of mathematics, statistics and operations research are telling their students of the different possibilities.

I always loved maths at school and was good at it. I liked teaching, so I decided to be a maths teacher. Along the way, at university, I discovered computer programming and operations research, both subjects that I enjoyed and excelled at. (My conversion to statistics came much later). Given more information at school level, I may well have taken a different path earlier. I didn’t know about engineering or meteorology or surveying, all subjects which need proficiency in mathematics. This may be because I attended a girls’ catholic school where the more able students took languages and I had to study chemistry by correspondence. Let us hope that students this century are better informed.

Teachers are really busy people. I was fortunate last week to attend three different events for mathematics teachers, and was impressed at the dedication they have and their desire to do the best for their pupils.

At university we are having trouble attracting students to the mathematical sciences, yet there is a clear market for graduates, as this article explains: You’re a data what?

There are sites on the internet dedicated to careers information related to mathematics.

The Mathematical Association of America has this link: maa.org/careers/

And the UK has a similar one: mathscareers.org.uk/

A New Zealand site gives suggestions to teachers: CareersNZ

I suspect all it may take is awareness, that the teaching of mathematics and statistics needs to be supplemented with a little careers information. Here are some ideas:

• Have students research mathematically based careers for homework.
• Put vignettes of mathematical workers on the screen for students to read as they enter the classroom.
• Apply the material covered to possible career paths – for example, “if you enjoy interpreting statistical graphs, you might like to work as a data scientist”.
• Use data from a range of mathematical careers as part of data analysis.

I certainly don’t have all the answers, and would love to hear what classroom teachers are doing. There is a wonderful array of information available at the end of a Google Search (which is only possible because of mathematical scientists who continually refine the search engine).

University lecturers need to talk about careers too

The responsibility to inform about careers is not just for school teachers. At University level we sometimes assume the students have a plan about what they are studying and where they would like to end up. I remember one student in particular who had a really clear plan about what he wanted to do. He knew the enrolment handbook better than I. I met him over ten years ago, and this is still a vivid memory, because so many of the students I have taught were unclear about their destination. I’m embarrassed to admit, I probably didn’t help as much as I should have. I think I had this uneasy feeling that I would be seen as “pushing my own barrow”. But heck – if I didn’t think Operations Research and Statistics were important, I should not have been teaching them.

The funny thing is I spent many years making negative comments about the subject of Marketing, seeing it as at time quite harmful. I had a friendly ongoing banter with a wonderful marketing lecturer, John Watson, where we would each poke fun at the others’ subject. And now that I am in the world of business and trying to make a living by selling my apps and on-line courses, I realise that not only do I need to use marketing, but I actually quite enjoy it.  It’s a funny old world!

Information is power, and when we help students learn about possible careers and disciplines we are giving them power to make better choices. And that is important.

The Sound of Music meets Linear Programming

“Let’s start at the very beginning – a very good place to start. When you read you begin with A, B,C!” When you do statistics you begin with…probability? the mean? graphs?

Begin at the end

But really, is the beginning a very good place to start? Sometimes, we need to begin at the end. And sometimes we need to go back before the beginning. Always we need to think about where to begin, because it is seldom obvious, and copying what other teachers and textbooks have done is often a bad idea.

Linear programming

Take Linear Programming, the flagship technique of Operations Research. Most text books start with a simple two variable example, one that can be drawn on a Cartesian plane. They begin by defining the decision variables and the objective function. Next they formulate the constraints and explain the non-negativity conditions. Then finally they get around to solving the problem – often through a graphical approach, and applying it to the trivial real-life imaginary example they started with.

Here is a better approach, with Linear programming as the example:

First ensure all the class members have the prerequisite mathematical skills for what you propose to teach. If they are not good at drawing equations on a plane, you will need to teach them again, or use a different approach such as using Excel Solver. If students are not sure which way around > and < signs go, you will need to go over it. If English is their second language you will need to make sure you explain words like constraint, objective and optimum. This won’t hurt the native English speakers either.

Second think about your destination. When children learn to read, they generally know what the outcome is going to be. They will be able to look at words on a page and make sense of them. When you learn to drive, you know the outcome – you will be able to get safely from one place to another behind the wheel of a car. When we learn to bake cakes, we like to have pictures of the finished product so that we can see where we are headed. Yet somehow we try to teach as if it is a voyage of discovery with no vision of the end. Now discovery is good, if it pertains to how we get to or understand a process, but students need to know what they are learning. It also helps to have a purpose. Reading, driving and baking are all purposeful, with a clear outcome. The same should be true of linear programming (or confidence intervals or decision trees or fitted lines or just about anything else we are learning.)

You give the students an illustration of the completed LP model of the problem, preferably complex enough to be realistic. You show them how it can be useful, and give them a chance to explore the model. This is SO much easier now that we have Excel and Solver to look after the solving. Let students find out all about one model and then another and another, before you begin to show how to formulate. When people know what they are trying to produce, the reasoning behind the steps is more obvious.

Linear Regression

The same approach can be applied to teaching Linear Regression analysis. First we need to make sure that students understand what a fitted line on a graph is. Get them to interpret several fitted graphs, and use them to make predictions and write statements about the nature of the relationships modelled. Then show how to make the fitted graphs once they know why they need to.

In last week’s post I talked about histograms. Students should learn to interpret histograms and other graphs before they are required to make their own. Having to read off pie charts should help immunise them against their use.

I was in a computer lab with some students from another first year statistics course, and noticed that the first thing they were taught was how to calculate the mean and standard deviation, including the finite population correction. Was this really the most interesting way to get them introduced to the joys of data analysis and interpretation? Why start with the mean, one of the most difficult concepts in statistics?

Work backwards from the end

There is an interesting technique used for teaching skills to children with special needs. When you teach a blind child to tie shoelaces, you start at the end. You do all but the last part, and let them finish it off. This gives a sense of success and purpose. Then gradually you add the steps backwards, so that they start earlier on in the process. This also means that the part of the skill that is getting the most repetition is the new part, not the part already mastered. The same is true of memorisation. Memorise the last line first, then the last two lines etc. I suspect the same approach may well apply to more abstract skills. Maybe we should teach how to read and critique a statistical analysis, then how to write one, then finally how to do the analysis.

The spiral approach is popular, in which topics are revisited each year and built on.  I would like to incorporate principles of mastery learning along with that. Mastery learning is based on the premise that you must master a skill before moving on to the next one. This is difficult to implement in a classroom, with mixed level of ability, but is more easily enacted with the help of a Learning Management System.

New math had odd beginnings

I was born in the early 1960s and was in the first cohort of children to learn “new math(s)”, devised in the US as a reaction to the humiliation of seeing the Russians put Sputnik into space before them. Even in New Zealand we were not immune to the influence of the Cold War on education!  I loved our bright new textbooks,  which started with Set Theory – even at age 6. Every year the first page of the text book had diagrams of herds of sheep, prides of lions and other sundry collections.  I loved the Venn diagrams and the intersections – even cardinal numbers, but to this day I’m not sure how that connected with mathematics, and learning to add and subtract. And to this day I ask, “What were they thinking?” It appears that set theory is the foundation of all mathematics, and thus these mathematicians decided to start there, baffling teachers and parents alike, who were alienated by these words and symbols.

I have no doubt that the intention was to improve learning, but it seems ill-advised now. I wonder how our attempts will be viewed with the benefits of 40 years of hindsight. These days constructivism is a popular, though not universal, theory and approach to learning. The idea is that we create knowledge through adding new ideas and experiences onto our current knowledge. Sometimes that involves undoing erroneous or primitive knowledge.

Sometimes a good approach is historical – to imitate in the learner (in an accelerated form) the learning process through which mankind has progressed, preferably missing out the stupid bits. (Roman numerals are fun for some children, but pretty pointless once you realise the power of zero). It is certainly worth contemplating as an alternative approach.

This post has touched on ideas regarding the sequencing of a learning/teaching approach. There are many considerations and serious thought needs to go into where we start. Sometimes we need to start at the end.

Beware of Excel Histograms

Excel histograms are a disgrace. Microsoft should be embarrassed to have them associated with their ubiquitous and generally wonderful spreadsheet, Excel. I have previously posted on how useful and versatile Excel is for enabling people to bypass the number crunching, and get to the ideas behind statistics. This is mostly true. But the histogram add-in should come with a health and safety warning.

To start with, the default look for the histogram is outrageously poor. I have some data on times a person takes to solve a Rogo puzzle. (Collected as part of our research on what factors affect solution time.) I put the data in the spreadsheet, and use the data analysis toolpak to create a histogram using the default settings. Voila!

Histogram produced using default settings in Excel

I’ll stretch it out a bit so you can see it in all its glory:

Histogram using Excel defaults, stretched out.

And here is what can be produced from this with a fair degree of manipulation:

Excel Histogram whipped into shape

This is not just a question of cosmetics. The way the horizontal axis is labelled makes it very difficult to read correctly an Excel-produced histogram unless the adaptation shown above is used. And sometimes, an Excel-produced histogram is just plain incorrect.

Bad histogram in NCEA sample exam

What prompted this tirade is a question from the sample external examination question in NCEA level 2 “Apply probability methods in solving problems”. This is an exam that over 15,000 students are likely to take. Fortunately this is only an examplar and not the real thing. It includes a badly labelled histogram, which I am almost certain was made in Excel.

Histogram taken from an exam exemplar

The introduction says: “Ali has a farm in Southland. She records the weights of 32 lambs born on her farm. The results are shown on the histogram above.”

The first question asks: “What proportion of the lambs weighs less than 1.25 kg?”

Go on – work it out.

I can’t answer this for certain. The labelling of the horizontal axis renders this question unanswerable. I suspect the desired answer is one out of 32. To get this answer I assume that the lambs are weighed to a precision of one decimal place, and the numbers under the graph are the inclusive upper bounds for the area above. I make this assumption because I know that is what Excel does. That is two too many assumptions for students in an exam. This is too many assumptions for any graph. Graphs exist in order to communicate, not confuse.

A histogram always has “bins” which cover a range of values. If you went to school last century and learnt to draw them by hand, you would put the boundary number between the bins on the tick mark on the graph that was the boundary between the bins. Intuitive!

A Google search on the word histogram shows most of the histograms with the tick marks at the boundaries, and quite a few using the Excel work around shown above. That is because the only way you can get the number and the tick mark to line up, is to move the tick mark to the centre. An Excel column chart is designed to be a value graph for nominal data, and it is being pressed into service in an unnatural way.

Another example

This is a simple mockup to illustrate

Example A

The question is, how many people scored 3 or less in the test?

It isn’t clear. Did one person score between 0 and 3, and then three between 3 and 6? The data is actually:  0, 0.5, 1, 2, 5, 5.5, 7, 8, 9,10 and the answer is that four people scored three or less. The following histogram shows this.

Example B (same data as Example A)

All it takes is some relabelling and the meaning is clear.

Teaching implications

We thought long and hard about the teaching of histograms within a Business Statistics course. We concluded that any student who is likely to need to produce a histogram in the future, is likely to (ought to?) have a better statistics package than Excel to use. Teaching them this bizarre work-around in Excel is a waste of student time (We decided this after we made students do this in a course.) It is more important for students to be able to interpret histograms correctly, and be aware of the pitfalls of badly labelled histograms. Consequently we taught students to interpret and then critique histograms, rather than construct them themselves, and assessed the same way.

If you ever use a histogram yourself, make sure you do not fall into the pit shown above!

And for those of you who persist in teaching histograms in Excel (or need help yourself in knowing how to do it – hence avoiding said pit), here is a pdf handout.

Drawing_a_histogram_2007

Good luck.

The best outcome would be that Microsoft get their very poor data analysis add-in fixed up, and the world would be a better place. Any chance of that?

Probability, Perception and False Positives

An understanding of probability empowers people to make informed choices in matters of great importance, including health screening, insurance, major weather events and terrorist threat. Unfortunately it has been shown that this understanding of probability eludes even some of our most educated professionals and decision-makers

Perceptions of Probability and Risk

There is a considerable body of work studying people’s perceptions of probability and risk, particularly by Amos Tversky and the Nobel prize-winning Daniel Kahnemann. This has uncovered many systematic errors humans make in judging the relative probabilities of uncertain events. The brain’s tendency to find patterns results in heuristics or rules that have consistent bias. For example, if we have recently experience or even heard of a bad random event, we perceive the probability to be higher than it really is. Having experienced two years of earthquakes in Christchurch, my estimation of the likelihood of an earthquake in other places is markedly increased. I (and many others from here) feel uneasy surrounded by tall buildings, street awnings and unsecured masonry in other cities, particularly Wellington, but even in cities with no known earthquake risk.

Cultural implications

The perception of probability is also found to be cultural. I analysed a probability-based task as part of the National Education Monitoring Project. I found that there was a statistical and practical difference between the responses of ten-year-old Pacific Island students and NZ European students. I hypothesised that different home experiences involving games of chance may have led to this.  Further reading uncovered other research which had identified other cultural differences. In particular, there are cultures in which everything is perceived to be decided by God and there is no chance but rather a lack of knowledge of God’s will.

In fact many things that we perceive to be subject to chance, would not be, if we had perfect knowledge. Increased understanding of weather patterns has made forecasting more reliable, which has reduced the level of uncertainty with regard to the arrival of bad storms like the recent Hurricane Sandy, or to a lesser extent, two heavy snowfalls in Christchurch in 2011. Even a coin toss is, strictly speaking, only a function of the placement of the coin and thumb, the amount of force applied and various other external factors. Because we cannot measure these factors, we are left to assume that the chance of a head or a tail is equal until shown otherwise.

Screening tests

In disease screening we generally do know the figures, and are not relying on subjective judgment as to the probabilities. However the interpretation of the figures is notoriously badly done. There is a great deal of money involved in the screening industry, and it is an emotive area. Neither money nor emotion aids rational decision-making. This is exacerbated by misinterpretation of probabilities, and selective cost-counting.

My eyes were opened to this issue by a keynote address by Gerd Gigerenzer, director at Max Planck Institute for Human Development . There is a very interesting 8 question quiz at the Harding Center. Try it now. http://www.harding-center.com/  (I was very excited to score 100%, but I put that down to having heard the address, and thought seriously about this.) It would be great if you could tell us your score and reaction to the quiz in the comments below.

A week ago Tim Harford wrote about the lack of understanding among physicians in his post, “Why aren’t we doing the maths? – The practical implications of misplaced confidence when dealing with statistical evidence are obvious and worrying.” This problem is not going away. Some of the comments on the post expressed regret that probability questions like these are not part of the school curriculum, and that it is difficult to find resources to learn on-line. In New Zealand a new curriculum is being introduced with a greater emphasis on statistics at all levels. At year 12 knowledge of understanding of risk, particularly using two-way tables, is examined. As we develop materials to help teach this, we will make them available to the general public.

Example

The following link takes you to a pdf of a powerpoint presentation that teaches a step-by-step approach to this: Risk and Screening – step-by-step approach
We have found that this approach is helpful to students.

In particular you need to make sure that the table has “What the test tells us” along the top, and “What is the reality” down the side. You do not have columns or rows saying “Correct” or “incorrect” as this is much more difficult.

At present there is no audio to go with this segment, but we hope it is self-explanatory.

The costs of screening

Just in case you are tempted to think that all screening must be good and more screening must therefore be better, here are some things to think about.

The following article Breast screening is harmful appeared recently and I found it after I had written the rest of this post. I am very excited to read that  “BreastScreen Aotearoa is revising its leaflets to incorporate information about the risks of overdiagnosis”.

Screening is big business. There are the obvious costs of the equipment and staffing, including nurses, doctors, technicians and clerical workers. Added to that is the cost of loss of productivity for the time taken for the test. The test itself may be harmful. The cost of a false positive is considerable, including unnecessary further tests and interventions, some of which do actual harm. When screening is increased to include people at low-risk, the number of false positives increases, which then takes up resources, and can prevent people who really need intervention from getting it. The emotional costs of a false positive are far-reaching, unnecessarily decreasing quality of life, as people lose confidence in their own health and medicine.

More screening can be harmful

Too often lobby groups,with well-intentioned but ill-informed leaders can do harm. This was possibly the case with breast cancer screening in New Zealand. The age of free screening was lowered to include a group for which the test is less accurate, resulting in many more false positives. A correct understanding of probability in the general populace might have prevented this.

What is clear is that information needs to be better explained in order for informed consent to occur.

Which type of error do you prefer?

Mayor Bloomberg is avoiding a Type 2 error

As I write this, Hurricane Sandy is bearing down on the east coast of the United States. Mayor Bloomberg has ordered evacuations from various parts of New York City. All over the region people are stocking up on food and other essentials and waiting for Sandy to arrive. And if Sandy doesn’t turn out to be the worst storm ever, will people be relieved or disappointed? Either way there is a lot of money involved. And more importantly, risk of human injury and death. Will the forecasters be blamed for over-predicting?

Types of error

There are two ways to get this sort of decision wrong. We can do something and find out it was a waste of time, or we can do nothing and wish that we had done something. In the subject of statistics these are known as Type 1 and Type 2 errors. Teaching about Type 1 and Type 2 errors is quite tricky and students often get confused. Does it REALLY matter if they get them around the wrong way? Possibly not, but what really does matter is that students are aware of their existence. We would love to be able to make decisions under certainty, but most decisions involve uncertainty, or risk. We have to choose between the possibility of taking an opportunity and finding out that it was a mistake, and the possibility of turning down an opportunity and missing out on something.

Earthquake prediction

In another recent event, Italian scientists have been convicted of manslaughter for failing to predict a catastrophic earthquake. This has particular resonance in Christchurch as our city has recently been shaken by several large quakes and a multitude of smaller aftershocks. You can see a graph of the Christchurch shakes at this site. In most part the people of Christchurch understand that it is not possible to predict the occurrence of earthquakes. However it seems that the scientists in Italy may have overstated the lack of risk. Just because you can’t accurately predict an earthquake, it doesn’t mean it won’t happen. Here is a link to a story by Nature of the Italian earthquake.

Tornado warnings

Laura McLay wrote a very interesting post entitled. “what is the optimal false alarm rate for tornado warnings?” . A high rate of false alarms is likened to the “boy who cried wolf”, to whom nobody listens any more. You would think that there is no harm in warning unnecessarily, but in the long term there is potential loss of life because people fail to heed subsequent warnings.

Operations Research and errors

Pure mathematicians tend not to like statistics much as it isn’t exact. It’s a little bit sullied by its contact with the real world. However Operations Research goes a step further into the messy world of reality and evaluates the cost of each type of error. Decisions are often converted into dollar terms within decision analysis. Like it or not, the dollar is the usual measure of worth, even for a human life, though sometimes a measure called “utility” is employed.

Costs of Errors

Sometimes there is very little cost to a type 2 error. A bank manager refusing to fund a new business is avoiding the risk of a type 1 error, which would result in a loss of money. They then become open to at type 2 error, that they missed out on funding a winner. The balance is very much on the side of avoiding a type 1 error. In terms of choosing a life partner, some people are happy to risk a type 1 error, and marry, while others, hold back, perhaps invoking a type 2 error by missing out on a “soul-mate”. Or it may be that we make this decision under the illusion of certainty and perfect information, and the possible errors do not cross our minds.

Cancer screening is a common illustration of type 1 and type 2 errors. With a type 1 error, we get a false positive and are told we have a cancer when we do not. With type 2, the test fails to detect a cancer. In this example the cost of a type 2 error seems to be much worse than type 1. Surely we would rather know if we have cancer? However in the case of prostate cancer, a type 1 error can lead to awful side-effects from unnecessary tests. Conversely a large number of men die from other causes, happily unaware that they have early stages of prostate cancer.

The point is that there is no easy answer when making such decisions.

Teaching about type 1 and type 2 errors

I have found the following helpful when teaching about type 1 and type 2 errors in statistics. Think first about the action that was taken. If the null hypothesis was rejected, we have said that there is an effect. After rejecting the null only two outcomes are possible. We have made the correct decision, or we have made a type 1 error. Conversely if we do not reject the null hypothesis, and do nothing, we have either been correct or made a type 2 error. You cannot make a type 1 error and a type 2 error in the same decision.

• Decision:Reject the Null. Outcome is:
  • Correct or
  • Type 1 error
• Decision:Do not reject the Null. Outcome is:
  • Correct or
  • Type 2 error.

Or another way of looking at it is:

• Do something and get it wrong – Type 1 error
• Do nothing and regret it – Type 2 error

Avoid error

Students may wonder why we have to have any kind of error. Can we not do something to remove error? In some cases we can – we can spend more money and take a larger sample, thus reducing the likelihood of error. However, that too has its cost. The three costs are important aspects of decision-making, and helping students to understand this will help them to make and understand decisions.

The Golden Rule doesn’t apply to teaching

Problems with the Golden Rule

The Golden rule is fundamental to most human cultures.

It reads, approximately, “Do unto others as you would have others do unto you,” or “Treat others as you would like to be treated.”

It sounds good at first glance, and I hesitate to argue with the wisdom of God and most cultures but I propose that the Golden Rule fails if applied mechanistically.

I’ll explain with some examples, and show why this is important in teaching, especially subjects like statistics and operations research.

Not long after after we were married, my husband was lying sick in bed. When I am sick I tend to get lonely, so I like someone to sit with me, or read to me, or visit at least occasionally. So I did that. A while later I was sick, and Mark didn’t come near me, leaving me to myself and my misery. I was lonely and a bit hurt. At birthdays I would give him caramello and dairy milk chocolate, and he would give me almond and peanut chocolate. The problem was that Mark liked to be left in peace when he was sick, so felt that was the best way to help me (and wished I would go away!) and I don’t like nut chocolate any more than he likes caramello. The problem was that we were doing for each other EXACTLY what we would like to be done for us.

Generally the people who are teachers succeeded, and enjoyed being taught. The teaching methods they espouse worked for them so they do unto others as they were done unto. The problem is that most teachers of statistics and operations research are not like most of our students. For instance I loved mathematics at school. The teaching method of giving pupils notes to copy off the board, followed by exercises from the textbook worked just fine for me. I usually understood it all, and had the confidence to persist or ask if I didn’t. I liked getting things right. So using the Golden Rule at a mechanistic level would suggest that I should do this for my students. But for many of our students – dare I say all – this isn’t really the best way to proceed.

In a response to my previous post one person commented that they were shy and didn’t like small groups. So he would not like to teach that way. I hated group work for other reasons, so would seldom use it, if I were just trying to give the students what I would want. However some students, particularly ones from more community-focussed cultures prefer to work in groups rather than stand out from the crowd.

The Golden Rule needs to be applied at a higher level than a mechanistic “this is what what works for me, therefore I will do the same for other people”.

I think we do understand this really, otherwise there would be more hardware stores advertising gifts for Mothers’ Day and jewellery stores with gifts for Fathers’ Day.

The point is not that we do unto others exactly as we would like them to do unto us, down to the details. Rather we would like people to care enough to work out (or ask) what we would like, and then do that, rather than assuming that what works for them works for us. Maybe we should rephrase the Golden Rule to be “Do unto others as you would others do unto you if you were them.”

Do unto others as you would others do unto you if you were them.

Eventually Mark and I worked out what was going on and modified our behaviour. I leave him alone when he is sick and he checks up on me from time to time. I buy him nut chocolate and he buys me Caramello. (Or tropical fruit if I am on Weight Watchers)

Sometimes it can be very difficult to find out what it is that people want. As a parent I had two children who could not have been much less like me. They don’t even resemble me in looks, except for my height. Rules and methods that worked for me just did not work for them. We had to think really hard and laterally to try to put ourselves in their place

People on the autistic spectrum struggle enormously with this principle. Many of them would rather not talk to people unless they have to, and then want to talk about what interests them, such as the history of Sesame Street, or trains, or cross-channel ferries. They would like people to talk to them about these things, so even if they can work out the Golden Rule, it would mean that other people must also want to talk about that.

We need to have enough imagination and will to put ourselves in their position and try to give them what they want. We need to research and share ideas to better understand what it is that others want to help them to learn.

The Golden Rule and nutrition

But even then, what people think they want, and what they really need, can be poles apart. This is clearly evident with many children’s eating choices. Given the choice they may avoid balanced nutrition as much as possible, with less than desirable outcomes. (I’ve read theories that children left to themselves will choose a balanced diet, but I’m sceptical.) Conscientious parents do NOT obey the Golden Rule and treat the children as they would like to be treated, giving them chocolate and chips at every mean, but rather they try to make sure the children  have a balanced diet to help them grow strong. One of my sons once thanked me for not giving him everything he wanted. He had reached the age when he realised that wants may not be for what we need.

Some students would like us to show them every step of a process so that they do not need to struggle and make the effort to learn. They feel that exploring and learning for themselves is a waste of time. At times some students actually do need that, to have a lot of “scaffolding” to help them to learn how to learn, and to gain confidence in their ability. But the scaffolding needs to be gradually removed so that they then become independent learners.

So now we modify the Golden Rule to:

“Do unto others as you would others do unto you if you were them and you had perfect understanding of consequences.”

So really, how the Golden Rule applies to teaching is that we need to do unto our students the best that we can to help them achieve their potential, bearing in mind their individual characteristics, abilities and tendencies. Isn’t that what we would like other people to do for us?

Optimal instruction in Statistics and Operations Research

Optimise everything!

I had a colleague who believed that everything could and should be optimised. He had a diet Linear Program which he used to plan his meals to provide optimal nutrition. Unfortunately the Linear Program didn’t seem to have a constraint to ensure the food was palatable, and he would eat combinations like sardines, broccoli and sunflower seeds for lunch. My colleague also believed that there must be an optimal way to teach, that would maximise the learning outcome. I am doubtful that there is such a thing, bearing in mind the diversity of human experience. However I like the idea of imagining the best possible method, unconstrained by class-size, resources or instructor time and competence.

I am currently developing on-line resources to help people learn statistics, and though I do not presume to claim that they will be optimal, I’d like for them to be really, really good. On-line provision can never do some of the things that human interaction can do, but by the same token, on-line materials are infinitely patient and calm, and should always be correct.

The problem with education and learning is that it involves people. People are not all the same and what works for one person may not work with another. Circumstances change. And there is the problem of motivation. In order for students to learn, they must engage with the material for an extended length of time. This requires motivation. Some students are intrinsically motivated, and will learn for the sheer joy of it. This intrinsic motivation is not common in operations research, and even less common in statistics. Surprisingly, most of our students are not enrolled in a statistics class for the intrinsic rewards it brings and the excitement that the subject engenders.

Sausage Survey

When I was in my second year of high school there was a city-wide mathematics competition, which included a class project. I decided our class should enter and set about running a statistical survey on, of all things, sausages. My maths teacher gave me the resources I needed and ran off the questionnaires on the Banda machine (mmm methylated spirits!) (Showing my age here, I know! – at least I was a student and not the teacher in this story). We had about three or four pages of questionnaire about what types of sausages people liked, how often they ate them, what they ate them with and other questions that I can’t remember. Each girl took home some of the questionnaires and had friends and family fill them out. (No informed consent was needed in those days). My friends and I counted the responses by hand and gave the summary values to class members to draw barcharts and (I’m embarrassed to admit) pie charts. An artistic friend drew a really great illustration in the centre, and we stuck all our graphs onto a large piece of paper – about 2m square. We didn’t win a prize, but we did get an honourable mention, and I got my photo in the school magazine. I do remember learning that our way of counting up did not allow for multivariate analysis. I would have liked to look at the relationships between our variables, but it would have taken too long to recount everything.

The photo from the School Magazine 1975. Dr Nic is second from the left.

Was that an optimal learning experience? Not really. I was motivated, and had fun, and learned things, but I could have learned a lot more with some more guidance from the teacher. To be fair I was possibly too stubborn to let her help us. I didn’t even know that I was doing a statistical analysis at the time, and didn’t encounter the subject of statistics until university.

Various methods of teaching and learning have gone in and out of fashion over the years. Someone once told me, “Never run after a bus, a woman or an educational theory, because there will be another one along soon.” A trifle sexist, though not as bad as “A woman, a dog and a walnut tree, the more you beat them the better they be,” which appeared in my school recipe book. But I digress.

My experience with the sausage survey was an example of Discovery Learning. It is a great way to learn if you have lots of time to make all the mistakes everyone else has ever made on the way to gaining knowledge. I’m quite keen on guided discovery, myself, and think it may be getting closer to optimality! Guided discovery reduces the number of dead-ends down which learners will travel, while helping them construct their own (correct) meaning from the experience.

I had a great experience learning to program at university. I had a friend who was a year ahead of me, majoring in computer science, so I had a tutor on tap. I would set about writing the assigned program, and when I got stuck I would ask Nick. He would give me just enough help to get me going again, and the odd hint if I were using a clumsy algorithm. Was that optimal? Maybe I should have been searching more to find the answers. It was pretty efficient, and as I rather fancied Nick, it was socially rewarding as well. I find programming is more intrinsically motivating than most learning. You write your program and run it, and there is nothing more exciting than seeing it work. It gives feedback of success, though the error statements are less illuminating.

Optimal learning

The optimal learning method should involve small groups of learners. As people discuss ideas they develop their own understanding. There needs to be an expert around who can help them when they get stuck and point them in the right direction. This sounds like the Oxbridge model, where students meet with tutors in very small groups and then go off for a week to do their learning. A week might be a bit too long in this instance.

So my unconstrained optimal solution involves small groups of 2 or 3 students being given projects, preferably related to their personal interests. They grapple with this project, while maintaining contact with a helpful and all-knowing tutor. Resources are available for analysis and for reference. The projects are carefully selected to provide different issues to help students develop the required competencies. The students present their project results to other students. Sounds wonderful. And expensive.

The constraints

• How can we apply this in a class of 30, 100 or 600 students?
• How do we incorporate testing and assessment in order to provide a grade?
• How do we provide such a service on a limited budget?

We examine the aspects that make this setup ideal, and try to replicate them in other ways. In particular we can think how technology can be used to enable this kind of learning. In another post I will examine how Coursera has approached teaching statistics, and see what is done well and how that can be improved.