# I’ve been thinking lately….

Sometimes it pays to stop and think. I have been reading a recent textbook for mathematics teachers, Dianne Siemon et al, Teaching mathematics: foundations to middle years (2011). On page 47 the authors asked me to “Take a few minutes to write down your own views about the nature of mathematics, mathematics learning and mathematics teaching.” And bearing in mind I see statistics as related to, but not enclosed by mathematics, I decided to do the same for statistics as well. So here are my thoughts:

# The nature of mathematics

Mathematicians love the elegance of mathematics

Mathematics is a way of modelling and making sense of the world. Mathematics underpins scientific and commercial endeavours as well as everyday life. Mathematics is about patterns and proofs and problem structuring and solution finding. I used to think it was all about the answer, but now I think it is more about the process. I used to think that maths was predominantly an individual endeavour, but now I can see how there is a social or community aspect as well. I fear that too often students are getting a parsimonious view of mathematics, thinking it is only about numbers, and something they have to do on their own. I find my understanding of the nature of mathematics is rapidly changing as I participate in mathematics education at different ages and stages. I have also been influenced by the work of Jo Boaler.

# To learn mathematics

My original idea of mathematics learning comes from my own successful experience of copying down notes from the board, listening to the teacher and doing the exercises in the textbook. I was not particularly fluent with my times-tables, but loved problem-solving. If I got something wrong, I was happy to try again until I nutted it out. Sometimes I even did recreational maths, like the time I enumerated all possible dice combinations in Risk to find out who had the advantage – attacker or defender. I always knew that it took practice to be good at mathematics. However I never really thought of mathematics as a social endeavour. I feel I missed out, now. From time to time I do have mathematical discussions with my colleague. It was an adventure inventing Rogo and then working out a solution method. Mathematics can be a social activity.

# To teach mathematics

When I became a maths teacher I perpetuated the method that had worked for me, as I had not been challenged to think differently. I did like the ideas of mastery learning and personalised system of instruction. This meant that learners progressed to the next step only when they had mastered the previous one. I was a successful enough teacher and enjoyed my work.

Then as a university lecturer I had to work differently, and experimented. I had a popular personalised system of instruction quantitative methods course, relying totally on students working individually, at their own pace. I am happy that many of my students were successful in an area they had previously thought out of their reach. For some students it was the only subject they passed.

# What I would do now

If I were to teach mathematics at school level again, I hope I would do things differently. I love the idea of “Number talks” and rich tasks which get students to think about different ways of doing things. I had often felt sad that there did not seem to be much opportunity to have discussions in maths, as things were either right or wrong. Now I see what fun we could have with open-ended tasks. Maths learning should be communal and loud and exciting, not solitary, quiet and routine. I have been largely constructivist in my teaching philosophy, but now I would like to try out social constructivist thinking.

# Statistics

And what about statistics? At school in the 1970s I never learned more than the summary statistics and basic probability. At uni level it was bewildering, but I managed to get an A grade in a first year paper without understanding any of the basic principles. It wasn’t until I was doing my honours year in Operations Research and was working as a tutor in Statistical methods that things stared to come together – but even then I was not at home with statistical ideas and was happy to leave them behind when I graduated.

# The nature of statistics

Statistics lives in the real world

My views now on the nature of statistics are quite different. I believe statistical thinking is related to mathematical thinking, but with less certainty and more mess. Statistics is about models of reality, based on imperfect and incomplete data. Much of statistics is a “best guess” backed up by probability theory. And statistics is SO important to empowered citizenship. There are wonderful opportunities for discussion in statistics classes. I had a fun experience recently with a bunch of Year 13 Scholarship students in the Waikato. We had collected data from the students, having asked them to interpret a bar chart and a pie chart. There were some outliers in the data and I got them to suggest what we should do about them. There were several good suggestions and I let them discuss for a while then moved on. One asked me what the answer was and I said I really couldn’t say – any one of their suggestions was valid. It was a good teaching and learning moment. Statistics is full of multiple good answers, and often no single, clearly correct, answer.

# Learning statistics

My popular Quantitative Methods for Business course was developed on the premise that learning statistics requires repeated exposure to similar analyses of multiple contexts. In the final module, students did many, many hypothesis tests, in the hope that it would gradually fall into place. That is what worked for me, and it did seem to work for many of the students. I think that is not a particularly bad way to learn statistics. But there are possibly better ways.

I do like experiential learning, and statistics is perfect for real life experiences. Perhaps the ideal way to learn statistics is by performing an investigation from start to finish, guided by a knowledgeable tutor. I say perhaps, because I have reservations about whether that is effective use of time. I wrote a blog post previously, suggesting that students need exposure to multiple examples in order to know what in the study is universal and what applies only to that particular context. So perhaps that is why students at school should be doing an investigation each year within a different context.

# The nature of understanding

This does beg the question of what it means to learn or to understand anything. I hesitate to claim full understanding. Of anything. Understanding is progressive and multi-faceted and functional. As we use a technique we understand it more, such as hypothesis testing or linear programming. Understanding is progressive. My favourite quote about understanding is from Moore and Cobb, that “Mathematical understanding is not the only understanding.” I do not understand the normal distribution because I can read the Gaussian formula. I understand it from using it, and in a different way from a person who can derive it. In this way my understanding is functional. I have no need to be able to derive the Gaussian function for what I do, and the nature and level of my understanding of the normal distribution, or multiple regression, or bootstrapping is sufficient for me, for now.

# Teaching statistics

I believe our StatsLC videos do help students to understand and learn statistics. I have put a lot of work into those explanations, and have received overwhelmingly positive feedback about the videos. However, that is no guarantee, as Khan Academy videos get almost sycophantic praise and I know that there are plenty of examples of poor pedagogy and even error in them. I have recently been reading from “Make it Stick”, which summarises theory based on experimental research on how people learn for recall and retention. I was delighted to find that the method we had happened upon in our little online quizzes was promoted as an effective method of reinforcing learning.

This has been an enlightening exercise, and I recommend it to anyone teaching in mathematics or statistics. Read the first few chapters of a contemporary text on how to teach mathematics. Dianne Siemon et al, Teaching mathematics: foundations to middle years (2011) did it for me. Then “take a few minutes to write down your own views about the nature of mathematics, mathematics learning and mathematics teaching.” To which I add my own suggestion to think about the nature of statistics or operations research. Who knows what you will find out. Maybe you could put a few of your ideas down in the comments.

# Divide and destroy in statistics teaching

## A reductionist approach to teaching statistics destroys its very essence

I’ve been thinking a bit about systems thinking and reductionist thinking, especially with regard to statistics teaching and mathematics teaching. I used to teach a course on systems thinking, with regard to operations research. Systems thinking is concerned with the whole. The parts of the system interact and cannot be isolated without losing the essence of the system. Modern health providers and social workers realise that a child is a part of a family, which may be a part of a larger community, all of which have to be treated if the child is to be helped. My sister, a physio, always finds out about the home background of her patient, so that any treatment or exercise regime will fit in with their life. Reductionist thinking, by contrast, reduces things to their parts, and isolates them from their context.

## Reductionist thinking in teaching mathematics

Mathematics teaching lends itself to reductionist thinking. You strip away the context, then break a problem down into smaller parts, solve the parts, and then put it all back together again. Students practise solving straight-forward problems over and over to make sure they can do it right. They feel that a column of little red ticks is evidence that they have learned something correctly. As a school pupil, I loved the columns of red ticks. I have written about the need for drill in some aspects of statistics teaching and learning, and can see the value of automaticity – or the ability to answer something without having to think too hard. That can be a little like learning a language – you need to be automatic on the vocabulary and basic verb structures. I used to spend my swimming training laps conjugating Latin verbs – amo, amas, amat (breathe), amamus, amatis, amant (breathe). I never did meet any ancient Romans to converse with, to see if my recitation had helped any, but five years of Latin vocab is invaluable in pub quizzes. But learning statistics has little in common with learning a language.

There is more to teaching than having students learn how to get stuff correct. Learning involves the mind, heart and hands. The best learning occurs when students actually want to know the answer. This doesn’t happen when context has been removed.

I was struck by Jo Boaler’s, “The Elephant in the Classroom”, which opened my eyes to how monumentally dull many mathematics lessons can be to so many people. These people are generally the ones who do not get satisfied by columns of red ticks, and either want to know more and ask questions, or want to be somewhere else. Holistic lessons, that involve group work, experiential learning, multiple solution methods and even multiple solutions, have been shown to improve mathematics learning and results, and have lifelong benefits to the students. The book challenged many of my ingrained feelings about how to teach and learn mathematics.

## Teach statistics holistically, joyfully

Teaching statistics is inherently suited for a holistic approach. The problem must drive the model, not the other way around. Teachers of mathematics need to think more like teachers of social sciences if they are to capture the joy of teaching and learning statistics.

At one time I was quite taken with an approach suggested for students who are struggling, which is to go step-by-step through a number of examples in parallel and doing one step, before moving on to the next step. The examples I saw are great, and use real data, and the sentences are correct. I can see how that might appeal to students who are finding the language aspects difficult, and are interested in writing an assignment that will get them a passing grade. However I now have concerns about the approach, and it has made me think again about some of the resources we provide at Statistics Learning Centre. I don’t think a reductionist approach is suitable for the study of statistics.

## Context, context, context

Context is everything in statistical analysis. Every time we produce a graph or a numerical result we should be thinking about the meaning in context. If there is a difference between the medians showing up in the graph, and reinforced by confidence intervals that do not overlap, we need to be thinking about what that means about the heart-rate in swimmers and non-swimmers, or whatever the context is. For this reason every data set needs to be real. We cannot expect students to want to find real meaning in manufactured data. And students need to spend long enough in each context in order to be able to think about the relationship between the model and the real-life situation. This is offset by the need to provide enough examples from different contexts so that students can learn what is general to all such models, and what is specific to each. It is a question of balance.

In my effort to help improve teaching of statistics, we are now developing teaching guides and suggestions to accompany our resources. I attend workshops, talk to teachers and students, read books, and think very hard about what helps all students to learn statistics in a holistic way. I do not begin to think I have the answers, but I think I have some pretty good questions. The teaching of statistics is such a new field, and so important. I hope we all keep asking questions about what we are teaching, and how and why.

# Why I am going to ICOTS9 in Flagstaff, Arizona

I was a university academic for twenty years. One of the great perks of academia is the international conference. Thanks to the tax-payers of New Zealand I have visited Vancouver, Edinburgh, Melbourne (twice), San Diego, Fort Lauderdale, Salt Lake City and my favourite, Ljubljana. This is a very modest list compared with many of my colleagues, as I didn’t get full funding until the later years of my employ.

Academic conferences enable university researchers and teachers from all over the world to gather together and exchange ideas and contacts. They range from fun and interesting to mind-bogglingly boring. My first conference was IFORS in Vancouver in 1996, and I had a blast. It helped that my mentor, Hans Daellenbach, was also there, and I got to meet some of the big names in operations research. I have since attended two other IFORS conferences, and it is amazing how connected you can feel to people whom you meet only every few years. I always try to go to most sessions of the conference as I feel an obligation to the people who have paid to have me there. It is unethical to be paid to go to a conference, and then turn up only for a couple of sessions and the banquet. Sometimes sessions that I have only limited connection with can turn out to be interesting. I found I could always listen for the real world application that I could then include in my teaching. That would usually take up the first few minutes of the talk. Once the formulas appeared I would glaze over and go to my happy place. Having said that, I also think mental health breaks are important, and would take time out to reflect. I get more out of conferences if I leave my husband at home. The quiet time in my hotel room was also important for invigorating my teaching and research.

Most academic conferences focus on research, though they often have a teaching stream, which I frequent. ICOTS is different though as it is mostly about teaching, with a research stream! ICOTS stands for International Conference on Teaching Statistics, and runs every four years. I attended my first ICOTS in Slovenia in 2010. What surprised me was how many people there were from New Zealand! At the welcome reception I wandered around introducing myself to people and more often than not found they were also from New Zealand. How ironic to spend 40 hours getting to this amazing place and meet large numbers of fellow kiwis! (Named for the bird, not the fruit!). Ljubljana is a wonderful city, with fantastic architecture and lots of bike routes and geocaches. I made good use of my spare time. The conference itself was inspiring too. I attended just about every session, and gave a paper about making videos to teach statistics. I saw the dance of the p-value, and learned about statistics teaching in some African countries. I was impressed by the keynote by Gerd Gigerenzer, and went home and cancelled my mammogram. I put faces to some of the names in statistics education, though I was sad not to see George Cobb there, or Joan Garfield. What struck me was how nice everyone was. I loved my trip to some caves on the half-day excursion.

The point of this post is to encourage readers to go to ICOTS 9 in July this year. I admit I was a little disappointed when they announced the venue. I was hoping for somewhere a little more exotic. However the great benefit is that it is going to cost considerably less to get there than to many countries, and take less time. (For people from New Zealand and Australia, a trip of less than 24 hours is a bonus.) Now that I am no longer paid by a university to go to conferences, the cost is a big consideration. If necessary I will sell our caravan. Another benefit of the venue is it is very convenient for teachers from the US to attend. I am hoping to find out more about AP statistics, and other US statistics teaching.

I am currently reviewing an edited book published by Springer, Probabilistic Thinking. As I read each chapter I am increasingly excited that most of the authors will be attending ICOTS9. This is a great opportunity to discuss with them their ideas, and how to apply them in the classroom and in our resources. I am particularly interested in the latest research on how children and adults learn statistics and probability. This ICOTS I am doing a presentation about setting up a blog, Twitter and YouTube. In four years’ time I hope to be able to add to the research using what we have learned from students’ responses on our on-line resources.

I am a little apprehensive about the altitude and temperature, but have planned to arrive a few days early in Phoenix to acclimatise myself. In the interests of economy I will be staying at the university dorms, and just found out there is no air-conditioning in the bedrooms. My daughter-in-law from Utah tells me to buy a fan. I’m pretty happy about a trip to the Grand Canyon on the afternoon off.  The names of presenters and their abstracts are now available on the ICOTS9 website, so you can see what interesting times await.

I really hope I see a lot of you there – and not just New Zealanders.

# Deterministic and Probabilistic models and thinking

The way we understand and make sense of variation in the world affects decisions we make.

Part of understanding variation is understanding the difference between deterministic and probabilistic (stochastic) models. The NZ curriculum specifies the following learning outcome: “Selects and uses appropriate methods to investigate probability situations including experiments, simulations, and theoretical probability, distinguishing between deterministic and probabilistic models.” This is at level 8 of the curriculum, the highest level of secondary schooling. Deterministic and probabilistic models are not familiar to all teachers of mathematics and statistics, so I’m writing about it today.

## Model

The term, model, is itself challenging. There are many ways to use the word, two of which are particularly relevant for this discussion. The first meaning is “mathematical model, as a decision-making tool”. This is the one I am familiar with from years of teaching Operations Research. The second way is “way of thinking or representing an idea”. Or something like that. It seems to come from psychology.

When teaching mathematical models in entry level operations research/management science we would spend some time clarifying what we mean by a model. I have written about this in the post, “All models are wrong.”

In a simple, concrete incarnation, a model is a representation of another object. A simple example is that of a model car or a Lego model of a house. There are aspects of the model that are the same as the original, such as the shape and ability to move or not. But many aspects of the real-life object are missing in the model. The car does not have an internal combustion engine, and the house has no soft-furnishings. (And very bumpy floors). There is little purpose for either of these models, except entertainment and the joy of creation or ownership. (You might be interested in the following video of the Lego Parisian restaurant, which I am coveting. Funny way to say Parisian!)

Many models perform useful functions. My husband works as a land-surveyor, and his work involves making models on paper or in the computer, of phenomenon on the land, and making sure that specified marks on the model correspond to the marks placed in the ground. The purpose of the model relates to ownership and making sure the sewers run in the right direction. (As a result of several years of earthquakes in Christchurch, his models are less deterministic than they used to be, and unfortunately many of our sewers ended up running the wrong way.)

Our world is full of models:

• a map is a model of a location, which can help us get from place to place.
• sheet music is a written model of the sound which can make a song
• a bus timetable is a model of where buses should appear
• a company’s financial reports are a model of one aspect of the company

## Deterministic models

A deterministic model assumes certainty in all aspects. Examples of deterministic models are timetables, pricing structures, a linear programming model, the economic order quantity model, maps, accounting.

## Probabilistic or stochastic models

Most models really should be stochastic or probabilistic rather than deterministic, but this is often too complicated to implement. Representing uncertainty is fraught. Some more common stochastic models are queueing models, markov chains, and most simulations.

For example when planning a school formal, there are some elements of the model that are deterministic and some that are probabilistic. The cost to hire the venue is deterministic, but the number of students who will come is probabilistic. A GPS unit uses a deterministic model to decide on the most suitable route and gives a predicted arrival time. However we know that the actual arrival time is contingent upon all sorts of aspects including road, driver, traffic and weather conditions.

## Model as a way of thinking about something

The term “model” is also used to describe the way that people make sense out of their world. Some people have a more deterministic world model than others, contributed to by age, culture, religion, life experience and education. People ascribe meaning to anything from star patterns, tea leaves and moon phases to ease in finding a parking spot and not being in a certain place when a coconut falls. This is a way of turning a probabilistic world into a more deterministic and more meaningful world. Some people are happy with a probabilistic world, where things really do have a high degree of randomness. But often we are less happy when the randomness goes against us. (I find it interesting that farmers hit with bad fortune such as a snowfall or drought are happy to ask for government help, yet when there is a bumper crop, I don’t see them offering to give back some of their windfall voluntarily.)

Let us say the All Blacks win a rugby game against Australia. There are several ways we can draw meaning from this. If we are of a deterministic frame of mind, we might say that the All Blacks won because they are the best rugby team in the world.  We have assigned cause and effect to the outcome. Or we could take a more probabilistic view of it, deciding that the probability that they would win was about 70%, and that on the day they were fortunate.  Or, if we were Australian, we might say that the Australian team was far better and it was just a 1 in 100 chance that the All Blacks would win.

I developed the following scenarios for discussion in a classroom. The students can put them in order or categories according to their own criteria. After discussing their results, we could then talk about a deterministic and a probabilistic meaning for each of the scenarios.

1. The All Blacks won the Rugby World Cup.
2. Eri did better on a test after getting tuition.
3. Holly was diagnosed with cancer, had a religious experience and the cancer was gone.
4. A pet was given a homeopathic remedy and got better.
5. Bill won \$20 million in Lotto.
6. You got five out of five right in a true/false quiz.

The regular mathematics teacher is now a long way from his or her comfort zone. The numbers have gone, along with the red tick, and there are no correct answers. This is an important aspect of understanding probability – that many things are the result of randomness. But with this idea we are pulling mathematics teachers into unfamiliar territory. Social studies, science and English teachers have had to deal with the murky area of feelings, values and ethics forever.  In terms of preparing students for a random world, I think it is territory worth spending some time in. And it might just help them find mathematics/statistics relevant!

# Twenty-first century Junior Woodchuck Guidebook

I grew up reading Donald Duck comics. I love the Junior Woodchucks, and their Junior Woodchuck Guidebook. The Guidebook is a small paperback book, containing information on every conceivable subject, including geography, mythology, history, literature and the Rubaiyat of Omar Khayyam.  In our family, when we want to know something or check some piece of information, we talk about consulting the Junior Woodchuck Guidebook. (Imagine my joy when I discovered that a woodchuck is another name for a groundhog, the star of my favourite movie!) What we are referring to is the internet, the source of all possible information! Thanks to search engines, there is very little we cannot find out on the internet. And very big thanks to Wikipedia, to which I make an annual financial contribution, as should all who use it and can afford to.

You can learn just about anything on the internet. Problem is, how do you know what is good? And how do you help students find good stuff? And how do you use the internet wisely? And how can it help us as learners and teachers of statistics and operations research? These questions will take more than my usual 1000 words, so I will break it up a bit. This post is about the ways the internet can help in teaching and learning. In a later post I will talk about evaluating resources, and in particular multimedia resources.

## Context

Both the disciplines in which I am interested, statistics and operations research, apply mathematical and analytic methods to real-world problems. In statistics we are generally trying to find things out, and in operations research we are trying to make them better. Either way, the context is important. The internet enables students to find background knowledge regarding the context of the data or problem they are dealing with. It also enables instructors to write assessments and exercises that have a degree of veracity to them even if the actual raw data proves elusive. How I wish people would publish standard deviations as well as means when reporting results!

## Data

Which brings us to the second use for on-line resources. Real problems with real data are much more meaningful for students, and totally possible now that we don’t need to calculate anything by hand. Sadly, it is more difficult than first appears to find good quality raw data to analyse, but there is some available. You can see some sources in a previous post and the helpful comments.

## Explanations

If you are struggling to understand a concept, or to know how to teach or explain it, do a web search. I have found some great explanations, and diagrams especially, that have helped me. Or I have discovered a dearth of good diagrams, which has prompted me to make my own.

## Video

Videos can help with background knowledge, with explanations, and with inspiring students with the worth of the discipline. The problem with videos is that it takes a long time to find good ones and weed out the others. One suggestion is to enlist the help of your students. They can each watch two or three videos and decide which are the most helpful. The teacher then watches the most popular ones to check for pedagogical value. It is great when you find a site that you can trust, but even then you can’t guarantee the approach will be compatible with your own.

## Social support

I particularly love Twitter, from which I get connection with other teachers and learners, and ideas and links to blogs. I belong to a Facebook group for teachers of statistics in New Zealand, and another Facebook group called “I love Operations Research”. These wax and wane in activity, and can be very helpful at times. Students and teachers can gain a lot from social networking.

## Software

There is good open-source software available, and 30-day trial versions for other software. Many schools in New Zealand use the R-based iNZight collection of programmes, which provide purpose-built means for timeseries analysis, bootstrapping and line fitting.

The other day I lost the volume control off my toolbar. (Windows Vista, I’m embarrassed to admit). So I put in the search box “Lost my volume control” and was directed to a YouTube video that took me step-by-step through the convoluted process of reinstating my volume control! I was so grateful I made a donation. Just about any computer related question can be answered through a search.

## Interactive demonstrations

I love these. There are two sites I have found great:

The National Library of Virtual Manipulatives, based in Utah.

A problem with some of these is the use of Flash, which does not play on all devices. And again – how do we decide if they are any good or not?

## On-line textbooks

Why would you buy a textbook when you can get one on-line. I routinely directed my second-year statistical methods for business students to “Concepts and Applications of Inferential Statistics”. I’ve found it just the right level. Another source is Stattrek. I particularly like their short explanations of the different probability distributions.

## Practice quizzes

There aren’t too many practice quizzes  around for free. Obviously, as a provider of statistical learning materials, I believe quizzes and exercises have merit for practice with immediate and focussed feedback. However, it can be very time-consuming to evaluate practice quizzes, and some just aren’t very good. On the other hand, some may argue that any time students spend learning is better than none.

## Live help

There are some places that provide live, or slightly delayed help for students. I got hooked into a very fun site where you earned points by helping students. Sadly I can’t find it now, but as I was looking I found vast numbers of on-line help sites, often associated with public libraries. And there are commercial sites that provide some free help as an intro to their services. In New Zealand there is the StudyIt service, which helps students preparing for assessments in the senior high school years. At StatsLC we provide on-line help as part of our resources, and will be looking to develop this further. From time to time I get questions as a result of my YouTube videos, and enjoy answering them ,unless I am obviously doing someone’s homework! I also discovered “ShowMe” which looks like a great little iPad app, that I can use to help people more.

This has just been a quick guide to how useful the internet can be in teaching and learning. Next week I will address issues of quality and equity.

# Parts and whole

The whole may be greater than the sum of the parts, but the whole still needs those parts. A reflective teacher will think carefully about when to concentrate on the whole, and when on the parts.

## Golf

If you were teaching someone golf, you wouldn’t spend days on a driving range, never going out on a course. Your student would not get the idea of what the game is, or why they need to be able to drive straight and to a desired length. Nor would it be much fun! Similarly if the person only played games of golf it would be difficult for them to develop their game. Practice driving and putting is needed.  A serious student of golf would also read and watch experts at golf.

## Music

Learning music is similar. Anyone who is serious about developing as a musician will spend a considerable amount of time developing their technique and their knowledge by practicing scales, chords and drills. But at the same time they need to be playing full pieces of music so that they feel the joy of what they are doing. As they play music, as opposed to drill, they will see how their less-interesting practice has helped them to develop their skills. However, as they practice a whole piece, they may well find a small part that is tripping them up, and focus for a while on that. If they play only the piece as a whole, it is not efficient use of time. A serious student of music will also listen to and watch great musicians, in order to develop their own understanding and knowledge.

## Benefits of study of the whole and of the parts

In each of these examples we can see that there are aspects of working with the whole, and aspects of working with the parts. Study of the whole contributes perspective and meaning to study, and helps to tie things together. It helps to see where they have made progress. Study of the parts isolates areas of weakness, develops skills and saves time in practice, thus being more efficient.

It is very important for students to get an idea of the purpose of their study, and where they are going. For this reason I have written earlier about the need to see the end when starting out in a long procedure such as a regression or linear programming model.

It is also important to develop “statistical muscle memory” by repeating small parts of the exercise over and over until it is mastered. Practice helps people to learn what is general and what is specific in the different examples.

# Teaching conditional probability

We are currently developing a section on probability as part of our learning materials. A fundamental understanding of probability and uncertainty are essential to a full understanding of inference. When we look at statistical evidence from data, we are holding it up against what we could reasonably expect to happen by chance, which involves a probability model. Probability lies in the more mathematical area of the study of statistics, and has some fun problem-solving aspects to it.

A popular exam question involves conditional probability. We like to use a table approach to this as it avoids many of the complications of terminology. I still remember my initial confusion over the counter-intuitive expression P(A|B) which means the probability that an object from subset B has the property of A. There are several places where students can come unstuck in Bayesian review, and the problems can take a long time. We can liken solving a conditional probability problem to a round of golf, or a long piece of music. So what we do in teaching is that first we take the students step by step through the whole problem. This includes working out what the words are saying, putting the known values into a table, calculating the unknown values in the table, and the using the table to answer the questions involving conditional probability.

Then we work on the individual steps, isolating them so that students can get sufficient practice to find out what is general and what is specific to different examples. As we do this we endeavour to provide variety such that students do not work out some heuristic based on the wording of the question, that actually stops them from understanding. An example of this is that if we use the same template each time, students will work out that the first number stated will go in a certain place in the table, and the second in another place etc. This is a short-term strategy that we need to protect them from in careful generation of questions.

As it turns out students should already have some of the necessary skills. When we review probability at the start of the unit, we get students to calculate probabilities from tables of values, including conditional probabilities. Then when they meet them again as part of the greater whole, there is a familiar ring.

Once the parts are mastered, the students can move on to a set of full questions, using each of the steps they have learned, and putting them back into the whole. Because they are fluent in the steps, it becomes more intuitive to put the whole back together, and when they meet something unusual they are better able to deal with it.

## Starting a course in Operations Research/Management Science

It is interesting to contemplate what “the whole” is, with regard to any subject. In operations research we used to begin our first class, like many first classes, talking about what management science/operations research is. It was a pretty passive sort of class, and I felt it didn’t help as first-year university students had little relevant knowledge to pin the ideas on. So we changed to an approach that put them straight into the action and taught several weeks of techniques first. We started with project management and taught critical path. Then we taught identifying fixed and variable costs and break-even analysis. The next week was discounting and analysis of financial projects. Then for a softer example we looked at multi-criteria decision-making, using MCDM. It tied back to the previous week by taking a different approach to a decision regarding a landfill. Then we introduced OR/MS, and the concept of mathematical modelling. By then we could give real examples of how mathematical models could be used to inform real world problems. It was helpful to go from the concrete to the abstract. This was a much more satisfactory approach.

So the point is not that you should always start with the whole and then do the parts and then go back to the whole. The point is that a teacher needs to think carefully about the relationship between the parts and the whole, and teach in a way that is most helpful.

# Why learning objectives are so important

Update in 2017

This is one of the most popular posts on this blog. You may also be interested in a case study of what happens when students do not get learning objectives: Why people hate statistics.

Original post:
The most useful thing I learned in my teacher training at Auckland College of Education in 1985 was to write learning objectives. Not many years, and two babies later, I began lecturing at the University of Canterbury in Management Science/Operations Research. I was the only academic staff member to have formal teacher-training. My first task, when put in charge of MSCI210, Statistical Methods for Management, was to write learning objectives. This was revolutionary, but the idea infiltrated through other courses over the years.

## A learning objective states specifically what a student should be able to do.

Here are some examples of good learning objectives:

Students will be able to:

• Identify different levels of data in new scenarios.
• Explain in context a confidence interval.
• Determine which probability distribution out of binomial, poisson or normal is most appropriate to model in an unfamiliar situation.
• Compare two time series models of the same data and evaluate which is more appropriate in a given context.

## Learning objectives need to be specific and measurable.

Here are some things that people might think are learning objectives, but are not:

• Students will understand the central limit theorem. (The term “understand” is not measurable)
• Students will learn about probability trees (“learn” is not measurable, and does not specify the level. Do students need to be able to interpret or create probability trees?)

## There are vast numbers of resources on learning objectives online.

Here is one I liked, with Bloom’s taxonomy of levels of learning. These are higher and lower levels of learning objectives, ranging from being able to state principles, through to synthesis and evaluation.

http://teachonline.asu.edu/2012/07/writing-measurable-learning-objectives/

And here are some useful verbs to use when writing learning objectives;

http://www.schreyerinstitute.psu.edu/pdf/SampleVerbs_for_LearningObjectives.pdf

It is not difficult to find material on developing learning objectives.

## Not just learning objectives

A course is more than the set of its learning objectives. The learning objectives specify the skills, but there are also attitudes and knowledge to be considered. The starting point for course design is the attitudes. What do we want the students to feel about the topics? What changes do we wish them to contemplate in their thinking? Then the skills and knowledge are specified, often starting at a quite general level, then working down to specifics.

For example, we might wish to teach about confidence intervals. We need to determine whether students need to be able to calculate them, interpret them, estimate or derive them.  We need to decide which confidence intervals we are interested in – for means alone, or proportions and slopes as well? Sometimes I find there are concepts I wish to include in the learning objectives, but they don’t really work as objectives. These I put as “important concepts and principles”.

I have put an example of learning objectives and concepts and principles at the end of this post.

## Learning objectives tell students what is important

Without learning objectives it is difficult for students to know what they are supposed to be learning. In a lecture, a teacher can talk extensively about a case, but unless she states explicitly, it can be difficult for the students to know where to direct their attention. Do they need to know the details of that specific case or what principles are they supposed to glean from the example? Or was it just a “war-story” to entertain the troops? Students can waste a great deal of time studying things that are not necessary, to the detriment of their learning as a whole. The uncertainty also causes unnecessary anxiety.

## Learning objectives enable good assessment development

Each year as we wrote our assessments we would go through the learning objectives and make sure they were assessed.  This way the assessment was fair and applied to the course. If we found it difficult to write a question to assess a learning objective we would think again about the learning objective, and what it is we really want the students to be able to do. It made it easier to write fair, comprehensive assessments.

## Learning objectives encourage reflection and good course design and development

As instructors write and review the learning objectives in a course, they can identify the level of learning that is specified in each. At an entry-level course, it is acceptable to have a number of lower level learning objectives. However, there needs to be some serious thinking done if a post-graduate course is not mainly made up of higher level learning objectives. I have seen tests in stage 2 and 3 papers that tested mainly recall and common-sense. It was evident that the instructor had not thought clearly about the level of learning that was expected.

Sometimes we find we are assessing things we have not specifically taught the students. The use of learning objectives, linked with assessment design, helps us to identify the background knowledge that we assume students have. One colleague was frustrated that the students did not seem able to apply the statistical results to a managerial context. However, nowhere had she specified that students would be required to do so, and nowhere had she actually taught students how to do this. She also assumed a level of understanding of business,  that was probably not appropriate in undergraduate students.

## Like it or not, assessment drives learning

I spoke recently to a maths advisor who informed me that teachers should be teaching to the curriculum not to the assessments. I felt he was idealistic, and told him so. My experience is that university students will learn what is assessed, and nothing else. I don’t know at what age this begins, but I suspect National Testing, the bane of good education, has lowered the age considerably. How wonderful it would be if our students learned for the sheer joy of learning! Where there are assessments looming, I fear this is unlikely.

When we write exams we are also writing learning materials for future students. One of the most common ways to prepare for an assessment is to do exercises from previous assessments. So when we feel that students were not really coming to grips with a concept, we include questions in the assessment, that can then be used by future students for review.

## Information promotes equity and reduces unnecessary stress

The use of learning objectives can help reduce the “gaming” aspects that can proliferate in the absence of clear information. This is apparent at present in the world of Year 13 Statistics in New Zealand. The information regarding the external standards for 2013 is still sketchy (1 July 2013). The exams are written by external examiners and will take place in November of this year. However there is still only vague and sometimes incorrect information as to exactly what may or may not be included in the exams. Because of this, teachers are trying to detect, from what is or isn’t in the formula sheet and the (not totally correct) exemplars what might be in the finals, and what to include in the school practice exams.  I suspect that some teachers or areas have more information than others. The way to make this fairer is to specify what is included in the material that may be included, as learning objectives. Let us hope that some clarity comes soon, for the sake of the teachers and the students.

So what were the learning objectives for this post?

• Reflect on their methods of course development and assessment with respect to using learning objectives.
• find further resources on the internet regarding learning objectives.
• Make comments on the good and bad aspects of this post! (oops – I didn’t teach that one)

## Coda – Example of some learning objectives

Here is a set of learning objectives for the final section of a service course in quantitative methods for management. It is based on Excel and traditional (normal-based) methods of statistical analysis.  They are far from perfect, including several ideas in many of them.

Evidence Section Learning Objectives

E1.   Explain the process underlying hypothesis tests.
E2.   Interpret a p-value in context for a given set of hypotheses.
E3.   Formulate a null and alternative hypothesis in words for problems involving means, proportions, differences of two means and differences of two proportions.
E4.   Use Excel to perform a hypothesis test on one or two means and interpret the results.
E5.   Use Excel to perform a hypothesis test on one or two proportions and interpret the results.
E6.   Use Excel and PivotTables to perform a Chi-sq test on table data, and interpret the results.
E7.   Explain the concept of Type I and Type II errors and identify which (or neither) has occurred in a given situation.
E8.   Use Excel to plot bi-variate data, find the correlation; interpret the output.
E9.   Use Excel to fit a linear regression line; interpret the output.
E10. Evaluate the validity of statements about the nature of statistical thinking, including the concepts of causation, sample size, models, experimentation, statistical significance, effect size and subjectivity.
E11. Determine which test is most appropriate in a given situation, from: test for a mean or a proportion, difference between proportions, difference of two means: independent samples or paired data, chi-sq test for independence, regression.

Important concepts or principles

E12. Inferential statistics uses information collected in a sample to make predictions or judgements about the population from which the data is drawn.
E13. An effect is statistically significant when there is evidence from the sample to reject the null hypothesis.
E14. The p-value for a hypothesis test of a claim about a population parameter is the probability of getting, by chance, a sample as least as extreme as the observed one if the null hypothesis is true.

# Why engineers and poets need to know about statistics

I’m kidding about poets. But lots of people need to understand the three basic areas of statistics, Chance, Data and Evidence.

Recently Tony Greenfield, an esteemed applied statistician, (with his roots in Operations Research) posted the following request on a statistics email list:

“I went this week to the exhibition and conference in the NEC run by The Engineer magazine. There were CEOs of engineering companies of all sizes, from small to massive. I asked a loaded question:  “Why should every engineer be a competent applied statistician?” Only one, from more than 100 engineers, answered: “We need to analyse any data that comes along.” They all seemed bewildered when I asked if they knew about, or even used, SPC and DoE. I shall welcome one paragraph responses to my question. I could talk all day about it but it would be good to have a succinct and powerful few words to use at such a conference.”

For now I will focus on civil engineers, as they are often what people think of as engineers. I’m not sure about the “succinct and powerful” nature of the words to follow, but here goes…

The subject of statistics can be summarised as three areas – chance, data and evidence (CDE!)

Chance includes the rules and perceptions of probability, and emphasises the uncertainty in our world. I suspect engineers are more at home in a deterministic world, but determinism is just a model of reality. The strength of a bar of steel is not exact, but will be modelled with a probability distribution. An understanding of probability is necessary before using terms such as “one hundred year flood”. Expected values are used for making decisions on improving roads and intersections. The capacity of stadiums and malls, and the provision of toilets and exits all require modelling that relies on probability distributions. It is also necessary to have some understanding of our human fallibility in estimating and communicating probability. Statistical process control accounts for acceptable levels of variation, and indicates when they have been exceeded.

The Data aspect of the study of statistics embraces the collection, summary and communication of data. In order to make decisions, data must be collected. Correct summary measures must be used, often the median, rather than the more popular mean. Summary measures should preferably be expressed as confidence intervals, thus communicating the level of precision inherent in the data. Appropriate graphs are needed, which seldom includes pictograms or pie charts.

Evidence refers to the inferential aspects of statistical analysis. The theories of probability are used to evaluate whether a certain set of data provides sufficient evidence to draw conclusions. An engineer needs to understand the use of hypothesis testing and the p-value in order to make informed decisions regarding data. Any professional in any field should be using evidence-based practice, and journal articles providing evidence will almost always refer to the p-value. They should also be wary of claims of causation, and understand the difference between strength of effect and strength of evidence. Our video provides a gentle introduction to these concepts.

Design of Experiments also incorporates the Chance, Data and Evidence aspects of the discipline of statistics.  By randomising the units in an experiment we can control for other extraneous elements that might affect the outcome in an observational study. Engineers should be at home with these concepts.

So, Tony, how was that? Not exactly succinct, and four paragraphs rather than one. I think the Chance, Data, Evidence framework helps provide structure to the explanation.

# So what about the poets?

I borrow the term from Peter Bell of Richard Ivey School of Business, who teaches operations research to MBA students, and wrote a paper, Operations Research For Everyone (including poets). If it is difficult to get the world to recognise the importance of statistics, how much harder is it to convince them that Operations Research is vital to their well-being!

Bell uses the term, “poet” to refer to students who are not naturally at home with mathematics. In conversation Bell explained how many of his poets, who were planning to work in the area of human resource management found their summer internships were spent elbow-deep in data, in front of a spreadsheet, and were grateful for the skills they had resisted gaining.

An understanding of chance, data and evidence is useful/essential for “efficient citizenship”, to paraphrase the often paraphrased H. G. Wells. I have already written on the necessity for journalists to have an understanding of statistics. The innovative New Zealand curriculum recognises the importance of an understanding of statistics for all. There are numerous courses dedicated to making sure that medical practitioners have a good understanding.

So really, there are few professions or trades that would not benefit from a grounding in Chance, Data and Evidence. And Operations Research too, but for now that may be a bridge too far.

# Our perception of chance affects our worldview

There are many reasons that I am glad that I majored in Operations Research rather than mathematics or statistics. My view of the world has been affected by the OR way of thinking, which combines hard and soft aspects. Hard aspects are the mathematics and the models, the stuff of the abstract world. Soft aspects relate to people and the reality of the concrete world.  It is interesting that concrete is soft! Operations Research uses a combination of approaches to aid in decision making.

My mentor was Hans Daellenbach, who was born and grew up in Switzerland, did his postgrad study in California, and then stepped back in time several decades to make his home in Christchurch, New Zealand. Hans was ahead of his time in so many ways. The way I am writing about today was his teaching about probability and our subjective views on the likelihood of events.

Thanks to Daniel Kahneman’s publishing and 2002 Nobel prize, the work by him and Amos Tversky is reaching into the popular realm and is even in the high school mathematics curriculum, in a diluted form. Hans Daellenbach required first year students to read a paper by Tversky and Kahneman in the late 1980’s, over a decade earlier. This was not popular, either with the students or the tutors who were trying to make sense of the paper. Eventually we made up some interesting exercises in tutorials, and structured the approach enough for students to catch on. (Sometimes nearly half our students were from a non-English speaking background, so reading the paper was extremely challenging for them.) As a tutor and later a lecturer, I internalised the thinking, and it changed the way I see the world and chance.

People’s understanding of probability and chance events has an impact on how they see the world as well as the decisions they make.

For example, Kahneman introduced the idea of the availability heuristic. This means that if someone we know has been affected by a catastrophic (or wonderful) unlikely event, we will perceive the possibility of such an event as more likely. For example if someone we know has had their house broken into, then we feel less secure, as we perceive the likelihood of that as increased.  Someone we know wins the lottery, and suddenly it seems possible for us. Nothing has changed in the world, but our perception has changed.

There is another easily understood concept of confirmation bias. We notice and remember events and sequences of events that reinforce or confirm our preconceived notions. “Bad things come in threes” is a wonderful example. Something bad or two things bad happen, so we look for or wait for the third, and then stop counting. Similarly we remember the times when our lucky number is lucky, and do not remember the unlucky times. We mentally record the times our hunches pay off, and quietly forget the times they don’t.

## So how does this affect us as teachers of statistics? Are there ethical issues involved in how we teach statistics?

I believe in God and I believe that He guides me in my decisions in life. However I do not perceive God as a “micro-manager”. I do not believe that he has time in his day to help me to find carparks, and to send me to bargains in the supermarket. I may be wrong, and I am prepared to be proven wrong, but this is my current belief. There are many people who believe in God (or in that victim-blaming book, “The Secret”), who would disagree with me. When they see good things happen, they attribute them to the hand of God, or karma or The Secret.  There are people in some cultures who do not believe in chance at all. Everything occurs as God’s will, hence the phrase, “ insha’Allah”, or “God willing”. If they are delayed in traffic, or run into a friend, or lose their job, it is because God willed it so. This is undoubtedly a simplistic explanation, but you get the idea.

Now along comes the statistics teacher and teaches probability.  Mathematically there are some things for which the probability is easily modelled. Dice, cards, counters, balls in urns, socks in drawers can all have their probability modelled, using the ratio of number of chosen events over number of possible events. There are also probabilities estimated using historic frequencies, and there are subjective estimates of probabilities. Tversky and Kahnemann’s work showed how flawed humans are at the subjective estimates.

For some (most?) students probability remains “school-knowledge” and makes no difference to their behaviour and view of the world. It is easy to see this on game-shows such as “Deal or No Deal”, my autistic son’s favourite. It is clear that except for the decision to take the deal or not, there is no skill whatsoever in this game. In the Australian version, members of the audience hold the different cases and can guess what their case holds. If they get it right they win \$500. When this happens they are praised – well done! When the main player is choosing cases, he or she is warned that they will need to be careful to avoid the high value cases. This is clearly impossible, as there is no way of knowing which cases contain which values. Yet they are praised, “Well done!” for cases that contain low values. Sometimes they even ask the audience members what they think they are holding in the case. This makes for entertaining television – with loud shouting at times to “Take the Deal!”. But it doesn’t imbue me with any confidence that people understand probability.

Having said that, I know that I act irrationally as well. In the 1990s there were toys called Tamagotchis which were electronic pets. To keep your pet happy you had to “play” with it, which involved guessing which way the pet would turn. I KNEW that it made NO difference which way I chose and that I would do just as well by always choosing the same direction. Yet when the pet had turned to the left four times in succession, I would choose turning to the right. Assuming a good random number generator in the pet, this was pointless. But it also didn’t matter!

So if I, who have a fairly sound understanding of probability distributions and chance, still think about which way my tamagotchi is going to turn, I suspect truly rational behaviour in the general populace with regard to probabilistic events is a long time coming! Astrologers, casinos, weather forecasters, economists, lotteries and the like will never go broke.

However there are other students for whom a better understanding of the human tendency to find patterns, and confirm beliefs could provide a challenge. Their parents may strongly believe that God intervenes often or that there is no uncertainty, only lack of knowledge. (In a way this is true, but that’s a topic for another day) Like the child who has just discovered the real source of Christmas bounty, probability models are something to ponder, and can be disturbing.

We do need to be sensitive in how we teach probability. Not only can we shake people’s beliefs, but we can also use insensitive examples. I used to joke about how car accidents are a poisson process with batching, which leads to a very irregular series. Then for the last two and a half years I have been affected by the Christchurch earthquakes.  I have no sense of humour when it comes to earthquakes. None of us do. When I saw in a textbook an example of probability a building falling down as a result of an earthquake, I found that upsetting. A friend was in such a building and, though she physically survived it will be a long time before she will have a full recovery, if ever. Since then I have never used earthquakes as an example of a probabilistic event when teaching in Christchurch. I also refrain as far as possible from using other examples that may stir up pain, or try to treat them in a sober manner. Breast cancer, car accidents and tornadoes kill people and may well have affected our pupils. Just a thought.

# Which comes first – problem or solution?

In teaching it can be difficult to know whether to start with a problem or a solution method. It seems more obvious to start with the problem, but sometimes it is better to introduce the possibility of the solution before posing the problem.

# Mathematics teaching

A common teaching method in mathematics is to teach the theory, followed by applications. Or not followed by applications. I seem to remember learning a lot of mathematics with absolutely no application – which was fine by me, because it was fun. My husband once came home from survey school, and excitedly told me that he was using complex numbers for some sort of transformation between two irregular surfaces. Who’d have thought? I had never dreamed there could be a real-life use for the square root of -1. I just thought it was a cool idea someone thought up for the heck of it.

But yet again we come to the point that statistics and operations research are not mathematics. Without context and real-life application they cease to exist and turn into … mathematics!

# Applicable mathematics

My colleague wrote a guest post about “applicable mathematics” which he separates from “applied mathematics”. Applicable maths appears when teachers make up applications to try to make mathematics seem useful. There is little to recommend about applicable maths. A form of “applicable maths” occurs in probability assessment questions where the examiner decides not to tell the examinee all the information, and the examinee has to draw Venn diagrams and use logical thinking to find out something that clearly anyone in the real world would be able to read in the data! I actually enjoy answering questions like that, and they have a point in helping students understand the underlying structure of the data. But I do not fool myself into thinking that they are anywhere near real-life. Nor are they statistics.

# Which first – theory or application?

So the question is – when teaching statistics and operations research, should you start with an application or a problem or a case, and work from there to the theory? Or do students need some theory, or at least an understanding of basic principles before a case or problem can have any meaning? Or in a sequence of learning do we move back and forward between theory and application?

My first off response is that of course we should start with the data, as many books on the teaching of statistics teach us. Well actually we should start with the problem, as that really precedes the collection of the data. But then, how can we know what sorts of problems to frame if we don’t have some idea of what is possible through modelling and statistics? So should we first begin with some theory? The New Zealand Curriculum emphasises the PPDAC cycle, Problem, Plan, Data, Analysis, Conclusion. However, in order to pose the problem in the first place, we need the theory of the PPDAC cycle itself. The answer is not simple and depends on the context.

I have recently made a set of three videos explaining confidence intervals and bootstrapping. These are two very difficult topics that become simple in an instant. What I mean by that is, until you understand a confidence interval, it makes no sense, and you can see no reason why it should make sense. You go through a “liminal space” of confusion and anxiety. Then when you emerge out the other side, instantly confidence intervals make sense, and it is equally difficult to see what it was that made them confusing. This dichotomy makes teaching difficult, as the teacher needs to try to understand what made the problem confusing.

I present the idea of a confidence interval first. Then I use examples. I present the idea of bootstrapping, then give examples. I think in this instance it is helpful to delineate the theory or the idea in reasonably abstract form, interspersed with examples. I also think diagrams are immensely useful, but that’s another topic.

# Critique of AtMyPace: Statistics

What prompted these thoughts about “which comes first” was a comment made about our “AtMyPace: Statistics” iOS app.

The YouTube videos used in AtMyPace:Statistics were developed to answer specific needs in a course. They generally take the format of a quick summary of the theory, followed by an example, often related to Helen and her business selling choconutties.

The iOS app, AtMyPace:Statistics was set up as a way to capitalise on the success of the YouTube videos, and we added two quizzes of ten True/false questions to complement each of the videos. We also put these same quizzes in our on-line course and found that they were surprisingly popular. In a way, they are a substitute for a textbook or notes, but require the person to commit one way or the other to an answer before reading a further explanation. We had happened on a effective way of engaging students with the material.

AtMyPace:Statistics is not designed to be a full course in statistics, but rather a tool to help students who might be struggling with concepts. We have also developed a web-based version of AtMyPace:Statistics for those who are not the happy owners of iOS devices. At present the web version is a copy of the app, but we will happily add other questions and activities when the demand arises.

I received the following critique of the AtMyPace: Statistics app:

“They are nicely done but very classical in scope. The approach is tools-oriented using a few “realistic” examples to demonstrate the tool. This could work for students who need to take exams and want accessible material.”

Very true. The material in AtMyPace:Statistics is classical in scope, as we focus on the material currently being taught in most business schools and first year statistics service courses. We are trying to make a living, and once that is happening we will set out to change the world!

The reviewer continues,

“ I think that in adult education you should reverse the order and have the training problem oriented. Take a six sigma DMAIC process as an example. The backbone is a problem scheduled to be solved. The path is DMAIC and the tools are supporting the journey. If you want to do it that way you need to tailor the problem to the audience. “

In tailored adult education it is likely that a problem-based approach will work. I would strongly recommend it.

I had an interesting discussion some time ago with a young lecturer working in a prestigious case-based MBA programme in North America. The entire MBA is taught using cases, and is popular and successful. My friend had some reservations about case-based teaching for a subject like Operations Research which has a body of skills which are needed as a foundation for analysis. Statistics would be similar. The question is making sure the students have the necessary skills and knowledge, with the ability to transfer to another setting or problem. Case-based learning is not an efficient way to accomplish this.

# Criticism on Choosing the Test procedure

In another instance, David Munroe commented on our video “Choosing which statistical test to use”, which receives about 1000 views a week.  In the video I suggest a three step process involving thinking about what kind of data we have, what kind of sample, and the purpose of the analysis. The comment was:

Myself I would put purpose first. 🙂 The purpose of the analysis determines what data should be collected – and more data is not necessarily more informative. In my view it is more useful to think ‘what am I trying to achieve’ with this analysis before collecting the data (so the right data have a chance to be collected). This in contrast to: collecting the data and then going ‘now what can I get from this data?’ (although this is sometimes an appropriate research technique). I think because we’ve already collected the data any time we’re illustrating particular modelling tools or statistical tests, we reinforce the ‘collect the data first then worry about analysis’ approach – at least subconsciously.

Thanks David! Good thinking, and if I ever redo the video I may well change the order. I chose the order I did, as it seemed to go from easy to difficult. (Actually I don’t remember consciously thinking about the order – it just fell out of individual help sessions with students.)  And the diagram was developed in response to the rather artificial problems I was posing!

I’ll step back a bit and explain. One problem I have seen in teaching Statistics and Operations Research is that students fail to make connections. They also compartmentalise the different aspects and find it difficult to work out when certain procedures would be most useful. I wrote a post about this. In the statistics course I wrote a set of scenarios describing possible applications of statistical methods in a business context. The students were required to work out which technique to use in each scenario and found this remarkably difficult. They could perform a test on difference of two means quite well, but were hard-pressed to discern when the test should be used. So I made up even more questions to give them more practice, and designed my three step method for deciding on the test.  This helped.

I had not thought of it as a way to decide in a real-life situation which test to use. Surely that would be part of a much bigger process.  So my questions are rather artificial, but that doesn’t make them bad questions. Their point was to help students make linkages between different parts of the course. And for that, it works.

# Bring on the criticism

I would like to finish by saying how much I appreciate criticism. It is nice when people tell me they like my materials. I feel as if I am doing something useful and helping people. I get frequent comments of this type on my YouTube site.  But when people make the effort to point out gaps and flaws in the material I am extremely grateful as it helps me to clarify my thinking and improve the approach. If nothing else, it gives me something to talk about in my blog. It is difficult producing material in a feedback vacuum.  So keep it coming!