Teaching with School League tables

NCEA League tables in the newspaper

My husband ran for cover this morning when he saw high school NCEA (National Certificates of Educational Achievement)  league tables in the Press. However, rather than rave at him yet again, I will grasp the opportunity to expound to a larger audience. Much as I loathe and despise league tables, they are a great opportunity to teach students to explore data rich reports with a critical and educated eye.  There are many lessons to learn from league tables. With good teaching we can help dispell some of the myths the league tables promulgate.

When a report is made short and easy to understand, there is a good chance that much of the ‘truth’ has been lost along with the complexity. The table in front of me lists 55 secondary and area schools from the Canterbury region. These schools include large “ordinary” schools and small specialist schools such as Van Asch Deaf Education Centre and Southern Regional Health School. They include single-sex and co-ed, private, state-funded and integrated. They include area schools which are in small rural communities, which cover ages 5 to 21. The “decile” of each of the schools is the only contextual information given, apart from the name of the school.  (I explain the decile, along with misconceptions at the end of the post.) For each school is given percentages of students passing at the three levels. It is not clear whether the percentages in the newspaper are of participation rate or school roll.

This is highly motivating information for students as it is about them and their school. I had an argument recently with a student from a school which scores highly in NCEA. She was insistent that her friend should change schools from one that has lower scores. What she did not understand was that the friend had some extra learning difficulties, and that the other school was probably more appropriate for her. I tried to teach the concept of added-value, but that wasn’t going in either. However I was impressed with her loyalty to her school and I think these tables would provide an interesting forum for discussion.

Great context discussion

You could start with talking about what the students think will help a school to have high pass rates. This could include a school culture of achievement, good teaching, well-prepared students and good resources. This can also include selection and exclusion of students to suit the desired results, selection of “easy” standards or subjects, and even less rigorous marking of internal assessment. Other factors to explore might be single-sex vs co-ed school, the ethnic and cultural backgrounds of the students, private vs state-funded schools.  All of these are potential explanatory variables. Then you can point out how little of this information is actually taken into account in the table. This is a very common occurrence, with limited space and inclusion of raw data. I suspect at least one school appears less successful because some of the students sit different exams, either Cambridge or International Baccalaureate. These may be the students who would have performed well in NCEA.

Small populations

It would be good to look at the impact of small populations, and populations of very different sizes in the data. Students should think about what impact their behaviour will have on the results of the school, compared with a larger or smaller cohort. The raw data provided by the Ministry of Education does give a warning for small cohorts. For a small school, particularly in a rural area, there may be only a handful of students in year 13, so that one student’s success or failure has a large impact on the outcome. At the other end of the scale, there are schools of over 2000, which will have about 400 students in year 13. This effect is important to understand in all statistical reporting. One bad event in a small hospital, for instance, will have a larger percentage effect than in a large hospital.

Different rules

We hear a lot about comparing apples and oranges. School league tables include a whole fruit basket of different criteria. Schools use different criteria for allowing students into the school, into different courses, and whether they are permitted to sit external standards. Attitudes to students with special educational needs vary greatly. Some schools encourage students to sit levels outside their year level.

Extrapolating from a small picture

What one of the accompanying stories points out is that NCEA is only a part of what schools do. Sometimes the things that are measurable get more attention because it is easier to report in bulk. A further discussion with students could be provoked using statements such as the following, which the students can vote on, and then discuss. You could also discuss what evidence you would need to be able to refute or support them.

  • A school that does well in NCEA level 3 is a good school.
  • Girls’ schools do better than boys’ schools at NCEA because girls are smarter than boys.
  • Country schools don’t do very well because the clever students go to boarding school in the city.
  • Boys are more satisfied with doing just enough to get achieved.

Further extension

If students are really interested you can download the full results from the Ministry of Education website and set up a pivot table on Excel to explore questions.

I can foresee some engaging and even heated discussions ensuing. I’d love to hear how they go.

Short explanation of Decile – see also official website.

The decile rating of the school is an index developed in New Zealand and is a measure of social deprivation. The decile rating is calculated from a combination of five values taken from census data for the meshblocks in which the students reside. A school with a low decile rating of 1 or 2 will have a large percentage of students from homes that are crowded, or whose parents are not in work or have no educational qualifications. A school with a decile rating of 10 will have the fewest students from homes like that. The system was set up to help with targeted funding for educational achievement. It recognises that students from disadvantaged homes will need additional resources in order to give them equal opportunity to learn. However, the term has entered the New Zealand vernacular as a measure of socio-economic status, and often even of worth. A decile 10 school is often seen as a rich school or a “top” school. The reality is that this is not the case.  Another common misconception is that one tenth of the population of school age students is in each of the ten bands. How it really works is that one tenth of schools is in each of the bands. The lower decile schools are generally smaller than other schools, and mostly primary schools. In 2002 there were nearly 40,000 secondary students in decile 10 schools, with fewer than 10,000 in decile 1 schools.

Parts and whole

The whole may be greater than the sum of the parts, but the whole still needs those parts. A reflective teacher will think carefully about when to concentrate on the whole, and when on the parts.

Golf

If you were teaching someone golf, you wouldn’t spend days on a driving range, never going out on a course. Your student would not get the idea of what the game is, or why they need to be able to drive straight and to a desired length. Nor would it be much fun! Similarly if the person only played games of golf it would be difficult for them to develop their game. Practice driving and putting is needed.  A serious student of golf would also read and watch experts at golf.

Music

Learning music is similar. Anyone who is serious about developing as a musician will spend a considerable amount of time developing their technique and their knowledge by practicing scales, chords and drills. But at the same time they need to be playing full pieces of music so that they feel the joy of what they are doing. As they play music, as opposed to drill, they will see how their less-interesting practice has helped them to develop their skills. However, as they practice a whole piece, they may well find a small part that is tripping them up, and focus for a while on that. If they play only the piece as a whole, it is not efficient use of time. A serious student of music will also listen to and watch great musicians, in order to develop their own understanding and knowledge.

Benefits of study of the whole and of the parts

In each of these examples we can see that there are aspects of working with the whole, and aspects of working with the parts. Study of the whole contributes perspective and meaning to study, and helps to tie things together. It helps to see where they have made progress. Study of the parts isolates areas of weakness, develops skills and saves time in practice, thus being more efficient.

It is very important for students to get an idea of the purpose of their study, and where they are going. For this reason I have written earlier about the need to see the end when starting out in a long procedure such as a regression or linear programming model.

It is also important to develop “statistical muscle memory” by repeating small parts of the exercise over and over until it is mastered. Practice helps people to learn what is general and what is specific in the different examples.

Teaching conditional probability

We are currently developing a section on probability as part of our learning materials. A fundamental understanding of probability and uncertainty are essential to a full understanding of inference. When we look at statistical evidence from data, we are holding it up against what we could reasonably expect to happen by chance, which involves a probability model. Probability lies in the more mathematical area of the study of statistics, and has some fun problem-solving aspects to it.

A popular exam question involves conditional probability. We like to use a table approach to this as it avoids many of the complications of terminology. I still remember my initial confusion over the counter-intuitive expression P(A|B) which means the probability that an object from subset B has the property of A. There are several places where students can come unstuck in Bayesian review, and the problems can take a long time. We can liken solving a conditional probability problem to a round of golf, or a long piece of music. So what we do in teaching is that first we take the students step by step through the whole problem. This includes working out what the words are saying, putting the known values into a table, calculating the unknown values in the table, and the using the table to answer the questions involving conditional probability.

Then we work on the individual steps, isolating them so that students can get sufficient practice to find out what is general and what is specific to different examples. As we do this we endeavour to provide variety such that students do not work out some heuristic based on the wording of the question, that actually stops them from understanding. An example of this is that if we use the same template each time, students will work out that the first number stated will go in a certain place in the table, and the second in another place etc. This is a short-term strategy that we need to protect them from in careful generation of questions.

As it turns out students should already have some of the necessary skills. When we review probability at the start of the unit, we get students to calculate probabilities from tables of values, including conditional probabilities. Then when they meet them again as part of the greater whole, there is a familiar ring.

Once the parts are mastered, the students can move on to a set of full questions, using each of the steps they have learned, and putting them back into the whole. Because they are fluent in the steps, it becomes more intuitive to put the whole back together, and when they meet something unusual they are better able to deal with it.

Starting a course in Operations Research/Management Science

It is interesting to contemplate what “the whole” is, with regard to any subject. In operations research we used to begin our first class, like many first classes, talking about what management science/operations research is. It was a pretty passive sort of class, and I felt it didn’t help as first-year university students had little relevant knowledge to pin the ideas on. So we changed to an approach that put them straight into the action and taught several weeks of techniques first. We started with project management and taught critical path. Then we taught identifying fixed and variable costs and break-even analysis. The next week was discounting and analysis of financial projects. Then for a softer example we looked at multi-criteria decision-making, using MCDM. It tied back to the previous week by taking a different approach to a decision regarding a landfill. Then we introduced OR/MS, and the concept of mathematical modelling. By then we could give real examples of how mathematical models could be used to inform real world problems. It was helpful to go from the concrete to the abstract. This was a much more satisfactory approach.

So the point is not that you should always start with the whole and then do the parts and then go back to the whole. The point is that a teacher needs to think carefully about the relationship between the parts and the whole, and teach in a way that is most helpful.

Let’s hear it for the Triangular Distribution!

Telly_Monster

Dr Nic meets Telly monster on the set of Sesame Street

Telly monster is my favourite character on Sesame Street, and a few years ago I was lucky enough to actually meet him. This morning I was delighted to find out from my resident Sesame Street expert that Telly monster is a triangle lover. I too am becoming a triangle lover.

I have learned recently about the triangular distribution. For some reason it is in the New Zealand curriculum and I wondered why, never having used it or seen it in any statistics textbook. I still don’t know the official motivation for including it, but it is a really good idea. The triangular distribution seems to be a useful teaching tool. I say “seems to be” as I haven’t actually used it in a class, but I can see the potential for some very good exercises and learning experiences.

There are three aspects of the triangular distribution that I find appealing.

Multiple models for the same scenario

Using the triangular distribution alongside the normal distribution encourages the idea that the distributions are models of a real-life process. Like many curricula, the previous NZ curriculum included binomial, Poisson and normal distributions. Any questions of what model to use in a specific scenario were determined pretty much by the form of the story. If the story involved continuous data it pretty much had to be “normally distributed”.  This could imbed a false impression that all continuous data was appropriately modelled by the normal distribution.

A nice learning experience is to take some real data such as weights of lemons from a tree, and see how well that is modelled by a normal distribution. This can be enriched by also modelling this as a triangular distribution and seeing how well the two compare. A further extension would be to use a uniform distribution model of the same data. The beauty of this exercise is that it reinforces the idea that a distribution is a model of reality, and that there are different models that may be more or less appropriate for different data and circumstances.

Different characteristics

I find contrasts are helpful for teaching. The triangular distribution provides a nice contrast with the normal distribution in a number of ways.

First is the requirements for specification. The triangular distribution is specified by the maximum, minimum and peak values. When we are making a subjective estimate, such as for completion time for a task, these are three easily pictured amounts – the longest time, the shortest time, and the most likely time for completion. This compares with the normal distribution, for which we need the mean and standard deviation, which would often be drawn from a sample.

The triangular distribution has a finite range, bounded by the maximum and minimum values. In a triangular distribution we can specify, for instance, that the results of a test will lie between 0 and 10. Modelling such a situation with a normal distribution can give results outside the range, as it theoretically goes to infinity in both directions.

The triangular distribution is not symmetric. It can be, but it is not a requirement. Where we have severely skewed data, it may well be that the triangular distribution is a better model than the normal distribution. This helps us when teaching the use of the normal distribution. The contrast is helpful.

Area under the pdf is the probability

The idea that the probability of a continuous distribution is the area under the probability mass function is a difficult one for many students to get their heads around. One way to teach this is to start with a discrete distribution and then cut it up into finer and finer points. But then finding the actual area is problematic. With the normal distribution, the computation is hidden in the calculator, spreadsheet or tables. With the uniform distribution, the computation is trivial and can seem contrived. But areas under the graph in the triangular distribution can be calculated, and the exercise is not trivial. It also very nicely shows how the pdf is not going to give them the probability of a single value.

So there you have it – three important lessons in one tidy little triangle-shaped package.

And to help you use the triangular distribution in your teaching, we have this handout: Notes on Triangle Distributions which you are welcome to use, so long as you leave our branding on it. And if you want your students to have some great practice exercises, they can always join our course!

Conceptualising Probability

The problem with probability is that it doesn’t really exist. Certainly it never exists in the past.

Probability is an invention we use to communicate our thoughts about how likely something is to happen. We have collectively agreed that 1 is a certain event and 0 is impossible. 0.5 means that there is just as much chance of something happening as not. We have some shared perception that 0.9 means that something is much more likely to happen than to not happen. Probability is also useful for when we want to do some calculations about something that isn’t certain. Often it is too hard to incorporate all uncertainty, so we assume certainty and put in some allowance for error.

Sometimes probability is used for things that happen over and over again, and in that case we feel we can check to see if our predication about how likely something is to happen was correct. The problem here is that we actually need things to happen a really big lot of times under the same circumstances in order to assess if we were correct. But when we are talking about the probability of a single event, that either will or won’t happen, we can’t test out if we were right or not afterwards, because by that time it either did or didn’t happen. The probability no longer exists.

Thus to say that there is a “true” probability somewhere in existence is rather contrived. The truth is that it either will happen or it won’t. The only way to know a true probability would be if this one event were to happen over and over and over, in the wonderful fiction of parallel universes. We could then count how many times it would turn out one way rather than another. At which point the universes would diverge!

However, for the interests of teaching about probability, there is the construct that there exists a “true probability” that something will happen.

Why think about probability?

What prompted these musings about probability was exploring the new NZ curriculum and companion documents, the Senior Secondary Guide and nzmaths.co.nz.

In Level 8 (last year of secondary school) of the senior secondary guide it says, “Selects and uses an appropriate distribution to solve a problem, demonstrating understanding of the relationship between true probability (unknown and unique to the situation), model estimates (theoretical probability) and experimental estimates.”

And at NZC level 3 (years 5 and 6 at Primary school!) in the Key ideas in Probability it talks about “Good Model, No Model and Poor Model” This statement is referred to at all levels above level 3 as well.

I decided I needed to make sense of these two conceptual frameworks: true-model-experimental and good-poor-no, and tie it to my previous conceptual framework of classical-frequency-subjective.

Here goes!

Delicious Mandarins

Let’s make this a little more concrete with an example. We need a one-off event. What is the probability that the next mandarin I eat will be delicious? It is currently mandarin season in New Zealand, and there is nothing better than a good mandarin, with the desired combination of sweet and sour, and with plenty of juice and a good texture. But, being a natural product, there is a high level of variability in the quality of mandarins, especially when they may have parted company with the tree some time ago.

There are two possible outcomes for my future event. The mandarin will be delicious or it will not. I will decide when I eat it. Some may say that there is actually a continuum of deliciousness, but for now this is not the case. I have an internal idea of deliciousness and I will know. I think back to my previous experience with mandarins. I think about a quarter are horrible, a half are nice enough and about a quarter are delicious (using the Dr Nic scale of mandarin grading). If the mandarin I eat next belongs to the same population as the ones in my memory, then I can predict that there is a 25% probability that the mandarin will be delicious.

The NZ curriculum talks about “true” probability which implies that any value I give to the probability is only a model. It may be a model based on empirical or experimental evidence. It can be based on theoretical probabilities from vast amounts of evidence, which has given us the normal distribution. The value may be only a number dredged up from my soul, which expresses the inner feeling of how likely it is that the mandarin will be delicious, based on several decades of experience in mandarin consumption.

More examples

Let us look at some more examples:

What is the probability that:

  • I will hear a bird on the way to work?
  • the flight home will be safe?
  • it will be raining when I get to Christchurch?
  • I will get a raisin in my first spoonful of muesli?
  • I will get at least one raisin in half of my spoonfuls of muesli?
  • the shower in my hotel room will be enjoyable?
  • I will get a rare Lego ® minifigure next time I buy one?

All of these events are probabilistic and have varying degrees of certainty and varying degrees of ease of modelling.

Easy to model Hard to model
Unlikely Get a rare Lego ® minifigure Raining in Christchurch
No idea Raisin in half my spoonfuls Enjoyable shower
Likely Raisin in first spoonful Bird, safe flight home

And as I construct this table I realise also that there are varying degrees of importance. Except for the flight home, none of those examples matter. I am hoping that a safe flight home has a probability extremely close to 1. I realise that there is a possibility of an incident. And it is difficult to model. But people have modelled air safety and the universal conclusion is that it is safer than driving. So I will take the probability and fly.

Conceptual Frameworks

How do we explain the different ways that probability has been described? I will now examine the three conceptual frameworks I introduced earlier, starting with the easiest.

Traditional categorisation

This is found in some form in many elementary college statistics text books. The traditional framework has three categories –classical or “a priori”, frequency or historical, and subjective.

Classical or “a priori” – I had thought of this as being “true” probability. To me, if there are three red and three white Lego® blocks in a bag and I take one out without looking, there is a 50% chance that I will get a red one. End of story. How could it be wrong? This definition is the mathematically interesting aspect of probability. It is elegant and has cool formulas and you can make up all sorts of fun examples using it. And it is the basis of gambling.

Frequency or historical – we draw on long term results of similar trials to gain information. For example we look at the rate of germination of a certain kind of seed by experiment, and that becomes a good approximation of the likelihood that any one future seed will germinate. And it also gives us a good estimate of what proportion of seeds in the future will germinate.

Subjective – We guess! We draw on our experience of previous similar events and we take a stab at it. This is not seen as a particularly good way to come up with a probability, but when we are talking about one off events, it is impossible to assess in retrospect how good the subjective probability estimate was. There is considerable research in the field of psychology about the human ability or lack thereof to attribute subjective probabilities to events.

In teaching the three part categorisation of sources of probability I had problems with the probability of rain. Where does that fit in the three categories? It uses previous experimental data to build a model, and current data to put into the model, and then a probability is produced. I decided that there is a fourth category, that I called “modelled”. But really that isn’t correct, as they are all models.

NZ curriculum terminology

So where does this all fit in the New Zealand curriculum pronouncements about probability? There are two conceptual frameworks that are used in the document, each with three categories as follows:

True, modelled, experimental

In this framework we start with the supposition that there exists somewhere in the universe a true probability distribution. We cannot know this. Our expressions of probability are only guesses at what this might be. There are two approaches we can take to estimate this “truth”. These two approaches are not independent of each other, but often intertwined.

One is a model estimate, based on theory, such as that the probability of a single outcome is the number of equally likely ways that it can occur over the number of possible outcomes. This accounts for the probability of a red brick as opposed to a white brick, drawn at random. Another example of a modelled estimate is the use of distributions such as the binomial or normal.

In addition there is the category of experimental estimate, in which we use data to draw conclusions about what it likely to happen. This is equivalent to the frequency or historical category above. Often modelled distributions use data from an experiment also. And experimental probability relies on models as well.  The main idea is that neither the modelled nor the experimental estimate of the “true” probability distribution is the true distribution, but rather a model of some sort.

Good model, poor model, no model

The other conceptual framework stated in the NZ curriculum is that of good model, poor model and no model, which relates to fitness for purpose. When it is important to have a “correct” estimate of a probability such as for building safety, gambling machines, and life insurance, then we would put effort into getting as good a model as possible. Conversely, sometimes little effort is required. Classical models are very good models, often of trivial examples such as dice games and coin tossing. Frequency models aka experimental models may or may not be good models, depending on how many observations are included, and how much the future is similar to the past. For example, a model of sales of slide rules developed before the invention of the pocket calculator will be a poor model for current sales. The ground rules have changed. And a model built on data from five observations of is unlikely to be a good model. A poor model is not fit for purpose and requires development, unless the stakes are so low that we don’t care, or the cost of better fitting is greater than the reward.

I have problems with the concept of “no model”. I presume that is the starting point, from which we develop a model or do not develop a model if it really doesn’t matter. In my examples above I include the probability that I will hear a bird on the way to work. This is not important, but rather an idle musing. I suspect I probably will hear a bird, so long as I walk and listen. But if it rains, I may not. As I am writing this in a hotel in an unfamiliar area I have no experience on which to draw. I think this comes pretty close to “no model”. I will take a guess and say the probability is 0.8. I’m pretty sure that I will hear a bird. Of course, now that I have said this, I will listen carefully, as I would feel vindicated if I hear a bird. But if I do not hear a bird, was my estimate of the probability wrong? No – I could assume that I just happened to be in the 0.2 area of my prediction. But coming back to the “no model” concept – there is now a model. I have allocated the probability of 0.8 to the likelihood of hearing a bird. This is a model. I don’t even know if it is a good model or a poor model. I will not be walking to work this way again, so I cannot even test it out for the future, and besides, my model was only for this one day, not for all days of walking to work.

So there you have it – my totally unscholarly musings on the different categorisations of probability.

What are the implications for teaching?

We need to try not to perpetuate the idea that probability is the truth. But at the same time we do not wish to make students think that probability is without merit. Probability is a very useful, and at times highly precise way of modelling and understanding the vagaries of the universe. The more teachers can use language that implies modelling rather than rules, the better. It is common, but not strictly correct to say, “This process follows a normal distribution”. As Einstein famously and enigmatically said, “God does not play dice”. Neither does God or nature use normal distribution values to determine the outcomes of natural processes. It is better to say, “this process is usefully modelled by the normal distribution.”

We can have learning experiences that help students to appreciate certainty and uncertainty and the modelling of probabilities that are not equi-probable. Thanks to the overuse of dice and coins, it is too common for people to assess things as having equal probabilities. And students need to use experiments.  First they need to appreciate that it can take a large number of observations before we can be happy that it is a “good” model. Secondly they need to use experiments to attempt to model an otherwise unknown probability distribution. What fun can be had in such a class!

But, oh mathematical ones, do not despair – the rules are still the same, it’s just the vigour with which we state them that has changed.

Comment away!

Post Script

In case anyone is interested, here are the outcomes which now have a probability of 1, as they have already occurred.

  • I will hear a bird on the way to work? Almost the minute I walked out the door!
  • the flight home will be safe? Inasmuch as I am in one piece, it was safe.
  • it will be raining when I get to Christchurch? No it wasn’t
  • I will get a raisin in my first spoonful of muesli? I did
  • I will get at least one raisin in half of my spoonfuls of muesli? I couldn’t be bothered counting.
  • the shower in my hotel room will be enjoyable? It was okay.
  • I will get a rare Lego minifigure next time I buy one? Still in the future!

The Knife-edge of Competence

I do my own video-editing using a very versatile and complex program called Adobe Premiere Pro. I have had no formal training, and get help by ringing my son, who taught me all I know and can usually rescue me with patient instructions over the phone. At times, especially in the early stages I have felt myself wobbling along the knife-edge of competence. All I needed was for something new to go wrong, or or click a button inadvertently and I would fall off the knife-edge and the whole project would disappear into a mass of binary. This was not without good reason. Premiere Pro wasn’t always stable on our computer, and at one point it took us several weeks to get our hard-drive replaced. (Apple “Time machine” saved me from despair). And sometimes I would forget to save regularly and a morning’s work was lost. (Even time-machine can’t help with that level of incompetence.)

But despite my severe limitations I have managed to edit over twenty videos that now receive due attention (and at times adulation!) on YouTube. It isn’t an easy feeling, to be teetering on the brink of disaster, real or imagined. But there was no alternative, and there is a sense of pride at having made it through with only a few scars and not too much inappropriate language.

There are some things at which I feel totally competent. I can speak to a crowd of any number of people and feel happy that they will be entertained, edified and perhaps even educated. I can analyse data using basic statistical methods. I can teach a person about inference. Performing these tasks is a joy, because I know I have the prerequisite skills and knowledge to cope with whatever happens. But on the way to getting to this point, I had to walk the knife-edge of competence.

Many teachers of statistics know too well this knife-edge. In New Zealand at present there are a large number of teachers of Year 13 statistics who are teaching about bootstrapping, when their own understanding of it is sketchy. They are teaching how to write statistical reports, when they have never written one themselves. They are assessing statements about statistics that they are not actually sure about. This is a knife-edge. They feel that any minute a student will ask them a question about the content that they cannot answer. These are not beginning teachers, but teachers with years and decades of experience in teaching mathematics and mathematical statistics. But the innovations of the curriculum have put them in an uncomfortable position. Inconsistent, tardy and even incorrect information from the qualification agency is not helping, but that is a story for another day.

In another arena there are professors and lecturers of statistics (in the antipodes we do not throw around the title “professor” with the abandon of our North American cousins) who are extremely competent at statistical mathematics and analysis but who struggle to teach in a satisfactory way. Their knife-edge concerns teaching, appropriate explanation and the generation of effective learning activities and assessments in the absence of any educational training. They fear that someone will realise one day that they don’t really know how to devise learning objectives, and provide fair assessments. I am hoping that this blog is going some way to helping these people to ask for help! Unfortunately the frequent response is avoidance behaviour, which is alarmingly supported by a system that rewards research publications rather than effective educational endeavours.

So what do you do when you are walking the knife-edge of competence?

You do the best you can.

And sometimes you fake it.

I am led to believe there is a gender-divide on this. Some people are better at hiding their incompetence than others, and just about all the people I know like that are men. I had a classmate in my honours year who was at a similar level of competence to me, but he applied for jobs I wouldn’t have contemplated. The fear of being shown up as a fake, or not knowing EXACTLY what to do at any point stopped me from venturing. He horrified me further a few years later when he set up his own company. Nearly three decades, two children and a PhD later I am not so fastidious or “nice” in the Jane Austen meaning of the word. If I think I can probably learn how to do something in time to make a reasonable fist of it and not cause actual harm, I’m likely to have a go. Hence taking my redundancy and running!

When I first lectured in statistics for management,  I did not know much beyond what I was teaching. I lived in fear that someone would ask me a question that I couldn’t answer and I would be revealed as the fake I was. Well you know, it never happened! I even taught students who were statistics majors, who did know more than I, and post-graduate students in psychology and heads of mathematics departments, and my fears were never realised. In fact the stats students told me that they finally understood the central limit theorem, thanks to my nifty little exercise using dotplots on minitab. (Which was how I had finally understood the central limit theorem – or at least the guts of it.)

I’m guessing that this is probably true for most of the mathematics teachers who are worrying. Despite their fear, they have not been challenged or called out.

The teachers’ other unease is the feeling that they are not giving the best service to their students, and the students will suffer, miss out on scholarships, decide not to get a higher education and live their lives on the street.  I may be exaggerating a little here, but certainly few of us like to give a service that is less than what we are accustomed to. We feel bad when we do something that feels substandard.

There are two things I learned in my twenty years of lecturing that may help here:

We don’t know how students perceive what we do. Every now and again I would come out of a lecture with sweat trickling down my spine because something had gone wrong. It might be that in the middle of an explanation I had had second thoughts about it, changed tack, then realised I was right in the first-place and ended up confusing myself. Or perhaps part way through a worked example it was pointed out to me that there was a numerical error in line three. To me these were bad, bad things to happen. They undermined my sense of competence. But you know, the students seldom even noticed. What felt like the worst lecture of my life, was in fact still just fine.

The other thing I learned is that we flatter ourselves when we think how much difference our knowledge may make.  Now don’t get me wrong here – teachers make an enormous difference. People who become teachers do so because we want to help people. We want to make a difference in students’ lives. We often have a sense of calling. There may be some teachers who do it because they don’t know what else to do with their degree, but I like to think that most of us teachers teach because to not teach is unthinkable. I despise, to the point of spitting as I talk, the expression “Those who can, do, and those who can’t, teach.” One day when the mood takes me I will write a whole post about the noble art of teaching and the fallacy of that dismissive statement. My next statement is so important I will give it a paragraph of its own.

A teacher who teaches from love, who truly cares about what happens to their students, even if they are struggling on the knife-edge of competence will not ruin their students’ lives through temporary incompetence in an aspect of the curriculum.

There are many ways that a teacher can have devastating effects on their students, but being, for a short time, on the knife-edge of competence, is not one of them.

Take heart, keep calm and carry on!

Oh Ordinal data, what do we do with you?

What can you do with ordinal data? Or more to the point, what shouldn’t you do with ordinal data?

First of all, let’s look at what ordinal data is.

It is usual in statistics and other sciences to classify types of data in a number of ways. In 1946, Stanley Smith Stevens suggested a theory of levels of measurement, in which all measurements are classified into four categories, Nominal, Ordinal, Interval and Ratio. This categorisation is used extensively, and I have a popular video explaining them. (Though I group Interval and Ratio together as there is not much difference in their behaviour for most statistical analysis.)

Nominal is pretty straight-forward. This category includes any data that is put into groups, in which there is no inherent order. Examples of nominal data are country of origin, sex, type of cake, or sport. Similarly it is pretty easy to explain interval/ratio data. It is something that is measured, by length, weight, time (duration), cost and similar. These two categorisations can also be given as qualitative and quantitative, or non-parametric and parametric.

Ordinal data

But then we come to ordinal level of measurement. This is used to describe data that has a sense of order, but for which we cannot be sure that the distances between the consecutive values are equal. For example, level of qualification has a sense of order

  • A postgraduate degree is higher than
  • a Bachelor’s degree,which is higher than
  • a high-school qualification, which is higher
  • than no qualification.

There are four steps on the scale, and it is clear that there is a logical sense of order. However, we cannot sensibly say that the difference between no qualification and a high-school qualification is equivalent to the difference between the high-school qualification and a bachelor’s degree, even though both of those are represented by one step up the scale.

Another example of ordinal level of measurement is used extensively in psychological, educational and marketing research, known as a Likert scale. (Though I believe the correct term is actually Likert item – and according to Wikipedia, the pronunciation should be Lick it, not Like it, as I have used for some decades!). A statement is given, and the response is given as a value, often from 1 to 5, showing agreement to the statement. Often the words “Strongly agree, agree, neutral, disagree, strongly disagree” are used. There is clearly an order in the five possible responses. Sometimes a seven point scale is used, and sometimes the “neutral” response is eliminated in an attempt to force the respondent to commit one way or the other.

The question at the start of this post has an ordinal response, which could be perceived as indicating how quantitative the respondent believes ordinal data to be.

What prompted this post was a question from Nancy under the YouTube video above, asking:

“Dr Nic could you please clarify which kinds of statistical techniques can be applied to ordinal data (e.g. Likert-scale). Is it true that only non-parametric statistics are possible to apply?”

Well!

As shown in the video, there are the purists, who are adamant that ordinal data is qualitative. There is no way that a mean should ever be calculated for ordinal, data, and the most mathematical thing you can do with it is find the median. At the other pole are the practical types, who happily calculate means for any ordinal data, without any concern for the meaning (no pun intended.)

There are differing views on finding the mean for ordinal data.

There are differing views on finding the mean for ordinal data.

So the answer to Nancy would depend on what school of thought you belong to.

Here’s what I think:

All ordinal data is not the same. There is a continuum of “ordinality” if you like.

There are some instances of ordinal data which are pretty much nominal, with a little bit of order thrown in. These should be distinguished from nominal data, only in that they should always be graphed as a bar chart (rather than a pie-chart)* because there is inherent order. The mode is probably the only sensible summary value other than frequencies. In the examples above, I would say that “level of qualification” is only barely ordinal. I would not support calculating a mean for the level of qualification. It is clear that the gaps are not equal, and additionally any non-integer result would have doubtful interpretation.

Then there are other instances of ordinal data for which it is reasonable to treat it as interval data and calculate the mean and median. It might even be supportable to use it in a correlation or regression. This should always be done with caution, and an awareness that the intervals are not equal.

Here is an example for which I believe it is acceptable to use the mean of an ordinal scale. At the beginning and the end of a university statistics course, the class of 200 students is asked the following question: How useful do you think a knowledge of statistics is will be to you in your future career? Very useful, useful, not useful.

Now this is not even a very good Likert question, as the positive and negative elements are not balanced. There are only three choices. There is no evidence that the gaps between the elements are equal. However if we score the elements as 3,2 and 1, respectively and find that the mean for the 200 students is 1.5 before the course, and 2.5 after the course, I would say that there is meaning in what we are reporting. There are specific tests to use for this – and we could also look at how many students changed their minds positively or negatively. But even without the specific test, we are treating this ordinal data as something more than qualitative. What also strengthens the evidence for doing this is that the test is performed on the same students, who will probably perceive the scale in the same way each time, making the comparison more valid.

So what I’m saying is that it is wrong to make a blanket statement that ordinal data can or can’t be treated like interval data. It depends on meaning and number of elements in the scale.

What do we teach?

And again the answer is that it depends! For my classes in business statistics I told them that it depends. If you are teaching a mathematical statistics class, then a more hard line approach is justified. However, at the same time as saying, “you should never calculate the mean of ordinal data”, it would be worthwhile to point out that it is done all the time! Similarly if you teach that it is okay to find the mean of some ordinal data, I would also point out that there are issues with regard to interpretation and mathematical correctness.

Please comment!

Foot note on Pie charts

*Yes, I too eschew pie-charts, but for two or three categories of nominal data, where there are marked differences in frequency, if you really insist, I guess you could possibly use them, so long as they are not 3D and definitely not exploding. But even then, a barchart is better. – perhaps a post for another day, but so many have done this.

Why learning objectives are so important

The most useful thing I learned in my teacher training at Auckland College of Education in 1985 was to write learning objectives. Not many years, and two babies later, I began lecturing at the University of Canterbury in Management Science/Operations Research. I was the only academic staff member to have formal teacher-training. My first task, when put in charge of MSCI210, Statistical Methods for Management, was to write learning objectives. This was revolutionary, but the idea infiltrated through other courses over the years.

A learning objective states specifically what a student should be able to do.

Here are some examples of good learning objectives:

Students will be able to:

  • Identify different levels of data in new scenarios.
  • Explain in context a confidence interval.
  • Determine which probability distribution out of binomial, poisson or normal is most appropriate to model in an unfamiliar situation.
  • Compare two time series models of the same data and evaluate which is more appropriate in a given context.

Learning objectives need to be specific and measurable.

Here are some things that people might think are learning objectives, but are not:

  • Students will understand the central limit theorem. (The term “understand” is not measurable)
  • Students will learn about probability trees (“learn” is not measurable, and does not specify the level. Do students need to be able to interpret or create probability trees?)

There are vast numbers of resources on learning objectives online.

Here is one I liked, with Bloom’s taxonomy of levels of learning. These are higher and lower levels of learning objectives, ranging from being able to state principles, through to synthesis and evaluation.

http://teachonline.asu.edu/2012/07/writing-measurable-learning-objectives/

And here are some useful verbs to use when writing learning objectives;

http://www.schreyerinstitute.psu.edu/pdf/SampleVerbs_for_LearningObjectives.pdf

It is not difficult to find material on developing learning objectives.

Not just learning objectives

A course is more than the set of its learning objectives. The learning objectives specify the skills, but there are also attitudes and knowledge to be considered. The starting point for course design is the attitudes. What do we want the students to feel about the topics? What changes do we wish them to contemplate in their thinking? Then the skills and knowledge are specified, often starting at a quite general level, then working down to specifics.

For example, we might wish to teach about confidence intervals. We need to determine whether students need to be able to calculate them, interpret them, estimate or derive them.  We need to decide which confidence intervals we are interested in – for means alone, or proportions and slopes as well? Sometimes I find there are concepts I wish to include in the learning objectives, but they don’t really work as objectives. These I put as “important concepts and principles”.

I have put an example of learning objectives and concepts and principles at the end of this post.

Learning objectives tell students what is important

Without learning objectives it is difficult for students to know what they are supposed to be learning. In a lecture, a teacher can talk extensively about a case, but unless she states explicitly, it can be difficult for the students to know where to direct their attention. Do they need to know the details of that specific case or what principles are they supposed to glean from the example? Or was it just a “war-story” to entertain the troops? Students can waste a great deal of time studying things that are not necessary, to the detriment of their learning as a whole. The uncertainty also causes unnecessary anxiety.

Learning objectives enable good assessment development

Each year as we wrote our assessments we would go through the learning objectives and make sure they were assessed.  This way the assessment was fair and applied to the course. If we found it difficult to write a question to assess a learning objective we would think again about the learning objective, and what it is we really want the students to be able to do. It made it easier to write fair, comprehensive assessments.

Learning objectives encourage reflection and good course design and development

As instructors write and review the learning objectives in a course, they can identify the level of learning that is specified in each. At an entry-level course, it is acceptable to have a number of lower level learning objectives. However, there needs to be some serious thinking done if a post-graduate course is not mainly made up of higher level learning objectives. I have seen tests in stage 2 and 3 papers that tested mainly recall and common-sense. It was evident that the instructor had not thought clearly about the level of learning that was expected.

Sometimes we find we are assessing things we have not specifically taught the students. The use of learning objectives, linked with assessment design, helps us to identify the background knowledge that we assume students have. One colleague was frustrated that the students did not seem able to apply the statistical results to a managerial context. However, nowhere had she specified that students would be required to do so, and nowhere had she actually taught students how to do this. She also assumed a level of understanding of business,  that was probably not appropriate in undergraduate students.

Like it or not, assessment drives learning

I spoke recently to a maths advisor who informed me that teachers should be teaching to the curriculum not to the assessments. I felt he was idealistic, and told him so. My experience is that university students will learn what is assessed, and nothing else. I don’t know at what age this begins, but I suspect National Testing, the bane of good education, has lowered the age considerably. How wonderful it would be if our students learned for the sheer joy of learning! Where there are assessments looming, I fear this is unlikely.

When we write exams we are also writing learning materials for future students. One of the most common ways to prepare for an assessment is to do exercises from previous assessments. So when we feel that students were not really coming to grips with a concept, we include questions in the assessment, that can then be used by future students for review.

Information promotes equity and reduces unnecessary stress

The use of learning objectives can help reduce the “gaming” aspects that can proliferate in the absence of clear information. This is apparent at present in the world of Year 13 Statistics in New Zealand. The information regarding the external standards for 2013 is still sketchy (1 July 2013). The exams are written by external examiners and will take place in November of this year. However there is still only vague and sometimes incorrect information as to exactly what may or may not be included in the exams. Because of this, teachers are trying to detect, from what is or isn’t in the formula sheet and the (not totally correct) exemplars what might be in the finals, and what to include in the school practice exams.  I suspect that some teachers or areas have more information than others. The way to make this fairer is to specify what is included in the material that may be included, as learning objectives. Let us hope that some clarity comes soon, for the sake of the teachers and the students.

Your learning

So what were the learning objectives for this post?

As a result of reading this post, readers will

  • Reflect on their methods of course development and assessment with respect to using learning objectives.
  • find further resources on the internet regarding learning objectives.
  • Make comments on the good and bad aspects of this post! (oops – I didn’t teach that one)

Coda – Example of some learning objectives

Here is a set of learning objectives for the final section of a service course in quantitative methods for management. It is based on Excel and traditional (normal-based) methods of statistical analysis.  They are far from perfect, including several ideas in many of them.

Evidence Section Learning Objectives

E1.   Explain the process underlying hypothesis tests.
E2.   Interpret a p-value in context for a given set of hypotheses.
E3.   Formulate a null and alternative hypothesis in words for problems involving means, proportions, differences of two means and differences of two proportions.
E4.   Use Excel to perform a hypothesis test on one or two means and interpret the results.
E5.   Use Excel to perform a hypothesis test on one or two proportions and interpret the results.
E6.   Use Excel and PivotTables to perform a Chi-sq test on table data, and interpret the results.
E7.   Explain the concept of Type I and Type II errors and identify which (or neither) has occurred in a given situation.
E8.   Use Excel to plot bi-variate data, find the correlation; interpret the output.
E9.   Use Excel to fit a linear regression line; interpret the output.
E10. Evaluate the validity of statements about the nature of statistical thinking, including the concepts of causation, sample size, models, experimentation, statistical significance, effect size and subjectivity.
E11. Determine which test is most appropriate in a given situation, from: test for a mean or a proportion, difference between proportions, difference of two means: independent samples or paired data, chi-sq test for independence, regression.

 Important concepts or principles

E12. Inferential statistics uses information collected in a sample to make predictions or judgements about the population from which the data is drawn.
E13. An effect is statistically significant when there is evidence from the sample to reject the null hypothesis.
E14. The p-value for a hypothesis test of a claim about a population parameter is the probability of getting, by chance, a sample as least as extreme as the observed one if the null hypothesis is true.

Difficult concepts in statistics

Recently someone asked: “I don’t suppose you’d like to blog a little on the pedagogical knowledge relevant to statistics teaching, would you? A ‘top five statistics student misconceptions (and what to do about them)’ would be kind of a nice thing to see …”

I wish it were that easy. Here goes:

Things that I have found students find difficult to understand and what I have done about them.

Observations

When I taught second year regression we would get students to collect data and fit their own multiple regressions. The interesting thing was that quite often students would collect unrelated data. The columns of the data would not be of the same observations. These students had made it all the way through first year statistics without really understanding about multivariate data.

So from them on when I taught about regression I would specifically begin by talking about observations (or data points) and explain how they were connected. It doesn’t hurt to be explicit. In the NZ curriculum materials for high school students are exercises using data cards which correspond to individuals from a database. This helps students to see that each card, which corresponds to a line of data, is one person or thing. In my video about Levels of measurement, I take the time to show this.

First suggestion is “Don’t assume”.  This applies to so much!

And this is also why it is vital that instructors do at least some of their own marking (grading). High school teachers are going, “Of course”. College professors – you know you ought to! The only way you find out what the students don’t understand, or misunderstand, or replicate by rote from your own notes, is by reading what they write. This is tedious, painful and sometimes funny in a head-banging sort of way, but necessary. I also check the prevalence of answers to multiple choice questions in my on-line materials. If there is a distracter scoring highly it is worthwhile thinking about either the question or the teaching that is leading to incorrect responses.

Inference

Well duh! Inference is a really, really difficult concept and is the key to inferential statistics. The basic idea, that we use information from a sample to draw conclusions about the population seems straight-forward. But it isn’t. Students need lots and lots of practice at identifying what is the population and what is the sample in any given situation. This needs to be done with different types of observations, such as people, commercial entities, plants or animals, geographical areas, manufactured products, instances of a physical experiment (Barbie bungee jumping), and times.

Second suggestion is “Practice”. And given the choice between one big practical project and a whole lot of small applied exercises, I would go with the exercises. A big real-life project is great for getting an idea of the big picture, and helping students to learn about the process of statistical analysis. But the problem with one big project is that it is difficult to separate the specific from the general. Context is at the core of any analysis in statistics, and makes every analysis different. Learning occurs through experiencing many different contexts and from them extracting what is general to all analysis, what is common to many analyses and what is specific to that example. The more different examples a student is exposed to, the better opportunity they have for constructing that learning. An earlier post extols the virtues of practice, even drill!

Connections

One of the most difficult things is for students to make connections between parts of the curriculum. A traditional statistics course can seem like a toolbox of unrelated but confusingly different techniques. It takes a high level of understanding to link the probability, data and evidence aspects together in a meaningful way. It is good to have exercises that hep students to make these connections. I wrote about this with regard to Operations Research and Statistics. But students need also to be making connections before they get to the end of the course.

The third suggestion is “get students to write”

Get students to write down what is the same and what is different between chi-sq analysis and correlation. Get them to write down how a poisson distribution is similar to and different from a binomial distribution. Get them to write down how bar charts and histograms are similar and different. The reason students must write is that it is in the writing that they become aware of what they know or don’t know. We even teach ourselves things as we write.

Graphs and data

Another type of connection that students have trouble with is that between the data and the graph, and in particular identifying variation and distribution in a histogram or similar. There are many different graphs, that can look quite similar, and students have problems identifying what is going on. The “value graph” which is produced so easily in Excel does nothing to help with these problems. I wrote a full post on the problems of interpreting graphs.

The fourth suggestion is “think hard”. (or borrow)

Teaching statistics is not for wusses. We need to think really hard about what students are finding difficult, and come up with solutions. We need to experiment with different ways of explaining and teaching. One thing that has helped my teaching is the production of my videos. I wish to use both visual and text (verbal) inputs as best as possible to make use of the medium. I have to think of ways of representing concepts visually, that will help both understanding and memory. This is NOT easy, but is extremely rewarding. And if you are not good at thinking up new ideas, borrow other people’s ideas. A good idea collector can be as good as or better than a good creator of ideas.

To think of a fifth suggestion I turned to my favourite book , “The Challenge of Developing Statistical Literacy, Reasoning and Thinking”, edited by Dani Ben-Zvi and Joan Garfield. I feel somewhat inadequate in the suggestions given above. The book abounds with studies that have shown areas of challenge or students and teachers. It is exciting that so many people are taking seriously the development of pedagogical content knowledge regarding the discipline of statistics. Some statisticians would prefer that the general population leave statistics to the experts, but they seem to be in the minority. And of course it depends on what you define “doing statistics” to mean.

But the ship of statistical protectionism has sailed, and it is up to statisticians and statistical educators to do our best to teach statistics in such a way that each student can understand and apply their knowledge confidently, correctly and appropriately.

Why engineers and poets need to know about statistics

I’m kidding about poets. But lots of people need to understand the three basic areas of statistics, Chance, Data and Evidence.

Recently Tony Greenfield, an esteemed applied statistician, (with his roots in Operations Research) posted the following request on a statistics email list:

“I went this week to the exhibition and conference in the NEC run by The Engineer magazine. There were CEOs of engineering companies of all sizes, from small to massive. I asked a loaded question:  “Why should every engineer be a competent applied statistician?” Only one, from more than 100 engineers, answered: “We need to analyse any data that comes along.” They all seemed bewildered when I asked if they knew about, or even used, SPC and DoE. I shall welcome one paragraph responses to my question. I could talk all day about it but it would be good to have a succinct and powerful few words to use at such a conference.”

For now I will focus on civil engineers, as they are often what people think of as engineers. I’m not sure about the “succinct and powerful” nature of the words to follow, but here goes…

The subject of statistics can be summarised as three areas – chance, data and evidence (CDE!)

Chance includes the rules and perceptions of probability, and emphasises the uncertainty in our world. I suspect engineers are more at home in a deterministic world, but determinism is just a model of reality. The strength of a bar of steel is not exact, but will be modelled with a probability distribution. An understanding of probability is necessary before using terms such as “one hundred year flood”. Expected values are used for making decisions on improving roads and intersections. The capacity of stadiums and malls, and the provision of toilets and exits all require modelling that relies on probability distributions. It is also necessary to have some understanding of our human fallibility in estimating and communicating probability. Statistical process control accounts for acceptable levels of variation, and indicates when they have been exceeded.

The Data aspect of the study of statistics embraces the collection, summary and communication of data. In order to make decisions, data must be collected. Correct summary measures must be used, often the median, rather than the more popular mean. Summary measures should preferably be expressed as confidence intervals, thus communicating the level of precision inherent in the data. Appropriate graphs are needed, which seldom includes pictograms or pie charts.

Evidence refers to the inferential aspects of statistical analysis. The theories of probability are used to evaluate whether a certain set of data provides sufficient evidence to draw conclusions. An engineer needs to understand the use of hypothesis testing and the p-value in order to make informed decisions regarding data. Any professional in any field should be using evidence-based practice, and journal articles providing evidence will almost always refer to the p-value. They should also be wary of claims of causation, and understand the difference between strength of effect and strength of evidence. Our video provides a gentle introduction to these concepts.

Design of Experiments also incorporates the Chance, Data and Evidence aspects of the discipline of statistics.  By randomising the units in an experiment we can control for other extraneous elements that might affect the outcome in an observational study. Engineers should be at home with these concepts. We have two new videos that illustrate this.

So, Tony, how was that? Not exactly succinct, and four paragraphs rather than one. The videos are optional. I think the Chance, Data, Evidence framework helps provide structure to the explanation.

So what about the poets?

I borrow the term from Peter Bell of Richard Ivey School of Business, who teaches operations research to MBA students, and wrote a paper, Operations Research For Everyone (including poets). If it is difficult to get the world to recognise the importance of statistics, how much harder is it to convince them that Operations Research is vital to their well-being!

Bell uses the term, “poet” to refer to students who are not naturally at home with mathematics. In conversation Bell explained how many of his poets, who were planning to work in the area of human resource management found their summer internships were spent elbow-deep in data, in front of a spreadsheet, and were grateful for the skills they had resisted gaining.

An understanding of chance, data and evidence is useful/essential for “efficient citizenship”, to paraphrase the often paraphrased H. G. Wells. I have already written on the necessity for journalists to have an understanding of statistics. The innovative New Zealand curriculum recognises the importance of an understanding of statistics for all. There are numerous courses dedicated to making sure that medical practitioners have a good understanding.

So really, there are few professions or trades that would not benefit from a grounding in Chance, Data and Evidence. And Operations Research too, but for now that may be a bridge too far.

The flipped classroom

Back in the mid1980s I was a trainee teacher at a high school in Rotorua. My associate teacher commented that she didn’t like to give homework much of the time as the students tended to practise things wrong, thus entrenching bad habits away from her watchful gaze. She had  a very good point! Bad habits can easily be developed when practising solving equations, trigonometry, geometry.

Recently the idea of the “flipped classroom” has gained traction, particularly enabled by near universal access to internet technology in some schools or neighbourhoods. When one “flips” the classroom, the students spend their homework time learning content – watching a video or reading notes. Then the classroom time is used for putting skills to practice, interactive activities, group work, problem-solving – all active things that are better with the teacher around. Having a teacher stand at the front of the room and lecture for a large percentage of the time is not effective teaching practice.

I ws surprised at a teaching workshop to find that many of the teachers were not even aware of the concept of “flipping”. To me this is a case for Twitter as a form of professional development. To address this gap, I am writing about the flipped classroom, especially with regard to statistics and mathematics.

There are two important aspects to flipping – what the students do when they are not in class, and what students do when they are in class.

Work away from class

In theory, classroom “flipping” has always been possible. You could set students notes to read or sections of the textbook to study. In some schools and cultures this is successful, though it does presuppose a high level of literacy. Universities expect students to read, though my experience is that they avoid it if possible – unless they are taking Law, which of course means they can’t avoid it.

Technology has changed the landscape for flipping. With ready access to the internet it is feasible for video and other work to be set remotely for students. Sometimes teachers prepare the material themselves, and sometimes they may specify a YouTube video or similar to watch. This is not as easy as it may sound. As you can see from my critique of videos about confidence intervals, there is a lot of dross from which to extract the gold. And Khan Academy is no exception.

One big advantage of video over a live lecture, even if the video is merely a talking head, is that the student can control the pace and repeat parts that aren’t clear. My experience of lecturing to classes of several hundred students was that the experience was far from personal. I would set the pace to aim at the middle, as I’m sure most do. In later years I put all my lectures into short audio files with accompanying notes. Students could control the pace and repeat parts they didn’t understand. They could stop and think for a bit and do the exercises as I suggested, sometimes using Excel in parallel. They could quickly look through the notes to see if they even needed to listen to the audio. It was much more individualised.

Another advantage was that you can remove errors, stumbles, gaps and tighten up the experience. I’ve found a fifty minute lecture can be reduced to about half the time, in terms of the recording.

Despite this much more individual approach I was still expected to give lectures (that’s what lecturers do isn’t it?) until the Christchurch earthquakes made my mode of delivery expedient and we were able to stop physical lectures. The students could view the delivery of the material without coming to the university. They could then do exercises, also set up on the LMS, with instant feedback.

Work in the classroom

People tend to focus on what happens away from the classroom, when talking about flipped classroom. It is equally important to think hard about what happens in class. Having the teacher and peers there to help when working through problems in mathematics is better than being at a dead-end at home, with no one to help.  But week after week of turning up to class, working on numbered exercises from the textbooks doesn’t sound like much fun.

Taking the content delivery out of the classroom frees up the teacher for all sorts of different activities. It can be a challenge for teachers to change how they think about how to use the time. There are opportunities for more active learning, based on the grounding done on-line. In a mathematics or statistics classroom there is room for creativity and imagination. Debates, group work, competitions, games, looking for errors, peer review and peer-grading are all possibilities. If anyone thinks there is no room for imagination in the teaching of mathematics, they should take a look at the excellent blog by Fawn Nyugen, Finding ways to Nguyen students over.  I wish she had taught my sons. Or me. (Nguyen is pronounced “Win”)

I am currently working with teachers on teaching statistical report-writing. This is something that benefits from peer review and discussion. Students can work separately to write up results, and then read each other’s work. This is done in English and Social Science classes, and language classes. There is much we can learn from teachers in other disciplines.

Potential Problems

Students can also be resistant to change, and some coaching may be needed at the start of the year.

There is a big investment by teachers if they wish to create their own materials. Finding suitable materials on line can take longer than making your own. A team approach could help here, where teachers pool their resources and provide the “at home” resources and links for each other’s classes. I would be cautious not to try to do too much at once in implementing “flipped classroom”. It would probably be wise to start with just one class at a time.

Where internet access is not universal, there needs to be adaptations. It may be that the students can use school resources out of school time. Or students could take the material home on a memory stick.

Special needs

One issue to consider is the students who have special learning needs. In one Twitter discussion it was suggested that the flipped classroom is great because the student can learn the content when they have a helper (parent!) to assist. This is an admirable theory and I might have agreed had I not been on the other side. As a mother of a son with special needs, the thought of homework was often too much for me. The daily battle of life was enough without adding further challenge. In addition my son had been full-on all day and had little capacity for homework even if I had been willing. We need to avoid assuming ideal circumstances.

Try it!

Overall though, in appropriate circumstances, the concept of flipping has a lot going for it. It is always good to try new things.

If you never have a bad lesson or a failed new idea, you aren’t being daring enough!