Pedagogical Content Knowledge

Pedagogical content knowledge for Statistics

Pedagogical content knowledge means knowing how to teach a specific subject, discipline or context. There is a school of thought that the skill of teaching is transferable between subjects, so long as the teacher knows the content. However others argue that teaching strategies differ sufficiently across disciplines to create individual but overlapping bodies of knowledge, called pedagogical content knowledge. To me it is clear that different skills and approaches are needed in the teaching of different disciplines. The methods for teaching a foreign language differ largely from those for teaching history or science or cake decorating or jazz piano. There are also commonalities in all teaching, such as the need to build a relationship between the teacher and student, and building on students’ previous knowledge.

I first learned about the concept of “pedagogical content knowledge” in one of my favourite books – How People Learn: Brain, Mind, Experience and School. This book brings together research into how the brain works, and how people learn, in such a way that teachers can gain from it in their practice. Regarding pedagogical content knowledge, it states “Expert teachers know the kinds of difficulties that students are likely to face; they know how to tap into students’ existing knowledge in order to make new information meaningful; and they know how to assess their students’ progress.”

I fear that one of the reasons that the subject of statistics is not as popular as it deserves to be, is because almost all the teachers at all levels lack pedagogical content knowledge with respect to teaching statistics. I am not saying that the teachers are bad teachers, or ill-meaning, or unintelligent. I am saying that most teachers of statistics do not really know how to teach statistics.

Let us look at some different groups of teachers, starting with the most influential and consequently worst paid.

Primary (elementary) school teachers)

My experience of primary school teachers is that they generally are less comfortable teaching mathematics than reading and writing. Their knowledge and understanding of statistics ranges between trivial and incorrect. Their pedagogical content knowledge for statistics is pretty low. These teachers often teach incorrect graphing methods, and may well perpetuate the idea that probability relates to dice, coins and counters. It is not really their fault. There is such a broad curriculum at that level, that it must be challenging to cover all possibilities in their training. Having said that, a well-funded initiative in professional development could address this issue.

High school teachers

Mostly statistics at high school level  is taught by mathematics teachers, from a mathematical background rather than a statistical one. I have already written about the problems when mathematicians fail to treat statistics as an allied but separate discipline from mathematics. I was greatly heartened last week to meet with forty committed teachers of high school statistics who are embracing the new approach of the New Zealand curriculum toward statistics. They have seen how interesting the subject is and are helping students to make real progress in their learning. This is testament to the dedication and collaboration of the teachers themselves, and the efforts of bodies such as Census @ School and my own Statistics Learning Centre, which are helping to support these teachers. The support from the official channels appears criminally lacking, unco-ordinated, and at times even conflicting.

These teachers were at my workshop on teaching statistical report-writing, because they were aware of their own inadequacies in this area. (Though some were doing a fantastic job already). It is hardly surprising that they feel unprepared for teaching this material when their expertise has been in teaching trigonometry, algebra, measurement and calculus. The pedagogical content knowledge for teaching statistics is very different from teaching mathematics. Statistics is, compared with mathematics, an inexact science, where context is vitally important, and where different correct approaches will produce different numbers as answers to a problem. In statistics the words used are critical, and one word can change the meaning of the sentence completely.

Fortunately there is research undertaken on how better to teach statistics, and the body of pedagogical content knowledge is increasing. Another of my favourite books is “The Challenge of Developing Statistical Literacy, Reasoning and Thinking”, edited by Dani Ben-Zvi and Joan Garfield. This brings together the results of thinking and experimentation to improve the efficacy of statistics teaching. One problem identified by Garfield some years ago was that even students who received A passes in statistics often had a very poor understanding of even the most basic concepts of the subject. It is exciting to read the progress that is being made in developing strategies for teaching statistics in a way that promotes deep understanding that transfers to other problems and disciplines. It is also exciting to live in New Zealand where the findings of the research have been applied to the development of a national curriculum in statistics.

Khan academy

I’d just like to pop in a reference to Khan Academy because, sadly, it has a great influence. I believe that many of the mathematics Khan Academy videos are fairly well taught, in a “boy-next-door” sort of way. However the statistics videos perpetuate the mathematical view of statistics, as they are a product of an archaic curriculum. Khan has NO pedagogical content knowledge of statistics. This is abundantly clear in the approach and errors. I have covered this in earlier posts.

AP Statistics

Advanced Placement Statistics is an American invention of which I have only a tenuous understanding. It appears to be a subject taken at high school level, examined nationally and can count for credit at a tertiary institution. Consequently, though the level is of first year college level, it is taught by high school teachers, which may or may not be to the advantage of the students. I suspect the level of pedagogical content knowledge among the teachers is highly skewed with  a very large bulge at the low end and very thin tail to the high end. (To me the word skewed goes the wrong way, so I prefer to describe the outcome).

Higher education

Statistics at universities is taught by a wide range of people. Teaching assistants have the advantage of recent experience learning the material and may thus be better able to see the challenges of learning the discipline. There will be truly great teachers of statistics among them. Some instructors specialise in the teaching of statistics and help to advance the corporate body of pedagogical content knowledge. Some academics really don’t care about teaching, and just present the material as painlessly (to them) as possible before they head back to their research.

Developing pedagogical content knowledge

I fear I have stated a problem, with very little in the way of solution. Sometimes it is a good start to identify that the problem exists. Part of my aim in my workshop is to validate the efforts of teachers in what is an unfamiliar environment, and explain why they are feeling out of their depth. This diagnosis helps to remove the blame from the teacher, who are then smart enough, with a few suggestions, to work to develop a solution.

It is my intention that this blog is part of a solution. The aim is that through my musings and the comments of others we are able to encourage progress in the teaching of statistics in such a way that will thrill and excite the masses! Failing that, at least not put them off statistics altogether.

Teaching statistical report-writing

Teaching how to write statistical reports

It is difficult to write statistical reports and it is difficult to teach how to write statistical reports.

When statistics is taught in the traditional way, with emphasis on the underlying mathematics the process of statistics is truncated at both ends. When we concentrate on the sterile analysis, the messy “writing stuff” is avoided. Students do not devise their own investigative questions, and they do not write up the results.

Here’s the thing though – in reality, the analysis step of a statistical investigation is a very small part of the whole, and performed at the click of a button or two.

Ultimately the embedding of the analysis back into an investigation should not be a problem. The really interesting part of statistics happens all around the analysis. Understanding the context enriches the learning, transforming the discipline from mathematics to statistics. We can help students embrace the excitement of a true statistical investiation. But in this time of transition, the report-writing aspects are a problem. They are a problem for the learner and for the teacher.

The new New Zealand curriculum for statistics requires report-writing as an essential component of the majority of assessment, particularly at the final year of high school. This is causing understandable concern among teachers, who come predominantly from a mathematical background. I can imagine myself a few years ago saying. “I became a maths teacher so I wouldn’t have to teach and mark essays!” In addition the results from the students are less than stellar, even from capable students. Teachers do not like their students to perform poorly.

All statistics courses should have a component of report-writing, unless they are courses in the mathematics of statistics. The problem here is, like the secondary school teachers in New Zealand, many statistics instructors are dealing with the mathematics more than the application of statistics, and are not confident of their own ability at report-writing themselves. Normal human behaviour is to avoid it. Having taught service statistics courses in a business school for two decades, I have gradually made the transition to more emphasis on report-writing and am convinced that statistical report-writing needs to be taught explicitly, and taught well.

Report-writing is a fundamental and useful skill

For teachers who are uncomfortable with teaching and marking reports, it would be nice to dismiss the process of report-writing  as “not important”. Much of statistics teaching is in a service course, as discussed in my previous blog. It is unlikely that any of these students will ever have to write a report on a statistical analysis, other than as part of the assessment for the course.  So why do we put them and ourselves through this?

You don’t realise whether you understand or not until you try to write it down.

The written word requires a higher level of precision than a thought or a spoken explanation. Your sentences look at you from the page and mock you with their vagueness and ambiguity. I find this out time and again as I blog. What seems like a well thought out argument in my head as I do my morning run, falls to shreds on paper, before being mustered into some semblance of order. It is in writing that we identify the flaws in our understanding. As we try to write our findings we become more aware of fuzzy thinking and gaps in reasoning. As we write we are required to organise our thoughts.

Better critics of other reports

A student who has been required to produce a report of a good standard will be exposed to examples of good and bad reports and will be better able to identify incorrect thinking in reports they read themselves. This is perhaps the most important purpose of a terminal course in statistics. Having said that, it is both heart-warming and alarming to hear from past-students the wonderful things they are doing with the statistics they learned in my one-semester course.

Useful skill for employment

Students need to be able to read and write as part of empowered citizenship. The skill of writing a coherent report in good English is highly sought after by employers, and of great use at university in just about every discipline. It is a transferable skill to many endeavours.

Reports are needed for assessment

On a practical level, if the teacher is going to evaluate understanding they need evidence to work from. A written report provides one form of evidence of understanding.

Report-writing is difficult to teach

Some maths teachers may feel inadequate in teaching “English”, as they see report-writing. They do not have the pedagogical content knowledge in teaching writing that they do for teaching algebra or percentages, for instance. Pedagogical content knowledge is more than the intersection of knowing a subject, and being able to teach in a general sort of way. It is the knowledge of how to teach a certain discipline, what is difficult to learners, and how to help them learn.

Some basic ideas for teaching report-writing

To write at good report you need to understand what is going on, have the appropriate vocabulary, and use a clear structure. Good teaching will emphasise understanding. Getting students to write sentences about output, and sharing them with their peers is a great way to identify misunderstandings. As these sentences are shared, the teacher can model the use of correct technical language. They can say, for instance, “You have the essence correct here, but there are some more precise terms you could use, such as …” Teachers can either give students outlines for reports, or they can give them several good reports and get the students to identify the underlying structure. I am a firm believer in the generous use of headings within a report. They provide signposts for writer and reader alike.

You can see this in my video, Writing up a Time Series Report.

Report-writing requires practice. The assessment report should not be the first report of that type that a student writes. In the world of motivated students with no other demands on their time, it would be great to have them write up one assignment for the practice and then learn from that to produce a better one. I am aware that students tend not to do the work unless there is a grade attached to it, so it can be difficult to get a student to do a “practice report” ahead of the “real assessment.”  There are other alternatives that approximate this, however, which require less input from the teacher. One of these, the use of templates, is explained in an earlier post, Templates for statistical reports – spoon-feeding?

There is nothing wrong with using templates and “sensible sentences”. (not to be confused with “sensible sentencing”, which seems devoid of sense.) There are only so many ways to say that “the median number of pairs of shoes owned by women is ten.” It is also a difficult sentence to make sound elegant. Good reports will look similar. This is not creative-writing – it is report-writing. Sure the marking may be boring when all the reports seem very similar, but it is a small price to pay when you avoid banging your head against the desk at the bizarre and disorganised offerings.

This is but a musing on the teaching of report-writing. Glenda Francis, in  “An approach to report writing in statistics courses” identifies similar issues, and provides a fuller background to the problem. She also indicates that there is much to be done in developing this area of teaching and research. I will be providing professional development in this area over the next month to at least three groups of teachers, and I look forward to learning a great deal from them, as we explore these issues together.

Teaching a service course in statistics

Most students who enrol in an initial course in statistics at university level do so because they have to. I did some research on attitudes to statistics in my entry level quantitative methods course, and fewer than 1% of the students had chosen to be in that course. This is a little demoralising, if you happen to think that statistics is worthwhile and interesting.

Teaching a service course in statistics is one of the great challenges of teaching. A “Service Course” is a course in statistics for students who are majoring in some other subject, such as Marketing or Medicine or Education. For some students it is a terminating course – they will never have to look at a p-value again (they hope). For some students it is the precursor to further applied statistics such as marketing research or biological research. Having said that, statistics for citizens is important and interesting and engaging if taught that way. And we might encourage some students to carry on.

Yet the teachers and textbook writers seem to do their best to remove the joy. Statistics is a difficult subject to understand. Often the way the instructor thinks is at odds with the way the students think and learn. The mathematical nature of the subject is invested with all sorts of emotional baggage.

Here are some of the challenges of teaching a statistics service course.

Limited mathematical ability

It is important to appreciate how limited the mathematical understanding is of some of the students in service courses. In my first year quantitative methods course, I made sure my students knew basic algebra, including rearranging and solving equations. This was all done within a business context. Even elementary algebra  was quite a stumbling block to some students, for whom algebra had been a bridge too far at school. There were students in a postgrad course I taught who were not sure which was larger, out of 0.05 and 0.1, and talked about crocodiles with regard to greater than and less than signs. And these were schoolteachers! Another senior maths teacher in that group had been teaching the calculation of confidence intervals, without actually understanding what they were.

The students are not like statisticians. Methods that worked to teach statisticians and mathematicians are unlikely to work for them. I wrote about this in my post about the Golden Rule, and how it applies at a higher level for teaching.

I realised a few years ago that I am not a mathematician. I do not have the ability to think in the abstract that is part of a true mathematician. Operations Research was my thing, because I was good at mathematics, but my understanding was concrete. This has been a surprising gift for me as a teacher, as it has meant that I can understand better what the students find difficult. Formulas do not tell them anything. Calculating by hand does not lead to understanding. It is from this philosophy that I approach the production of my videos. I am particularly pleased with my recent video about confidence intervals, which explains the ideas, with nary a formula in sight, but plenty of memorable images.

Software

One of my more constantly accessed posts is  Excel, SPSS, Minitab or R?. This consistent interest indicates that the course of software is a universal problem.  People are very quick to say how evil Excel is, and I am under no illusions as to many of the shortcomings. The main point of my post was, however, that it depends on the class you are teaching.

As I have taught mainly business students, I still hold that for them, Excel is ideal. Not so much for the statistical aspects, but because they learn to use Excel. Last Saturday the ideas for today’s posts were just forming in my mind when the phone rang, and despite my realising it was probably a telemarketer (we have caller ID on our phone) I answered it. It was a nice young woman asking me to take part in a short survey about employment opportunities for women in the Christchurch Rebuild. After I’d answered the questions, explaining that I was redundant from the university because of the earthquakes and that I had taught statistics, she realised that I had taught her. (This is a pretty common occurrence for me in our small town-city – even when I buy sushi I am served by ex-students). So I asked her about her experience in my course, and she related how she would never have taken the course, but enjoyed it and passed. I asked about Excel, and she told me that she had never realised what you could do with Excel before, and now still used it. This is not an isolated incident. When students are taught Excel as a tool, they use it as a tool, and continue to do so after the course has ended.

When business students learn using Excel, it has the appearance of relevance. They are aware that spreadsheets are used in business. It doesn’t seem like time wasted. So I stand by my choice to use Excel. However if I were still teaching at University, I would also be using iNZight. And if I taught higher levels I would continue to use SPSS, and learn more about R.

Textbooks

As I said in a previous post Statistics Textbooks suck out all the fun. Very few textbooks do no harm. I wonder if this site could provide a database of statistics texts and reviews. I would be happy to review textbooks and include them here. My favourite elementary textbook is, sadly, out of print. It is called “Taking the Fear out of Data Analysis”, by the fabulously named Adamantis Diamantopoulos and Bodo Schlegelmilch. It takes a practical approach, and has a warm, nurturing style. It lacks exercises. I have used extracts from it over the years. The choice of textbook, like the choice of software, is “horses for courses”, but I think there are some horses that should not be put anywhere near a course. I do wonder how many students use textbooks as anything other than a combination lucky charm and paper weight.

In comparison with the plethora of college texts of varying value, at high-school level the pickings for textbooks are thin. This probably reflects the newness of the teaching of statistics at high-school level.  A major problem with textbooks is that they are so quickly out of date, and at school level it is not practical to replace class sets too often.

Perhaps the answer is online resources, which can be updated as needed, and are flexible and give immediate feedback. 

Emotional baggage

I was less than gentle with a new acquaintance in the weekend. When asked about my business, I told him that I make on-line materials to help people teach and learn statistics. He proceeded to relate a story of a misplaced use of a percentage as a reason why he never takes any notice of statistics. I have tired of the “Lies, damned lies, and statistics” jibe and decided not to take it lying down. I explained that the world is a better place because of statistical analysis. Much research, including medical would not be possible in the absence of methods for statistical analysis. An understanding of the concepts of statistics is a vital part of intelligent citizenship, especially in these days of big and ubiquitous data.

I stopped at that point, but have pondered since. What is it that makes people so quick to denigrate the worth of statistics? I suspect it is ignorance and fear. They make themselves feel better about their inadequacies by devaluing the things they lack. Just a thought.

This is not an isolated instance. In fact I was so surprised when a lighthouse keeper said that statistics sounded interesting and wanted to know more, that I didn’t really know what to say next! You can read about that in a previous post. Statistics is an interesting subject – really!

But the students in a service course in statistics may well be in the rather large subset of humanity who have yet to appreciate the worth of the subject. They may even have fear and antipathy towards the subject, as I wrote about previously. Anxiety, fear and antipathy for maths, stats and OR.

People are less likely to learn if they have negative attitudes towards the subject. And when they do learn it may well be “learning to pass” rather than actual learning which is internalised.

So what?

Keep the faith! Statistics is an important subject. Keep trying new things. If you never have a bad moment in your teaching, you are not trying enough new things. And when you hear from someone whose life was changed because of your teaching, there is nothing like it!

The median outclasses the mean

The median suffers from poor marketing.

All my time at school the “average” was always calculated as the arithmetic mean, by adding up all the scores and then dividing by the number of scores. When we were taught about the median, it seemed like an inferior version of the mean. It was the thing you worked out when you weren’t smart enough to add and divide. It was used for house prices, and that was about it. Of course the mean was the superior product! Why wouldn’t you use the mean?

I’ve been preparing resources for teaching the fabulous new New Zealand curriculum, and have been brought face-to-face with my prejudices. It strikes me that the median has had very poor representation.

Public opinion of the median and mean

I put a question on Facebook and Twitter to see what people felt about the mean and the median. I briefly explained what each was, then asked which one they thought was better. Some people had no idea what I was talking about, but most felt that the mean was the superior statistic. The following are a selection of responses:

The mean, but I don’t know why.. maybe that’s just what we were taught to use when I was back in school (a long time ago!) lol

When I think of “average” I always think of the mean. I don’t know if it’s actually better though

well the median is a real pain to work out. you have to make a list of all the numbers, in order, and then count how many they are and then go to the middle. PAIN IN THE BUM. the average… well that is somewhat quicker to do, no? and i don’t see the point in the median at all. unless well no, there is just no need for it. who cares what the15th person in the class got on a test? the lowes and highes is much more interesting. As i remember it, the mode is the most commonly occuring number out of a set of numbers… i think of this as the “mode” or in English (not French), the ‘fashionable” number. oh and it stresses me how all 3 start with Ms cos that is confusing. which is why i like to use the word average.

The mean, which I’m guessing is the same as the average? When the media refer to real estate stats they always use median price, which can distort reality, we would prefer the average price. (From a real estate agent)

I don’t really think it’s a case of which is better. They’re two different things aren’t they? I think it’s usually easier to work out the average.

A number of my Facebook friends did know about statistics, and responded in favour of the median in most cases. This was an interesting comment:

“It depends. Everyone who proof read my thesis was like why on earth are you using the median – no one uses it. And most of the other similar primate studies I’ve read use the mean (except one, that was published by my associate supervisor). But my means were off their rocker, and I’m pretty sure my medians were a much better representation of reality in this case. It makes making comparisons between studies a little awkward though.

Why NOT use the median all the time?

I am hard pressed to find an instance where the mean is actually a better measure of central tendency than the median. The purpose of the mean or median (or mode) is to provide a one number summary of a set of data. The whole idea of the mean is actually quite tricky, as you can read in one of my early posts about explaining what the mean is. Generally the summary value is used to compare with another sample or population.

In my lectures I often illustrated times when the median is a better summary measure of a sample or population than the mean. This is quite common in notes and YouTube videos. Never once did I show where the mean was preferred to the median! So why were/are we so loyal to the mean, bringing out the median for special occasions and real estate?

I think there are two answers, both of them no longer valid. It is a question of legacy.

Time and ease to calculate

Despite first appearances, for anything larger than a trivial sample the mean is actually easier to calculate than the median. Putting a set of 100 values in order by hand is no easy task. (Pain in the bum, as my friend so elegantly expressed it.) Adding up scores and dividing by 100 is a walk in the park in comparison.  In the early 1980s when I learned programming (in Fortran, Pascal and Cobol), writing a sorting program was far from trivial and a large set of numbers would take a large amount of time to sort. Only in later years, as computing power has expanded, has it been possible to get a computer to calculate a median.

Formulas for confidence intervals

Means behave nicely and give nice mathematical results when manipulated. Because of this we can calculate confidence intervals using a nifty little formula and statistical tables. Until bootstrapping by computer  became do-able on a large and small scale, there was no practical way to perform inference on a number of very useful statistics, including the median and the inter-quartile range.

Conclusion: the median is better

A median is intrinsically understandable. It is the middle number when the values are put in order. End of story. – Well not quite – you do have that slightly tricky thing where the sample is even and you have to average the middle two terms, but apart from that it is easy!

A median is not affected by outliers. I learned a new term for this when I was reading up in preparation for writing this post. The term is “resistant” and I learned it from one of Mr Tarrou’s videos for AP Statistics. I found these videos after my tirade against videos on confidence intervals. Tarrou’s videos are long and a bit more mathematical than I would like. (He can’t help it – he is a maths teacher and the AP Statistics syllabus seems to have been devised by mathematical statisticians trying to put students off ever taking the subject again.) But they are GOOD. Tarrou’s videos are sound, and interesting and well put together. I will be recommending them as complementary to my own offerings. (Because I sure as heck don’t want to have to do all that icky mathsy stuff).

But I digress. The median is “resistant” because it is not at the mercy of outliers. There are lots of great examples, including in Mr Tarrou’s video. If you have a median of 5 and then add another observation of 80, the median is unlikely to stray far from the 5. However a mean is a fickle beast, and easily swayed by a flashy outlier.

The main disadvantage I can see for the median is that it can be a bit jumpy in small samples made up of discrete values. I guess if you have two well-behaved populations that are very similar and you want to see precise differences then the means might just be better – but even then you would possibly be over-interpreting small differences.

I have found it very interesting observing the behaviour of confidence intervals for the difference of two medians, compared with confidence intervals for the difference in two means. While I was preparing materials for our on-line resource, I performed nine such tests on different real data taken from students at university. The scores are very jumpy, and the differences between the medians often include exactly zero. Consequently the confidence intervals of the difference of two medians quite often have zero as their lower bound. This provides a challenge in interpretation, as I had not met this often when looking at the differences between means. However, it also illuminates the odd relationship we have with zero. Just because a confidence interval for a difference of two means is (-0.13, 3.98) and includes a zero, it is tempting to conclude that there is no significant difference. But is -0.13 really any different from zero in practical terms? The other point is that we should be leaving the confidence interval as it is, rather than stretching it into further inference.

Word on the web

I did a little surfing to see what the word on the web was.  To find out who said what, drop the entire phrase into Google. (Ah ‘tis a wonderful we live in, indeed)

• “The mean is the one to use with symmetrically distributed data; otherwise, use the median.” Hmm – but if the data is symmetric, surely the mean = the median?
• “An important property of the mean is that it includes every value in your data set as part of the calculation. In addition, the mean is the only measure of central tendency where the sum of the deviations of each value from the mean is always zero. “ Ok – hard to argue with that.
• “Calculation of medians is a popular technique in summary statistics and summarizing statistical data, since it is simple to understand and easy to calculate, while also giving a measure that is more robust in the presence of outlier values than is the mean.” Totally!
• “However, when the sample size is large and does not include outliers, the mean score usually provides a better measure of central tendency. “(Then goes on to give an example of when the median is better.)
• “Use the median to describe the middle of a set of data that does have an outlier. Advantages of the median: Extreme values (outliers) do not affect the median as strongly as they do the mean, useful when comparing sets of data, it is unique – there is only one answer.
  Disadvantages of the median:  Not as popular as mean.” (Not as popular??!)

Sorry median  – you do not win X-Factor for summary statistics. You may be more robust, and less fickle, not to mention easier to understand, but you just aren’t as popular!

I can feel a video coming on – the median has been relegated to the periphery long enough!

Is statistical enquiry a cycle?

What is the statistical enquiry cycle and why is it a cycle? Is it really a cycle?

The New Zealand curriculum for Mathematics and statistics was recently held up as an example of good practice with regard to statistics. Yay us! In New Zealand the learning of statistics starts at the beginning of schooling and is part of the curriculum right through the school years. Statistics is developed as a discipline alongside mathematics, rather than as a subset of it. There are mathematics teachers who view this as an aberration, and believe that when this particular fad is over statistics will go back where it belongs, tucked quietly behind measurement, algebra and arithmetic. But the statisticians rejoice that the rich and exciting world of real data and detective work is being opened up to early learners. The outcome for mathematics and statistics remains to be seen.

A quick look over the Australian curriculum shows ostensibly a similar emphasis with regard to content at most levels.  The big difference (at first perusal) is that the New Zealand curriculum has two strands of statistics – statistical investigation, and statistical literacy, whereas the Australian curriculum has the more mathematical approach of “Data representation and interpretation”.  Both include probability as another strand.

Data Detective Cycle

In the New Zealand curriculum, the statistical investigation strand at every level refers to the “Statistical enquiry cycle”, shown here, which is also known as the PPDAC cycle. This is a unifying theme and organising framework for teachers and learners.

The data detective poster

This link takes you to a fuller explanation of the statistical enquiry cycle and its role at the different levels of the school curriculum. Note that the levels do not correspond to years. Click here to see the correspondence. The first five levels correspond to about 2 years each, whereas levels 6,7 and 8 correspond to the final three years of high school. So a child working on level 3 is generally aged about 10 or 11.

As I provide resources to support teaching and learning within the NZ curriculum I have become more aware of this framework, and have some questions and suggestions. I have made a table from which I hope to develop another diagram that students at higher levels can engage with, particularly with regard to the reporting aspects. As this is a work in progress you will have to wait!

Origins

Let’s look at the origins of the diagram and terminology. Maxine Pfannkuch (an educator) worked with Chris Wild (a statistician) to articulate what it is that statisticians do. They published their results in the international statistical review in 1999 and contributed the chapter “Towards an understanding of statistical thinking” in “The Challenge of Developing Statistical Literacy, Reasoning and Thinking”, edited by Dani Ben-Zvi and Joan Garfield. The statistical enquiry cycle has consequently been promulgated in the diagram and description referred to above. There is sound research behind this, and it makes good sense as a way of explaining what statisticians do.

Diagrams

I love diagrams. Anyone who has viewed my videos will know this. I spend a great deal of mental energy (usually while running) trying to work out ways to convey ideas in a visual way that will help people to learn, understand and remember. I also do NOT believe in the fad of learning styles, but rather I believe that all learners will gain from different presentations of concepts. I also believe that it is a useful discipline for a teacher to create different ways of expressing concepts. I am rather fussy about diagrams, however, as our Honours students would attest. I have a particular problem with arrows which mean different things in different places. If an arrow denotes passage of time in one instance it should do so in all instances, or a different style of arrow should be employed.

No way in or out

A problem I have with the PPDAC “Cycle” being a cycle is that it seems to imply that we can come in at any point and that there is no escape. If there is a logical starting point, and the link back to it is not one of process, then that should be indicated. Because the arrows are all the same style in the PPDAC diagram, it is also difficult to see a way out of the cycle. As a learner I would find it a little daunting to think that I could never escape! I am also concerned about understanding in what way does a Conclusion lead to a Problem? Surely the whole point of the word “Conclusion” is that it concludes or ends something?

To me there are at least three linkages between the Problem and the Conclusion. First of all, while in the Problem stage, we need to think about what we want to be able to say in the future Conclusion stage.  We may not know which way our conclusion will go, though we will probably have an opinion, or even a hope! (I am too post-modern in my thinking to believe in the objectivity of the researcher.) For instance we may want to be able to say – There is (or is not) evidence that women own more pairs of shoes than men. Another linkage is that when we write up our conclusion we must refer back to the original problem. And the third linkage comes from a comment Jean Thompson made on my blog about teaching time series without many computers. “Often the answer from a good statistical analysis is more questions”.  One conclusion can lead to a new problem.

I found a similar diagram online which is more sequential, starting with the problem and working vertically through the steps, with a link at the end going back to the beginning. I like this, because it does give an idea of conclusion and moving on, rather than being caught in some endless cycle. The reality for students is that they will generally do some project, which will start with a problem and end with a conclusion. Then they will move on to an unrelated project. It has also been my experience as a practitioner.

In my experience the cyclical behaviour which this diagram portrays is generally more within the cycle than over the whole cycle. For instance one may be part way through the data collection and realise that it isn’t going to work, and go back to the “Plan” stage. Some of these extra loops are suggested in my table.

Reporting

For students at a higher level who are required to write reports, it is difficult to see how the report fits in with the cycle. The “Conclusion” step includes “communication”, which could imply a report. However reports often include most of the steps, particularly when their purpose is to satisfy an assessment requirement.

Existing datasets

It is also difficult to apply the cycle in a non-cynical way to work with existing datasets. Often, in the interests of time and quality control, students are given a dataset. In reality they start, not at the Problem step, but somewhere between the Data step and the Analysis step. In their assessments they are required to read around the topic and use their imaginations to come up with the problem, look at how the data was collected, and move on from there.  This is not always the case, but it is for NCEA level 3 Bivariate Investigation, Time Series analysis and Formal Inference areas (called ‘standards’). The only area where they really do plan and collect the data is in the Experimental Design standard. Might it not be helpful to provide an adapted plan that takes into account these exigencies? Let us be explicit about it rather than coyly pretend that the data wasn’t driving everything?

In general I like the concept of the statistical enquiry cycle, and I am happy that it is providing a unifying theme to the curriculum. However, particularly at higher levels, I think it needs a bit of tweaking, taking into account the experience of teachers and learners.  If it is to hold such an important place in a curriculum that is leading the world, it deserves on-going attention.

Disclaimer

This is a blog and not an academic journal. The ideas I have contemplated need a lot more thought and background reading, but I do not have the time or the university salary to support such a luxury right now. Maybe someone else does!

Good, Bad and Wrong: Videos about Confidence Intervals

Videos are useful teaching and learning resources

There is much talk about “flipped classrooms” and the wonders of Khan Academy, YouTube abounds with videos about everything…really! Even television news reports show YouTube clips. Teachers and instructors use videos in their teaching, and get their students to watch them at home, ready to build on in class time. A well put-together video can provide a different way of looking at a problem that helps a student to learn. Videos are endlessly patient and can be paused and watched at the students’ pace. (See my earlier post on multimedia for a fuller discourse on good multimedia.) The problem is: How is a teacher to know what is good and what is not? This seems to be especially difficult in an area like statistics.

I decided this week to see what was on offer and summarise for you all. To narrow it down I chose the topic of Confidence intervals. This topic is pretty universal to statistics courses, and is conceptually tricky. I wanted to see if there was a quick way of working out if a video is any good or not, without having to watch them. I was prepared to suffer so that my readers would not have to.

Videos about confidence intervals are mostly awful

And suffer I did. Not to beat around the bush – many of the videos I watched were awful. There is no other word for it. Not only were they slow, boring, mathematically based, unscripted and unedited, but many of them were just plain wrong. Back in 2008 I went looking for a video about confidence intervals for my students, and realised I had to make my own. It is still true. I do not doubt that the videos are well intentioned. Many of them may have been made (as mine were originally) for a specific class (or family member), and thus were not intended for a larger audience. Maybe those ones should come with disclaimers. “I’m not sure I really know what I am talking about – view at your own risk.”

I put “Confidence Intervals” into the YouTube search engine and examined nine of the top offerings. Mostly I went for the videos with the most views, as this would appear to be a way of filtering out poor material. (wrong again, as you will see) I also included two of my own videos.

Most these videos should not be seen by a wider audience. No – that’s not right – most of them should not be seen – by anyone. The impression they give of statistics is of a bumbling professor talking about formulas and looking up tables. Nothing in them gives a single hint about how interesting, applied and relevant the subject of statistics is. Maybe there needs to be a wikipedia approach to supposedly educational videos to provide quality control.There is just one video other than my own two that I approve of – by Keith Bower. (Biased, I know – feel free to respond.)

A possible way if you wish to find useful videos, might be to get the students to find a video they think is good, then you check that it is correct. Trying to find a good video about statistics is not a good use of your time – unless of course, you just go straight to the Statistics Learning Centre channel or Keith Bower. 

If any of you gentle readers have a video you think is worth a second look, please put the link in the comments.

Brief reviews of ten videos on Confidence Intervals in no especial order except that I left the three good ones until last

I started with the videos that appeared at the top. They have paid to be first in the list, so I thought they might be good. As it turns out they are very similar to each other and from the same stable, it seems. I found them lacklustre, though not totally harmful.

1. Statistics – Confidence Intervals
Channel: EducatorVids2                Videos on the channel: 1192
17268 views  Loaded 24 Oct 2011 (32 views per day)    Duration 3:25

This video and the next one are part of some sort of course. This video seemed to start in the middle of a lecture “Now let’s go on to some examples”. The layout was utilitarian, with a talking head, and a screen showing the  working. The video, like most of them, was not scripted. The content was based on a Mathematical example with no context. I don’t really know what she was talking about. But at least it was short.

---

2. Statistics: Confidence Intervals (Difference in Means)
Channel Educator.com                    Videos on the channel: 1914
7381 views, loaded 5 Nov 2009  (6 views per day) Duration 3:46

Very similar format to EducatorVids2. The bulk of the content was around a medical example with  7 subjects. Again it was not scripted. The  computation was tedious so that I had to fast forward.

---

And here is the one you have all been waiting for: Khan Academy. I should know better than to suggest that the mighty Khan is less than perfect (my previous post about KA  continues to provoke defensive vitriol.) But here goes:

3. Confidence Interval 1
Khan Academy                                  Videos on the channel: 3492
167, 213 views, loaded 28 Oct 2010 (186 views per day) Duration 14:03
246 likes 20 dislikes.

Like all Khan Academy videos (as far as I know) the format is very simple with a black screen with printed example. Again the video is not scripted, and consists of a lot of repetition as Khan doesn’t like empty air while he is writing.It is actually a lead into confidence intervals, doing a theoretical exercise involving the sampling distribution. Thus it talks about probabilities.  It would have been better to entitle it, preparation for confidence intervals, as it doesn’t actually teach about confidence intervals, and includes probability. Khan included steps to using tables to find a t value. This video was really not nice.  And it took 14 minutes! That is 14 minutes I will never have again. It is also a long time to find out that it doesn’t actually teach about confidence intervals. This video is one of the worst of the ten I viewed, and has far more views than it ought.

---

4. Statistics is easy: Confidence Interval
aghasemi4u                           Videos on the channel: 4 about statistics
a remarkable 296,456 views, loaded 23 March 2007, (136 per day)  Duration 5:00
186 likes and 83 dislikes

This video was simple and reasonably well put together, with nice diagrams, but only three slides in its five minutes.  The narration is unscripted and uses probability to describe the confidence interval (wrong!). There was a focus on the mathematical formula.

---

5. . Confidence Intervals
Madonna USI                          Videos on the channel: 22
18,403 views        Uploaded 9 Nov 2009 (15 per day) Duration 9:42
102 likes, 2 dislikes

A brief description of what confidence intervals are as well as a couple examples.Live person working on a whiteboard. Refers to a textbook. Very slow. Definition wrong – Says that we are 95% confident that the value that we found is within the range. I’m hoping this is just a slip of the tongue, but it should have been editted out.

---

6. How to calculate Confidence Intervals and Margin of Error
Statistics is fun                  Videos on the channel: 80
25,750 views.   Uploaded 12 July 2011 (40 per day) Duration: 6:44
145 likes, 3 dislikes

Summarised before and after, which can be tedious. Mathematically based. Slick graphics, but glacially slow in parts. Gives an example with no context. This is not statistics! Tedious. To be fair, there are lots of positive comments, and as the title says “how to calculate confidence intervals” there is no requirement to explain what they are when you get them. The channel is all about “how to calculate” and nothing about context, so I think it is a bit of a misnomer to call it “Statistics is fun”.

---

7. Confidence Intervals Part1 YouTube
Larry Shrewsbury Videos on the channel: 15
82,006 views. Uploaded on 12 Jul 2009 (128  per day) Duration 7:42
136 likes 53 dislikes

Part of an enterprise “Taking the Sadistics out of learning Statistics”
I found the voice irritating as it seems patronising. However some people find my accent distracting, (wot eksent?) so I can’t really be too hard on that. Very formula based, looking at the mathematics rather than the interpretation. The best part was an interesting animation – very nice way of looking at traditional confidence intervals that I hadn’t seen before.

---

Here are the three good videos:

8. The history, use and certain limitations of confidence intervals in statistics.
Keith Bower   Videos on the channel: 49
32,883 views. Uploaded 5 Jan 2009 (21 views per day) Duration:3:25
66 likes, 5 dislikes

Keith’s presentation isn’t visually exciting, but he is correct and clear and that goes a long way. His is just a talking head – but he is an interesting presenter and very fluent. His video has branching, such that you can click to go to another video if needed. I’ve found all his videos sound and sensible. (I got “p is low, null must go” from one of his videos.)

---

9. Confidence Intervals in Excel
UCMSCI              Videos on the channel
17797 views  Loaded 25 Dec 2008 (11 per day)
26 likes 0 dislikes

This was one of my earlier videos. It is scripted with visuals to help in comprehension. It takes the classical approach to confidence intervals and puts emphasis on the idea of level of confidence. Addresses the aspects that affect the width of confidence intervals. Discusses the formula for confidence intervals, and shows how to use Excel to calculate them. (I don’t think I really loaded it on Christmas day! Maybe some strange dateline thing)

10. Understanding Confidence Intervals: Statistics Help
Statistics Learning Centre   Videos on the channel: 19
550 views    Uploaded 26 March 2013  (31 per day)  6 likes 0 dislikes Duration: 4:02

This video will disappoint the mathematicians, as there are no formulas. But students love it. The point of the video is to explain what a confidence interval is, and what things affect the size of the interval. It makes use of diagrams and examples to help students understand the concepts. It is tightly scripted and edited with no wasted time. People can always pause if they need to, but it is difficult to speed up a slow presentation.

---

Epilogue (Obituary?)

And there you have it, folks – there is no easy way to work out which videos are going to be useful for your students without watching them all. Sorry. And if you expect me to go through this again with another topic, you clearly didn’t get the subtext.

Which comes first – problem or solution?

In teaching it can be difficult to know whether to start with a problem or a solution method. It seems more obvious to start with the problem, but sometimes it is better to introduce the possibility of the solution before posing the problem.

Mathematics teaching

A common teaching method in mathematics is to teach the theory, followed by applications. Or not followed by applications. I seem to remember learning a lot of mathematics with absolutely no application – which was fine by me, because it was fun. My husband once came home from survey school, and excitedly told me that he was using complex numbers for some sort of transformation between two irregular surfaces. Who’d have thought? I had never dreamed there could be a real-life use for the square root of -1. I just thought it was a cool idea someone thought up for the heck of it.

But yet again we come to the point that statistics and operations research are not mathematics. Without context and real-life application they cease to exist and turn into … mathematics!

Applicable mathematics

My colleague wrote a guest post about “applicable mathematics” which he separates from “applied mathematics”. Applicable maths appears when teachers make up applications to try to make mathematics seem useful. There is little to recommend about applicable maths. A form of “applicable maths” occurs in probability assessment questions where the examiner decides not to tell the examinee all the information, and the examinee has to draw Venn diagrams and use logical thinking to find out something that clearly anyone in the real world would be able to read in the data! I actually enjoy answering questions like that, and they have a point in helping students understand the underlying structure of the data. But I do not fool myself into thinking that they are anywhere near real-life. Nor are they statistics.

Which first – theory or application?

So the question is – when teaching statistics and operations research, should you start with an application or a problem or a case, and work from there to the theory? Or do students need some theory, or at least an understanding of basic principles before a case or problem can have any meaning? Or in a sequence of learning do we move back and forward between theory and application?

My first off response is that of course we should start with the data, as many books on the teaching of statistics teach us. Well actually we should start with the problem, as that really precedes the collection of the data. But then, how can we know what sorts of problems to frame if we don’t have some idea of what is possible through modelling and statistics? So should we first begin with some theory? The New Zealand Curriculum emphasises the PPDAC cycle, Problem, Plan, Data, Analysis, Conclusion. However, in order to pose the problem in the first place, we need the theory of the PPDAC cycle itself. The answer is not simple and depends on the context.

I have recently made a set of three videos explaining confidence intervals and bootstrapping. These are two very difficult topics that become simple in an instant. What I mean by that is, until you understand a confidence interval, it makes no sense, and you can see no reason why it should make sense. You go through a “liminal space” of confusion and anxiety. Then when you emerge out the other side, instantly confidence intervals make sense, and it is equally difficult to see what it was that made them confusing. This dichotomy makes teaching difficult, as the teacher needs to try to understand what made the problem confusing.

I present the idea of a confidence interval first. Then I use examples. I present the idea of bootstrapping, then give examples. I think in this instance it is helpful to delineate the theory or the idea in reasonably abstract form, interspersed with examples. I also think diagrams are immensely useful, but that’s another topic.

Critique of AtMyPace: Statistics

What prompted these thoughts about “which comes first” was a comment made about our “AtMyPace: Statistics” iOS app.

The YouTube videos used in AtMyPace:Statistics were developed to answer specific needs in a course. They generally take the format of a quick summary of the theory, followed by an example, often related to Helen and her business selling choconutties.

The iOS app, AtMyPace:Statistics was set up as a way to capitalise on the success of the YouTube videos, and we added two quizzes of ten True/false questions to complement each of the videos. We also put these same quizzes in our on-line course and found that they were surprisingly popular. In a way, they are a substitute for a textbook or notes, but require the person to commit one way or the other to an answer before reading a further explanation. We had happened on a effective way of engaging students with the material.

AtMyPace:Statistics is not designed to be a full course in statistics, but rather a tool to help students who might be struggling with concepts. We have also developed a web-based version of AtMyPace:Statistics for those who are not the happy owners of iOS devices. At present the web version is a copy of the app, but we will happily add other questions and activities when the demand arises.

I received the following critique of the AtMyPace: Statistics app:

“They are nicely done but very classical in scope. The approach is tools-oriented using a few “realistic” examples to demonstrate the tool. This could work for students who need to take exams and want accessible material.”

Very true. The material in AtMyPace:Statistics is classical in scope, as we focus on the material currently being taught in most business schools and first year statistics service courses. We are trying to make a living, and once that is happening we will set out to change the world!

The reviewer continues,

“ I think that in adult education you should reverse the order and have the training problem oriented. Take a six sigma DMAIC process as an example. The backbone is a problem scheduled to be solved. The path is DMAIC and the tools are supporting the journey. If you want to do it that way you need to tailor the problem to the audience. “

In tailored adult education it is likely that a problem-based approach will work. I would strongly recommend it.

I had an interesting discussion some time ago with a young lecturer working in a prestigious case-based MBA programme in North America. The entire MBA is taught using cases, and is popular and successful. My friend had some reservations about case-based teaching for a subject like Operations Research which has a body of skills which are needed as a foundation for analysis. Statistics would be similar. The question is making sure the students have the necessary skills and knowledge, with the ability to transfer to another setting or problem. Case-based learning is not an efficient way to accomplish this.

Criticism on Choosing the Test procedure

In another instance, David Munroe commented on our video “Choosing which statistical test to use”, which receives about 1000 views a week.  In the video I suggest a three step process involving thinking about what kind of data we have, what kind of sample, and the purpose of the analysis. The comment was:

Myself I would put purpose first. The purpose of the analysis determines what data should be collected – and more data is not necessarily more informative. In my view it is more useful to think ‘what am I trying to achieve’ with this analysis before collecting the data (so the right data have a chance to be collected). This in contrast to: collecting the data and then going ‘now what can I get from this data?’ (although this is sometimes an appropriate research technique). I think because we’ve already collected the data any time we’re illustrating particular modelling tools or statistical tests, we reinforce the ‘collect the data first then worry about analysis’ approach – at least subconsciously.

Thanks David! Good thinking, and if I ever redo the video I may well change the order. I chose the order I did, as it seemed to go from easy to difficult. (Actually I don’t remember consciously thinking about the order – it just fell out of individual help sessions with students.)  And the diagram was developed in response to the rather artificial problems I was posing!

I’ll step back a bit and explain. One problem I have seen in teaching Statistics and Operations Research is that students fail to make connections. They also compartmentalise the different aspects and find it difficult to work out when certain procedures would be most useful. I wrote a post about this. In the statistics course I wrote a set of scenarios describing possible applications of statistical methods in a business context. The students were required to work out which technique to use in each scenario and found this remarkably difficult. They could perform a test on difference of two means quite well, but were hard-pressed to discern when the test should be used. So I made up even more questions to give them more practice, and designed my three step method for deciding on the test.  This helped.

I had not thought of it as a way to decide in a real-life situation which test to use. Surely that would be part of a much bigger process.  So my questions are rather artificial, but that doesn’t make them bad questions. Their point was to help students make linkages between different parts of the course. And for that, it works.

Bring on the criticism

I would like to finish by saying how much I appreciate criticism. It is nice when people tell me they like my materials. I feel as if I am doing something useful and helping people. I get frequent comments of this type on my YouTube site.  But when people make the effort to point out gaps and flaws in the material I am extremely grateful as it helps me to clarify my thinking and improve the approach. If nothing else, it gives me something to talk about in my blog. It is difficult producing material in a feedback vacuum.  So keep it coming!

Context – if it isn’t fun…

The role of context in statistical analysis

The wonderful advantage of teaching statistics is the real-life context within which any applicaton must exist. This can also be one of the difficulties. Statistics without context is merely the mathematics of statistics, and is sterile and theoretical.  The teaching of statistics requires real data. And real data often comes with a fairly solid back-story.

One of the interesting aspects for practicing statisticians, is that they can find out about a wide range of applications, by working in partnership with specialists. In my statistical and operations research advising I have learned about a range of subjects, including the treatment of hand injuries, children’s developmental understanding of probability, the bed occupancy in public hospitals, the educational needs of blind students, growth rates of vegetables, texted comments on service at supermarkets, killing methods of chickens, rogaine route choice, co-ordinating scientific expeditions to Antarctica and the cost of care for neonatals in intensive care. I found most of these really interesting and was keen to work with the experts on these projects. Statisticians tend to work in teams with specialists in related disciplines.

Learning a context can take time

When one is part of a long-term project, time spent learning the intricacies of the context is well spent. Without that, the meaning from the data can be lost. However, it is difficult to replicate this in the teaching of statistics, particularly in a general high school or service course. The amount of time required to become familiar with the context takes away from the time spent learning statistics. Too much time spent on one specific project or area of interest can mean that the students are unable to generalise. You need several different examples in order to know what is specific to the context and what is general to all or most contexts.

One approach is to try to have contexts with which students are already familiar. This can be enabled by collecting the data from the students themselves. The Census at School project provides international data for students to use in just this way. This is ideal, in that the context is familiar, and yet the data is “dirty” enough to provide challenges and judgment calls.

Some teachers find that this is too low-level and would prefer to use biological data, or dietary or sports data from other sources. I have some reservations about this. In New Zealand the new statistics curriculum is in its final year of introduction, and understandably there are some bedding-in issues. One I perceive is the relative importance of the context in the students’ reports. As these reports have high-stakes grades attached to them, this is an issue. I will use as an example the time series “standard”. The assessment specification states, among other things, “Using the statistical enquiry cycle to investigate time series data involves: using existing data sets, selecting a variable to investigate, selecting and using appropriate display(s), identifying features in the data and relating this to the context, finding an appropriate model, using the model to make a forecast, communicating findings in a conclusion.”

The full “standard” is given here: Investigate Time Series Data This would involve about five weeks of teaching and assessment, in parallel with four other subjects.(The final 3 years of schooling in NZ are assessed through the National Certificate of Educational Achievement (NCEA). Each year students usually take five subject areas, each of which consists of about six “achievement standards” worth between 3 and 6 credits. There is a mixture of internally and externally assessed standards.)

In this specification I see that there is a requirement for the model to be related to the context. This is a great opportunity for teachers to show how models are useful, and their limitations. I would be happy with a few sentences indicating that the student could identify a seasonal pattern and make some suggestions as to why this might relate to the context, followed by a similar analysis of the shape of the trend. However there are some teachers who are requiring students to do independent literature exploration into the area, and requiring references, while forbidding the referencing of Wikipedia.

This concerns me, and I call for robust discussion.

Statistics is not research methods any more than statistics is mathematics. Research methods and standards of evidence vary between disciplines. Clearly the evidence required in medical research will differ from that of marketing research. I do not think it is the place of the statistics teacher to be covering this. Mathematics teachers are already being stretched to teach the unfamiliar material of statistics, and I think asking them and the students to become expert in research methods is going too far.

It is also taking out all the fun.

Keep the fun

Statistics should be fun for the teacher and the students. The context needs to be accessible or you are just putting in another opportunity for antipathy and confusion. If you aren’t having fun, you aren’t doing it right. Or, more to the point, if your students aren’t having fun, you aren’t doing it right.

Some suggestions about the role of context in teaching statistics and operations research

• Use real data.
• If the context is difficult to understand, you are losing the point.
• The results should not be obvious. It is not interesting that year 12 boys weigh more than year 9 boys.
• Null results are still results. (We aren’t trying for academic publications!)
• It is okay to clean up data so you don’t confuse students before they are ready for it.
• Sometimes you should use dirty data – a bit of confusion is beneficial.
• Various contexts are better than one long project.
• Avoid the plodding parts of research methods.
• Avoid boring data. Who gives a flying fish about the relative sizes of dolphin jaws?
• Wikipedia is a great place to find out the context for most high school statistics analysis. That is where I look. It’s a great starting place for anyone.

Less is more

“Less is More” is a bit of a funny title for a mathematical blog!

Garlic bread and Ice Cream Sundaes

Back in the seventies, garlic bread became very popular in our household. I loved its buttery, salty, garlicky goodness, and made it quite often. One time I decided that if a little bit of garlic was yummy, then lots of garlic would be even more delicious. I was wrong! The garlic bread was barely edible, and the house and its occupants gave off a distinctive aroma for several days. More garlic did not mean “better”. From then on whenever I used garlic, I would recite in my head “More is not always better.”

Similarly it is fun to see children given a whole range of ice cream flavours, sauces and toppings and watch them create a dessert with EVERYTHING. From experience we know that there are only so many different forms of sugar and fat that should be added to ice cream at one time. If we are smart, we have several bowls, one with chocolate and nuts, one with caramel and crunchy toffee, one with raspberry and biscuit crumbs. That way we can appreciate the different flavours, without having them overridden. Having said that, we then discover that there comes a point of diminishing or even negative returns on investment. The final bowl of ice cream is often regretted.

Enough of food!

“Less is more” applies to teaching

The statement “Less is more” applies to teaching, and particularly subjects like Statistics and Operations Research.

As I learned with the garlic bread, we need to be careful not to give students too much. It is tempting, when developing on-line resources to keep including every possible activity, video and link that is relevant. However we have found that too many activities become overwhelming. It is tempting, as instructors to want to give plenty of practice and every possible resource. We assume that students can pick which items are useful for them. Instead we found that conscientious students want to complete EVERYTHING, and get discouraged when there is so much to do. They possibly don’t need to do all the activities, and waste their time on the easy ones.

We need to be selective about how we use our students’ time. Unless the homework or activity is going to help them learn and accomplish the goals of the course, it should not be there. I am reminded of the hell that was homework for my older son and me when he was going through middle-school. The teacher believed that more homework was better, and the result was misery in our family. Eventually I cried, “Enough!” and arranged an interview with her. I asked her for the specific learning objectives of the “worksheet”, which I know was an unfair question. Clearly the objective of worksheets is to keep the parents of conscientious girls (and the very uncommon conscientious boys) happy because their children were getting homework to do. She never did come up with learning objectives that satisfied me, so William (or rather, I) ceased to do her homework sheets, concentrating instead on times-tables, reading and handwriting. (Or generally nothing at all!)

But I digress. The point is – don’t waste student time on “busy” work. If students understand the process and internalise a skill after ten examples, then they do not need another ten. I DO believe in drill or practice, but it needs to be well developed and practising the skills we wish students to develop. For example there is no need for students to calculate by hand the standard deviation of ten sets of numbers devoid of context. However there is great value in large numbers of questions getting students to determine which test is appropriate in a given scenario.

If you really want to make more resources, rather than making more tests, provide a larger question bank for the current quizzes. That way students can do the quiz multiple times to achieve mastery, but those who have mastered the material immediately can move on.

We should not teach all we know

And as with the ice cream sundaes, when choosing content, what we leave out is as important than what we put in. We should not attempt to teach all we know. When writing the scripts for my videos I find it is important to stick to the main ideas and get them well explained. Sometimes total accuracy is sacrificed in the interests of comprehensibility. I come back to the dreaded question, “Where do babies come from?”, the answer to which depends enormously on the source of the question and context. Seldom is a full biological explanation required or even desirable.

Leonardo Da Vinci is purported to have said, “Simplicity is the ultimate sophistication.” It is the art of the true teacher to be able to reduce complex ideas into a simple form. Bill Bryson is the master of this. In his book, “A Short History of Nearly Everything”, Bryson puts forth complex ideas in ways that a layperson can understand. This is a skill I seek after as a teacher, and try to use in my videos and resources.

Choosing the statistical test – in simple terms

I was unhappy with the branching diagrams commonly used to teach how to choose a statistical test. I felt that there was a more integrative way to express this that would also help peoples understanding. I came up with quite a different diagram that is featured in our most popular video to date.

The students love it. But there are aspects about the diagram which could be looked at a different way. For example I ask “How many samples?”, and say that an independent samples t-test is used on two separate samples. Really it could also be defined as one sample with two variables – the measurement variable and another variable for group membership. When people are just coming to grips with new ideas, they don’t need to see multiple ways of doing things. If they are at the stage to see the other way of looking at it, they aren’t going to need the diagram.

Another very cool thing Da Vinci said was “Art is never finished, only abandoned.” On that note, I will stop now.

Confidence Intervals: informal, traditional, bootstrap

Confidence Intervals

Confidence intervals are needed because there is variation in the world. Nearly all natural, human or technological processes result in outputs which vary to a greater or lesser extent. Examples of this are people’s heights, students’ scores in a well written test and weights of loaves of bread. Sometimes our inability or lack of desire to measure something down to the last microgram will leave us thinking that there is no variation, but it is there. For example we would check the weights of chocolate bars to the nearest gram, and may well find that there is no variation. However if we were to weigh them to the nearest milligram, there would be variation. Drug doses have a much smaller range of variation, but it is there all the same.

You can see a video about some of the main sources of variation – natural, explainable, sampling and due to bias.

When we wish to find out about a phenomenon, the ideal would be to measure all instances. For example we can find out the heights of all students in one class at a given time. However it is impossible to find out the heights of all people in the world at a given time. It is even impossible to know how many people there are in the world at a given time. Whenever it is impossible or too expensive or too destructive or dangerous to measure all instances in a population, we need to take a sample. Ideally we will take a sample that gives each object in the population an equal likelihood of being chosen.

You can see a video here about ways of taking a sample.

When we take a sample there will always be error. It is called sampling error. We may, by chance, get exactly the same value for our sample statistic as the “true” value that exists in the population. However, even if we do, we won’t know that we have.

The sample mean is the best estimate for the population mean, but we need to say how well it is estimating the population mean. For example, say we wish to know the mean (or average) weight of apples in an orchard. We take a sample and find that the mean weight of the apples in the sample  is 153g. If we only took a few apples, it is only a rough idea and we might say we are pretty sure the mean weight of the apples in the orchard is between 143g and 163g. If someone else took a bigger sample, they might be able to say that they are pretty sure that the mean weight of apples in the orchard is between 158g and 166g. You can tell that the second confidence interval is giving us better information as the range of the confidence interval is smaller.

There are two things that affect the width of a confidence interval. The first is the sample size. If we take a really large sample we are getting a lot more information about the population, so our confidence interval will be more exact, or smaller. It is not a one-to-one relationship, but a square-root relationship.  If we wish to reduce the confidence interval by a factor of two, we will need to increase our sample size by a factor of 4.

The second thing to affect the width of a confidence interval is the amount of variation in the population. If all the apples in the orchard are about the same weight, then we will be able to estimate that weight quite accurately. However, if the apples are all different sizes, then it will be harder to be sure that the sample represents the population, and we will have a larger confidence interval as a result.

Three ways to find confidence intervals

Traditional (old-fashioned?) Approach

The standard way of calculating confidence intervals is by using formulas developed on the assumptions of normality and the Central Limit Theorem. These formulas are used to calculate the confidence intervals of means, proportions and slopes, but not for medians or standard deviations. That is because there aren’t nice straight-forward formulas for these. The formulas were developed when there were no computers, and analytical methods were needed in the absence of computational power.

In terms of teaching, these formulas are straight-forward, and also include the concept of level of confidence, which is part of the paradigm. You can see a video teaching the traditional approach to confidence intervals, using Excel to calculate the confidence interval for a mean.

Rule of Thumb

In the New Zealand curriculum at year 12, students are introduced to the concept of inference using an informal method for calculating a confidence interval. The formula is median +/-  1.5 times the interquartile range divided by the square-root of the sample size. There is a similar formula for proportions.

Bootstrapping

Bootstrapping is a very versatile way to find a confidence interval. It has three strengths:

1. It can be used to calculate the confidence interval for a large range of different parameters.
2. It uses ALL the information the sample gives us, rather than the summary values
3. It has been found to aid in understanding the concepts of inference better than the traditional methods.

There are also some disadvantages

1. Old fogeys don’t like it. (Just kidding) What I mean is that teachers who have always taught using the traditional approach find it difficult to trust what seems like a hit-and-miss method without the familiar theoretical underpinning.
2. Universities don’t teach bootstrapping as much as the traditional methods.
3. The common software packages do not include bootstrap confidence intervals.

The idea behind a bootstrap confidence interval is that we make use of the whole sample to represent the population. We take lots and lots of samples of the same size from the original sample. Obviously we need to sample with replacement, or the samples would all be identical. Then we use these repeated samples to get an idea of the distribution of the estimates of the population parameter. We chop the tails off at a given point, and we give the confidence interval.  Voila!

Answers to the disadvantages (burn the straw man?)

1. There is a sound theoretical underpinning for bootstrap confidence intervals. A good place to start is a previous blog about George Cobb’s work. Either that or – “Trust me, I’m a Doctor!” (This would also include trusting far more knowledgeable people such as Chris Wild and Maxine Pfannkuch, and the team of statistical educators led by Joan Garfield.
2. We have to start somewhere. Bootstrap methods aren’t used at universities because of inertia. As an academic of twenty years I can say that there is NO PAY OFF for teaching new stuff. It takes up valuable research time and you don’t get promoted, and sometimes you even get made redundant. If students understand what confidence intervals are, and the concept of inference, then learning to use the traditional formulas is trivial. Eventually the universities will shift. I am aware that the University of Auckland now teaches the bootstrap approach.
3. There are ways to deal with the software package problem. There is a free software interface called “iNZight” that you can download. I believe Fathom also uses bootstrapping. There may be other software. Please let me know of any and I will add them to this post.

In Summary

Confidence intervals involve the concepts of variation, sampling and inference. They are a great way to teach these really important concepts, and to help students be critical of single value estimates. They can be taught informally, traditionally or using bootstrapping methods. Any of the approaches can lead to rote use of formula or algorithm and it is up to teachers to aim for understanding. I’m working on a set of videos around this topic. Watch this space.