Which comes first – problem or solution?

In teaching it can be difficult to know whether to start with a problem or a solution method. It seems more obvious to start with the problem, but sometimes it is better to introduce the possibility of the solution before posing the problem.

Mathematics teaching

A common teaching method in mathematics is to teach the theory, followed by applications. Or not followed by applications. I seem to remember learning a lot of mathematics with absolutely no application – which was fine by me, because it was fun. My husband once came home from survey school, and excitedly told me that he was using complex numbers for some sort of transformation between two irregular surfaces. Who’d have thought? I had never dreamed there could be a real-life use for the square root of -1. I just thought it was a cool idea someone thought up for the heck of it.

But yet again we come to the point that statistics and operations research are not mathematics. Without context and real-life application they cease to exist and turn into … mathematics!

Applicable mathematics

My colleague wrote a guest post about “applicable mathematics” which he separates from “applied mathematics”. Applicable maths appears when teachers make up applications to try to make mathematics seem useful. There is little to recommend about applicable maths. A form of “applicable maths” occurs in probability assessment questions where the examiner decides not to tell the examinee all the information, and the examinee has to draw Venn diagrams and use logical thinking to find out something that clearly anyone in the real world would be able to read in the data! I actually enjoy answering questions like that, and they have a point in helping students understand the underlying structure of the data. But I do not fool myself into thinking that they are anywhere near real-life. Nor are they statistics.

Which first – theory or application?

So the question is – when teaching statistics and operations research, should you start with an application or a problem or a case, and work from there to the theory? Or do students need some theory, or at least an understanding of basic principles before a case or problem can have any meaning? Or in a sequence of learning do we move back and forward between theory and application?

My first off response is that of course we should start with the data, as many books on the teaching of statistics teach us. Well actually we should start with the problem, as that really precedes the collection of the data. But then, how can we know what sorts of problems to frame if we don’t have some idea of what is possible through modelling and statistics? So should we first begin with some theory? The New Zealand Curriculum emphasises the PPDAC cycle, Problem, Plan, Data, Analysis, Conclusion. However, in order to pose the problem in the first place, we need the theory of the PPDAC cycle itself. The answer is not simple and depends on the context.

I have recently made a set of three videos explaining confidence intervals and bootstrapping. These are two very difficult topics that become simple in an instant. What I mean by that is, until you understand a confidence interval, it makes no sense, and you can see no reason why it should make sense. You go through a “liminal space” of confusion and anxiety. Then when you emerge out the other side, instantly confidence intervals make sense, and it is equally difficult to see what it was that made them confusing. This dichotomy makes teaching difficult, as the teacher needs to try to understand what made the problem confusing.

I present the idea of a confidence interval first. Then I use examples. I present the idea of bootstrapping, then give examples. I think in this instance it is helpful to delineate the theory or the idea in reasonably abstract form, interspersed with examples. I also think diagrams are immensely useful, but that’s another topic.

Critique of AtMyPace: Statistics

What prompted these thoughts about “which comes first” was a comment made about our “AtMyPace: Statistics” iOS app.

The YouTube videos used in AtMyPace:Statistics were developed to answer specific needs in a course. They generally take the format of a quick summary of the theory, followed by an example, often related to Helen and her business selling choconutties.

The iOS app, AtMyPace:Statistics was set up as a way to capitalise on the success of the YouTube videos, and we added two quizzes of ten True/false questions to complement each of the videos. We also put these same quizzes in our on-line course and found that they were surprisingly popular. In a way, they are a substitute for a textbook or notes, but require the person to commit one way or the other to an answer before reading a further explanation. We had happened on a effective way of engaging students with the material.

AtMyPace:Statistics is not designed to be a full course in statistics, but rather a tool to help students who might be struggling with concepts. We have also developed a web-based version of AtMyPace:Statistics for those who are not the happy owners of iOS devices. At present the web version is a copy of the app, but we will happily add other questions and activities when the demand arises.

I received the following critique of the AtMyPace: Statistics app:

“They are nicely done but very classical in scope. The approach is tools-oriented using a few “realistic” examples to demonstrate the tool. This could work for students who need to take exams and want accessible material.”

Very true. The material in AtMyPace:Statistics is classical in scope, as we focus on the material currently being taught in most business schools and first year statistics service courses. We are trying to make a living, and once that is happening we will set out to change the world!

The reviewer continues,

“ I think that in adult education you should reverse the order and have the training problem oriented. Take a six sigma DMAIC process as an example. The backbone is a problem scheduled to be solved. The path is DMAIC and the tools are supporting the journey. If you want to do it that way you need to tailor the problem to the audience. “

In tailored adult education it is likely that a problem-based approach will work. I would strongly recommend it.

I had an interesting discussion some time ago with a young lecturer working in a prestigious case-based MBA programme in North America. The entire MBA is taught using cases, and is popular and successful. My friend had some reservations about case-based teaching for a subject like Operations Research which has a body of skills which are needed as a foundation for analysis. Statistics would be similar. The question is making sure the students have the necessary skills and knowledge, with the ability to transfer to another setting or problem. Case-based learning is not an efficient way to accomplish this.

Criticism on Choosing the Test procedure

In another instance, David Munroe commented on our video “Choosing which statistical test to use”, which receives about 1000 views a week.  In the video I suggest a three step process involving thinking about what kind of data we have, what kind of sample, and the purpose of the analysis. The comment was:

Myself I would put purpose first. The purpose of the analysis determines what data should be collected – and more data is not necessarily more informative. In my view it is more useful to think ‘what am I trying to achieve’ with this analysis before collecting the data (so the right data have a chance to be collected). This in contrast to: collecting the data and then going ‘now what can I get from this data?’ (although this is sometimes an appropriate research technique). I think because we’ve already collected the data any time we’re illustrating particular modelling tools or statistical tests, we reinforce the ‘collect the data first then worry about analysis’ approach – at least subconsciously.

Thanks David! Good thinking, and if I ever redo the video I may well change the order. I chose the order I did, as it seemed to go from easy to difficult. (Actually I don’t remember consciously thinking about the order – it just fell out of individual help sessions with students.)  And the diagram was developed in response to the rather artificial problems I was posing!

I’ll step back a bit and explain. One problem I have seen in teaching Statistics and Operations Research is that students fail to make connections. They also compartmentalise the different aspects and find it difficult to work out when certain procedures would be most useful. I wrote a post about this. In the statistics course I wrote a set of scenarios describing possible applications of statistical methods in a business context. The students were required to work out which technique to use in each scenario and found this remarkably difficult. They could perform a test on difference of two means quite well, but were hard-pressed to discern when the test should be used. So I made up even more questions to give them more practice, and designed my three step method for deciding on the test.  This helped.

I had not thought of it as a way to decide in a real-life situation which test to use. Surely that would be part of a much bigger process.  So my questions are rather artificial, but that doesn’t make them bad questions. Their point was to help students make linkages between different parts of the course. And for that, it works.

Bring on the criticism

I would like to finish by saying how much I appreciate criticism. It is nice when people tell me they like my materials. I feel as if I am doing something useful and helping people. I get frequent comments of this type on my YouTube site.  But when people make the effort to point out gaps and flaws in the material I am extremely grateful as it helps me to clarify my thinking and improve the approach. If nothing else, it gives me something to talk about in my blog. It is difficult producing material in a feedback vacuum.  So keep it coming!

Context – if it isn’t fun…

The role of context in statistical analysis

The wonderful advantage of teaching statistics is the real-life context within which any applicaton must exist. This can also be one of the difficulties. Statistics without context is merely the mathematics of statistics, and is sterile and theoretical.  The teaching of statistics requires real data. And real data often comes with a fairly solid back-story.

One of the interesting aspects for practicing statisticians, is that they can find out about a wide range of applications, by working in partnership with specialists. In my statistical and operations research advising I have learned about a range of subjects, including the treatment of hand injuries, children’s developmental understanding of probability, the bed occupancy in public hospitals, the educational needs of blind students, growth rates of vegetables, texted comments on service at supermarkets, killing methods of chickens, rogaine route choice, co-ordinating scientific expeditions to Antarctica and the cost of care for neonatals in intensive care. I found most of these really interesting and was keen to work with the experts on these projects. Statisticians tend to work in teams with specialists in related disciplines.

Learning a context can take time

When one is part of a long-term project, time spent learning the intricacies of the context is well spent. Without that, the meaning from the data can be lost. However, it is difficult to replicate this in the teaching of statistics, particularly in a general high school or service course. The amount of time required to become familiar with the context takes away from the time spent learning statistics. Too much time spent on one specific project or area of interest can mean that the students are unable to generalise. You need several different examples in order to know what is specific to the context and what is general to all or most contexts.

One approach is to try to have contexts with which students are already familiar. This can be enabled by collecting the data from the students themselves. The Census at School project provides international data for students to use in just this way. This is ideal, in that the context is familiar, and yet the data is “dirty” enough to provide challenges and judgment calls.

Some teachers find that this is too low-level and would prefer to use biological data, or dietary or sports data from other sources. I have some reservations about this. In New Zealand the new statistics curriculum is in its final year of introduction, and understandably there are some bedding-in issues. One I perceive is the relative importance of the context in the students’ reports. As these reports have high-stakes grades attached to them, this is an issue. I will use as an example the time series “standard”. The assessment specification states, among other things, “Using the statistical enquiry cycle to investigate time series data involves: using existing data sets, selecting a variable to investigate, selecting and using appropriate display(s), identifying features in the data and relating this to the context, finding an appropriate model, using the model to make a forecast, communicating findings in a conclusion.”

The full “standard” is given here: Investigate Time Series Data This would involve about five weeks of teaching and assessment, in parallel with four other subjects.(The final 3 years of schooling in NZ are assessed through the National Certificate of Educational Achievement (NCEA). Each year students usually take five subject areas, each of which consists of about six “achievement standards” worth between 3 and 6 credits. There is a mixture of internally and externally assessed standards.)

In this specification I see that there is a requirement for the model to be related to the context. This is a great opportunity for teachers to show how models are useful, and their limitations. I would be happy with a few sentences indicating that the student could identify a seasonal pattern and make some suggestions as to why this might relate to the context, followed by a similar analysis of the shape of the trend. However there are some teachers who are requiring students to do independent literature exploration into the area, and requiring references, while forbidding the referencing of Wikipedia.

This concerns me, and I call for robust discussion.

Statistics is not research methods any more than statistics is mathematics. Research methods and standards of evidence vary between disciplines. Clearly the evidence required in medical research will differ from that of marketing research. I do not think it is the place of the statistics teacher to be covering this. Mathematics teachers are already being stretched to teach the unfamiliar material of statistics, and I think asking them and the students to become expert in research methods is going too far.

It is also taking out all the fun.

Keep the fun

Statistics should be fun for the teacher and the students. The context needs to be accessible or you are just putting in another opportunity for antipathy and confusion. If you aren’t having fun, you aren’t doing it right. Or, more to the point, if your students aren’t having fun, you aren’t doing it right.

Some suggestions about the role of context in teaching statistics and operations research

• Use real data.
• If the context is difficult to understand, you are losing the point.
• The results should not be obvious. It is not interesting that year 12 boys weigh more than year 9 boys.
• Null results are still results. (We aren’t trying for academic publications!)
• It is okay to clean up data so you don’t confuse students before they are ready for it.
• Sometimes you should use dirty data – a bit of confusion is beneficial.
• Various contexts are better than one long project.
• Avoid the plodding parts of research methods.
• Avoid boring data. Who gives a flying fish about the relative sizes of dolphin jaws?
• Wikipedia is a great place to find out the context for most high school statistics analysis. That is where I look. It’s a great starting place for anyone.

Less is more

“Less is More” is a bit of a funny title for a mathematical blog!

Garlic bread and Ice Cream Sundaes

Back in the seventies, garlic bread became very popular in our household. I loved its buttery, salty, garlicky goodness, and made it quite often. One time I decided that if a little bit of garlic was yummy, then lots of garlic would be even more delicious. I was wrong! The garlic bread was barely edible, and the house and its occupants gave off a distinctive aroma for several days. More garlic did not mean “better”. From then on whenever I used garlic, I would recite in my head “More is not always better.”

Similarly it is fun to see children given a whole range of ice cream flavours, sauces and toppings and watch them create a dessert with EVERYTHING. From experience we know that there are only so many different forms of sugar and fat that should be added to ice cream at one time. If we are smart, we have several bowls, one with chocolate and nuts, one with caramel and crunchy toffee, one with raspberry and biscuit crumbs. That way we can appreciate the different flavours, without having them overridden. Having said that, we then discover that there comes a point of diminishing or even negative returns on investment. The final bowl of ice cream is often regretted.

Enough of food!

“Less is more” applies to teaching

The statement “Less is more” applies to teaching, and particularly subjects like Statistics and Operations Research.

As I learned with the garlic bread, we need to be careful not to give students too much. It is tempting, when developing on-line resources to keep including every possible activity, video and link that is relevant. However we have found that too many activities become overwhelming. It is tempting, as instructors to want to give plenty of practice and every possible resource. We assume that students can pick which items are useful for them. Instead we found that conscientious students want to complete EVERYTHING, and get discouraged when there is so much to do. They possibly don’t need to do all the activities, and waste their time on the easy ones.

We need to be selective about how we use our students’ time. Unless the homework or activity is going to help them learn and accomplish the goals of the course, it should not be there. I am reminded of the hell that was homework for my older son and me when he was going through middle-school. The teacher believed that more homework was better, and the result was misery in our family. Eventually I cried, “Enough!” and arranged an interview with her. I asked her for the specific learning objectives of the “worksheet”, which I know was an unfair question. Clearly the objective of worksheets is to keep the parents of conscientious girls (and the very uncommon conscientious boys) happy because their children were getting homework to do. She never did come up with learning objectives that satisfied me, so William (or rather, I) ceased to do her homework sheets, concentrating instead on times-tables, reading and handwriting. (Or generally nothing at all!)

But I digress. The point is – don’t waste student time on “busy” work. If students understand the process and internalise a skill after ten examples, then they do not need another ten. I DO believe in drill or practice, but it needs to be well developed and practising the skills we wish students to develop. For example there is no need for students to calculate by hand the standard deviation of ten sets of numbers devoid of context. However there is great value in large numbers of questions getting students to determine which test is appropriate in a given scenario.

If you really want to make more resources, rather than making more tests, provide a larger question bank for the current quizzes. That way students can do the quiz multiple times to achieve mastery, but those who have mastered the material immediately can move on.

We should not teach all we know

And as with the ice cream sundaes, when choosing content, what we leave out is as important than what we put in. We should not attempt to teach all we know. When writing the scripts for my videos I find it is important to stick to the main ideas and get them well explained. Sometimes total accuracy is sacrificed in the interests of comprehensibility. I come back to the dreaded question, “Where do babies come from?”, the answer to which depends enormously on the source of the question and context. Seldom is a full biological explanation required or even desirable.

Leonardo Da Vinci is purported to have said, “Simplicity is the ultimate sophistication.” It is the art of the true teacher to be able to reduce complex ideas into a simple form. Bill Bryson is the master of this. In his book, “A Short History of Nearly Everything”, Bryson puts forth complex ideas in ways that a layperson can understand. This is a skill I seek after as a teacher, and try to use in my videos and resources.

Choosing the statistical test – in simple terms

I was unhappy with the branching diagrams commonly used to teach how to choose a statistical test. I felt that there was a more integrative way to express this that would also help peoples understanding. I came up with quite a different diagram that is featured in our most popular video to date.

The students love it. But there are aspects about the diagram which could be looked at a different way. For example I ask “How many samples?”, and say that an independent samples t-test is used on two separate samples. Really it could also be defined as one sample with two variables – the measurement variable and another variable for group membership. When people are just coming to grips with new ideas, they don’t need to see multiple ways of doing things. If they are at the stage to see the other way of looking at it, they aren’t going to need the diagram.

Another very cool thing Da Vinci said was “Art is never finished, only abandoned.” On that note, I will stop now.

Shibboleth, Mixolydian, Heteroscedasticity – and Kipling

All areas of human endeavour have specific language. Cricket commentators, art critics and wines buff make this very obvious.

Mixolydian

My son, who is blind, autistic and plays the piano like an angel, is studying Jazz, and I’m helping him. You can read more about this in my other blog Never Ordinary. There is a specific language around Jazz, and I’m not talking about ‘scat’. (Hmm just realised the other meaning for that word!) In the Jazz course they use words like Mixolydian, Chromatisism, Quartal Harmony…  I nod and smile. This language expresses ideas clearly and uniquely and is outside my comprehension. (Mixolydian is based on the Major scale, but with a flat 7. – clearer now?)

Trumpetty yellow, Daffodils, Narcissus

This week there was a statistics list discussion about the meaning of the term “multivariate”. As part of the ongoing discussion, someone suggested that using exact terminology exactly avoids a situation such as saying “I have yellow flowers in my garden with trumpetty bits, that come out in spring and have oniony looking bits in the ground.” This can also be said as “I have daffodils in my garden”.  However it can also be said as “I have Narcissus pseudonarcissus  in my garden”. Each of those phrases expresses the same idea, but with differing clarity or exclusiveness depending on the audience.

Shibboleth

Language can be used to exclude, as well as to inform or communicate. The term “shibboleth” comes from the book of Judges. When the Gileadites wished to find out if people crossing the river were Ephraimites, they would ask them to say the word “shibboleth”. If they said it as sibboleth, they killed them. The Old Testament can be a bit like that. The word “shibboleth” is now used to mean a code word, or knowledge that only a certain culture or group will know. Sometimes it can seem that statistical terms are used so only the initiated will be able to understand.

Virtue and Common Touch

As statisticians, operations researchers and teachers of statisticians and operations researchers we have many different opportunities to select the language we use. We must always be aware of our audience. In the poem, “If”, Kipling encourages people to be able to “…talk with crowds and keep your virtue, Or walk with Kings – nor lose the common touch,” Academics “walk with kings” when they write academic papers, using highly specialised and exclusive language. We need to make sure we do not lose the common touch. At the same time we should “keep our virtue”, and use the correct statistical term when the circumstances arise, making sure that we retain the common touch so that all understand.

Heteroscedasticity

When I use the term heteroscedasticity I am usually doing so for one of two reasons. First, that the data in question has non-constant variance, and I am explaining the concept and technical term to a client, student or colleague. Second, because I really like the word. “Heteroscedasticity” is eight syllables of tongue-twisting goodness! But, really, “non-constant variance” says exactly the same thing, has only six syllables and is easier to understand. I suspect a degree of linguistic snobbery appearing.

Communicating Statistics

Greenfield wrote a paper in 1993, which is still disappointingly relevant today. In “Communicating Statistics” (http://greenfieldresearch.co.uk/papers/Communicating%20stats.pdf) he suggests that statisticians have a great deal to offer the world, and that we aren’t doing a good job of making people aware of that. He was damning of the type of language used in academic publications, which ensure that any potentially useful results are obscured by “prolix and pseudo-objective style”.

This flows over into our consulting endeavours, where the aim should be to communicate rather than exclude. Greenfield gives the example fictionalised in this comic:

Depiction of almost true event. Click to view.

Greenfield’s parting provocative statement was to suggest that statisticians produce more cookery-books and more easy-to-use programs, and encourage their use by everybody who can benefit. These books and programs can carry the message that if they want to do better they should study more and seek the guidance of statisticians.

In closing he says “Our audience, our customers are out there. They need us, even if they do not realise it. We must change our culture, our philosophy, our public relations and our use of language to reach them.”

Greenfield Challenge

I’m not sure I want to be telling you about the Greenfield challenge, as I’m thinking of entering it, and would really like a trip to Ankara for the ENBIS conference. But in pursuit of the greater good, I am putting a link here: The Greenfield Challenge. The blurb explains:

“We would like to encourage you to report immediately whenever you’ve had dealings with non-statisticians – in whichever form (face-to-face, in writing, in form of an audio or video recording, in interactive social media … ) or context (interactions with students, educators, managers and employees of organizations in private and public sectors … ).”

Greenfield even suggests “You might even write a short story or a play.”

Still thinking about that one. I guess there is always “The Goal” to look to for an example. In the meantime I’ll stick to this work of mostly non-fiction, interspersed with opinion and anecdote.

Choose our words

When we use very specific technical terms we need to make sure that they are really necessary. Is there a simpler, and just as accurate way of saying the same thing? If our audience is statisticians, then really we can indulge in specific technical language. But if the audience includes students, non-statisticians and the general public, then we should probably use simpler terms, or at least “gloss”, or say what the word means along with its use. (There was an example of glossing right there!)

I have written earlier about the minefield of statistical terminology, particularly when the statistical word also has an everyday meaning which is not quite the same. Examples of this are “significant”, “random” and “relationship”. The post includes some suggestions for teaching statistical language.

But as well as teachers, we are also communicators, and need to get our message across in the best way possible. It is vital to determine our audience, and make sure we bring them along with us.

I contemplate the new New Zealand curriculum with excitement. Through the efforts of a group of statisticians we are able to inculcate a greater understanding of the essentials of statistics from an early age to much of the population. The role of the statisticians is to help the teachers feel at home in the world of statistics, so that they can invite their students along. These are exciting times. The rest of the world is watching.

Assessment – a necessary evil

My northern hemisphere twitter buddies are well into the academic year, and facing the demands of grading, while here in New Zealand we are enjoying the sunshine and trying hard not to think about going back to work. However the teachers of High School statistics in New Zealand are facing (or trying not to) an interesting challenge in the coming year. They are going to have to mark (our word for grade) essays. Eek.

One of the main reasons I majored in operations research, and became a mathematics teacher was that I was required neither to write nor grade essays. This must sound funny coming from someone who can’t stop blogging! I remember exam time as a new high school teacher, happily putting little red ticks against numerically correct answers, and occasionally pausing to decide if the working were adequate. Next to me was the also new English teacher having to give grades for essays, agonising over what the work was telling her and what grade to assign. It was hard not to feel smug.  In the end I felt sorry for her and helped with some of her marking.

Then at university I taught operations research and statistics, both of which I thought could be reasonably assessed with mainly numeric questions. But as time went on and I gained a greater understanding of my discipline and of teaching and learning I realised there was no escape. I dabbled with orals, essays, assignments and on-line assessment. I put on a brave face, and tried to focus on what I was learning from the mistakes they were making. To be sure you do learn a lot from marking student work, but the effect wears off on the 50^th script. Or sooner.

There is no escaping it…

Marking/Grading is difficult, often unpleasant and extremely important.

Feedback is a vital part of learning. Research into education and learning has shown that specific, timely feedback is possibly THE most useful thing to help people to learn. Well duh!  As an aside, this is one of the reasons I have reservations about giving too much homework in mathematics. If the students don’t check their work as they go, in the absence of correct feedback they can often entrench wrong procedures and thinking.

As we learn we need to make sure that we are learning correctly, in a physically and emotionally safe environment. This is why flight simulators were invented, so pilots could practice crashing – or rather not crashing, while remaining alive.

This is one of the reasons I fell in love with my Learning Management System. Originally it was hosted by WebCT, then Blackboard and (I hope finally) Moodle. (Clearly a non-specific love-affair!) A good LMS can give non-judgmental, correct, timely, specific feedback FOREVER! It never gets tired. We had a student who struggled with English, who sat one of our on-line tests over 70 times. He got there in the end, with the help of several of my more patient tutors. But there is no way we could have given him the time he needed, in the way the LMS did.

Of course there is only a certain range of assessment possible for automatic grading, but I have been experimenting with different ideas, and have managed to come up with ways to provide worthwhile automated feedback to students in most circumstances. Another great thing about the LMS is that you can collect the results and quickly see what the students are getting wrong, and which distractors are most distracting!

The first reason we need to grade is to give feedback, to help students learn. This is known as formative assessment. In a school setting we can usually make this low stakes, and the students will still participate, but at university level, time pressure means that unless the assessment is “worth something”, the students who need it most are least likely to do it. We found a sizable correlation between participation in the small tests and grades in the course as a whole. We don’t claim causation, but that doesn’t mean it isn’t there!

The other main reason for assessment is to evaluate the learning at the end of the course. The formal term for this is summative assessment. This is what tells the student and future employer how well they did in the student did in the assessment at the time. It may or may not tell anyone how much the student knows, especially some time later.

Miscellaneous thoughts on assessment and grading:

• Align assessment with learning objectives. Don’t ask what you haven’t specified and taught. (Except for scholarship exams when you can do what you like!)
• Students will only learn what is assessed – if you want them to learn something, put it in the objectives, tell the students and then assess it.
• Be clear in your mind what the assessment is for. Normative or summative? Mastery or brilliance? Encouragement through success or scaring them to do some work? Signalling important points to students in later years? Propaganda?
• Spend the time devising a good test, and you will save time and pain in the marking.  Write-on answer booklets save time.  On-line saves even more!
• Don’t ask more questions of the same type than you need to. You aren’t getting more information.
• Be careful with the word “how”. It is almost always ambiguous.
• Make sure that ignorance of the non-subject-specific context of the question will not affect the ability to answer. An example – There can be questions involving reading tables that assume that the person knows that Shirley is a girl and will therefore use the female sizing chart. For non-native speakers (and even people from other English-speaking countries) this is not a reasonable expectation.
• Don’t agonise – if a student is borderline you are probably being too generous. It isn’t personal.
• Do not assign half marks.
• Be creative. Try orals.

This is not the last you will hear about assessment. I am currently developing a suite of videos, quizzes, writing guides and an app for teaching and learning basic time-series analysis. Assessing learning for this topic is an interesting problem. Watch this space.

Mathematical recreation

Here in New Zealand it is still the summer holidays, and it is difficult to feel too excited about topics of great moment, even if it is the International Year of Statistics and the start of Mathematics of Planet Earth 2013!

While the sun shines in a clear blue Christchurch sky, in the interests of mathematical recreation I will tell you about Rogo, a new number puzzle that we hope will soon become as popular as Sudoku.

A  simple Rogo puzzle

We came up with the idea for Rogo a few years ago. I have always loved board games, and was trying to invent one based around the sport of Rogaining. (I still have hopes to bring that one to fruition one day.) Instead we came up with a puzzle that can be done with pen and paper, or on an iPhone/iPad/iPod touch. It is based on the traveling salesperson problem, with prize collection and subset selection, all on a rectilinear grid.

It’s surprisingly fun and engrossing (and to use a somewhat overused term, addictive!).

Children as young as six or seven like to play on the Rogo app, as do adults all over the world. Our greatest fan is Martin, in Austria, and having solved the 384 puzzles on the iPad, he is keen for us to make another set. I think we need to sell some more of our current version first!

You can see a YouTube video on how to play on the iPhone here:

Or an early Youtube video on how to solve Rogo on paper here:

You can buy the app here (sorry only iOS at present): Link to the AppStore. (Please do!)

You can get daily paper puzzles here: Daily puzzles

We have a website dedicated to Rogo, where there is a useful page about how Rogo can be used in teaching. This is aimed at school level problem-solving, science fair, extension and numeracy development. For a University course involving heuristics, Rogo is a great medium through which to illustrate and teach different search algorithms.

We have done some research into what makes a Rogo difficult. We came up with twelve potential  factors in the determination of difficulty.  You can read about that in our paper:Determining Degree Of Difficulty In Rogo, A TSP-based Paper Puzzle

Solving Rogo

The computational solution of Rogo is mathematically challenging. In the early days of developing our algorithm my laptop would overheat and shut down if I tried to solve too many bigger Rogos.

Hakan Kjellerstrand wrote about solving Rogo in a blog about constraint programming.

Recently Chris Kuip blogged about Rogo in AIMMS Rogo Solver using constraint programming

You can read about our algorithm in the Journal of Information Processing.

Have fun!

Careers advice in Mathematical Sciences

Mathematics teachers as Careers advisors

What can you do if you are good at mathematics? Become a maths teacher, of course!

I wonder how many of our students are aware of what wonderful and exciting career opportunities are out there for the mathematically competent, including being a mathematics teacher.

I also wonder how many teachers of mathematics, statistics and operations research are telling their students of the different possibilities.

I always loved maths at school and was good at it. I liked teaching, so I decided to be a maths teacher. Along the way, at university, I discovered computer programming and operations research, both subjects that I enjoyed and excelled at. (My conversion to statistics came much later). Given more information at school level, I may well have taken a different path earlier. I didn’t know about engineering or meteorology or surveying, all subjects which need proficiency in mathematics. This may be because I attended a girls’ catholic school where the more able students took languages and I had to study chemistry by correspondence. Let us hope that students this century are better informed.

Teachers are really busy people. I was fortunate last week to attend three different events for mathematics teachers, and was impressed at the dedication they have and their desire to do the best for their pupils.

At university we are having trouble attracting students to the mathematical sciences, yet there is a clear market for graduates, as this article explains: You’re a data what?

There are sites on the internet dedicated to careers information related to mathematics.

The Mathematical Association of America has this link: maa.org/careers/

And the UK has a similar one: mathscareers.org.uk/

A New Zealand site gives suggestions to teachers: CareersNZ

I suspect all it may take is awareness, that the teaching of mathematics and statistics needs to be supplemented with a little careers information. Here are some ideas:

• Have students research mathematically based careers for homework.
• Put vignettes of mathematical workers on the screen for students to read as they enter the classroom.
• Apply the material covered to possible career paths – for example, “if you enjoy interpreting statistical graphs, you might like to work as a data scientist”.
• Use data from a range of mathematical careers as part of data analysis.

I certainly don’t have all the answers, and would love to hear what classroom teachers are doing. There is a wonderful array of information available at the end of a Google Search (which is only possible because of mathematical scientists who continually refine the search engine).

University lecturers need to talk about careers too

The responsibility to inform about careers is not just for school teachers. At University level we sometimes assume the students have a plan about what they are studying and where they would like to end up. I remember one student in particular who had a really clear plan about what he wanted to do. He knew the enrolment handbook better than I. I met him over ten years ago, and this is still a vivid memory, because so many of the students I have taught were unclear about their destination. I’m embarrassed to admit, I probably didn’t help as much as I should have. I think I had this uneasy feeling that I would be seen as “pushing my own barrow”. But heck – if I didn’t think Operations Research and Statistics were important, I should not have been teaching them.

The funny thing is I spent many years making negative comments about the subject of Marketing, seeing it as at time quite harmful. I had a friendly ongoing banter with a wonderful marketing lecturer, John Watson, where we would each poke fun at the others’ subject. And now that I am in the world of business and trying to make a living by selling my apps and on-line courses, I realise that not only do I need to use marketing, but I actually quite enjoy it.  It’s a funny old world!

Information is power, and when we help students learn about possible careers and disciplines we are giving them power to make better choices. And that is important.

The Sound of Music meets Linear Programming

“Let’s start at the very beginning – a very good place to start. When you read you begin with A, B,C!” When you do statistics you begin with…probability? the mean? graphs?

Begin at the end

But really, is the beginning a very good place to start? Sometimes, we need to begin at the end. And sometimes we need to go back before the beginning. Always we need to think about where to begin, because it is seldom obvious, and copying what other teachers and textbooks have done is often a bad idea.

Linear programming

Take Linear Programming, the flagship technique of Operations Research. Most text books start with a simple two variable example, one that can be drawn on a Cartesian plane. They begin by defining the decision variables and the objective function. Next they formulate the constraints and explain the non-negativity conditions. Then finally they get around to solving the problem – often through a graphical approach, and applying it to the trivial real-life imaginary example they started with.

Here is a better approach, with Linear programming as the example:

First ensure all the class members have the prerequisite mathematical skills for what you propose to teach. If they are not good at drawing equations on a plane, you will need to teach them again, or use a different approach such as using Excel Solver. If students are not sure which way around > and < signs go, you will need to go over it. If English is their second language you will need to make sure you explain words like constraint, objective and optimum. This won’t hurt the native English speakers either.

Second think about your destination. When children learn to read, they generally know what the outcome is going to be. They will be able to look at words on a page and make sense of them. When you learn to drive, you know the outcome – you will be able to get safely from one place to another behind the wheel of a car. When we learn to bake cakes, we like to have pictures of the finished product so that we can see where we are headed. Yet somehow we try to teach as if it is a voyage of discovery with no vision of the end. Now discovery is good, if it pertains to how we get to or understand a process, but students need to know what they are learning. It also helps to have a purpose. Reading, driving and baking are all purposeful, with a clear outcome. The same should be true of linear programming (or confidence intervals or decision trees or fitted lines or just about anything else we are learning.)

You give the students an illustration of the completed LP model of the problem, preferably complex enough to be realistic. You show them how it can be useful, and give them a chance to explore the model. This is SO much easier now that we have Excel and Solver to look after the solving. Let students find out all about one model and then another and another, before you begin to show how to formulate. When people know what they are trying to produce, the reasoning behind the steps is more obvious.

Linear Regression

The same approach can be applied to teaching Linear Regression analysis. First we need to make sure that students understand what a fitted line on a graph is. Get them to interpret several fitted graphs, and use them to make predictions and write statements about the nature of the relationships modelled. Then show how to make the fitted graphs once they know why they need to.

In last week’s post I talked about histograms. Students should learn to interpret histograms and other graphs before they are required to make their own. Having to read off pie charts should help immunise them against their use.

I was in a computer lab with some students from another first year statistics course, and noticed that the first thing they were taught was how to calculate the mean and standard deviation, including the finite population correction. Was this really the most interesting way to get them introduced to the joys of data analysis and interpretation? Why start with the mean, one of the most difficult concepts in statistics?

Work backwards from the end

There is an interesting technique used for teaching skills to children with special needs. When you teach a blind child to tie shoelaces, you start at the end. You do all but the last part, and let them finish it off. This gives a sense of success and purpose. Then gradually you add the steps backwards, so that they start earlier on in the process. This also means that the part of the skill that is getting the most repetition is the new part, not the part already mastered. The same is true of memorisation. Memorise the last line first, then the last two lines etc. I suspect the same approach may well apply to more abstract skills. Maybe we should teach how to read and critique a statistical analysis, then how to write one, then finally how to do the analysis.

The spiral approach is popular, in which topics are revisited each year and built on.  I would like to incorporate principles of mastery learning along with that. Mastery learning is based on the premise that you must master a skill before moving on to the next one. This is difficult to implement in a classroom, with mixed level of ability, but is more easily enacted with the help of a Learning Management System.

New math had odd beginnings

I was born in the early 1960s and was in the first cohort of children to learn “new math(s)”, devised in the US as a reaction to the humiliation of seeing the Russians put Sputnik into space before them. Even in New Zealand we were not immune to the influence of the Cold War on education!  I loved our bright new textbooks,  which started with Set Theory – even at age 6. Every year the first page of the text book had diagrams of herds of sheep, prides of lions and other sundry collections.  I loved the Venn diagrams and the intersections – even cardinal numbers, but to this day I’m not sure how that connected with mathematics, and learning to add and subtract. And to this day I ask, “What were they thinking?” It appears that set theory is the foundation of all mathematics, and thus these mathematicians decided to start there, baffling teachers and parents alike, who were alienated by these words and symbols.

I have no doubt that the intention was to improve learning, but it seems ill-advised now. I wonder how our attempts will be viewed with the benefits of 40 years of hindsight. These days constructivism is a popular, though not universal, theory and approach to learning. The idea is that we create knowledge through adding new ideas and experiences onto our current knowledge. Sometimes that involves undoing erroneous or primitive knowledge.

Sometimes a good approach is historical – to imitate in the learner (in an accelerated form) the learning process through which mankind has progressed, preferably missing out the stupid bits. (Roman numerals are fun for some children, but pretty pointless once you realise the power of zero). It is certainly worth contemplating as an alternative approach.

This post has touched on ideas regarding the sequencing of a learning/teaching approach. There are many considerations and serious thought needs to go into where we start. Sometimes we need to start at the end.

Probability, Perception and False Positives

An understanding of probability empowers people to make informed choices in matters of great importance, including health screening, insurance, major weather events and terrorist threat. Unfortunately it has been shown that this understanding of probability eludes even some of our most educated professionals and decision-makers

Perceptions of Probability and Risk

There is a considerable body of work studying people’s perceptions of probability and risk, particularly by Amos Tversky and the Nobel prize-winning Daniel Kahnemann. This has uncovered many systematic errors humans make in judging the relative probabilities of uncertain events. The brain’s tendency to find patterns results in heuristics or rules that have consistent bias. For example, if we have recently experience or even heard of a bad random event, we perceive the probability to be higher than it really is. Having experienced two years of earthquakes in Christchurch, my estimation of the likelihood of an earthquake in other places is markedly increased. I (and many others from here) feel uneasy surrounded by tall buildings, street awnings and unsecured masonry in other cities, particularly Wellington, but even in cities with no known earthquake risk.

Cultural implications

The perception of probability is also found to be cultural. I analysed a probability-based task as part of the National Education Monitoring Project. I found that there was a statistical and practical difference between the responses of ten-year-old Pacific Island students and NZ European students. I hypothesised that different home experiences involving games of chance may have led to this.  Further reading uncovered other research which had identified other cultural differences. In particular, there are cultures in which everything is perceived to be decided by God and there is no chance but rather a lack of knowledge of God’s will.

In fact many things that we perceive to be subject to chance, would not be, if we had perfect knowledge. Increased understanding of weather patterns has made forecasting more reliable, which has reduced the level of uncertainty with regard to the arrival of bad storms like the recent Hurricane Sandy, or to a lesser extent, two heavy snowfalls in Christchurch in 2011. Even a coin toss is, strictly speaking, only a function of the placement of the coin and thumb, the amount of force applied and various other external factors. Because we cannot measure these factors, we are left to assume that the chance of a head or a tail is equal until shown otherwise.

Screening tests

In disease screening we generally do know the figures, and are not relying on subjective judgment as to the probabilities. However the interpretation of the figures is notoriously badly done. There is a great deal of money involved in the screening industry, and it is an emotive area. Neither money nor emotion aids rational decision-making. This is exacerbated by misinterpretation of probabilities, and selective cost-counting.

My eyes were opened to this issue by a keynote address by Gerd Gigerenzer, director at Max Planck Institute for Human Development . There is a very interesting 8 question quiz at the Harding Center. Try it now. http://www.harding-center.com/  (I was very excited to score 100%, but I put that down to having heard the address, and thought seriously about this.) It would be great if you could tell us your score and reaction to the quiz in the comments below.

A week ago Tim Harford wrote about the lack of understanding among physicians in his post, “Why aren’t we doing the maths? – The practical implications of misplaced confidence when dealing with statistical evidence are obvious and worrying.” This problem is not going away. Some of the comments on the post expressed regret that probability questions like these are not part of the school curriculum, and that it is difficult to find resources to learn on-line. In New Zealand a new curriculum is being introduced with a greater emphasis on statistics at all levels. At year 12 knowledge of understanding of risk, particularly using two-way tables, is examined. As we develop materials to help teach this, we will make them available to the general public.

Example

The following link takes you to a pdf of a powerpoint presentation that teaches a step-by-step approach to this: Risk and Screening – step-by-step approach
We have found that this approach is helpful to students.

In particular you need to make sure that the table has “What the test tells us” along the top, and “What is the reality” down the side. You do not have columns or rows saying “Correct” or “incorrect” as this is much more difficult.

At present there is no audio to go with this segment, but we hope it is self-explanatory.

The costs of screening

Just in case you are tempted to think that all screening must be good and more screening must therefore be better, here are some things to think about.

The following article Breast screening is harmful appeared recently and I found it after I had written the rest of this post. I am very excited to read that  “BreastScreen Aotearoa is revising its leaflets to incorporate information about the risks of overdiagnosis”.

Screening is big business. There are the obvious costs of the equipment and staffing, including nurses, doctors, technicians and clerical workers. Added to that is the cost of loss of productivity for the time taken for the test. The test itself may be harmful. The cost of a false positive is considerable, including unnecessary further tests and interventions, some of which do actual harm. When screening is increased to include people at low-risk, the number of false positives increases, which then takes up resources, and can prevent people who really need intervention from getting it. The emotional costs of a false positive are far-reaching, unnecessarily decreasing quality of life, as people lose confidence in their own health and medicine.

More screening can be harmful

Too often lobby groups,with well-intentioned but ill-informed leaders can do harm. This was possibly the case with breast cancer screening in New Zealand. The age of free screening was lowered to include a group for which the test is less accurate, resulting in many more false positives. A correct understanding of probability in the general populace might have prevented this.

What is clear is that information needs to be better explained in order for informed consent to occur.

Which type of error do you prefer?

Mayor Bloomberg is avoiding a Type 2 error

As I write this, Hurricane Sandy is bearing down on the east coast of the United States. Mayor Bloomberg has ordered evacuations from various parts of New York City. All over the region people are stocking up on food and other essentials and waiting for Sandy to arrive. And if Sandy doesn’t turn out to be the worst storm ever, will people be relieved or disappointed? Either way there is a lot of money involved. And more importantly, risk of human injury and death. Will the forecasters be blamed for over-predicting?

Types of error

There are two ways to get this sort of decision wrong. We can do something and find out it was a waste of time, or we can do nothing and wish that we had done something. In the subject of statistics these are known as Type 1 and Type 2 errors. Teaching about Type 1 and Type 2 errors is quite tricky and students often get confused. Does it REALLY matter if they get them around the wrong way? Possibly not, but what really does matter is that students are aware of their existence. We would love to be able to make decisions under certainty, but most decisions involve uncertainty, or risk. We have to choose between the possibility of taking an opportunity and finding out that it was a mistake, and the possibility of turning down an opportunity and missing out on something.

Earthquake prediction

In another recent event, Italian scientists have been convicted of manslaughter for failing to predict a catastrophic earthquake. This has particular resonance in Christchurch as our city has recently been shaken by several large quakes and a multitude of smaller aftershocks. You can see a graph of the Christchurch shakes at this site. In most part the people of Christchurch understand that it is not possible to predict the occurrence of earthquakes. However it seems that the scientists in Italy may have overstated the lack of risk. Just because you can’t accurately predict an earthquake, it doesn’t mean it won’t happen. Here is a link to a story by Nature of the Italian earthquake.

Tornado warnings

Laura McLay wrote a very interesting post entitled. “what is the optimal false alarm rate for tornado warnings?” . A high rate of false alarms is likened to the “boy who cried wolf”, to whom nobody listens any more. You would think that there is no harm in warning unnecessarily, but in the long term there is potential loss of life because people fail to heed subsequent warnings.

Operations Research and errors

Pure mathematicians tend not to like statistics much as it isn’t exact. It’s a little bit sullied by its contact with the real world. However Operations Research goes a step further into the messy world of reality and evaluates the cost of each type of error. Decisions are often converted into dollar terms within decision analysis. Like it or not, the dollar is the usual measure of worth, even for a human life, though sometimes a measure called “utility” is employed.

Costs of Errors

Sometimes there is very little cost to a type 2 error. A bank manager refusing to fund a new business is avoiding the risk of a type 1 error, which would result in a loss of money. They then become open to at type 2 error, that they missed out on funding a winner. The balance is very much on the side of avoiding a type 1 error. In terms of choosing a life partner, some people are happy to risk a type 1 error, and marry, while others, hold back, perhaps invoking a type 2 error by missing out on a “soul-mate”. Or it may be that we make this decision under the illusion of certainty and perfect information, and the possible errors do not cross our minds.

Cancer screening is a common illustration of type 1 and type 2 errors. With a type 1 error, we get a false positive and are told we have a cancer when we do not. With type 2, the test fails to detect a cancer. In this example the cost of a type 2 error seems to be much worse than type 1. Surely we would rather know if we have cancer? However in the case of prostate cancer, a type 1 error can lead to awful side-effects from unnecessary tests. Conversely a large number of men die from other causes, happily unaware that they have early stages of prostate cancer.

The point is that there is no easy answer when making such decisions.

Teaching about type 1 and type 2 errors

I have found the following helpful when teaching about type 1 and type 2 errors in statistics. Think first about the action that was taken. If the null hypothesis was rejected, we have said that there is an effect. After rejecting the null only two outcomes are possible. We have made the correct decision, or we have made a type 1 error. Conversely if we do not reject the null hypothesis, and do nothing, we have either been correct or made a type 2 error. You cannot make a type 1 error and a type 2 error in the same decision.

• Decision:Reject the Null. Outcome is:
  • Correct or
  • Type 1 error
• Decision:Do not reject the Null. Outcome is:
  • Correct or
  • Type 2 error.

Or another way of looking at it is:

• Do something and get it wrong – Type 1 error
• Do nothing and regret it – Type 2 error

Avoid error

Students may wonder why we have to have any kind of error. Can we not do something to remove error? In some cases we can – we can spend more money and take a larger sample, thus reducing the likelihood of error. However, that too has its cost. The three costs are important aspects of decision-making, and helping students to understand this will help them to make and understand decisions.