Drill and Rote in teaching LP and Hypothesis Testing

Drill and rote-learning are derogatory terms in many education settings. They have the musty taint of “old-fashioned” ways of teaching. They evoke images of wooden classrooms and tight-lipped spinsters dressed in grey looming over trembling pupils as they recite their times-tables. Drill and rote-learning imply mindless repetition, devoid of understanding.

Much more attractive educational terms are “discovery”, “exploration”, “engagement”. Constructivism requires that learners engage with their materials and create learning by building on existing knowledge and experiences.

But (and I’m sure you could see this coming) I think there is a place for something not far from drill or rote-learning when teaching statistics and operations research. However I like to call it “well-designed repetitive practice”, rather than drill or rote-learning. With another name it smells a little sweeter.

Students need repeated exposure to and exploration of spreadsheet Linear Programming models in order to generalize and construct their own understanding correctly. Students benefit from repeated exposure to hypothesis testing in different contexts in order to discern the general from the specific. But this is not “mindless repetition” of similar examples where wrong generalizations can (and will) be constructed. The different examples should be carefully managed to make effective use of students’ time, and avoid reinforcement of incorrect concepts.

Reason for well-designed repetitive practice

A single instance of a phenomenon does not provide enough information to transfer to another instance. It is only by being exposed to multiple instances that learners can decide which aspects are in common or general, and which are specific to that particular example. Exploring one instance of a linear program (LP) in a standard format gives an initial understanding, but in order to generalize, there must be multiple examples.

Learners, in general, endeavor to make sense of the material by making generalizations about the different examples they are given. If the common elements they perceive are not relevant, the learners make incorrect generalizations. If the first three examples of an LP spreadsheet have all decision variables in the same units, students can reasonably assume that LPs require decision variables to use the same units. To avoid this, the set of examples used must be carefully constructed. If all the hypothesis testing examples result in rejecting the null hypothesis, students gain an incorrect generalization that this is the usual result.

It is popular practice in entry-level statistics courses to require students to collect their own data, analyse and report on it. This is a wonderful way for students to learn and engage with the process of statistical analysis. My concern is that it gives only one example from which the student can construct their understanding of the process. Ideally students would have exposure to many different examples before embarking on their own project.

A learning management system is invaluable. We have a bank of very carefully constructed examples which students work through, to help them gradually develop understanding. The data is real – from questionnaires they or earlier classes completed. There is immediate feedback on submission of their answers, again to reinforce correct concepts. We explain to students that they should not to wait until they understand the process completely before they begin, but rather that the understanding comes with doing. There are many parallels for this kind of learning. Chess, sports, driving and speaking a language all develop through practice. Understanding follows practice.

What’s more, this method seems to work. Students are motivated to work through multiple examples so that they internalize the process and improve their understanding. And they gain a sense of accomplishment and confidence at correctly completing the examples.


No more lectures!

The lecture is the mainstay of higher education, but is it really a good way for people to learn?

Here is a guest blog about my statistics course. In No more lectures I explain how my course, “Quantitative Methods for Business” uses Moodle to deliver self-paced learning materials in a blended course. This became especially useful after the Christchurch earthquake, which occurred a year ago today.

Another interesting discussion about doing away with lectures can be found here: How to replace the lecture

And here is another interesting description of four things a lecture is good for.

Just like textbooks, the lecture needs rethinking.

Giving students dirty data

Dirty data is real data as it is collected before someone gets hold of it and takes out the tricky bits. You won’t find dirty data in textbooks. Dirty data is what real researchers have to deal with. And even amateur researchers and students doing real-life projects will have to deal with dirty data. Yet not much is said about dirty data, and what to do with it.

Elements of dirty data

Mistakes – people put down the current year for their date of birth, give their weight in the wrong unit, put an extra decimal point.

Missing data – people leave gaps, possibly by mistake and possibly intentionally, or give up before the end.

Mindless response – people just tick all the middle responses to Likert scales, or answer “no” to everything.

Silly answers – people state that they expect to earn over 1 billion dollars next year, say they want to have 28 children or are 105 years old and weigh 500 pounds.

Detecting dirty data

Messy girl

Assume that all data collected from people is dirty

First of all assume your data is dirty, particularly if humans have been involved, and even more so if students have been involved. To find the problem areas you need to make tables, graphs and summary statistics of all the variables, and look for outliers. Look for consistent missing values. Look at the highest and lowest values. Scatter-charts are also good for identifying anomalous data.

Dealing with dirty data

Well – this is where mathematics and statistics inextricably part company. There is no single right answer. It all depends! (Students hate that phrase.) Sometimes you should take the response out. Sometimes you should make it a missing value. Sometimes you should correct it. Sometimes you should remove a complete record or observation. Always you should document and justify your decisions, and be aware of any possible implications. There is a fine line between cleaning data and massaging it into something that will give the results you are seeking. There are some actions that are insupportable.

Teaching with dirty data

If students do their own projects they will need to deal with dirty data. It is a wonderful opportunity to make them suffer help them learn. Don’t give them the answers, but get them to make the judgment calls – that’s what real researchers have to do.

However not all statistics courses include student projects. (Our first year course doesn’t for reasons I will cover in a later post). I do give postgraduate business students a set of data as it was collected, raw from the students. Part of their assignment is to clean it up before they start, and provide a report on what they have done and why.

For the introductory course for undergraduate students I clean up the data, so that the missing and spurious values don’t injure their fragile confidence. The point in this particular instance is to practice multiple examples of different types of testing, in order to generalise the principles of hypothesis testing. Excel, which I use with reservations (another later post) doesn’t cope well with missing values and would provide barriers too early in their learning. Whether the data is given to them clean or dirty depends on the learning objective of the exercise – and the nature of the students.

I would like to get them using the original data, but the course is not quite long enough. I’m still mulling over that one. Having written this post, I am convinced I need to do something about it.  I’ll get back to you.

Effective multimedia teaching videos

I have converted lectures into considerably shorter videos that students view at their own time and pace, and as many times as they like. Hosted on YouTube, they are open to an international audience and have proved popular. Here are some tips that may be useful to other instructors interested in doing likewise. (Though teachers of statistics, Excel or linear programming are welcome to use ours!)

Pictures and words

As much as possible two channels, pictures and words are used. Narration and illustrations complement each other. There are no talking heads.

There is considerable research regarding the effectiveness of multimedia instruction. Mayer shows in his experiments that “students learn more deeply from a multimedia explanation presented in words and pictures than in words alone”, which he calls “the multimedia effect.”

Levels of Measurement

Illustrations complement the narrative

Mayer developed a framework, based on aspects of cognitive science, which helps to explain this multimedia effect. The framework assumes that humans process pictures and words using different parts of working memory, both of which are limited. However, the total amount of information that can be taken in is increased by using both input channels (pictures and words), which appear to have independent capacities.

Fast conversational narrative

Conversational narrative is used, which Mayer suggests is more effective. In addition the rate of speech is fast. This is based on the premise that students can stop, pause and go back if they didn’t understand something, but will lose interest in a slow and ponderous delivery. All the narrative is tightly scripted so that there is no wasted time, and the best possible explanations are used.

Avoid extraneous material

Though I accept that generally extraneous material must be avoided, we do add humour to our videos, as we believe it keeps people’s attention, and lightens the atmosphere. Diagrams are developed carefully to aid memory and support the message, rather than distract from it.

Diagrams are used to aid memory and understanding

No summaries

In general we do not have summaries at the ends of the videos. The videos are less than ten minutes long, so students can go back and watch them again. From examining the YouTube viewer statistics we found that the viewing levels plummeted when we began the summary.

Keep them short

We try to keep videos between 5 and 7 minutes, although one has grown to 10 minutes. Define a small set of knowledge or skills and focus on that. If there is too much material, make two videos. There is a lot of padding in a traditional lecture, whereas our videos are scripted to say exactly and effectively what we want to say; consequently so you can fit at least 30 minutes of lecture content into about five to seven minutes of video.

Think REALLY hard

The time limit on the explanation means that you need to examine what makes the material difficult to grasp, and what are some ways to make it clearer. There are well-known Youtube mathematics videos that use a rambling approach with little or no prior preparation. We can do better than that. Each of our videos is the product of years of experience of explanations to thousands of students, accompanied by deep thought and experimentation to find ways to explain things.

Try new things – be creative – have fun

To begin with our videos under the UCMSCI banner were quite primitive and quirky, focussing mainly on Excel implementation. Our later videos as Creative Heuristics are more polished, and also use diagrams and animations to get our material across. And the best is yet to come!

Teaching Operations Research with food

Food is a universal context

What I like about Operations Research is its applied nature. It is mathematical and useful. We need to make sure that our students recognize that. As students have often had little experience in the world of business and manufacturing, it can be helpful to use examples based around food. Food is a universal topic. We all need food and most of us enjoy it and probably eat too much. For this reason food is a useful context to use in teaching.

Linear programming diet problems

The linear programming diet problem is an obvious starting place. For decades linear programs have been used to find optimal combinations of different types of feed for animals such as pigs, cattle and poultry. A popular teaching diet problem is based on McDonald’s fast food. Information on nutritional content and requirements is easily sourced these days on the web. Students can create their own diet problems and find the least cost solution to feed themselves a minimum cost, balanced (within reason) diet at McDonalds. When we used this as a student assignment the solution involved a large number of soft-serve ice cream cones. This sort of solution helps teach students that the computer finds the optimal solution to the model, which may not be even feasible in reality, if there are too many implicit constraints.

A popular variation of the McDonalds diet problem was the Hiking diet problem. This was a little different as the objective was minimising weight rather than cost. Again the initial “optimal” solution lacked variety. (This gave us the opportunity to teach that the number of non-zero variables at optimality will not exceed the number of binding constraints.) The students enjoyed the assignment and some were inspired to the extent that they used it to cater for a hike. Another example was refugee boxes, aiming to provide a balanced diet for a family for a week, at minimum cost.In all these examples the problem of non-integer solutions can also be addressed.

Other Operations Research topics can be taught using food.

When teaching about fixed and variable costs, and developing a spreadsheet model, our example is that of a sausage sizzle. In it the students are setting up to sell sausages wrapped in bread, with onions, cooked on a barbecue. The aim is to find the breakeven point.

Critical path can be taught in the context of planning a three course dinner.

Many examples are built around Choconutties

I have been accused of making everything about food. Our series of videos for teaching statistics and spreadsheet modelling centre around Helen, who sells choconutties. In our test and practice examples we have bakeries, candy manufacturers, jam-makers and cafes. Food examples are used for inventory control, break-even analysis, linear programming, decision analysis and critical path.

Students need to be engaged with the material, and it helps if the examples and contexts are familiar. There is a case for using business examples as well, as these help develop their understanding of business and its vocabulary. But food is a good place to start.