I don’t like the Khan Academy videos about statistics. But I can see why some people do. Some are okay, though some are very bad. I’m rather sorry they exist though, as they perpetuate the idea of statistics as mathematics.
Khan Academy, critics and supporters
Just in case you have been living under a rock, with respect to mathematics education, I will explain what Khan Academy is.
Sal Khan made little YouTube videos to teach a family member maths. Other people watched them and found them useful. Bill Gates discovered them and threw money at them. Now there are heaps of videos, with some back up exercises, and some people think this is the best thing to happen to maths (and other) education. Other people think that the videos lack pedagogical content knowledge. Sal agrees – he says he just makes them up as he goes along.
Diane Ravitch linked into the Khan Academy debate, beginning with this post, which is what got me looking into this. Two mathematics teachers made videos after the style of Mystery Science Theater 3000 starring two of Khan’s poorer mathematical contributions. The one on multiplying negative numbers was particularly poor and has since been replaced. Critiques of Khan seem to meet with two kinds of comments. One group is people who know about teaching, who are pleased that someone is pointing out that the emperor, though not naked, is poorly clad. The other lot are generally telling the mean teachers to leave Khan alone, that he is the saviour of mathematics teaching, and they would never have understood mathematics without him. The supporters also either suggest vested interest (for people who make educational materials) or that the writers should try to do better (for those people who don’t make educational materials). To be fair, the first group are also calling for other people to make better videos and put them out there.
For a good summary of the pros and cons of KA, here is a recent article in the Washington Post: “How well does Khan Academy teach?”
Khan Academy Statistics videos
So I took a look at Khan Academy statistics videos. I know something about the teaching of statistics. I have many years of experience of successful teaching, I have done research and I have read some of the literature. I have pedagogical content knowledge (I understand what makes it hard for people to learn statistics.) And I have made my own statistics teaching videos, which have been well received. I wrote some time ago about the educational principles based on research into multimedia, which have been used in developing these videos. Unlike Khan I have thought hard and long about how to present these tricky concepts. I have written and rewritten the scripts. I have edited my audio to remove errors and hesitations, I have…anyway – back to Khan Academy.
To be fair, statistics is one of the most difficult subjects to teach, so I didn’t have high hopes.
To start with the list of topics under the statistics heading showed a strong mathematics influence. This may reflect the state of the curriculum in the United States, but in no way reflects current understanding of how statistics is best understood. I couldn’t find anything about variation, levels of measurement and sampling methods, which are all foundation concepts of statistics. I think it would be more correct to call the collection of videos “the mathematics of statistics”. It starts with the “Mean, Median and Mode.” Not exactly a great way to enter the exciting world of statistics. And he mispronounces the adjectival use of “arithmetic”, which is a bit embarrassing.
Reading Pie Graphs
I summoned up courage to view the video on reading Pie Graphs. It was not good. The example was percentages of ticket sales for Mediterranean cruises over a year. That data should never have been put into a Pie Graph. For two reasons! First there are too many slices of pie. A pie chart should never be used for that many categories. But worse than that, the categories are ordinal – they are months. The best choice of graph is a bar or column chart, with the months in order, as you would then be able to see trend! (I have to stop myself here or I could rave on much longer). My point is that Khan has used a very BAD graph as his example. This is one of the worst things a teacher can do, as it entrenches in the students’ minds that this is acceptable. The only thing good about the graph was that it was not three dimensional, and it is not exploding. It didn’t even have a title. Bad, bad, bad. (Sorry I was meant to be stopping)
Confidence Intervals
I am tempted to say Khan is arrogant to think he can produce something after a few minutes thought. Actually I was tempted to say something rather stronger than that. I have to admit I haven’t watched many of the videos, but I really don’t want to spend too much of my life doing that. I chose one on Confidence Intervals, which nearly had me throwing things at the computer. It never explains what a confidence interval is. The bumbling around was so painful I couldn’t watch the video in its entirety. I’m pretty sure he got it wrong. He was so confused by the end that I can’t say for sure. My own confidence intervals video is one of my earlier ones, so it is a little rough, but I’d wager most people understand better what a confidence interval is after watching it.
So then I decided I should look at the video entitled p-value and hypothesis tests. This is something I know many people struggle with. It is crucial to understanding inferential statistics. I have spent many hours working out ways in which to teach this that will help people to understand.
The p-value and hypothesis testing
Well I watched most of the p-value video, and was pleasantly surprised. The explanation of how we get the p-value is sound, and once he gets into his flow, the hesitations get less irritating. There is a small error – talking about 100 samples, rather than a sample of 100 observations. Also it is a bad idea to have a sample size of 100 in an example as this can get confused with the 100 in the expression of the confidence interval as a percentage. But it does give a good mathematical explanation of how the p-value is calculated. I’m not sure how well it helps students to understand what a p-value is. For a mathematically capable student, this would probably be enlightening. I have my doubts about most of the business students I have taught over the last two decades.
My main criticism is that the video is dull. It doesn’t provide anything more than the mathematics. But apart from alienating non-mathematical students it isn’t harmful. In fact if I had a student who wanted to know the mathematics behind the statistics, I would be happy to send them there. People have commented that my videos don’t tell you how the p-value is calculated. This is true. That is not the aim. Maybe I’ll do one about that one day, but I figured it was more important to know what to do with one.
Khan Academy videos on statistics aren’t good
My point is, surely we can do better than that! Bill Gates has thrown money at the Khan Academy. Wouldn’t it be wonderful if it were the purveyor of really good practice rather than mediocrity? One blogger suggests that if Khan Academy could use really good videos, it really could be useful.
I have gone on long enough.
I realise now, that asking a busy person to watch my videos is a bit of a cheek. They aren’t that long though. They are funny and clever. They are NOT like Khan Academy. I think they are worth the six to ten minutes each.
Here are links to my three most popular ones. Enjoy.
Thanks. I do hope so.
Oh, but of course… the only thing that matters is that The Students Like It! (At least the ones they care to listen to).
I liken it to the latest diet craze… people who are desperate for something that looks like success with something that’s confounded them are prone to having conversion experiences.
Pingback: Khan Academy Statistics videos are not good | Business and Economics: E-Learning and Blended Learning | Scoop.it
Reblogged this on Things I grab, motley collection .
Pingback: Khan Academy Statistics videos are not good | Web 2.0 for juandoming | Scoop.it
I’ve been following the criticisms and discussions around this as well. Right of the bat, I think Salman Khan is a hero — I always thought that he was going to set of a revolution in education. That has been true, with 3 Stanford’s courses being offered online, now followed by hundreds on Coursera and this fall edX by MIT and Harvard.
Can’t we praise what he’s done, rather than nit-pick on the few little errors here and there, glossing over the fact that he is helping millions?
Even though his methods/material is not perfect, he’s still better than most Math teachers out there. For that majority, Sal provides an alternative. He makes Match “fun” for those who always sleep through math lectures.
They are working hard to make things better. The lead developer at Khan responded on HN: “We have four ex-teachers as full-time employees. We have two high school math teachers as consultants. One Harvard Doctoral candidate in Education and one post-doc in neuroscience at Stanford are in residence. One UPenn Professor is also likely to begin a sabbatical with us. We have a 3 person team dedicated to working with and getting feedback from our 50 pilot classrooms and the 15,000 teachers actively using KA in classrooms.”
What I deem more important, is that Khan Academy is revolutionizing education first-hand, flipping the classroom. Problem-solving gets done in class with peers, while lectures are being watched at home (at their own pace).
We need more heroes like him and his team. Khan Acedemy is an NGO, nobody is doing it for profit and we need to rally behind these initiatives. The education system as is has way too many holes in it — in dire need of a refresher.
My thoughts too. Instead of being jealous of Khan’s success, ride the wave. Before Khan online learning was something relegated to the college setting. Maybe teachers and academics had vague hopes for something like this, but the for mainstream audience online learning was a neat idea that could never really work.
Khan changed that perception. Online teaching and resources for learners are being taken seriously. Real money and attention is being poured into this area and it is not just Khan Academy who benefits. His success encourages others that this is possible. Sometimes the rising tide really does lift all boats: ride the wave.
We can not realistically expect Khan to teach every subject perfectly. Even where he falls short his material is better than the instruction many students have available to them.
If it were not for the Khan Academy we would not even be having this conversation. Critic his material, that is healthy, but it is petty to attack an institution that is dramatically changing the ways we teach for the better. Why do those who disagree with Khan insist on making this such a divisive issue? Surely there are ways to improve upon Khan without resorting to this us vs. them mentality. Why does your success seem predicated on Khan’s failure?
Sal Khan is “still better than most Math teachers out there.” This line gets used over and over by those defending Khan and claiming he’s the second coming of whatever messiah you believe in. I’d really like to see where this data is coming from. Not a single one of the posts making this assertion provide any research as support or even cite an unbiased poll.
The Khan Academy has great potential to do great good, but a product that has both content errors and pedagogical errors throughout dozens of its videos needs to do better quality control.
In addition, what Khan is doing is not revolutionary; he’s taking ideas pioneered by hardworking classroom teachers years before him and claiming them as his own. The reason KA seems revolutionary is because money shouts very loudly and drowns out other voices so Khan can easily claim that he’s discovered something no other teacher has ever done, thereby bashing the very people whose ideas are making him famous.
In terms of data, it might not be scientific, but telling nonetheless: 174M lessons delivered. This number cannot come from one-time visitors — these are return visitors that love his way of teaching. Millions are learning more, where they had no (free) alternative before. Also telling is the amount of likes and positive comments on his videos (vs negative). These are people who appreciate the alternative, because they consider their Math teacher dull & torturous, hence leaving them unable to retain/comprehend Math..
A survey to back it up would be great, but here’s what Andrew Hacker wrote in “Is Algebra Necessary?”: “A TYPICAL American school day finds some six million high school students and two million college freshmen struggling with algebra.” He goes with a lot of data on the failing matter of algebra among high-school students.
IMHO, the main cause is the de-facto standard teaching style. There are many points I could list why I think KA’s approach is better for the kids struggling with Math, but Sal does a better job at his TED talk: http://www.ted.com/talks/lang/en/salman_khan_let_s_use_video_to_reinvent_education.html
The ideas are not what’s revolutionary — it’s the execution of a simple idea that makes it genius. Online teaching has been around for years pre-KA, but never was there a platform that attracted such a user base.
Google did not invent the idea of indexing the web. Facebook did not invent the idea of Social Media. Apple did not invent the tablet. They all took simple ideas and brought it to excellent execution — and are revered as the real heroes.
It is exactly the zealotry displayed here that led me to examine the statistics videos.
If Khan were still producing charmingly bumbling videos to help his cousin with her schoolwork I couldn’t give a flying fig what they were like.
Precisely because KA is heralded as the saviour of the education system it is imperative that he GET IT RIGHT!
Being free is not an excuse for misleading people.
Khan is not a hero. He is lucky. But he is out of his depth. There are many people in history who have had an immensely loyal and devoted following, sometimes with disastrous results for their country and even the world.
I don’t claim to speak on any other subject, but the statistics videos need peer review.
Khan Academy truly could be many of the things its followers think it is, with examination and critical review by people with deep knowledge of the subject matter and pedagogy.
Dr. Nic wrote:
There are many people in history who have had an immensely loyal and devoted following, sometimes with disastrous results for their country and even the world.
Are you comparing the Khan Academy web site to dictators like Hitler and Stalin? Who’s acting like a zealot here?
Dr Nic: Perhaps someone more like Andrew Wakefield?
From Wikipedia:
” a British former surgeon and medical researcher, known for his fraudulent 1998 research paper in support of the now-discredited claim that there is a link between the administration of the measles, mumps and rubella (MMR) vaccine and the appearance of autism and bowel disease”
Not to imply that Khan is in any way fraudulent, but in many areas, wrong! Unfortunately, I don’t have to scour the scientific press very diligently to find examples of scientific research that is based on an incorrect understanding of statistical principles for use in my statistics classes. These things lead to policy decisions that can affect millions.
Make your own videos then.
I have made several, which you can see on the CreativeHeuristics channel of YouTube. Unfortunately they take a lot of effort to make when you do it properly, and I don’t have the backing of Bill Gates. But I feel I have gone for quality and pedagogical foundations, so I hope you will find them useful. They are pretty popular and get lots of good feedback.
I just wanted to say that I appreciate your dedication to pedagogy of mathematics. One of the reasons I believe people are defensive about KA is because the pedagogical difference may not matter to them. They may already understand statistics (or they may have even learned through KA) and not hit many of the common stumbling blocks that other students hit. Or worse, they may have internalized certain misconceptions about stats that they don’t fully appreciate.
One of the big problems at my university was that professors were brought in because of their research genius, not their teaching ability. Many of them were phenomenal thinkers in their respective fields, but had a hard time with language. It made it a requirement that students in these areas basically teach themselves from the homework problems.
I think this is a common thing. People in mathy areas get used to needing to teach themselves things. This is great! People should be able to teach themselves things! But if you don’t have the same background, a good teacher can make all the difference. And more pessimistically, as I said, you may come to an erroneous conclusion if you don’t have someone to correct your mistakes.
Unfortunately, misconceptions in statistics can mean drugs that shouldn’t go to market, changes in political policies that are based on faulty information, incorrect diagnoses, unhealthy fads, etc.
I think that your focus on pedagogy is phenomenal. I wish I’d had more teachers at college of whom I could say the same.
Thank you. I really appreciate your kind words. I was recently made redundant, in part because my emphasis was too little on research. Research is useful, but I feel that helping people really understand statistics is an important role.
I don’t know if I’m the only one seeing that but I got the feeling that the only reason you bash him so hard is for promoting your own videos. And I don’t think it’s with this kind of attitude that you’ll get some attention of the Bill Gates Foundation.
Oh excuse me, I’m not worthy to write about this since I don’t got any «… deep knowledge of the subject matter and pedagogy».
I’m pretty sure this is a “no win” situation. Half the Khan supporters say “Make your own then”, but if you do, then they say you are pushing your own barrow. Feel free to watch my videos and critique them!
Khan does videos on MANY different topics, not just statistics. You focus on just one topic, so i would assume it would be a lot easier for you to really go in-depth. Khan’s videos cover chemistry, calculus, bio, organic chemistry etc. He does a good job, because i use those videos to get a general idea and understand the topics in the subject before actually taking the class (or while taking the class).
That is my point, really. Because Khan is not an expert in many areas, sometimes the material is not well presented, and sometimes it is not correct. I wouldn’t presume to make videos on subjects I don’t know much about. Just because you choose to use them, doesn’t mean he has done a good job. I think what you mean is that it is an easy place to find a lot of adequate videos that seem to help students get a general idea. I suggest that it would be preferable to use better videos if they are available, but if not, then Khan is possibly fine in other areas, but is a bit limited in statistics.
I can understand your frustration with the Khan Academy videos, but I believe that his intentions are good and that he is doing the best that he can. He is looking for math content creators who actually know what they are doing more than he does. Have you contacted them about the errors in the videos? Perhaps if they realize that there are problems, they will try harder to get a statistician to redo the content.
Pingback: Khan Academy Statistics videos are not good | news_ali | Scoop.it
Dear Dr Nic
You, yourself, is a very smart individual. I would think that you should join the Khan Academy to help them succeed in the Statistics field. This way people will respect you more.
There are a lot of people who think they are superior and others people’s efforts are inferior. Very immature. No offense, but your videos on statistics are run of the mill, average videos I was ‘forced’ to watch in math class in the 80′s. Khan’s videos keep me awake and alert and are enjoyable to watch. I keep coming back to them for more ‘voluntarily.’ Your view is biased, you are clearly jealous, you call him lucky when he has helped people pass exams and advance in school. His contribution is much more giving and broader than yours. Please fill out a job application with him and hope you can also be a contributor instead of an antagonist. By the way, we have enough 1980 style videos out there.
Pingback: Good, Bad and Wrong: Videos about Confidence Intervals | Learn and Teach Statistics and Operations Research
Hi Dr. Nic,
Have you considered posting your concerns in the comments of Sal’s videos or writing your suggestions to him? I read an article about 2 physics teachers discovering an error in one of his videos; when they contacted him about the error, Sal swiftly corrected his video and thanked them for it.
I’ve watched him in several interviews, and he strikes me as a friendly and reasonable person who would welcome feedback, especially from a passionate teacher like you. One nice feature about his videos is how quickly people can comment on any confusing parts or mistakes and interact with each other in resolving them; I’ve seen Sal responding to those comments directly and acknowledging those mistakes.
I hope that you two dedicated educators can collaborate to teach all of us better.
Sincerely,
Eric Cai – The Chemical Statistician
http://chemicalstatistician.wordpress.com
https://twitter.com/chemstateric