In grad school I managed to take advantage of the pacing effect in an educational setting. I was teaching linear algebra. What I did was make the homework incremental - 1/3 of homework on today's material, 1/3 on the previous week, and 1/3 anything in the course. Those thirds were in increasing order of difficulty.
I also started every class with a question/answer period. The rules were simple, the questioning will last at least 10 minutes, and you don't want me to ask the questions. :-) Anything that had come up in the questions that seemed to be a point of confusion was sure to be added to the next homework set.
I won't go into what else I did with that class, but the end result is worth thinking about. First note that I gave a ridiculously hard final. Other grad students who saw it thought that the class would bomb. Secondly they aced the test. What do I mean by aced? Well I had a bonus question which fellow grad students thought nobody would get. 70% of the class got that question, and a good fraction were over 100% on the test. So they must have studied hard, right? Nope. I ran into some students several months later. They told me that they tried to study for the final and stopped after a few minutes because it was useless, they knew everything. And several months later they still knew much of the material cold!
The thing is that none of what I did was very radical. The principles have been known for a century. Psychologists have been trying to get people to listen for that long. I learned about it in the 80s from a university course I watched on TV. (British Columbia had a TV channel devoted to lectures for correspondence courses.)
Yet, despite how dramatic the effects are, nobody listens and nobody takes advantage of it.
What an excellent Hacker News Post. Please do write up a bit more about your method, what your students got out of the course specifically, and whether you've had success with it in other places. You have a clean, readable, and interesting writing style.
I think we can all recall a teacher or teachers in our life who truly made a difference in teaching us something - sometimes _despite_ our interest (or lack thereof) of learning it.
I was forced in University to take a course in Propositional Calculus (it was a breadth requirement for computing science) - it was mid-day so I attended the class (unlike early morning classes in which my attendance was abysmal) - I went in with zero desire - no coding, or even math. But, the timing was right so I just sat in the class.
The instructor spent thirteen weeks walking us through Reductio Ad absurdum proof, tautologies, conditional proofs, etc.. He gave us homework assignments that built on earlier knowledge, and just as we were getting into new material, he'd throw some earlier sequent proofs at us to solve and refresh our memories.
Here is the thing - I had _no_ interest in this course, the only thing going for it was it's timing, and repetition. I was _killing_ myself trying to learn integration by parts, linear algebra, discrete math, spending 5 hours outside of class for every hour in class trying to master those other course, and basically just sitting in this one distraction course. I was a C+ student in the math courses (after _massive_ effort) - I walked out of the Propositional Calculus course with an A+.
Sixteen years later I look back at University and can honestly say that the most useful course, that I _continue_ to have rock solid knowledge of PLUS use almost every single hour of the day was that stupid Philosphy 210 Propositional Calculus course. For the life of me I can't remember much of graph theory, and wouldn't be able to invert a matrix if you held a gun to my head, but, I'll sit in a meeting and see a ton of possible alternative and immediately start applying Disjunction elimination to break us out of a log jam.
It could also be that propositional calculus is dead simple - or at least that has always been my impression of it. Calculus and linear algebra work on complex problems in a large domain. Propositional calculus has a small domain and fairly simple theories.
I think in general, the knowledge domain is important. Linear algebra is a very unified field by itself but when you get courses that involve a grab bag of algebra, linear algebra, trigonometry and calculus, I think the "stew" they present is incredibly hard to comprehend on a high level and students wind-up either failing or learning by a horrible kind of rote.
Uh, this is from biased personal experience. I indeed only taught one college math course when I doing my math MA, it was "Trignometry". It was a complete failure. I still blame the "grab bag" quality of the material but perhaps I was to blame instead.
If you made this story into a blog post, people would listen. (I'm actually serious, it's an interesting story and a lot of people might benefit from hearing more of the details and the thought process behind it.
"If you made this story into a blog post, people would listen."
That's what I hate about people. What makes people pay attention isn't the ideas themselves, but whether it looks like you worked hard in writing it. I've made comments on HN that no one paid any attention to, and then turned them into blog posts and had them get to the top of Reddit or Digg or whatever. Exact same ideas, all I did was visually change the text to make it look more professionally laid out, and carefully rework some of the phrasing to make it sound better to the ear. (Plus add a couple really good first sentences and a good headline.)
I've found that if you actually leave a couple minor mistakes in your blog post it gets more votes, because it look like you're thinking at the edge of your intellectual ability, and people like that. It's the exact same thing Seth said in his post about the Chris Bliss Diss video:
"Today, I got a video, featuring Jason, who just might be the best juggler I have ever seen. Same music, similar routine. Except... five balls. Not three, five. Infinitely more difficult. And Jason makes it look easy.
The thing is, even though I know how much more difficult Jason's routine is and how skilled he is, the very ease of his delivery makes it less likely an audience would give him that same ovation. Interesting how important effort seems to be."
* all I did was visually change the text to make it look more professionally laid out, and carefully rework some of the phrasing to make it sound better to the ear. (Plus add a couple really good first sentences and a good headline.)*
Those changes are non-trivial. Layout and writing style are integral components in the user interface for comprehending text.
There are a lot of hn news posts I've made that I think I could make into blog posts or even articles. But I can't claim they already are in a form people will pay attention to.
For example, the posts I'm most proud of generally get few mods whereas simplistic but irrefutable objections to some other story tend to get the highest mods. But that's logical. People just aren't reading the ten or fifty posts on a paper to find the gems. They might read the first that way if you're lucky. Otherwise, they are more likely to skim. Even on the web, the writer has to present their ideas so they are accessible.
(And here I go again. This post will get far less mods than statements like "where's your evidence")
I like Seth's writing, but I don't like this post. He's missed the point entirely.
In the Chris Bliss video, he's on a stage with all kinds of lights and scenery, performing an original routine in front of a crowd. In the Jason Garfield video, he's copying someone else's routine in a high school gym with no crowd. Sure, Jason's performance may be harder, but Chris Bliss's performance is more interesting. Its not that Jason made it look too easy, its that he made it boring.
Instead the takeaway could be this:
You users don't care if what you did was hard, they care about whether they like the product. And that's as much presentation (i.e. UI) as nuts and bolts.
I'd love to hear about it as soon as possible (i.e., early enough in the semester for me to act on it).
I'm already updating webassign to follow your suggestion, current material + old stuff each week. Anything else you did (beyond SRS) would also be useful to know.
I'm implementing all of btilly's specific suggestions.
As for why not "be creative"? I don't see how that suggestion is helpful. Btilly wasn't creative, he implemented proven techniques. I want to do the same, I just don't know what those proven techniques are (beyond the ones he mentioned in his post).
When I was learning to play the violin, my teacher had a very strict method for going through the repertoire. You had your working piece, which was supposed to be hard - it was stretching the abilities of what you could do as a violinist. You had your polishing piece, which was your previous working piece where you were finessing all the fine points of technique and musicality. You had review, which was all the other pieces you'd learned so far. And you had your preps, which were little passages (4 measures or so) from your working piece that were so hard that you played them over and over again, way slowed down, until you got them right and could speed them up and incorporate them into your working piece.
I've tried to do something similar with my work so far - a working codebase that I'm just learning, a polishing codebase that I basically know my way around in, and various tweaks and bugfixes that I have to do for previous projects. It's a bit harder in the corporate world though - while Google engineers are given a lot of freedom to pick their projects, they're still subject to the needs of the business, and sometimes a project will come up that's such a great opportunity for professional visibility that I'd want to take it even if it involves working hard on something I already know well and dropping the polishing of stuff I just learned.
This remind me of a drawing book called "Drawing on the Right Side of the Brain". It teaches people how to draw realistically using techniques like contour drawing among other stuff.
However, middle school art teachers seem stuck in their little art project. I am not sure about high school, but it doesn't look like they teach the skills.
> It teaches people how to draw realistically using techniques like contour drawing among other stuff.
A great book. One of the exercises is to copy a picture turned 180 degrees, so that you're drawing exactly what you're seeing without the interference of your mind's prior knowledge of the subject.
Not in high school, and not in collegiate art school.
I have a copy that I pass around via samizdat, because the mandatory drawing classes here are useless to students without the basic principles.
(Also worth noting: 'The Memory Book' by Lucas and Loraynne. Now back in print, an falling-apart copy got me through high school. I can still memorize a deck of cards in under ten minutes.)
I'm just about to start lecturing the tutorials for a linear algebra class next week. The prof that is running the course taught it to me 4 years ago and is the reason I'm now in grad school. She did more or less the same thing that you mention, and it definitely works. I remember all of this material from back when I took the class; it's still familiar and I use it all the time.
But despite being very useful for a half to two-thirds of her students, there is always about 1/3 of the class who does terrible in this course. How did you get these people to keep up?
Anyway, I'm glad you posted this. It's a good pick-me-up motivator as I'm about to be trying to do the same thing!
I wish you were my teacher. That's a theory of education I've been pondering about for ages. Glad to see it formalized like this. Please consider running for office and fixing how we go about learning. :)
Actually you should be glad that I'm not your teacher. I set a number of goals for that course around learning, and forgot to set any around making the students happy. I met the goals I set but...
I'm sure that if I had stayed in academia I would have sorted out those kinds of issues. However it is clear to me that I'm much happier not being an academic.
The question of why is there so much crappy education is interesting. I think part of the answer is that society is well-served by the failings of the educational system in that widespread educational crappiness helps support the class system. I think it would be fairly straightforward to turn 90% of the population into well educated upper middle class types, using techniques like btilly describes...
... But if we did that, who would drink Coors and drive forklift for Walmart and not complain about it?
So I think that a huge function of the educational system is to educate a large part of the population badly. I think that the teacher training system serves this is as well, by selecting for mediocre teachers and then making them more mediocre via training. Additionally, in the schools themselves a lot of effort is made to classify young people into dumb and smart categories, usually unfairly (http://en.wikipedia.org/wiki/Pygmalion_effect), and this classification stamps them for the rest of their lives and creates a population of hopeless lower class workers.
If there are SOME good teachers in the system like btilly, then SOME kids go on to get out of their class, which is perfect in that (1) we need to claim that it is possible to pull yourself up by your bootstraps, and (2) we need to recruit SOME kids to grow up to fill management positions, but not too many that they all can't find careers. Historically, underemployed educated people are the people who become union leaders.
The beauty of it is that the people who fail think it is their fault!
(Sorry to rant about the educational / class system in general, rather than the topic of how to teach advanced math, but I couldn't stop myself.)
"But if we did that, who would drink Coors and drive forklift for Walmart and not complain about it?"
Robots? If the education system elevated more of the population, there would be a shortage of blue-collar workers, more pressure for system automation, and more well-educated minds to develop the robots. Of course, there would be no shortage of immigrants to fill the blue-collar positions, but perhaps in the long term, a "better" equilibrium would be reached.
Could you go into more details into SRS for math? "Fact Memorizing" type systems seem to fit well with things like supermemo (languages, law, history, biology, a bunch of sciences), but not so much with things like math (other than memorizing formulas). And did you really add homework questions like find the kernel of this subspace in the first few weeks of the course?
I'm not sure what details to go into. Being reminded of something as we're forgetting it pushes that thing into longer term memory. It also internalizes the lesson. This applies to both rote facts and intellectual ideas. If you think of it as fact memorization, then you're missing the real power.
There are a lot of things you need to learn to master a mathematical topic. You need to learn definitions and terminology. You need to learn key theorems. You should be able to apply procedures and derive formulas. At a higher level you need to learn available proof techniques and the key ideas behind important proofs.
Of course learning these things intellectually is not quite sufficient. You need skills as well. However failing to learn these things will keep you from learning the topic. And that is where most students fall down.
This is particularly true in subjects that build on themselves like mathematics. If you missed an important point earlier, you may develop a bigger misconception about a current topic that will make it impossible to learn a future one. Anyone who has tutored students should be aware of this.
As for what to review, students did not get the same questions over and over again. Instead I gave them questions from the same section over and over again. Since I only asked a few questions from the current day's section, there were always plenty of questions left to go back to for future homework sets. As a bonus doing many kinds of questions at the same time makes connections obvious. When a single homework set asks you to solve a set of equations, find a basis for a vector space, and invert a matrix, you see how they tie together.
You reminded me of learning abstract algebra, in the last year of my college. The subject is extremely abstract, and I try hard to learn the key ideas, without realizing I need do it intellectually, instead of just thinking it more, and reading it with more care. Besides, I could not make any progress on that book, it just got me frustrated. And then I gave up, I hope to continuing my study in grad school, but I could just not get my feet in.
I'm slow, I think, and I can not do well in math. So I just let math get off my life, and trying to be a computing hacker as well as an entrepreneur.
Speaking from my personal experience: you can get very far just by dumping a few hundred questions/examples/small-problems into your favorite SRS. That's so many that your mind, rather than memorize them, will instead learn just how to do them. This works as well for math or CS as for languages.
I've proposed for Mnemosyne 2.0 that there be a card type which is just some Python code which is evaluated: http://groups.google.com/group/mnemosyne-proj-users/browse_t... The idea is that if you want to learn, say, multiplication, you would write some Python code that generates 2 random ints (as the question) and their product (as the answer), and now the user must solve it. You get much the same benefit as if you generated several hundred problems by hand or by script, but without cluttering your deck or risking memorizing some.
I recently heard that newly educated teachers come to schools with a strong desire to teach their specialties. And to transmit some of their true passion over to their students. But since being a teacher (here in Norway) means they have to administrate and organize the kids so much, they have little energy left for teaching. Meaning the teacher appears as a boring, pale fuck (not very inspiring). This is of course a organizational problem, and not a pedagogical one.
Bottom line is that to maximize retention, there's an optimal time for being re-exposed to information: right before you're about to forget it. Seems like re-covering the previous material in the class may have accomplished this.
Hey OP, was your technique influenced by Mihaly Csikszentmihalyi's Flow? I'm referring to the relationship between ability level and challenge. Do I see similarities with your clearly effective teaching style and some of the elements Video Game designers have almost unanimously implemented to great success?
I'm not sure "the relationship between ability level and challenge" is what MC means by flow, if that's what you're saying.
Q: "What is "the flow experience" and what does it have to do with motivation?
A: The flow experience is when a person is completely involved in what he or she is doing, when the concentration is very high, when the person knows moment by moment what the next steps should be, like if you are playing tennis, you know where you want the ball to go, if you are playing a musical instrument you know what notes you want to play, every millisecond, almost. And you get feedback to what you're doing. That is, if you're playing music, you can hear whether what you are trying to do is coming out right or in tennis you see where the ball goes and so on. So there's concentration, clear goals, feedback, there is the feeling that what you can do is more or less in balance with what needs to be done, that is, challenges and skills are pretty much in balance."
(Although it's on the shortlist, I didn't read 'Flow' yet so I may not be able to speak with authority about his work.)
Hey cool, thanks for the link. It's been a long time since I was in school but back then every homework assignment was 100% what we covered that day. So if the "lessons" became unclear many folks just got further and further behind and for many there was no recovering at some point because the teachers never wanted to deviate from their schedule. Btillys 1/3 1/3 1/3 structure really resonates with me because I can imagine I would feel confident and never get in over my head as the complexity of the work escalates. (And if I fell behind I could use his 10 minute improvisational time to get clear) But back to the confidence, Video Games Designers want the player to be challenged but also to feel confident and of course this balance has to remain throughout all levels of the game. Video Game Designers have long used MC's work to help them incorporate this balance. So, yes I do believe the relationship between ability and challenge is part of MC's agenda- there is more, of course. It is my understanding that Video Games are the most common contact most people usually have with MC's work.
I made a connection between the successful feeling of playing video games with the successful learning experiences in Btillys class's and it would be interesting if Btilly is aware of MC's work or if he had ever thought of what he does as MCesque. In closing, I think you and I are saying the same thing here. Thanks for the comment..
It's sad that professors are promoted for research, rather than either research or teaching. It amazes me that the private U.S. universities that are most highly ranked for undergraduate education are the ones with the best research.
Please do a more detailed version, if it makes sense to do so. You actually managed to say a lot in this little space. I had a teacher too who did this on a small scale - the first 5 mins was a summary of the previous calls. The first day of each week he summarized the entire chapter or section in the first 10 mins. This bird's eye view of the material went a long way in keeping the concepts 'live',
That's really awesome. I'm a huge proponent of finding the real, actual insights and lessons behind a subject (plug: I blog at betterexplained.com).
The dirty little secret of education is that it's "ok" to cram and forget. Most classes reward that; education is about "being able to do something once" vs. actually remembering and applying it.
I think a good self-teacher would be better a teacher than who just hold some fancy degrees. Since good self-teachers always know what students need, and where they failed to grasp the essential, and most importantly how to get them out of obstacles.
And I bet you must be a good self-teacher. I wish I have ever had you as my teacher!
The advantage of holding "fancy degrees" is usually that you have been exposed to a wide variety of viewpoints and insignt that self-taught people might have. People tend to study what they are interested in. Unfortunately, this can produce self-taught academics with horse blinders on.
Imaging you only got your news from Reddit & Huffington Post, saying you were becoming a self-taught news commentator. You'd probably end up incredibly informed... about liberal viewpoints, perhaps not realizing that a large portion of the country doesn't share them. Getting taught in a formal education system (providing the system works) would expose you to a wider variety of views.
There is one glaring fault that a self-taught teacher may have; people learn in different ways. What works for one may not work for all. So the self-taught teacher may not realise that their own personal methods are not necessarily valid for everyone.
I'm wondering what type of university you were at that let you write a final. The most flexibility given to grad students in introductory courses I've seen is, at most, to write a quiz or two. Most don't even let you write homeworks.
Actually, at my department (a "top-10" school for CS), for each course we TA, we're expected to write at least one problem set and come up with at least one question for each exam given. There are also half-courses on intro to Unix or intro to C++ that are taught entirely by grad students.
I suspect the OP was teaching an intro-level course, designed to get freshmen up to speed on basic linear algebra, filling any gaps they might have had in their high school curriculum.
I've toyed with the idea of keeping a practice journal, but I've always been too lazy to stick to it. Your comment has inspired me though! This way, I can go back X number of days and review whatever I was working on then.
I also started every class with a question/answer period. The rules were simple, the questioning will last at least 10 minutes, and you don't want me to ask the questions. :-) Anything that had come up in the questions that seemed to be a point of confusion was sure to be added to the next homework set.
I won't go into what else I did with that class, but the end result is worth thinking about. First note that I gave a ridiculously hard final. Other grad students who saw it thought that the class would bomb. Secondly they aced the test. What do I mean by aced? Well I had a bonus question which fellow grad students thought nobody would get. 70% of the class got that question, and a good fraction were over 100% on the test. So they must have studied hard, right? Nope. I ran into some students several months later. They told me that they tried to study for the final and stopped after a few minutes because it was useless, they knew everything. And several months later they still knew much of the material cold!
The thing is that none of what I did was very radical. The principles have been known for a century. Psychologists have been trying to get people to listen for that long. I learned about it in the 80s from a university course I watched on TV. (British Columbia had a TV channel devoted to lectures for correspondence courses.)
Yet, despite how dramatic the effects are, nobody listens and nobody takes advantage of it.