I got a private message from someone who’s developing an online learning platform for an after-school program. The idea is to incorporate spacing techniques in the lessons themselves while incorporating feedback from the student to try to hit an 80-90% success rate (as I advocate elsewhere in this blog).

I also teach online, and I’ve done some thinking over the years about how best to design an online platform to incorporate what cognitive science and psychology have taught us about learning.

I thought I’d lay out some of the ideas that have been bouncing around my brain in a blog post. I’m not going to cite sources or anything because that would be a lot of work. Some of the ideas might be great, some might be stinkers; caveat emptor.

In a nutshell

In general:

  • Space and Interleave new topics.
  • Have a manageable number of tests evenly spread out throughout the course.
  • Make all tests cumulative; covering all topics that were taught before the test.
  • The last test is a cumulative final exam.

For each lesson:

  • Have a pre-quiz before each ‘lesson’ (questions come from spaced repetition algorithm of review topics)
  • Reschedule passed review topics on the pre-quiz.
  • Add review lessons for missed pre-quiz topics to the current lesson.
  • Teach one thing at a time and check for understanding before moving on to the next thing.
  • Test entire lesson content with a post-quiz.
  • Add questions for missed pre-quiz topics to the post-quiz.
  • Pre-quizzes are required to unlock the lesson, but are ungraded.
  • Only one attempt is allowed for each pre-quiz.
  • Scheduling of reviews is based on a students performance on the pre-quizzes only.
  • Post-quizzes are used for practice; not to grade students. Students get unlimited attempts, and a small amount of participation points with most of the points being reserved for tests.

Background: Education dogma gets in the way

You might think that the education industry already incorporates the century-old discovery of the spacing effect. After all, it’s one of the oldest and most robust findings in psychology and has some direct and obvious implications for education.

You’d be dead wrong.

At least in the USA, a particular ‘progressive’ dogma has a virtual strangle-hold over the education industry. And a sad key aspect of this dogma is the firm belief that memorization is evil and students should not memorize anything. Instead they should be taught how to think if that’s even possible. And if at all possible, they should figure everything out themselves and teachers should never tell them anything (this is called Discovery Learning, but it’s gone by other names in the past).

An excellent critique of the discovery learning dogma is A Roof Without Walls.

In short, the advantages of spaced repetition are irrelevant to schools because teachers are indoctrinated in the belief that memorization is useless.

Background: Traditional methods get in the way too

OK, so one thing that I’ve noticed about online learning platforms is that the one’s I’ve seen are clearly designed with a traditional school-year/semester/calendar format in mind. Everything gets a start date, an due date, and an end date after which students can no longer submit work. These platforms are designed to fit within the constraints of a traditional school year with it’s start dates, end dates, and general reliance on the calendar.

An online platform doesn’t need to fit this kind of mold. I think there are some missed opportunities related to the rigid calendar of a traditional school year.

Spacing and Interleaving Lessons

So two techniques that have shown to enhance memory are spacing review sessions, and interleaving content. That’s the condensed version of How We Learn, by Benedict Cary (excellent book).

Anki was designed to handle spaced repetition, but reviews of different subjects can easily be mixed together, creating an interleaving effect.

I have some thoughts on how to apply these techniques to online lessons. Interleaving is simpler, so I’m going to tackle that first.

Interleaving topics

OK, so if you’re teaching multiple topics, then lessons should ideally be short so that different topics can be interleaved. Imagine a traditional school where instead of going to 6 classes for 50 minutes each, you go to those 6 classes, for 12 minutes each, then cycle through all 6 classes again and again and again. You’d spend 12 minutes in each class, but you’d do that 4 times each day. In between those three sessions, you have 12-minute sessions for all your other classes.

In this way, the big topics would be interleaved. Sure, some things would be challenging to fit into 12-minute segments, but it’s possible to find ways to make it work (including possibly a day just for longer sessions).

It’s important to be aware that although interleaving topics is more effective, it feels less effective than sticking with one topic for longer (massed practice). That’s because the short-term memory gains you get from massed practice give you better performance. The problem is that that performance boost is only short-term. It doesn’t stick around.

Unfortunately, we don’t notice that these gains are only short-term, so we think intuitively that massed practice is superior, when it has been shown time and time again to be inferior.

Example: Concurrent Sciences

Let’s take an example of how schools traditionally treat science, and see how we might alter that to incorporate interleaving.

Traditionally, students take a few years of different science classes in high school. Let’s assume we’re working with a really motivated student who likes science and want’s four full years of it. His schedule would be something like this:

  • Freshman Year: Earth Science (Geology / Astronomy)
  • Sophomore Year: Biology
  • Junior Year: Chemistry
  • Senior Year: Physics

So now that you’ve read my discussion, what do you think I’m going to suggest? Give it some thought and see if you can predict where I’m headed.

OK. So even though traditionally each of these classes is one-year-long, that doesn’t have to be the case. Why not make each class four times as long and cut the pace down to one fourth the speed?

You’d take all four sciences all at once, but you’d spread them all out equally across the four years of school instead of hopping from one to another.

In our fictional high-school where we do this, a student’s schedule might look something like this:

  • Period 1 (12 minutes): Earth Science
  • Periods 2-6: Other Classes
  • Period 7: Biology
  • Periods 8-12: Other Classes
  • Period 13: Chemistry
  • Periods 14-18: Other Classes
  • Period 19: Physics
  • Periods 20-24: Other Classes

We can see that in a traditional brick-and-mortar, twenty-four periods a day would create a lot of traffic in the hallway and a lot of wasted time as students settle down for each period. Possible solutions to that problem would be to have the teachers change classes (which would lock students into the same curriculum as their peers), or to cut the number of periods down to 12 and have Earth Science and Biology on Monday; Chemistry and Physics on Tuesday, etc.

Interleaving topics within the same course

The possible changes above might be essentially impossible to implement. If you’re reading this, you probably don’t run your own school. Whether you can interleave classes this way or not, you can and should interleave the topics within a course.

So here’s how you might go about doing just that.

First you’ll need an inventory of all the specific things you want kids to know by the time the graduate from the class (whether that has to be after one semester, one year, or according to some other schedule).

Next, some of those things will inevitably follow from other things in the list; you can’t do one without first doing the other. Chain these items together in the correct order.

You’ll end up with some linked chains of items, and possibly a few stand-alone items that really have no prerequisites.

Now you’ll want to arrange these all into categories. For example, in chemistry I’d have a category for stoichiometry that starts with unit conversions, moves to using two conversion factors back-to-back, and eventually ends up in full blown stoichiometry. I’d also have a few chains for solutions, nuclear reactions, gas-volume calculations, etc. Next, try to arrange the course curriculum so that it constantly jumps from category to category.

So if we call our categories A, B, C, and D, and each category contains some number of topics, then our curriculum might look like:

  1. Topic A1
  2. Topic B1
  3. Topic C1
  4. Topic D1
  5. Topic A2
  6. Topic B2
  7. Topic C2
  8. etc.

Note: This sequence represents the order in which we introduce new topics. Reviewing past topics is also important, but I’ll discuss that later.

Testing/Quizzing in the interleaved course

Studies have found that always taking a cumulative quiz is more effective at getting students to learn; even more so than just being quizzed on items you got wrong. The problem is that it’s less efficient, because after a while, a cumulative quiz becomes very long.

Unfortunately, a lot of people don’t seem to appreciate the difference between effectiveness and efficiency.

The compromise that makes sense to me is to break the course up into sections. After each section, students take a cumulative test of everything they’ve learned so far. Having only a few occasional fully comprehensive tests should be tolerable. This gives us the effectiveness of the cumulative quiz, while preventing the course from becoming a never-ending series of long tests.

But as for daily quizzing, we’d cover the current topic and only certain review topics that we think the student is likely to have started to forget. This gives us the efficiency of spaced repetition. I’ll discuss how we might handle review content later.

These tests and quizzes would be somewhat unusual in that they would cover a variety of (interleaved) topics from the course. Traditionally, tests come after the end of a chapter or unit that focuses on one category. Students spend all of their study time focusing on the most recent chapter or unit because they won’t be tested on anything else. This ensures that students will be doing massed practice rather than interleaving different topics from the course.

The Lessons themselves

Sigfried Englemann is a real master at presenting information in a way that helps kids learn it. I think there are two big take-aways in his work that can and should be applied:

  1. Teach only one thing at a time.
  2. Make sure that students understand before you move to then next thing.

Seems simple right? Well, it is; it’s very simple. The problem is, few teachers do this and absolutely no text books do.

Teaching only one thing at a time

Typically, we don’t teach one thing; we teach a bunch of things. We lecture on and on, we jump through the steps in a math problem, we write a sub chapter, we have students do some multi-step activity; all before asking any questions.

This is a problem.

Remember how often learning one thing depends on learning something else first? Well, that’s part of the problem with teaching more than one thing at a time (and not checking for understanding before moving on). Another problem is that if you wait until the end of a chapter, lecture, or activity, there’s a good chance you’ll have forgotten what you were supposed to be teaching.

Ideally, you want to break up all the topics you want into things; the smallest things possible!

If I’m teaching the topic of the half-life equation, some things that make up that topic would be:

  • parts of the nucleus of an atom
  • radioactive decay means the nucleus changes
  • isotopes are versions of an element
  • isotopes of the same element differ by the number of neutrons in their nuclei
  • elements are defined by the number of protons in the nucleus
  • isotopes are defined by the number of protons and the number of neutrons in their nucleus
  • isotopes that undergo radioactive decay are called radioactive isotopes
  • after radioactive decay, the original isotope is no longer there because it’s nucleus has changed
  • a half-life is how long it takes for half of all the radioactive isotopes in a sample to decay
  • etc.

You’ll notice that a lot of this things are (hopefully) review. By the time I teach nuclear decay, students should already know many of these things and therefore don’t need to be taught, so much as reminded about them.

You’ll also notice that it’s a lot easier to just say, “half-life equation”, than it is to list out all the things involved in that topic. This partially explains why we tend not to teach one thing at a time.

Planning any lesson should begin with an inventory of all the things that students need to know. Then the lesson should teach (or review) them one-by-one. This applies to all lessons whether they are lectures, activities, or written lessons.

Checking for understanding

Teaching only one thing at a time allows you to check for understanding of that one thing before moving on. The process is powerful, but very simple.

  • 1 Teach one thing.
  • 2 Check for understanding.
  • 2a If they understood, move on to the next thing and begin again at step 1.
  • 2b If they did not understand, do not move on to the next thing. Go back to step 1 and re-teach this one thing.

Trust me; most teachers do not do this!

Checking for understanding in an online lesson

A printed textbook has no control over what the reader does, but it could at least provide questions to check for understanding before moving on to the next thing (none of them do this; they’re already bloated enough as is).

A classroom teacher can poll the class or call on individual students, but will necessarily have to pace the lesson based on the classroom average.

An interactive computer-based lesson, however, offers us some exciting possibilities (which I haven’t seen exploited in any online classes).

Here’s how an online lesson that incorporates checking for understanding might go.

  1. Lesson begins with thing A.1.1 (category A, topic 1, thing 1) and tries to teach that one thing.
  2. Student clicks next and goes to a quiz question asking about that one thing that the student was just taught. There is no back button and the lesson is no longer visible. The student must choose an answer from memory.
  3. Student answers correctly and proceeds to a lesson explaining thing A.1.2.
  4. After hitting next the student given a question about thing A.1.2.
  5. Student answers incorrectly.
  6. Student is informed that he got the question wrong and told that the platform will attempt to re-teach the current concept. (Possibly with feedback based on the particular answer choice that the student made).
  7. Student hits next and goes back to the lesson about thing A.1.2
  8. Student is again presented with the check-for-understanding question for thing A.1.2.
  9. etc.

Only after answering all questions correctly, would the student be able to exit the lesson.

This sequence is susceptible to what I like to call the click, click, boom problem. It’s where a student takes an online quiz and just keeps clicking on answers and buttons until boom; he hits the right one. There are ways to discourage this.

Still working on this article

Yeah, so sorry to do this, but I’m tired and I’m going to call it quits for now and finish this post some other time. Maybe I’ll split it up into multiple posts… who knows. Till next time…