Anki Reset (sometimes it’s necessary)
I’ve written before about SRS. I stated that I had my “misgivings” (a post still unwritten), but that I think it’s a good technology which will eventually become more pervasive. In the meantime it’s very DIY. It’s hard for most of us to like, and it’s easy to get it wrong.
Yes, it’s easy to get wrong. Khatzumoto frequently tells us about some of the mistakes he’s made and how to avoid them, and John Biesnecker has some tips as well. I’d like to share one of mine.
The mistake I made was big enough to destroy my enthusiasm for SRS and Anki (a great program). In fact, I’ve come to the conclusion that the only way forward, short of abandoning SRS as method, is a total Anki reset. Deleting all your SRS data is something you don’t ordinarily want to do (it builds on itself and evolves over time), but in my case I have no choice.
I made two major mistakes:
Mistake #1: Adding word lists
Yeah, this is kind of a newbie mistake, but I wanted to learn lots of obscure country names, so I just entered them all in. Only problem is I never talk or write about those countries in Chinese. I don’t even like politics or geography. I was entering data into Anki, which was dutifully passing it on into a “memory black hole.” And then I kept having to review those names over and over again, and then forgetting them.
Lesson learned: Don’t enter language you’re pretty sure you’ll never need.
Mistake #2: Adding all unfamiliar words in my readings
Around the time I was getting more enthusiastic about Anki, I was also reading a lot more Chinese literature as part of an effort to sophisticate my Chinese. So I added a bunch of semi-archaic vocabulary from Lu Xun stories. Mistake!
The problem was that these were words I would basically only see in writing, and many of them were fairly easy to figure out in context. Driven to totally master that vocabulary, I was trying to force into my active vocabulary quite a few items which really had no business being there. They would have been perfectly fine just chilling in my passive vocabulary, and simply continuing to read more would reinforce them enough.
Lesson learned: Don’t enter language you’re pretty sure you’ll never need.
What I’m doing now
So after learning my lessons, I’ve wiped my Anki data clean. Now the data I enter is vocabulary I can imagine myself actually using. This does wonders for my motivation to use Anki, becomes reinforcing these fun and useful terms puts me that much closer to better speaking ability. Rather than (potentially) improving my reading speed, I’m working on enhancing my human interactions. That is way more motivating.
That’s why I use several lists…
That way you would be able to divide your vocab between recognition and production, so all that obscure Lu Xun stuff would go onto the recognition list…
Also, when words get too obscure, I just suspend them, so they get out of the queue, and I don’t lose the scheduling info for the rest.
That’s one way to do it, but personally I find it too high-maintenance for my situation. For simplicity’s sake, I use Anki for production, and just rely on extensive reading to continually reinforce recognition.
Suspending is a good tool, but I was finding that really, almost everything should be suspended (or deleted)… hence the reset.
I’d definitely second suggestion about having a place for low frequency, “passive memory is just fine” items. I have a model specifically for stuff like that, with recognition only, and it works quite well.
Also, w/r/t the inadvisability of adding all the unknown words in your reading, I would say it depends on what you’re reading, and how much of that or similar reading you plan to do. One thing I’ve found when reading long, dense material is that by adding all of the words that I encountered that I didn’t know I was able to pretty quickly get a grasp on the author’s style, and my reading accelerated the further into the book I went.
Hey John,
Long time reader but this is my first comment. I have been using SRS (Anki, Mnemosyne, Khatzumemo`s new form surusu) for about 3 years.
I have used it for Japanese for several years and have achieved great results as I have been able to read Japanese books newspapers etc without dictionaries for around 2 years. I also now live and work in Japan.
I agree with it not worth adding stuff you will never need. It simply increases your daily review count and can increase boredom. I used to add lots of Japanese sentence with very obscure words and complicated phrases but they definintely do not stick in the mind as much as the phrases and words you envision yourself using on a daily basis.
One thing that must be said about an SRS is its benefits are almost certainly in the long term, when you have accumulated thousands of sentences and begin to be able to read freely. I remember beign at this stage with Japanese – I did not even have to write the sentence or speak it alound as I had read and heard so much I could just hear it in my head and just
knewI could write it by hand.I currently am using Surusu for Mandarin Chinese at the moment. And I am receiving great benefits in terms of reading but frankly very little in terms of speaking at the moment -though I am working to change this.
Anyway, sorry for the long post – keep up the great site.
James
Anki? I thought you were using the pleco flashcards, don’t they do the same? What is the advantage?
I have committed the same mistake that you mention of the Luxun-esque vocabulary. But the problem is it is not always so obvious which words are too rare and whihc are common. If you read a lot, depending on which fields you get into rare words become suddenly common (OK, the problem is I am very curious and I end up reading in many different fields)
Anyway “passive” is the key, I agree. Especially for Chengyus, where I have noticed that even most Chinese know a large part of them only passively.
My method of studying vocabulary is to distinguish between goal-specific and general knowledge. When studying a particular text, I do use word lists and all unknown words from the text, but I study these with simple flashcard software and not an SRS. I also make the distinction between words that are used just once in the same article and those used more than once, focusing more on the latter. I could be more choosy about what to study (as John suggests), but I happen to enjoy memorizing words.
What I put into the SRS file are words that I consider generally useful to retain long-term, independent of where they are found. That’s the whole point of spaced repetition, and it works great for that purpose.
John, if you love Anki so much – why then Anki support within ChinesePod is so poor?
I would like to download all the expansion & lesson sentences of a CPod podcast – audios (zipped) + texts (in Anki input format), so that I can easily reinforce them with Anki!
THAT would be great!
I don’t make all the decisions at ChinesePod, but adding Anki support was my idea.
ChinesePod’s Anki support is actually quite good; what you’re asking for is a matter of what data is directly offered to ChinesePod users.
I look forward to your “misgivings” post.
Learning so-called “trivia” (like country names and whatnot) is not a mis-use of Anki, on the contrary, a system like SRS is probably one of the most effective ways to remember that kind of content. On the other hand, SRS is certainly not as “magical” as the hype sometimes makes it out to be. It still requires time and motivation. In fact, if anything it makes you realize just how many times a fact needs to be repeated before you can remember it for more than even a few minutes!
What I’m saying here is, it’s not a question of how much you need to learn the content, but of how much you want to learn it…as in, do you really, really want to learn it!
I think the point was that you should be really selective with what you put into anki because you can get discouraged quickly.
I couldn’t imagine starting from scratch. It really seems like throwing the baby out with the bathwater. If I was tired of SRSing for a while, then I’d just do something like the the suspend approach. You could try temporarily suspending all items with an interval over say, 3 months, which would reduce a large portion of your daily workload and free up some time for you to learn fresh new stuff, and consequently keep you more motivated.
The good thing about suspending items with a higher interval is, if you do wait longer to review them, the chances of it falling out of your memory when the go overdue is less than for newer less familiar items.
Even if the miss rate is slightly higher, it’s not a big deal because when things fall out of your memory, they don’t usually fall out completely.
Another thing I tend to do is enforce a time limit on any individual flashcard. If I can’t recall it in a specific amount of time, then I go ahead and mark it wrong. It might cost you more work in the short run, but in the long run, I think it will mean time spent because you’ll go through flashcards at a much higher rate.
I use Pleco for my SRS flashcard needs, quite heavily. There’s definitely a phenomena whereby words which you just can’t identify with just get thrown back at you to forget time and time again so I completely identify with what John says here.
The way I choose to deal with that is look for compounds with that character or other words that share a character and I add that, after finding something that’s suitably interesting or useful sounding. That seems to work but of course only if you can easily find such a word, if you’re highly advanced in vocab it might be harder.
I gave Anki a spin on mobile and it’s serviceable enough but since your flashcards end up being based on super terse CC-CEDICT entries, it just doesn’t really hold a candle to using Pleco when the reveal shows a highly detailed commercial dictionary definition complete with example sentences. I quite like this sort of expanded flashcard study.
I tried Anki, thanks to this post, and I like it a lot. This seems to be a great method for me. A friend of mine loves lingt.com, but you can’t import anything. He uses it for HSK, somehow. My only problem with Anki is that the characters and pinyin get switched around on some cards — and the program doesn’t let you fix it (It lets you try, but then makes its own decision about whether to save your changes or not). This is really bad for a beginner like me! Would love to know if there are any other programs out there that might work better.
Carl, if your questions and answers somehow seems to be getting swapped around in Anki, it may be because you have accidently generated a “reverse” card (meaning that you have only entered a single item, but it has produced two flashcards – one “forward” and one “reverse”). Go to Edit -> Browse items and if there are two cards, you can delete the one you don’t want.
I guess I didn’t explain myself well, since that’s not what I meant. The front card, for example, will show the correct characters for yi wan fan, a bowl of rice. On the back however, the pinyin is wan fan yi. Anki has flipped things around, or else they got messed up when exporting/importing from ChinesePod.
If I try to fix this, Anki MAY accept my changes — but it just as likely may not. In this case, I know what is correct, yi wan fan, so there’s no damage to my learning. But with other vocabulary items, I might not know that the pinyin is messed up. That’s what concerns me. So I should probably find another SRS application, since this is the first method that has actually worked for me when studying Chinese.
I agree that this is good advice; I try and avoid adding in lists now, and I’ve actually done a reset once or twice before. Now, my strategy is to write down words I don’t know but have needed in a conversation, and then I add almost exclusively these into the list. Other words come from CPod lessons that I’ve studied; in that case, I’ve seen the word in context before, and so the SRS (in Supermemo) is reinforcing something I encountered elsewhere.
This has been more productive than just dumping stuff into the list. I’m sure everyone reading this has had the experience of seeing the same hated items over and over again. “See you tomorrow, complicated word!”
SRS is only as good as the items you put into it. Difficult material that is not very useful CAN be remembered, but the reward or payoff is not very high. This doesn’t do well in providing motivation for keeping up with any SRS system. Rather than just putting stuff into your SRS because “you need something to do,” think of SRS as “saving your progress” when playing a video game. Learn by watching TV, reading books, basically doing stuff, then take the interesting results and put them into your SRS. Think of it as a “scrapbook of your progress.” Once your collection reflects this, SRS will be much more fun and enjoyable. Adding pictures (Not too many, as adding them takes time) can also help “personalize” your flashcards.
One other big factor is how the SRS algorithm deals with difficult or frequently failed items. As a user of Supermemo, they are referred to as “leeches,” in that they leech your time away and are (scheduled to be) reviewed too frequently. It usually comes down to three choices: 1. Rewrite the item, 2. Reset the learning progress for the item, or 3. Delete the item.
1. Rewrite: If the item is poorly worded you might never get it right, this was commonly a problem for me.
2. Reset: Sometimes the word just won’t “stick,” and the problem is only your own ability to retain a weird-sounding word. After five “fails,” I usually tell Supermemo to reset the item, and I’ll give it another run. Perhaps I’ll forget it again, but given enough time (Perhaps even two or three resets), I will eventually become familiar with the item.
3. Delete: If it is a waste of time and effort, sometimes you have to get rid of it.
Here is an article on leeches: http://www.supermemo.com/help/leech.htm
This is getting pretty in-depth when it comes to “Supermemo theory,” though.
When a majority of your items fit the above criteria for being “leeches,” it is understandable that such seemingly drastic action was taken. I recommend that you check this article out: http://www.supermemo.com/articles/20rules.htm
It applies to all kinds of knowledge, but you can easily adapt the rules to apply to learning vocabulary. If they are strictly followed, an enjoyable time will be had.
I had an interesting experience with SRS techniques. I started studying Chinese on my own, and was a faithful user of iFlash for Mac, which has a spaced repetition program built into it. At that point, it worked great. About 2 years in, I started studying at an intensive program in Beijing where we were learning about 50-100 words per day. I tried to keep up with my old system by putting every new word into iFlash and testing myself at night, but my system eventually broke down. I was getting 300 flash cards thrown at me every day, so I had to increase the length of the intervals. I had to keep increasing them to the point where the whole SRS thing became ineffective. I was simply trying to cram too much vocabulary in my head at once.
One other thought – I liken SRS to juggling. That is, you use SRS to keep all these words in your active memory at once. When I got to Beijing, I felt like I was getting all of these balls tossed my way until there were too many and I dropped all of them.
So I don’t think SRS is the ideal system for an intensive language learning environment. It’s much better done at a more relaxed pace with words you want at your disposal for colloquial situations.
The standard theory does not properly account for the interactions between the items in the deck and those (mostly the same) items seen outside the deck.
The idea seems to be that ren2 e.g. will be promoted out of sight, and so that is not a problem. I think it still is but I don’t have time to restate the whole argument. I’ll skip to the solution.
If you could prove you know everything in an xfold composite object, and could ensure that it would reappear often enough for you to be satisfied that it will remain mastered, then you could (optionally as you feel like it) suspend all of its (x-1)fold or smaller constituents, down to and including the “minimum-information” atoms.
In the extreme, the Lu Xun story (or a pointer to it) would be on a card to be assigned every x months. Your eye would quickly find the low-frequency items which you could confirm or unsuspend as necessary. All of the chars, words, and sentences that the story might have been broken down into could be suspended if desired. (I think they should be suspended if justified.) The multiplier (the aggressiveness of the memory-stretching) should decrease as the card-complexity increases (this is how you “ensure that it reappears often enough”, the bigger the chunk the more conservative), to the point where you have something close to a periodic rereading of whole stories or books.
The general solution is to take a book or set of books (like NPCR e.g.), break them down recursively, activate the minimum-information bits, and then roll-up the bits back into the bigger pieces as that is justified (apply some criterion based on the intervals on all of the constituent bits). Someone will implement this eventually.
Your “misgivings” post could be very difficult, and also unrewarding, I’ve found.
Let me say it another way, with an easy example…
Seeing 人 elsewhere in one’s studies every day makes the interval on that card meaningless; you haven’t proven that you can remember it that long. This one appears to be a “don’t care” case. But other less frequently occurring words get caught in this and receive unjustified promotion too.
Let’s say you’re studying vol 1, and most of those cards get promoted too far because you’re immersed in vol 1.
Months later you’re in vol 4. You aren’t bothered when 人 from vol 1 pops up because you’ve been seeing it. You are bothered by the many low frequency items that were promoted too far that now pop up that you haven’t seen since vol 1. Your testimony about the Lu Xun stories and the stress this caused is a good illustration. Thank you.
I already tried to offer a general solution. Here I’ll give something more specific (with some approximate numbers).
My deck is build mostly from vols 1-8 of a series. It has 6000 cards. 3500 of those are from vols 1-6. Those volumes contain (basically) 100 children’s poems. Vols 7-8 have moved on to real stories and I’m in vol 9. I could justify suspending those 3500 cards (and I argue I should), in let’s say one of these two ways:
1. put those volumes on a rotation to be read periodically, with any words missed unsuspended as needed. With this you exercise the retention and prove it (wholesale) at the same time.
2. let those 100 poems be represented and tested in the hardest mode (for me): audio flashcards. Audio on the front, ensure you understand everything, ensure you can write everything. Don’t forget to place an interval limit on those cards. (I’m already set up to do exactly this.)
In either case, the 3500 could be confidently suspended.
As you said in a later blog post, maybe you were unwise to flashcard the Lu Xun stories. On the other hand, if you really want to preserve your mastery of those, you would put the whole stories on a regular rotation. You would remember most of those lower frequency items in their native context (exactly where you say you’ll probably see them anyway) and can unsuspend the rest as needed.
SRS works great at short intervals, but I suggest that materials be rotated out in favor of “bigger” modes as soon as this is justified (when that is can be argued). Limits on the maximum interval and on the number of “minimum interval” cards are consistent with my argument, and I suspect that the numbers should be surprisingly small, let’s say, 1000 items, 30 days?, 2000 items, 60 days? That would vary for each person. Then your low-frequency items would be kept on a leash.
I developed this point of view after noticing how easily I could read those earlier volumes. Despite believing that I can and should, I haven’t actually suspended those cards yet because I’m a hoarding fanatic like most of you. But, there’s a price to be paid for that (as John testifies) and a reasoned adjustment would be wise. Unfortunately, neither the cause of the problem nor the varied possible solutions to it are discussed much.
Oops, typo. I meant to say “Limits on the maximum interval and on the number of “minimum INFORMATION” cards… (These should be rolled back up into larger blocks, either inside or outside the flashcard program.)
SuperDuper-people, please don’t ask me to show my scientific studies. I’ve tried to understand real situations such as John described and offer some common-sense solutions. The problem does not exist in your world so go and be happy!
Alright, I’ve simplified it further:
You can’t prove you can remember something for a year if you just saw it yesterday. Thus, for the typical student seeing most of his items outside of the flashcard program, the intervals on most of his cards quickly become not logically justified- the easy ones, but also an unknowable number of the hard ones. (Later, randomly missing these, is one cause of the stress and discouragement.)
The only way I can think of to mitigate this is to set a maximum interval limit. (Plecodict- setting, Anki- plugin)
Suspend items as they build up. Let these graduate to a routine of scheduled readings/listenings of real language in which they appear with hundreds of others.
O.K. I’m done now! Thanks.
[…] using an SRS program with great efficacy. But I know there are more people out there like me who suddenly found their reviews weren’t helping, or people who skipped a few weeks of study and found it too intimidating to open up their SRS […]
I don’t know. I have 21 000 words in my Anki deck, yet if I haven’t added lots of new words recently, I only spend about 30 minutes/day maintaining it. I definitely think it’s worth it and I will probably never reset my deck. Also, how do you know which words you will never use? Naturally, you should be very careful with adding random word lists, but that’s kind of obvious. As long as you’re ading useful words, it’s cool.
What I’m trying to say it that it depends on what your goals and what kind of learner you are. Personally, I see my Anki deck as an extremely valuable resource, not as a heavy burden. Naturally, if I started seeing it as something negative, I would have to do something (perhaps something along the lines you suggest here), but I don’t (yet). I understand your argument, I just don’t view it that way myself.
I can’t agree with what you say here. To have basic ability with a language indeed you don’t need word lists and words in readings and all that stuffs, but to really master and be fluent in a language, you have to do it. I am from China; when I did the SAT exam I memorized the whole Barron’s 3500 Word List. I even made an individual sentence out of each and every of them! Sure in daily conversations you use them much less than other words, but, all of that is a part of getting you really imbued in that language and become truly proficient. After that I was able to read whatever English book I wanted, and use all the abstruse words as if they have always been a part of my thinking. My oral, listening, writing and reading abilities all took great leaps forward. Believe me, if you only stop yourself in a basic level you’ll never truly be a language speaker of that language! Unless what you are memorizing is ancient Chinese or some domain-specific words which is totally another business. Otherwise word lists and words from readings obviously help a lot and those are the things you’re gonna use for your whole life!
[…] I’ll give you a personal example. I was reading a Lu Xun story, and it contained a fair amount of vocabulary with which I was unfamiliar. After looking up the new words, I dutifully copied them into Anki (my SRS client of choice). There was a fair amount of vocabulary just from that Lu Xun story. Over time, I found that the Lu Xun vocabulary just wasn’t sticking. The words were semi-archaic, and I had virtually no chance of running into them in my modern daily life in Shanghai. I found they were useful only for reading Lu Xun (or possibly other Chinese literature of that era), and yet I wasn’t spending a lot of time reading that literature. The vocabulary was effectively “clogging up” my SRS review sessions as I had to repeatedly review those words, which meant I had less time to spend on review of more useful vocabulary, and I was rapidly losing motivation to use SRS altogether. When I found myself going a week or more without doing any review at all, I eventually realized that I had effectively killed my review sessions and needed an “Anki Reset.” […]
[…] Anki Reset (sometimes it’s necessary) […]
That is a SERIOUS reset! 😉