Learning about learning from a learning computer

Computer scientists are enabling computers to learn about the real world by reading and parsing massive quantities of text. This article discusses some cutting-edge research.

What I find interesting is the critical importance of human intervention to correct mistaken inferences:

For the first six months, NELL ran unassisted. But the research team noticed that while it did well with most categories and relations, its accuracy on about one-fourth of them trailed well behind. Starting in June, the researchers began scanning each category and relation for about five minutes every two weeks. When they find blatant errors, they label and correct them, putting NELL’s learning engine back on track.

When Dr. Mitchell scanned the “baked goods” category recently, he noticed a clear pattern. NELL was at first quite accurate, easily identifying all kinds of pies, breads, cakes and cookies as baked goods. But things went awry after NELL’s noun-phrase classifier decided “Internet cookies” was a baked good. (Its database related to baked goods or the Internet apparently lacked the knowledge to correct the mistake.)

NELL had read the sentence “I deleted my Internet cookies.” So when it read “I deleted my files,” it decided “files” was probably a baked good, too. “It started this whole avalanche of mistakes,” Dr. Mitchell said. He corrected the Internet cookies error and restarted NELL’s bakery education.

His ideal, Dr. Mitchell said, was a computer system that could learn continuously with no need for human assistance. “We’re not there yet,” he said. “But you and I don’t learn in isolation either.”

The longer such errors go uncorrected, the worse the computer performs because one mistaken inference cascades into more and more mistaken inferences. Untangling those mistaken inferences requires time and consumes valuable processing cycles that would otherwise be used to learn new things.

The implication for learning is obvious: Consuming information is not sufficient because we inevitably draw mistaken inferences from that information. The earlier we can catch and correct an individual’s mistaken beliefs, the more rapidly that individual — human or computer — will learn. Educators and parents should strive to shorten the gap between the development of false beliefs and the correction of those false beliefs. This requires active listening and observation to quickly detect and correct mistakes.

Posted by James on Tuesday, October 05, 2010