Artificial Intelligence meets Natural Stupidity

In this very good, short and very quotable paper, Drew McDermott points out a number of common problems in the AI world of the 1970s (mostly). Considering how familiar many of them sound in 2002, this was both very necessary and not entirely successful. If only I had read this before I set out on my own thesis.

His chosen method is humor:

This paper is an effort to ridicule some of these mistakes. Almost everyone I know should find himself the target at some point or other; if you don't, you are encouraged to write up your own favorite fault. The three described here I suffer from myself.

Since this is such a quotable paper, and since it's already quite short (18 pages) so it's hard to compress further, I will use a lot of quotes in these notes. Although I did not get permission, I believe they should still fall under Fair Use laws.

He discusses three main problems, each of which I will touch on now.

Wishful Mnemonics
This part is about naming things in your program. Call a procedure "UNDERSTAND", and you're begging the question. What you are programming is, presumably, some novel idea that might make your program understand something. Quoth McDermott:

What he should do instead is refer to his main loop as "G0034", and see if he can convince himself or anyone else that G0034 implements some part of understanding. Or he could give it a name that reveals its intrinsic properties, like NODE-NET-INTERSECTION-FINDER, it being the substance of his theory that finding intersections in networks of nodes constitutes understanding. If Quillian (1969) had called his program the "Teachable Language Node Net Intersection Finder", he would have saved us some reading (except for those of us fanatic about finding the part on teachability).

It is important to name your things by what they do, not by what you hope their effect will be.

He then goes into a long attack on semantic nets, of which I can't really give a good account because I'm not familiar enough with them yet; it certainly reads like they were everywhere back then.

The attack on the "IS-A" construct is good though; it's overused everywhere, including in current-day OO hype. "is a" is a complex concept, and shouldn't be used as name for a link, where what the link does is more accurately described by something like "INHERITS-INDICATORS".

Unnatural Language
This is an argument against using human language for internal knowledge representation. His main argument is that although there are certain problems in natural language processing, and there are certain problems in knowledge representation, these problems are only very superficially related, and it's nonsensical to try to solve them the same way.

For instance, it's possible to use different phrases to refer to the same thing in a sentence, and it's also possible to refer to the same thing in different ways by different roles they play in a knowledge base. But the similarity is superficial; the process of finding out that (the internal representations of) "Mary" and "John's mother" relate to the same person has nothing to with the process of recognizing that subsequent uses of "the left arm of the chair", then "the arm", and then "it" refer to the same thing. In speech, the shorter ways to refer to the thing are merely used to spare time and breath. The speaker doesn't think of the arm in three different ways.

Speakers will introduce a hand and easily refer to "the finger"; which finger they mean depends on context. Internal representations that automatically pick one finger when they meet "the finger" miss the point.

McDermott argues for using an internal representation that is as far from English as possible, putting as much of the interpretation work into the language processor as possible. He believes these confusions come from the problem of letting different programs "talk" with each other. Many researchers believe that English is the ideal notation for this:

They are aware that message-passing channels are the most frustrating bottleneck through which intelligence must pass, so they wish their way into the solution: let the modules speak in human tongues! Let them use metaphor, allusion, hints, polite requests, pleading, flattery, bribes, and patriotic exhortations to their fellow modules!

With these examples he drives home the point that sending information is only occasionally the point of human speech. On the other hand, human speech cannot use the specific powers that computer communication can; actually transmitting information is pretty much trivial, since a pointer to the encoded knowledge can be handed around without further problems. The computer can give any extra information it may believe useful. Brevity, ambiguity, even understanding through naming differences (by means of handing around translation tables) are not problems if human language isn't used.

He then has a short discussion on the superficiality that human language is often treated with. He first discusses that the difference between "a" and "the" isn't at all obvious, and hard to define (this is well known to anyone who has ever conversed in English with a native speaker of Russian...) He then talks about how English terms like "a", "the", "all", "or", "and", etc, are often seen as "embellished or ambiguous versions" of their logical "counterparts".

To cure yourself of this, try examining two pages of a book for ten-year olds, translating the story as you go into an internal representation (I found Kenny, 1963, pp. 14-15 useful.) If you can do this without difficulty, your case is hopeless.

Further discussion shows that natural language is also not very well suited for information queries (database lookups). Similar arguments apply; information querying is but a very small part of human language, and it's often very context dependent in a way that database lookups aren't; and an obsession with natural language equivalence can lead to database designs that are based around trying to fit natural sentences, which probably aren't the best possible designs.

"**Only a Preliminary Version of the Program was Actually Implemented"
This part hurts for me, as this is a flaw I definitely have. You have a great idea, you implement it, but as you do that you become aware of more and more flaws in the system, and you have new ideas about how they could be solved. Two things happen: first, you believe having found new ideas that could solve the problems is equivalent to having solved them; and second, you write a paper that reports on the fixed version and all its neat features, except it was never actually implemented. I can't help but quote much of the story from the book:

Having identified a problem, the ambitious researcher stumbles one day upon a really good idea that neatly solves several related subproblems of it at once. (Sometimes the solution actually comes before the problem is identified.) The idea is formally pretty and seems to mesh smoothly with the way a rational program ought to think. Let us call it "sidetracking control structure" for concreteness. The researcher immediately implements an elegant program embodying automatic sidetracking, with an eye toward applying it to his original problem. As always, implementation takes much longer than expected, but matters are basically tidy.

However, as he develops and debugs this piece of code, he becomes aware that there are several theoretical holes in his design and that it doesn't work. It doesn't work for good and respectable reasons, most of them depending on the fact that the solution to the problem requires more than one good idea. But, having got a framework, he becomes more and more convinced that those small but numerous holes are where the good ideas are to fit. He may even be right.

Here, however, he begins to lose his grip. Implementing Version I, whose shortcomings are all too obvious, was exhausting; it made him feel grubby for nothing. (Not at all like the TECO macros he took time out for along the way!) He feels as though he's paid his dues; now he can join the theoreticians. What's more, he should. Implementation details will make his thesis dull. The people want epistemology.

Simultaneously, he enjoys the contradictory feeling that the implementation of Version II would be easy. He has reams of notes on the holes in Version I and how to fill them. When he surveys them, he feels their master. Though a stranger to the trees, he can talk with confidence about the forest. Indeed, that is precisely what he does in his final document. It is full of allusions to a program he seems to be claiming to have written. Only in a cautious footnote does he say, "the program was never actually finished", or, "a preliminary version of the program was actually written".

The effects of this are disastrous. Readers believe the results were pretty good, but as the author doesn't claim this will solve everything, it probably won't.

Later researchers (like grad students) will be put off by this research; expanding Version I doesn't seem like a good research subject, since it seems all the work is already done. It would be great to make a Version III from Version II, but this doesn't exist. The last option is to try some other new idea to solve the original problem. Much the same thing happens.

Eventually there may have been five or six different ways to attack a problem, but none of them really worked. The conclusion is reached that the problem will not be solved. Five theses, building on each other, might have solved the problem! If only the inventor of the first great idea had left it an open question!

McDermott explains how all research should be based on actual implementations, and be a thorough report on them. What is needed is a very clear picture of what was tried, what worked, what didn't, why didn't that work. And there must be a working program that later researchers can play with. Later research can build on these partial solutions, and report the exact improvements made since the previous version, the improvement in performance, etc. As McDermott states:

The standard for such research should be a partial success, but AI as a field is starving for a few carefully documented failures. Anyone can think of several theses that could be improved stylistically and substantively by being rephrased as reports on failures. I can learn more by just being told why a technique won't work than by being made to read between the lines.

The paper concludes with a small list of some minor points that also irritate the author, without further explanations. Of note are "3. The notion that a semantic network is a network." and "4. The notion that a semantic network is semantic." Man, there was some controversy going on, I take it :-)

And at last, there's this gem:

Most AI workers are responsible people who are aware of the pitfalls of a difficult field and produce good work in spite of them. However, to say anything good about anyone is beyond the scope of this paper.

Notes written on a rainy Monday afternoon, October 14 2002, with minor editing and adding of links done on December 26.

If you're downvoting this, could you please tell me why? Thanks!