Paper by Drew McDermott
, 1981, part of Mind Design
, which was edited by John Haugeland
In this very good, short and very quotable paper, Drew McDermott
points out a number of common problems in the AI world of the 1970s
(mostly). Considering how familiar many of them sound in 2002, this
was both very necessary and not entirely successful. If only I had
read this before I set out on my own thesis.
His chosen method is humor:
This paper is an effort to ridicule some of these
mistakes. Almost everyone I know should find himself the target at
some point or other; if you don't, you are encouraged to write up
your own favorite fault. The three described here I suffer from
Since this is such a quotable paper, and since it's already quite
short (18 pages) so it's hard to compress further, I will use a lot of
quotes in these notes. Although I did not get permission, I believe
they should still fall under Fair Use laws.
He discusses three main problems, each of which I will touch on now.
This part is about naming things in your program. Call a procedure
"UNDERSTAND", and you're begging the question. What you are programming
is, presumably, some novel idea that might make your
program understand something. Quoth McDermott:
What he should do instead is refer to his main loop as
"G0034", and see if he can convince himself or anyone
else that G0034 implements some part of understanding. Or he could
give it a name that reveals its intrinsic properties, like
NODE-NET-INTERSECTION-FINDER, it being the substance of his theory
that finding intersections in networks of nodes constitutes
understanding. If Quillian (1969) had called his program the
"Teachable Language Node Net Intersection Finder", he would have
saved us some reading (except for those of us fanatic about finding
the part on teachability).
It is important to name your things by what they
do, not by what you hope their
effect will be.
He then goes into a long attack on semantic
nets, of which I can't really give a good account because I'm not
familiar enough with them yet; it certainly reads like they were
everywhere back then.
The attack on the "IS-A" construct is good though; it's overused
everywhere, including in current-day OO hype. "is a" is a complex
concept, and shouldn't be used as name for a link, where what the link
does is more accurately described by something like
This is an argument against using human language for internal
knowledge representation. His main argument is that although there are
certain problems in natural language processing, and there are certain
problems in knowledge representation, these problems are only very
superficially related, and it's nonsensical to try to solve them the
For instance, it's possible to use different phrases to refer to
the same thing in a sentence, and it's also possible to refer to the
same thing in different ways by different roles they play in a
knowledge base. But the similarity is superficial; the process of
finding out that (the internal representations of) "Mary" and "John's
mother" relate to the same person has nothing to with the process of
recognizing that subsequent uses of "the left arm of the chair", then
"the arm", and then "it" refer to the same thing. In speech, the
shorter ways to refer to the thing are merely used to spare time and
breath. The speaker doesn't think of the arm in three different
Speakers will introduce a hand and easily refer to "the finger"; which finger they mean depends on context. Internal representations that automatically pick one finger when they meet "the finger" miss the point.
McDermott argues for using an internal representation that is as
far from English as possible, putting as much of the interpretation
work into the language processor as possible. He believes these
confusions come from the problem of letting different programs "talk"
with each other. Many researchers believe that English is the ideal
notation for this:
They are aware that message-passing channels are
the most frustrating bottleneck through which intelligence must pass,
so they wish their way into the solution: let the modules speak in
human tongues! Let them use metaphor, allusion, hints, polite requests,
pleading, flattery, bribes, and patriotic exhortations to their
With these examples he drives home the point that sending
information is only occasionally the point of human speech. On the
other hand, human speech cannot use the specific powers that computer
communication can; actually transmitting information is pretty much
trivial, since a pointer to the encoded knowledge can be handed around
without further problems. The computer can give any extra information
it may believe useful. Brevity, ambiguity, even understanding through
naming differences (by means of handing around translation tables) are
not problems if human language isn't used.
He then has a short discussion on the superficiality that human
language is often treated with. He first discusses that the difference
between "a" and "the" isn't at all obvious, and hard to define (this
is well known to anyone who has ever conversed in English with a
native speaker of Russian...) He then talks about how English terms
like "a", "the", "all", "or", "and", etc, are often seen as
"embellished or ambiguous versions" of their logical
To cure yourself of this, try examining two pages
of a book for ten-year olds, translating the story as you go into an
internal representation (I found Kenny, 1963, pp. 14-15 useful.) If
you can do this without difficulty, your case is hopeless.
Unfortunately, when I made these notes I didn't write down what all these references referred to; and the book is back to the library. This was some children's book.
Further discussion shows that natural language is also not very
well suited for information queries (database lookups). Similar
arguments apply; information querying is but a very small part of
human language, and it's often very context dependent in a way that
database lookups aren't; and an obsession with natural language
equivalence can lead to database designs that are based around trying
to fit natural sentences, which probably aren't the best possible
"**Only a Preliminary Version of the Program was Actually
This part hurts for me, as this is a flaw I
definitely have. You have a great idea, you implement it, but as you
do that you become aware of more and more flaws in the system, and you
have new ideas about how they could be solved. Two things happen:
first, you believe having found new ideas that could solve the
problems is equivalent to having solved them; and second, you write a
paper that reports on the fixed version and all its neat features,
except it was never actually implemented. I can't help but quote much
of the story from the book:
Having identified a problem, the ambitious researcher
stumbles one day upon a really good idea that neatly solves
several related subproblems of it at once. (Sometimes the solution
actually comes before the problem is identified.) The idea is
formally pretty and seems to mesh smoothly with the way a rational
program ought to think. Let us call it "sidetracking control
structure" for concreteness. The researcher immediately implements
an elegant program embodying automatic sidetracking, with an eye
toward applying it to his original problem. As always,
implementation takes much longer than expected, but matters are
However, as he develops and debugs this piece of code,
he becomes aware that there are several theoretical holes in his
design and that it doesn't work. It doesn't work for good and
respectable reasons, most of them depending on the fact that the
solution to the problem requires more than one good idea. But,
having got a framework, he becomes more and more convinced that
those small but numerous holes are where the good ideas are to
fit. He may even be right.
Here, however, he begins to lose his grip. Implementing
Version I, whose shortcomings are all too obvious, was exhausting;
it made him feel grubby for nothing. (Not at all like the TECO
macros he took time out for along the way!) He feels as though
he's paid his dues; now he can join the theoreticians. What's
more, he should. Implementation details will make his thesis dull.
The people want epistemology.
Simultaneously, he enjoys the contradictory feeling that the
implementation of Version II would be easy. He has reams of notes
on the holes in Version I and how to fill them. When he surveys
them, he feels their master. Though a stranger to the trees, he
can talk with confidence about the forest. Indeed, that is
precisely what he does in his final document. It is full of
allusions to a program he seems to be claiming to have written.
Only in a cautious footnote does he say, "the program was never
actually finished", or, "a preliminary version of the program was
The effects of this are disastrous. Readers believe the results
were pretty good, but as the author doesn't claim this will solve
everything, it probably won't.
Later researchers (like grad students) will be put off by this
research; expanding Version I doesn't seem like a good research
subject, since it seems all the work is already done. It would be
great to make a Version III from Version II, but this doesn't
exist. The last option is to try some other new idea to solve the
original problem. Much the same thing happens.
Eventually there may have been five or six different ways to attack
a problem, but none of them really worked. The conclusion is reached
that the problem will not be solved. Five theses, building on each
other, might have solved the problem! If only the inventor of the
first great idea had left it an open question!
McDermott explains how all research should be based on actual
implementations, and be a thorough report on them. What is needed is a
very clear picture of what was tried, what worked, what didn't, why
didn't that work. And there must be a working program that later
researchers can play with. Later research can build on these
partial solutions, and report the exact improvements
made since the previous version, the improvement in performance,
etc. As McDermott states:
The standard for such research should be a partial
success, but AI as a field is starving for a few carefully
documented failures. Anyone can think of several theses that could
be improved stylistically and substantively by being rephrased as
reports on failures. I can learn more by just being told why a
technique won't work than by being made to read between the lines.
The paper concludes with a small list of some minor points that
also irritate the author, without further explanations. Of note are
"3. The notion that a semantic network is a network." and "4. The
notion that a semantic network is semantic." Man, there was some
controversy going on, I take it :-)
And at last, there's this gem:
Most AI workers are responsible people who are aware of
the pitfalls of a difficult field and produce good work in spite of
them. However, to say anything good about anyone is beyond the scope
of this paper.
Notes written on a rainy Monday afternoon, October 14 2002, with minor editing and adding of links done on December 26.
If you're downvoting this, could you please tell me why? Thanks!