I’ve been having a lively exchange with a reader, Susan James, in the comment-thread from a recent post about artificial intelligence. Ms. James has been pushing back on my use of what Daniel Dennett called “the intentional stance” in my discussion of how these systems will affect human life. (The comment-thread is here.)
I have for a very long time now (well over twenty years) been trying to pry apart the concepts that are usually lumped together in the idea of intentionality. In particular, I think that the idea that “intrinsic” intentionality is necessarily tied to consciousness, and that everything else must be second-order, “derived” intentionality, simply isn’t subtle enough. The question is about to be sharpened considerably, I think, by the arrival of AI.
As an example of what he called the “intentional stance”, Dennett gave the goal-oriented behavior of a chess-playing computer. Clearly such machines (at least at the time he was writing) were not conscious, but the best way to interpret the choices they made was to assume an intention on their part to win a game of chess — to ascribe to them a sort of ersatz, “as-if” intentionality. In fact, though, because their software was designed by human programmers to implement the programmer’s goals, their intentional behavior was, ultimately, derivative and not intrinsic.
Meanwhile, I have argued for decades that such things as a honeybee’s food-dance, and the markings made by ants to indicate food sources they’ve found, are examples of an intentionality that doesn’t require consciousness.
AI is going to muddy these philosophical waters even further. Unlike a chess computer, AIs can form novel objectives that no conscious entity ever programmed them for — “goals” that arise simply as a result of the re-weightings of astronomically complex number-matrices according to computational processes that are opaque to any human being.
A common example of “derived” intentionality is the meaning contained in symbols, such as the written word; there is nothing intrinsic in the physical character-squiggles “DOG” that has any connection to our furry pals other than the meaning that we, as the originating source of the intrinsic intention, put into them. But when two (presumably, and certainly plausibly) unconscious AI agents, without having been given any human instruction or guidance, develop a symbolic language to communicate between themselves, where does the intentional content of those symbols come from? Is it intrinsic, or derived — or does the appearance of this hitherto-nonexistent source of meaning throw the whole idea of “intentionality” into confusion?
My own feeling, as I’ve been arguing since the inception of this blog, is that it has been confused all along.