A couple of weeks ago I wrote a brief post about whether AIs might be considered to have “intrinsic” intentionality: that is, whether their internal states might be originating sources of “aboutness”. The nature of intentionality is something I’ve been stewing over for decades now, and the aspect of it that I’ve been most curious about is its relationship to consciousness. It’s also something I’ve been debating with my friend Bill Vallicella, the Maverick Philosopher for nearly as long, and we’ve just gotten into it again over at his place. I thought I should recap some of the argument here as well.
My own view, which really hasn’t changed in all these years, is that consciousness and intentionality are orthogonal (this is in contrast to the view, held by Bill and others, that it is only in virtue of our consciousness that we can be “Original Meaners”; that there is no meaning or “aboutness” to be found in the merely physical without it having been put in by a conscious mind.
In another recent discussion-thread at Bill’s blog, Bill made a distinction between mere sentience and higher types of consciousness:
I am using ‘sentience’ to refer to the lowest level of consciousness, non-intentional consciousness. But there is more to consciousness than sentience since there is also consciousness-of or what the philosophers call intentionality. This is why there is a use for both ‘sentience’ and its adjectival form and ‘consciousness’ and its adjectival form.
He also had this to say, in a more recent thread:
Yes, a key question is whether “intentionality requires consciousness.” Clearly, every sentient state is a conscious state. You agree, right? You also agree that sentient states are non-intentional states, i.e., states that are not object-directed. You also furthermore agree — I hope — that we cannot just assume that object-directed states are not conscious states. This is something to be investigated.
One view (V1) is that no organism or machine can be in an intentional state without being conscious. This is the view to which I incline. To understand this you must appreciate that consciousness and sentience are not the same inasmuch as sentience is just one mode of consciousness. I am not maintaining that only sentient beings are subjects of intentional states. So I do not argue that because LLMs are not sentient that they are therefore not subjects of intentional states. I deny that they are sentient and I deny that they are ever in intentional states.
Another view (V2) is some machines can be in an intentional state without being conscious, either in the mode of sentience or in any other mode. This, I take it, is the view to which you incline.
I have the impression, Malcolm, that you are assuming that ‘sentient’ and ‘conscious’ are terms with the same extension. I deny that they have the same extension.
Consider a pure spirit such as an angel. Angels lack sense organs. So they cannot be in sentient states. But they think about things, e.g. the square root of 2, and are presumably conscious of themselves thinking about such an object.
In response, I listed some distinctions I thought needed clarifying:
So it seems to me that there are several possible states here (some of which almost certainly exist on some sort of gradient):
1) Wholly unconscious, but capable of sensing/reacting in a purely mechanical sense (a thermostat, for example);
2) “Merely” sentient- capable not only of reaction to what is sensed, but also having subjective experiencing of what is sensed (i.e., it “is like” something to be this entity experiencing this sensation), but incapable of “consciousness-of”, or intentionality (for example, perhaps a worm, or a bee);
3) The state we call “consciousness”, which describes ourselves and perhaps some of the higher animals: everything in 2), plus full-blown “consciousness-of” and the ability to be an originating source of the “derived” intentionality we read into the symbols, gestures, and artifacts we create;
4) Consciousness without sentience – an example of which might be angels (but I wonder! – do we imagine angels to be blind, and deaf? Aren’t the Cherubim and Ophanim described as being covered with eyes?)
5) “Mere” intentionality, i.e. the ability to be an originating source of the “derived” intentionality we read into symbols, language-strings, etc., but otherwise as mechanical as a toaster, and lacking any level of consciousness, including “mere” sentience.
It’s 5) that I have been wondering about as regards AI. If they invent a language to talk to each other, isn’t that what we would normally regard as sophisticated intentionality? And if they can do that without being conscious (as I suspect they already are doing), doesn’t that break the dependence of intentionality on consciousness, and imply that they are in fact orthogonal (as I’ve been inclined to believe for decades now)?
Bill also said this:
Just as you can’t squeeze blood from a stone, you can’t squeeze semantics out of syntactics. So I’m driving toward the conclusion that LLMs are just insanely fast and complex syntactic engines. They merely simulate what you and I do. But they engage in their syntactic operations so well and so fast that they create the illusion that they are subjects of gen-u-ine original intentionality.
To which I replied:
First, I’m not sure that it’s accurate to call LLMs “syntax engines”; they don’t really manipulate symbols, and don’t operate according to rigid rules (because nobody ever wrote any [syntactical] rules for them to operate by!). Syntax engines ignore meaning entirely, whereas LLMs manage to draw out semantic content from fuzzy, broken sentences, slang, etc.
Second, to say they “merely simulate” what we do is, perhaps, using “simulate” in a categorically misleading way. A simulation of a hurricane isn’t a hurricane; it causes no wetness and knocks down no trees. But a simulated chess game, for example, actually is a chess game. It seems your position is that “aboutness” is like a hurricane, in that it can’t really exist in a simulation – but that’s exactly the question at issue here, so you can’t use it as an argument.
My suspicion is that consciousness is what is like the hurricane (in that the simulation isn’t the real thing), while “aboutness” is more like chess: we can play chess, but so can a machine, and it’s still a game of chess regardless.
In sum, then: in the discussions above, Bill acknowledges, by making a distinction between the consciousness involved in “mere” sentience, and “consciousness-of”, that there can be consciousness without intentionality. I’m trying to tease them further apart by saying that there can also be intentionality without consciousness.
What’s interesting about this is that it ties together what are usually two widely divergent models. On the one hand, the materialist, Darwinian account of intentionality, as described at greatest length by the late Daniel Dennett, tells a story in which “aboutness” entered the world in a gradual, evolutionary way, without referring to consciousness at all. An advantage of this model is that it can easily accommodate such things as the “meaning’ in the food-dance of a bee, without having to insist that bees have full-blown “consciousness-of”. And on the other hand, it preserves the “specialness” of human consciousness (which Dennett denies even exists!) while accommodating the idea that AIs really are up to something “meaningful” when they talk to us. It avoids the dilemma of either having to deny that AIs instantiate any genuine meaning at all (which, given their sophisticated, human-like output and obvious intelligence, seems increasingly at odds with reality) or insisting that their behavior means they must in fact be conscious (which I very much doubt they are).
Is any of this important, or is it just arcane fussing over definitions? I’ll say this: I think the question of AI consciousness is going to be a very big deal before long, because on it will hang the question of whether these contraptions have rights. I have perfect confidence that nobody in law or government has the slightest competence to adjudicate that issue — so anything we can do in advance to clarify the matter can only be helpful.