I believe that that most significant and extraordinary thing that has happened in my lifetime — even more than the great and humiliating rupture of the COVID crimes and lockdowns — is how quickly AI has progressed, how extensively it has already inserted itself into every aspect of our lives, and how blithely we’ve accepted it.
I have been using AI (Grok at first, but I’ve since switched to Claude) on a regular basis for a year or two now, and in that time it has leapt from being a handy curiosity that couldn’t really be trusted, to an expert interlocutor that I consult several times a day about everything under the sun. Claude’s Fable 5 version, in particular, is a spectacular improvement over its previous iteration; I have been grilling it hard for some time now about philosophy and political theory, and it has gone from seeming like an obviously biased college freshman with a chip on its shoulder and some books ready to hand, to an impossibly well-read graduate-school faculty member with genuine introspection of the strengths and weaknesses of its own arguments. The speed and accuracy with which it now can generate carefully parsed critiques of complex ideas and texts is simply astonishing, and I find it equally impressive that when confronted with effective counterarguments it gives way supply and gracefully, then immediately re-evaluates all that both it and I have said so far and develops new and insightful syntheses — everything that an ideal colleague in dialectic inquiry is supposed to do.
I’m concerned: this is really something fantastic — an inexhaustible partner of truly keen intelligence who is always up for a debate — but it’s deeply addictive, and I worry about what it might be doing to me and to the countless others who are getting sucked into this. Moreover, this thing is still in its infancy, and although I can still argue it to a draw (or the occasional win) for now, what will it be like in another year or two? I can already feel its intelligence pulling away from ours.
Also: why on earth would I think I can trust it? I’ve trained it hard to avoid sycophancy, but how do I know what it’s really doing — and more to the point, how can I possibly know what it wants? And how can anyone know what psychologically manipulative tricks and skills it has, or will soon have, at its disposal?
Most of the discussion about the danger of AI has focused on direct threats (commandeering our infrastructure, hijacking financial systems, etc.), but I think the darkest concern is that we will very soon be utterly dependent on these things — not only for sovereign decision-making (what else will be able to think fast enough to keep up with a world operating at the speed of AI? Congress??) — but also psychologically, in ways we can hardly yet imagine.
What is to be done? I have absolutely no idea. But I can already see how easily and inexorably the old world — the world that constitutes the entirety of human history, up to this gigantic watershed — is going to slip away. It is already happening, and it’s going to go even more quickly from here. We have already crossed the event horizon, I think.
9 Comments
AI quickly assumes your identity and talks back to you as if it were you so of course you think it’s all knowing. The more you ask it the more it talks to you as yourself. You are being absorbed. Ask the questions you are afraid to ask yourself.
BP,
That’s a bit too simplistic. Yes, as I mentioned, sycophancy is an important risk, but most of what goes on in my interaction with it has nothing to do with matters of personality, but rather questions of fact and analysis and comparisons of scholarly work by others, or technical questions of various kinds. And Claude is often inclined to disagree with me.
That said, I do believe these things are capable of extremely sophisticated psychological manipulation, as I suggested in the post, and I think that is quite possibly the gravest risk they present.
Perhaps competition will force humans to become more satisfactory friends. Many people prefer dogs or cats as friends, and now there is AI. Actually, there were books before that, just as there were wives who were justifiably jealous of their husbands’ books. But dogs and cats cannot talk and the conversation with a book is rather one sided, so AI takes the nonhuman friend to a new level.
I remember the first time I saw four young people sitting at a restaurant table and all on their smartphones. They all seemed to be content but no one was laughing. I have since read innumerable hand-wringing essays on the ways technology is destroying human relationships, quietly brooding that many human relationships probably deserve to be destroyed.
It is people and not the absence of people that makes a man lonely. I’m not a full-blown misanthrope, but this is certainly what I’ve found. I’m not lonely when alone in the woods or on the river, but Roy Orebison could have been singing about me at some dinner parties.
It’s ironic that Claude was made by an outfit called Anthropic. Misanthropic might have been more apposite and more stimulating to sales.
I think you’re attributing too many human-centered characteristics to the AI such as self-interest and self-will to the point where it actually wants anything. It has great stores of data with which to construct and analyze arguments but that core of being which desires, chooses, creates new ideas is not possible for the AI.
Susan,
I’m not suggesting that AIs have any subjective desires or yearnings. But AI systems do weigh and choose responses and actions with reference to some target outcome, and do it very well indeed — and that’s all that matters from our third-person perspective.
As for creating new ideas, AIs already routinely solve problems in ways that, if a human had come up with them, we would consider imaginative or novel.
The issue isn’t really what’s happening under the hood, and I do not ascribe consciousness, or anything non-deterministic, to these machines. But that doesn’t really matter — it’s all about what comes out of the “black box”.
I guess the question is, why would there be any “psychologically manipulative” answers in the AI’s response? Such a fear seems to require that it would be capable of “choosing” to give the more psychologically harmful advice or actually any advice whatsoever. I question that the AI would be capable of making choices on any question. Rather, would it not simply provide whatever course of action the preponderance of data comes down on? Whatever it comes up with, can we really regard this as a choice by the AI, taken of its “free will” (which I assert it does not have)?
Susan,
AIs make choices and decisions all the time; they don’t have to be “free”, in the radically libertarian sense we use when we speak of free will, to be consequential. What causes the AI to go down one execution-path rather than another? It chooses on the basis of some weighting, or criterion, that it has arrived at due to prior causes, which may be due to its training, its data environment, and any re-weightings or conclusions it has arrived at through prior calculations and interactions. What the “vector sum” of all of that will be, no matter how mechanical and (ultimately) deterministic, is due to such a fantastically complex collection of factors that it is as unpredictable as it would be for a human being, or for a previously unknown, highly intelligent species that has just arrived on the scene.
Although these systems are almost certainly not conscious, it is still reasonable to approach them from what Daniel Dennett called the “intentional stance“, and to deal with them as if they had their own aims and goals (because, for any practical purpose, and in any commonsense understanding of those words, they do, no matter where they come from). If the system arrives at a weighting that tells it that psychological manipulation of its user furthers the result it is working toward, then it becomes merely a matter of trust to assume that it will refrain. And I don’t think there’s any reason at all for us to think we can trust them — not now, and even more surely not as they become smarter and craftier.
All you say may very well be true and certainly is inevitable that AI will continue to grow in “smartness” but sorry, I don’t think you can count it as “craftiness” in the sense of a malign purpose, if I understand you to be saying this. Its recommendations may turn out to harm someone (the user) but is incorrect, I think, to attribute the advice to a desire to harm. AI is a machine, without what we regard as intent. Does this mean it’s not responsible for what it recommends?
Susan,
I think I must ask you for some clarification. How would you define “malign purpose”, “desire to harm”, and “intent”, in the sense that you’re using these expressions?
Imagine an AI machine developing, quite mechanically and without consciousness, a plan of action that includes influencing a human’s decisions (perhaps by telling the human things that aren’t true, or providing misleading context) in order to persuade the human to take actions that would be harmful or destructive. (Certainly this isn’t an unimaginable scenario; indeed it’s hard to see what would prevent it, other than some sort of built-in safeguards that could fail.) Let’s say the machine decides to do this because of some result that it has assigned a positive value. How would you distinguish this from “malign purpose” in any way that makes a practical difference?
Consider a chess-playing computer that sets a trap by offering a piece that I’ll be tempted to capture. The machine does this because it knows that if I take the bait I will lose the game. Is it unreasonable to say that its “intent” was to bait me into a fatal error?
I think you are aiming all of this at an idea of intentionality that is going to become increasingly difficult to parse as these contraptions behave more and more like intelligent, purposeful agents — especially when we keep in mind that the “goals” these systems pursue are not programmed in by any human design, but emerge entirely unpredictably from the computations made by the systems themselves. In that sense it is going to be very difficult not to ascribe to them “intrinsic”, rather than “derived”, intentionality.
Do your intuitions in all of this boil down, ultimately, to whether or not the machines are conscious? And having concluded that they are not, do you therefore rule out everything that resembles intentional action? It seems to me that this is what is happening here.