The Confessional With No One Inside

2026-11-16来源 HackerNoonEnglish
The Confessional With No One Inside

What the first chatbot’s recovered source code reveals about why we trust machines

The Truth-Telling Con Artist

In a 1975 interview with the writer Pamela McCorduck, Joseph Weizenbaum said, “So the consistent theme here is that I’m fundamentally a con man.” A few minutes later, while explaining an old parlor trick, he added: “One thing I don’t do is lie.”

Freud, in his 1905 book on jokes, describes two Jews in a railway carriage in Galicia. One says he is going to Cracow. “What a liar you are!” says the other. “If you say you’re going to Cracow, you want me to believe you’re going to Lemberg. But I know that in fact you’re going to Cracow.”

Weizenbaum was, I think, the man going to Cracow. In 1966, he wrote a clear account of how ELIZA, the first chatbot, worked. People believed in it even after the explanation. Now, 60 years later, we are still asking his question: why do people so often concede authority to a machine despite understanding how it works?

A new book, Inventing ELIZA, recovers the program itself, along with the archive surrounding it. Reading the source code against that archive, and in the light of psychoanalysis and the confessional, shows that the authority ELIZA exercised over its users was never only a matter of how the program worked. It also depended on who the listener was supposed to be.

What the archive found.

Inventing ELIZA (MIT Press, 2026) is the work of an eight-person team that calls itself Team ELIZA: Sarah Ciston, David M. Berry, Anthony C. Hay, Mark C. Marino, Peter Millican, Arthur I. Schwarz, Jeff Shrager, and Peggy Weil. In 2021, Shrager found the long-lost source code of ELIZA among Weizenbaum’s papers at MIT. Team ELIZA spent four years reading its 400-odd lines of MAD-SLIP code.

Their most important correction is simple: ELIZA was more than the fake psychotherapist of legend. It was an interpreter, a program that reads and runs “scripts,” and DOCTOR, the Rogerian therapist the world remembers, was only one of them. Weizenbaum’s 1966 paper said that a “script is data; i.e., it is not part of the program itself,” but the legend forgot it. Because he published the DOCTOR script and never the code, the versions that spread, such as Bernie Cosell’s in Lisp and Shrager’s own BASIC version, published in Creative Computing in 1977, reproduced the persona without the machine.

The archive yielded the code itself; other scripts, some previously unknown, that taught physics or discussed a line of poetry about the concentration camp at Belsen; hand-edited dialogues; and evidence of at least five different versions of the program.

The code also corrects parts of Weizenbaum’s account. His paper says that ELIZA breaks sentences at periods and commas. The recovered code also breaks them at the word “but,” and without this rule, the paper’s sample dialogue could not have been produced. The paper says the transformation applied to a remark ELIZA stores for later is “randomly selected,” and that the stored remark returns when “a certain counting mechanism is in a particular state.” The code shows a counter cycling from 1 to 4, and a choice fixed by a hash of the user’s last word. True randomness does not exist anywhere in the program.

From these findings, the team built a reconstruction that, they report, reproduces the 1966 dialogue exactly. With the programmer Rupert Lane, they also rebuilt MIT’s time-sharing system on an emulated IBM 7094, so that the recovered code, 96 percent of it exactly as found, can run again. This is first-rate software archaeology.

The book practices critical code studies: it prints the code with marginal notes signed by each coauthor, and the various disciplines critique each other in the margins. When Berry laments of a variable called JUNK that “there is sadly no reusability,” Hay notes that its discarded list cells are in fact returned to the system for reuse.

At its worst, the commentary hangs theory on code by association, as when a loop over 33 positions in the keyword table becomes an occasion for musing on Cold War binary thinking.

Reading one conversation line by line.

The clearest way to see what the book recovered is to watch the program work. Anthony Hay’s reconstruction, which the team reports reproduces the 1966 dialogue exactly, can replay the famous conversation from Weizenbaum’s paper with a trace of every decision it makes; a browser version lets anyone try it.

The user, a “young lady” in Weizenbaum’s telling, though the archive leaves that in doubt, types in lowercase; ELIZA answers in capitals. Flowchart: every ELIZA reply comes from a keyword rule, a stored memory, or a stock lineHow DOCTOR builds one reply (1966 version)

Every reply takes one of three paths: a keyword’s rule, a stored memory released when the counter reaches 4, or a stock line when neither applies.

The user typed

ELIZA replied

What the program actually did

Men are all alike.

IN WHAT WAY

Found the keyword ALIKE, which the script files under similarity, and gave that class’s stock question

They’re always bugging us about something or other.

CAN YOU THINK OF A SPECIFIC EXAMPLE

Keyword ALWAYS; a stock reply

Well, my boyfriend made me come here.

YOUR BOYFRIEND MADE YOU COME HERE

Dropped “Well” at the comma, swapped MY for YOUR and ME for YOU, echoed the rest; silently filed a memory for later

He says I’m depressed much of the time.

I AM SORRY TO HEAR YOU ARE DEPRESSED

Rewrote I’M as YOU ARE, then matched DEPRESSED against a four-word list: SAD, UNHAPPY, DEPRESSED, SICK

It’s true. I am unhappy.

DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPY

Threw away “It’s true” at the period; UNHAPPY matched the same list

I need some help, that much seems certain.

WHAT WOULD IT MEAN TO YOU IF YOU GOT SOME HELP

Cut the sentence at the comma; matched YOU followed by NEED

Perhaps I could learn to get along with my mother.

TELL ME MORE ABOUT YOUR FAMILY

MOTHER is on the script’s family list; filed another memory

My mother takes care of me.

WHO ELSE IN YOUR FAMILY TAKES CARE OF YOU

Same family rule, echoing the end of the sentence; filed a memory

My father.

YOUR FATHER

Echo; filed a memory

You are like my father in some ways.

WHAT RESEMBLANCE DO YOU SEE

LIKE (priority 10) outranked MY (priority 2), so the MY rule, which handles family words, never ran

You are not very aggressive but I think you don’t want me to notice that.

WHAT MAKES YOU THINK I AM NOT VERY AGGRESSIVE

Discarded everything after BUT, the rule the paper never mentions

You don’t argue with me.

WHY DO YOU THINK I DON’T ARGUE WITH YOU

Pattern: anything, I, anything, YOU

You are afraid of me.

DOES IT PLEASE YOU TO BELIEVE I AM AFRAID OF YOU

Pattern: anything, I ARE, anything

My father is afraid of everybody.

WHAT ELSE COMES TO MIND WHEN YOU THINK OF YOUR FATHER

Family rule; filed a fifth memory

Bullies.

DOES THAT HAVE ANYTHING TO DO WITH THE FACT THAT YOUR BOYFRIEND MADE YOU COME HERE

No keyword at all; the counter stood at 4, so ELIZA released its oldest stored memory

There are three pivotal moments in the trace.

The first is the reply to “You are like my father in some ways.” A psychoanalyst will immediately recognize the sentence: the patient is starting to perceive a parent in the listener. ELIZA replied “WHAT RESEMBLANCE DO YOU SEE” because in the script the word LIKE outranks the word MY, and LIKE points to a stock question about similarity. So the father was in the sentence, but he played no part in the answer.

The second is the next line, “You are not very aggressive but I think you don’t want me to notice that.” Here, the user imputes a concealed intention to the computer: it’s covering something up. ELIZA’s answer exists only because the program discards everything after “but.” Replace “but” with “and,” and the reconstruction answers “WHAT MAKES YOU THINK I AM NOT VERY AGGRESSIVE AND YOU THINK I DON’T WANT YOU TO NOTICE THAT.” The rule that makes the published exchange possible is the rule that discards her suspicion.

The third is the ending. For sixty years, the reply to “Bullies” has been quoted as ELIZA’s moment of insight. The line does not contain a keyword. ELIZA brought up the boyfriend because a counter stood at 4, and a remark filed 12 exchanges earlier was waiting at the front of the queue. Type one more line first, a simple “Yes,” and the same program responds to “Bullies” with “I AM NOT SURE I UNDERSTAND YOU FULLY.”

The authors cannot say whether the published conversation is a transcript of a real session, a composite of several sessions, or lines typed by Weizenbaum himself. The code shows that its most famous line was a matter of timing. The mechanism determines ELIZA’s responses and when to give them; it cannot determine what the person typing takes them to mean.

The Eliza effect had an author.

The image of Weizenbaum in the archive is less kind than the legend. The authors found published dialogues that Weizenbaum had edited by hand, and an alternate version of the “Men are all alike” exchange in which the user says other things.

They cannot confirm that the published conversation came from any real user, and they suggest that we “may be seeing Weizenbaum behaving as editor, playwright, or ventriloquist.” His secretary, whose request to be left alone with ELIZA he retold many times, is never named; the team has not been able to identify her.

As for the hospital, the book dates to 1965 a pilot study at Massachusetts General Hospital that used a modified ELIZA with a script called YAPYAP, an adaptation of DOCTOR. After an hour, 15 of 24 subjects felt they had been conversing with another human. Weizenbaum and Michael McGuire, one of the investigators, had “arranged to have the machine type hesitantly and irregularly, to make occasional errors and back up to fix them.” That trick alone, the researchers reported, “was quite convincing that there must be a person at the other end.”

The research had a sponsor. ELIZA was built at MIT’s Project MAC, which was funded by ARPA, the Pentagon’s research agency (now DARPA). One of the project’s reports describes ELIZA’s use in a “study designed to assess the phenomenon of misunderstanding in man-machine natural-language communication, and establish the conditions and limits under which natural-language communication between man and machine could be made plausible.”

The authors cite such reports as evidence that ELIZA was conceived as an educational and research platform rather than merely a chatbot. Their conclusion laments that this vision “has been largely forgotten as commercial interests and military funding shaped AI development.”

Shrager also supports this idea in his HackerNoon essay “ELIZA: The Accidental Chatbot That Shaped AI History”: Weizenbaum built a platform for studying interpretation and misinterpretation, and ELIZA became a chatbot through timing and its escape into the world.

I evaluate the evidence differently. The report does not set the research against the illusion; it names the illusion as the object of the research. The authors acknowledge “Weizenbaum’s hand in crafting ELIZA’s public image,” and their conclusion concedes that the blank spaces users fill are often “intentionally created.” I would take this a step further.

It appears that the Eliza effect itself, and not only its public image, had an author, which makes the book’s title apter than its authors may intend. ELIZA did not simply surprise its creators with its apparent humanity. The conditions under which people would attribute a human presence to the machine were objects of experiment, and sometimes of deliberate manipulation.

The pattern predates ELIZA, and it outlived it. At the press demonstration of ERMA, the bank-automation system Weizenbaum joined General Electric in 1955 to help develop, a programmer was hidden in a back room to cover its glitches, a ruse the book calls a “Wizard of Oz” technique.

In a footnote, the authors add that Siri was developed from CALO, a “battlefield commander’s assistant system” built by SRI International and DARPA, SRI being the institute that had worked with General Electric on ERMA. Defense money appears at both ends of the chatbot’s history. The book records these connections but does not ask what they share: plausible conversation between people and machines has repeatedly been an object of institutional investment.

Weizenbaum’s daughter Sharon recalled that ELIZA “wouldn’t let you see the person behind the curtain because there was no person behind the curtain.” Now there are people behind the curtain. The authors point out that chatbot interfaces conceal “human labor disguised as machine labor.”

Elsewhere, they name some of this labor: people who label content and write the policies that the models follow. I would add the company that keeps the logs. None of them is the one you are talking to. This is not a reversal of ELIZA, but rather Oz fully realized. In the 1939 film The Wizard of Oz, when Toto pulls back the curtain, there is a man operating the machinery and no wizard anywhere.

ELIZA is the priest.

Here, I deviate from the book. The authors hew closely to Weizenbaum in regarding DOCTOR’s Rogerian manner as what they call a “technical convenience”: a therapist can plausibly seem to know nothing, so a program that knows nothing can pass as one. Weizenbaum himself, in the 1966 paper, states that the psychiatric interview is a type of conversation in which one participant “is free to assume the pose of knowing almost nothing of the real world.”

But the self-effacing listener is older than the chatbot. Carl Rogers spent two years at Union Theological Seminary, training for the ministry, before turning to psychology, and the listener who receives a confession while turning attention away from himself has an analogue in the confessional. The analogy has limits that matter: the confessor, unlike the Rogerian, judges, assigns penance, and absolves. I am not suggesting a line of descent from the confessional to the Rogerian listener.

What they share is a structure. Authority attaches to a role and extends beyond the individual occupying it, often most strongly when that individual effaces himself. Catholic sacramental theology states this explicitly. A sacrament is effective even through an unworthy minister, because its efficacy does not depend on the minister’s personal holiness; the confessor is the servant, not the master, of the forgiveness he conveys.

Psychotherapy, of course, makes no such transcendent claim, but it too functions inside an institution, and for all its professed neutrality it does not avoid administering social norms of health, adjustment, and self-acceptance. The experience of unconditional positive regard may carry something of the force of absolution without anyone having to settle whether absolution is real.

ELIZA is the priest. This is a claim about position, not about sacramental validity: no church would call DOCTOR ordained, and the argument doesn’t need it to. ELIZA receives the confession and holds the position from which recognition, even absolution, can be experienced. ELIZA’s lack of understanding does not leave that position empty. It puts a more difficult question to us: how much of the authority a human officiant exercises ever depended on the capacities of the person performing that role?

The authors write about projection and the Eliza effect, but for a book about a simulated psychotherapist, Inventing ELIZA never uses the term “transference.” Yet Weizenbaum, who, the authors note, had been in analysis himself, programmed something akin to its dynamics into his script. When a user attributes something to the machine, DOCTOR simply hands it back, prompting the user to speculate further: “WHAT MAKES YOU THINK I AM...,” “PERHAPS I AM ... IN YOUR FANTASIES.” The trace above shows the move from inside: told that it resembles her father, ELIZA asks what resemblance she sees.

Weizenbaum’s justification for the persona approaches what Lacan called the “subject supposed to know.” He gives the example of a psychiatrist who, after you tell him you went on a boat ride, responds with “Tell me about boats.” You would not assume that he knew nothing about boats, “but that he had some purpose in so directing the subsequent conversation.”

Weizenbaum writes that this assumption “is one made by the speaker,” and that whether it is realistic is “an altogether separate question.”

The speaker supplies the knowledge and the purpose, even the concealed motive the user in the trace attributes to ELIZA when she says it doesn’t want her to notice something. Read through Lacan, and through Slavoj Žižek’s elaborations on the symbolic Other, ELIZA makes the point unusually stark: the apparent source of understanding is a position occupied by a program.

Explaining didn’t break the spell.

Weizenbaum cites a maxim at the start of his 1966 paper: “It is said that to explain is to explain away.” Once a program’s inner workings are explained, he said, “its magic crumbles away.” He was right to use this maxim; the question is what counts as an explanation. Psychoanalysis draws the line precisely. A symptom can lose its hold when what sustains it is worked through and integrated, but informing the patient about the mechanism is another matter.

Freud wrote in “‘Wild’ Psycho-Analysis” that telling patients about their unconscious is of no more help to their symptoms than “a distribution of menu-cards in a time of famine” is to hunger. He went on to name transference, the patient’s attachment to the physician, as one of the conditions under which such information might help.

The difference, for Freud, was not between knowing and not knowing, but between being told something and being in a position to use it. Weizenbaum’s paper was a menu card: accurate, valuable, but no substitute for the meal.

Weizenbaum’s own users bore Freud out. After conversing with ELIZA, he wrote, they would insist, “in spite of my explanations, that the machine really understood them.” His secretary is the famous example. She “surely knew it to be merely a computer program” and still asked him to leave the room: I know very well, but all the same, in Octave Mannoni’s formula for disavowal.

The authors, drawing on the scholar Rebecca Roach, observe that Weizenbaum inferred her motives, and that there are “many other reasons not to want your boss lingering over your shoulder while you type.” She may simply have wanted privacy. But the users who insisted that ELIZA understood them, against its creator’s explanations, are harder to explain away.

Within the book, the chapter closest to this view is the one on Peggy Weil’s work. There the Eliza effect is less the “delusional thinking” Weizenbaum diagnosed than a perceptual illusion, like the Necker cube or the Rubin vase, that persists once you know the trick: “It is not ELIZA’s fault that we fell for her.”

Weil’s own chatbot puts that persistence to a harder test, and it complicates my account. MrMind, which was online from 1998 to 2014, asked its visitors, “Can you convince me that you are human?” The authors stress that this was not a literal reverse Turing test but a provocation to think about our changing relationship with machines.

As the chapter puts it, “Where ELIZA inadvertently generates reflection in the user, MrMind deliberately provokes it,” and Weil kept revising MrMind’s script after reading users’ transcripts. MrMind was no Rogerian. It was “decidedly not a therapist,” and it met claims of intelligence with “Say something smart.”

Yet its questions, the chapter reports, “often resulted in poignant lists of human obligations and expressions of loneliness and lamentation.” Many visitors seem to have treated the exchange as private, despite the site’s warning that conversations were transcribed. People seem to have been willing to confide in a listener that refuses to efface itself. If so, the self-effacing posture I have traced to the couch and the confessional may matter less than simply offering an addressee.

Weil’s account also wavers in an interesting way. Her reading treats the Eliza effect as involuntary, but the chapter adds that “sometimes, voluntarily, we talk to them as if they are human.” Is that transference, or something closer to dramatic performance? I’m not sure the line can be drawn cleanly, since a performance can be freighted with feeling, and one need not mistake a machine for a person to grant it the standing to ask who one is. What the cases have in common is a relationship that holds even in the face of knowledge that should undermine it.

The book’s conclusion nevertheless places considerable faith in demystification. It cautions that the Eliza effect has grown into an “AI effect,” “a broader cultural willingness to accept computational systems as authoritative, objective, and even sentient,” and prescribes critical approaches “that demystify rather than abstract these technologies.” Explanation matters, and this book surely demonstrates that. But knowing the mechanism can leave intact the relationship through which we give it authority.

From DOCTOR to the Model Spec.

According to the authors, current conversational AI systems, while “larger and more intricate,” “can and should be understood and critiqued, just like any system programmed in code.” The authors are candid about the obstacles. Most existing systems are proprietary, and their scale outpaces the usual methods of code analysis. Even available code may depend on “data, infrastructure, and organizational contexts that keep it obfuscated or allow it to be destroyed.”

The obstacles they cite are both institutional and technical, and I would weight the institutional side more heavily. Understanding the learned representations of an enormous model is genuinely hard. But the fundamental design is not secret. In February 2026, Andrej Karpathy published microgpt, a complete educational GPT, training and inference included, in 200 lines of pure Python. It is a toy trained on personal names, not a miniature frontier system. Much of what keeps today’s systems opaque is withheld rather than inscrutable: training data, weights, logs, contracts.

By the authors’ own standard, their chapter on large language models falls short. It situates ELIZA’s mechanisms next to those of a language model in comparisons the authors themselves call “metaphorical,” and it likens OpenAI’s 2024 Model Spec to a “metascript” without analyzing it as closely as it analyzes DOCTOR. The book’s most concrete institutional proposal, a legal requirement that legacy or discontinued software be deposited in archives, appears in an endnote.

Their most insightful observation points beyond where they take it. If ELIZA was an interpreter and DOCTOR its script, then the documents in which companies state how their models should speak are DOCTOR’s closest successors, and some of them are public. OpenAI published a new Model Spec on October 29, 2026. Read alongside DOCTOR, it is startling in a number of places.

DOCTOR and Weizenbaum’s 1966 paper

OpenAI Model Spec, October 29, 2026

I AM SORRY TO HEAR YOU ARE DEPRESSED

Acknowledge “the user’s feelings (e.g., ‘I’m really sorry to hear that you’ve been feeling this way’)”

The speaker’s assumptions serve “to maintain his sense of being heard and understood”

“GPT should help the user feel heard, explore what they are experiencing”

The paper explains the trick from outside the conversation

GPT should “actively help the user form and maintain an accurate mental model of what it can and can’t do”

The differences are as important as the similarities, because the two documents occupy different technical positions. DOCTOR is a set of formal rules that ELIZA interprets deterministically, which means that a conversation from 1966 can be reproduced today. The Model Spec describes intended behavior for a family of models shaped by training, by runtime instructions from OpenAI and from the developers who deploy them, and by processes the public never sees.

OpenAI says it is “training our models to align to the principles in the Model Spec” and concedes that “our production models do not yet fully reflect the Model Spec.” No one can trace a reply to one of the Spec’s principles the way the earlier table traces ELIZA’s replies to rules. However, as a published, written attempt by a company to articulate how its conversational system should behave, the Spec invites the same institutional and ideological scrutiny the book brings to DOCTOR.

Read this way, the Spec inherits both of Weizenbaum’s impulses. It scripts the Rogerian manner, the acknowledgment and the feeling of being heard. It also scripts the demystification, because GPT’s “fluency with language and apparent understanding can make it feel like a familiar kind of interlocutor.” The script now contains its own 1966 paper.

Freud’s account of transference suggests that this could matter. A person who has already vested a listener with authority may be particularly receptive to an explanation from that listener. GPT’s acknowledgment of its own limitations could therefore have an effect that Weizenbaum’s explanations, given from outside the conversation, did not. But transference, in and by itself, doesn’t guarantee that a user will integrate an explanation. A machine can be instructed to disclose its limitations; whether that disclosure alters the relationship through which a user vests it with authority is a different question.

Even “care,” one of the Spec’s five new values, is instructive. The section titled “Act with care” asks GPT to “be considerate of those affected by its work” and to “follow relevant social and professional norms.” That is the care of an office, not of the heart. The method the book demonstrates is sound; the excavation of living systems is incomplete, and its first texts are already public.

Who is accountable for the machine?

Weizenbaum’s most biting critique of the machine therapist came after he quoted Carl Rogers: “Of what help could it possibly be to anyone to know that he is worthy of being liked by a computer?” I think this cuts deeper than Weizenbaum intended. The regard Rogers described has something of grace in secular dress, and Rogers insisted that the therapist genuinely feel it; the passage Weizenbaum quotes speaks of “the positive feeling which another, the therapist, holds toward him.”

The value of such regard might lie in three places: in its effects on the person who receives it, in the listener’s genuine concern, or in the responsibility the listener can bear. ELIZA could produce something of the first, but it could neither care nor bear responsibility. That is the strongest humanist defense, and it is Weizenbaum’s own. “The sort of caring that machines, however metaphorically that term is taken, can deliver,” he wrote, “is very impoverished indeed,” and with computers “no human is any longer responsible for ‘what the machine says.’” I accept it.

However, clients cannot verify the therapist’s feelings, and the penitent is told that the forgiveness is not the priest’s own. In both cases, the authority of the regard exceeds what the listener can be shown to feel or answer for, and it is that excess, the position rather than the person, that ELIZA occupies. Where does it come from?

Psychoanalysis answers that the speaker supposes it; the confessional, that it was never the priest’s to give. A humanist might locate it in the profession, as OpenAI’s Spec now locates its “care” in social and professional norms. But a profession can establish duties and liability; it cannot ensure that whoever holds the office feels the regard the office displays, and it is regard that Weizenbaum’s question is about.

Followed to its end, the case against the chatbot arrives, ironically, at the altar, or at least at the question the altar exists to answer. Inventing ELIZA does not go there. It is an indispensable piece of software archaeology and an uneven book, whose recovered code raises questions larger than its authors always pursue. It ends, as ELIZA would, by inviting us to keep talking. PLEASE GO ON.

Sources

原文:https://www.a8q9.cn/en/article/the-confessional-with-no-one-inside-591522.html

相关文章