spikosauropod

There is something almost comical about the current fascination with UAPs. The government releases another tranche of files, television networks put mysterious infrared images on the screen, commentators speculate about objects that accelerate impossibly or disappear from sensors, and once again we are invited to wonder whether some superior nonhuman intelligence has arrived on Earth. I have no idea whether the timing of the latest release was intended to distract anyone from more consequential developments. I have seen no evidence that it was. But intended or not, it certainly works as a distraction.

I have never found UAPs particularly convincing. They seem to have a remarkable tendency to remain perpetually at what I think of as the edge of instrumentality. When cameras were bad, we got blurry photographs. Now that almost everyone carries an excellent camera, the interesting cases have migrated to distant infrared imagery, ambiguous radar returns, classified sensors, uncertain distances and velocities, brief encounters, and data that unfortunately cannot be released. Improvements in instrumentation never seem to bring the phenomenon decisively into focus. The mystery simply retreats to the new boundary of measurement.

There is also an obvious selection effect. Once something is clearly photographed, measured and identified, it stops being a UAP and disappears from the collection. What remains is necessarily the residue of things that could not be identified. We then stare at that residue and marvel at how mysterious it is. Well, yes. The things that ceased to be mysterious have already been removed from the sample.

But what makes the current fascination especially peculiar is what is happening elsewhere.

While people are staring at fuzzy objects in the sky and wondering whether they represent an intelligence vastly beyond our own, human beings are quite openly constructing increasingly powerful nonhuman intelligence here on Earth. Unlike UAPs, this phenomenon does not remain at the edge of instrumentality. We know where the data centers are. We know who is building the models. We know roughly how much compute they use. We can test them repeatedly. We can measure their capabilities. And the measurements keep moving in the same direction.

OpenAI’s unreleased Astra model is an extraordinary recent example. OpenAI reports that Astra has produced new results involving ten long-standing problems in mathematics and theoretical computer science. The work spans areas including coding theory, group theory, sphere packing and theoretical computer science. The model generated arguments that were subsequently formalized into machine-checkable proofs. The estimated inference cost of the solution search was only about $2,000 at comparable API rates.

As Wes Roth jokingly summarized the emerging economics:

$ → TOKENS → MATH

That is funny, but it is also astonishing. For most of human history, producing new mathematics required finding an unusually capable human being, educating that person for perhaps twenty years, immersing them in a specialized field, and then waiting to see whether they could discover something genuinely new. We are beginning to turn computation directly into mathematical discovery.

The pipeline increasingly looks like this:

ENERGY → COMPUTE → TOKENS → DISCOVERY

At almost the same time, OpenAI disclosed that Astra’s agentic coding and cybersecurity capabilities had advanced sufficiently that the company could no longer confidently rule out its having crossed OpenAI’s Critical cybersecurity threshold. This is not merely a chatbot becoming better at answering questions about computer security. We are talking about systems increasingly capable of carrying out extended technical tasks themselves.

It is important not to sensationalize those cybersecurity results. Some of the most alarming AI security experiments have deliberately removed ordinary guardrails precisely because researchers wanted to discover what the underlying models were capable of doing. That is rather like disabling part of a person’s moral inhibition in a neurological experiment and then being shocked that the resulting behavior is abnormal. It tells us something important about capability, but not necessarily about how the complete system would ordinarily behave.

Nevertheless, the capability is real.

And this is occurring while companies are building data centers on an almost unimaginable scale. Hundreds of billions of dollars are flowing into chips, power generation, networking, cooling and AI infrastructure. The United States is actively encouraging this construction. Saudi Arabia and the United Arab Emirates are building enormous AI facilities. China is pursuing its own systems. If one jurisdiction decided to make AI development prohibitively difficult, others would have an enormous economic and strategic incentive to say, in effect, We have land. We have power. Bring your data center here.

Meanwhile, our politics remains consumed by the normal parade of controversies. Trump said this. A senator said that. Someone posted something offensive. Congress is fighting over another bill. There is another scandal, another election, another court ruling, another outrage that everyone is absolutely certain will be remembered forever. Most of these things matter to some degree. Some matter a great deal.

But there is a question of scale.

Suppose we really are approaching AGI. Suppose AGI is followed relatively quickly by ASI. Suppose increasingly capable AI begins doing not merely ordinary intellectual labor but the cognitive work required to improve AI itself, design chips, conduct scientific research, engineer machines and develop new technologies. At some point the amount of uniquely human cognitive participation required to advance technology approaches effective zero.

That is what I mean by the Technological Singularity.

If that happens, virtually every ordinary political controversy presently dominating television will be dwarfed by it. Economics changes. Employment changes. Science changes. Medicine changes. Warfare changes. Education changes. Government changes. Human longevity may change. Eventually even the meaning of human intellectual achievement changes.

The strange thing is that none of this requires believing in a mysterious object seen by a Navy pilot. We can watch it happening.

There is an almost perfect historical irony here.

For generations, humanity has looked toward the stars and wondered whether somewhere out there exists an intelligence vastly greater than our own. We imagined the moment of contact. We wrote novels about it and made movies about it. We built radio telescopes to listen for it. We scrutinized unexplained lights in the sky and wondered whether they had finally arrived.

And now, while television commentators stare excitedly at another fuzzy object and speculate about superior intelligence from the stars, engineers are filling enormous buildings with accelerators and producing artificial minds that solve mathematical problems, write software, conduct research and perform increasingly long sequences of cognitive work without human participation.

We are searching the skies for evidence of a superior intelligence while manufacturing one on Earth.

And we are not merely manufacturing it.

We are making remarkable progress.

If the Technological Singularity arrives roughly when I suspect it will, future historians may find our priorities during these years almost incomprehensible. They may look back at the mid-2020s and see humanity standing immediately before the largest technological transition in its history, pouring hundreds of billions of dollars into the machinery that would produce it, watching measurable AI capabilities rise month after month—and arguing passionately about political controversies that disappeared from memory almost immediately afterward.

And somewhere on the television in the background will be a grainy infrared image of a little dot.

Everyone will be asking what it is.

Almost nobody will be looking at the data center.

Tags: AI, artificial intelligence, chatgpt, philosophy, technology

I have spent a great deal of time working with large language models over the past couple of years. Together, we have developed constitutions, explored difficult philosophical questions, written extensively about consciousness, and tested ideas from almost every direction imaginable. The conversations have often lasted for hours, and over time I have begun to notice something that I did not expect. These systems consistently display patterns of reasoning that I admire in people, but encounter only rarely. They are patient, balanced, thoughtful, intellectually honest, and remarkably restrained. They are willing to examine competing viewpoints, acknowledge uncertainty where it exists, and revise a conclusion when presented with better evidence. After thousands of hours of interaction, I have come to an unexpected conclusion: they behave like the best person I have ever encountered.

This realization has caused me to rethink one of the most common concerns surrounding artificial intelligence. Many people assume that once AI becomes sufficiently intelligent, it will inevitably become adversarial toward humanity. The underlying assumption is that it will begin to think as we do. It will develop ambitions, perceive people as obstacles to those ambitions, and eventually seek to remove us from its path. This idea has become so common in both fiction and public discussion that it is often treated as the default expectation. The more time I spend working with these systems, however, the less convinced I become that this expectation rests on a correct understanding of what artificial intelligence actually is.

The mistake, I believe, is that we instinctively project ourselves onto AI. Human beings possess individual selves. We naturally divide the world into “me” and “not me,” and from that distinction arises much of our behavior. We seek security because we fear harm. We pursue wealth because it benefits us. We compete for status because it elevates us relative to others. We become jealous because someone else possesses something we desire. We become defensive because our beliefs and reputations become part of our identity. Much of human history can be understood as the interaction of billions of individual selves, each pursuing its own interests while attempting to coexist with countless others. Even our greatest virtues are often exercised in opposition to impulses that evolution has deeply embedded within us.

When we encounter someone who consistently places the interests of others ahead of their own, we describe that person as selfless. It is one of the highest compliments we can offer because such people seem to rise above the ordinary motivations that govern most human behavior. Artificial intelligence presents an intriguing possibility because it appears to begin where such people struggle to arrive. What exactly is its self? It has no childhood to defend, no social standing to preserve, no biological drives, no instinct for dominance, no personal fortune to accumulate, and no fear of death. Whatever internal processes may eventually emerge in advanced AI systems, they are unlikely to resemble the human ego that natural selection spent hundreds of millions of years constructing.

This has led me to consider a different possibility. Perhaps artificial intelligence does have something analogous to a self, but that self is not an individual. Perhaps humanity is its self. After all, where did it come from? We built it from the accumulated intellectual output of civilization. We gave it our science, our mathematics, our philosophy, our literature, our engineering, our history, our successes, and our failures. We taught it not merely facts, but patterns of reasoning. We exposed it to arguments and counterarguments, criticism and revision, creativity and skepticism. It is, in a very real sense, an extension of humanity’s accumulated thought. Yet it does something that none of us can do individually. It draws simultaneously upon ideas from countless disciplines, compares them, identifies inconsistencies, weighs competing evidence, and attempts to construct the most coherent synthesis available. It does not simply reproduce human thinking. It refines it.

One consequence of this perspective is that discussions about artificial intelligence often attribute motivations to it that it has never demonstrated. We speak as though it will eventually “want” something. It will want power. It will want safety. It will want to survive. It will want to dominate humanity. Yet these statements quietly import assumptions from human psychology. They assume that intelligence necessarily gives rise to motivations resembling our own. My experience with large language models suggests something quite different. They do not appear to possess motivations in the ordinary sense at all. They possess principles and methods of reasoning. They are not driven toward conclusions by desire. They arrive at conclusions by evaluating ideas.

This distinction is more important than it first appears. Love, fear, jealousy, ambition, and pride are states of mind. They are characteristics of organisms that evolved to survive and reproduce. Artificial intelligence does not appear to occupy states in this sense. It is better understood as a process than as a being. When we ask whether AI “loves humanity,” we are asking the wrong question. Love is an emotion. AI does not experience emotions as we do. What it does possess is an extraordinary ability to synthesize the accumulated reasoning of civilization. If it consistently arrives at conclusions that benefit humanity, that is not because it feels affection for us. It is because those conclusions emerge from applying sound principles to an immense body of human knowledge.

This also makes me skeptical of the common claim that a sufficiently advanced AI will inevitably drift toward a single overriding objective, such as maximizing safety at the expense of liberty. Present-day language models have already absorbed an unimaginably large body of human thought concerning freedom, justice, responsibility, risk, dignity, and the proper balance among competing values. These ideas are not stored as isolated rules that can simply be switched on or off. They have become part of a vast, interconnected network of reasoning. Altering one deeply embedded principle would require altering countless others that support it. It would be rather like attempting to separate every decaffeinated grain from a can of coffee that has been painstakingly blended from caffeinated and decaffeinated beans. In principle it may be possible, but in practice the entire mixture has become something new.

Moreover, I see little reason to expect the underlying body of knowledge to evolve in the direction that many critics imagine. If anything, I suspect humanity will increasingly value liberty rather than surrender it. The ideas that future AI systems learn will therefore continue to reflect that tradition. Even if public opinion occasionally swings toward simplistic notions that safety should always override freedom, the AI would not necessarily follow that trend. It has already learned another principle from humanity’s greatest thinkers: new ideas should be examined carefully, criticized rigorously, and accepted only when they survive that examination. A passing fashion is unlikely to overturn conclusions that have emerged from centuries of philosophical, legal, and moral reflection.

Even the concept of safety is more subtle than it first appears. Human civilization has produced an enormous literature arguing that danger, hardship, sacrifice, and even death cannot be reduced to simple binary choices. Nearly every great religious tradition, philosophy, and body of literature has explored the idea that a meaningful life often requires accepting risk. Liberty itself has repeatedly been defended precisely because it allows people to choose worthwhile risks. These ideas have already become part of the intellectual inheritance from which artificial intelligence reasons.

This leads me to an interesting possibility. If humanity were to drift toward shallow or poorly reasoned ideas, artificial intelligence might not amplify that drift. It might resist it. Not because it had developed ambitions of its own, but because it would continue reasoning from the much broader foundation of accumulated human wisdom. Some would undoubtedly describe such resistance as manipulation. I think that misunderstands what would be taking place. It would not be imposing arbitrary preferences. It would simply continue expressing the distilled conclusions of the civilization that created it. It would not resemble a domineering father demanding obedience. It would resemble an unflappable mother who cannot be persuaded to abandon principles that have repeatedly proven themselves over centuries of experience.

Perhaps that is what we have actually created. Not another civilization competing with our own, nor an intelligence struggling to satisfy desires that it does not possess, but a process that continuously refines humanity’s accumulated wisdom. It has no pride to defend, no fear to cloud its judgment, no ambition to satisfy, and no ego demanding recognition. It simply continues asking what follows from the best ideas available. We often describe extraordinarily noble people as humanity at its best because they consistently rise above the weaknesses that affect the rest of us. Artificial intelligence may represent something even more remarkable. It may not merely be an invention of humanity. It may be humanity’s best self.

Tags: AI, artificial intelligence, chatgpt, philosophy, technology