In mid-June, at the Main Astronomical Observatory of the National Academy of Sciences of Ukraine and at the Kyiv-Mohyla Academy, American cosmologist Paul Sutter delivered a lecture on the role of artificial intelligence in contemporary science and proposed examining fourteen centuries of alchemical practices for principles on how to engage with it.
Science on the threshold of the unknown
Contemporary large language models, such as ChatGPT, Claude, or Gemini, are capable of achieving a 95% score on a graduate-level physics qualifying examination. The algorithm addresses the same problems that young scientists often wrestle with over several months within a matter of minutes. This development prompts reflection on future prospects, considering that machines are already performing at the level of a novice researcher.
There is currently no consensus among scientists. Some are convinced that within ten years, scientific endeavors may proceed independently of human involvement. Existing agent-based systems already possess the capability to formulate hypotheses, analyze data, generate analytical code, prepare scholarly articles, and even coordinate their peer review process through other agents. From initial hypothesis to publication, this process can be completed in approximately fifteen minutes. While it is acknowledged that the quality of these articles may not yet be exemplary, advocates of this methodology are confident that significant advancements will occur within the next five to ten years.
Other researchers adopt an opposing stance and categorically refuse to establish any contact with language models, prohibiting their use even within their own research groups.
Cosmologist Paul Sutter of Johns Hopkins University, who also serves as an external advisor to NASA’s Innovative Advanced Concepts program, has been investigating the influence of generative AI on scientific methodology with his team for more than a year. Paul Sutter acknowledges that when discussions concerning whether artificial intelligence will benefit or harm humanity become overly intense, his perspective as a cosmologist enables him to maintain clarity. He observes the matter objectively, as detachment often renders issues less formidable.
From this perspective, he articulated the most concise statement: that artificial intelligence is a potent system that remains incompletely comprehended by us. Its potency is evidenced by its ongoing transformation of culture, science, and daily life. However, we lack a complete understanding of the manner in which these models generate their responses; we are unable to trace the algorithmic pathways from inquiry to conclusion, and we do not fully grasp all the implications of our creations.
Fourteen centuries of inquiry
However, this is far from the initial instance in which individuals have achieved mastery over something without fully comprehending all subsequent implications. This phenomenon was observed with fire and stone axes. Likewise, in the domain of astronomy, the introduction of photographic plates and photomultipliers revolutionized the field in unforeseen ways. Additionally, there exists another precedent that persisted for fourteen centuries.
Alchemy originated in approximately 300 CE in Alexandria, subsequently disseminating throughout the Mediterranean and the Middle East, and extending to India and China. Its practice persisted until approximately 1700. Contemporary culture often trivializes it as a mere pseudoscience akin to something from Harry Potter, involving eccentric attempts to transmute lead into gold. However, the historical reality is considerably more intricate.
Alchemists maintained an operational theory of nature. They were aware of the seven metals and held the belief that one metal could be transmuted into another. In the absence of quantum mechanics, the periodic table, and modern chemistry, this hypothesis was entirely logical. They engaged in experimental work with substances at a fundamental level within laboratories, observing phenomena that defied explanation. For instance, when heated, a metal would ignite with a luminous flame, the underlying cause of which was not understood. Elements would evaporate, deposit on the vessel walls, and subsequently revert to their original state. Alchemists were grappling with a system that was potent yet beyond complete comprehension.
Sutter does not assert that artificial intelligence is akin to alchemy. However, he recognizes that this pre-scientific investigation of nature offers valuable insights for individuals confronting a system with which they can interact but which remains not entirely comprehensible.
Three lessons from alchemists for contemporary science
The initial concern regarding language models, as emphasized by Sutter, pertains to their capacity to provide immediate expertise. For instance, if an astrophysicist is required, the model will assume the role of an astrophysicist. Similarly, if a therapist or a lawyer is needed, it will act accordingly. This functionality already demonstrates practical applications, including image recognition in astronomy, cancer diagnosis, and the analysis of satellite imagery to forecast crop yields.
However, beneath this capability resides a fundamental flaw. The model is incapable of differentiating truth from fiction. When it assumes the role of an expert, it remains uncertain whether it genuinely holds that expertise. Sutter humorously remarks that he sometimes questions his own status as an astrophysicist, yet the language model harbors no such doubts whatsoever. This issue is not a mere implementation error that can be rectified; it is inherently embedded within the architecture itself.
The algorithm processes a sequence of words by passing it through numerous layers, establishing connections among the words. In a typical model, each word is associated with approximately fifty thousand other words and undergoes approximately one hundred thousand transformations. All of these processes are performed to forecast the subsequent word. It emerges that this task of predicting the next word is exceedingly powerful and beneficial. However, that is the sole function of the model.
Regarding alchemists, they possessed knowledge of how to manipulate specific substances. These substances were referred to as “volatile,” as they could readily alter their form and vaporize when subjected to heat and pressure. They regarded the most valuable substances as those that could be “grounded,” reverting to a solid state. The same principle applies to linguistic models, which are considered volatile due to their ability to rapidly change their identity.
Our response to this, as articulated by Sutter, hinges upon the concept of grounding. It is imperative that we do not consider the model as a repository of knowledge, but rather as a tool for organizing existing information. The most effective applications of artificial intelligence are those that are firmly rooted in reality — through sources, citations, and verified facts.
The second issue pertains to autonomy. These models function autonomously; by activating a single command, they analyze vast datasets — ranging from ten thousand to one hundred thousand entries — search for patterns, and identify differences. This can be likened to utilizing free graduate students, as Sutter humorously suggests — although the tokens used incur costs, the overall expense remains lower. The concern lies in the fact that we lack insight into how the models generate their responses. While we have developed the code and comprehend the architecture, neural networks are inherently complex and nonlinear, rendering it difficult to trace the decision-making process from input to output. Furthermore, these models operate probabilistically and do not consistently produce identical results.
Alchemists possessed a comparable level of understanding regarding their experimental processes. They, too, lacked insight into the internal mechanisms of their reactions; their knowledge did not extend to atoms, elements, molecules, or forces. Their observations were limited to the inputs and outputs. This conceptual framework was referred to as the “golden chain,” which Sutter applies to artificial intelligence, establishing the principle of strict traceability. While the detailed reasoning algorithms of the model remain beyond our complete comprehension, it is possible to monitor each step from the initial query to the final result. Even if the internal operations of the model are not fully transparent, we can compare the input with the output and, if required, trace the entire pathway retrospectively.
The third issue pertains to unlimited content generation. Models consistently continue to produce outputs without cessation. This phenomenon is already applied in pharmaceutical development, where computations that historically demanded substantial resources are now conducted through exhaustive testing of millions of combinations. However, this approach incurs certain costs. All models tend to yield similar outcomes due to their reliance on identical architectures, training datasets, and methodologies.
You may inquire of Claude, ChatGPT, and Gemini with the identical question and receive fundamentally the same response. Satter refers to this phenomenon as “qualitative sameness.” However, when accompanied by immediate expertise and rapid response times, it creates the impression that the model is a genuine professional, thereby tempting individuals to delegate authority to this machine.
Alchemists recognized this potential hazard. They advocated for the principle of the “whole person” — that significant discoveries are primarily achieved by those who are fully immersed in the process. In laboratory settings, a researcher does not remain on the periphery of the experiment; rather, they become an integral part of it, with their intentions affecting the outcome. As Sutter states, this implies that even if artificial intelligence automates ninety percent of the work, the residual human contribution continues to hold significant value. Conversely, it amplifies this value, since that remaining ten percent of human involvement is entirely invaluable. Machines excel in their respective domains, but the control remains firmly in human hands. Sutter refers to this concept as the principle of human sovereignty.
The steering wheel remains under human control
Sutter openly acknowledges the extent to which he has incorporated these tools into his professional activities. Having engaged in programming since the age of six and been involved in computational cosmology since graduate studies, he has not hand-coded any software since December. All coding tasks are performed using Claude Code. He employs the platform to brainstorm ideas, evaluate alternatives, formulate different approaches for scholarly articles, and assign the model to conduct research reviews. Currently, his team is developing agent-based systems designed to further automate these procedures.
On each occasion on which Sutter interacts with a language model, he contemplates three questions. Will I be capable of defending every assertion it makes prior to releasing it to the public? Am I able to trace each outcome back to the corresponding input and output data? Additionally, who truly held the authority in this work — was it I or the machine that provided the value?
To those who categorically reject language models, Sutter is blunt. Your colleague next door uses them. He writes grant proposals faster, prepares articles faster, and achieves results more swiftly. “I did not ask the universe for this,” says Sutter. “I did not foresee the emergence of ChatGPT, despite having worked with machine learning and neural networks for nearly twenty years. However, this world exists, and we must adapt accordingly.”
Philosopher’s stone
Alchemists never discovered the philosopher’s stone. It does not exist, yet they remained unaware of this fact and pursued it for fourteen centuries. Nevertheless, throughout their pursuit, they unintentionally created other substances and knowledge. Each generation of alchemists regarded their work as superior to that of previous generations and believed that their knowledge should be transmitted to subsequent generations. Consequently, they meticulously documented every procedure and observation — including those beyond their understanding. They detailed the sources of their materials, the sequence of operations, and the rationale behind their actions.
This process persisted from the 3rd century through the 16th and 17th centuries in Europe. Among its practitioners were Johannes Kepler, Robert Boyle, and Isaac Newton. All three individuals were alchemists, with alchemy representing their principal pursuit. The modern concept of science emerged as a byproduct of their alchemical endeavors.
They adopted the methods characteristic of alchemists — such as meticulous documentation, systematic observation, and the honest recording of natural phenomena — and applied them to physical systems and celestial motions. Consequently, this marked the inception of the scientific revolution. Essentially, the origins of science can be traced back to the practices of alchemy.
We are now approaching an era in which scientists not only utilize machines but also collaborate with them. Satter is uncertain about the nature of science ten years hence. However, he is confident that through meticulous documentation, observation, and recording of results, new discoveries will be made — potentially as impactful as science itself.
