In a modest office in Sheffield, England, a compact server hums with activity, housing eight Nvidia chips. This unassuming setup mirrors the massive data centers popping up worldwide, which are the physical embodiments of cutting-edge AI models. Here, engineers from Amodo Design are testing a monitoring system that could one day be integral to every data center, addressing the growing concerns of AI researchers about the potential risks of unchecked AI development.
In late July, over 1,300 employees from leading AI companies signed an open letter expressing alarm at the rapid advancement of AI, warning that it may soon surpass human control. They emphasized the need to slow down AI development to allow for safety research, but acknowledged the intense competition between companies and countries makes this difficult. The letter called for the U.S. government to support international efforts to build tools that could enable all sides to slow down the AI race.
Amodo Design’s AI Verification Initiative
Amodo Design’s CEO, Tom Milton, estimates that there are fewer than 50 engineers worldwide working full-time on so-called AI verification tools, with nine of them at Amodo. Despite the vast resources poured into making AI systems more powerful, efforts to build slowdown tools are primarily funded by academia and philanthropy. Amodo’s work in this area is supported by the Survival and Flourishing Fund and Longview Philanthropy, both of which have heavily invested in reducing AI-related risks.
Milton, who entered the field somewhat by chance, notes the surprising lack of people working on AI verification. In Sheffield, three workers are focused on their compute cluster, which, despite running hot, is not yet ready for prime time. The prototype faces both technical and political hurdles, with the latter being the more significant challenge. Milton stresses that the system won’t be useful unless the U.S. and China agree on an AI slowdown treaty.
How AI Verification Could Work
Amodo’s engineers believe that any AI slowdown treaty will likely require monitoring data centers, where AI models reside. Their current prototype aims to provide two key assurances: first, that a data center is only being used for inference, meaning the running of existing AI models rather than training new ones; and second, that a data center is running a specific, agreed-upon model that has passed certain safety tests.
To demonstrate the system, an Amodo engineer logs into the server rack, where two separate systems are running. The first system contains an AI model, while the second, the verifier samples data from the first system and reruns it to confirm the model’s identity. In the demonstration, the verifier correctly identifies the model as GPT-OSS-120B, showcasing the system’s potential.
Challenges and Limitations
Despite its promise, Amodo’s solution has several limitations. Currently, it only works with unencrypted data, making it unsuitable for sensitive workloads. Milton notes that the next version will utilize zero-knowledge cryptography to reduce the need for unencrypted data. Another challenge is the requirement for data centers to be retrofitted, including a process called network tapping which involves copying data from working chips to verification systems. This is a daunting task given the high-security nature of data centers.
The verifier system also requires significant computing power, which could reduce the capacity and profitability of data centers. However, Amodo’s engineers expect to achieve substantial efficiency gains, potentially by monitoring only random samples of a data center’s computation. Milton acknowledges that the technology isn’t perfect yet but believes it will improve over time, becoming minimally invasive and maximally privacy-preserving.
The Political Landscape
While AI verification technology is still in its infancy, recent advancements in AI capabilities have sparked interest in the field. The Institute for Progress recommended that the U.S. government collaborate with AI labs, chipmakers, and data center builders to accelerate the development of AI verification tools. Similarly, the authors of AI 2040 emphasized the importance of data center monitoring technologies in their plan for navigating the arrival of superintelligent AI.
Anthropic recently announced its commitment to devoting resources to building systems that would enable a credible slowdown or pause in AI development. Milton has held preliminary discussions with governments about Amodo’s work, although he declines to share specifics. Despite the current lack of enthusiasm from the U.S. government, Milton remains optimistic that both the U.S. and Chinese governments will eventually be sufficiently scared by more powerful AI models to come to the table and agree on an AI treaty.
For now, the political feasibility of implementing this technology remains uncertain. However, Milton believes that the possibility of future agreements justifies the investment in building the optionality for AI verification tools.
