AI has grown so adept at solving mathematical puzzles that critics fear the flood of new proofs it creates will overwhelm academics and actually set the field back. Until now, AI has had limited impact on other areas of science, but there are signs that physics may be about to experience a similar upset.
at Yale University says AI models may have an unfairly bad reputation when it comes to physics because of the way they are tested. Cutting-edge models have previously scored as low as 30 per cent on six different tests designed specifically to assess AI鈥檚 ability on tough physics problems, but this doesn鈥檛 square with the successes Sous and his colleagues have seen when using them, he says.
Checking the problems in the benchmark revealed a litany of errors and ambiguous definitions. Once corrected, . The problem didn鈥檛 lay with AI, but in the tests not providing the necessary information.
Advertisement
鈥淚 think anything that鈥檚 well defined, even if it鈥檚 very hard, I suspect that AI will be very good at it 鈥� arguably better than the best experts,鈥� says Sous. 鈥淭his is backed up by our data.鈥�
But while some parts of physics can be very well defined 鈥� physicist that mathematics is the part of physics where experiments are cheap 鈥� many aren鈥檛. And it is in the new, uncharted, fuzzy and cutting-edge parts of physics where AI may struggle.
Sous says that interesting physics comes when we find something confusing, some phenomenon we observe in nature, a material or the cosmos that we don鈥檛 understand. It is then a physicist鈥檚 job to create a model, an approximation or a simulation that captures and predicts that phenomenon.
鈥淭hat鈥檚 a very artistic, a very creative process,鈥� says Sous. 鈥淐an AI really take the next step and do the full job of a theoretical physicist? Do end-to-end research: define, create, explore, test? The picture is not clear yet.鈥�
One reason mathematics has been at the tip of the spear of AI progress is formalisation. This technique turns a mathematical idea into computer code that can be methodically checked; the process is so thorough that if a theorem comes through intact, it is considered correct beyond all reasonable doubt. It allows AI to process a problem and know when it has arrived at a correct solution.
While some advances have been made in using formalisation to test physics, it is early days in comparison to mathematics. And physics problems can often be messier and harder to pin down.
What is clear is that AI is already being used by physicists, not only to write computer code and do maths, but even to design and run experiments. , also at Yale University, is now teaching a course on using AI in physics research, and says that the syllabus is constantly changing to catch up with developments.
The key, says Wright, is to create experiments where the inputs and outputs can be computer controlled. Then that control can be handed over to an AI that is able to pull the appropriate levers to explore the problem space and arrive at meaningful results.
鈥淚t doesn鈥檛 have hands, it doesn鈥檛 have eyes, it just interacts with the world through language and code,鈥� says Wright. 鈥淪o if we can set up an experiment in which all the observations are reduced to code, and all the actions are reduced to code鈥� then AI can actually be very helpful.鈥�
One such experiment currently underway in Wright鈥檚 laboratory involves analysing how different shapes of laser pulses affect a rubidium atom. The researchers use Anthropic鈥檚 AI Claude to control the laser pulses and gather data on their effects.
鈥淐laude can now basically say 鈥業 think this laser pulse is gonna produce this output鈥�, right,鈥� says Wright. 鈥淎nd it can test that.鈥�
Wright says that such experiments can run overnight and complete as much work as a human would in a week 鈥� and human physicists can even use an app on their phone to suggest ideas that come to them while they are away from the lab, and get the results when they return.
鈥淚t鈥檚 not going to be [the case that] all of the universe can be explored in that way. It鈥檒l be like very constrained domains,鈥� says Wright. 鈥淚 think part of the next frontier of physics will be to make as much progress as we can in those domains.鈥�
Wright says that it is already entirely possible that AI will soon design an experiment that humans don鈥檛 understand, ask humans to build it and generate exciting new results. 鈥淏ut just to be clear, just like in math, there鈥檚 a debate about does it count as science if we get a result that we don鈥檛 understand anything about,鈥� says Wright.
