Artificial intelligence has long achieved superhuman performance in digital domains such as chess, Go, and video games. However, the leap into the physical world has remained one of the greatest challenges in robotics. With Project Ace, Sony AI has made that leap: an autonomous table tennis robot that defeats professional players under official competition conditions.
What Is Physical AI?
Physical AI describes AI systems that perceive, decide, and act in the real world – in real time and under physical conditions. Unlike purely digital AI, Physical AI systems must cope with unpredictability, speed, and physical forces. This requires the fusion of advanced sensing, machine learning, and precision robotics.
The term deliberately distinguishes itself from "Embodied AI," which often remains confined to simulated environments. Physical AI operates exclusively in the real world and must contend with latency, friction, air resistance, and the unpredictability of human opponents. The demands on reliability and safety are incomparably higher than in purely virtual scenarios.
From Gran Turismo Sophy to Project Ace
Sony AI has systematically approached Physical AI. The first milestone was Gran Turismo Sophy – an AI agent that achieved superhuman performance in the PlayStation racing simulation and was published in Nature in 2022. Sophy proved that reinforcement learning works in highly dynamic competitive situations, albeit initially only in simulation.
Project Ace transfers exactly this methodology to the physical world. The crucial difference: while Sophy received perfect information from the game engine, Ace must reconstruct its entire perception from camera data in real time – at ball speeds exceeding 100 km/h and reaction times of just a few milliseconds.
The Sensor System in Detail
Ace's perception pipeline consists of two complementary camera systems:
- Nine APS Cameras (Active Pixel Sensor): These conventional high-speed cameras provide precise 3D positional data of the ball through triangulation. The cameras operate at extremely high frame rates to capture the ball's trajectory without gaps.
- Three EVS Cameras (Event-Based Vision Systems): Event-based cameras register only brightness changes at the pixel level, achieving temporal resolution in the microsecond range. This enables them to measure the ball's spin – a critical parameter that decisively influences the trajectory after bounce.
The combination of both systems allows Ace to simultaneously capture both position and rotation of the ball – a capability that even experienced human players possess only intuitively and imprecisely. Spin detection achieves accuracy at rotation speeds up to 450 rad/s, equivalent to approximately 4,300 revolutions per minute.
Model-Free Reinforcement Learning
Ace uses model-free reinforcement learning – meaning the robot does not learn from a predefined physical model of the game, but exclusively from experience. During training, the AI runs through millions of simulated rallies, optimizing its strategy through trial and error.
This approach has a decisive advantage: Ace can dynamically adapt to new game situations that were not anticipated by any model. During a match, the system continuously analyzes the opponent's playing style and adjusts shot selection, placement, and timing accordingly.
Particularly noteworthy is the sim-to-real transfer: strategies learned in the simulator are directly transferred to the physical robot. The discrepancy between simulated and real physics – the so-called "sim-to-real gap" – is minimized through domain randomization and robust policy architectures. This enables Ace to execute shots that were never explicitly trained in the real world.
The Robot Arm: Precision at Maximum Speed
Ace's hardware is designed for maximum speed and precision. The robot arm features six degrees of freedom and can achieve accelerations far exceeding human capabilities. Motion planning occurs in under 5 milliseconds – from detecting the ball to calculating the optimal stroke.
Ace commands a broad repertoire of shot techniques: topspin, backspin, sidespin, and flat shots are dynamically selected based on the game situation. The robot can return the ball with variable spin and precise placement – capabilities that are rare at the professional level and unprecedented at the machine level.
Results Under Competition Conditions
Ace was tested under official International Table Tennis Federation (ITTF) rules against five elite and two professional players. The results are impressive:
- Three victories in five matches against elite players (rating above 1,800)
- Over 75% return rate at spin values up to 450 rad/s
- 16 direct points on serve – twice as many as the human opponents combined
- Average shot speed exceeding 9 m/s with peaks above 12 m/s
- In follow-up tests in December 2025 and March 2026, Ace defeated multiple professional players and demonstrated higher shot speeds and more aggressive placement
Particularly revealing was the behavior of the human players: several professionals reported that playing against Ace "feels different" than against other players. The consistency and unpredictability of the robot's shots forced players to abandon their usual strategies and reorient themselves in real time.
Why Table Tennis?
Table tennis may seem like a niche application at first glance, but it is an ideal testing environment for Physical AI. The sport combines extreme speed (ball velocities over 100 km/h), complex physics (spin, air resistance, material friction), and strategic depth (opponent adaptation, serve variation) within a clearly defined ruleset. This makes table tennis a perfect benchmark for the question of whether AI systems can operate at expert human level in dynamic physical environments.
Significance Beyond Sport
The implications extend far beyond table tennis. Peter Stone, Chief Scientist at Sony AI, emphasizes: "Once AI can operate at an expert human level under these conditions, it opens the door to an entirely new class of real-world applications."
Potential application areas include:
- Autonomous Manufacturing: Robots that can react to unforeseen situations on production lines
- Surgical Assistance: Precise, fast interventions under variable conditions
- Safety-Critical Environments: Deployment in disaster zones, underwater, or in space
- Logistics and Warehousing: Dynamic grasping and sorting of heterogeneous objects
- Human-Robot Collaboration: Robots that react to human movements in real time and work cooperatively
The core achievement of Project Ace – the integration of high-speed perception, adaptive learning, and precision mechanics – is a blueprint for any application requiring physical intelligence in dynamic environments.
Outlook: The Next Level of Physical Intelligence
With Project Ace, Sony AI has provided important proof: Physical AI is no longer a future vision but technical reality. The next steps will show how quickly the technology can be transferred to other physical domains. The combination of simulation-based training and high-precision sensing offers a scalable approach that points beyond individual applications.
The research was published on the cover of the journal Nature. More details at ace.ai.sony.