Everyone wants to improve their lap times. A research team programmed an AI for super-human performances.
Photo credit: imago images / Science Photo Library
In Gran Turismo, a gray-red painted Audi rushes across the asphalt on the Tōkyō Expressway. The racing car exits the tunnel, takes every corner perfectly and sets an absolute best time. It’s not a professional esports driver behind the wheel – but an artificial intelligence that controls the car.
Drivers can have many individual goals or motives to engage in racing. Some want to have fun with it, others love the community aspect, still others prefer to challenge themselves. But in the end, it’s all about one question: Who is the fastest driver? And to be the fastest of them all, you need to drive the perfect racing line.
A team from the University of Zurich guided by Italian Professor Doctor Davide Scaramuzza developed a Deep Learning Artificial Intelligence (AI) for Gran Turismo Sport which was able to find the perfect trajectory autonomously. The AI learned how to drive a car via Reinforcement Learning on a flawless racing line and was able to outperform over 50,000 reference drivers. We invited Prof. Scaramuzza for an interview to talk about his recent project, and also discuss how esports racers can learn from the AI to improve their lap times.
We make use of maximum-entropy deep reinforcement learning and a new reward design to train a sensorimotor policy to complete a given racetrack as fast as possible. We outperform the fastest known times in a dataset of personal best lap times of over 50,000 human drivers!
— Davide Scaramuzza (@davsca1) August 19, 2020
Experimenting with car racing
Davide Scaramuzza is a professor of Robotics and Perception at the University of Zurich, as well as the leader of the Robotics and Perception Group. The research team has a specific goal in mind for all their studies:
Our mission is to research the fundamental challenges of robotics and computer vision that will benefit all of humanity. Our key interest is to develop autonomous machines that can navigate all by themselves using only onboard cameras and computation, without relying on external infrastructure, such as GPS or position tracking systems, nor off-board computing.
To achieve this goal, the team is currently experimenting with autonomous drone racing and autonomous car racing. The next step towards their intermediate objective is to make autonomous machines more agile and outperform humans in races, as Prof. Scaramuzza explained.
Their experiment surrounding Gran Turismo Sport was the most recent part of their work. You can find the autonomous agent in action in the video below. As you can see, the vehicle was able to beat the expert driver as it pursued a different approach for the racing line.
Super-human performance in Gran Turismo
The UZH Robotics and Perception Group worked together with SONY AI Zurich for this project with one goal in mind: to create a Deep Learning AI that is able to drive the perfect trajectory. The fully autonomous agent controlled a race car in Gran Turismo Sport and outperformed all human reference drivers.
A total of three different agents for three different settings was programmed. The team used two car models and two circuits for that. You can find the three settings on the picture below. To validate the results, they also compared the performance to that of a human expert driver called “TG”. The training was done with four PlayStation 4 consoles and one single desktop PC. Overall, it took less than 73 hours for the AI to achieve super-human performance. As developer Polyphony Digital provided the comparables and data, the team was able to compare the AI with the best lap times of over 50,000 Gran Turismo racers.
Faster than everyone else
The AI went for an aggressive approach, which allowed the car to keep up higher speed. According to Prof. Scaramuzza, the vehicle was forced to its physical limits:
Autonomous car racing is an extremely challenging problem in robotics. Its goal is to complete a given track as fast as possible, which requires the algorithm to generate fast and precise actions even when the vehicle reaches its physical limits. In addition, it features the principle importance in emergency driving scenario: how to control the vehicle at extreme high speed.
As you can see in the table above, the AI is faster than the performances of all human players compared to it in the project. It is also faster than the built-in GTS NPC. But the limits that come with human abilities are not only an issue within driving, but also with the problem of autonomous car racing itself which Prof. Scaramuzza mentioned above. This is why the team used a reinforcement learning system:
Traditional approaches provide little promise to address the problem since these systems are normally handcrafted by human experts, and hence, they may impose a ceiling on the performance of the system. By contrast, reinforcement learning systems are learned from scratch by trial and error, in principle allowing them to exceed human capabilities, and to operate in domains where human expertise is lacking.
Why did they choose Gran Turismo Sport?
There is a wide range of racing games available, but the team settled for Gran Turismo Sport for several reasons. First, the game offers advanced graphics and very realistic modelling of different race cars and tracks. On top of that, Prof. Scaramuzza explains:
It is uniquely suited to act as a general platform for studying autonomous racing systems, as it allows simulation of a large number of realistic racing environments with sophisticated physics and complex interactions between vehicles. With over 80 million copies sold, GTS allows us to directly compare the performance of our approach with professional human players, bringing invaluable comparison and evaluation for our autonomous systems.
What can players learn from the AI?
The experiment not only brought insights for the Robotics Group, as esports racers can also benefit from the AI. The deep learning program provides a flawless way to find the best route around a circuit, and players can simply try to follow it:
In theory, there is only one time-optimal trajectory given each race car & racetrack combination. If you can follow this time optimal trajectory precisely, there is no chance for other players to play against you. However, even though our agent can find a trajectory that is faster than the best human, we don’t know yet if this is the optimal one.
Of course, human drivers cannot reproduce the performance to 100% accuracy. As Prof. Scaramuzza explains, the perception of a human is not perfect and there is lots of randomness and many delays in the human brain. He continues to state that mathematically speaking, human performance (high-level planning and control) is not the optimal solution.
So far, the AI from Prof. Scaramuzza’s team only targeted the ideal trajectory. However, the researcher is confident that in theory, it would also be possible to create an AI to test the perfect pit stop strategy and tire management in more complex games such as iRacing. The Robotics Group has also already built a system that can do very aggressive overtaking.
If you want to know more about the technical details of the project, check out Prof. Scaramuzza’s research paper which you can find here. For more studies and experiments of the UZH Robotics and Perception Group, follow their YouTube channel.
What other projects involving AI would you like to see in racing games? Tell us on Twitter at @overtake_gg!