by Key Takeaways
- Waymo has developed a new training model called EMMA, which is based on Google’s multimodal large language model.
- The model creates driving strategies for self-driving cars based on sensor data.
- In the study, Waymo admits that there are still issues with the model that must be fixed before it is released.
Waymo has released a new study on creating a training model for its robotaxis by utilizing Google’s Gemini large language AI model, which is a sibling company.
The training model, known as EMMA (End-to-End Multimodal Model for Autonomous Driving), creates driving plans for self-driving cars using sensor data, in this case, camera images. A group of programmers won’t create EMMA’s internal architecture or make any particular adjustments based on driving experience, in contrast to previous autonomous driving initiatives like Tesla and Wayve.
Gemini in EMMA has real-world information beyond typical driving logs because it has been educated on enormous volumes of internet data. This aids in overcoming the drawbacks of traditional driving system modules, which frequently have preset settings that make it difficult for them to adapt to new environments.
Gemini is also capable of chain-of-thought thinking, which is comparable to human reasoning. This will help the driving system make the best choices possible.
Since EMMA is built on a natural language LLM, it can be instructed via written commands regarding navigation, weather, and road conditions, thereby assisting Waymo’s robotaxis in navigating challenging environments. According to the research article, EMMA has been evaluated on both an internal dataset and a number of publicly accessible datasets. The model successfully assisted its autonomous vehicles in determining the proper routes in the majority of these tests.
In the research report, Waymo admits that EMMA still needs a lot of work. Due to the computational cost of processing this data, the model is currently unable to include 3D sensor inputs from LiDAR or radar. Additionally, EMMA has a restriction on how many image frames it can handle at once. Waymo has made it clear that more investigation is required before the model can be used in practical situations.
