I am elated about the coming of autonomous vehicles. I truly believe that road safety, traffic efficiency, and transportation accessibility would be greatly improved if cars were driven by computers instead of humans. Not only would there be fewer crashes, but there would be less traffic, more efficient use of cars and roads, and we would have a lot more time on our hands. The following is mostly a summary of Waymo’s safety report.

Imagine if, you could drive with 360-degree vision around your car simultaneously at all times, and not have to only rely on vision, but you had supervision that lets you see even in the dark. Imagine if you had superhuman focus on driving and never got distracted. Imagine you were also emotionless on the road and were never phased by other incompetent drivers. What if you could do all of these things so consistently that you were near flawless? This is how autonomous vehicles will eventually work, and we’re not far from it.

There are dozens of companies devoting tons of resources to advance autonomous vehicle technology, but Google’s Waymo is at the forefront of the revolution. Started in 2009, Waymo is an autonomous driving technology development company working to build “the world’s most experienced driver” and improve road safety.

Waymo currently has a fully autonomous ride-hailing service called Waymo One, which operates within parts of the Phoenix metropolitan area.

Every year, 1.35 million lives are lost to traffic crashes around the world, and in the U.S., the number of tragedies is growing. Humans have endless distractions fighting for our attention. Meanwhile, self-driving AI exists with one purpose and one purpose only: To drive safely.

As of early 2021, Waymo released a safety report sharing their data and processes for developing a fully autonomous car. Although this technology is still fairly new, they’ve collected a surprising amount of data thus far. Waymo is on its 5th generation of autonomous vehicles, has driven over 20 million miles on public roads, and another 15 billion miles in simulation.

How does the technology work?

Human drivers need to answer four basic questions: Where am I? (the environment around you), What’s around me? (processing that information), What will happen next? (predicting how others in that environment will behave), and What should I do? (making driving decisions based on that information). An autonomous driving system needs to answer those questions, too.

Where am I? Before Waymo cars drive anywhere, they build detailed 3D maps that highlight information such as traffic lights, stop signs, road profiles, curbs and sidewalks, lane markers, crosswalks, and other road features. The Waymo Driver cross-references the pre-built maps with real-time sensor data to precisely determine their location on the road, rather than relying on GPS.

What’s around me? Waymo sensors and software constantly scan for objects around the vehicle —vehicles, cyclists, pedestrians, road work, obstructions — and continuously read traffic controls, from traffic light color and railroad crossing gates to temporary stop signs. The Waymo sensor suite has a 360-degree view around the vehicle and is designed to respond to objects up to up to 300 meters away.

What will happen next? For every dynamic object on the road, Waymo software predicts future movements based on current speed and trajectory. It understands that a vehicle will move differently than a pedestrian or cyclist. The software then uses that information to predict the many possible paths that other road users may take. Their software also considers how changing road conditions such as a blocked lane may impact the behavior of others around it.

What should I do? Waymo’s software considers all of this information as it finds an appropriate route for the vehicle to take. Then it selects the exact trajectory, speed, lane, and steering maneuvers needed to progress along this route safely. Because the Waymo Driver constantly monitors the environment and predicts the future behavior of other road users in 360-degrees around their vehicles, the Driver can respond quickly and safely to any changes on the road.

Sensors

To meet the complex demands of autonomous driving, Waymo has developed an array of sensors that allows their vehicle to see 360-degrees, both in the daytime and at night, and up to 300 meters away. This multi-layered sensor suite works together seamlessly to paint a detailed 3D picture of the world, showing dynamic and static objects including pedestrians, cyclists, other vehicles, traffic lights, construction cones, and other road features.

Lidar (Laser) System

Lidar (Light Detection and Ranging) works day and night by beaming out millions of laser pulses per second—in 360-degrees—and measuring how long it takes to reflect off a surface and return to the vehicle. Waymo’s system includes three types of lidar developed in-house: a short-range lidar that gives their vehicle an uninterrupted view directly around it, a high-resolution mid-range lidar, and a powerful long-range lidar that can see 300 meters away.

Vision (Camera) System

Waymo’s vision system includes cameras designed to see the world in context, as a human would, but with a simultaneous 360-degree field of view. Because their high-resolution vision system detects color, it can help their system spot traffic lights, construction zones, school buses, and the flashing lights of emergency vehicles. Waymo’s vision system is comprised of several sets of high-resolution cameras, designed to work well at long range, in daylight and low-light conditions.

Radar System

Radar uses electromagnetic waves to perceive objects and movement. Radar remains effective in rain, fog, and snow, and operates equally well day or night. Waymo’s radar system has a continuous 360-degree view, so it can track the presence and speed of road users in front, behind, and to both sides of the vehicle.

Inertial Measurement Unit

This module uses accelerometers and gyroscopes with input from GPS, maps, wheels speeds, and laser and radar measurements to provide highly accurate position, velocity, and heading information to the vehicle. This information remains highly accurate even in the event of a sensor, vehicle component, or other system failures.

Supplemental Sensors

Waymo vehicles also have a number of additional sensors, including an audio detection system that can hear police and emergency vehicle sirens up to hundreds of feet away, and GPS to supplement their vehicles’ extensive understanding of their physical locations in the world.

Software

All of these sensors and information would not work without the software. Waymo’s autonomous driving software is the “brain” of the Driver. It makes sense of the information coming from their sensors and uses that info to make the best driving decisions for each situation.

Their system possesses a deep, contextual understanding of the world. It doesn’t just detect the presence of other objects; it actually understands what an object is, how it’s likely to behave, and how that should affect our vehicle’s own behavior on the road.

While the software is made up of many different pieces, there are three main components: perception, behavior prediction, and planner.

Perception

Perception is the part of the software that detects and classifies objects on the road and estimates their speed, heading, and acceleration, while also understanding the environment. The software takes the myriad of details coming from the sensors and turns them into a cohesive real-time view of the world. Perception helps the vehicle distinguish pedestrians, cyclists, motorcyclists, vehicles, and more. It also distinguishes the color of static objects such as traffic signals. For these kinds of objects, perception enables the system to semantically understand the situation around the vehicle—whether there’s a construction zone or a lane is blocked because of the many cones in front of it.

Behavior Prediction 

With behavior prediction, the software is designed to model, predict, and understand the intent of each object on the road. Because Waymo has millions of miles of on-road driving experience, their vehicles have highly accurate models of how different road users are likely to behave. For example, their software understands that, though pedestrians, cyclists, and motorcyclists may look similar, their behavior can vary dramatically. Pedestrians move more slowly than either cyclists or motorcyclists, but they can change direction more suddenly.

Planner

The planner considers all the information their software has gathered from perception and behavior prediction and plots out a path for the vehicle. The best drivers are defensive drivers. That’s why Waymo has baked in defensive driving behaviors, such as staying out of other drivers’ blind spots and leaving extra room for cyclists and pedestrians. Waymo’s planner can also think several steps ahead.

For example, if the software perceives that an adjacent lane ahead is closed due to construction, and predicts that a cyclist in that lane will move over, the planner can make the decision to slow down or make room for the cyclist well ahead of time. Using their on-road experience, they’re also refining their driving so their movements on the road are smooth and comfortable for passengers inside their vehicles, and natural and predictable for other road users.

To put this into perspective, Waymo’s autonomous driving technology is classified as a level 4 while Tesla’s popular autopilot is a Level 2. The 6 levels of autonomous driving range from 0 to 6 and are recognized as the standard for legalities and driving ability.

As a quick overview, Level 2 has ADAS (advanced driver assistance systems) where the vehicle can control both steering and accelerating/decelerating. At this level, humans are still required to be attentive at the wheel. Level 4 vehicles can intervene by themselves if there’s a system failure. They can legally operate in self-driving mode, however, only within a limited area until legislation and infrastructure evolve.

So, why is this technology so important? Why should you be excited? I’m sure many of us will miss the occasional afternoon cruise to clear our mind, and the subtle vibration in the steering wheel and feeling in control, but there’s a lot to be excited about!

Huge environmental gains will come from autonomous driving as a result of less traffic and more efficient use of cars via car sharing. This also means we can get to our destination quicker, and we can use that traveling time to be more productive. Autonomous vehicles will help us avoid a lot of crashes and eliminate much of the direct and indirect costs they have on the economy. come from them. Plus our insurance premiums will go down significantly!

Autonomous vehicles will allow people with disabilities to have more personal freedom and independence to go places by themself. Government data identifies driver behavior or error as a factor in 94 percent of crashes, and self-driving vehicles can help reduce and eventually eliminate driver error. We’re still a ways off from level 6 autonomy, but self-driving cars are already on our roads and they’re improving every single day.