How Do Self-Driving Cars Know When to Stop?

You’re approaching an intersection and the stoplight ahead flashes from green to yellow. Instead of accelerating before the red, you make the responsible decision and slow to a stop, glancing out your window at the car pulling up alongside you: a white car bulging with strange, bulky cutouts on its sides and topped with a spinning contraption that resembles a whirling top hat. 

But that’s not what draws your eye — there’s no one in the driver’s seat. 

It might feel like you’ve stepped into a sci-fi film, but this is a common sight in cities such as Phoenix, Arizona, and Austin, Texas, where Waymo, Alphabet’s self-driving car initiative, operates its fleet of autonomous vehicles. 

But how exactly do these self-driving cars know how to stop at lights, change lanes, or make left turns? 

No eyes, no ears, no problem

Autonomous vehicles can “see,” for lack of a better term, through a combination of cameras and sensors that provide the vehicle with a comprehensive view of its surroundings. Combined with advanced AI algorithms, this data enables driverless cars to navigate independently, though there are still plenty of puzzle pieces to be figured out before they’re considered truly autonomous. 

Multiple car manufacturers and technology companies are currently experimenting with autonomous vehicles (AVs), and while their methods and technology can vary, most self-driving cars operate similarly. AVs use multiple types of sensors to understand their surroundings: Radar sensors (as seen in the above diagram #1) use radio waves to determine the distance between objects; lidar sensors use light beams to do the same; ultrasonic sensors (#2) determine closer distances and help with parking; infrared sensors detect lane lines and pedestrians in low-light conditions. Meanwhile, high-definition cameras (#3) add to the information captured by sensors to complete the picture of a car’s surroundings. 

Even with all this input, the vehicle requires some level of intelligence to determine how it should move. In regular cars, that analysis is done by the driver (for better or worse), but in an autonomous vehicle, advanced AI algorithms do the legwork. These algorithms have been trained to recognize common objects like stop signs, traffic lights, pedestrians, oncoming traffic, etc., and direct the car’s response. Training cars to identify and adapt to brand-new or unexpected situations — such as quickly flooding roads or steep slope changes — is much trickier, however, which is why you won’t find a self-driving car off-roading in the wild just yet.  

Despite the impressive advancements in this technology, there are still things that can’t replace a human driver’s intuition. Making eye contact with a pedestrian or driver to determine right of way, or navigating the free-for-all chaos of a crowded parking lot, can introduce problems for autonomous vehicles. Unfavorable weather conditions also play a huge role in a driverless car’s ability to read its surroundings, especially when weather covers and obscures signs and lane markings, or the AV’s sensors themselves. 

Until manufacturers figure out how to account for these drawbacks, cars won’t be truly autonomous. On the plus side, that strange-looking Waymo beside you isn’t likely to challenge you to a drag race anytime soon.