Robotaxis represent one of the most complex engineering challenges of the modern era. Getting a vehicle to navigate busy city streets, respond to unpredictable human behaviour and safely transport passengers — all without a human driver — requires an intricate combination of hardware, software and artificial intelligence.
This article provides an accessible overview of the key technologies that make autonomous taxi services possible.
The Sensor Suite
Every robotaxi relies on a combination of sensors to perceive its surroundings. The most common sensor types include:
- LiDAR (Light Detection and Ranging) — uses laser pulses to create detailed 3D maps of the environment. LiDAR can measure distances with centimetre-level accuracy, making it effective for detecting obstacles, pedestrians and other vehicles. Companies like Waymo and Zoox use roof-mounted LiDAR units as a core part of their sensor stack.
- Cameras — provide visual information including colour, texture and the ability to read signs, traffic lights and lane markings. Most autonomous vehicles use multiple cameras positioned around the vehicle to achieve 360-degree coverage.
- Radar — uses radio waves to detect objects and measure their speed. Radar is particularly useful in poor visibility conditions such as rain, fog or glare, where cameras and LiDAR may be less effective.
- Ultrasonic sensors — short-range sensors typically used for close-proximity detection, such as during parking manoeuvres or when navigating tight spaces.
Most leading robotaxi operators use a combination of all four sensor types — an approach known as sensor fusion — to create redundancy and ensure reliable perception in a wide range of conditions.
The AI Brain: Perception, Prediction and Planning
Raw sensor data alone is not enough. The real intelligence of a robotaxi lies in its software stack, which typically operates in three stages:
1. Perception
The perception system processes data from all sensors to identify and classify objects in the vehicle’s environment. Using deep learning models trained on millions of kilometres of driving data, the system can distinguish between cars, trucks, cyclists, pedestrians, animals, traffic cones and other objects. It also tracks their position and movement over time.
2. Prediction
Once objects are identified, the prediction system estimates what they are likely to do next. Will that pedestrian step off the kerb? Is the car ahead about to change lanes? Prediction models use behavioural patterns learned from real-world driving data to anticipate the actions of other road users, typically projecting several seconds into the future.
3. Planning
The planning system uses perception and prediction outputs to decide what the vehicle should do. This includes route planning (which streets to take), tactical decisions (when to merge, when to yield) and low-level control (steering angle, acceleration, braking). The planner must optimise for safety, comfort, efficiency and compliance with road rules — often making hundreds of decisions per second.
High-Definition Maps
Most robotaxi operators rely on pre-built, highly detailed maps of their operating areas. These HD maps include information that goes far beyond standard navigation maps: lane positions, kerb heights, traffic signal locations, speed limits and even the geometry of intersections.
Waymo, for example, uses mapping vehicles to survey and catalogue its service areas before launching operations. The HD map provides a baseline understanding of the environment, which the vehicle’s real-time sensors then update with dynamic information like moving vehicles and temporary obstacles.
Connectivity and Remote Support
While robotaxis are designed to operate autonomously, most operators maintain remote support centres where human operators can monitor vehicle status and provide assistance in unusual situations. This might include helping a vehicle navigate around an unexpected road closure or communicating with passengers via an in-vehicle intercom.
This remote support capability is not the same as remote driving — the human operators do not control the vehicle’s steering or speed. Rather, they provide high-level guidance that the vehicle’s autonomous systems then execute.
Safety and Redundancy
Given the safety-critical nature of autonomous driving, robotaxis are built with extensive redundancy. This typically includes:
- Backup computing systems that can take over if the primary system fails
- Redundant braking and steering mechanisms
- Multiple independent sensor systems so that the loss of one sensor type does not compromise the vehicle’s ability to perceive its environment
- Fail-safe protocols that bring the vehicle to a safe stop if a critical system failure is detected
The Road Ahead
The technology behind robotaxis continues to advance. Areas of active development include improving performance in adverse weather conditions, reducing the cost of sensor hardware, expanding operational design domains (the conditions under which the vehicle can safely operate) and developing vehicles purpose-built for autonomous operation — such as Zoox’s bidirectional pod, which has no traditional front or back.
For Australia, understanding these technologies is important context for the regulatory and infrastructure decisions that will need to be made as the country prepares for the eventual arrival of autonomous taxi services.
Sources
- Waymo — How Waymo’s autonomous driving technology works
- Zoox — Purpose-built autonomous vehicle design
- NHTSA — Automated Vehicles Safety Overview
- SAE International — J3016 Levels of Driving Automation
- National Highway Traffic Safety Administration — Automated Vehicles
- National Transport Commission — Automated Vehicle Program (Australia)
