It took more than 25 minutes to realize that Lucas Bolster was not driving in busy San Francisco traffic.
The Mercedes-Benz engineer was alert and monitoring where he was going, but his vehicle, the electric CLA, was making all the driving decisions.
Bolster's feet were firmly placed on the floor, his hands only lightly touching the steering wheel. The sedan guided Bolster along the desired route, stopping and accelerating when necessary. The experience was nearly seamless.
Navigating around San Francisco can be nerve-racking for a human driver: there are formidable hills, aggressive city buses, distracted pedestrians, confusing traffic lights and infinite Waymo autonomous taxis scurrying around. Add in the trolleys packed with tourists and the driving experience becomes an obstacle course.
Bolster and his team have spent years refining the company's MB.Drive Assist Pro technology, which arrives in the U.S. later this year. The German automaker partnered with Nvidia on the software, which launches first in the CLA. More Mercedes models will included in the near future, according to Bolster.
Of course, customers can choose not to opt in, and Mercedes makes it clear that the tech is not fully self-driving. It's SAE Level 2, meaning it requires the driver to be engaged, responsive and ultimately reliable for the vehicle's operation. If the vehicle detects that the driver is distracted or sleeping when the program is on, the vehicle will come to a complete stop and call emergency services. MB.Drive Assist Pro costs $3,950 for three years; a monthly subscription is available too.
Advanced driving technology has become increasingly popular in the industry, with Tesla's Full Self-Driving (Supervised) and GM's Super Cruise widely accepted by owners. Other automakers are in the process of rolling out their semi-autonomous systems. Bolster said he has been carefully fine-tuning the tech so it can recognize all possible situations.
"We are working on street and garage parking," he told ABC News. "Tunnels are a challenge. The system does not handle roundabouts as well as I want. There are challenges in every city. We're trying to get it really perfect."
Here's how the tech works, as explained by Bolster.
The interview below has been edited for clarity.
Q: How is this different from the Mercedes-Benz Drive Pilot program that launched in 2023?
A: Drive Pilot was basically Level 3, or conditionally automated driving for freeways. Customers had the freedom to take their eyes off the road and hands off the wheel and engage in secondary activities while they were in freeway traffic jams.
This system is still considered Level 2, so hands on wheel and eyes on road. But what it's meant to do is basically alleviate the stresses of city driving. It's also really proficient at being a second pair of eyes.
It's not fully autonomous -- you're collaborating with the system. We do a lot of data collection around the world with human drivers. We study naturalistic driving. We have taken this all across North America and do customer studies to get feedback.
MORE:Mercedes-Benz's CEO is a 'petrol head.' He's also convinced EVs are the future
Q: How long have you been working on this tech in San Francisco?
A: I personally have been working on the driver assistance for 10 years. This particular system has probably been in development for five years in various iterations.
There are five radars and 10 cameras. Four are mostly for parking. The other six are for driving. There are fender-mounted front cameras. In urban driving, you have to think a lot about when you're going into an intersection. Those cameras are great for looking out your side window.
Suburbia brings it own challenges but they're mostly easier than in a dense urban [environment].
Advertisement
Q: How does Mercedes' technology compare to what else is available on the market?
A: What we're trying to do is bring features that we think will add value to our customers. We have all the ingredients to basically offer every single technology that our competitors do. We were first with Level 3.
Q: Explain how MB.Drive Assist Pro works.
A: Once you hit the resume button on the steering wheel the system becomes available. It does everything it needs to do to follow the route we programmed into the navigation. It's making sure we're in the right lane, it's doing lane changes to overtake slow [cars], yielding to pedestrians, watching traffic lights and stopping for red lights.
The system basically requires a light grasp on the steering wheel. I don't have to provide steering torque but I just need to be touching it. And the camera here is also monitoring my gaze and it will remind me to remain alert. When the car is going to make a turn, there is no physical warning in the driver seat. You can counter steer to cancel a turn, though.
A big part of what makes city driving hard is basically dealing with random and unclear situations. Double parked cars -- are they actually double parked? Pedestrians -- are they looking to cross or are they waiting for the bus, for example? All of these sort of situations that are really hard to get right. The left most lane, for example, can make left turns, but it doesn't have turn protection. It can be difficult for the system to figure out: should it make a lane change? When would that seem like a nervous behavior?
How do you assert yourself at a four-way stop? What do we want the car to do there? We're still in the final tuning stages.
Q: How do you train the computer to be more assertive?
A: It learns through watching behaviors. We are feeding the model thousands and thousands of hours of driving. You can very specifically train behaviors either by finding data or creating data of behaviors you want. If you want the system to nudge when there is an open car door, you show it videos of cars nudging. And you also give it counterfactuals. You can show it videos of bad driving behaviors and collisions as a negative -- a behavior you should avoid doing.
Q: Competitors' driving assistance technology will tell the driver to take over when there's construction. How would this system respond?
A: The system designed to handle construction. It should handle all of that with your attention. It's never meant to replace you. It's meant to help you.
Q: There are still a lot of people who are nervous to let a car make the driving decisions. How do you get people more comfortable to give up that control?
A: I think one thing is to get the drive as intuitive as you can so [the car] is driving the way you would drive. The other part is the UI [user interface]. It's not integrated yet but there's going to be a really rich driver assistance graphic -- basically all the detected objects in the road and associated lane markers will appear [in a 3D layout] to build your confidence in what the system is seeing and reacting to.
It's all about having option. It's a system someone can you decide not to use for one trip but use for another.
MORE: What it's like to experience Mercedes-Benz's Drive Pilot automated system
Q: The car will know to slam on the brakes or swerve if an animal or person gets in the way?
A: Yeah. The model is incorporating all this human collision-avoidant behaviors.
Q: What about in bad weather -- can the car do the driving?
A: You will see some performance degradation [in bad weather] the same way that a human has degradation performance driving.
I would pretty cautious about using the system up to and including heavy rain. There's no promise that the system is going to be 100% flawless, or we would be calling it Level 3.
Q: Has your personal driving style changed since you've been tinkering with this technology?
A: Maybe? [laughs]. We were working on the blinking behavior for overtaking double parked cars. Someone on the team was like, "I always blink when I am overtaking." Then one day he was driving behind me and he did not put on his blinker. Everyone thinks they're an above average driver.