A Senior Software Engineer at Mercedes-Benz R&D North America, US patent holder and IEEE Senior Member, tells us what it takes to make a car understand human speech without a data center behind it
In-vehicle artificial intelligence assistants have turned into serious business. Analysts at The Business Research Company put the market at $3.46 billion in 2026 and, in their latest report, projected growth to $8.71 billion by 2030, a compound annual rate of 26 percent. Money on that scale flows toward one stubborn engineering question: how do you run a language model in a machine that drives through tunnels, loses network coverage and cannot afford a frozen interface? Answering it in production cars is the job of Sivakumar Balu, Senior Software Engineer at Mercedes-Benz Research & Development North America in San Jose. After integrating large language models into the HeyMercedes Voice Assistant of Mercedes-Benz vehicles, he moved part of that intelligence onto the car's own edge computer, cutting the number of cloud calls and their cost. Earlier in his career came eleven years at Honeywell, a climb from rank-and-file engineer to technical lead on communication and inertial navigation equipment for Boeing 777 and 787 aircraft under Federal Aviation Administration rules. We asked him why the industry wants language models on wheels, what breaks first when you put them there, and what aviation taught him about software that is not allowed to fail.
Sivakumar, forecasts have the in-vehicle assistant market more than doubling in four years. Yet voice control has existed in cars since the early 2010s, and most drivers used it for little beyond dialing a number. What changed?
Old systems matched your phrase against a fixed list of commands. Say the phrase slightly wrong, and nothing happened, so people stopped trying after two failures. Large language models remove that wall: they understand intent, not wording. You can say "I'm freezing and the windows are fogging up," and the assistant will figure out which climate settings to change without you naming any of them. That shift matters for safety more than for comfort. Every second a driver spends scrolling a touchscreen menu is a second of attention taken from the road, and speech is the only interface that keeps both hands on the wheel and both eyes ahead. Beyond commands, we now build products around personalization, so the car responds to its owner rather than to an average user. Demand followed capability, which is what those market figures reflect.
At Mercedes-Benz you first connected the assistant to cloud language models, and then integrated an edge LLM that runs on the vehicle's own computer, reducing cloud calls and cost. Computing power in a data center is nearly unlimited. So why give any of it up?
Three reasons, and each one alone would justify the work. Latency comes first. Each request that travels to a server and back can take a couple of seconds; conversation dies at that pace, and in a moving car a delayed response to a control request is worse than no response. Connectivity is second. Vehicles spend real time in parking garages, tunnels and rural dead zones, and an assistant that goes silent there trains the driver to distrust it everywhere. Cost closes the list: a single inference in the cloud is cheap, but multiply it by every utterance in millions of cars over years of ownership and it becomes a line item that shapes the whole business case. Edge models earn their keep by handling frequent, well-bounded requests locally, while the cloud takes the long tail of open questions. Making that split work is harder than it sounds, because automotive hardware brings strict limits on memory, power and heat, and the model must be compressed to fit them without losing the accuracy people already expect. Our goal was never a laboratory exercise. Bringing this to millions of Mercedes-Benz cars meant that every design decision either saved or burned real money at scale.
Before cars there were airplanes: during eleven years at Honeywell you grew from engineer to technical lead and led research and development of communication and inertial navigation devices for commercial aircraft, including the Boeing 777 and 787. Getting avionics through certification under Federal Aviation Administration rules is considered one of the hardest paths in engineering. Just how unforgiving is that process from the inside?
Unforgiving is accurate. Certification authorities in aerospace assume nothing and verify everything, because a device that talks to air traffic control or tells a Boeing 787 where it is in space has no permission to fail. Compliance at that level is not a final exam; it shapes the work from day one, so we sat with certification engineers throughout development instead of meeting them at the finish line. Traceability became my personal obsession there. Requirements had to flow through design, code, review and testing without a single orphan among them, an experience I later turned into a published method for carrying safety-critical requirements through the whole product life cycle. Leading a team added one more layer, since a technical lead in avionics answers not only for his own work but for the rigor of every engineer he mentors.
Later your career took a sharp turn toward consumer devices, with firmware for Nest smart-home products, and then toward automotive AI. Which of those aviation habits survive in a car cabin?
Discipline, mostly. More than any single tool, aerospace leaves you with the reflex of proving things rather than assuming them, and cars reward that reflex, because they are certified machines too. Nest added a different lesson, about doing more with less, since a small consumer device gives you no spare memory to hide inefficiency in. Put together, those two schools are exactly what edge AI needs: constrained computers running software that people trust with their lives at highway speed.
One of your inventions has been approved by the US Patent and Trademark Office, and a second application has been filed in Great Britain; among them is a next-action mechanism for system-generated messages. Where does a production engineer find patentable ideas?
In friction, honestly. Patents rarely come from sitting down to invent something; they come from a moment when the standard approach fails and you are forced to build a workaround, then realize the workaround is general. Colleagues sometimes ask where to begin, and I tell them to write those moments down before deadline pressure erases them.
Alongside the day job, you have refereed more than ninety articles and journal publications, and this summer are judging at the AITEX Summit 2026. How does unpaid work of this kind pay an engineer back?
An early view of the field, for one. Refereeing puts new ideas in front of you a year or two before they reach products, which quietly keeps your own thinking current. Judging adds something else: you learn to articulate why one solution is better than another, and that skill transfers directly into design reviews at work. There is also plain obligation. Peer review only exists because practicing engineers donate hours to it, and everyone who benefits from that system owes some of those hours.
Two technical books and five journal papers also carry your name, alongside an IEEE publication. To close: what should an engineer entering automotive AI learn first, and where are you taking your own work next?
Learn constraints before models. Anyone can call an API; the profession begins where compute, safety and cost push back. My own plans stay on the same road: developing AI products for automobiles and intelligent transportation systems, so that the intelligence we now put into one HeyMercedes Voice Assistant gradually spreads across everything a vehicle does.