Thoughts on Automotive IoT

Most of the current (and future) features of the cars we drive (or that drive us) depend on Internet of Things (IoT) technologies. These features go beyond simple sensors that allow us to get in and out of a garage or inform us of when automobile maintenance is necessary.

This virtual roundtable includes four automotive industry expert panelists. The panelists were given a set of three questions, and we are publishing their responses with minor editing.

COMPUTER: The IoT is heavily focused on the sensors that produce the data that the software algorithms feed on to make decisions. Is automotive IoT any different?

ALEXANDER WYGLINSKI: There is more to the IoT than just sensors, including automotive IoT applications. For instance, my research activities primarily focus on enabling/implementing reliable and seamless wireless connectivity for a range of technologies that include the IoT (automotive, smart agriculture, and opportunistic radio frequency localization). For the IoT in general, connectivity is an essential ingredient for moving data off these sensing devices; sending them to data centers for processing; and pushing decisions back to platforms.

Within the context of automotive IoT, connectivity is critical, especially if the IoT applications being supported involve multiple vehicles on the road trying to cooperate with each other and exchanging mission-critical information, such as the platooning of autonomous vehicles, vehicles navigating through intelligent intersections, etc. It’s the aspect of (potentially high-speed) mobility that differentiates automotive IoT from other IoT applications. Not only is the environment being sensed by these IoT devices rapidly changing, but the connectivity infrastructure supporting the movement of this information from sensors to data centers back to other IoT devices is varying quickly over time and space (especially the network topology and protocols needed to form these connections between vehicles and with roadside infrastructure). As a result, this makes designing reliable and dynamic IoT rather challenging.

SCOTT J. McCORMICK: There are some fundamental differences in automotive sensors. First, they must be what is considered “automotive grade.” In-vehicle electronics are subject to both vibration and temperature ranges that most in-device sensors don’t experience. Therefore, they must be more physically robust. In many IoT applications, the sensors can transmit wirelessly. In-vehicle wireless transmission is relegated to non-safety-critical systems, such as tire pressure monitoring systems. The complexity of in-vehicle software and the fact that we are fusing data from multiple sensors to gain higher order knowledge about road, weather, and traffic systems are also differentiators. As the vehicle is consuming energy and traversing the road infrastructure, some sensors are geared to reduce the car’s impact on the environment. On the sensing side, new contactless techniques are appearing from physical sensors based on light, electromagnetic, or acoustics waves (that is, video cameras, lidar, radar, and ultrasonics).

Importantly, as we are dealing with a moving vehicle, safety and liability are the critical concerns. Therefore, the sensors must have a higher mean time between failure, redundancies, and failover systems (such as inertial guidance when the GPS signal is lost). Safety cannot be relegated off-vehicle, so using cloud data processing is still in its infancy for safety-critical systems. For building or roadside sensors, there are much fewer safety-critical restraints and manageable environmental impacts.

FLAVIO BONOMI: In a broader sense, there are profound similarities between what we are witnessing in IoT applications and what we see in the evolution of the automotive architectures and technologies.

Both IoT and automotive applications do the following:

1. They interact with the physical world through sensors (in fast evolution through MEMS technologies, radar, lidar, and mobile phone developments).

2. They use evolving communications technologies (wired and wireless; local and wide area; and the Internet infrastructure) to move the sensor data to computing and storage resources and embedded or distributed infrastructure (cyberspace).

3. They use evolving applications deployed across the computing-networking-storage infrastructure to clean, share, fuse, and analyze sensor data to extract insights and to derive decisions to modify and optimize the future behavior of the physical systems under consideration. These decisions are based on human experience and knowledge or on digital models build with artificial intelligence (AI) or physical models.

4. They apply these decisions to the physical systems through actuators or control points, closing a feedback loop that may involve humans or be closed autonomously locally or through edge or cloud computers.

What we are part of is the evolution of cyberphysical systems and the digital transformation, where we hope to apply technologies developed in the IT domain to improve the operations and control of physical systems.

As we get close to the physical boundary and as we want to directly impact the behavior of the physical systems through closed-loop controls, we need to meet more stringent requirements, including safety, real time, physical security, failure prevention, etc. The systems we are considering are also becoming more sophisticated, distributed, and collaborative. Consider, for example, the future vehicles interacting with each other and with applications at the roadside, in the edge and cloud infrastructures. We are just at the beginning of a huge journey!

DAVID McNAMARA: In agreement with Scott’s comments, automotive sensors, and aerospace, for that matter, stand apart as having performance, environmental, and also validation requirements that are additional and can be costly. The automotive engineer, possibly differing from the aerospace engineer, is looking for technologies that are affordable and scalable as well as robust (Scott’s point). That said, often, the consumer industry will offer up technologies in the area of connectivity and computing that are robust and scalable. Lidar is a good case in point where the industry is still working on that problem—not so with cameras and now capable and low-cost CMOS radars. Bottom-line volume applications always come to the rescue, which has been true of camera- and radar-based advanced driver assistance systems (ADASs); customer wants and needs drive volume.

COMPUTER: Many challenging requirements were mentioned by the panel: connectivity infrastructure; robustness for both vibration and temperature changes; data aggregation where data need to be fused from multiple sensors; more rigorous safety and security (for example, higher mean time among failure; redundancies; and failover systems); performance; and affordable and scalable environmental and validation requirements.

That said, how close is the auto industry to addressing these requirements? Are there IoT features/systems that are on hold until those requirements are addressed? Automotive is just one safety-critical transportation vertical domain. How much does automotive IoT leverage other domains, such as rail and plane, in terms of assuring safety and trust?

McCORMICK: How close is the auto industry to addressing these requirements?

There isn’t an end game, per se. As the vehicle continually evolves with advances in each of the areas noted, new challenges arise.

For example, the three documents 1) SAE J1938: Design/Process Checklist for Vehicle Electronic Systems; 2) SAE J3083: Reliability Prediction for Automotive Electronics; and 3) SAE 1J1879 Revised 2_2014: Handbook for Robustness Validation of Semiconductor Devices in Automotive Applications are essentially always a work in process. As the electronic systems and semiconductors change, they have to be reevaluated and the specs updated.

Are there IoT features/systems that are on hold until those requirements are addressed? No, they are proceeding in unison in my opinion. ADASs, for example, can synchronize their own systems and fuse data. With information from other vehicles and the infrastructure, they can evolve to higher levels of automation. We know all the communication protocols necessary to make this happen, with cellular, dedicated short-range communications, satellite, and Wi-Fi. So, it’s really an adoption rate issue. We replace fewer than 7% of the national fleet every year, so if tomorrow all new vehicles were sold with the capability to send and receive road, weather, and traffic data, it would take 15 years before the entire fleet was equipped. Progress is needed on object recognition so that it can build a 3D map for at least a 100-m range and a high angular resolution image for a quarter mile out. A recent analysis by a major tier found that cameras do not see a pedestrian or bicyclist 2 s before colliding. At just 35 mi/h, that is more than 50 ft away. The vision systems don’t see well in snow or rain or at night. And only one car can hear, but it recognizes only one sound. But the industry is working on all of these things and many others. The IoT has many things to work on until those requirements are met.

Automotive is just one safety-critical transportation vertical domain. How much does automotive IoT leverage other domains, such as rail and plane, in terms of assuring safety and trust? To my knowledge, it doesn’t. Each of the transportation domains mentioned, along with maritime, has specific trust and safety issues that are, for the most part, unique to their modality. There is no common frame of reference for ships, aircraft, trains, or cars to intersect. They all want to avoid that intersection, but how each does it is specific to the boundaries of their conveyance mode. The safety and trust must be paramount and established. But below that, the implementation is unique.

McNAMARA: The auto industry is fortunately a very collaborative community, and our engineers participate in many forums and standard organizations (ISO, SAE, IEEE, etc.) that have participation from the communications, transportation, and consumer industries. With that said, many requirements and most of the test methods and certifications are specific and customized to automotive. We leverage in these forums the knowledge gained as to the readiness, test methods, and requirements for emerging technology.

Scott has correctly characterized that we are chasing a moving target relative to requirements. The issue is that as technology adoption increases, we are not keeping pace, and the customer is used as the test bed. Tesla has been accused of employing this methodology. The automotive aftermarket thrives on lower costs by reduced specification and testing: possibly a strategy to improve time to market but dangerous. As consumers demand new ADASs and safety features, every industry ups its game.

Even though we borrow and reference other industry specs (to capture requirements), the automotive system testing (key cost and time to market impact) is unique. Our different mission and operating environment require different system testing for performance and robustness. You can argue that we can borrow component test methods (for example, cameras), but the bulk of the engineering effort is largely centered on vehicle system testing. In the area of automation, connected vehicles (CVs), and electrification, the original equipment manufacturers (OEMs) have spent considerable effort to system-test fleets under real-life conditions. The Tampa CV pilot (Honda, Toyota, and Hyundai testing CVs) and Ford testing automated deliveries in Miami are examples of extensive fleet system testing. The IoT consumer industries, for example, don’t employ that level of preproduction testing, but aerospace does.

WYGLINSKI: Yes, there are definitely several automotive IoT features that are on hold due to the challenges listed from the previous question. For example, for self-driving car applications involving information exchanges performed wirelessly across a large number of vehicles (think a 1-mi stretch of the Mass Pike leaving Boston during the afternoon rush hour), providing reliable real-time situational awareness for every vehicle on the road when a complex emergency situation occurs is an issue; for 99.9% of the time, the traffic will be mundane bumper to bumper at 2 mi/h, but for that 0.1% of the time when something serious and irregular happens, all vehicles within the vicinity of that unique time-sensitive event must be able to react with as much reliable information as possible.

There is significant R&D being pursued by the wireless/CV sector to provide very low-latency wireless connectivity in applications such as vehicle-to-vehicle information transfers (for example, the latest 5G releases have a low-latency mode), but it will take time (maybe another five years or so) to achieve sufficiently low latency across a very large number of vehicles all operating along the same stretch of road whose actions are highly correlated with each other. It might be possible to look to other verticals, such as aviation and rail, for best practices and lessons learned, but their operating environments are different; rail is 1D, whereas automotive traffic is 2D, while aviation operating environments are highly controlled with the aerial vehicle density being substantially lower compared to ground vehicles.

BONOMI: Automotive is one of the many verticals that are in the middle of an accelerated transformation, a manifestation of the broader digital transformation. Automotive, like aerospace, robotics, intelligent transportation, energy, industrial automation, etc., belongs to the class of cyberphysical systems, where humans touch complex physical systems with the help of cyberspace resources (computing, networking, AI, etc.).

All these verticals are in the process of adopting a range of new technologies, often coming from the IT domain, with the goal of achieving ambitious improvements in efficiency, cost, performance, autonomy, and more. These technologies include:

1. more sophisticated sensors, for example, lidar, radar, cameras, etc.

2. more advanced networking technologies, including more time-sensitive networking (TSN), such as 5G/6G and Ethernet with TSN

3. more powerful nonhomogeneous computing with multicore CPUs; GPUs; field-programmable gate arrays; tensor processing units; and organized and multichip systems

4. more advanced and mission-critical virtualization, using hypervisors previously adopted in advanced embedded systems used in avionics

5. more advanced operating systems, real time and non-real time, coexisting on the same computing infrastructure

6. applications such as AI and digital twins to interpret data; gain insight; and make control decisions in a more scalable and powerful way

7. modeling tools to capture more complex system behaviors

8. more advanced software lifecycle management systems, based on containers and orchestration (for example, Kubernetes)

9. more advanced security methodologies

10. and more…

Maturity in the adoption of this huge range of new and complex technologies, when the ultimate solutions need to satisfy safety and reliability levels not typical of the IT solutions, is still far away, across all the domains mentioned earlier.

It will take time and a more effective collaboration across domains, with support from academia, industry consortia, standardization bodies, and precompetitive industry efforts, to accelerate this evolution. It will also take patience and more long-term investment from governments and venture capitalists to bring about the required breakthroughs.

Many problems are still wide open. Just to name a few:

1. It is difficult to produce reliably executable code from current modeling tools (the model-code gap).

2. It is not easy to safety-certify and validate systems controlled by AI algorithms. AI is usually not explainable, black box, and statistical.

3. The data middleware (for example, Data Distribution Service) required to share data across distributed applications, typical of solutions in automotive and the other verticals named earlier, is immature and only now trying to address time sensitivity and determinism.

4. There are no distributed deterministic operating systems working across distributed computing and communications resources.

5. We are just beginning to address the broad problem of ultimate system validation and certification, with the help of simulation, hardware in the loop, continuous integration, and deployment.

Above all, we need to have the humility to know we are not there yet and there is much more collaborative work to do. But the journey is exciting!

COMPUTER: Have the supply chain issues related to chip shortages affected the “Things” in automotive IoT? Does the CHIPS Act, just signed into law, do anything to help the automotive industry?

McCORMICK: The biggest impact on the OEMs from the chip shortage is falling revenue. OEMs have not corrected the long, disruption-prone logistics routes between them and their chip suppliers. They won’t be able to produce as many vehicles. Martin Banks^1, managing editor at Modded, stated that “Leading automotive manufacturers could see US$110 billion in losses in 2022 due to the chip shortage. Automakers may see higher revenues than in 2021 and 2020, but they should anticipate lower-than-usual sales.” Similar shortages will likely recur in the future. Semiconductor manufacturers have accelerated production to make up for their considerable backlogs, but disruptions with raw material suppliers could have cascading impacts. Due to availability and prices, demand will likely fluctuate. If demand picks up with the availability of chips, prices will likely rise. OEMs may focus on producing and selling vehicles with fewer semiconductors, that is, without extra features such as wireless charging, to stretch their inventories further. Demand for electric vehicles (EVs) is also rising, and those vehicles require more semiconductors, further exacerbating the problem. A positive change that will result from the chip shortage is a move to restructure supply chains. I expect OEMs to move away from lean inventories to holding more reserves. I expect the semiconductor shortage to have lasting effects into the future.

The bipartisan CHIPS and Science Act that President Joe Biden signed into law gives automakers hope that the semiconductor industry will now be able to keep up with surging auto industry demand. The regulation gives about US$52 billion in subsidies for semiconductor analysis, design, and manufacturing within the U.S., together with US$2 billion put aside for the mature node “legacy chips,” which can be generally utilized by automakers and suppliers. The invoice additionally features a 25% tax credit score for investments in microchip manufacturing by means of 2026. However, it takes as many as five years for a new semiconductor fabrication plant to stand up and begin producing, so that relief won’t come for some time.

McNAMARA: A great and important question. The auto industry developed relationships with an electronics ecosystem that provides them the highest quality volume supply at the lowest cost. What that meant historically was significant offshoring.

It is not clear that the investment in the U.S. “chip” infrastructure will directly benefit the auto industry. Automotive has special requirements relative to peripheral circuits, for example, power supplies and custom input–output circuits for ADASs versus the high margin versus the servers and GPUs of the consumer market, Historically, many semiconductor companies have left the automotive market for these higher margin businesses/clients. Therefore, it remains to be seen if we see any significant supply relief beyond the slow progress toward relief in late 2023 and the benefit of onshoring. For example, in the area of 5G for connected cars and EV applications, suppliers are looking to China to launch these new applications in 2023 and to move to the United States, possibly in 2024. Many of the suppliers consider the Chinese OEMs as the key customer with first-to-market applications and volume.

WYGLINSKI: Absolutely! Automotive IoT was definitely impacted by supply chain issues related to chip shortages. The foundation of all of the IoT is based on semiconductor devices, from which operations such as embedded computing, sensing, and communications are performed across these complex networks. Without the availability of these chips, creating IoT environments, especially those used in automotive applications, are just simply impossible. And this is not even referring to self-driving cars and their IoT networks; today’s conventional ground vehicles possess a plethora of sensing, computing, and communication devices that all interact with each other to enable safe and reliable vehicular operations while on the road, many of which are real-time mission-critical functions, such as the transmission, power steering, power braking, fuel injection, etc. With an absence of the necessary chips to form these automotive IoT networks, these vehicles cannot even pull out of the garage safely (or even start in many cases).

The recently signed CHIPS Act will directly have an impact on the automotive sector with respect to fueling new innovation in the design, implementation, and fabrication of these semiconductor devices that would make automotive IoT more reliable, cost-effective, and capable as well as insusceptible to supply chain issues. This includes opportunities for supporting both existing companies as well as enabling new start-ups to emerge and grow in a market seeking solutions to challenging problems. Additionally, the CHIPS Act will also assist with the workforce development of future engineers and technologists capable of coming up with new processes and designs for chips and other semiconductor devices to address future challenges and emerging applications that are not realizable with today’s technology.

BONOMI: The number of electronics (that is, chips) deployed in automotive is exploding both because more intelligence, driver’s assistance, and entertainment are required and because more sensors for safety, performance, and comfort are featured in new car designs. This requires more powerful CPUs to support control and entertainment and more embedded CPUs for the expanding range of sensors deployed.

The supply chain challenges of the past years have definitely conditioned the availability of new automotive products and also contributed to the skyrocketing of the value of used vehicles. It had led to the dramatic rethinking about how to reduce the OEM dependence on outsourcing many of their chips. It has led to foundational investments across the world and within the OEMs, including:

• the CHIPS Act legislation by the Biden Administration
• OEMs’ decisions to invest in their own special silicon.

I believe that one of the architectural directions that may alleviate the supply chain issues experienced in automotive electronics is the consolidation of multiple, often specialized, electronic control units into stronger multicore chips, possibly more similar to those used in other operational industries and even in the IT industry.

The extreme optimization and differentiation on computing chips with a limited set of suppliers may be counterproductive with respect to the mentioned supply chain issues. The software-defined vehicle may more and more resemble a data center on wheels.

COMPUTER: Thank you all for your participation. You provided valuable insights into the current automotive IoT feature challenges as well as the progress needed to reach industry goals and offer increased consumer functionality.

This discussion highlights the development focus in this industry, such as object recognition; improving vision in bad weather; automotive testing issues; adoption rate; and more. To quote one of the panelists—there is much more work to do, but “the journey is exciting!”