Beyond Spotify and Netflix- the future of in-vehicle infotainment systems in connected cars

 When talking about data monetization in the automotive industry, we tend to focus on technology, safety, sensors, or cloud solutions. However, all these elements fade when confronted with the ultimate element - the driver of the vehicle. Without taking into account their needs and expectations, there can be no question of generating revenue. Any vehicle data monetization strategy must be mindful of this.

We can fine-tune the system, we can find exceptional partners to implement the software in the vehicle, but without a deep understanding of the vehicle user, no one will benefit from the solutions developed. Our organization will put a considerable amount of effort into building the team and implementing the technology, but the new vehicle features will not be used by the driver.

For this to happen, we need two factors: a value proposition of the brand- which explains clearly and transparently what the user will get out of it, and a coherent action strategy based on a market-back methodology that stems from specific market needs and allow us to develop services that are desired by the customer.

What benefits do customers most often look for in a software-defined vehicle?

Remember that just because people want to use a service, it doesn't mean that they will pay for it. What matters here is not just the benefit, but also the way it is presented, the user experience, and the pricing model. Only the combination of all these elements determines the success of the service. First of all, it is worth focusing on the benefits themselves and only then selecting the right technology to match them.

What are users willing to actually pay for and what are they willing to share only? Many studies indicate that the main factor motivating consumers to share data is gamification and rivalry - this aspect has not changed for years, as we can see for example in social media or e.g. "free" applications, which from time to time appear on the market, gather millions of interested users and vanish in no time. However, when it comes to paying for such "services", users are not so willing to use them.

In vehicles, it looks slightly different. Capgemini's research shows that the connected car services that are most popular with consumers are those related to the "core" functionality of vehicles, such as:

•     safety,  
•     driving comfort,  
•     time saving  
•     reduction of vehicle operating costs.  

Among them, however, the services that are most willingly paid for are:

•     hazard warning,  
•     collision warning,  
•     theft detection systems / vehicle finder.  

Of course, just because entertainment or gamification isn't on the list doesn't mean that automotive companies should avoid them. It's also a way to distinguish and find their own individual voice that corresponds to the broad brand strategy and allows them to stand out in the market. It's about the way they are served, presented to the consumer, and showing that they can actually derive real benefit from them.

It also works the opposite way. Simply creating a "hazard warning" service in a connected car does not immediately guarantee success. It still needs to be packaged properly, run smoothly, and be provided with a payment model that suits the consumer.

Examples of customized connected car services

In-vehicle ads based on navigation and user experience

Is it possible that a driver will like the ads that will be displayed in the car? If we adopt the message to their needs and preferences, in all likelihood, it is. For example, if we often go to McDonald’s, the navigation system can mark such places on our route. We have our favorite clothing brand, right? We will certainly react differently to a sale offer in a shopping mall we just happen to be driving past. The context of shopping and the consumer’s needs are decisive, and the software-defined vehicle is perfectly suited to ensuring that the advertising message is 100% tailored to the driver.

Contextual payments

Removing barriers to shopping and being able to buy everything everywhere is a popular trend in modern commerce. In a vehicle where the driver is focused on the road and has their hands full, such a service makes even more sense. With the development of voice assistants, drivers will be able to pay this way not only for fuel or tolls but also for purchases beyond typical vehicle-related payments. Voice shopping on the way back home from work, instead of looking for a parking space in front of the mall and returning in traffic jams in the evening? Why not?

Sharing information about driver behaviour

Sharing data about the way we drive may not appeal to everyone. But if in return for sharing this information, a company gives us a huge discount on our car insurance or a super attractive leasing offer, then things may take a totally different turn. In cooperation with an insurance company or a bank, such services become a specific bargaining chip the OEM can play with when dealing with the driver.

Manufacturer's connected car applications

Saving money on car maintenance and taking care of the overall condition of the car is a benefit that most drivers will appreciate. A practical and thoughtful manufacturer app that warns of potential breakdowns, component replacements, or servicing will allow the user to enjoy a well-functioning vehicle for longer and sell it at a higher profit. In this way, the OEM gets the driver used to have the vehicle repaired at an authorized service center, and the user, due to the loyalty shown to the brand, can expect future discounts and lucrative offers.

Practical use of telemetry

Sharing telemetry data may seem profitable only to OEMs - after all, as they draw better conclusions based on the collected information and save on R&D processes. However, it is important for companies to make vehicle users aware of the benefits of such services, as well. After all, driving style data can be used to suggest solutions that improve road safety, work on fuel efficiency or reduce overall vehicle operating costs. In each of these cases, the winner is the driver. Example? When a vehicle frequently skids and triggers the ESP/TC system, the system can suggest that the driver should get better tyres (by a specific brand, of course).

Unlocking extra features on the subscription model

Paying for heated seats, just to use them for three months a year, may not be worthwhile for everyone. Well-known to us from streaming portals, the subscription model definitely meets the users’ needs. The customers themselves choose which functionalities they want to pay for and over what period of time. The OEM only has to take care of the right vehicle software that will enable that. And, of course, be careful not to alienate those customers who see this as "yet another" way to squeeze additional payments out of them. That’s how manufacturers can provide both functionalities directly related to the vehicle itself - e.g. better lights or engine boost - as well as those associated with in-car entertainment providers such as Spotify or Apple CarPlay.

What can be done to make the user more eager to pay for data monetization services?

A well-thought-out user experience is essential

In today's digital world, UX and mobile-friendly approaches decide whether a service is viable. If the product is presented in an unclear and incomprehensible way, and it is difficult for the user to find the desired options - they will not use it. The size and color of buttons, the messages displayed, the stability of the application - all of the above is of paramount importance and determine the popularity of the product. Keeping in mind the latest trends, mapping the market, and adapting to consumer trends is necessary to offer the vehicle user service of the quality known to them from e-commerce or their own AppStore.

UX itself is not only a practical tool that helps better track consumer behavior and how they use the service, but also a constant theme to promote and boost brand interest. Does Apple really need to upgrade iOS every year and does Instagram have to offer users a new feed layout every quarter? The answer is obvious. It's simply profitable for the brand.

Start with anonymized data

When creating a strategy for in-vehicle data monetization efforts, it's a good idea to start by developing services that don't require the sharing of personal data. A lower "pain threshold" will make it quicker for the user to learn the benefits of the system and how convenient or useful the service can be. Thus, it will be easier to convince people to use products that require more openness to data sharing. And this may be the next step in the implementation of technological solutions.

Focus on heavy-vehicle users

People who spend most of their day in the car or drive long and demanding routes happily embrace any technical innovations designed to make driving easier and safer for them. It is this group that should be targeted at the beginning of developing your own data monetization model.

Minimizing risks and accurately selecting the group will not solve all challenges, but it will increase the chance of success and help gain a new, loyal group of consumers who will help transfer the technology to other users.

Last, but not least: a flexible payment model

Convenience should accompany the user at every stage of the use of a new service. Not only when it is most beneficial to the user, but also when it is easiest for the user to give it up: whilst paying for the next billing period.

It is worth taking care of the flexibility of the payment model (e.g. one-off payment, freemium model, annual or monthly settlement), adjusting it to the user's needs and not hindering payments.

The smoother and more tailored to the user's needs the whole process of interacting with the service is - from understanding the need to using it to making payments - the greater the chance that the stream of data flowing from a given vehicle will not dry up after a short period of use (read: being frustrated using an underdeveloped product for the first time).

Let's remember that data monetization can succeed provided that it really understands the user, is fair and transparent to them and focuses on user experience. If we didn't have time to get to know the customer's needs, why should they waste their time on services they don't understand and don't need?

    Brook. Not a stream yet, though. But in the foreseeable future, it is going to be a proper river. What are we talking about? Data obtained from vehicles. Experts estimate that data inflow is likely to rise from approximately 33 zettabytes (this is how much we obtained in 2018) to 175 zettabytes in 2052. For OEMs and companies from the broadly-defined automotive industry, this means one thing. Endless monetization possibilities. Providing that they face the challenges connected with data capture, filtering and storage, and become familiar with the in-vehicle technologies enabling that.  

The potential is enormous. However, the Capgemini report shows that there is still a long way ahead before reaching its full potential. Today, as many as 44% of OEM customers do not yet avail of any online service in their cars, and still,  connecting to the network is just the starting point because without the Internet there is no option of monetizing data. And even if the vehicle is already connected to the network, only every second driver declares frequent use of this type of service.

Anyway, the condition of the Internet is a challenge in itself. Today, in modern vehicles, there are around 100 points from which information can be downloaded (in the future it is estimated that there will be up to 10,000 of them!)

Before we get to know the technologies that enable it (about which we will write in the second part of the article), let's have a look at the challenges and checkpoints that must be considered when creating a data monetization strategy for a software-defined vehicle.

5 things to bear in mind if you want to monetize vehicle data

1. Developing the customer value proposition

This is where it all begins- from creating a sales offer and an environment in which drivers will believe you have something unique and valuable for them. Without trade, no technology will guarantee your success. Customers will simply not want to share data.

Think about the unique offer you want to present to them and develop a clear data management policy. As a result, it should be followed by the selection of appropriate technologies, and then their implementation in vehicles.

Obtaining data to offer the driver safety or a good sense of direction differs from getting information related to entertainment or directing the customer to a sale in a nearby shopping mall.

It would be perfect if the developed customer value proposition was consistent with your brand's DNA and features that have always been associated with it. This would make it easier to convince users, remain in line with your business assumptions, and stand out from the competition. Focus on technology application, not on technology just to be used.

2. Consider matching technology with the data for which users are most likely to "pay"

Speaking of users’ preferences, even today, at the stage when the technologies of obtaining data from vehicles are not fully-fledged yet, it can be seen that for some services customers are willing to give up some of their privacy, while they are largely opposed or reluctant towards others.

Capgemini's research shows that the group with the greatest potential includes services related to safety and facilitating driving:

•  hazard warning;
•  collision warning;
•  theft detection system;
•  e-call;
•  interactive language assistance.  
  

On the other hand, the greatest objection among users is aroused by services related to broadly -defined shopping:

•  In-car delivery;
•  in car e-commerce.

Keep this in mind when choosing technology to help you monetize your data.

3. Data collector strategy

The data in the vehicle is acquired by means of special sensors and then sent to collectors, which are supposed to gather this data and enable it to be transferred to the cloud. To effectively filter this data and derive maximum benefit from it, you need reliable technology to facilitate it. Due to the huge amount of data and the interaction between various sensors, the universal data collector is the best solution, as it collects all information obtained from sensors in the car.

In order to fully use its potential, during the implementation phase of this technology, it is crucial to ensure close work of the engineering team with people responsible for digital data management (see the next section). Close cooperation of both teams will help to obtain more interesting data and implement new services more efficiently.

4. Provider of IoT data platform

Collecting data from vehicles is impossible without an  IoT platform connected to cloud solutions dedicated to the  automotive industry - this is where data is sent and analyzed to be later collected by the vehicle sensors.

Regardless of which platform you choose (the most popular solutions on the market today are: Microsoft Azure, Amazon AWS, and Otonomo, operating in the SaaS system), 5 features that such a platform should have are of paramount importance to enable the efficient flow of information.

 You can read more about it in     our article on this issue    .

5. Data enrichment

While this article focuses on technologies directly related to obtaining data from the vehicle, it should not be overlooked that the software-defined vehicle operates in a wider ecosystem. Monetization of data from vehicles will not be possible without technologies related to infrastructure (e.g. smart-road infrastructure,  V2X communication , or high-speed data towers), as well as coordination of back-end processes for which entities such as policymaker, cybersecurity specialist, technical regulator, road infrastructure operator or billing/tolling player are accountable.

To create more valuable and attractive services, a coherent policy is necessary, as it will enrich the data stream from third parties and the user themselves, and will improve cooperation between elements of the ecosystem.

Checkpoints inside the car

In-car technologies are not the only gateway for data that companies can obtain from drivers (another entry point may be, for instance, the driver's smartphone or road infrastructure). However, they are the ones over which OEMs and manufacturers have the greatest control, technically at least.

Before we directly describe the technologies in the vehicle allowing that data to be obtained, let's focus on the  checkpoints that are crucial for the capture of information, its quality, and value for building services.

In the software-defined vehicle ecosystem, we can identify three such areas, a kind of bottleneck on which the flow of data depends. These are:

1.     Vehicle interior and infrastructure.  
2.     Connection to cloud.  
3.     Data cloud.  

Let's have a look at the first area, which is practically entirely the responsibility of the automotive company and is directly related to the equipment in the vehicle.

We can list the following groups of such checkpoints which require closer attention when building a data monetization strategy.

1. Gateway to the customer

Key points due to the start of data gathering and the user's experience - their willingness to share data, and thus increasing the value of the gathered data for the manufacturer.

•     HMI    (i.e. a set of technologies enabling the driver to activate the vehicle and begin collecting data, e.g. touch screens, visual sensors, voice commands, etc. - certainly a topic for a separate article)
•     Data gateway    (port, mobile data connection, USB port, radio connection)
•     Customer ID  

2. Points that build loyalty and the need to buy

That is, the contact points with the offer that allow you to easily download new applications, pay bills and influence the user's willingness to renew the service. The more transparent, engaging, and easy-to-use, the more likely the user is to continue their subscription.

•     App store / ecosystem  
•     Billing platform  
•     In-vehicle infotainment (IVI)  
•     Apps/ content  
   

3. Key points for data security, data analysis and usability

•     CPU/ control unit  
•     Car sensors / actuators  

Software-defined vehicles do not run in a vacuum

When creating a data monetization strategy for a software-defined vehicle, one should always bear in mind the wide ecosystem in which such a vehicle operates. It is not enough to equip it with the technology itself and wait for the flow of  data that will turn into specific value for the enterprise . In such a complex and extensive ecosystem, nothing happens by itself. There is no room for improvisation, omitting checkpoints, and presenting half-baked offers. Yes, the technology that downloads data from the vehicle is crucial, but it won't work unless we bear in mind the broader data management context that reaches beyond collecting and analyzing it.

Machine Learning combined with edge computing gains a lot of interest in industries leveraging AI at scale - healthcare, automotive, or insurance. The proliferation of use cases such as autonomous driving or augmented reality, requiring low latency, real-time response to operating correctly, made distributed data processing a tempting solution. Computation offloading to edge IoT devices makes the distributed cloud systems smaller - and in this case, smaller is cheaper. That’s the first most obvious benefit of moving machine learning from the cloud to edge devices.

 Why is this article worth reading? See what we provide here:

•     Explaining why regular ML training flow might not be enough.  
•     Presenting the idea behind federated learning.  
•     Describing the advantages and risks associated with this technology.  
•     Introducing technical architecture of a similar solution.  

How can federated learning be used in the automotive industry?

Using  the automotive industry as an example, modern cars already contain the edge device with processors capable of making complex computations. All ADAS (Advanced Driver Assistance Systems) and autonomous driving calculations happen on-board and require rather significant compute power. Detecting obstacles, road lanes, other vehicles, or road signs happens right now using onboard vehicle systems. That’s why collaboration with companies like  Nvidia becomes crucial for OEMs, as the need for better onboard SoCs does not stop.

Even though the prediction happens in the vehicle, the model is trained and prepared using regular, complex, and costly training systems built on-premises or in the cloud.  The training data grows bigger and bigger making the training process computationally expensive, slower, and requiring significant storage, especially if incremental learning is not used. The updated model may take time to be passed to the vehicle, and storing the user driving patterns, or even images from the onboard camera, requires both user consent and adherence to local law regulations.

The possible solution for that problem is using a local dataset from each vehicle as small, distributed training sets and training the model in the form of “federated learning”, where the local model is trained using smaller data batches and then aggregated into a singular global model. This is both more computational and memory efficient.

What are the benefits of federated learning?

One of the important concepts highly associated with machine learning at edge is building Federated Learning on top of edge ML. The combination of federated learning and edge computing gives important, measurable advantages:

•  Reduced training time - edge devices calculate simultaneously which improves velocity compared to a monolithic system.
•  Reduced inference time - compared to the cloud, at the edge inference results are calculated immediately.
•  Collaborative learning - instead of single, huge training dataset learning happens simultaneously using smaller datasets - which makes it both easier and more accurate enabling bigger training sets.
•  Always up-to-date model in vehicle - the new model is propagated to the vehicle after validation which makes the learning process of the network automatic.
•  Exceptional privacy - the omnipresent problem of secure channels for passing sensitive user data, anonymization, and storing personal user data for training purposes is now gone. The learning happens on local data in the edge device, and the data never leaves the vehicle. The weights which are being shared cannot be used to identify the user or even his driving patterns.
•  Lack of single point of failure - the data loss of the training set is not a threat.

Benefits from these concepts contain both cost savings and accuracy improved, visible as an overall better user experience when using the vehicle systems. As autonomous driving and ADAS systems are critical, better model accuracy is also directly associated with better security. For example, if the system can identify pedestrians on the road in front of vehicles with accuracy higher by  10%, it can mean that an additional 10% of collisions with pedestrians can be avoided. That is a measurable and important difference.

Of course, the solution does not come only with benefits. There are certain risks that have to be taken into account when deciding to transition to federated learning. The main one is that compared to the regular training mechanisms, federated learning is based on heterogeneous training data - disconnected datasets stored on edge devices. This means the global model accuracy is hard to control, as the global model is derived based on local models and changes dynamically.

This can be solved by building a hybrid solution, where part of the model is built using safe, predefined data, and it is gradually enhanced by federated learning. This brings both worlds closer together - amounts of data impossible to handle by a singular training system and stable model based on a verified training set.

Architectural overview

To build this kind of system, we need to start with the overall architecture. Key assumptions are that the infrastructure is capable of running distributed, microservices-based systems and has queueing and load balancing capabilities. Edge devices have some kind of storage, sensors, and SoC with CPU, and GPU capable of  training the ML model .

Let’s split it into multiple subsystems and consider them one by one:

1.  Swarm of connected vehicle edge devices, each one with connected sensors and ability to recalculate model gradient (weights.)
2.  Connection medium, in this case fast, 5G network available in the car
3.  Cloud connector, being a secure, globally available public API where each of the vehicle IoT edge devices connect to.
4.     Kubernetes cluster    with federated learning system split into multiple scalable microservices:

a) Gradient verification / Firewall - system rejecting the gradient that looks counterfeit - either manipulated by 3rd party or being based on fictional data.
b) Model aggregator - system merging the new weights into the existing model and creating an updated model.
c) Result verification automated test system - system verifying the new model on a predefined dataset with known predictions to score the model compared to the original.
d) Propagating queue connected to (S)OTA - automatic or triggered by user propagation of updated model in the form of an over-the-air update to the vehicle.

A firewall?

The firewall here, inside the system, is not a mistake. It is not guarding the network against attacks. It is guarding the model against being altered by cyber attacks.

Security is a very important aspect of AI, especially when the model can be altered by unverified data from the outside. There are multiple known attack vectors:

•  Byzantine attack - regarding the situation, when some of the edge devices are compromised and uploading wrong weights. In our case, it is unlikely for the attacker to be omniscient (to know the data of all participants), so the uploaded weights are either randomized but plausible, like generated Gaussian noise, or flip-bit of result calculation. The goal is to make the model unpredictable.
•  Model Poisoning - this attack is similar to the byzantine attack, but the goal is to inject the malicious model, which as a result alters the global model to misclassify objects. The dangerous example of such an attack is by injecting multiple fake vehicles into a model, which incorrectly identifies the trees as “stop” road signs. As a result, an autonomous car would not be able to operate correctly and stop near all trees as it would be a cross-section.
•  Data Poisoning - this attack is the hardest to avoid and easiest to execute, as it does not require a vehicle to be compromised. The sensor, for example, camera, is fed with a fake picture, which contains minor, but present changes - for example, a set of bright green pixels, like on the picture:

This can be a printed picture or even a sticker on a regular road sign. If the network learns to treat those four pixels as a “stop” sign. This can be painted, for example, on another vehicle and cause havoc on the road when an autonomous car encounters this pattern.

As we can see, those attacks are specific to distributed learning systems or machine learning in general. Taking this into account is critical, as the malicious model may be impossible to identify by looking at the weights or even prediction results if the way of attack was not determined.

There are multiple countermeasures that can be used to mitigate those attacks. Median or distance to the global model can be calculated and quickly identify rogue data. The other defense is to check the score of the global model after merging and revert the change if the score is significantly worse.

In both cases, the notification about the situation should be notified, both to operators as a metric and to a service that gives scores to the vehicle edge devices. If the device gets repeatedly flagged as wrong-doing, it should be kicked out of the network, and investigation is required to figure out if this is a cyberattack and who is the attacker.

Model aggregation and test

As we know, taking care of the cybersecurity threats specific to our use case, now the important step is merging the new weights with the global model.

There is no one best function or algorithm that can be used to aggregate the local models into global models by merging the individual results (weights). In general, very often average, or weighted average gives sufficient results to start with.

The Aggregation step is not final. The versioned model is then tested in the next step using the predefined data with automated verification. This is a crucial part of the system, preventing the most obvious faults - like the lane assist system stopping to recognize roadside lines.

If the model passes the test with a score at least as good as the current model (or predefined value), it’s being saved.

Over-the-air propagation

The last step of the pipeline is enqueueing the updated model to be propagated back to vehicles. This can be either an automatic process as in  continuous deployment directly to the car or may require a manual trigger if the system requires additional manual tests on the road.

A safe way of distributing the update is using the container image. The same image may be used for tests and then run in vehicles greatly reducing the chance of deploying failing updates. With this process, rollback is also simple as long as the device is able to store the previous version of the model.

The results

Moving from legacy, monolithic training method to federated learning gives promising results in both reduced overall system cost and improved accuracy. With quick expansion of 5G low-latency network and IoT edge devices into vehicles, this kind of system can move from theoretical discussions, scientific labs, and proofs of concepts to fully capable and robust production systems. The key part of building such a system is to consider the cybersecurity threats and crucial metrics like global model accuracy from the start.