Vehicle automation - where we are today and what problems we are facing

 Accidents, traffic congestion, lack of parking lots and poor state of roads. These are the 4 Horsemen of the Road Apocalypse that on occasion haunt cities around the globe. Have they come to settle in the largest agglomerations for good? Can AI in transportation combat them and make mobility smoother, more comfortable, and safer? Practical solutions introduced by the biggest transport companies from all over the world show that it is possible. And we do not have to wait for fully self-driving cars to use the advantages of AI. The changes are happening right before our eyes.

In 1900, the number of vehicles in the USA - the only country that produced cars at the time - reached 4192 vehicles. Today, the number of motor cars is estimated to be around 600 million, and with the current growth in production, this number is expected to double in the next 30 years. Our cities are congested, polluted and in many places getting around in a car during rush hour borders on the miraculous. Not to mention the real endurance test that drivers' nerves are put to.

Government agencies and shipping companies must explore solutions that reduce the number of vehicles in cities and equip urban infrastructure and cars with tools that effectively offset the side effects of technological globalization. The  Internet of Things and artificial intelligence are coming to the rescue to facilitate a new class of intelligent transportation systems (ITS), not only for  automotive but also for rail, marine, and aircraft transportation.

By  analyzing massive amounts of data from vehicles and  connecting the road infrastructure into a seamless network of information exchange , many aspects of transportation can be successfully addressed. The benefits of using AI in this market area are not only for cities and drivers but also for transport companies, pedestrians, and the environment. The whole transport ecosystem benefits from it, not just one of its constituent parts. We should all care about the development of these technologies and the broadest possible use of them in transport.

Thanks to the above-mentioned technologies, new trends are developing, such as micro-mobility,  shared mobility or, especially in the Netherlands and Scandinavia, the idea of mobility-as-a-service (MaaS), which encourages drivers to give up their own vehicle and exchange it for one in which transport is provided as a service.

Benefits of introducing AI in transportation

According to Market Data Forecast, the global transportation AI market will be worth around $3.87 billion by 2026 and is estimated to grow at a CAGR of 15.8% between 2021 and 2026. And it's no wonder that more and more businesses are embracing these solutions. The benefits of using AI technology in transportation are truly far-reaching and, indeed, their future is looking bright. With the development of data analytics and more modern sensors gathering information, new and innovative applications are bound to emerge.

Today, key benefits of using AI in transportation include:

•  increasing transportation safety;
•  detecting market trends;
•  relieving traffic congestion;
•  reducing greenhouse gas emissions, air pollution, and noise;
•  improved transportation design and management;
•  better management of urban space and reclaiming specific urban areas for residents;
•  analyzing travel needs and pedestrian behavior.

9 use cases of AI in transportation

When talking about using AI in transportation, self-driving cars are the most often mentioned examples that stir the imagination. Although such solutions have already been tested on the city streets (e.g. Waymo and Cruise in California) and occasionally we hear news about reaching by the manufacturer the highest (5th) level of automation, we are still a little away from the dissemination of vehicles that do not need any attention of the driver.

The main challenges faced by autonomous driving remain unchanged. First,  detection of objects on the road and their categorization, and second,  making the right decisions by the neural network, decision tree, or, in most cases, complicated hybrid model.

In 95% of cases, the neural network controlling the vehicles is already behaving correctly and making the best possible decisions. But there is still a marginal 5%, and this level is the most difficult to achieve at the moment. It simply takes time and more data to "train" a neural network. With the dropping price of LIDARs [light detection and ranging sensors], high resolutions camera, and the computing power of the GPUs [graphic processing units] increasing, it is only a matter of the next few years before this barrier is overcome - first in limited controlled areas (e.g. factories and harbors), the form of autonomized truck transport, and then using city vehicles.

Meanwhile, there are already more than a dozen advanced technologies on the road today that are  taking advantage of the AI ‘’goodies’’ and changing the way we control vehicle flow, driver safety, and driving behavior. Let's take a closer look at them.

1.Traffic detection & traffic signs

If traffic regulations were boiled down to one simple rule that even a few-year-old child could understand, red and green lights would definitely be second to none. Meanwhile, there are hundreds of road accidents each year related to running the red light and not stopping the vehicle at the right moment. Many factors contribute to this, such as driver fatigue, inclement weather, misuse of cell phones while driving, or simply rushing and time pressure.

People make mistakes and always will, these cannot be avoided. However, we have started to teach the machine to recognize traffic lights and eradicate such mistakes (the first attempts were made by BMW and Mercedes). With this technology, the braking system will react automatically when the driver tries to run a red light, and thus we can prevent disaster.

2. Pedestrian detection

The unpredictability of pedestrians and their different behavior on the road is one of the main factors holding back the mass introduction of autonomous cars. Thanks to computer vision,  AI already recognizes trees, unusual objects, and pedestrians without much of a struggle, and can warn drivers of a human approaching the roadway. The problem arises when a pedestrian is carrying groceries, holding a dog on a lead, or is in a wheelchair. Their unusual shape increases the difficulty for the machine to properly identify a human. Although it must be admitted that by using various object detection functions - based on motion, textures, shapes, or gradients - it is practically 100% successful.

However, the pedestrian's intention still remains a great challenge. Will he or she step onto the road or not? Are they only walking by the side of the road, or do they intend to cross it? These elements are always ambiguous and a neural network is needed to predict them effectively. To this end, the human pose estimation method comes in handy. It is based on the dynamics of the human skeleton and is capable of predicting human intentions in real-time.

3.Traffic Flow Analysis

Noise, smog, clogged city arteries,  stressed drivers, economic losses, greenhouse gas emissions - traffic congestion and vehicle crowding in cities give rise to numerous undesirable phenomena. AI can effectively help counteract all of them and make transportation much more efficient and convenient.

By relying on in-vehicle sensors, municipal CCTV cameras, and even drones to monitor vehicle flow, the algorithms can watch and keep track of the traffic both on highways and in the city. This allows them to warn drivers of potential congestion or accidents and direct the flow of vehicles in an efficient manner. It is also invariably useful for the town and urban planners involved in constructing new roads and improving the city's infrastructure. With prior traffic analysis and the vast amount of data available,  AI can identify the best planning solutions and help reduce undesirable situations right at the planning stage.

4.Inspection of dangerous turns, traffic circles and bike lanes

On a macro scale AI can help us change the entire road network, and on a micro-scale- a single intersection or traffic circle that needs repair. The analysis of the material provided by intelligent algorithms can calculate the trajectory of vehicles entering the bend, analyze the risk of potential conflicts between vehicles - pedestrians - cyclists, the speed at which vehicles enter the bend, or the waiting time at the traffic lights. The analysis of all this invaluable information can help optimize a given road section, and improve the safety and convenience of transport.

5.Computer Vision-Powered Parking Management

Entering the city center by car and finding a parking lot is often a struggle. If we connect the city's parking lots into an efficient network of sensors that monitor available spaces, the length of time vehicles are parked, and the hours when vehicles are most heavily congested, this key aspect of traffic can be greatly enhanced. With maps embedded in vehicles, AI can facilitate finding free parking spots, alert you to potential parking overcrowding, and - something actually pretty common - allow you to find your car when you forget where you parked it.

Such solutions are particularly useful in places such as airports, sports stadiums or arenas, where traffic must be smooth, and a high volume of visitors may pose a threat to safety.

6. Automated license plate recognition

A useful application of AI and computer vision is car license plate recognition. This type of technology is often used when entering highways, tunnels, ferries, or restricted areas constrained by gates or barriers. AI helps verify whether a given vehicle is on the list of registrations that, due to the fee paid or the drivers' status, are allowed to access a given area.

License plate recognition by algorithms is also a well-proven tool in the hands of the police and security services, who in this way are able to pinpoint the route of a particular vehicle or verify the driver's alibi.

7. Road condition monitoring

Each year potholes cause $3,000,000,000 worth of damage to vehicles in the U.S. alone. Intelligent algorithms can warn drivers of surprises lurking on the roads and monitor the condition of the road surface, so they can notify the authorities in advance of potential spots that will soon need fixing. This is enabled by linking the camera to ADAS, which applies machine learning to gather real-time information from the road surface where it is moving.

In this way, the driver can be warned not only of roadway damage but also of wet surfaces, ice, potholes or dangerous road debris. All of this improves safety for travelers, prevents accidents, and saves money - both in terms of drivers' finances and city funds.

8. Automatic Traffic Incident Detection

Video surveillance has been with us on the roads for ages, but it wasn't until the system was supported by AI solutions that it became possible to detect traffic incidents more efficiently, respond faster and provide information to traffic users practically in real-time.

By linking cameras within an ITS system, using computer vision technology, and equipping vehicles with intelligent sensors, we can detect different types of accidents. Intelligent algorithms save lives, prevent serious accidents and warn road users of hazardous situations by recommending safer travel options.

 The most commonly detected traffic incidents include:

•  pedestrians or animals entering on the road;
•  vehicles moving too fast or too slowly;
•  vehicles blocking the passage;
•  detection of debris on the road;
•  identification of vehicles moving in the wrong direction

9. Driver Monitoring

Finally, there is a full category of artificial intelligence solutions that apply directly in the interior of the car and affect the drivers themselves (we covered this in more depth  in this article ). Among them, three are particularly noteworthy:

•     driver’s fatigue monitoring    - by detecting the driver's face and estimating the position of the head, the system can detect drowsiness and emotions of the driver and thus prevent an accident.
•     alerts when the driver gets distracted    - for instance, when they reach for their cell phone, veer out of their lane, or turn around in the back seat to talk to fellow passengers.
•     emergency assist systems    - when the driver is not responsive and does not operate the vehicle, the car first tries to wake the driver by braking and pulling safety belts, and if it fails pulls over and calls emergency.

AI in transportation: setting the course for change

Given the speed at which computer processing power is changing and the number of sensors from which  data is being collected , fully automated cars on city roads are likely to be a question of the nearest 5-10 years. Change is happening at an exponential rate and today's applications of AI in transportation are just the first glimpse of the possibilities offered by intelligent algorithms. Change is essential and inevitable, e.g. due to the challenges facing the global community when it comes to global warming.

An increasing number of people live in cities, own not one but two vehicles, and want to travel to work or do their shopping in comfort. Transport companies and city managers must join forces with IT companies to fully tap into the potential of AI and change transport to be more efficient, environmentally friendly and suited to the way we want to use our cities. This is the only way we can make transportation sustainable and remove obstacles on the way to a zero-carbon economy and smart cities. Otherwise, we may face a vision of the future in which scientists predict traffic congestion 10 times worse than we experience today.

 When talking about data monetization in the automotive industry, we tend to focus on technology, safety, sensors, or cloud solutions. However, all these elements fade when confronted with the ultimate element - the driver of the vehicle. Without taking into account their needs and expectations, there can be no question of generating revenue. Any vehicle data monetization strategy must be mindful of this.

We can fine-tune the system, we can find exceptional partners to implement the software in the vehicle, but without a deep understanding of the vehicle user, no one will benefit from the solutions developed. Our organization will put a considerable amount of effort into building the team and implementing the technology, but the new vehicle features will not be used by the driver.

For this to happen, we need two factors: a value proposition of the brand- which explains clearly and transparently what the user will get out of it, and a coherent action strategy based on a market-back methodology that stems from specific market needs and allow us to develop services that are desired by the customer.

What benefits do customers most often look for in a software-defined vehicle?

Remember that just because people want to use a service, it doesn't mean that they will pay for it. What matters here is not just the benefit, but also the way it is presented, the user experience, and the pricing model. Only the combination of all these elements determines the success of the service. First of all, it is worth focusing on the benefits themselves and only then selecting the right technology to match them.

What are users willing to actually pay for and what are they willing to share only? Many studies indicate that the main factor motivating consumers to share data is gamification and rivalry - this aspect has not changed for years, as we can see for example in social media or e.g. "free" applications, which from time to time appear on the market, gather millions of interested users and vanish in no time. However, when it comes to paying for such "services", users are not so willing to use them.

In vehicles, it looks slightly different. Capgemini's research shows that the connected car services that are most popular with consumers are those related to the "core" functionality of vehicles, such as:

•     safety,  
•     driving comfort,  
•     time saving  
•     reduction of vehicle operating costs.  

Among them, however, the services that are most willingly paid for are:

•     hazard warning,  
•     collision warning,  
•     theft detection systems / vehicle finder.  

Of course, just because entertainment or gamification isn't on the list doesn't mean that automotive companies should avoid them. It's also a way to distinguish and find their own individual voice that corresponds to the broad brand strategy and allows them to stand out in the market. It's about the way they are served, presented to the consumer, and showing that they can actually derive real benefit from them.

It also works the opposite way. Simply creating a "hazard warning" service in a connected car does not immediately guarantee success. It still needs to be packaged properly, run smoothly, and be provided with a payment model that suits the consumer.

Examples of customized connected car services

In-vehicle ads based on navigation and user experience

Is it possible that a driver will like the ads that will be displayed in the car? If we adopt the message to their needs and preferences, in all likelihood, it is. For example, if we often go to McDonald’s, the navigation system can mark such places on our route. We have our favorite clothing brand, right? We will certainly react differently to a sale offer in a shopping mall we just happen to be driving past. The context of shopping and the consumer’s needs are decisive, and the software-defined vehicle is perfectly suited to ensuring that the advertising message is 100% tailored to the driver.

Contextual payments

Removing barriers to shopping and being able to buy everything everywhere is a popular trend in modern commerce. In a vehicle where the driver is focused on the road and has their hands full, such a service makes even more sense. With the development of voice assistants, drivers will be able to pay this way not only for fuel or tolls but also for purchases beyond typical vehicle-related payments. Voice shopping on the way back home from work, instead of looking for a parking space in front of the mall and returning in traffic jams in the evening? Why not?

Sharing information about driver behaviour

Sharing data about the way we drive may not appeal to everyone. But if in return for sharing this information, a company gives us a huge discount on our car insurance or a super attractive leasing offer, then things may take a totally different turn. In cooperation with an insurance company or a bank, such services become a specific bargaining chip the OEM can play with when dealing with the driver.

Manufacturer's connected car applications

Saving money on car maintenance and taking care of the overall condition of the car is a benefit that most drivers will appreciate. A practical and thoughtful manufacturer app that warns of potential breakdowns, component replacements, or servicing will allow the user to enjoy a well-functioning vehicle for longer and sell it at a higher profit. In this way, the OEM gets the driver used to have the vehicle repaired at an authorized service center, and the user, due to the loyalty shown to the brand, can expect future discounts and lucrative offers.

Practical use of telemetry

Sharing telemetry data may seem profitable only to OEMs - after all, as they draw better conclusions based on the collected information and save on R&D processes. However, it is important for companies to make vehicle users aware of the benefits of such services, as well. After all, driving style data can be used to suggest solutions that improve road safety, work on fuel efficiency or reduce overall vehicle operating costs. In each of these cases, the winner is the driver. Example? When a vehicle frequently skids and triggers the ESP/TC system, the system can suggest that the driver should get better tyres (by a specific brand, of course).

Unlocking extra features on the subscription model

Paying for heated seats, just to use them for three months a year, may not be worthwhile for everyone. Well-known to us from streaming portals, the subscription model definitely meets the users’ needs. The customers themselves choose which functionalities they want to pay for and over what period of time. The OEM only has to take care of the right vehicle software that will enable that. And, of course, be careful not to alienate those customers who see this as "yet another" way to squeeze additional payments out of them. That’s how manufacturers can provide both functionalities directly related to the vehicle itself - e.g. better lights or engine boost - as well as those associated with in-car entertainment providers such as Spotify or Apple CarPlay.

What can be done to make the user more eager to pay for data monetization services?

A well-thought-out user experience is essential

In today's digital world, UX and mobile-friendly approaches decide whether a service is viable. If the product is presented in an unclear and incomprehensible way, and it is difficult for the user to find the desired options - they will not use it. The size and color of buttons, the messages displayed, the stability of the application - all of the above is of paramount importance and determine the popularity of the product. Keeping in mind the latest trends, mapping the market, and adapting to consumer trends is necessary to offer the vehicle user service of the quality known to them from e-commerce or their own AppStore.

UX itself is not only a practical tool that helps better track consumer behavior and how they use the service, but also a constant theme to promote and boost brand interest. Does Apple really need to upgrade iOS every year and does Instagram have to offer users a new feed layout every quarter? The answer is obvious. It's simply profitable for the brand.

Start with anonymized data

When creating a strategy for in-vehicle data monetization efforts, it's a good idea to start by developing services that don't require the sharing of personal data. A lower "pain threshold" will make it quicker for the user to learn the benefits of the system and how convenient or useful the service can be. Thus, it will be easier to convince people to use products that require more openness to data sharing. And this may be the next step in the implementation of technological solutions.

Focus on heavy-vehicle users

People who spend most of their day in the car or drive long and demanding routes happily embrace any technical innovations designed to make driving easier and safer for them. It is this group that should be targeted at the beginning of developing your own data monetization model.

Minimizing risks and accurately selecting the group will not solve all challenges, but it will increase the chance of success and help gain a new, loyal group of consumers who will help transfer the technology to other users.

Last, but not least: a flexible payment model

Convenience should accompany the user at every stage of the use of a new service. Not only when it is most beneficial to the user, but also when it is easiest for the user to give it up: whilst paying for the next billing period.

It is worth taking care of the flexibility of the payment model (e.g. one-off payment, freemium model, annual or monthly settlement), adjusting it to the user's needs and not hindering payments.

The smoother and more tailored to the user's needs the whole process of interacting with the service is - from understanding the need to using it to making payments - the greater the chance that the stream of data flowing from a given vehicle will not dry up after a short period of use (read: being frustrated using an underdeveloped product for the first time).

Let's remember that data monetization can succeed provided that it really understands the user, is fair and transparent to them and focuses on user experience. If we didn't have time to get to know the customer's needs, why should they waste their time on services they don't understand and don't need?

 Connected Car ,  Software Development and  Autonomous Driving are the three most repeated words in the automotive industry. It’s hard not to notice that all three are basically different use cases heavily dependent on different kinds of software: cloud, AI, edge computing, or internal applications. Analysts, investors, management, and even regular employees of OEMs seem to believe and agree that software is the future of the automotive industry. But why?

Automotive software - how did we get there?

To understand the origins of this trend, let’s briefly look at the last 20 years of automotive history. On the market, where 99% of vehicles were based on combustion engines, a new entrant appeared. Tesla Motors Inc. A company with no background in building cars, named to pay tribute to the well-known electrical engineer, Nikola Tesla. A year later, famous entrepreneur, Elon Musk, decided to invest in this dream of building electric vehicles for the masses.

Fast forward to 2012 and we have the world premiere of the Tesla Model S. The Electric Vehicle, being the biggest disruption in the automotive industry in years, immediately receiving several automotive awards, including Car of The Year. Designed and developed by a company with 10 years of experience on the market and literally, a single vehicle developed earlier (the original Tesla Roadster). This showed that there is a big, unoccupied market for electric vehicles.

Just a year later, the Tesla Autopilot was introduced, and the whole world joined the hype for autonomous driving.

Why did Tesla get so popular?

It was not just because the market desperately needed an electric vehicle. Since the beginning, Tesla has been designing its cars to be software-centric. Big on-board CPUs from Nvidia support, not just Autopilot but also a multitude of applications and services available in the largest (at the time at least) central screen of a road car.

And the software has been updated very often using Over-The-Air upgrades,  giving the customers the feeling that the software was always fresh and the producer quickly reacted to feedback with new changes. Effectively, making the software a major selling point.

Electrification

Apart from the software-defined vehicle focus, electrification started as a solution to reduce the CO2 footprint of the industry. Both BEV and PHEV vehicles development was caused partially by new legislation and sustainability requirements, and partially of course by the success of Tesla. The EVs offering is increasing year by year, and most of the brands announced the potential timeline of reducing the combustion engines offering to 0 models.

The industry today

It seems like all of the large OEMs treated Tesla as their very own R&D department and allowed the company to conduct the world’s biggest ever market study. Tesla was able to prove that people actually care for the CO2 emission and want to drive electric cars and also showed that software in a vehicle may be more appealing to end-users than the sound of V8.

On the other hand, we compared Tesla to an R&D department because their cars are not always built with top quality and software sometimes have glitches – all in all, it’s a tremendous idea, but not an ideal car. VW Group, Toyota, or Stellantis could never afford to make such mistakes.

 Software defined-vehicles became a real future trend, not when Tesla S was first shown to the world. That happened when all of the world’s top OEMs decided that enough is enough, the experiment was over and the time to “productionize” Tesla’s “concept” had come.

And here we are today, a few days after Stellantis Software Day, an investor meeting purely focused on the Software-Defined Vehicles and their new platform, STLA (pronounced `Stella`). A few months after Mercedes-Benz announced that they are hiring developers to work on their own Operating System, MB.OS, as part of a greater “Digital First” brand strategy. A year after the CARIAD by Volkswagen Group was fully defined to provide unified software platforms for all vehicles in the group, called ODP (One Digital Platform) or VW.OS and VW.AC (VW Automotive Cloud).

Everyone is fully committed. But what exactly is the automotive industry committed to? Let’s dissect the latest event, Stellantis Software Day, to see the core topics they want to focus on in the next few years.

1.  Disconnecting hardware and software lifecycle.
2.  Broadening the scope of software in the vehicle.
3.  OTA software updates for adding new features.
4.  Using software to create a unique offering for all brands in the group.
5.     Connected Car data monetization    .
6.  Software to support EV and sustainability.

    #SWDAY21Stellantis    | Carlos Tavares, CEO: "We are transforming     #Stellantis    into a     #tech       #mobility    company. We owe it to our customers. We owe it to our Brands. We owe it to the principle on which     #Stellantis    was founded".     pic.twitter.com/iMYHSLpMwL    — Stellantis (@Stellantis)  December 7, 2021

Those are predicted to generate ~€20B in incremental annual revenues by 2030. That, of course, partially answers the “why?” question, but is there more to it?

Coming back to why

If we summarize the situation, we see that electrification and disruption forced the industry to change. The side effect of electrification is making the previous key differentiator - powertrain - much less important. With electric vehicles, the engines are not the key. Most of them are very similar and technology focuses more on batteries. This makes the different models similar, especially in terms of acceleration and horsepower.

So, where is the differentiator? Where do companies look for unique selling points for their brands, and how do they separate the offering of different models when the platform is almost exactly the same?

  https://twitter.com/Herbert_Diess/status/1469218343068614657

As you might have already guessed - that is the software. Of course, it’s not just electrification, the other key aspect is also digitalization of our lives, but the disruption already happened and the industry tries to follow.

The people fueling the future of automotive software

Certainly, when everyone decides at the same time to do a similar shift, it can get complicated rapidly. From the resourcing perspective, in the market with such a shortage of skilled software engineers, when everyone tries to quickly build their software competencies, it cannot come without problems. Hiring an experienced software developer is hard, and it gets harder if a company is fully focused on vehicle manufacturing, with a limited budget for IT and IT recruitment departments. The problems with building teams can result in delays in project start or extending their timeline.

This is where partnerships with companies like Grape Up come into play. Partnering with a software development company with strong experience in the automotive industry can help mitigate those issues - having skilled engineers available to help frame the project, architect, develop, and productionize significantly reduces the risk of shifting towards software development, and in the meantime also allows to train internal staff by working together, hands-on, on the actual projects.

The end

 We are an endangered species, you and me. We fans of speed, we devotees of power, we lovers of performance and beauty, and mechanical soul. We dare not speak of cams or cranks or double wishbones. We fear for our love of roaring V8s and the smell of burnt rubber. We're told to think of the economy, the environment, and not excitement and enjoyment. In an age of hybrid-this and automatic-that, we are the odd ones out. Yet there is hope. There is a haven. A place that celebrates speed, grip, gears, and fun. And it's all here for you to explore.

Jeremy Clarkson