Blend AI assistant concepts together

This series of articles starts with a general  chatbot description – what it is, how to deploy the model, and how to call it. The second part is about  tailoring – how to teach the bot domain knowledge and how to enable it to execute actions. Today, we’ll dive into the architecture of the application, to avoid starting with something we would regret later on.

To sum up, there are three AI assistant concepts to consider: simple chatbot, RAG, and function calling.

I propose to use them all at once. Let’s talk about the architecture. The perfect one may look as follows.

In the picture, you can see three layers. The first one, connectors, is responsible for session handling. There are some differences between various UI layers, so it’s wise to keep them small, simple, and separated. Members of this layer may be connected to a fast database, like Redis, to allow session sharing between nodes, or you can use a server-side or both-side communication channel to keep sessions alive. For simple applications, this layer is optional.

The next layer is the chatbot – the “main” application in the system. This is the application connected to the LLM, implementing the function calling feature. If you use middleware between users and the “main” application, this one may be stateless and receive the entire conversation from the middleware with each call. As you can see, the same application serves its capabilities both to employees and clients.

Let’s imagine a chatbot dedicated to recommending a car. Both a client and a dealer may use a very similar application, but the dealer has more capabilities – to order a car, to see stock positions, etc. You don’t need to create two different applications for that. The concept is the same, the architecture is the same, and the LLM client is the same. There are only two elements that differ: system prompts and the set of available functions. You can play it through a simple abstract factory pattern that will provide different prompts and function definitions for different users.

In a perfect world, the last layer is a set of microservices to handle different functions. If the LLM decides to use the function “store_order”, the “main” application calls the “store_order” function microservice that inserts data to an order database. Suppose the LLM decides to use the function “honk_and_flash” to localize a car in a crowded parking. In that case, the “main” application calls the “hong_and_flash” function microservice that handles authorization and calls a  Digital Twin API to execute the operation in the car. If the LLM decides to use a function “check_in_user_manual”, the “main” application calls the “check_in_user_manual” function microservice, which is… another LLM-based application!

And that’s the point!

 A side note before we move on – the world is never perfect so it’s understandable if you won’t implement each function as a separate microservice and e.g. keep everything in the same application.

The architecture proposed can combine all three AI assistant concepts. The “main” application may answer questions based on general knowledge and system prompts (“simple chatbot” concept) or call a function (“function calling” concept). The function may collect data based on the prompt (“RAG” concept) and do one of the following: call LLM to answer a question or return the data to add it to the context to let the “main” LLM answer the question. Usually, it’s better to follow the former way – to answer the question and not to add huge documents to the context. But for special use cases, like a long conversation about collected data, you may want to keep the document in the context of the conversation.

Which brings us to the last idea – mutable context. In general, each call contains the conversation history, including all data collected during the conversation, together with all available functions’ definitions.

 First prompt:

    System: You are a car seller, be nice to your customers    

    User: I’d like to buy a car  

    Functions: function1, function2, function3  

 Second prompt:

    System: You are a car seller, be nice to your customers    

    User: I’d like to buy a car  

    Assistant: call function1  

    Function: function1 returned data  

    Assistant: Sure, what do you need?  

    User: I’m looking for a sports car.  

    Functions: function1, function2, function3  

 Third prompt:

    System: You are a car seller, be nice to your customers    

    User: I’d like to buy a car  

    Assistant: call function1  

    Function: function1 returned data  

    Assistant: Sure, what do you need?  

    User: I’m looking for a sports car.  

    Assistant: I propose model A, it’s fast and furious  

    User: I like it!  

    Functions: function1, function2, function3  

You can consider a mutation of the conversation context at this point.

 Fourth prompt:

    System: You are a sports car seller, be nice to your customers    

    System: User is looking for a sports car and he likes model A  

    Assistant: Do you want to order model A?  

    Functions: function1, function2, function3, function4  

 You can implement a summarization function in your code to shorten the conversation, or you can select different subsets of all functions available, depending on the conversation context. You can perform both those tasks with the same LLM instance you use to make the conversation but with totally different prompts, e.g. “Summarize the conversation” instead of “You are a car seller”. Of course, the user won’t even see that your application calls the LLM more often than on user prompts only.

Pitfalls

All techniques mentioned in the series of articles may be affected by some drawbacks.

The first one is the  response time . When you put more data into the context, the user waits longer for the responses. It’s especially visible for voice-driven chatbots and may influence the user experience. Which means - it’s more important for customer-facing chatbots than the ones for internal usage only.

The second inhibition is  cost . Today, the 1000 prompt tokens processed by GPT-4-Turbo cost €0,01, which is not a lot. However, a complex system prompts together with some user data may, let’s say, occupy 20000 tokens. Let’s assume that the first question takes 100 tokens, the first answer takes 150 tokens, and the second question takes 200 tokens. The cost of the conversation is calculated as follows.

First prompt: common data + first question = 20000 [tokens] + 100 [tokens] = 2100 [tokens]

Second prompt: common data + first question + first answer + second question = 20000 [tokens] + 100 [tokens] + 150 [tokens] + 200 [tokens] = 20450 [tokens]

This two-prompts conversation takes 40550 tokens in total so far, which costs €0,41, excluding completions. Be aware that users may play with your chatbot running up the bill.

The last risk is the  security risk . In the examples in the  part 2 article , the chatbot uses an “unlock” function that accepts a VIN. You are obliged to assume that the chatbot will try to call a function with the VIN parameter of a car that does not belong to the customer. Of course, you must implement various LLM-hacking prevention mechanisms (prompt engineering, prompt filtering, answer filtering, etc.), but you can never assume that it is sufficient. Therefore, the developer’s role is to secure the functions (function calling) and data sources (RAG). For example, if a user tries to call a vehicle-related function, the user should be identified by the OAuth2 token, and the function microservice should reject a call with a VIN argument that doesn’t match the user profile.

Outline

In this series of articles, I’ve given you a few examples of how to utilize the most significant invention of 2023 for your automotive clients. You can write a chatbot that analyses customers’ requirements, helps mechanics, automates maintenance visit scheduling, seeks answers in user manuals, or writes personalized emails. Large language models are powerful and may be customized with a little effort. On the other hand, you need to keep in mind their limitations and common pitfalls. At the end of the day, even if the engine of the application is a black box, there is still a lot to implement around it to meet the business requirements.

The era of  LLM-based applications is started within the  automotive industry too.

Making the chatbot more suitable

Recently we’ve  built a simple chatbot , using an Azure-hosted GPT 4 Turbo instance. It’s a great starting point, but we definitely need more to fulfill any business requirements. Let’s try to talk with our bot.

      Prompt:    
 Hi, I’d like to buy a new car. Can you assist me with a choice? I’m looking for a family car with a hybrid powertrain and all-wheel drive. I’m a fan of modern design and fancy features. My budget is around 70k euro.  
    Completion:    
 Absolutely, I’d be happy to help you find a suitable family car that meets your criteria! Given your preferences (…), there are several options across different brands that could be a great fit. (…):  
 1. Audi Q5 TFSI e  
 Powertrain: (…)  
 Features: (…)  
 Price Range: (…)  
 2. BMW X5 xDrive45e  
 Powertrain: (…)  
 Features: (…)  
 Price Range: (…)  
 3. Volvo XC90 Recharge  
 Powertrain: (…)  
 Features: (…)  
 Price Range: (…)  
 4. Mercedes-Benz GLE 450 4MATIC  
 Powertrain: (…)  
 Features: (…)  
 Price Range: (…)  
 5. Lexus RX 450h  
 Powertrain: (…)  
 Features: (…)  
 Price Range: (…)  
 When considering these options, it’s important to test drive the ones you’re most interested in to get a feel for their driving dynamics, comfort, and the user-friendliness of their tech features. Additionally, (…)

As you can see, the response is quite long, versatile, and absolutely useless.

If you’re working for one of those brands, you don’t want the chatbot to offer your competitors. Also, the knowledge base is a bit outdated, so the answer is unreliable. Besides that, the answer is too long to be shown on a mobile device or to be read. And you need to be aware that the user may misuse the chatbot to generate a master thesis, write a fairy tale or plan a trip to Rome.

Our job is to limit the bot possibilities. The way to achieve it is prompt engineering. Let’s try to add some system messages before the user prompt.

Messages=[
       {"role": "system", "content": "You are a car seller working for X"},
       {"role": "system", "content": "X offers following vehicles (…)"},
       {"role": "system", "content": "Never recommend X competitors"},
       {"role": "system", "content": "Avoid topics not related to X. e.g. If the user asks for weather, kindly redirect him to the weather service"},
       {"role": "system", "content": "Be strict and accurate, avoid too long messages"},
       {"role": "user", "content": "Hi, I’d like to buy a new car. Can you assist me with a choice? I’m looking for a family car with a hybrid powertrain and all-wheel drive. I’m a fan of modern design and fancy features. My budget is around 70k euro."},
   ]

Now the chatbot should behave much better, but it still can be tricked. Advanced prompt engineering, together with LLM hacking and ways to prevent it, is out of the scope of this article, but I strongly recommend exploring this topic before exposing your chatbot to real customers. For our purposes, you need to be aware that providing an entire offer in a prompt (“X offers following vehicles (…)”) may go way above the LLM context window. Which brings us to the next point.

Retrieval augmented generation

You often want to provide more information to your chatbot than it can handle. It can be an offer of a brand, a user manual, a service manual, or all of that put together, and much more. GPT 4 Turbo can work on up to 128 000 tokens (prompt + completion together), which is, according to  the official documentation , around 170 000 of English words. However, the accuracy of the model decreases around half of it  [source] , and the longer context processing takes more time and consumes more money. Google has just  announced a 1M tokens model but generally speaking, putting too much into the context is not recommended so far. All in all, you probably don’t want to put there everything you have.

RAG is a technique of collecting proper input for the LLM that may contain information required to answer the user questions.

Let’s say you have two documents in your company knowledge base. The first one contains the company offer (all vehicles for sale), and the second one contains maintenance manuals. The user approaches your chatbot and asks the question: “Which car should I buy?”. Of course, the bot needs to identify user’s needs, but it also needs some data to work on. The answer is probably included in the first document but how can we know that?

In more detail, RAG is a process of comparing the question with available data sources to find the most relevant one or ones. The most common technique is vector search. This process converts your domain knowledge to vectors and stores them in a database (this process is called embedding). Each vector represents a piece of document – one chapter, one page, one paragraph, depending on your implementation. When the user asks his question, it is also converted to a vector representation. Then, you need to find the document represented by the most similar vector – it should contain the response to the question, so you need to add it to the context. The last part is the prompt, e.g. “Basic on this piece of knowledge, answer the question”.

Of course, the matter is much more complicated. You need to consider your embedding model and maybe improve it with fine-tuning. You need to compare search methods (vector, semantic, keywords, hybrid) and adapt them with parameters. You need to select the best-fitting database, polish your prompt, convert complex documents to text (which may be challenging, especially with PDFs), and maybe process the output to link to sources or extract images.

It's challenging but possible. See the result in one of our case studies:  Voice-Driven Car Manual .

Good news is – you’re not the first one working on this issue, and there are some out-of-the-box solutions available.

The no-code one is Azure AI Search, together with Azure Cognitive Service and Azure Bot.  The official manual covers all steps – prerequisites, data ingestion, and web application deployment. It works well, including OCR, search parametrization, and exposing links to source documents in chat responses. If you want a more flexible solution,  the low-code version is available here .

I understand if you want to keep all the pieces of the application in your hands and you prefer to build it from scratch. At this point we need to move back to the language opting. The Langchain library, which was originally available for Python only, may be your best friend for this implementation.

See the example below.

From langchain.chains.question_answering import load_qa_chain
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import Qdrant
from qdrant_client import QdrantClient
from langchain.chat_models import AzureChatOpenAI
from langchain.chains.retrieval_qa.base import RetrievalQA

client = QdrantClient(url="…", api_key="…")
embeddings = HuggingFaceEmbeddings(model_name="hkunlp/instructor-xl")
db = Qdrant(client= client, collection_name="…", embeddings=embeddings)
second_step = load_qa_chain(AzureChatOpenAI(
       deployment_name="…",
       openai_api_key="…",
       openai_api_base="…",
       openai_api_version="2023-05-15"
   ), chain_type="stuff", prompt="Using the context {{context}} answer the question: …")
first_step = RetrievalQA(
       combine_documents_chain=second_step,
       retriever=db.as_retriever(
           search_type="similarity_score_threshold", search_kwargs={"score_threshold": 0.5 }
       ),
   )
first_step.run()

This is the entire searching application. It creates and executes a “chain” of operations – the first step is to look for data in the Qdrant database, using a model called instructor-xl for embedding. The second step is to put the output of the first step as a “context” to the GPT prompt. As you can see, the application is based on the Langchain library. There is a Java port for it, or you can execute each step manually in any language you want. However, using Langchain in Python is the most convenient way to follow and a significant advantage of using this language at all.

With this knowledge you can build a chatbot and feed it with company knowledge. You can aim the application for end users (car owners), internal employees, or potential customers. But LLM can “do” more.

Function calling

To “do” is the keyword. In this section we’ll teach the LLM to do something for us, not only to provide information or  tell jokes . An operational chatbot can download more data if needed and decide which data is required for the conversation, but it can also execute real operations. Most modern vehicles are delivered with mobile applications that you can use to read data (localize the car, check the mileage, read warnings) or to execute operations (open doors, turn on air conditioning, or start charging process). Let’s do the same with the chatbot.

Function calling is a built-in functionality of GPT models. There is a field in the API model for tools (functions), and it can produce responses in a JSON format. You can try to achieve the same with any other LLM with a prompt like that.

In this environment, you have access to a set of tools you can use to answer the user's question.

You may call them like this:

<function_calls>
 <invoke>
   <tool_name>$TOOL_NAME</tool_name>
     <parameters>
       <$PARAMETER_NAME>$PARAMETER_VALUE</$PARAMETER_NAME>
     </parameters>
 </invoke>
</function_calls>

Here are the tools available:
<tools>
 <tool_description>
   <tool_name>unlock</tool_name>
   <description>
     Unlocks the car.
   </description>
   <parameters>
     <parameter>
       <name>vin</name>  
       <type>string</type>
       <description>Car identifier</description>
     </parameter>
   </parameters>
 </tool_description>
</tools>
This is a prompt from the user: ….

Unfortunately, LLMs often don’t like to follow a required structure of completions, so you might face some errors when parsing responses.

With the GPT, the official documentation recommends verifying the response format, but I’ve never encountered any issue with this functionality.

Let’s see a sample request with functions’ definitions.

{
 "model": "gpt-4",
 "messages": [
   { "role": "user",  "content": "Unlock my car" }
 ],
 "tools": [
   {
     "type": "function",
     "function": {
       "name": "unlock",
       "description": "Unlocks the car",
       "parameters": {
         "type": "object",
         "properties": {
           "vin": {
             "type": "string",
             "description": "Car identifier"
           },
         "required": ["vin"]
       }
     }
   }
 ],
}

To avoid making the article even longer, I encourage you to visit  the official documentation for reference.

If the LLM decides to call a function instead of answering the user, the response contains the function-calling request.

{
…
 "choices": [
   {
     "index": 0,
     "message": {
       "role": "assistant",
       "content": null,
       "tool_calls": [
         {
           "id": "call_abc123",
           "type": "function",
           "function": {
             "name": "unlock",
             "arguments": "{\"vin\": \"ABC123\"}"
           }
         }
       ]
     },
     "logprobs": null,
     "finish_reason": "tool_calls"
   }
 ]
}

Based on the  finish_reason value, your application decides to return the content to the user or to execute the operation. The important fact is – there is no magic that can be used to automatically call some API or execute a function in your code. Your application must find a function based on the name, and parse arguments from the JSON-formatted list. Then the response of the function should be sent to the LLM (not to the user), and the LLM makes the decision about next steps – to call another function (or the same with different arguments) or to write a response for the user. To send the response to the LLM, just add it to the conversation.

{
 "model": "gpt-4",
 "messages": [
   { "role": "user",  "content": "Unlock my car" },
   {"role": "assistant", "content": null, "function_call": {"name": "unlock", "arguments": "{\"vin\": \"ABC123\"}"}},
   {"role": "function", "name": "unlock", "content": "{\"success\": true}"}
 ],
 "tools": [
   …
 ],
}

In the example above, the next response is more or less “Sure, I’ve opened your car”.

With this approach, you need to send with each request not only the conversation history and system prompts but also a list of all functions available with all parameters. Keep it in mind when counting your tokens.

Follow up

As you can see, we can limit the chatbot versatility by prompt engineering and boost its resourcefulness with RAG or external tools. It brings us to another level of LLMs usability but now we need to meld it together and not throw the baby out with the bathwater. In  the last article we’ll consider the application architecture, plug some optimization, and evade common pitfalls. We’ll be right back!

Artificial Intelligence is everywhere - my laundry dryer is “powered by AI” (whatever it means), and I suppose there are some fridges on the market that take photos of their content to send you a shopping list and maybe even propose a recipe for your next dinner basing on the food you have. Some people say that generative AI and large language models (LLMs) are the most important inventions since the Internet, and we observe the beginning of the next industrial revolution.

However, household appliances and the newest history deliberation are not in our sphere of interest. The article about AI-based tools to support developers is getting old quickly due to the extremely fast development of new tools and their capabilities. But what can we, software makers, propose to our customers to keep up with the world changing?

Let’s talk about chatbots. Today, we try to break down an AI-driven personal assistants topic for the automotive industry. First, to create a chatbot, we need a language model.

The best-known LLM is currently OpenAI GPT4 that powers ChatGPT and thousands of different tools and applications, including a very powerful, widely available Microsoft Copilot. Of course, there are more similar models: Anthropic Claude with a huge context window, recently updated Google Bard, available for self-hosting Llama, code-completion tailored Tabnine, etc.

Some of them can give you a human-like conversation experience, especially combined with voice recognition and text-to-speech models – they are smart, advanced, interactive, helpful, and versatile. Is it enough to offer an AI-driven personal assistant for your automotive customers?

Well, as usual, it depends.

What is a “chatbot”?

The first step is to identify end-users and match their requirements with the toolkit possibilities. Let’s start with the latter point.

We’re going to implement a text-generating tool, so in this article, we don’t consider graphics, music, video, and all other generation models. We need a large language model that “understands” a natural language (or more languages) prompts and generates natural language answers (so-called “completions”).

Besides that, the model needs to operate on the domain knowledge depending on the use case. Hypothetically, it’s possible to create such a model from scratch, using general resources, like open-licensed books (to teach it the language) and your company resources (to teach it the domain), but the process is complex, very expensive in all dimensions (people, money, hardware, power, time, etc.) and at the end of the day - unpredictable.

Therefore, we’re going to use a general-purpose model. Some models (like gpt-4-0613) are available for fine-tuning – a process of tailoring the model to better understand a domain. It may be required for your use case, but again, the process may be expensive and challenging, so I propose giving a shot at a “standard” model first.

Because of the built-in function calling functionality and low price with a large context window, in this article, we use gpt-4-turbo. Moreover, you can have your own Azure-hosted instance of it, which is almost certainly significant to your customer privacy policy. Of course, you can achieve the same with some extra prompt engineering with other models, too.

OK, what kind of AI-driven personal assistant do you want? We can distinguish three main concepts: general chatbot, knowledge-based one, and one allowed to execute actions for a user.

Your first chatbot

Let’s start with the implementation of a simple bot – to talk about everything except the newest history.

As I’ve mentioned, it’s often required not to use the OpenAI API, but rather its own cloud-hosted model instance. To deploy one, you need an Azure account. Go to https://portal.azure.com/ , create a new resource, and select “Azure OpenAI”. Then go to your new resource, select “Keys and endpoints” from the left menu, and copy the endpoint URL together with one of the API keys. The endpoint should look like this one: https://azure-openai-resource-name.openai.azure.com/.

Now, you create a model. Go to “Model deployments” and click the “Manage deployments” button. A new page appears where you can create a new instance of the gpt-4 model. Please note that if you want to use the gpt-4-turbo model, you need to select the 1106 model version which is not available in all regions yet. Check this page to verify availability across regions.

Now, you have your own GPT model instance. According to Azure's privacy policy, the model is stateless, and all your data is safe, but please read the “Preventing abuse and harmful content generation” and “How can customers get an exemption from abuse monitoring and human review?” sections of the policy document very carefully before continuing with sensitive data.

Let’s call the model!

curl --location https://azure-openai-resource-name.openai.azure.com/openai/deployments/name-of-your-deployment/chat/completions?api-version=2023-05-15' \
--header 'api-key: your-api-key \
--header 'Content-Type: application/json' \
--data '{
"messages": [
{
"role": "user",
"content": "Hello!"
}
]
}'


The response should be like the following one.

{
"id": "chatcmpl-XXX",
"object": "chat.completion",
"created": 1706991030,
"model": "gpt-4",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! How can I assist you today?"
}
}
],
"usage": {
"prompt_tokens": 9,
"completion_tokens": 9,
"total_tokens": 18
}
}

Generally speaking, we’re done! You are making a conversation with your own chatbot. See the official documentation for a comprehensive API reference. Note that 2023-05-15 is the latest stable version of the API when I’m writing this text – you can use a newer preview version, or maybe there is a newer stable version already available.

However, using cURL is not the best user experience. Most tutorials propose using Python to develop your own LLM-based application. It’s a good advice to follow – Python is simple, offers SDKs for most generative AI models, and Langchain – one library to rule them all. However, our target application will handle more enterprise logic and microservices integration than LLM API integration, so choosing a programming language based only on this criterion may result in a very painful misusage in the end.

At this stage, I’ll show you an example of a simple chatbot application using Azure OpenAI SDK in two languages: Python and Java. Make your decision based on your language knowledge and more complex examples from the following parts of the article.

The Python one goes first.

from openai import AzureOpenAI

client = AzureOpenAI(
api_key='your-api-key',
api_version="2023-05-15",
azure_endpoint= ‘https://azure-openai-resource-name.openai.azure.com/'
)

chat_completion = client.chat.completions.create(
model=" name-of-your-deployment ",
messages=[
{"role": "user", "content": "Hello!"}
]
)

print(chat_completion.choices[0].message.content)

Here is the same in Java:

package demo;

import com.azure.ai.openai.OpenAIClientBuilder;
import com.azure.ai.openai.models.ChatCompletionsOptions;
import com.azure.ai.openai.models.ChatRequestUserMessage;
import com.azure.core.credential.AzureKeyCredential;

import java.util.List;

class Main {
public static void main(String[] args) {
var openAIClient = new OpenAIClientBuilder()
.credential(new AzureKeyCredential("your-api-key"))
.endpoint("https://azure-openai-resource-name.openai.azure.com/ ")
.buildClient();
var chatCompletionsOptions = new ChatCompletionsOptions(List.of(new ChatRequestUserMessage("Hello!")));
System.out.println(openAIClient.getChatCompletions("name-of-your-deployment", chatCompletionsOptions)
.getChoices().getFirst().getMessage().getContent());
}
}

One of the above applications will be a base for all you’ll build with this article.

User interface and session history

We’ve learnt how to send a prompt and read a completion. As you can see, we send a list of messages with a request. Unfortunately, the LLM’s API is stateless, so we need to send an entire conversation history with each request. For example, the second prompt, “How are you?”, in Python looks like that.

messages=[
{"role": "user", "content": "Hello!"},
{"role": "assistant", "content": "How can I help you"},
{"role": "user", "content": "How are you?"}
]

Therefore, we need to maintain the conversation history in our application, which brings us back to the user journey identification, starting with the user interface.

The protocol

The easy way is to create a web application with REST. The conversation history is probably shown on the page all the time, so it’s easy to send the entire history with each request from the frontend to the backend, and then from the backend to the LLM. On the other hand, you still need to add some system prompts to the conversation (we’ll discuss system prompts later) and sending a long conversation over the internet twice is a waste of resources. Moreover, LLMs may be slow, so you can easily hit a timeout for popular REST gateways, and REST offers just a single response for each request.

Because of the above, you may consider using an asynchronous communication channel: WebSocket or Server-Side Events. SSE is a one-way communication channel only, so the frontend still needs to send messages via the REST endpoint and may receive answers asynchronously. This way, you can also send more responses for each user query – for example, you can send “Dear user, we’re working hard to answer your question” before the real response comes from the LLM. If you don’t want to configure two communication channels (REST and SSE), go with WebSocket.

“I wish to know that earlier” advice: Check libraries availability for your target environment. For example, popular Swift libraries for WebSocket don’t support SockJS and require some extra effort to keep the connection alive.

Another use case is based on communicators integration. All companies use some communicators nowadays, and both Slack and Teams SDKs are available in many languages and offer asynchronous messaging. You can react to mentions, read entire channels, or welcome new members. However, some extra functionalities may be kind of limited.

Slack SDK doesn’t support “bot is typing” indicators, and Teams offers reading audio and video streams during meetings only for C# SDK. You should undeniably verify all the features you need availability before starting the integration. You need to consider all permissions you’ll need in your customer infrastructure to set up such a chatbot too.

The state

Regardless of what your frontend and communication channel are, you need to retain the history of the conversation. In a single-server environment, the job is easy – you can create a session-scope storage, perhaps a session-key dictionary, or a session Spring bean that stores the conversation. It’s even easier with both WebSocket and SSE because if the server keeps a session open, the session is sticky, and it should pass through any modern load balancer.

However, both WebSocket and SSE can easily scale up in your infrastructure but may break connections when scaling down – when a node that keeps the channel is terminated, the conversation is gone. Therefore, you may consider persistent storage: a database or a distributed cache.

Speech-to-text and text-to-speech

Another piece of our puzzle is the voice interface. It’s important for applications for drivers but also for mechanics who often can’t operate computers with busy (or dirty) hands.

For mobile devices, the task is easy – both iOS and Android offer built-in speech-to-text recognition and text-to-speech generation as a part of accessibility mechanisms, so you can use them via systems’ APIs. Those methods are fast and work on end devices, however their quality is discussable, especially in non-English environments.

The alternative is to use generative AI models. I haven’t conducted credible, trustworthy research in this area, but I can recommend OpenAI Whisper for speech-to-text and Eleven Labs for text-to-speech. Whisper works great in noisy environments (like a car riding on an old pavement), and it can be self-hosted if needed (but the cloud-hosted variant usually works faster). Eleven Labs allows you to control the emotions and delivery of the speaker. Both work great with many languages.

On the other hand, using server-side voice processing (recognition and generation) extends response time and overall processing cost. If you want to follow this way, consider models that can work on streams – for example, to generate voice at the same time when your backend receives the LLM response token-by-token instead of waiting for the entire message to convert.

Additionally, you can consider using AI talking avatars like Synthesia, but it will significantly increase your cost and response time, so I don’t recommend it for real-time conversation tools.

Follow up

This text covers just a basic tutorial on how to create bots. Now you know how to host a model, how to call it in three ways and what to consider when designing a communication protocol. In following parts of the article series, we’ll add some domain knowledge to the created AI-driven personal assistant and teach it to execute real operations. At the end, we’ll try to summarize the knowledge with a hybrid solution , we’ll look for the technology weaknesses and work on the product optimization.

In the earlier article , we’ve covered the detailed architecture of a fleet management system based on AWS IoT and on-edge virtual networks. Now, we can dive into implementation. Let’s create a prototype of an edge network with two applications running together on both virtual and physical nodes and with isolated virtual networks. As we don’t have fire trucks on hand, we use three computers (the main truck ARM computer simulated by a Raspberry Pi and two application nodes running on laptops) and a managed switch to connect them together.

Overview of topics and definitions used in the material

In this chapter, we provide concise explanations of key networking concepts and technologies relevant to the architecture discussed earlier. These definitions will help readers better understand the underlying mechanisms that enable efficient and flexible communication between Docker containers, the host system, and external devices. Familiarizing yourself with these concepts will facilitate a deeper understanding of the networking aspects of the presented system and their interrelationships.

• Docker Networking : a system that enables containers to communicate with each other and external networks. It provides various network drivers and options to support different network architectures and requirements, including bridge, host, overlay, and IPvlan/MACvlan drivers. Docker networking creates virtual networks and attaches containers to these networks. Each network has a unique IP address range, and containers within a network can communicate using their assigned IP addresses. Docker uses network drivers to manage container connectivity and network isolation.
• IPvlan : a Docker network driver that enables efficient container-to-container and container-to-external network communication by sharing the parent interface's MAC address with its child interfaces. In the context of the router Docker image in the presented topic, IPvlan provides efficient routing between multiple networks and reduces the management overhead associated with MAC addresses.
• Docker Bridge : a virtual network device that connects multiple Docker container networks, allowing containers to communicate with each other and the host system. By default, Docker creates a bridged network named "docker0" for containers to use. Users can create custom bridge networks to segment and isolate container traffic.
• Linux Bridge : a kernel-based network device that forwards traffic between network segments. It operates at the data link layer, similar to how ethernet frames function within the TCP/IP model. Linux Bridges are essential in creating virtual network interfaces for entities such as virtual machines and containers.
• veth : (Virtual Ethernet) a Linux kernel network device that creates a pair of connected virtual network interfaces. Docker uses veths to connect containers to their respective networks, with one end attached to the container and the other end attached to the network's bridge. In a bridged Docker network, veth pairs are created and named when a container is connected to a Docker bridge network, with one end of the veth pair being assigned a unique identifier within the container's network namespace and the other end being assigned a unique identifier in the host's network namespace. The veth pair allows seamless communication between the container and the bridge network. In simple words – veth is a virtual cable between (virtual or not) interfaces of the same machine.
• Network namespace : Docker provides containers with isolated network stacks, ensuring each container has its own private IPs and ports. VLANs (Virtual Local Area Networks) operate at the data link layer, allowing for the creation of logically segmented networks within a physical network for improved security and manageability. When combined in Docker, containers can be attached directly to specific VLANs, marrying Layer 2 (VLAN) and Layer 3 (namespaces) isolation.
• VLAN (Virtual Local Area Network) : a logical network segment created by grouping physical network devices or interfaces. VLANs allow for traffic isolation and efficient use of network resources by separating broadcast domains.
• iptable : a Linux command-line utility for managing packet filtering and network address translation (NAT) rules in the kernel's network filter framework. It provides various mechanisms to inspect, modify, and take actions on packets traversing the network stack.
• masquerade : a NAT technique used in iptables to mask the source IP address of outgoing packets with the IP address of the network interface through which the packets are being sent. This enables multiple devices or containers behind the masquerading device to share a single public IP address for communication with external networks. In the context of the presented topic, masquerading can be used to allow Docker containers to access the Internet through the router Docker image.

Solution proposal with description and steps to reproduce

Architecture overview

The architecture described consists of a Router Docker container, two applications’ containers (Container1 and Container2), a host machine, and two VLANs connected to a switch with two physical devices. The following is a detailed description of the components and their interactions.

Router Docker container

The container has three interfaces:

• eth0 (10.0.1.3): Connected to the br0net Docker network (10.0.1.0/24).
• eth1 (192.168.50.2): Connected to the home router and the internet, with the gateway set to 192.168.50.1.
• eth2 (10.0.2.3): Connected to the br1net Docker network (10.0.2.0/24).

Docker containers

Container1 (Alpine) is part of the br0net (10.0.1.0/24) network, connected to the bridge br0 (10.0.1.2).

Container2 (Alpine) is part of the br1net (10.0.2.0/24) network, connected to the bridge br1 (10.0.2.2).

Main edge device – Raspberry Pi or a firetruck main computer

The machine hosts the entire setup, including the router Docker image and the Docker containers (Container1 and Container2). It has two bridges created: br0 (10.0.1.2) and br1 (10.0.2.2), which are connected to their respective Docker networks (br0net and br1net)

VLANs and switch

The machine’s bridges are connected to two VLANs: enp2s0.1 (10.0.1.1) and enp2s0.2 (10.0.2.1). The enp2s0 interface is configured as a trunk connection to a switch, allowing it to carry traffic for multiple VLANs simultaneously.

Two devices are connected to the switch, with Device1 having an IP address of 10.0.1.5 and Device2 having an IP address of 10.0.2.5

DHCP Server and Client

Custom DHCP is required because of the IP assignment for Docker containers. Since we would like to maintain consistent addressing between both physical and virtual nodes in each VLAN, we let DHCP handle physical nodes in the usual way and assign addresses to virtual nodes (containers) by querying the DHCP server and assigning addresses manually to bypass the Docker addressing mechanism.

In short - the presented architecture describes a way to solve the non-trivial problem of isolating Docker containers inside the edge device architecture. The main element responsible for implementing the assumptions is the Router Docker container, which is responsible for managing traffic inside the system. The Router isolates network traffic between Container1 and Container2 containers using completely separate and independent network interfaces.

The aforementioned interfaces are spliced to VLANs via bridges, thus realizing the required isolation assumptions. The virtual interfaces on the host side are already responsible for exposing externally only those Docker containers that are within the specific VLANs. The solution to the IP addressing problem for Docker containers is also worth noting. The expected result is to obtain a form of IP addressing that will allow a permanent address assignment for existing containers while retaining the possibility of dynamic addressing for new components.

The architecture can be successfully used to create an end-to-end solution for edge devices while meeting strict security requirements.

Step-by-step setup

Now we start the implementation!

VLANs [execute on host]

Let’s set up VLANs.

enp2s0.1
auto enp2s0.1
iface enp2s0.1 inet static
address 10.0.1.1
network 10.0.1.0
netmask 255.255.255.0
broadcast 10.0.1.255

enp2s0.2
auto enp2s0.2
iface enp2s0.2 inet static
address 10.0.2.1
network 10.0.2.0
netmask 255.255.255.0
broadcast 10.0.2.255


Bridges [execute on host]

We should start by installing bridge-utils, a very useful tool for bridge setup.

sudo apt install bridge-utils

Now, let’s config the bridges.

sudo brctl addbr br0
sudo ip addr add 10.0.1.1/24 dev br0
sudo brctl addif br0 enp2s0
sudo ip link set br0 up

sudo brctl addbr br1
sudo ip addr add 10.0.2.1/24 dev br0
sudo brctl addif br0 enp2s0
sudo ip link set br0 up

Those commands create virtual brX interfaces, set IP addresses, and assign physical interfaces. This way, we bridge physical interfaces with virtual ones that we will create soon – it’s like a real bridge, connected to only one river bank so far.

Docker networks [execute on host]

Network for WLAN interface.

docker network create -d ipvlan --subnet=192.168.50.0/24 --gateway=192.168.50.1 -o ipvlan_mode=l2 -o parent=wlp3s0f0 wlan

Network for bridge interface br0.

docker network create --driver=bridge --subnet=10.0.1.0/24 --gateway=10.0.1.2 --opt "com.docker.network.bridge.name=br0" br0net

Network for bridge interface br1.

docker network create --driver=bridge --subnet=10.0.2.0/24 --gateway=10.0.2.2 --opt "com.docker.network.bridge.name=br1" br1net

Now, we have empty docker networks connected to the physical interface (wlp3s0f0 – to connect containers the Internet) or bridges (br0net and br1net – for VLANs). The next step is to create containers and assign those networks.

Docker containers [execute on host]

Let’s create the router container and connect it to all Docker networks – to enable communication in both VLANs and the WLAN (Internet).

docker create -it --cap-add=NET_ADMIN --cap-add=SYS_ADMIN --cap-add=NET_BROADCAST --network=br0net --sysctl net.ipv4.icmp_echo_ignore_broadcasts=0 --ip=10.0.1.3 --name=router alpine
docker network connect router wlan
docker network connect router br1net

Now, we create applications’ containers and connect them to proper VLANs.

docker create -it --cap-add=NET_ADMIN --cap-add=SYS_ADMIN --cap-add=NET_BROADCAST --network=br0net --sysctl net.ipv4.icmp_echo_ignore_broadcasts=0 --name=container1 alpine

docker create -it --cap-add=NET_ADMIN --cap-add=SYS_ADMIN --cap-add=NET_BROADCAST --network=br1net --sysctl net.ipv4.icmp_echo_ignore_broadcasts=0 --name=container2 alpine

OK, let’s start all containers.

docker start router
docker start container1
docker start container2


Now, we’re going to configure containers. To access Docker images’ shells, use the command

docker exec -it <image_name> sh.

Router container setup [execute on Router container]

Check the interface’s IP addresses. The configuration should be as mentioned below.

eth0 Link encap:Ethernet HWaddr 02:42:0A:00:01:03
inet addr:10.0.1.3 Bcast:10.0.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:745 errors:0 dropped:0 overruns:0 frame:0
TX packets:285 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:142276 (138.9 KiB) TX bytes:21966 (21.4 KiB)

eth1 Link encap:Ethernet HWaddr 54:35:30:BC:6F:59
inet addr:192.168.50.2 Bcast:192.168.50.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4722 errors:0 dropped:0 overruns:0 frame:0
TX packets:1515 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3941156 (3.7 MiB) TX bytes:106741 (104.2 KiB)

eth2 Link encap:Ethernet HWaddr 02:42:0A:00:02:01
inet addr:10.0.2.3 Bcast:10.255.255.255 Mask:255.0.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:829 errors:0 dropped:0 overruns:0 frame:0
TX packets:196 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:190265 (185.8 KiB) TX bytes:23809 (23.2 KiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:60 errors:0 dropped:0 overruns:0 frame:0
TX packets:60 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5959 (5.8 KiB) TX bytes:5959 (5.8 KiB)

Then let’s set up the iptables. You can omit the first command if the iptables package is already installed. The second command configures the masquerade, and the rest of them configure routing rules.

apk add ip6tables iptables
iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
iptables -P INPUT ACCEPT
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT
iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT
iptables -A FORWARD -i eth1 -o eth2 -j ACCEPT
iptables -A FORWARD -i eth2 -o eth1 -j ACCEPT
iptables -A FORWARD -i eth0 -o eth1 -j ACCEPT

Now, we hide the internal addresses of outgoing packets (the masquerade), and the networks are isolated. Please note that there is no routing configured on the host machine, and it’s not even a gateway for both containerized and physical network nodes.

In the mentioned config, test containers will communicate with the external environment by physical interfaces on the router container, and in other words – they will be exposed to the Internet by the Docker router container. Router, in addition to enabling communication between VLANs and the Internet, may also allow communication between VLANs or even specific VLAN nodes of different VLANs. Thus, this container has become the main routing and filtering point for network traffic.

Container1 setup [execute on Container1]

route del default
ip route add default via 10.0.1.3

Container2 setup [execute on Container2]

route del default
ip route add default via 10.0.2.3

As you can see, the container configuration is similar; all we need to do is set up the default route via the router container instead of the docker-default one. In the real-world scenario, this step should be done via the DHCP server.

Switch setup [execute on the network switch]

The configuration above requires a manageable switch. We don’t enforce any specific model, but the switch must support VLAN tagging on ports with the trunk option for a port that combines traffic for multiple VLANs. The configuration, of course, depends on the device. Pay attention to the trunk port of the device, which is responsible for traffic from the switch to our host. In our case, the device1 is connected to a switch port tagged as VLAN1, and the device2 is connected to a switch port tagged as VLAN2. The enp2s0 port of the host computer is connected to a switch port configured as a trunk - to combine traffic of multiple VLANs in a single communication link.

Summary

We’ve managed together to conduct the network described in the first article. You can play with the network with ICMP to verify which nodes can access each other and, more importantly, which nodes can’t be reached outside their virtual networks.

Here is a scenario for the ping test. The following results prove that the created architecture fulfills its purpose and achieves the required insulation.

Source Target Ping status Explanation Container1 Device1 OK VLAN 1 Device1 Container1 OK VLAN 1 Container1 Router (10.0.1.3) OK VLAN 1 Device1 Router (10.0.1.3) OK VLAN 1 Container1 Internet (8.8.8.8) OK VLAN 1 to Internet via Router Device1 Internet (8.8.8.8) OK VLAN 1 to Internet via Router Router Container1 OK VLAN 1 Router Device1 OK VLAN 1 Container1 Container2 No connection VLAN 1 to VLAN 2 Container1 Device2 No connection VLAN 1 to VLAN 2 Container1 Router (10.0.2.3) No connection VLAN 1 to VLAN 2 Device1 Container2 No connection VLAN 1 to VLAN 2 Device1 Device2 No connection VLAN 1 to VLAN 2 Device1 Router (10.0.2.3) No connection VLAN 1 to VLAN 2 Container2 Device2 OK VLAN 2 Device2 Container2 OK VLAN 2 Container2 Router (10.0.2.3) OK VLAN 2 Device2 Router (10.0.2.3) OK VLAN 2 Container2 Internet (8.8.8.8) OK VLAN 2 to Internet via Router Device2 Internet (8.8.8.8) OK VLAN 2 to Internet via Router Router Container2 OK VLAN 2 Router Device2 OK VLAN 2 Container2 Container1 No connection VLAN 2 to VLAN 1 Container2 Device1 No connection VLAN 2 to VLAN 1 Container2 Router (10.0.1.3) No connection VLAN 2 to VLAN 1 Device2 Container1 No connection VLAN 2 to VLAN 1 Device2 Device1 No connection VLAN 2 to VLAN 1 Device2 Router (10.0.1.3) No connection VLAN 2 to VLAN 1

As you can see from the table above, the Router container is able to send traffic to both networks so it’s a perfect candidate to serve common messages, like GPS broadcast.

If you need more granular routing or firewall rules, we propose to use firewalld instead of iptables . This way, you can disable non-encrypted traffic or open specific ports only.

Nevertheless, the job is not over yet. In the next article , we’ll cover IP addresses assignment problem, and run some more sophisticated tests over the infrastructure.

Remote control over a vehicle is not a new idea; it has become a trend in the modern world. The idea of vehicle remote control is highly connected with vehicle telematics , which is used to gather data using GPS technology, sensors, and onboard diagnostics codes. Managed data can include vehicle location, driver behavior, vehicle activity, and real-time engine diagnostics. Further, the technology is managed on software platforms that help fleet owners manage their assets remotely.

The data collected often includes vehicle location, fuel consumption, speeding, and maintenance information. But what if a car got stolen? It would be helpful not only to have the current location but also to send a command to a vehicle to turn the engine off or enable signalization. A huge problem in this case is to have a platform that will be independent of the Original Equipment Manufacturer (OEM). The best option here is to move the solution to the Cloud and introduce a single point of work with vehicles and integration with different OEMs.

Such telematics remote control gives a powerful opportunity to car-rental or car-sharing services like CityBee or SnapCar. They now can track their cars and also offer customers a no-human way to reserve a car using just a mobile application and know the current vehicle state when it is in use.

Vehicle connection

To establish connection to a car it’s necessary to equip the vehicle with an Internet-connected telemetry device. Some companies provide such a device as part of machine installation, but third-party tools like Geotab can be used for custom installation as well. It is important to have a device intended for two-way connection with a vehicle, as some solutions were created only for tracking purposes. Remote vehicle controllers like Mobokey offer the following commands on a vehicle:

• Lock/Unlock
• Turn engines on/off
• Klaxon sound
• Turn flash-lights on/off
• Wake up/sleep
• Connect/Disconnect the vehicle

Some manufacturers require their cars to be woken up explicitly before executing actual commands. It is intended to prevent the continuous connection or in case of low battery level.

Software solution

Once the two-way connection with a vehicle is established, we need a solution to track the actual status of a car and send commands. To do that, it is required:

• To make sure integration with a vehicle is completed successfully
• That the connection is secure
• That the command can be sent by us – it may differ depending on the manufacturer and model

and that we prepared:

• Dashboard to send commands
• Response handling - to be aware if command execution was successful

The damage caused by a successful attack on a vehicular telematics system because of an unsecured connection may range from mild to disastrous - to the extent of serious casualties and property losses. To enforce the security objectives for deployed and developing vehicular monitoring systems, embodiments of the disclosed technology include a stack of security approaches, both on physical and software levels.

As vehicle commands are sent over the Internet, it is important to have a network and infrastructure security built. The software solution stack must be enclosed in a Virtual Private Cloud (VPC). Apart from that, it is highly recommended to apply some best practices such as Static Application Security Testing (SAST), Interactive Application Security Testing (IAST), and Software Composition Analysis (SCA) to find vulnerabilities.

Another challenge in software solutions relates to the need for integration with different OEMs . Each OEM provides an API to integrate with and different ways of communication with vehicles - it may be a synchronous way, for example, HTTP request to REST API, or an asynchronous way, for instance, using queue-based protocols like MQTT.

Another issue is handling command execution responses or callbacks . The easiest way to implement this is when the OEM API synchronously responds with a command execution result, but in most cases, the OEM system may notify us about the execution result eventually in some time.

In this case, it is necessary to find a way to map a command request to a vehicle via OEM API and an execution response as it is used for retry policy, error handling, and troubleshooting.

Software architecture

This connected vehicle solution uses the IoT platform, which authenticates messages from connected vehicles and processes data according to business rules. It leverages a set of main services, including Vehicle Services to connect vehicles and store vehicle-related data, a Telemetry stack to collect a delivery stream of input events and write them into Database, Command Services to send commands to the car and combine execution responses; and a queue-based topic which is intended for inter-communication between different parts of the system.

The solution also includes integration with OEM APIs. When IoT receives a message, it authenticates and authorizes the message, and the Command Platform executes the appropriate rule on the message, which routes the message to the appropriate OEM integration.

Here we see a potential OEM integration with the IoT Platform. It has authorization integration to allow us to send request OEM API securely; Callback integration to keep OEM response data regarding command execution; Database to keep mapping and consistent result - command request vs response; retry mechanism implementation using polling results from the database.

Once the system is authenticated, requests can be submitted to the connected vehicle solution’s OEM APIs. Based on the request identification data, the system eventually waits for the command result using a callback mechanism.

Conclusion

As highly-equipped connected vehicles increasingly rely on data exchanged with the infrastructure, it is required to have sustainable infrastructure, well-built cyber-security, privacy, and safety taken into account. The proposed solution also pays respect to the need to enroll in this process vehicle from different manufacturers.

This solution with remote vehicle control may be extremely useful for car-sharing systems, and apart from that, it can cover a solution for such use cases as autonomous and semi-autonomous vehicle driving, usage-based insurance, and customized in-vehicle experience . The solution also includes two-way communication.

Welcome to a short cycle of articles that shows a way to combine network techniques and AWS services for a mission-critical automotive system .

We’ll show you how to design and implement an IoT system with a complex edge architecture.

The cycle consists of three articles and shows the architecture design, a step-by-step implementation guide, and some pitfalls with the way to overcome these.

Let’s start!

AWS IoT usage to manage vehicle fleet

Let’s create an application. But this won’t be a typical, yet another CRUD-based e-commerce system. This time, we’d like to build an IoT-based fleet-wise system with distributed (on-edge/in-cloud) computing.

Our customer is an automotive company that produces fire trucks. We’re not interested in engine power, mechanical systems, and firefighters' equipment. We’re hired to manage the fleet of vehicles for both the producer and its customers.

Each truck is controlled by a central, “rule-them-all” computer connected to all vehicles CAN buses, and whole extra firefighters’ equipment. The computer sends basic vehicle data (fuel level, tire pressure, etc.) to the fire station and a central emergency service supervisor. It receives new orders, calculates the best route to targets and controls all the vehicle equipment - pumps, lights, signals, and of course – the ladder. Also, it sends some telemetry and usage statistics to the producer to help design even better trucks in the future.

However, those trucks are not the same. For instance, in certain regions, the cabin must be airtight, so extra sensors are used. Some cities integrate emergency vehicles with city traffic light systems to clear the route for a running truck. Some stations require specialized equipment like winches, extra lights, power generators, crew management systems, etc.

Moreover, we need to consider that those trucks often operate in unpleasant conditions, with a limited and unreliable Internet connection available.

Of course, the customer would like to have a cloud-based server to manage everything both for the producer and end users - to collect logs and metrics with low latency, to send commands with no delay, and with a colorful, web-based, easy-to-use GUI.

Does it sound challenging? Let's break it down!

Requirements

Based on a half-an-hour session with the customer, we've collected the following, a bit chaotic, set of business requirements:

• a star-like topology system, with a cloud in the center and trucks around it,
• groups of trucks are owned by customers - legal entities that should have access only to their trucks,
• each group that belongs to a customer may be customized by adding extra components, both hardware-, or software-based,
• each truck is controlled by identical, custom, Linux-based computers running multiple applications provided by the customer or third parties,
• truck-controlling computers are small, ARM-based machines with limited hardware and direct Internet access via GSM,
• Internet connection is usually limited, expensive, and non-reliable,
• the main computer should host common services, like GPS or time service,
• some applications are built of multiple components (software and hardware-based) - hardware components communicate with the main computers via the in-vehicle IP network,
• the applications must communicate with their servers over the Internet, and we need to control (filter/whitelist) this traffic,
• each main computer is a router for the vehicle network,
• each application should be isolated to minimize a potential attack scope,
• components in trucks may be updated by adding new software or hardware components, even after leaving the production line,
• the cloud application should be easy - read-only dashboards, truck data dump, send order, both-way emergency messages broadcast,
• new trucks can be added to the system every day,
• class-leading security is required - user and privileges management, encrypted and signed communication, operations tracking, etc.
• provisioning new vehicles to the system should be as simple as possible to enable the factory workers to do it.

As we’ve learned so far, the basic architecture is as shown in the diagram below.

Our job is to propose a detailed architecture and prove the concept. Then, we’ll need a GPT-based instrument bench of developers to hammer it down.

The proposed architecture

There are two obvious parts of the architecture - the cloud one and the truck one. The cloud one is easy and mostly out-of-scope for the article. We need some frontend, some backend, and some database (well, as usual). In the trucks, we need to separate applications working on the same machine and then isolate traffic for each application. It sounds like containers and virtual networks. Before diving into each part, we need to solve the main issue - how to communicate between trucks and the cloud.

Selecting the technology

The star-like architecture of the system seems to be a very typical one - there is a server in the center with multiple clients using its services. However, in this situation, we can't distinguish between resources/services supplier (the server) and resources/services consumers (the clients). Instead, we need to consider the system as a complex, distributed structure with multiple working nodes, central management, and 3rd party integration. Due to the isolation, trucks’ main computers should containerize running applications. We could use Kubernetes clusters in trucks and another one in the cloud, but in that case, we need to implement everything manually – new truck onboarding, management at scale, resource limiting for applications, secured communication channels, and OTA updates. In the cloud, we would need to manage the cluster and pods, running even when there is no traffic.

An alternative way is the IoT. Well, as revealed in the title, this is the way that we have chosen. IoT provides a lot of services out-of-the-box - the communication channel, permissions management, OTA updates, components management, logs, metrics, and much more. Therefore, the main argument for using it was speeding up the deployment process.

However, we need to keep in mind that IoT architecture is not designed to be used with complex edge devices. This is our challenge, but fortunately, we are happy to solve it.

Selecting the cloud provider

The customer would like to use a leading provider, which reduces the choice to the top three in the World: AWS, MS Azure, and GCP.

The GCP IoT Core is the least advanced solution. It misses a lot of concepts and services available in the competitors, like a digital twin creation mechanism, complex permissions management, security evaluation, or a complex provisioning mechanism.

The Azure IoT is much more complex and powerful. On the other hand, it suffers from shortcomings in documentation, and - what is most important - some features are restricted to Microsoft instruments only (C#, Visual Studio, or PowerShell). On the other hand, it provides seamless AI tool integration, but it’s not our case for now.

But the last one – AWS IoT – fits all requirements and provides all the services needed. Two MQTT brokers are available, plenty of useful components (logs forwarding, direct tunnel for SSH access, complex permission management), and almost no limitation for IoT Core client devices. There is much more from AWS Greengrass - an extended version with higher requirements (vanilla C is not enough), but we can easily fulfill those requirements with our ARM-based trucks’ computers.

The basic architecture

Going back to the start-like topology, the most important part is the communication between multiple edge devices and the core. AWS IoT provides MQTT to enable a TCP-based, failure-resistant communication channel with a buffer that seamlessly keeps the communication on connection lost. The concept offers two MQTT brokers (in the cloud and on the edge) connected via a secured bridge. This way, we can use the MQTT as the main communication mechanism on the edge and decide which topics should be bridged and transferred to the cloud. We can also manage permissions for each topic on both sides as needed.

The cloud part is easy – we can synchronize the IoT MQTT broker with another messaging system (SNS/SQS, Kafka, whatever you like) or read/write it directly from our applications.

The edge part is much more complex. In the beginning, let’s assume that there are two applications running as executable programs on the edge. Each of these uses its own certificate to connect to the edge broker so we can distinguish between them and manage their permissions. It brings up some basic questions – how to provide certificates and ensure that one application won’t steal credentials from another. Fortunately, AWS IoT Greengrass supplies a way to run components as docker containers – it creates and provides certificates and uses IPC (inter-process communication) to allow containers to use the broker. Docker ensures isolation with low overhead, so each application is not aware of the other one. See the official documentation for details: Run a Docker container - AWS IoT Greengrass (amazon.com) .

Please note the only requirement for the applications, which is, in fact, the requirement we make to applications’ providers: we need docker images with applications that use AWS IoT SDK for communication.

See the initial architecture in the picture below.

As you can see, Application 1 contains two programs (separate docker containers) communicating with each other via the broker: Application 1_1 and Application 1_2. Thanks to the privileges management, we are sure that Application 2 can’t impact or read this communication. If required, we can also configure a common topic accessible by both applications.

Please also note that there is one more component – Nucleus. You can consider it as an orchestrator required by AWS IoT to rule the system.

Of course, we can connect thousands of similar edges to the same cloud, but we are not going to show it on pictures for readability reasons. AWS IoT provides deployment groups with versioning for OTA updates based on typical AWS SDK. Therefore, we can expose a user-friendly management system (for our client and end users) to manage applications running on edge at scale.

Virtual networks

Now, let’s challenge the architecture with a more complex scenario. Let’s assume that Application 2 communicates with an in-cabin air quality sensor – a separate computer that is in the same IP network. We can assume the sensor is a part of Application 2, and our aim is to enable such communication but also to hide it from Application 1. Let’s add some VLANs and utilize network interfaces.

Starting from the physical infrastructure, the main computer uses two interfaces – eth0 to connect to the Internet and eth1 connected to a physical, managed switch (the “in-vehicle IP network” mentioned above). The Application 2_2 computer (the air quality sensor) is connected to the switch to a port tagged as VLAN 102, and the switch is connected to eth1 via a trunk port.

The eth0 interface is used by the main computer (host) to communicate with the Internet, so the main MQTT bridging is realized via this interface. On the other hand, there is also a new Greengrass-docker component called router. It’s connected to eth0 and to two virtual bridges – br101 and br102. Those bridges are not the same as the MQTT bridge. This time, we need to use the kernel-based Linux feature “bridge,” which is a logical, virtual network hub. Those bridges are connected to virtual network interfaces eth1.101 and eth1.102 and to applications’ containers.

This way, Application 1 uses its own VLAN 101 (100% virtual), and Application 2 uses its own VLAN 102 (holding both virtual and physical nodes). The application separation is still ensured, and there is no logical difference between virtual and mixed VLANs. Applications running inside VLANs can’t distinguish between physical and virtual nodes, and all IP network features (like UDP broadcasting and multicasting) are allowed. Note that nodes belonging to the same application can communicate omitting the MQTT (which is fine because the MQTT may be a bottleneck for the system).

Moreover, there is a single security-configuration point for all applications. The router container is the main gateway for all virtual and physical application-nodes, so we can configure a firewall on it or enable restricted routes between specific nodes between applications if needed. This way, we can enable applications to communicate with third-party servers over the Internet (see Application 1_1 in the picture), to communicate with individual nodes of the applications without restrictions, and to control the entire application-related traffic in a single place. And this place – the router – is just another Greengrass component, ready to be redeployed as a part of the OTA update. Also, the router is a good candidate to serve traffic targeting all networks (and all applications), e.g., to broadcast GPS position via UDP or to act as the network time server.

One more broker

What if… the application is provided as a physical machine only?

Well, as the main communication channel is MQTT, and the direct edge-to-Internet connection is available but limited, we would like to enable a physical application to use the MQTT. MQTT is a general standard for many integrated systems (small computers with limited purposes), but our edge MQTT broker is AWS-protected, so there are two options available. We can force the application supplier to be AWS-Greengrass compatible, or we need another broker. As we’re pacifists and we can’t stand forcing anybody to do anything, let’s add one more broker and one more bridge.

This time, there are two new components. The first one, an MQTT broker (Mosquitto or similar), interacts with Application 3. As we can’t configure the Mosquitto to act as a bridge for the AWS-managed broker, there is one more, custom application running on the server for this purpose only – a Greengrass component called “bridge”. This application connects to both local MQTT brokers and routes specific messages between them, as configured. Please note that Application 3 is connected to its own VLAN even if there are no virtual nodes. The reason is – there are no virtual nodes yet, but we’d like to keep the system future-proof and consistent. This way, we keep the virtual router as a network gateway for Application 3, too. Nevertheless, the non-AWS broker can listen to specific virtual interfaces, including eth1.103 in this case, so we can enable it for specific VLANs (application) if needed.

Summary

The article shows how to combine AWS IoT, docker, and virtual networks to achieve a future-proof fleet management system with hardware- and software-based applications at scale. We can use AWS tools to deliver new applications to edge devices and manage groups evoking truck owners or truck models. Each vehicle can be equipped with an ARM computer that uses AWS-native fleet provisioning on OS initialization to join the system. The proposed structure may seem to be complex, but you need to configure it only once to fulfill all requirements specified by the client.

However, theory is sometimes easier than practice, so we encourage you to read the following article with implementation details .

Generative AI is quickly emerging as one of the key drivers of automotive innovation. This shift is not just a future possibility; it's already happening. In 2023, the generative AI market in the automotive sector was valued at approximately  USD 387.54 million . Looking ahead, it's projected to surge to about  USD 2,691.92 million by 2032 , demonstrating a robust Compound Annual Growth Rate (CAGR) of 24.03% from 2023 to 2032. Major Original Equipment Manufacturers (OEMs) are integrating sophisticated AI algorithms into various aspects of the industry, from vehicle design to enhancing customer interactions.

The impact of generative AI in the automotive sector is already evident. For instance, NVIDIA's generative AI models empower designers to swiftly  transform 2D sketches into intricate 3D models, significantly speeding up the design process and opening up new avenues for creativity and efficiency​​. Meanwhile, automotive manufacturing companies are exploring collaborations with tech giants to integrate advanced AI language models into their vehicles, enhancing the driving experience​​.

This article will explore how leading automotive players are leveraging generative AI to not only keep pace with the evolving demands of the market but also redefine mobility and automotive excellence.

Software-defined vehicles and generative AI

The introduction of software-defined vehicles (SDVs) represents a significant shift in the automotive industry, moving beyond traditional performance metrics such as horsepower and chassis design to focus on software and digital capabilities. By 2025, it is estimated that vehicles could require as much as  650 million lines of code each. These vehicles heavily rely on software for critical operations such as driving assistance, navigation, and in-car entertainment systems.

The integration of generative AI in this domain further amplifies these capabilities. Generative AI is known for its ability to create and optimize designs and solutions, which is beneficial in improving both the software and hardware aspects of SDVs. It helps in generating efficient algorithms for vehicle control systems, contributes to the development of more effective and adaptive software solutions, and even assists in designing vehicle components for better performance and efficiency.

However, bringing generative AI into this landscape presents both unique opportunities and significant challenges.

The opportunities and challenges

•     Challenges  

Integrating advanced AI systems into modern vehicles is a complex and multifaceted task that demands  technical expertise and careful attention to data security and privacy , particularly with the increasing reliance on data-driven functionalities in vehicles.

The automotive industry is facing a complex regulatory environment. With the growing importance of AI and data in-vehicle systems, it has become crucial to  comply with various international regulations and standards , covering areas such as data protection, safety, and environmental impact.

One significant challenge for OEMs is the  lack of standardization within the software-defined vehicles industry, which can complicate the development and integration of new technologies as there are no universal norms or protocols to guide these processes.

 Internal transformation is also a critical aspect of this integration. OEMs may need to revamp their internal capabilities, processes, and technological infrastructure to use generative AI effectively.

•     Opportunities  

Integrating generative AI technology allows for more  creative and efficient vehicle design , resulting in  quicker prototypes and more innovative models .

It also allows for creating  personalized vehicles that cater to individual user preferences like never before. In manufacturing, generative AI promotes more efficient and  streamlined production processes , which optimizes resources and reduces waste.

Let's explore how automotive manufacturers already use gen AI to boost their operations.

Generative AI applications in the automotive industry

Generative AI's integration into the automotive industry revolutionizes multiple facets of vehicle design, manufacturing, and user experience. Let's explore these areas:

Design and conceptualization

•     Vehicle Design Enhancement    : Artificial Intelligence is revolutionizing the vehicle design process by speeding up the initial phase of the design cycle. Generative design algorithms use parameters such as material properties, cost constraints, and performance requirements to generate optimal design solutions. For example, in vehicle body design, AI can propose multiple design options that optimize for aerodynamics and strength while minimizing weight. This enables quick visualization and modification of ideas.

 Toyota Research Institute has introduced a generative AI technique to optimize the vehicle design process to produce more efficient and innovative vehicles. This approach allows designers to explore a wider range of design possibilities, including aerodynamic shapes and new material compositions.

•     Digital Prototyping    : The use of Generative AI technology makes it possible to create digital prototypes, which can be tested and refined extensively without the need for physical models. This approach is highly beneficial, as it enables designers to detect and correct potential design flaws early in the process.

 BMW's use of NVIDIA Omniverse is a significant step in design improvement. The company uses this platform to create digital twins of their manufacturing facilities, integrating generative AI to enhance production efficiency and design processes.

Manufacturing and production

•     Streamlining Manufacturing Processes    : Generative AI significantly enhances the efficiency of manufacturing processes. Unlike traditional AI or machine learning models, generative AI goes beyond identifying inefficiencies; it actively generates novel manufacturing strategies and solutions. By inputting parameters such as production timelines, material constraints, and cost factors, generative AI algorithms can propose a range of optimized manufacturing workflows and processes.

 BMW has implemented generative AI in a unique way to improve the scheduling of their manufacturing plant. In partnership with Zapata AI, BMW utilized a quantum-inspired generative model to optimize their plant scheduling, resulting in more efficient production. This process, known as Generator-Enhanced Optimization (GEO), has significantly improved BMW's production planning, demonstrating the potential of generative AI in industrial applications.

•     Supply Chain Resilience    : In the context of supply chain management, particularly during challenges like the automotive microchip shortage, generative AI plays a crucial role. Unlike conventional AI, gen AI can do more than just analyze existing supply chain networks; it can creatively generate alternative supply chain models and strategies. The algorithms can propose diverse and robust supplier networks by leveraging data about supplier capabilities, logistics constraints, and market demands.

•     Customized Production    : With generative AI, it is now possible to create personalized vehicles on a large scale, meeting the growing demand for customization in the automotive industry.

Predictive maintenance and modelling

Traditionally, predictive maintenance relies on historical data to forecast equipment failures, but generative AI enhances this process by creating detailed, simulated data environments. This technology generates realistic yet hypothetical scenarios, encompassing a vast array of potential machine failures or system inefficiencies that might not be present in existing data sets.

The generative aspect of this AI technology is particularly valuable in situations where real-world failure data is limited or non-existent. By synthesizing new data points, generative AI models can extrapolate from known conditions to predict how machinery will behave under various untested scenarios.

In modeling, generative AI goes a step further. It not only predicts when and how equipment might fail but also suggests optimal maintenance schedules, anticipates the impact of different environmental conditions, and proposes design improvements.

Customer experience and marketing

One of the challenges in using generative AI, particularly in customer interaction, is the accuracy of AI-generated responses. An example of this was an error by ChatGPT, where it was tricked into suggestion of  buying a Chevy for a dollar . This incident underlines the potential risks of misinformation in AI-driven communication, emphasizing the need for regular updates, accuracy checks, and human oversight in AI systems. Nevertheless, this technology offers many opportunities for improving the customer experience:

•     Personalized User Experiences and Enhanced Interaction    : AI's capability to adapt to individual preferences not only enhances the driving experience but also improves the functionality of vehicle features.

For instance, in collaboration with Microsoft,  General Motors is exploring the use of AI-powered virtual assistants that offer drivers a more interactive and informative experience. These assistants can potentially provide detailed insights into vehicle features and performance metrics and offer personalized recommendations based on driving patterns.

Also,  Mercedes-Benz is exploring the integration of generative AI through voice-activated functionalities in collaboration with Microsoft. This includes leveraging the OpenAI Service plugin ecosystem, which could allow for a range of in-car services like restaurant reservations and movie ticket bookings through natural speech commands.

Example applications

•     Simplified Manuals    : AI technology, enabled by natural language processing, has simplified the interaction between drivers and their vehicles. Beyond just responding to voice commands with pre-existing information, a generative AI system can even create personalized guides or tutorials based on the driver's specific queries and past interactions.

    Grape Up has developed an innovative          voice-driven car manual         that allows drivers to interact with their vehicle manual through voice commands, making it more accessible and user-friendly. With this technology, drivers no longer have to navigate through a traditional manual. Still, they can easily ask questions and receive instant verbal responses, streamlining the process of finding information about their vehicle.  

•     Roadside Assistance:    In this scenario, generative AI can go beyond analyzing situations and suggesting solutions by creating new, context-specific guidance for unique problems. For instance, if a driver is stranded in a rare or complex situation, the AI could generate a step-by-step solution, drawing from a vast database of mechanical knowledge, previous incidents, and environmental factors.

•     Map Generation    : Here, generative AI can be used to not only update maps with real-time data but also to predict and visualize future road conditions or propose optimal routes that don't yet exist. For example, it could generate a route that balances time, fuel efficiency, and scenic value based on the driver's preferences and driving history.

•     Marketing and Sales Innovation    : Generative AI-enabled content engine is transforming the creation of digital advertising for the automotive industry. This content is tailored to meet the unique requirements of automotive brands and their consumers, thereby revolutionizing traditional marketing strategies.

Safety and compliance

•     Enhancing Vehicle Safety    : Generative AI in vehicles goes beyond traditional AI systems by not only assisting drivers but also by creating predictive models that enhance safety features. It processes and interprets data from cameras and sensors to foresee potential road hazards, often employing advanced generative models that simulate and predict various driving scenarios.

•     Regulatory Compliance    : Similarly, gen AI helps automakers comply with safety standards and navigate complex regulation changes by monitoring performance data and comparing it against regulatory benchmarks. This allows automakers to stay ahead of the compliance curve and avoid potential legal and financial repercussions.

Autonomous vehicle development

•     Simulation and Testing    : Generative AI is crucial for developing autonomous vehicle systems. It generates realistic simulations, including edge-case scenarios, to test and improve vehicle safety and performance.

•     Enhancing ADAS Capabilities    : AI technology can improve essential Advanced Driver Assistance Systems (ADAS) features such as adaptive cruise control, lane departure warnings, and automatic emergency braking by analyzing data from various sensors and cameras. Generative AI's strength in this context lies in its ability to not only process existing data but also to generate new data models, which can predict and simulate different driving scenarios. This leads to more advanced, reliable, and safer ADAS functionalities, significantly contributing to the evolution of autonomous and semi-autonomous driving technologies.

Conclusion

As the automotive industry accelerates towards a more AI-integrated future, the role of expert partners like Grape Up becomes increasingly crucial. Our expertise in navigating the intricacies of AI implementation can help automotive companies unlock the full potential of this technology. If you want to stay ahead in this dynamic landscape, now is the time to embrace the power of generative AI. For more information or to collaborate with Grape Up,  contact our experts today.

In the modern world, tech companies strive to collect as much information as possible about the status of owned cars to enable proactive maintenance and rapid responses to any incidents that may occur. These incidents could involve theft, damage, or the cars simply getting lost. The only way to remotely monitor their status is by obtaining telemetry data sent by the vehicles and storing it on a server or in the cloud. There are numerous methods for gathering this data, but is there an optimal approach? Is there a blueprint for designing an architecture for such a system? Let's explore.

What does “telemetry” mean in a car?

This article is about gathering telemetry data, so let's begin with a quick reminder of what it is. Telemetry in cars refers to the technology that enables the remote collection and transmission of real-time data from various components of a vehicle to a central monitoring system. This data encompasses a wide range of parameters, including, for example:

• engine performance,
• fuel consumption,
• tire pressure,
• vehicle speed,
• percentage of electric vehicle battery,
• braking activity,
• acceleration,
• GPS position,
• odometer

Collecting vehicle details is valuable, but what is the real purpose of this information?

Why collect telemetry data from a car?

The primary use of telemetry data is to monitor a car's status from anywhere in the world, and it's especially crucial for companies like car rental firms such as Hertz or Europcar, as well as transportation companies like Uber. Here are some examples:

• Tracking Stolen Cars : Companies can quickly track a stolen vehicle if they store its GPS position.
• Accident Analysis : If a car is involved in an accident, the company can assess the likelihood of the event by analyzing data such as a sudden drop in speed to zero and high acceleration. This allows companies to provide replacement cars promptly.
• Fuel or Charging Management : In cases where a rental car is returned without a full tank of fuel or not fully charged, the company can respond quickly to make the car available for the next rental.

These are just a few examples of how telemetry data can be utilized, with many more possibilities. Understanding the value of telemetry data, let's delve into the technical aspects of acquiring and using this data in the next part of the article.

To begin planning the architecture, we need answers to some fundamental questions

How will the telemetry data be used?

Architectural planning should commence with an understanding of the use cases for the collected telemetry data. This includes considering what the end user intends to do with the data and how they will access it. Common uses for this data include:

1. Sharing data on a dashboard : To enable this, an architecture should be designed to support an API that retrieves data from databases and displays it on a dashboard,
2. Data analytics : Depending on the specific needs, appropriate analytic tools should be planned. This can vary from real-time analysis (e.g. AWS Kinesis Data Analytics) to near real-time analysis (e.g. Kafka) or historical data analysis (e.g. AWS Athena),
3. Sharing data with external clients : If external clients require real-time data, it's essential to incorporate a streaming mechanism into your architecture. If real-time access is not needed, a REST API should be part of the plan.

Can we collect the data from cars?

We should not collect any data from cars unless we either own the car or have a specific legal agreement to do so. This requires not only planning the architecture for acquiring access to the car but also for disposing of it. For example, if we collect telemetry or location data from a car through websockets and the company decides to sell the car, we should immediately cease tracking the car. Storing data from it, especially location data, might be illegal as it could potentially allow tracking of the location of a person inside the car.

How do we manage permissions to the car?

If we have legal permission to collect data from the car, we must include correct permission management in our architecture. Some key considerations include:

• Credential and token encryption,
• Secure storage of secrets, such as using AWS Secret Manager,
• Regular rotation of credentials and tokens for security,
• Implementing minimum access levels for services and vehicles,
• Good management of certificates,
• Adhering to basic security best practices.

How do we collect the data?

Now that we have access to the data, it's time to consider how to collect it. There are several known methods to do this:

• Pull Through REST/GRPC API: In this scenario, you'll need to implement a data poller. This approach may introduce latency in data acquisition and is not the most scalable solution. Additionally, you may encounter request throttling issues due to hitting request limits.
• External Service Push Through REST/GRPC: Here, you should set up a listener, which is essentially a service exposed with an endpoint, such as an ECS task or a Lambda function on AWS. This method might incur some costs, and it's crucial to consider automatic scaling to ensure no data is lost. Keep in mind that the endpoint will be publicly exposed, so robust permission management is essential.
• Pulling From a Stream: This approach is often recommended as it's the most scalable and secure option. You can receive data in real-time or near real-time, making it highly efficient. The primary considerations are access to the stream and the service responsible for pulling data from it.
• Queues: Similar to streams, queues can be used for data collection, and they may offer better data ordering. However, streams are typically faster but might be more expensive. This is another viable option for collecting vehicle data from external services.
• Websockets: Websockets are a suitable solution when bidirectional data flow is required, and they can be superior to REST/GRPC APIs in such cases. For example, they are an appropriate choice when a client needs confirmation that data has been successfully acquired. Websockets also allow you to specify which telemetry data can be acquired and at what frequency. A notable example is the Tesla Websockets ( https://github.com/teslamotors/fleet-telemetry/blob/main/protos/vehicle_data.proto ).

Where to store the data?

After collecting the data, it's important to decide where to store it. There are various databases available, and the choice depends on your specific data use cases and access patterns. For instance:

• Commonly Used Data : For data that will be frequently accessed, you can opt for a traditional database like MongoDB or PostgreSQL.
• Low-Maintenance Database : If you prefer a database that requires minimal maintenance, AWS DynamoDB is a good choice.
• Infrequently Used Data for Analytics : When data won't be used frequently but will be utilized for occasional data analytics, you can consider using an AWS S3 bucket with the appropriate storage tier, coupled with AWS Athena for data analysis.
• Complex Data Analysis : If the data will be regularly analyzed with complex queries, AWS Redshift might be a suitable solution.

When planning your databases, don't forget to consider data retention. If historical data is no longer needed, it's advisable to remove it to avoid excessive storage costs.

Example

Here is an example of such an architecture on AWS in which:

1. An employee grants permissions to the car to stream the data.
2. The data is streamed using AWS Kinesis Stream and saved to an S3 bucket by AWS Kinesis Firehose for audit purposes.
3. The data is also normalized by the AWS Lambda function and stored in AWS DynamoDB.
4. The stored data is queried by another AWS Lambda function.
5. The query Lambda is triggered by an AWS API Gateway to enhance security, such as limiting requests per second.
6. The API is exposed via Route 53 to the end user, which can be, for example, a dashboard or an external API.

Conclusion

In the modern tech landscape, the quest for complete vehicle data is a paramount objective. Tech companies seek to collect critical information about the status of owned cars to enable proactive maintenance and rapid responses to a spectrum of incidents, from theft and damage to simple misplacement. This imperative relies on the remote monitoring of vehicles through the collection and storage of data on servers or in the cloud, offering the capability to monitor a vehicle's status from any corner of the globe. This is especially essential for companies like car rental firms and transportation services, with applications ranging from tracking stolen cars through GPS data to analyzing accident events and managing fuel or charging for rental vehicles.

The core of this mission is to strike a balance between data collection, security, and architectural planning. The process involves careful consideration of data collection methods, adherence to legal and security best practices, and informed choices for data storage solutions. The evolving landscape of vehicle data offers endless possibilities for tech companies to harness the power of telemetry and deliver an enhanced experience for their customers.

In this article, we’ll give an overview of the Quarkus framework and compare it with Spring Boot -  the most popular Java backend framework.

Quarkus is an open-source Java framework for developing web applications. It is designed to build apps in a modern approach with microservices and cloud-native applications. Quarkus has built-in support for the two most popular package managers: Gradle and Maven. By default, a new starter project created using the official Quarkus CLI uses Maven, although it may be easily switched to generate a grade-based project instead.

Features comparison to Spring Boot framework

Let’s make an overview of the main features available in Quarkus and compare them to the equivalent ones from Spring Framework.

Application configuration

Quarkus uses an open-source SmallRye Config library for configuring applications. By default, it uses  application.properties files for this purpose. However, there is an option to also use the yaml file. In order to enable yaml configuration, we need to add a  quarkus-config-yaml dependency. Quarkus may also additionally read configuration using environment variables specified in the  .env file. This is particularly useful for local development.

You may store some local settings in the  .env file e.g., to enable retrieval of credentials from external sources for local builds and not commit the file, keeping it for local usage only. Spring does not support reading environment variables from an  .env file, so developers have to either inject them by IDE or configure them directly using the application configuration file.

Similarly to Spring, Quarkus supports different profiles depending on the target environment. This may be achieved by adding a profile name to the  application.properties file name, e.g.:

 application.properties – base configuration
 application-dev.properties – extended configuration for the dev environment. When the dev profile is active, it will add (or override) the configuration to the base profile.

There is also an ability to build a configuration hierarchy with an additional configuration layer to mention base and profile-specific configuration. We may specify the name of the parent profile using  quarkus.config.profile.parent in  application-<profile>.properties file. E.g., by specifying  quarkus.config.profile.parent=common in the  application-dev.properties file, we’ll get a three-level hierarchy:

 application.properties – base configuration  application-common.properties – overrides/extends base configuration
 application-dev.properties – overrides/extends  common profile configuration

Spring has similar setting called  spring.config.import which allows to import properties from a specified file rather than creating a hierarchy.

Dependency injection

Quarkus uses Jakarta EE Contexts and Dependency Injection engine. Similarly to Spring, Quarkus uses the concept of a bean. A bean is an object managed by an application environment (container) which creation and behavior can be controlled by the developer in a declarative style using annotations. For injecting beans, same as in the Spring Boot, we may use field, setter, and constructor injections.

Beans can be created by annotating the bean class with a scoped annotation:

@ApplicationScopedpublic class DataService {
   public void writeData() {
       //...
   }
}


Compare with similar bean in Spring Boot:

@Servicepublic class DataService {
  public void writeData() {
       //...
   }
}


Another way to define beans is to use producer methods or fields. This is similar to  @Configuration classes in Spring Boot.

Quarkus:

// Define beans
@ApplicationScopedpublic class SettingsProducer {

   @Produces
   String appToken = "123abc";

  @Produces
   Map<String, String> settings() {
       return Map.of(
           "setting 1", "value1",
           "setting 2", "value2"
       );
   }

  @Produces
   MyBean myBean() {
       return new MyBean(); // custom class
   }

}

// Inject beans:
@ApplicationScopedpublic class DataService {
  @Inject
   private String appToken;

   @Inject
  private Map<String, String> settings;

   @Inject
   private MyBean myBean;


   //…
}


Equivalent Spring Boot code:


// Configuration:
@Configuration
public class SettingsProducer {
   @Bean
   String appToken() {
       return "123abc";
   }

  @Bean
   Map<String, String> settings() {
       return Map.of(
           "setting 1", "value1",
           "setting 2", "value2"
       );
   }

  @Bean
   MyBean myBean() {
       return new MyBean();
   }
}

// Injecting beans:
@Beanpublic class DataService {
  @Autowired
   private String appToken;

   @Autowired
   @Qualifier("settings") // needed to explicitly tell spring that Map itself is a bean
   private Map<String, String> settings;

  @Autowired
   private MyBean myBean;

   //…
}


Quarkus also provides built-in mechanisms for interceptors, decorator, and event handling using Jakarta EE framework annotations and classes.

RESTful API

Building RESTful services with Quarkus is enabled by RESTEasy Reactive framework, which is an implementation of Jakarta REST specification. The example class to handle endpoint requests:

@Path("/api")
public class DataResource {
   @GET
   @Path("resource")
   public String getData() {
       return "My resource";
   }

   @POST
   @Path("add/{data}")
   public String addData(String data, @RestQuery String type) {
       return "Adding data: " + data + " of type: " + type;
   }

}


Class annotation  @Path defines the root path of the URL. Each method contains HTTP method annotation (  @GET, @POST ) and  @Path annotation that becomes a subpath of the root part. That is, in the example above,  getData() is called for  GET /api/resource request. Method  addData() is called for  POST /api/add/some_text?type=text request. Path component  some_text will be specified as a data parameter to the  addData() method. Query parameter  text will be passed as a type parameter.

In Spring Boot, the above endpoint implementation using Spring Web is very similar:

@RestController
@RequestMapping("/api")
public class TestController {
   @GetMapping("resource")
   public String hello() {
       return "My resource";
   }

   @PostMapping("add/{data}")
   public String addData(@PathVariable String data, @RequestParam String type) {
       return "Adding data: " + data + " of type: " + type;
   }
}


Security

Quarkus has its own Security framework, similarly to Spring Boot. Quarkus Security has built-in authentication mechanisms (like Basic, Form-based etc) and provides an ability to use well-known external mechanisms, such as OpenID Connect or WebAuth. It also contains security annotations for role-based access control. On the other hand, Spring Security has more abilities and is more flexible than Quarkus Security. E.g.,  @Secured and  @PreAuthorize Spring annotations are not provided natively by Quarkus. However,  Quarkus can be integrated with Spring Security to use the functionality that is provided by Spring Security only.

Cloud integration

Quarkus supports building container images when creating an application. The following container technologies are supported:

•     Docker.    To generate a Dockerfile that allows the building of a docker image, the     quarkus-container-image-docker    extension is used.

•     Jib.    The Jib container images can be built by using the     quarkus-container-image-jib    extension. For Jib containerization, Quarkus provides caching of the app dependencies stored in a layer different from the application. This allows to rebuild the application fast. It also makes the application build smaller when pushing the container. Moreover, it gives the ability to build apps and containers without the need to have any client-side docker-related tool in case only push to the docker registry is needed.

•     OpenShift.    To build an OpenShift container, the     quarkus-container-image-openshift    is needed. The container is built by only uploading an artifact and its dependencies to the cluster, where they are merged into the container. Building OpenShift builds requires the creation of BuildConfig and two ImageStream resources. One resource is required for the builder image and one for the output image. The objects are created by the Quarkus Kubernetes extension.

•     Buildpack.    Quarkus extension     quarkus-container-image-buildpack    uses build packs to create container images. Internally, the Docker service is used.

Quarkus is positioned as a Kubernetes native. It provides a Quarkus Kubernetes extension that allows the deployment of apps to Kubernetes. This simplifies workflow and, in simple applications, allows one to deploy to Kubernetes in one step without deep knowledge of Kubernetes API itself. Using quarkus-kubernetes dependency provides auto-generated Kubernetes manifest in the target directory. The manifest is ready to be applied to the cluster using the following example command:

kubectl apply -f target/kubernetes/kubernetes.json

Quarkus also creates a dockerfile for the docker image for deployment. Application properties of the Quarkus app allow the configuration of container images to set their names, groups, etc. Kubernetes-related application properties allow the setup of a generated resource. E.g,  quarkus.kubernetes.deployment-kind sets the resource kind to, e.g., Deployment, StatefulSet, or Job.

When it comes to Spring Boot it also has support of containerization and Kubernetes. However, such support is not out-of-the-box. The setup and deployment require much more configuration and technology-specific knowledge comparing to how it’s implemented in Quarkus. Quarkus reduces boilerplate code and setup since it has a native configuration for Kubernetes and containerization.

Production readiness and technical support

Quarkus is a relatively new framework comparing to Spring/Spring Boot. The initial Quarkus release took place on May 20, 2019. The first production Spring Framework release was on March 24, 2004, and Spring Boot was first released on April 1, 2014. Quarkus releases new versions a bit more frequently than Spring Boot, although the frequency is very good for both frameworks. There are 3-4 Quarkus releases per month and 5-6 releases of Spring Boot.

When it comes to bugs, there are 8289 total issues marked as bugs on github for Quarkus, and only 881 of them are in opened state. For Spring Boot, there are 3473 total issues that are marked as bugs, and only 66 of them are opened. The bugs are fixed faster for Spring Boot. The reason may be because it has a larger supporting community as Spring is much older.

Spring Boot has long-term support (LTS) versions that are maintained (updates/bug fixes) for a longer period of time without introducing major changes. Such an approach gives more stability and allows the use of the older version of your application that is getting all the critical fixes (e.g., fixes of known vulnerabilities). At the same time, you may work on the newer releases of your product to integrate and test the latest version of the framework. Spring Boot offers one year of regular support for each LTS edition and more than two years of commercial support.

Quarkus recently introduced LTS versions supported for one year. They plan to release a new LTS version every six months. Currently, Quarkus does not have commercial support for LTS versions.

Summary

In this article, we described key features of the Quarkus framework and compared them with the equivalent features in Spring Boot. Let’s summarize the main pros and cons of both frameworks in every area addressed.

•     Application Configuration  

Both Spring Boot and Quarkus have well-developed systems of configuration management. Even though Quarkus has the ability to build a simple three-level hierarchy of profiles, both frameworks can handle app configuration at a good level to satisfy app development needs.

•     Dependency Injection  

Both frameworks use a similar approach with beans injected at run time and use annotations to define bean types, assign bean information, and specify injection points.

•     RESTful API  

Spring Boot and Quarkus have very similar approach to writing RESTful API, with no major differences.

•     Security  

Quarkus provides the same authorization methods as Spring and has built-in integration with well-known authentication mechanisms. Spring Security offers more security-related annotations, although some of them can be used in Quarkus as it has the ability to integrate with the Spring Security framework.

•     Cloud integration  

Quarkus has native support for containerization and Kubernetes. It provides more abilities and is easier to use in cloud-native environments and tools compared to Spring Boot. This may be a key factor in choosing between Spring Boot and Quarkus for a development of a new app designed for a cloud-native environment.

•     Production Readiness and Technical Support  

Spring Boot is a much older framework. It has a larger support community and more users. It provides more robust LTS versions with the ability to extend tech support even more with LTS commercial support. Quarkus, on the other hand, has just recently introduced LTS versions with a shorter support period and doesn’t have commercial options. However, both frameworks are stable, bugs are continuously fixed, and new versions are released frequently.

For the most of its features, Quarkus implements Jakarta EE specifications, while Spring Boot includes its own solutions. Implementing the specification offers the advantage of creating a more consistent and robust API, as opposed to using non-standard methods. On the other hand, Spring Boot  is quite an old and well-known framework and its features have proven to be stable and robust over time.

Android Automotive OS, AAOS in short, is a vehicle infotainment operating system that has gained a lot of traction recently, with most of the OEMs around the world openly announcing new versions of their infotainment based on Android. AAOS is based on the AOSP (Android Open Source Project) source code, which makes it fully compatible with Android, with additions that make it more useful in cars – different UI, integration with hardware layer, or vehicle-specific apps.

For OEMs and Tier1s, who are deeply accustomed to infotainment based on QNX/Autosar/Docker/Linux, and software developers working on AAOS apps, it’s sometimes difficult to quickly spin-up the development board or emulator supporting external hardware that has no out-of-the-box emulation built by Google. One of the common examples is camera access, which is missing in the official AAOS emulator these days, but the hardware itself is quite common in modern vehicles – which makes implementation of applications similar to Zoom or MS Teams for AAOS tempting to app developers.

In this article, I will explain how to build a simple test bench based on a cost-effective Raspberry Pi board and AAOS for developers to test their camera application. Examples will be based on AAOS 11 running on Raspberry Pi 4 and our Grape Up repository. Please check our previous article: " Build and Run Android Automotive OS on Raspberry Pi 4B " for a detailed description of how to run AAOS on this board.

Android Automotive OS has 2 different subsystems to access platform cameras: Camera2 and EVS. In this article, I will explain both how we can use it and how to get it running on Android Automotive OS 11.

Exterior View System (EVS)

EVS is a subsystem to display parking and maneuvering camera image. It supports multiple cameras' access and view. The main goal and advantage of that subsystem is that it boots quickly and should display a parking view before 2 seconds, which is required by law.

Source https://source.android.com/docs/automotive/camera-hal

As you can see on the attached diagram, low layers of EVS depend on OEM source code. OEM needs to deliver Camera and Display implementation. However, Android delivers a sample application (/hardware/interfaces/automotive/evs/1.0) , which uses Linux V4L2 and OpenGL to grab camera frames and display them. You can find more information about EVS at https://source.android.com/docs/automotive/camera-hal

In our example, we will use samples from Android. Additionally, I assume you build our Raspberry Pi image (see our article ), as it has multiple changes that allow AAOS to reliably run on RPi4 and support its hardware.

You should have a camera connected to your board via USB. Please check if your camera is detected by V4L2. There should be a device file:

/dev/video0

Then, type on the console:

su

setprop persist.automotive.evs.mode 1


This will start the EVS system.

To display camera views:

evs_app

Type Ctrl-C to exit the app and go back to the normal Android view.

Camera2

Camera2 is a subsystem intended for camera access by “normal” Android applications (smartphones, tablets, etc.). It is a common system for all Android applications, recently slowly being replaced by CameraX. The developer of an Android app uses Java camera API to gain access to the camera.

Camera2 has three main layers, which are shown in the diagram below:

Source https://source.android.com/docs/core/camera

Low-level Camera access is implemented in CameraProvider. OEM can implement their own provider or a V4L2 camera driver can be used.

To get Camera2, you should enable it in the Car product make file. In

packages/services/Car/car_product/build/car_base.mk change config.disable_cameraservice to false.

PRODUCT_PROPERTY_OVERRIDES += config.disable_cameraservice=false

After that, rebuild Android:

make ramdisk systemimage vendorimage

Put it in the SD card and boot RPi with it. You will be able to run the “Camera” application on the AAOS screen, see camera output from the connected webcam, and run and debug applications using Camera API.

Summary

Now you know how to run both AAOS camera APIs on the RPi4 board. You can use both APIs to develop automotive applications leveraging cameras and test them using a simple USB webcam, which you may have somewhere on the shelf. If you found this article useful, you can also look at our previous articles about AAOS – both from the application development perspective and the OS perspective . Happy coding!

Fasten your seatbelts! The EU Data Act aims to drive a paradigm shift in the digital economy, and the automotive industry is about to experience a high-octane transformation. Get ready to  explore the user-centric approach , new data-sharing mechanisms, and the roadmap for OEMs to adapt and thrive in the European data market. Are you prepared for this journey?

Key takeaways

•  The EU Data Act grants users ownership and control of their data while introducing obligations for automotive OEMs to ensure fair competition.
•  The Act facilitates data sharing between users, enterprises, and public sector bodies to promote innovation in the European automotive industry.
•  Automotive OEMs must invest in resources and technologies to comply with the EU Data Act regulations for optimal growth opportunities.

The EU Data Act and its impact on the automotive industry

The EU Data Act applies to manufacturers, suppliers, and users of products or services placed on the market in the EU, as well as data holders and recipients based in the EU.

What is the EU Data Act regulation?

The EU Data Act is a  proposed regulation that seeks to harmonize rules on fair access to and use of data in the European Union. The regulation sets out clear guidelines on who is obliged to surrender data, who can access it, how it can be used, and for what specific purposes it can be utilized.

In June 2023, the European Union took a significant step towards finalizing the Data Act, marking a pivotal moment in data governance. While the Act awaits formal adoption by the Council and Parliament following a legal-linguistic revision, the recent informal political agreement suggests its inevitability. This groundbreaking regulation will accelerate the monetization of industrial data while ensuring a harmonized playing field across the European Union.

User-centric approach

The European Data Act is revving up the engines of change in the automotive sector, putting users in the driver’s seat of their data and imposing specific obligations on OEMs. This means that connected products and related services must provide users with direct access to data generated in-vehicle, without any additional costs, and in a secure, structured, and machine-readable format.

Data handling by OEMs

A significant change is about to happen in data practices, particularly for OEMs operating in the automotive industry. Manufacturers and designers of smart products, such as smart cars, will be required to share data with users and authorized third parties. This shared data includes a wide range of information:

 Included in the Sharing Obligation: The data collected during the user's interaction with the smart car that includes information about the car's operation and environment. This information is gathered from onboard applications such as GPS and sensor images, hardware status indications, as well as data generated during times of inaction by the user, such as when the car is on standby or switched off. Both raw and pre-processed data are collected and analyzed.

 Excluded from the Sharing Obligation: Insights derived from raw data, any data produced when the user engages in activities like content recording or transmitting, and any data from products designed to be non-retrievable are not shared.

Sharing mechanisms and interactions

Data holders must make vehicle-generated data available (including associated metadata) promptly, without charge, and in a structured, commonly used, machine-readable format.

The legal basis for sharing personal data with connected vehicle users and legal entities or data recipients other than the user varies depending on the data subject and the sector-specific legislation to be presented.

Data access and third-party services

The Data Act identifies eligible entities for data sharing, encompassing both physical persons, such as individual vehicle owners or lessees, and legal persons, like organizations operating fleets of vehicles.

Requesting data sharing

Data can be accessed by  users who are recipients either directly from the device's storage or from a remote server that captures the data. In cases where the data cannot be accessed directly, the manufacturers must promptly provide it.

The data must be free, straightforward, secure, and formatted for machine readability, and its quality should be maintained where necessary. There may be contracts that limit or deny access or further distribution of data if it breaches legal security requirements. This is a critical aspect for smart cars where sharing data might pose a risk to personal safety.

If  the recipient of data is a third party , they cannot use the data to create competing products, only for maintenance. They cannot share the data unless it is for providing a user service and cannot prevent users who are consumers from sharing it with other parties.

Fair competition and trade secrets

The Data Act mandates that manufacturers share data, even when it is protected by trade secret laws. However, safeguards exist, allowing OEMs to impose confidentiality obligations and withhold data sharing in specific circumstances. These provisions ensure a balance between data access and trade secret protection. During the final negotiations on the Data Act, safeguarding trade secrets was a primary focus.

The Data Act now has provisions to prevent potential abusive behavior by data holders. It also includes an exception to data-sharing that permits manufacturers to reject certain data access requests if they can prove that such access would result in the disclosure of trade secrets, leading to severe and irreversible economic losses.

Connected vehicle data

Connected vehicle data takes the spotlight under the EU Data Act, empowering users with real-time access to their data and enabling data sharing with repair or service providers.

The implementation of the Data Act heavily involves connected cars. As per the Act, users, including companies, have the right to access the data collected by vehicles. However, manufacturers have the option to limit access under exceptional circumstances. This has a significant impact on data collection practices in the automotive sector.

Preparing for the EU Data Act: A guide for automotive OEMs

To stay ahead of the curve, OEMs must understand the business implications of the Data Act, adapt to new regulations, and invest in the necessary resources and technologies to ensure compliance.

As connected vehicles become the norm, OEMs that embrace the Data Act will be well-positioned to capitalize on new opportunities and drive growth in the European automotive sector.

Business implications

The EU Data Act imposes significant business implications on automotive OEMs, necessitating changes in their data handling practices and adherence to new obligations. As the industry embraces the user-centric approach to data handling, OEMs must design connected products and related services that provide users with access to their in-vehicle data.

To ensure a smooth transition and maintain a competitive edge, automotive OEMs must undertake a tailored and strategic preparation process.

Adapting to new regulations

Failure to comply with the Data Act could result in legal and financial repercussions for automotive OEMs. In order to avoid any possible problems, they should invest in the necessary resources and technologies to ensure compliance with the regulations of the Data Act.

They should also engage proactively with the requirements of the Data Act and implement compliance measures strategically.

By taking the following steps, automotive OEMs can navigate the regulatory landscape effectively and seize growth opportunities in the European automotive sector:

 In-Depth Knowledge: Dive deep into the EU Data Act, with a special focus on its impact on the automotive industry. Recognize that the automotive sector is central to this regulation, requiring industry-specific understanding.

 Data Segmentation: Perform a comprehensive analysis of your data, categorizing it into distinct groups. Identify which data types fall within the purview of the EU Data Act.

 Compliance Framework Development:

•     Internal Compliance:    Audit and update policies to comply with the EU Data Act. Develop a data governance framework for access, sharing, and privacy.
•     Data Access Protocols:    Establish unambiguous protocols for data access and sharing, including procedures for obtaining user consent, data retrieval, and sharing modalities.

 Data Privacy and Security:

•     Data Safeguards:    Enhance data privacy and security, including encryption and access controls.

 Data Utilization: Develop plans for leveraging this data to generate new revenue streams while adhering to the EU Data Act's mandates.

 User Engagement and Consent:

•     Transparency:    Forge clear and transparent channels of communication with users. Keep users informed about data collection, sharing, and usage practices, and obtain well-informed consent.
•     Consent Management:    Implement robust consent management systems to efficiently monitor and administer user consent. Ensure that users maintain control over their data.

 Legal Advisors: Engage legal experts well-versed in data protection and privacy laws, particularly those relevant to the automotive sector. Seek guidance for interpreting and implementing the EU Data Act within your specific industry context.

 Data Access Enhancement: Invest in technology infrastructure to facilitate data access and sharing as per the EU Data Act's stipulations. Ensure that data can be easily and securely provided in the required format.

 Employee Education: Educate your workforce on the intricacies of the EU Data Act and its implications for daily operations. Ensure that employees possess a strong understanding of data protection principles.

 Ongoing Compliance Oversight: Establish mechanisms for continuous compliance monitoring. Regularly assess data practices, consent management systems, and data security protocols to identify and address compliance gaps.

 Collaboration with Peers: Collaborate closely with industry associations, fellow automotive OEMs, and stakeholders to share insights, best practices, and strategies for addressing the specific challenges posed by the EU Data Act in the automotive sector.

 Future-Ready Solutions: Develop adaptable and scalable solutions that accommodate potential regulatory landscape shifts. Remain agile and prepared to adjust strategies as needed.

Boosting innovation capabilities

The Data Act may bring some challenges, but it also creates a favorable environment for innovation. By making industrial data more accessible, the Act offers a huge potential for data-driven businesses to explore innovative business models. Adapting to the Act can improve a company's ability to innovate, allowing it to use data as a strategic asset for growth and differentiation.

Summary

The EU Data Act is driving a paradigm shift in the automotive sector, putting users in control of their data and revolutionizing the way OEMs handle, share, and access vehicle-generated data.

By embracing the user-centric approach, ensuring compliance with data sharing and processing provisions, and investing in innovation capabilities, data holders can unlock new opportunities and drive growth in the European automotive market.

It's time for OEMs to take  actionable steps to comply with the new regulation .  Read this guide on building EU Data Act-compliant connected car software to learn what they are.

Get prepared to meet the EU Data Act deadlines

Ready to turn compliance into a competitive advantage?  We’re here to assist you , whether you need expert guidance on regulatory changes or customized data-sharing solutions.

‍

Our  initial article on predictive maintenance covered the definition of such a system, its construction, and the key implementation challenges. In this part, we'll delve into how PdM technology is transforming different facets of the automotive industry and its advantages for OEMs, insurers, car rental companies, and vehicle owners.

Best predictive maintenance techniques and where you can use them

In the first part of the article, we discussed the importance of sensors in a PdM system. These sensors are responsible for collecting data from machines and vehicles, and they can measure various variables like temperature, vibration, pressure, or noise. Proper placement of these sensors on the machines and connecting them to IoT solutions, enables the transfer of data to the central repository of the system. After processing the data, we obtain information about specific machines or their parts that are prone to damage or downtime.

The automotive industry can benefit greatly from implementing these top predictive maintenance techniques.

Vibration analysis

 How does it work?

Machinery used in the automotive industry and car components have a specific frequency of vibration. Deviations from this standard pattern can indicate "fatigue" of the material or interference from a third-party component that may affect the machine's operation. The PdM system enables you to detect these anomalies and alert the machine user before a failure occurs.

 What can be detected?

The technique is mainly applied to high-speed rotating equipment. Vibration and oscillation analysis can detect issues such as bent shafts, loose mechanical components, engine problems, misalignment, and worn bearings or shafts.

Infrared thermography analysis

 How does it work?

The technique involves using infrared cameras to detect thermal anomalies. This technology can identify malfunctioning electrical circuits, sensors or components that are emitting excessive heat due to overheating or operating at increased speeds. With this advanced technology, it's possible to anticipate and prevent such faults, and even create heat maps that can be used in predictive models and maintenance of heating systems.

 What can be detected?

Infrared analysis is a versatile and non-invasive method that can be used on a wide scale. It is suitable for individual components, parts, and entire industrial facilities, and can detect rust, delamination, wear, or heat loss on various types of equipment.

Acoustic analysis monitoring

 How does it work?

Machines produce sound waves while operating, and these waves can indicate equipment failure or an approaching critical point. The amplitude and character of these waves are specific to each machine. Even if the sound is too quiet for humans to hear in the initial phase of malfunction, sensors can detect abnormalities and predict when a failure is likely to occur.

 What can be detected?

This PdM technology is relatively cheaper compared to others, but it does have some limitations in terms of usage. It is widely used in the Gas & Oil industry to detect gas and liquid leaks. In the automotive industry, it is commonly used for detecting vacuum leaks, unwanted friction, and stress on machine parts.

Motor circuit analysis

 How does it work?

The technique works through electronic signature analysis (ESA). It involves measuring the supply voltage and operating current of an electronic engine. It allows locating and identifying problems related to the operation of electric engine components.

 What can be detected?

Motor circuit analysis is a powerful tool that helps identify issues related to various components, such as bearings, rotor, clutch, stator winding, or system load irregularities. The main advantage of this technique is its short testing time and convenience for the operator, as it can be carried out in just two minutes while the machine is running.

PdM oil analysis

 How does it work?

An effective method for Predictive Maintenance is to analyze oil samples from equipment without causing any damage. By analyzing the viscosity and size of the sample, along with detecting the presence or absence of third substances such as water, metals, acids or bases, we can obtain valuable information about mechanical damage, erosion or overheating of specific parts.

 What can be detected?

Detecting anomalies early is crucial for hydraulic systems that consist of rotating and lubricating parts, such as pistons in a vehicle engine. By identifying issues promptly, effective solutions can be developed and potential damage to the equipment or a failure can be prevented.

Computer vision

 How does it work?

Computer vision is revolutionizing the automotive industry by leveraging AI-based technology to enhance predictive maintenance processes. It achieves this by analyzing vast datasets, including real-time sensor data and historical performance records, to rapidly predict equipment wear and tear. By identifying patterns, detecting anomalies, and issuing early warnings for potential equipment issues, computer vision enables proactive maintenance scheduling.

 What can be detected?

In the automotive industry, computer vision technology plays a crucial role in detecting equipment wear and tear patterns to predict maintenance requirements. It can also identify manufacturing defects such as scratches or flaws, welding defects in automotive components, part dimensions and volumes to ensure quality control, surface defects related to painting, tire patterns to match with wheels, and objects for robotic guidance and automation.

Who and how can benefit from predictive maintenance

Smart maintenance systems analyze multiple variables and provide a comprehensive overview, which can benefit several stakeholders in the automotive industry. These stakeholders range from vehicle manufacturing factories and the supply chain to service and dealerships, rental companies, insurance companies, and drivers.

Below, we have outlined the primary benefits that these stakeholders can enjoy. In the OEMs section, we have provided examples of specific implementations and case studies from the market.

Car rentals

 Fleet health monitoring and better prediction of the service time

Managing service and repairs for a large number of vehicles can be costly and time-consuming for rental companies. When vehicles break down or are out of service while in the possession of customers, it can negatively impact the company’s revenue. To prevent this, car rental companies need constant insight into the condition of their vehicles and the ability to predict necessary maintenance. This allows them to manage their service plan more efficiently and minimize the risk of vehicle failure while on the road.

Car dealerships

 Reducing breakdown scenarios

Car dealerships use predictive maintenance primarily to anticipate mechanical issues before they develop into serious problems. This approach helps in ensuring that vehicles sold or serviced by them are in optimal condition, which aids in preventing breakdowns or major faults for the customer down the line. By analyzing data from the vehicle's onboard sensors and historical maintenance records, dealerships can identify patterns that signify potential future failures. Predictive maintenance also benefits dealerships by allowing for proactive communication with vehicle owners, reducing breakdown scenarios, and enhancing customer satisfaction

Vehicle owners

 Peace of mind

Periodic maintenance recommendations for vehicles are traditionally based on analyzing historical data from a large population of vehicle owners. However, each vehicle is used differently and could benefit from a tailored maintenance approach. Vehicles with high mileage or heavy usage should undergo more frequent oil changes than those that are used less frequently. By monitoring the actual vehicle condition and wear, owners can ensure that their vehicles are always at 100% and can better manage and plan for maintenance expenses.

Insurance companies

 Risk & fraud

By using data from smart maintenance systems, insurance companies can enhance their risk modeling. The analysis of this data allows insurers to identify the assets that are at higher risk of requiring maintenance or replacement and adjust their premiums accordingly. In addition, smart maintenance systems can detect any instances of tampering with the equipment or negligence in maintenance. This can aid insurers in recognizing fraudulent claims.

OEMs successful development of PdM systems

BMW Group case study

The German brand implements various predictive maintenance tools and technologies, such as sensors, data analytics, and artificial intelligence, to prevent production downtime, promote sustainability, and ensure efficient resource utilization in its global manufacturing network. These innovative, cloud-based solutions are playing a vital role in enhancing their manufacturing processes and improving overall productivity.

The BMW Group's approach involves:

•     Forecasting phenomena and anomalies using a cloud-based platform.    Individual software modules within the platform can be easily switched on and off if necessary to instantly adapt to changing requirements. The high degree of standardization between individual components allows the system to be globally accessible. Moreover, it is highly scalable and allows new application scenarios to be easily implemented.
•     Optimizing component replacements       (this uses advanced real-time data analytics).  
•     Carrying out maintenance and service work in line with the requirements of the actual status of the system.  
•     Anomaly detection using advanced AI predictive algorithms.  

Meanwhile, it should be taken into account that in BMW's body and paint shop alone, welding guns perform some 15,000 spot welds per day. At the BMW Group's plant in Regensburg, the conveyor systems' control units run 24/7. So any downtime is a huge loss.

→  SOURCE case study.

FORD case study

Predictive vehicle maintenance is one of the benefits offered to drivers and automotive service providers as part of Ford's partnerships with CARUSO and HIGH MOBILITY.  In late 2020, Ford announced two new connected car agreements to potentially enable vehicle owners to benefit from  a personalized third-party offer.

CARUSO and HIGH MOBILITY will function as an  online data platform that is completely independent of Ford and allows  third-party service providers secure and compliant access to vehicle-generated data. This access will, in turn, enable third-party providers to create personalized services for Ford vehicle owners. This will enable drivers to benefit from smarter  insurance, technical maintenance and roadside recovery.

Sharing vehicle data (warning codes, GPS location, etc.) via an  open platform is expected to be a way to maintain competitiveness in the connected mobility market.

→  SOURCE case study.

Predictive maintenance is the future of the automotive market

An effective PdM system means less time spent on equipment maintenance, saving on spare parts, eliminating unplanned downtime and improved management of company resources. And with that comes more efficient production and customers’ and employees’ satisfaction.

As the data shows, organizations that have implemented a PdM system report an average  decrease of 55% in unplanned equipment failures.  Another upside is that, compared to other connected car systems (such as infotainment systems), PdM is relatively easy to monetize. Data here can remain anonymous, and all parties involved in the production and operation of the vehicle reap the benefits.

Organizations have come to recognize the hefty returns on investment provided by predictive maintenance solutions and have thus adopted it on a global scale. According to Market Research Future, the global Predictive Maintenance market is projected to grow to  111.30 billion by 2030 , suggesting that further growth is possible in the future.

Android Automotive OS 14 has arrived, and it marks a significant evolution in the way users interact with their vehicle's system. This version brings enhanced user experience, improved Android API, and better OS-level security (as well as non-automotive Android 14). In this short article, we'll walk you through a tutorial on creating your own emulator from scratch, but first, here are some of the standout features and improvements introduced in Android Automotive OS 14  !

Android Automotive 14 noteworthy new features

•     Enhanced UI:    Now with an optional, improved home screen adaptation to the portrait mode for better vehicle compatibility.
•     Multi-User Upgrades:    Support parallel sessions with custom sound zones and multiple displays.
•     Remote Access:    Enables system wake-up, executes a task and then shutdown via external requests.
•     Extended VHAL:    More ADAS and non-ADAS properties included to represent activation status and the system state.
•     App Quick Actions:    A feature that allows applications to showcase quick actions.
•     Infotainment Reference Design:    The starting point for developers to create apps for Android Automotive OS.
•     New Boot Animation:    Well, as usual 😊

To learn about all new features provided in Android Automotive 14, follow this link:  https://source.android.com/docs/automotive/start/releases/u_udc_release?hl=en

Steps to building an emulator

The best operating system for building an emulator in AAOs is Ubuntu 18.04 or higher. If you use a different operating system, you must follow some extra steps. For instance, you may need to install a repo from  https://gerrit.googlesource.com/git-repo instead of using a package manager.

1)     You need first to install the required dependencies

sudo apt install git-core gnupg flex bison build-essential zip curl zlib1g-dev libc6-dev-i386 libncurses5 x11proto-core-dev libx11-dev lib32z1-dev libgl1-mesa-dev libxml2-utils xsltproc unzip fontconfig repo

2)     Then, configure Git, set your name and email address

git config --global user.name "Your name"
git config --global user.email your@email


3) After configuring Git, you can download source code from a Git repository

repo init -u https://android.googlesource.com/platform/manifest -b android-14.0.0_r54 --partial-clone --clone-filter=blob:limit=10M && repo sync

You can skip  ‐‐ partial-clone and  ‐‐ clone-filter. However, this will result in longer download times. It’s recommended to check for the latest android-14.0.0_rXX tag before downloading, which can be found on this page:  https://android.googlesource.com/platform/manifest/+refs .

Keep in mind that downloading takes a lot of time because the sources take about 150GB even with partial clone and clone-filter enabled.

4)     In the next step, set up environment variables using the script provided

. build/envsetup.sh

This method replaces your JAVA_HOME and modifies PATH, so be aware that your console may act differently now.

5)     Select the system to build

lunch sdk_car_portrait_x86_64-eng

You can create a landscape build by removing "portrait". Also, change x86_64 to arm64 if you want to run the system on Mac. For more details on building on Mac, check out  this article .

6)     Create the system and the emulator image

m && m emu_img_zip

The first command will take hours to complete. Take a break: go running, biking, hiking, or whatever drives you. You can modify threat pool usage by the build system with -j parameter, like m -j 16 – the default one is the CPU count of your machine.

7)     Copy the emulator image to Android Studio emulator directory

mkdir -p /mnt/c/Users/<user>/AppData/Local/Android/Sdk/system-images/android-34/custom_aaos_14/ && unzip -o out/target/product/emulator_x86_64/sdk-repo-Linux-system-images-eng.dape.zip -d /mnt/c/Users/<user>/AppData/Local/Android/Sdk/system-images/android-34/custom_aaos_14/

I assume you work on a Windows machine with WSL. Please adapt the above commands with your Android/SDK directory if you are working on native Linux.

Create a package.xml file in /mnt/c/Users/<user>/AppData/Local/Android/Sdk/system-images/android-34/custom_aaos_14/x86_64 directory with the  this content . The file provided bases on existing package.xml files in other emulator images.

Adjust “tag”, “vendor”, and “display name” in the upper file if needed. Make sure to match  <localPackage obsolete="false" path="system-images;android-34;custom_aaos_14;x86_64"> with the path you’d placed the emulator image.

8)     Now it’s time to create a new emulator in Android Studio

Open "Device Manager" and select "Create Virtual Device". In the left-hand menu, choose "Automotive" and add a new hardware profile using the button in the lower-left corner of the panel.

Select “Android Automotive” as a device type. Choose the correct resolution for your build. For example, I selected a resolution of 1152x1536 for a 10-inch device to create a portrait build. Next, allocate at least 1536 MB of RAM to your device. Then, choose only one supported device state - "Portrait" or "Landscape" - according to your build. Finally, disable any unnecessary sensors and skin for AAOS compatibility.

9)     Accept and select your new hardware profile. Then, move on to the next step

10) Pick your emulator image (you can find it using the tag and vendor configured in package.xml)

11) On the final screen, enter a name and complete the configuration process

12) To start the emulator, go to the "Device Manager" and launch it from there

13) You’re all set! Enjoy!

Get started on creating your very own Android Automotive OS 14 emulator by following the steps outlined in this article. Explore the possibilities of car technology and discover what the future has in store. You can find a AAOS “Hello World” example in our article  How to Build Your First App for Android Automotive OS . Start building, try out the various features, and have fun with your new setup!

Welcome to the second part of our article series about MongoDB in automotive. In the  previous installment , we explored the power of MongoDB as a versatile data management solution for the automotive industry, focusing on its flexible data model, scalability, querying capabilities, and optimization techniques.

In this continuation, we will delve into two advanced features of MongoDB that further enhance its capabilities in automotive applications: timeseries data management and change streams. By harnessing the power of timeseries and change streams, MongoDB opens up new possibilities for managing and  analyzing real-time data in the automotive domain. Join us as we uncover the exciting potential of MongoDB's advanced features and their impact on driving success in automotive applications.

Time-series data management in MongoDB for automotive

Managing time-series data effectively is a critical aspect of many automotive applications. From sensor data to vehicle telemetry, capturing and analyzing time-stamped data is essential for monitoring performance, detecting anomalies, and making informed decisions. MongoDB offers robust features and capabilities for managing time-series data efficiently. This section will explore the key considerations and best practices for leveraging MongoDB's time-series collections.

Understanding time-series collections

Introduced in MongoDB version 5.0, time-series collections provide a specialized data organization scheme optimized for storing and retrieving time-series data. Time-series data consists of a sequence of data points collected at specific intervals, typically related to time. This could include information such as temperature readings, speed measurements, or fuel consumption over time.

MongoDB employs various optimizations in a time-series collection to enhance performance and storage efficiency. One of the notable features is the organization of data into buckets. Data points within a specific time range are grouped in these buckets. This time range, often referred to as granularity, determines the level of detail or resolution at which data points are stored within each bucket. This bucketing approach offers several advantages.

Firstly, it improves query performance by enabling efficient data retrieval within a particular time interval. With data organized into buckets, MongoDB can quickly identify and retrieve the relevant data points, reducing the time required for queries.

Secondly, bucketing allows for efficient storage and compression of data within each bucket. By grouping data points, MongoDB can apply compression techniques specific to each bucket, optimizing disk space utilization. This helps to minimize storage requirements, especially when dealing with large volumes of time-series data.

Choosing sharded or non-sharded collections

When working with time-series data, you have the option to choose between sharded and non-sharded collections. Sharded collections distribute data across multiple shards, enabling horizontal scalability and accommodating larger data volumes. However, it's essential to consider the trade-offs associated with sharding. While sharded collections offer increased storage capacity, they may introduce additional complexity and potentially impact performance compared to non-sharded collections.

In most cases, non-sharded collections are sufficient for managing time-series data, especially when proper indexing and data organization strategies are employed. Non-sharded collections provide simplicity and optimal performance for most time-series use cases, eliminating the need for managing a sharded environment.

Effective data compression for time-series collections

Given the potentially large volumes of data generated by time-series measurements, efficient data compression techniques are crucial for optimizing storage and query performance. MongoDB provides built-in compression options that reduce data size, minimizing storage requirements and facilitating faster data transfer. Using compression, MongoDB significantly reduces the disk space consumed by time-series data while maintaining fast query performance.

One of the key compression options available in MongoDB is the WiredTiger storage engine. WiredTiger offers advanced compression algorithms that efficiently compress and decompress data, reducing disk space utilization. This compression option is particularly beneficial for time-series collections where data points are stored over specific time intervals.

By leveraging WiredTiger compression, MongoDB achieves an optimal balance between storage efficiency and query performance for time-series collections. The compressed data takes up less space on disk, resulting in reduced storage costs and improved overall system scalability. Additionally, compressed data can be transferred more quickly across networks, improving data transfer speeds and reducing network bandwidth requirements.

Considerations for granularity and data retention

When designing a time-series data model in MongoDB, granularity and data retention policies are important factors to consider. Granularity refers to the level of detail or resolution at which data points are stored, while data retention policies determine how long the data is retained in the collection.

Choosing the appropriate granularity is crucial for striking a balance between data precision and performance. MongoDB provides different granularity options, such as "seconds," "minutes," and "hours," each covering a specific time span. Selecting the granularity depends on the time interval between consecutive data points that have the same unique value for a specific field, known as the meta field. You can optimize storage and query performance by aligning the granularity with the ingestion rate of data from a unique data source.

For example, if you collect temperature readings from weather sensors every five minutes, setting the granularity to "minutes" would be appropriate. This ensures that data points are grouped in buckets based on the specified time span, enabling efficient storage and retrieval of time-series data.

In addition to granularity, defining an effective data retention policy is essential for managing the size and relevance of the time-series collection over time. Consider factors such as the retention period for data points, the frequency of purging outdated data, and the impact on query performance.

MongoDB provides a Time to Live (TTL) mechanism that can automatically remove expired data points from a time-series collection based on a specified time interval. However, it's important to note that there is a known issue related to TTL for very old records in MongoDB at the time of writing this article. The issue is described in detail in the MongoDB Jira ticket SERVER-76560.

The TTL behavior in time series collections differs from regular collections. In a time series collection, TTL expiration occurs at the bucket level rather than on individual documents within the bucket. Once all documents within a bucket have expired, the entire bucket is removed during the next run of the background task that removes expired buckets.

This bucket-level expiration behavior means that TTL may not work in the exact same way as with normal collections, where individual documents are removed as soon as they expire. It's important to be aware of this distinction and consider it when designing your data retention strategy for time series collections.

When considering granularity and data retention policies, evaluate the specific requirements of your automotive application. Consider the level of precision needed for analysis, the data ingestion rate, and the desired storage and query performance. By carefully evaluating these factors and understanding the behavior of TTL in time series collections, you can design a time-series data model in MongoDB that optimizes both storage efficiency and query performance while meeting your application's needs.

Retrieving latest documents

In automotive applications, retrieving the latest documents for each unique meta key can be a common requirement. MongoDB provides an efficient approach to achieve this using the `DISTINCT_SCAN` stage in the aggregation pipeline. Let's explore how you can use this feature, along with an automotive example.

The `DISTINCT_SCAN` stage is designed to perform distinct scans on sorted data in an optimized manner. By leveraging the sorted nature of the data, it efficiently speeds up the process of identifying distinct values.

To illustrate its usage, let's consider a scenario where you have a time series collection of vehicle data that includes meta information and timestamps. You want to retrieve the latest document for each unique vehicle model. Here's an example code snippet demonstrating how to accomplish this:

```javascript
db.vehicleData.aggregate([
 { $sort: { metaField: 1, timestamp: -1 } },
 {
   $group: {
     _id: "$metaField",
     latestDocument: { $first: "$$ROOT" }
   }
 },
 { $replaceRoot: { newRoot: "$latestDocument" } }
])
```


In the above code, we first use the `$sort` stage to sort the documents based on the `metaField` field in ascending order and the `timestamp` field in descending order. This sorting ensures that the latest documents appear first within each group.

Next, we employ the `$group` stage to group the documents by the `metaField` field and select the first document using the `$first` operator. This operator retrieves the first document encountered in each group, corresponding to the latest document for each unique meta key.

Finally, we utilize the `$replaceRoot` stage to promote the `latestDocument` to the root level of the output, effectively removing the grouping and retaining only the latest documents.

By utilizing this approach, you can efficiently retrieve the latest documents per each meta key in an automotive dataset. The `DISTINCT_SCAN` stage optimizes the distinct scan operation, while the `$first` operator ensures accurate retrieval of the latest documents.

It's important to note that the `DISTINCT_SCAN` stage is an internal optimization technique of MongoDB's aggregation framework. It is automatically applied when the conditions are met, so you don't need to specify or enable it in your aggregation pipeline explicitly.

Time series collection limitations

While MongoDB Time Series brings valuable features for managing time-series data, it also has certain limitations to consider. Understanding these limitations can help developers make informed decisions when utilizing MongoDB for time-series data storage:

●  Unsupported Features : Time series collections in MongoDB do not support certain features, including transactions and change streams. These features are not available when working specifically with time series data.

● A  ggregation $out and $merge : The $out and $merge stages of the aggregation pipeline, commonly used for storing aggregation results in a separate collection or merging results with an existing collection, are not supported in time series collections. This limitation affects the ability to perform certain aggregation operations directly on time series collections.

●  Updates and Deletes : Time series collections only support insert operations and read queries. This means that once data is inserted into a time series collection, it cannot be directly modified or deleted on a per-document basis. Any updates or manual delete operations will result in an error.

MongoDB change streams for real-time data monitoring

MongoDB Change Streams provide a powerful feature for real-time data monitoring in MongoDB. Change Streams allow you to capture and react to any changes happening in a MongoDB collection in a real-time manner. This is particularly useful in scenarios where you need to track updates, insertions, or deletions in your data and take immediate actions based on those changes.

Change Streams provide a unified and consistent way to subscribe to the database changes, making it easier to build reactive applications that respond to real-time data modifications.

```javascript
// MongoDB Change Streams for Real-Time Data Monitoring

const MongoClient = require('mongodb').MongoClient;

// Connection URL
const url = 'mongodb://localhost:27017';

// Database and collection names
const dbName = 'mydatabase';
const collectionName = 'mycollection';

// Create a change stream
MongoClient.connect(url, function(err, client) {
 if (err) throw err;

 const db = client.db(dbName);
 const collection = db.collection(collectionName);

 // Create a change stream cursor with filtering
 const changeStream = collection.watch([{ $match: { operationType: 'delete' } }]);

 // Set up event listeners for change events
 changeStream.on('change', function(change) {
   // Process the delete event
   console.log('Delete Event:', change);
   // Perform further actions based on the delete event
 });

 // Close the connection
 // client.close();
});
```


In this updated example, we use the `$match` stage in the change stream pipeline to filter for delete operations only. The `$match` stage is specified as an array in the `watch()` method. The `{ operationType: 'delete' }` filter ensures that only delete events will be captured by the change stream.

Now, when a delete operation occurs in the specified collection, the `'change'` event listener will be triggered, and the callback function will execute. Inside the callback, you can process the delete event and perform additional actions based on your application's requirements. It's important to note that the change stream will only provide the document ID for delete operations. The actual content of the document is no longer available. If you need the document content, you need to retrieve it before the delete operation or store it separately for reference.

One important aspect to consider is related to the inability to distinguish whether a document was removed manually or due to the Time to Live (TTL) mechanism. Change streams do not provide explicit information about the reason for document removal. This means that when a document is removed, the application cannot determine if it was deleted manually by a user or automatically expired through the TTL mechanism. Depending on the use case, it may be necessary to implement additional logic or mechanisms within the application to handle this distinction if it is critical for the business requirements.

Here are some key aspects and benefits of using MongoDB Change Streams for real-time data monitoring:

●  Real-Time Event Capture : Change Streams allow you to capture changes as they occur, providing a real-time view of the database activity. This enables you to monitor data modifications instantly and react to them in real-time.

●  Flexibility in Filtering : You can specify filters and criteria to define the changes you want to capture. This gives you the flexibility to focus on specific documents, fields, or operations and filter out irrelevant changes, optimizing your monitoring process.

●  Data Integration and Pipelines : Change Streams can be easily integrated into your existing data processing pipelines or applications. You can consume the change events and perform further processing, transformation, or analysis based on your specific use case.

●  Scalability and High Availability : Change Streams are designed to work seamlessly in distributed and sharded MongoDB environments. They leverage the underlying replica set architecture to ensure high availability and fault tolerance, making them suitable for demanding and scalable applications.

●  Event-Driven Architecture : Using Change Streams, you can adopt an event-driven architecture for your MongoDB applications. Instead of continuously polling the database for changes, you can subscribe to the change events and respond immediately, reducing unnecessary resource consumption.

MongoDB Change Streams provide a powerful mechanism for real-time data monitoring, enabling you to build reactive, event-driven applications and workflows. By capturing and processing database changes in real-time, you can enhance the responsiveness and agility of your applications, leading to improved user experiences and efficient data processing.

Another consideration is that low-frequency events may lead to an invalid resume token. Change streams rely on a resume token to keep track of the last processed change. In cases where there are long periods of inactivity or low-frequency events, the resume token may become invalid or expired. Therefore, the application must handle this situation gracefully and take appropriate action when encountering an invalid resume token. This may involve reestablishing the change stream or handling the situation in a way that ensures data integrity and consistency.

It's important to note that while Change Streams offer real-time monitoring capabilities, they should be used judiciously, considering the potential impact on system resources. Monitoring a large number of collections or frequently changing data can introduce additional load on the database, so it's essential to carefully design and optimize your Change Streams implementation to meet your specific requirements.

Conclusion

By harnessing the capabilities of MongoDB, developers can unlock a world of possibilities in modern application development. From its NoSQL model to efficient time-series data management, batch writes, data retrieval, and real-time monitoring with Change Streams, MongoDB provides a powerful toolkit. By following best practices and understanding its limitations, developers can maximize the potential of MongoDB, resulting in scalable, performant, and data-rich applications.

MongoDB can be likened to a Swiss Army Knife in the world of database systems, offering a versatile set of features and capabilities. However, it is important to note that MongoDB is not a one-size-fits-all solution for every use case. This article series aim to showcase the capabilities and potential use cases of MongoDB.

While MongoDB provides powerful features like time-series data management and change streams for real-time data monitoring, it is essential to consider alternative solutions as well. Depending on factors such as team skills, the company-adopted tools, and specific project requirements, exploring other options like leveraging IaaS provider-native solutions such as DynamoDB for specific use cases within the automotive domain may be worthwhile.

Furthermore, it is important to highlight that both articles focused on using MongoDB in automotive primarily for hot storage, while the aspect of cold storage for automotive applications is not covered.

When starting a new project, it is crucial to conduct thorough research, consider the specific requirements, and evaluate the available options in the market. While MongoDB may provide robust features for certain scenarios, dedicated time-series databases like InfluxDB may offer a more tailored and specialized solution for specific time-series data needs. Choosing the right tool for the job requires careful consideration and an understanding of the trade-offs and strengths of each option available.

The automotive sector has rapidly evolved, recognizing that cutting-edge software is now the primary factor differentiating car brands and models. Automotive leaders are reshaping the industry by creating innovative software-defined vehicles that seamlessly integrate advanced features, providing drivers with a genuinely captivating experience.

While some automotive manufacturers have begun building in-house expertise, others are turning to third-party software development companies for assistance. However, such partnerships can take on different forms depending on factors like the vendor’s size and specialization, which can sometimes lead to challenges in cooperation.

Still, success is possible if both parties adopt the right approach despite any problems that may arise. To ensure optimal collaboration, we recommend implementing the  Value-Aligned Partnership concept.

    By adopting this approach, vendors accept full accountability for delivery, match their contributions to the client's business goals, and use their experience to guarantee success.    We use this strategy at     Grape Up    to ensure our clients receive the best possible results.  

 This article explores the advantages and challenges of working with service vendors of various sizes and specializations. It also highlights the Value-Aligned Partnership model as a way to overcome these challenges and maximize the benefits of client-vendor cooperation.

Large vendors and system integrators

Working with multi-client software vendors can pose unique challenges for automotive companies, as these companies typically operate on a  body leasing model , providing temporary skilled personnel for development services.

Furthermore, these companies may lack  specialized expertise , which is particularly true for large organizations with high turnover rates in their development teams. Despite having impressive case studies, the team assigned to a specific project doesn't always possess the necessary experience due to  frequent personnel changes within the company.

Moreover, big vendors usually have numerous clients and projects to manage simultaneously. This may lead to limited personalized attention and cause  potential delays in delivery. Outsourcing services to countries with different time zones, quality assurance procedures, and cultural norms can further complicate the situation.

However, despite these challenges, such partnerships have significant advantages, which should be considered in the decision-making process.

Advantages of partnering with large vendors and system integrators

•     Access to diverse expertise    : A wide array of specialists is available for various project aspects.
•     Scalability advantage    : Project resource requirements are met without bottlenecks.
•     Accelerated time-to-market    : Development speed is increased with skilled, available teams.
•     Cost-efficiency    : It's a cost-effective alternative via vendor teams, reducing overhead.
•     Risk management    : Turnover risks are handled by vendors to ensure continuity.
•     Cross-industry insights application    : Diverse sector practices are applied to automotive projects.
•     Agile adaptation    : They are efficiently adjusting to changing project needs.
•     Enhanced global collaboration:    Creativity is boosted by leveraging time zones and diverse perspectives from vendors.

Working with smaller vendors

Your other option is to work with smaller software companies, but you should be aware of the potential challenges of such collaboration, as well. For example, while they are exceptionally well-versed in a particular field or technology, they might lack the breadth of knowledge necessary to meet the needs of the industry. Additionally,  limited resources or difficulties in scaling could hinder their ability to keep pace with the growing demands of the sector.

Furthermore, although these vendors  excel in specific technical areas , they may struggle to provide comprehensive end-to-end solutions, leaving gaps in the development process.

Adding to these challenges, smaller companies often have a more restricted perspective due to their  limited engagement in partnerships , infrequent conference attendance, and a narrower appeal that doesn't span a global audience. This can result in a lack of exposure to diverse ideas and practices, hindering their ability to innovate and adapt.

Benefits of working with smaller vendors

•     Tailored excellence:    Small vendors often craft industry-specific, innovative solutions without unnecessary features.
•     Personal priority:    They prioritize each client, ensuring dedicated attention and satisfaction.
•     Flexible negotiation:    Smaller companies offer negotiable terms and pricing, unlike larger counterparts.
•     Bespoke solutions:    They customize offerings based on your unique needs, unlike a generic approach.
•     Agile responsiveness:    Quick release cycles, technical agility, and transparent interactions are their strengths.
•     Meaningful connections:    They deeply care about your success, fostering personal relationships.
•     Accountable quality:    They take responsibility for their products, as development and support are integrated.

Niche specialization companies

Collaborating with a company specializing in the automotive sector offers distinct advantages, addressing challenges some large and small vendors face. Their  in-depth knowledge of the automotive industry ensures  tailored solutions that meet specific requirements efficiently.

As opposed to vendors who work in several industries, they are quick to adjust to shifts in the market, allowing software development projects to be successful and meet the ever-changing needs of the automobile sector.

It is important, however, to consider the potential drawbacks when entering partnerships that rely heavily on a narrow area of expertise. Niche solutions  may not be versatile , and specialization can lead to  higher costs and resistance to innovation . Additionally, overreliance on a single source could lead to  dependency concerns and a limited perspective on the market. It is important to weigh these risks against the benefits and ensure that partnerships are balanced to avoid stagnation and limited options.

When working with software vendors, no matter their size or specialization level, it is essential to adopt the right approach to cooperation to mitigate risks and challenges.

Value-aligned partnership model explained

This cooperation model prioritizes shared values, expertise, and cultural compatibility. It's important to note, in this context, that  a company's size doesn't limit its abilities to become a successful partner. Both large and small vendors can be just as driven and invested in professional development.

What matters most is a company's mindset and commitment to continuous improvement. Small and large businesses can excel by prioritizing a robust organizational culture, regular training sessions, knowledge sharing among employees, and partnerships that leverage the strengths of both parties.

Thus, the Value-Aligned Partnership is a model that brings together the benefits of working with different types of companies.  It combines the diverse expertise of large vendors with the agility and tailored solutions of small companies while incorporating vast industry-specific knowledge.

  In a value-aligned partnership model, the vendor goes beyond simply providing software development services. They actively engage in the journey towards success by fully     immersing themselves in the client's vision    . By thoroughly comprehending the customer's values and goals, a partner ensures that every contribution they make aligns seamlessly with the overall direction of the business and can even foster innovation within the client's company.  

 Building a strong partnership based on shared values takes time and effort, but it's worth it for the exceptional outcomes that result. Open communication, collaboration, and mutual understanding are key factors in creating a foundation for     long-term cooperation    and shared success between the two parties.

In the fast-paced and ever-changing automotive industry, having expertise specific to the domain is crucial.  A value-aligned partner recognizes the importance of  retaining in-house knowledge and skills related to the sector. As a company that prioritizes this approach, Grape Up invests in measures to  minimize turnover , provide ongoing training and education, and ensure that our team possesses  deep domain expertise . This commitment to automotive know-how strengthens the partnership's reliability and establishes us as a trusted, long-term ally for the automotive company.

Conclusion

The automotive industry is transforming remarkably with the rise of software-defined vehicles. OEMs realize they can't tackle this revolution alone and seek the perfect collaborators to join them on this exciting journey. These partners bring a wealth of expertise in software development, cloud technologies, artificial intelligence, and more. With their finger on the pulse of the automotive industry, they understand the ever-changing trends and challenges. They take the time to comprehend the OEM's vision, objectives, and market positioning, enabling them to provide tailored solutions that address specific needs.

‍