How to Build Software Architecture for New Mobility Services – Gathering Telemetry Data

In the modern world, tech companies strive to collect as much information as possible about the status of owned cars to enable proactive maintenance and rapid responses to any incidents that may occur. These incidents could involve theft, damage, or the cars simply getting lost. The only way to remotely monitor their status is by obtaining telemetry data sent by the vehicles and storing it on a server or in the cloud. There are numerous methods for gathering this data, but is there an optimal approach? Is there a blueprint for designing an architecture for such a system? Let’s explore.

What does “telemetry” mean in a car?

This article is about gathering telemetry data, so let’s begin with a quick reminder of what it is. Telemetry in cars refers to the technology that enables the remote collection and transmission of real-time data from various components of a vehicle to a central monitoring system. This data encompasses a wide range of parameters, including, for example:

• engine performance,
• fuel consumption,
• tire pressure,
• vehicle speed,
• percentage of electric vehicle battery,
• braking activity,
• acceleration,
• GPS position,
• odometer

Collecting vehicle details is valuable, but what is the real purpose of this information?

Why collect telemetry data from a car?

The primary use of telemetry data is to monitor a car’s status from anywhere in the world, and it’s especially crucial for companies like car rental firms such as Hertz or Europcar, as well as transportation companies like Uber. Here are some examples:

• Tracking Stolen Cars: Companies can quickly track a stolen vehicle if they store its GPS position.
• Accident Analysis: If a car is involved in an accident, the company can assess the likelihood of the event by analyzing data such as a sudden drop in speed to zero and high acceleration. This allows companies to provide replacement cars promptly.
• Fuel or Charging Management: In cases where a rental car is returned without a full tank of fuel or not fully charged, the company can respond quickly to make the car available for the next rental.

These are just a few examples of how telemetry data can be utilized, with many more possibilities. Understanding the value of telemetry data, let’s delve into the technical aspects of acquiring and using this data in the next part of the article.

To begin planning the architecture, we need answers to some fundamental questions

How will the telemetry data be used?

Architectural planning should commence with an understanding of the use cases for the collected telemetry data. This includes considering what the end user intends to do with the data and how they will access it. Common uses for this data include:

1. Sharing data on a dashboard: To enable this, an architecture should be designed to support an API that retrieves data from databases and displays it on a dashboard,
2. Data analytics: Depending on the specific needs, appropriate analytic tools should be planned. This can vary from real-time analysis (e.g. AWS Kinesis Data Analytics) to near real-time analysis (e.g. Kafka) or historical data analysis (e.g. AWS Athena),
3. Sharing data with external clients: If external clients require real-time data, it’s essential to incorporate a streaming mechanism into your architecture. If real-time access is not needed, a REST API should be part of the plan.

Can we collect the data from cars?

We should not collect any data from cars unless we either own the car or have a specific legal agreement to do so. This requires not only planning the architecture for acquiring access to the car but also for disposing of it. For example, if we collect telemetry or location data from a car through websockets and the company decides to sell the car, we should immediately cease tracking the car. Storing data from it, especially location data, might be illegal as it could potentially allow tracking of the location of a person inside the car.

How do we manage permissions to the car?

If we have legal permission to collect data from the car, we must include correct permission management in our architecture. Some key considerations include:

• Credential and token encryption,
• Secure storage of secrets, such as using AWS Secret Manager,
• Regular rotation of credentials and tokens for security,
• Implementing minimum access levels for services and vehicles,
• Good management of certificates,
• Adhering to basic security best practices.

How do we collect the data?

Now that we have access to the data, it’s time to consider how to collect it. There are several known methods to do this:

• Pull Through REST/GRPC API: In this scenario, you’ll need to implement a data poller. This approach may introduce latency in data acquisition and is not the most scalable solution. Additionally, you may encounter request throttling issues due to hitting request limits.
• External Service Push Through REST/GRPC: Here, you should set up a listener, which is essentially a service exposed with an endpoint, such as an ECS task or a Lambda function on AWS. This method might incur some costs, and it’s crucial to consider automatic scaling to ensure no data is lost. Keep in mind that the endpoint will be publicly exposed, so robust permission management is essential.
• Pulling From a Stream: This approach is often recommended as it’s the most scalable and secure option. You can receive data in real-time or near real-time, making it highly efficient. The primary considerations are access to the stream and the service responsible for pulling data from it.
• Queues: Similar to streams, queues can be used for data collection, and they may offer better data ordering. However, streams are typically faster but might be more expensive. This is another viable option for collecting vehicle data from external services.
• Websockets: Websockets are a suitable solution when bidirectional data flow is required, and they can be superior to REST/GRPC APIs in such cases. For example, they are an appropriate choice when a client needs confirmation that data has been successfully acquired. Websockets also allow you to specify which telemetry data can be acquired and at what frequency. A notable example is the Tesla Websockets (https://github.com/teslamotors/fleet-telemetry/blob/main/protos/vehicle_data.proto).

Where to store the data?

After collecting the data, it’s important to decide where to store it. There are various databases available, and the choice depends on your specific data use cases and access patterns. For instance:

• Commonly Used Data: For data that will be frequently accessed, you can opt for a traditional database like MongoDB or PostgreSQL.
• Low-Maintenance Database: If you prefer a database that requires minimal maintenance, AWS DynamoDB is a good choice.
• Infrequently Used Data for Analytics: When data won’t be used frequently but will be utilized for occasional data analytics, you can consider using an AWS S3 bucket with the appropriate storage tier, coupled with AWS Athena for data analysis.
• Complex Data Analysis: If the data will be regularly analyzed with complex queries, AWS Redshift might be a suitable solution.

When planning your databases, don’t forget to consider data retention. If historical data is no longer needed, it’s advisable to remove it to avoid excessive storage costs.

Example

Here is an example of such an architecture on AWS in which:

1. An employee grants permissions to the car to stream the data.
2. The data is streamed using AWS Kinesis Stream and saved to an S3 bucket by AWS Kinesis Firehose for audit purposes.
3. The data is also normalized by the AWS Lambda function and stored in AWS DynamoDB.
4. The stored data is queried by another AWS Lambda function.
5. The query Lambda is triggered by an AWS API Gateway to enhance security, such as limiting requests per second.
6. The API is exposed via Route 53 to the end user, which can be, for example, a dashboard or an external API.

Conclusion

In the modern tech landscape, the quest for complete vehicle data is a paramount objective. Tech companies seek to collect critical information about the status of owned cars to enable proactive maintenance and rapid responses to a spectrum of incidents, from theft and damage to simple misplacement. This imperative relies on the remote monitoring of vehicles through the collection and storage of data on servers or in the cloud, offering the capability to monitor a vehicle’s status from any corner of the globe. This is especially essential for companies like car rental firms and transportation services, with applications ranging from tracking stolen cars through GPS data to analyzing accident events and managing fuel or charging for rental vehicles.

The core of this mission is to strike a balance between data collection, security, and architectural planning. The process involves careful consideration of data collection methods, adherence to legal and security best practices, and informed choices for data storage solutions. The evolving landscape of vehicle data offers endless possibilities for tech companies to harness the power of telemetry and deliver an enhanced experience for their customers.