Integrating HVAC control in Android with DDS
As modern vehicles become more connected and feature-rich, the need for efficient and reliable communication protocols has grown. One of the critical aspects of automotive systems is the HVAC (Heating, Ventilation, and Air Conditioning) system , which enhances passenger comfort. This article explores how to integrate HVAC control in Android with the DDS (Data Distribution Service) protocol, enabling robust and scalable communication within automotive systems.
This article builds upon the concepts discussed in our previous article, "Controlling HVAC Module in Cars Using Android: A Dive into SOME/IP Integration." It is recommended to read that article first, as it covers the integration of HVAC with SOME/IP, providing foundational knowledge that will be beneficial for understanding the DDS integration described here.
What is HVAC?
HVAC systems in vehicles are responsible for maintaining a comfortable cabin environment. These systems regulate temperature, airflow, and air quality within the vehicle. Key components include:
- Heaters : Warm the cabin using heat from the engine or an electric heater.
- Air Conditioners : Cool the cabin by compressing and expanding refrigerant.
- Ventilation : Ensures fresh air circulation within the vehicle.
- Air Filters : Remove dust and pollutants from incoming air.
Effective HVAC control is crucial for passenger comfort, and integrating this control with an Android device allows for a more intuitive user experience.
Detailed overview of the DDS protocol
Introduction to DDS
Data Distribution Service (DDS) is a middleware protocol and API standard for data-centric connectivity . It enables scalable, real-time, dependable, high-performance, and interoperable data exchanges between publishers and subscribers. DDS is especially popular in mission-critical applications like aerospace, defense, automotive, telecommunications, and healthcare due to its robustness and flexibility.
Key functionalities of DDS
- Data-Centric Publish-Subscribe (DCPS) : DDS operates on the publish-subscribe model where data producers (publishers) and data consumers (subscribers) communicate through topics. This model decouples the communication participants in both time and space, enhancing scalability and flexibility.
- Quality of Service (QoS) : DDS provides extensive QoS policies that can be configured to meet specific application requirements. These policies control various aspects of data delivery, such as reliability, durability, latency, and resource usage.
- Automatic Discovery : DDS includes built-in mechanisms for the automatic discovery of participants, topics, and data readers/writers. This feature simplifies the setup and maintenance of communication systems, as entities can join and leave the network dynamically without manual configuration.
- Real-Time Capabilities : DDS is designed for real-time applications, offering low latency and high throughput. It supports real-time data distribution, ensuring timely delivery and processing of information.
- Interoperability and Portability : DDS is standardized by the Object Management Group (OMG), which ensures interoperability between different DDS implementations and portability across various platforms.
Structure of DDS
Domain Participant : The central entity in a DDS system is the domain participant. It acts as the container for publishers, subscribers, topics, and QoS settings. A participant joins a domain identified by a unique ID, allowing different sets of participants to communicate within isolated domains.
Publisher and Subscriber :
- Publisher : A publisher manages data writers and handles the dissemination of data to subscribers.
- Subscriber : A subscriber manages data readers and processes incoming data from publishers.
Topic : Topics are named entities representing a data type and the QoS settings. They are the points of connection between publishers and subscribers. Topics define the structure and semantics of the data exchanged.
Data Writer and Data Reader :
- Data Writer : Data writers are responsible for publishing data on a topic.
- Data Reader : Data readers subscribe to a topic and receive data from corresponding data writers.
Quality of Service (QoS) Policies : QoS policies define the contract between data writers and data readers. They include settings such as:
- Reliability : Controls whether data is delivered reliably (with acknowledgment) or best-effort.
- Durability : Determines how long data should be retained by the middleware.
- Deadline : Specifies the maximum time allowed between consecutive data samples.
- Latency Budget : Sets the acceptable delay from data writing to reading.
Ensuring communication correctness
DDS ensures correct communication through various mechanisms:
- Reliable Communication : Using QoS policies, DDS can guarantee reliable data delivery. For example, the Reliability QoS can be set to "RELIABLE," ensuring that the subscriber acknowledges all data samples.
- Data Consistency : DDS maintains data consistency using mechanisms like coherent access, which ensures that a group of data changes is applied atomically.
- Deadline and Liveliness : These QoS policies ensure that data is delivered within specified time constraints. The Deadline policy ensures that data is updated at expected intervals, while the Liveliness policy verifies that participants are still active.
- Durability : DDS supports various durability levels to ensure data persistence. This ensures that late-joining subscribers can still access historical data.
- Ownership Strength : In scenarios where multiple publishers can publish on the same topic, the Ownership Strength QoS policy determines which publisher's data should be used when conflicts occur.
Building the CycloneDDS Library for Android
To integrate HVAC control in Android with the DDS protocol, we will use the CycloneDDS library. CycloneDDS is an open-source implementation of the DDS protocol, providing robust and efficient data distribution. The source code for CycloneDDS is available at Eclipse CycloneDDS GitHub , and the instructions for building it for Android are detailed at CycloneDDS Android Port .
Prerequisites
Before starting the build process, ensure you have the following prerequisites installed:
- Android NDK: Download and install the latest version from the Android NDK website .
- CMake: Download and install CMake from the CMake website.
- A suitable build environment (e.g., Linux or macOS).
Step-by-step build instructions
1. Clone the CycloneDDS Repository : First, clone the CycloneDDS repository to your local machine:
git clone https://github.com/eclipse-cyclonedds/cyclonedds.git
cd cyclonedds
2. Set Up the Android NDK : Ensure that the Android NDK is properly installed and its path is added to your environment variables.
export ANDROID_NDK_HOME=/path/to/your/android-ndk
export PATH=$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin:$PATH
3. Create a Build Directory : Create a separate build directory to keep the build files organized:
mkdir build-android
cd build-android
4. Configure the Build with CMake : Use CMake to configure the build for the Android platform. Adjust the ANDROID_ABI parameter based on your target architecture (e.g., armeabi-v7a , arm64-v8a , x86 , x86_64 ):
cmake -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_HOME/build/cmake/android.toolchain.cmake \
-DANDROID_ABI=arm64-v8a \
-DANDROID_PLATFORM=android-21 \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=OFF \
-DCYCLONEDDS_ENABLE_SSL=NO \
..
5. Build the CycloneDDS Library : Run the build process using CMake. This step compiles the CycloneDDS library for the specified Android architecture:
cmake --build .
Integrating CycloneDDS with VHAL
After building the CycloneDDS library, the next step is to integrate it with the VHAL (Vehicle Hardware Abstraction Layer) application.
1. Copy the Built Library : Copy the libddsc.a file from the build output to the VHAL application directory:
cp path/to/build-android/libddsc.a path/to/your/android/source/hardware/interfaces/automotive/vehicle/2.0/default/
2. Modify the Android.bp File : Add the CycloneDDS library to the Android.bp file located in the hardware/interfaces/automotive/vehicle/2.0/default/ directory:
cc_prebuilt_library_static {
name: "libdds",
vendor: true,
srcs: ["libddsc.a"],
strip: {
none: true,
},
}
3. Update the VHAL Service Target : In the same Android.bp file, add the libdds library to the static_libs section of the android.hardware.automotive.vehicle@2.0-default-service target:
cc_binary {
name: "android.hardware.automotive.vehicle@2.0-default-service",
srcs: ["VehicleService.cpp"],
shared_libs: [
"liblog",
"libutils",
"libbinder",
"libhidlbase",
"libhidltransport",
"android.hardware.automotive.vehicle@2.0-manager-lib",
],
static_libs: [
"android.hardware.automotive.vehicle@2.0-manager-lib",
"android.hardware.automotive.vehicle@2.0-libproto-native",
"android.hardware.automotive.vehicle@2.0-default-impl-lib",
"libdds",
],
vendor: true,
}
Defining the Data Model with IDL
To enable DDS-based communication for HVAC control in our Android application, we need to define a data model using the Interface Definition Language (IDL). In this example, we will create a simple IDL file named hvacDriver.idl that describes the structures used for HVAC control, such as fan speed, temperature, and air distribution.
hvacDriver.idl
Create a file named hvacDriver.idl with the following content:
module HVACDriver
{
struct FanSpeed
{
octet value;
};
struct Temperature
{
float value;
};
struct AirDistribution
{
octet value;
};
};
Generating C Code from IDL
Once the IDL file is created, we can use the idlc (IDL compiler) tool provided by CycloneDDS to generate the corresponding C code. The generated files will include hvacDriver.h and hvacDriver.c , which contain the data structures and serialization/deserialization code needed for DDS communication.
Run the following command to generate the C code:
idlc hvacDriver.idl
This command will produce two files:
- hvacDriver.h
- hvacDriver.c
Integrating the generated code with VHAL
After generating the C code, the next step is to integrate these files into the VHAL (Vehicle Hardware Abstraction Layer) application.
Copy the Generated Files : Copy the generated hvacDriver.h and hvacDriver.c files to the VHAL application directory:
cp hvacDriver.h path/to/your/android/source/hardware/interfaces/automotive/vehicle/2.0/default/
cp hvacDriver.c path/to/your/android/source/hardware/interfaces/automotive/vehicle/2.0/default/
Include the Generated Header : In the VHAL source files where you intend to use the HVAC data structures, include the generated header file. For instance, in VehicleService.cpp , you might add:
#include "hvacDriver.h"
Modify the Android.bp File : Update the Android.bp file in the hardware/interfaces/automotive/vehicle/2.0/default/ directory to compile the generated C files and link them with your application:
cc_library_static {
name: "hvacDriver",
vendor: true,
srcs: ["hvacDriver.c"],
}
cc_binary {
name: "android.hardware.automotive.vehicle@2.0-default-service",
srcs: ["VehicleService.cpp"],
shared_libs: [
"liblog",
"libutils",
"libbinder",
"libhidlbase",
"libhidltransport",
"android.hardware.automotive.vehicle@2.0-manager-lib",
],
static_libs: [
"android.hardware.automotive.vehicle@2.0-manager-lib",
"android.hardware.automotive.vehicle@2.0-libproto-native",
"android.hardware.automotive.vehicle@2.0-default-impl-lib",
"libdds",
"hvacDriver",
],
vendor: true,
}
Implementing DDS in the VHAL Application
To enable DDS-based communication within the VHAL (Vehicle Hardware Abstraction Layer) application, we need to implement a service that handles DDS operations. This service will be encapsulated in the HVACDDSService class, which will include methods for initialization and running the service.
Step-by-step implementation
1. Create the HVACDDSService Class : First, we will define the HVACDDSService class with methods for initializing the DDS entities and running the service to handle communication.
2. Initialization : The init method will create a DDS participant, and for each structure (FanSpeed, Temperature, AirDistribution), it will create a topic, reader, and writer.
3. Running the Service : The run method will continuously read messages from the DDS readers and trigger a callback function to handle data changes.
void HVACDDSService::init()
{
/* Create a Participant. */
participant = dds_create_participant (DDS_DOMAIN_DEFAULT, NULL, NULL);
if(participant < 0)
{
LOG(ERROR) << "[DDS] " << __func__ << " dds_create_participant: " << dds_strretcode(-participant);
}
/* Create a Topic. */
qos = dds_create_qos();
dds_qset_reliability(qos, DDS_RELIABILITY_RELIABLE, DDS_SECS(10));
dds_qset_durability(qos, DDS_DURABILITY_TRANSIENT_LOCAL);
topic_temperature = dds_create_topic(participant, &Driver_Temperature_desc, "HVACDriver_Temperature", qos, NULL);
if(topic_temperature < 0)
{
LOG(ERROR) << "[DDS] " << __func__ << " dds_create_topic(temperature): "<< dds_strretcode(-topic_temperature);
}
reader_temperature = dds_create_reader(participant, topic_temperature, NULL, NULL);
if(reader_temperature < 0)
{
LOG(ERROR) << "[DDS] " << __func__ << " dds_create_reader(temperature): " << dds_strretcode(-reader_temperature);
}
writer_temperature = dds_create_writer(participant, topic_temperature, NULL, NULL);
if(writer_temperature < 0)
{
LOG(ERROR) << "[DDS] " << __func__ << " dds_create_writer(temperature): " << dds_strretcode(-writer_temperature);
}
.....
}
void HVACDDSService::run()
{
samples_temperature[0] = Driver_Temperature__alloc();
samples_fanspeed[0] = Driver_FanSpeed__alloc();
samples_airdistribution[0] = Driver_AirDistribution__alloc();
while (true)
{
bool no_data = true;
rc = dds_take(reader_temperature, samples_temperature, infos, MAX_SAMPLES, MAX_SAMPLES);
if (rc < 0)
{
LOG(ERROR) << "[DDS] " << __func__ << " temperature dds_take: " << dds_strretcode(-rc);
}
/* Check if we read some data and it is valid. */
if ((rc > 0) && (infos[0].valid_data))
{
no_data = false;
Driver_Temperature *msg = (Driver_Temperature *) samples_temperature[0];
LOG(INFO) << "[DDS] " << __func__ << " === [Subscriber] Message temperature(" << (float)msg->value << ")";
if (tempChanged_)
{
std::stringstream ss;
ss << std::fixed << std::setprecision(2) << msg->value;
tempChanged_(ss.str());
}
}
......
if(no_data)
{
/* Polling sleep. */
dds_sleepfor (DDS_MSECS (20));
}
}
}
Building and deploying the application
After implementing the HVACDDSService class and integrating it into your VHAL application, the next steps involve building the application and deploying it to your Android device.
Building the application
1. Build the VHAL Application : Ensure that your Android build environment is set up correctly and that all necessary dependencies are in place. Then, navigate to the root of your Android source tree and run the build command:
source build/envsetup.sh
lunch <target>
m -j android.hardware.automotive.vehicle@2.0-service
2. Verify the Build : Check that the build completes successfully and that the binary for your VHAL service is created. The output binary should be located in the out/target/product/<device>/system/vendor/bin/ directory.
Deploying the application
1. Push the Binary to the Device : Connect your Android device to your development machine via USB, and use adb to push the built binary to the device:
adb push out/target/product/<device>/system/vendor/bin/android.hardware.automotive.vehicle@2.0-service /vendor/bin/
2. Restart device
Conclusion
In this article, we have covered the steps to integrate DDS (Data Distribution Service) communication for HVAC control in an Android Automotive environment using the CycloneDDS library. Here's a summary of the key points:
1. CycloneDDS Library Setup :
- Cloned and built CycloneDDS for Android.
- Integrated the built library into the VHAL application.
2. Data Model Definition :
- Defined a simple data model for HVAC control using IDL.
- Generated the necessary C code from the IDL definitions.
3. HVACDDSService Implementation :
- Created the HVACDDSService class to manage DDS operations .
- Implemented methods for initialization ( init ) and runtime processing ( run ).
- Set up DDS entities such as participants, topics, readers, and writers.
- Integrated DDS service into the VHAL application's main loop.
4. Building and Deploying the Application
- Built the VHAL application and deployed it to the Android device.
- Ensured correct permissions and successfully started the VHAL service.
By following these steps, you can leverage DDS for efficient, scalable, and reliable communication in automotive systems, enhancing HVAC systems' control and monitoring capabilities in Android Automotive environments. This integration showcases the potential of DDS in automotive applications, providing a robust framework for data exchange across different components and services.
Data powertrain in automotive: Complete end-to-end solution
We power your entire data journey, from signals to solutions
Check related articles
Read our blog and stay informed about the industry's latest trends and solutions.
Controlling HVAC module in cars using Android: A dive into SOME/IP integration
In modern automotive design, controlling various components of a vehicle via mobile devices has become a significant trend, enhancing user experience and convenience. One such component is the HVAC (Heating, Ventilation, and Air Conditioning) system, which plays a crucial role in ensuring passenger comfort. In this article, we'll explore how to control the HVAC module in a car using an Android device , leveraging the power of the SOME/IP protocol.
Understanding HVAC
HVAC stands for Heating, Ventilation, and Air Conditioning. In the context of automotive engineering, the HVAC system regulates the temperature, humidity, and air quality within the vehicle cabin. It includes components such as heaters, air conditioners, fans, and air filters. Controlling the HVAC system efficiently contributes to passenger comfort and safety during the journey.
Introduction to SOME/IP
In the SOME/IP paradigm, communication is structured around services, which encapsulate specific functionalities or data exchanges. There are two main roles within the service-oriented model:
Provider: The provider is responsible for offering services to other ECUs within the network. In the automotive context, a provider ECU might control physical actuators, read sensor data, or perform other tasks related to vehicle operation. For example, in our case, the provider would be an application running on a domain controller within the vehicle.
The provider offers services by exposing interfaces that define the methods or data structures available for interaction. These interfaces can include operations to control actuators (e.g., HVAC settings) or methods to read sensor data (e.g., temperature, humidity).
Consumer: The consumer, on the other hand, is an ECU that utilizes services provided by other ECUs within the network. Consumers can subscribe to specific services offered by providers to receive updates or invoke methods as needed. In the automotive context, a consumer might be responsible for interpreting sensor data, sending control commands, or performing other tasks based on received information.
Consumers subscribe to services they are interested in and receive updates whenever there is new data available. They can also invoke methods provided by the service provider to trigger actions or control functionalities. In our scenario, the consumer would be an application running on the Android VHAL (Vehicle Hardware Abstraction Layer), responsible for interacting with the vehicle's network and controlling HVAC settings.
SOME/IP communication flow
The communication flow in SOME/IP follows a publish-subscribe pattern, where providers publish data or services, and consumers subscribe to them to receive updates or invoke methods. This asynchronous communication model allows for efficient and flexible interaction between ECUs within the network.
Source: https://github.com/COVESA/vsomeip/wiki/vsomeip-in-10-minutes
In our case, the application running on the domain controller (provider) would publish sensor data such as temperature, humidity, and HVAC status. Subscribed consumers, such as the VHAL application on Android, would receive these updates and could send control commands back to the domain controller to adjust HVAC settings based on user input.
Leveraging VHAL in Android for vehicle networking
To communicate with the vehicle's network, Android provides the Vehicle Hardware Abstraction Layer (VHAL). VHAL acts as a bridge between the Android operating system and the vehicle's onboard systems , enabling seamless integration of Android devices with the car's functionalities. VHAL abstracts the complexities of vehicle networking protocols, allowing developers to focus on implementing features such as HVAC control without worrying about low-level communication details.
Source: https://source.android.com/docs/automotive/vhal/previous/properties
Implementing SOMEIP Consumer in VHAL
To integrate a SOMEIP consumer into VHAL on Android 14, we will use the vsomeip library. Below are the steps required to implement this solution:
Cloning the vsomeip Repository
Go to the main directory of your Android project and create a new directory named external/sdv:
mkdir -p external/sdv
cd external/sdv
git clone https://android.googlesource.com/platform/external/sdv/vsomeip
Implementing SOMEIP Consumer in VHAL
In the hardware/interfaces/automotive/vehicle/2.0/default directory, you can find the VHAL application code. In the VehicleService.cpp file, you will find the default VHAL implementation.
int main(int /* argc */, char* /* argv */ []) {
auto store = std::make_unique<VehiclePropertyStore>();
auto connector = std::make_unique<DefaultVehicleConnector>();
auto hal = std::make_unique<DefaultVehicleHal>(store.get(), connector.get());
auto service = android::sp<VehicleHalManager>::make(hal.get());
connector->setValuePool(hal->getValuePool());
android::hardware::configureRpcThreadpool(4, true /* callerWillJoin */);
ALOGI("Registering as service...");
android::status_t status = service->registerAsService();
if (status != android::OK) {
ALOGE("Unable to register vehicle service (%d)", status);
return 1;
}
ALOGI("Ready");
android::hardware::joinRpcThreadpool();
return 0;
}
The default implementation of VHAL is provided in DefaultVehicleHal which we need to replace in VehicleService.cpp .
From:
auto hal = std::make_unique<DefaultVehicleHal>(store.get(), connector.get());
To:
auto hal = std::make_unique<VendorVehicleHal>(store.get(), connector.get());
For our implementation, we will create a class called VendorVehicleHal and inherit from the DefaultVehicleHal class. We will override the set and get functions.
class VendorVehicleHal : public DefaultVehicleHal {
public:
VendorVehicleHal(VehiclePropertyStore* propStore, VehicleHalClient* client);
VehiclePropValuePtr get(const VehiclePropValue& requestedPropValue,
StatusCode* outStatus) override;
StatusCode set(const VehiclePropValue& propValue) override;
};
The get function is invoked when the Android system requests information from VHAL, and set when it wants to set it. Data is transmitted in a VehiclePropValue object defined in hardware/interfaces/automotive/vehicle/2.0/types.hal.
It contains a variable, prop, which is the identifier of our property. The list of all properties can be found in the types.hal file.
We will filter out only the values of interest and redirect the rest to the default implementation.
StatusCode VendorVehicleHal::set(const VehiclePropValue& propValue) {
ALOGD("VendorVehicleHal::set propId: 0x%x areaID: 0x%x", propValue.prop, propValue.areaId);
switch(propValue.prop)
{
case (int)VehicleProperty::HVAC_FAN_SPEED :
break;
case (int)VehicleProperty::HVAC_FAN_DIRECTION :
break;
case (int)VehicleProperty::HVAC_TEMPERATURE_CURRENT :
break;
case (int)VehicleProperty::HVAC_TEMPERATURE_SET:
break;
case (int)VehicleProperty::HVAC_DEFROSTER :
break;
case (int)VehicleProperty::HVAC_AC_ON :
break;
case (int)VehicleProperty::HVAC_MAX_AC_ON :
break;
case (int)VehicleProperty::HVAC_MAX_DEFROST_ON :
break;
case (int)VehicleProperty::EVS_SERVICE_REQUEST :
break;
case (int)VehicleProperty::HVAC_TEMPERATURE_DISPLAY_UNITS :
break;
}
return DefaultVehicleHal::set(propValue);
}
Now we need to create a SOME/IP service consumer. If you're not familiar with the SOME/IP protocol or the vsomeip library, I recommend reading the guide "vsomeip in 10 minutes" .
It provides a step-by-step description of how to create a provider and consumer for SOME/IP.
In our example, we'll create a class called ZoneHVACService and define SOME/IP service, instance, method, and event IDs:
#define ZONE_HVAC_SERVICE_ID 0x4002
#define ZONE_HVAC_INSTANCE_ID 0x0001
#define ZONE_HVAC_SET_TEMPERATURE_ID 0x1011
#define ZONE_HVAC_SET_FANSPEED_ID 0x1012
#define ZONE_HVAC_SET_AIR_DISTRIBUTION_ID 0x1013
#define ZONE_HVAC_TEMPERATURE_EVENT_ID 0x2011
#define ZONE_HVAC_FANSPEED_EVENT_ID 0x2012
#define ZONE_HVAC_AIR_DISTRIBUTION_EVENT_ID 0x2013
#define ZONE_HVAC_EVENT_GROUP_ID 0x3011
class ZoneHVACService {
public:
ZoneHVACService(bool _use_tcp) :
app_(vsomeip::runtime::get()->create_application(vsomeipAppName)), use_tcp_(
_use_tcp) {
}
bool init() {
if (!app_->init()) {
LOG(ERROR) << "[SOMEIP] " << __func__ << "Couldn't initialize application";
return false;
}
app_->register_state_handler(
std::bind(&ZoneHVACService::on_state, this,
std::placeholders::_1));
app_->register_message_handler(
ZONE_HVAC_SERVICE_ID, ZONE_HVAC_INSTANCE_ID, vsomeip::ANY_METHOD,
std::bind(&ZoneHVACService::on_message, this,
std::placeholders::_1));
app_->register_availability_handler(ZONE_HVAC_SERVICE_ID, ZONE_HVAC_INSTANCE_ID,
std::bind(&ZoneHVACService::on_availability,
this,
std::placeholders::_1, std::placeholders::_2, std::placeholders::_3));
std::set<vsomeip::eventgroup_t> its_groups;
its_groups.insert(ZONE_HVAC_EVENT_GROUP_ID);
app_->request_event(
ZONE_HVAC_SERVICE_ID,
ZONE_HVAC_INSTANCE_ID,
ZONE_HVAC_TEMPERATURE_EVENT_ID,
its_groups,
vsomeip::event_type_e::ET_FIELD);
app_->request_event(
ZONE_HVAC_SERVICE_ID,
ZONE_HVAC_INSTANCE_ID,
ZONE_HVAC_FANSPEED_EVENT_ID,
its_groups,
vsomeip::event_type_e::ET_FIELD);
app_->request_event(
ZONE_HVAC_SERVICE_ID,
ZONE_HVAC_INSTANCE_ID,
ZONE_HVAC_AIR_DISTRIBUTION_EVENT_ID,
its_groups,
vsomeip::event_type_e::ET_FIELD);
app_->subscribe(ZONE_HVAC_SERVICE_ID, ZONE_HVAC_INSTANCE_ID, ZONE_HVAC_EVENT_GROUP_ID);
return true;
}
void send_temp(std::string temp)
{
LOG(INFO) << "[SOMEIP] " << __func__ << " temp: " << temp;
std::shared_ptr< vsomeip::message > request;
request = vsomeip::runtime::get()->create_request();
request->set_service(ZONE_HVAC_SERVICE_ID);
request->set_instance(ZONE_HVAC_INSTANCE_ID);
request->set_method(ZONE_HVAC_SET_TEMPERATURE_ID);
std::shared_ptr< vsomeip::payload > its_payload = vsomeip::runtime::get()->create_payload();
its_payload->set_data((const vsomeip_v3::byte_t *)temp.data(), temp.size());
request->set_payload(its_payload);
app_->send(request);
}
void send_fanspeed(uint8_t speed)
{
LOG(INFO) << "[SOMEIP] " << __func__ << " speed: " << (int)speed;
std::shared_ptr< vsomeip::message > request;
request = vsomeip::runtime::get()->create_request();
request->set_service(ZONE_HVAC_SERVICE_ID);
request->set_instance(ZONE_HVAC_INSTANCE_ID);
request->set_method(ZONE_HVAC_SET_FANSPEED_ID);
std::shared_ptr< vsomeip::payload > its_payload = vsomeip::runtime::get()->create_payload();
its_payload->set_data(&speed, 1U);
request->set_payload(its_payload);
app_->send(request);
}
void start() {
app_->start();
}
void on_state(vsomeip::state_type_e _state) {
if (_state == vsomeip::state_type_e::ST_REGISTERED) {
app_->request_service(ZONE_HVAC_SERVICE_ID, ZONE_HVAC_INSTANCE_ID);
}
}
void on_availability(vsomeip::service_t _service, vsomeip::instance_t _instance, bool _is_available) {
LOG(INFO) << "[SOMEIP] " << __func__ << "Service ["
<< std::setw(4) << std::setfill('0') << std::hex << _service << "." << _instance
<< "] is "
<< (_is_available ? "available." : "NOT available.");
}
void on_temperature_message(const std::shared_ptr<vsomeip::message> & message)
{
auto payload = message->get_payload();
temperature_.resize(payload->get_length());
temperature_.assign((char*)payload->get_data(), payload->get_length());
LOG(INFO) << "[SOMEIP] " << __func__ << " temp: " << temperature_;
if(tempChanged_)
{
tempChanged_(temperature_);
}
}
void on_fanspeed_message(const std::shared_ptr<vsomeip::message> & message)
{
auto payload = message->get_payload();
fan_speed_ = *payload->get_data();
LOG(INFO) << "[SOMEIP] " << __func__ << " speed: " << (int)fan_speed_;
if(fanspeedChanged_)
{
fanspeedChanged_(fan_speed_);
}
}
void on_message(const std::shared_ptr<vsomeip::message> & message) {
if(message->get_method() == ZONE_HVAC_TEMPERATURE_EVENT_ID)
{
LOG(INFO) << "[SOMEIP] " << __func__ << "TEMPERATURE_EVENT_ID received";
on_temperature_message(message);
}
else if(message->get_method() == ZONE_HVAC_FANSPEED_EVENT_ID)
{
LOG(INFO) << "[SOMEIP] " << __func__ << "ZONE_HVAC_FANSPEED_EVENT_ID received";
on_fanspeed_message(message);
}
}
std::function<void(std::string temp)> tempChanged_;
std::function<void(uint8_t)> fanspeedChanged_;
private:
std::shared_ptr< vsomeip::application > app_;
bool use_tcp_;
std::string temperature_;
uint8_t fan_speed_;
uint8_t air_distribution_t;
};
In our example, we will connect ZoneHVACService and VendorVehicleHal using callbacks.
hal->fandirectionChanged_ = [&](uint8_t direction) {
ALOGI("HAL fandirectionChanged_ callback direction: %u", direction);
hvacService->send_fandirection(direction);
};hal->fanspeedChanged_ = [&](uint8_t speed) {
ALOGI("HAL fanspeedChanged_ callback speed: %u", speed);
hvacService->send_fanspeed(speed);
};
The last thing left for us to do is to create a configuration for the vsomeip library. It's best to utilize a sample file from the library: https://github.com/COVESA/vsomeip/blob/master/config/vsomeip-local.json
In this file, you'll need to change the address:
"unicast" : "10.0.2.15",
to the address of our Android device.
Additionally, you need to set:
"routing" : "service-sample",
to the name of our application.
The vsomeip stack reads the application address and the path to the configuration file from environment variables. The easiest way to do this in Android is to set it up before creating the ZoneHVACService object.
setenv("VSOMEIP_CONFIGURATION","/vendor/etc/vsomeip-local-hvac.json",1);
setenv("VSOMEIP_APPLICATION_NAME," "hvac-service",1);
That’s it. Now, we shoudl replace vendor/bin/hw/android.hardware.automotive.vehicle@2.0-default-service with our new build and reboot Android.
If everything was configured correctly, we should see such logs, and the provider should get our requests.
04-25 06:52:12.989 3981 3981 I automotive.vehicle@2.0-default-service: Starting automotive.vehicle@2.0-default-service ...
04-25 06:52:13.005 3981 3981 I automotive.vehicle@2.0-default-service: Registering as service...
04-25 06:52:13.077 3981 3981 I automotive.vehicle@2.0-default-service: Ready
04-25 06:52:13.081 3981 4011 I automotive.vehicle@2.0-default-service: Starting UDP receiver
04-25 06:52:13.081 3981 4011 I automotive.vehicle@2.0-default-service: Socket created
04-25 06:52:13.082 3981 4010 I automotive.vehicle@2.0-default-service: HTTPServer starting
04-25 06:52:13.082 3981 4010 I automotive.vehicle@2.0-default-service: HTTPServer listen
04-25 06:52:13.091 3981 4012 I automotive.vehicle@2.0-default-service: Initializing SomeIP service ...
04-25 06:52:13.091 3981 4012 I automotive.vehicle@2.0-default-service: [SOMEIP] initInitialize app
04-25 06:52:13.209 3981 4012 I automotive.vehicle@2.0-default-service: [SOMEIP] initApp initialized
04-25 06:52:13.209 3981 4012 I automotive.vehicle@2.0-default-service: [SOMEIP] initClient settings [protocol=UDP]
04-25 06:52:13.210 3981 4012 I automotive.vehicle@2.0-default-service: [SOMEIP] Initialized SomeIP service result:1
04-25 06:52:13.214 3981 4028 I automotive.vehicle@2.0-default-service: [SOMEIP] on_availabilityService [4002.1] is NOT available.
04-25 06:54:35.654 3981 4028 I automotive.vehicle@2.0-default-service: [SOMEIP] on_availabilityService [4002.1] is available.
04-25 06:54:35.774 3981 4028 I automotive.vehicle@2.0-default-service: [SOMEIP] on_message Message received: [4002.0001.2012] to Client/Session [0000/0002]
04-25 06:54:35.774 3981 4028 I automotive.vehicle@2.0-default-service: [SOMEIP] on_messageZONE_HVAC_FANSPEED_EVENT_ID received
04-25 06:54:35.774 3981 4028 I automotive.vehicle@2.0-default-service: [SOMEIP] on_fanspeed_message speed: 1
04-25 06:54:35.775 3981 4028 I automotive.vehicle@2.0-default-service: SOMEIP fanspeedChanged_ speed: 1
04-25 06:54:36.602 3981 4028 I automotive.vehicle@2.0-default-service: [SOMEIP] on_message Message received: [4002.0001.2012] to Client/Session [0000/0003]
04-25 06:54:36.602 3981 4028 I automotive.vehicle@2.0-default-service: [SOMEIP] on_messageZONE_HVAC_FANSPEED_EVENT_ID received
04-25 06:54:36.603 3981 4028 I automotive.vehicle@2.0-default-service: [SOMEIP] on_fanspeed_message speed: 2
04-25 06:54:36.603 3981 4028 I automotive.vehicle@2.0-default-service: SOMEIP fanspeedChanged_ speed: 2
04-25 06:54:37.605 3981 4028 I automotive.vehicle@2.0-default-service: [SOMEIP] on_message Message received: [4002.0001.2012] to Client/Session [0000/0004]
04-25 06:54:37.606 3981 4028 I automotive.vehicle@2.0-default-service: [SOMEIP] on_messageZONE_HVAC_FANSPEED_EVENT_ID received
04-25 06:54:37.606 3981 4028 I automotive.vehicle@2.0-default-service: [SOMEIP] on_fanspeed_message speed: 3
04-25 06:54:37.606 3981 4028 I automotive.vehicle@2.0-default-service: SOMEIP fanspeedChanged_ speed: 3
Summary
In conclusion, the integration of Android devices with Vehicle Hardware Abstraction Layer (VHAL) for controlling HVAC systems opens up a new realm of possibilities for automotive technology. By leveraging the power of SOME/IP communication protocol and the vsomeip library, developers can create robust solutions for managing vehicle HVAC functionalities.
By following the steps outlined in this article, developers can create custom VHAL implementations tailored to their specific needs. From defining service interfaces to handling communication callbacks, every aspect of the integration process has been carefully explained to facilitate smooth development.
As automotive technology continues to evolve, the convergence of Android devices and vehicle systems represents a significant milestone in the journey towards smarter, more connected vehicles. The integration of HVAC control functionalities through VHAL and SOME/IP not only demonstrates the potential of modern automotive technology but also paves the way for future innovations in the field.
Android AAOS 14 - EVS network camera
The automotive industry has been rapidly evolving with technological advancements that enhance the driving experience and safety. Among these innovations, the Android Automotive Operating System (AAOS) has stood out, offering a versatile and customizable platform for car manufacturers.
The Exterior View System (EVS) is a comprehensive camera-based system designed to provide drivers with real-time visual monitoring of their vehicle's surroundings. It typically includes multiple cameras positioned around the vehicle to eliminate blind spots and enhance situational awareness, significantly aiding in maneuvers like parking and lane changes. By integrating with advanced driver assistance systems, EVS contributes to increased safety and convenience for drivers.
For more detailed information about EVS and its configuration, we highly recommend reading our article "Android AAOS 14 - Surround View Parking Camera: How to Configure and Launch EVS (Exterior View System)." This foundational article provides essential insights and instructions that we will build upon in this guide.
The latest Android Automotive Operating System , AAOS 14, presents new possibilities, but it does not natively support Ethernet cameras. In this article, we describe our implementation of an Ethernet camera integration with the Exterior View System (EVS) on Android.
Our approach involves connecting a USB camera to a Windows laptop and streaming the video using the Real-time Transport Protocol (RTP). By employing the powerful FFmpeg software, the video stream will be broadcast and described in an SDP (Session Description Protocol) file, accessible via an HTTP server. On the Android side, we'll utilize the FFmpeg library to receive and decode the video stream, effectively bringing the camera feed into the AAOS 14 environment.
This article provides a step-by-step guide on how we achieved this integration of the EVS network camera, offering insights and practical instructions for those looking to implement a similar solution. The following diagram provides an overview of the entire process:
Building FFmpeg Library for Android
To enable RTP camera streaming on Android, the first step is to build the FFmpeg library for the platform. This section describes the process in detail, using the ffmpeg-android-maker project. Follow these steps to successfully build and integrate the FFmpeg library with the Android EVS (Exterior View System) Driver.
Step 1: Install Android SDK
First, install the Android SDK. For Ubuntu/Debian systems, you can use the following commands:
sudo apt update && sudo apt install android-sdk
The SDK should be installed in /usr/lib/android-sdk .
Step 2: Install NDK
Download the Android NDK (Native Development Kit) from the official website:
https://developer.android.com/ndk/downloads
After downloading, extract the NDK to your desired location.
Step 3: Build FFmpeg
Clone the ffmpeg-android-maker repository and navigate to its directory:
git clone https://github.com/Javernaut/ffmpeg-android-maker.gitcd ffmpeg-android-maker
Set the environment variables to point to the SDK and NDK:
export ANDROID_SDK_HOME=/usr/lib/android-sdk
export ANDROID_NDK_HOME=/path/to/ndk/
Run the build script:
./ffmpeg-android-maker.sh
This script will download FFmpeg source code and dependencies, and compile FFmpeg for various Android architectures.
Step 4: Copy Library Files to EVS Driver
After the build process is complete, copy the .so library files from build/ffmpeg/ to the EVS Driver directory in your Android project:
cp build/ffmpeg/*.so /path/to/android/project/packages/services/Car/cpp/evs/sampleDriver/aidl/
Step 5: Add Libraries to EVS Driver Build Files
Edit the Android.bp file in the aidl directory to include the prebuilt FFmpeg libraries:
cc_prebuilt_library_shared {
name: "rtp-libavcodec",
vendor: true,
srcs: ["libavcodec.so"],
strip: {
none: true,
},
check_elf_files: false,
}
cc_prebuilt_library {
name: "rtp-libavformat",
vendor: true,
srcs: ["libavformat.so"],
strip: {
none: true,
},
check_elf_files: false,
}
cc_prebuilt_library {
name: "rtp-libavutil",
vendor: true,
srcs: ["libavutil.so"],
strip: {
none: true,
},
check_elf_files: false,
}
cc_prebuilt_library_shared {
name: "rtp-libswscale",
vendor: true,
srcs: ["libswscale.so"],
strip: {
none: true,
},
check_elf_files: false,
}
Add prebuilt libraries to EVS Driver app:
cc_binary {
name: "android.hardware.automotive.evs-default",
defaults: ["android.hardware.graphics.common-ndk_static"],
vendor: true,
relative_install_path: "hw",
srcs: [
":libgui_frame_event_aidl",
"src/*.cpp"
],
shared_libs: [
"rtp-libavcodec",
"rtp-libavformat",
"rtp-libavutil",
"rtp-libswscale",
"android.hardware.graphics.bufferqueue@1.0",
"android.hardware.graphics.bufferqueue@2.0",
android.hidl.token@1.0-utils,
....]
}
By following these steps, you will have successfully built the FFmpeg library for Android and integrated it into the EVS Driver.
EVS Driver RTP Camera Implementation
In this chapter, we will demonstrate how to quickly implement RTP support for the EVS (Exterior View System) driver in Android AAOS 14. This implementation is for demonstration purposes only. For production use, the implementation should be optimized, adapted to specific requirements, and all possible configurations and edge cases should be thoroughly tested. Here, we will focus solely on displaying the video stream from RTP.
The main files responsible for capturing and decoding video from USB cameras are implemented in the EvsV4lCamera and VideoCapture classes. To handle RTP, we will copy these classes and rename them to EvsRTPCamera and RTPCapture . RTP handling will be implemented in RTPCapture . We need to implement four main functions:
bool open(const char* deviceName, const int32_t width = 0, const int32_t height = 0);
void close();
bool startStream(std::function<void(RTPCapture*, imageBuffer*, void*)> callback = nullptr);
void stopStream();
We will use the official example from the FFmpeg library, https://github.com/FFmpeg/FFmpeg/blob/master/doc/examples/demux_decode.c, which decodes the specified video stream into RGBA buffers. After adapting the example, the RTPCapture.cpp file will look like this:
#include "RTPCapture.h"
#include <android-base/logging.h>
#include <errno.h>
#include <error.h>
#include <fcntl.h>
#include <memory.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <unistd.h>
#include <cassert>
#include <iomanip>
#include <stdio.h>
#include <stdlib.h>
#include <iostream>
#include <fstream>
#include <sstream>
static AVFormatContext *fmt_ctx = NULL;
static AVCodecContext *video_dec_ctx = NULL, *audio_dec_ctx;
static int width, height;
static enum AVPixelFormat pix_fmt;
static enum AVPixelFormat out_pix_fmt = AV_PIX_FMT_RGBA;
static AVStream *video_stream = NULL, *audio_stream = NULL;
static struct SwsContext *resize;
static const char *src_filename = NULL;
static uint8_t *video_dst_data[4] = {NULL};
static int video_dst_linesize[4];
static int video_dst_bufsize;
static int video_stream_idx = -1, audio_stream_idx = -1;
static AVFrame *frame = NULL;
static AVFrame *frame2 = NULL;
static AVPacket *pkt = NULL;
static int video_frame_count = 0;
int RTPCapture::output_video_frame(AVFrame *frame)
{
LOG(INFO) << "Video_frame: " << video_frame_count++
<< " ,scale height: " << sws_scale(resize, frame->data, frame->linesize, 0, height, video_dst_data, video_dst_linesize);
if (mCallback) {
imageBuffer buf;
buf.index = video_frame_count;
buf.length = video_dst_bufsize;
mCallback(this, &buf, video_dst_data[0]);
}
return 0;
}
int RTPCapture::decode_packet(AVCodecContext *dec, const AVPacket *pkt)
{
int ret = 0;
ret = avcodec_send_packet(dec, pkt);
if (ret < 0) {
return ret;
}
// get all the available frames from the decoder
while (ret >= 0) {
ret = avcodec_receive_frame(dec, frame);
if (ret < 0) {
if (ret == AVERROR_EOF || ret == AVERROR(EAGAIN))
{
return 0;
}
return ret;
}
// write the frame data to output file
if (dec->codec->type == AVMEDIA_TYPE_VIDEO) {
ret = output_video_frame(frame);
}
av_frame_unref(frame);
if (ret < 0)
return ret;
}
return 0;
}
int RTPCapture::open_codec_context(int *stream_idx,
AVCodecContext **dec_ctx, AVFormatContext *fmt_ctx, enum AVMediaType type)
{
int ret, stream_index;
AVStream *st;
const AVCodec *dec = NULL;
ret = av_find_best_stream(fmt_ctx, type, -1, -1, NULL, 0);
if (ret < 0) {
fprintf(stderr, "Could not find %s stream in input file '%s'\n",
av_get_media_type_string(type), src_filename);
return ret;
} else {
stream_index = ret;
st = fmt_ctx->streams[stream_index];
/* find decoder for the stream */
dec = avcodec_find_decoder(st->codecpar->codec_id);
if (!dec) {
fprintf(stderr, "Failed to find %s codec\n",
av_get_media_type_string(type));
return AVERROR(EINVAL);
}
/* Allocate a codec context for the decoder */
*dec_ctx = avcodec_alloc_context3(dec);
if (!*dec_ctx) {
fprintf(stderr, "Failed to allocate the %s codec context\n",
av_get_media_type_string(type));
return AVERROR(ENOMEM);
}
/* Copy codec parameters from input stream to output codec context */
if ((ret = avcodec_parameters_to_context(*dec_ctx, st->codecpar)) < 0) {
fprintf(stderr, "Failed to copy %s codec parameters to decoder context\n",
av_get_media_type_string(type));
return ret;
}
av_opt_set((*dec_ctx)->priv_data, "preset", "ultrafast", 0);
av_opt_set((*dec_ctx)->priv_data, "tune", "zerolatency", 0);
/* Init the decoders */
if ((ret = avcodec_open2(*dec_ctx, dec, NULL)) < 0) {
fprintf(stderr, "Failed to open %s codec\n",
av_get_media_type_string(type));
return ret;
}
*stream_idx = stream_index;
}
return 0;
}
bool RTPCapture::open(const char* /*deviceName*/, const int32_t /*width*/, const int32_t /*height*/) {
LOG(INFO) << "RTPCapture::open";
int ret = 0;
avformat_network_init();
mFormat = V4L2_PIX_FMT_YUV420;
mWidth = 1920;
mHeight = 1080;
mStride = 0;
/* open input file, and allocate format context */
if (avformat_open_input(&fmt_ctx, "http://192.168.1.59/stream.sdp", NULL, NULL) < 0) {
LOG(ERROR) << "Could not open network stream";
return false;
}
LOG(INFO) << "Input opened";
isOpened = true;
/* retrieve stream information */
if (avformat_find_stream_info(fmt_ctx, NULL) < 0) {
LOG(ERROR) << "Could not find stream information";
return false;
}
LOG(INFO) << "Stream info found";
if (open_codec_context(&video_stream_idx, &video_dec_ctx, fmt_ctx, AVMEDIA_TYPE_VIDEO) >= 0) {
video_stream = fmt_ctx->streams[video_stream_idx];
/* allocate image where the decoded image will be put */
width = video_dec_ctx->width;
height = video_dec_ctx->height;
pix_fmt = video_dec_ctx->sw_pix_fmt;
resize = sws_getContext(width, height, AV_PIX_FMT_YUVJ422P,
width, height, out_pix_fmt, SWS_BICUBIC, NULL, NULL, NULL);
LOG(ERROR) << "RTPCapture::open pix_fmt: " << video_dec_ctx->pix_fmt
<< ", sw_pix_fmt: " << video_dec_ctx->sw_pix_fmt
<< ", my_fmt: " << pix_fmt;
ret = av_image_alloc(video_dst_data, video_dst_linesize,
width, height, out_pix_fmt, 1);
if (ret < 0) {
LOG(ERROR) << "Could not allocate raw video buffer";
return false;
}
video_dst_bufsize = ret;
}
av_dump_format(fmt_ctx, 0, src_filename, 0);
if (!audio_stream && !video_stream) {
LOG(ERROR) << "Could not find audio or video stream in the input, aborting";
ret = 1;
return false;
}
frame = av_frame_alloc();
if (!frame) {
LOG(ERROR) << "Could not allocate frame";
ret = AVERROR(ENOMEM);
return false;
}
frame2 = av_frame_alloc();
pkt = av_packet_alloc();
if (!pkt) {
LOG(ERROR) << "Could not allocate packet";
ret = AVERROR(ENOMEM);
return false;
}
return true;
}
void RTPCapture::close() {
LOG(DEBUG) << __FUNCTION__;
}
bool RTPCapture::startStream(std::function<void(RTPCapture*, imageBuffer*, void*)> callback) {
LOG(INFO) << "startStream";
if(!isOpen()) {
LOG(ERROR) << "startStream failed. Stream not opened";
return false;
}
stop_thread_1 = false;
mCallback = callback;
mCaptureThread = std::thread([this]() { collectFrames(); });
return true;
}
void RTPCapture::stopStream() {
LOG(INFO) << "stopStream";
stop_thread_1 = true;
mCaptureThread.join();
mCallback = nullptr;
}
bool RTPCapture::returnFrame(int i) {
LOG(INFO) << "returnFrame" << i;
return true;
}
void RTPCapture::collectFrames() {
int ret = 0;
LOG(INFO) << "Reading frames";
/* read frames from the file */
while (av_read_frame(fmt_ctx, pkt) >= 0) {
if (stop_thread_1) {
return;
}
if (pkt->stream_index == video_stream_idx) {
ret = decode_packet(video_dec_ctx, pkt);
}
av_packet_unref(pkt);
if (ret < 0)
break;
}
}
int RTPCapture::setParameter(v4l2_control&) {
LOG(INFO) << "RTPCapture::setParameter";
return 0;
}
int RTPCapture::getParameter(v4l2_control&) {
LOG(INFO) << "RTPCapture::getParameter";
return 0;
}
std::set<uint32_t> RTPCapture::enumerateCameraControls() {
LOG(INFO) << "RTPCapture::enumerateCameraControls";
std::set<uint32_t> ctrlIDs;
return std::move(ctrlIDs);
}
void* RTPCapture::getLatestData() {
LOG(INFO) << "RTPCapture::getLatestData";
return nullptr;
}
bool RTPCapture::isFrameReady() {
LOG(INFO) << "RTPCapture::isFrameReady";
return true;
}
void RTPCapture::markFrameConsumed(int i) {
LOG(INFO) << "RTPCapture::markFrameConsumed frame: " << i;
}
bool RTPCapture::isOpen() {
LOG(INFO) << "RTPCapture::isOpen";
return isOpened;
}
Next, we need to modify EvsRTPCamera to use our RTPCapture class instead of VideoCapture . In EvsRTPCamera.h , add:
#include "RTPCapture.h"
And replace:
VideoCapture mVideo = {};
with:
RTPCapture mVideo = {};
In EvsRTPCamera.cpp , we also need to make changes. In the forwardFrame(imageBuffer* pV4lBuff, void* pData) function, replace:
mFillBufferFromVideo(bufferDesc, (uint8_t*)targetPixels, pData, mVideo.getStride());
with:
memcpy(targetPixels, pData, pV4lBuff->length);
This is because the VideoCapture class provides a buffer from the camera in various YUYV pixel formats. The mFillBufferFromVideo function is responsible for converting the pixel format to RGBA. In our case, RTPCapture already provides an RGBA buffer. This is done in the
int RTPCapture::output_video_frame(AVFrame *frame) function using sws_scale from the FFmpeg library.
Now we need to ensure that our RTP camera is recognized by the system. The EvsEnumerator class and its enumerateCameras function are responsible for detecting cameras. This function adds all video files from the /dev/ directory.
To add our RTP camera, we will append the following code at the end of the enumerateCameras function:
if (addCaptureDevice("rtp1")) {
++captureCount;
}
This will add a camera with the ID "rtp1" to the list of detected cameras, making it visible to the system.
The final step is to modify the EvsEnumerator: :openCamera function to direct the camera with the ID "rtp1" to the RTP implementation. Normally, when opening a USB camera, an instance of the EvsV4lCamera class is created:
pActiveCamera = EvsV4lCamera::Create(id.data());
In our example, we will hardcode the ID check and create the appropriate object:
if (id == "rtp1") {
pActiveCamera = EvsRTPCamera::Create(id.data());
} else {
pActiveCamera = EvsV4lCamera::Create(id.data());
}
With this implementation, our camera should start working. Now we need to build the EVS Driver application and push it to the device along with the FFmpeg libraries:
mmma packages/services/Car/cpp/evs/sampleDriver/
adb push out/target/product/rpi4/vendor/bin/hw/android.hardware.automotive.evs-default /vendor/bin/hw/
Launching the RTP Camera
To stream video from your camera, you need to install FFmpeg ( https://www.ffmpeg.org/download.html#build-windows ) and an HTTP server on the computer that will be streaming the video.
Start FFmpeg (example on Windows):
ffmpeg -f dshow -video_size 1280x720 -i video="USB Camera" -c copy -f rtp rtp://192.168.1.53:8554
where:
- -video_size is video resolution
- "USB Camera" is the name of the camera as it appears in the Device Manager
- "-c copy" means that individual frames from the camera (in JPEG format) will be copied to the RTP stream without changes. Otherwise, FFmpeg would need to decode and re-encode the image, introducing unnecessary delays.
- "rtp://192.168.1.53:8554": 192.168.1.53 is the IP address of our Android device. You should adjust this accordingly. Port 8554 can be left as the default.
After starting FFmpeg, you should see output similar to this on the console:
Here, we see the input, output, and SDP sections. In the input section, the codec is JPEG, which is what we need. The pixel format is yuvj422p, with a resolution of 1920x1080 at 30 fps. The stream parameters in the output section should match.
Next, save the SDP section to a file named stream.sdp on the HTTP server. Our EVS Driver application needs to fetch this file, which describes the stream.
In our example, the Android device should access this file at: http://192.168.1.59/stream.sdp
The exact content of the file should be:
v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
c=IN IP4 192.168.1.53
t=0 0
a=tool:libavformat 61.1.100
m=video 8554 RTP/AVP 26
Now, restart the EVS Driver application on the Android device:
killall android.hardware.automotive.evs-default
Then, configure the EVS app to use the camera "rtp1". For detailed instructions on how to configure and launch the EVS (Exterior View System), refer to the article "Android AAOS 14 - Surround View Parking Camera: How to Configure and Launch EVS (Exterior View System)".
Performance Testing
In this chapter, we will measure and compare the latency of the video stream from a camera connected via USB and RTP.
How Did We Measure Latency?
- Setup Timer: Displayed a timer on the computer screen showing time with millisecond precision.
- Camera Capture: Pointed the EVS camera at this screen so that the timer was also visible on the Android device screen.
- Snapshot Comparison: Took photos of both screens simultaneously. The time displayed on the Android device was delayed compared to the computer screen. The difference in time between the computer and the Android device represents the camera's latency.
This latency is composed of several factors:
- Camera Latency: The time the camera takes to capture the image from the sensor and encode it into the appropriate format.
- Transmission Time: The time taken to transmit the data via USB or RTP.
- Decoding and Display: The time to decode the video stream and display the image on the screen.
Latency Comparison
Below are the photos showing the latency:
USB Camera
RTP Camera
From these measurements, we found that the average latency for a camera connected via USB to the Android device is 200ms , while the latency for the camera connected via RTP is 150ms . This result is quite surprising.
The reasons behind these results are:
- The EVS implementation on Android captures video from the USB camera in YUV and similar formats, whereas FFmpeg streams RTP video in JPEG format.
- The USB camera used has a higher latency in generating YUV images compared to JPEG. Additionally, the frame rate is much lower. For a resolution of 1280x720, the YUV format only supports 10 fps, whereas JPEG supports the full 30 fps.
All camera modes can be checked using the command:
ffmpeg -f dshow -list_options true -i video="USB Camera"
Conclusion
This article has taken you through the comprehensive process of integrating an RTP camera into the Android EVS (Exterior View System) framework, highlighting the detailed steps involved in both the implementation and the performance evaluation.
We began our journey by developing new classes, EvsRTPCamera and RTPCapture , which were specifically designed to handle RTP streams using FFmpeg. This adaptation allowed us to process and stream real-time video effectively. To ensure our system recognized the RTP camera, we made critical adjustments to the EvsEnumerator class. By customizing the enumerateCameras and openCamera functions, we ensured that our RTP camera was correctly instantiated and recognized by the system.
Next, we focused on building and deploying the EVS Driver application, including the necessary FFmpeg libraries, to our target Android device. This step was crucial for validating our implementation in a real-world environment. We also conducted a detailed performance evaluation to measure and compare the latency of video feeds from USB and RTP cameras. Using a timer displayed on a computer screen, we captured the timer with the EVS camera and compared the time shown on both the computer and Android screens. This method allowed us to accurately determine the latency introduced by each camera setup.
Our performance tests revealed that the RTP camera had an average latency of 150ms, while the USB camera had a latency of 200ms. This result was unexpected but highly informative. The lower latency of the RTP camera was largely due to the use of the JPEG format, which our particular USB camera handled less efficiently due to its slower YUV processing. This significant finding underscores the RTP camera's suitability for applications requiring real-time video performance, such as automotive surround view parking systems, where quick response times are essential for safety and user experience.
Interested in our services?
Reach out for tailored solutions and expert guidance.