CarToolForge is a tool built on Android Automotive OS (AAOS) that connects a Large Language Model (LLM) to vehicle systems. It uses the Car API to access vehicle functions and makes them available to the LLM through the App Functions framework.
- Query and Control Vehicle Properties: Enables an LLM to query the status of vehicle properties and perform control actions (e.g., turning on the A/C) via a simple API.
To see this in action, check out the CarToolPlayground project, which provides an agent-based chat experience built on CarToolForge.
You can build the project using Gradle:
./gradlew clean build
App Functions is enabled by default starting with android-16.0.0_r4. However, on earlier versions of Android 16, this is considered an experimental feature and may be disabled by default in your system.
You can check whether the essential feature flags are enabled on your device or emulator using the following command:
adb shell aflags list | grep "enable_app_functions_schema_parser"
If this flag is not enabled, you need to enable them in your AOSP build. For detailed instructions, refer to the official guide on Set feature launch flag values.
Tip
You can manually backport the necessary configuration by referencing the textproto values from the android-16.0.0_r4 tag.
CarToolForge must be installed as a privileged system application to access vehicle APIs.
To facilitate out-of-the-box integration, the necessary configuration and build files are already included in the integration/cartoolforge directory. These files are bundled into a package for each release, which you can download from the Releases page. For step-by-step instructions, follow the integration_guide.md.
For development and debugging, you can sideload the application onto a rooted device or emulator.
- Disable Permission Enforcement: First, modify the
build.propfile to disable privileged permission enforcement.
adb root
adb remount
adb shell "sed -i 's/ro.control_privapp_permissions=enforce/ro.control_privapp_permissions=log/g' /vendor/build.prop"
- Install as a Privileged App: Push the APK to the privileged apps directory, such as
/system/priv-app.
adb push app-debug.apk /system/priv-app
adb reboot
- Grant Runtime Permissions: After rebooting, use the
adb install -gcommand to reinstall the app and grant all permissions defined in its manifest.
adb install -g app-debug.apk
For security, CarToolForge uses an allowlist to control the agent's access to vehicle properties. This list is defined in the vehicle_properties.yaml file. For details, see config/README.md.
-
Simplified API Surface: To make the API easier for an LLM to consume, CarToolForge intentionally omits some of the more complex features of the Car API. This may affect results in certain cases. Unsupported features include：
- Area-level access, see AreaIdConfig#getAccess.
- Dynamic min/max supported values, see CarPropertyManager#getMinMaxSupportedValue.
BYTESandMIXEDproperty data types.- VEHICLE_AREA_TYPE_VENDOR area type.
-
Model Requirements: Achieving good results with the current function complexity requires a model with excellent reasoning capabilities and a large parameter size (>20B). Further work is required to explore potential avenues for optimization.
Take a look at the CONTRIBUTING.md.