Release v6.0.1

What's Changed

• Add map to resolve the device id to client id #381

• Define Connector as singleton, only one instance can be created per tenant #380

• Feature/outbound sub auto by @ck-c8y in #375

• Allow to define additional headers for webhook by @ck-c8y in #374

• Allow to define deviceName and deviceType when a device is implicitly created (createNonExistingDevice = true) by @ck-c8y in #373

• Allow to send validation errors to devices if payload does not conform to defined rule by @ck-c8y in #371

• Allow to define mappings to use processingMode other than PERSISTENT by @ck-c8y in #370

• Automatically add devices to an existing outbound description by @svetlcky in #368

Breaking changes

• Rename field "message" to "payload" in wrapped message for mappings of type HEX, FLAT_FILE by @ck-c8y in #372

Bug Fixes

• Mapping to items in array in the payload are not working by @ck-c8y in #377
• Json name's with "." in the name are not selected correct in Mapping by @kreinhar in #367
• When CONTEXT_DATA is mapped as object it is not correctly interpreted and result in error: com.jayway.jsonpath.PathNotFoundException: Path: CONTEXT_DATA not found by @ck-c8y in #365

Full Changelog: v6.0.0...v6.0.1

Release v6.0.1-pre

What's Changed

Enhancements

• Add map to resolve the device id to client id #381

• Define Connector as singleton, only one instance can be created per tenant #380

• Feature/outbound sub auto by @ck-c8y in #375

• Allow to define additional headers for webhook by @ck-c8y in #374

• Allow to define deviceName and deviceType when a device is implicitly created (createNonExistingDevice = true) by @ck-c8y in #373

• Allow to send validation errors to devices if payload does not conform to defined rule by @ck-c8y in #371

• Allow to define mappings to use processingMode other than PERSISTENT by @ck-c8y in #370

• Automatically add devices to an existing outbound description by @svetlcky in #368

Breaking changes

• Rename field "message" to "payload" in wrapped message for mappings of type HEX, FLAT_FILE by @ck-c8y in #372

Bug Fixes

• Mapping to items in array in the payload are not working by @ck-c8y in #377
• Json name's with "." in the name are not selected correct in Mapping by @kreinhar in #367
• When CONTEXT_DATA is mapped as object it is not correctly interpreted and result in error: com.jayway.jsonpath.PathNotFoundException: Path: CONTEXT_DATA not found by @ck-c8y in #365

Full Changelog: v6.0.0...v6.0.1-pre

Release v6.0.0

What's Changed

AI Empowered

With a 6.0.0 release we made a big step into fully AI empower the dynamic mapper. With this pre-release we introduce a capability to auto-generate substitutions either for graphical mappings or JavaScript mappings with just a single click.

To use it, you need to have:

the AI Agent Manager deployed to your tenant (currently in private preview - reach out to your c8y contact),
an AI model subscription & API key and
optional the latest C8Y MCP Server deployed to your tenant.
Check the updated Installation Guide for more details.

Afterwards you just need to (re-)deploy the 6.0.0 Dynamic Mapper microservice and it will configure / create all AI Agents automatically and enable the feature to be useable within the UI.

We tested a lot of templates with a very good output, but there might be always templates where the AI agents won't generate what you are probably expecting. Therefore this feature should be considered as preview and might be changed and improved in final & upcoming releases.

Feedback: Please share your feedback so we can improve the AI agents.

See related issue #350 #360 #361

Microservice Pre-builds

Before the 6.0.0 release we only provided one microservices with 0.5 CPUs + 2 GB RAM hard-coded in the manifest. This is working for most starting use cases to play around with dynamic mapper and up to 625 messages per seconds. For more performance we are now providing 4 builds with different CPU + Memory configuration. Here are all 4 pre-builds:

• Starter - Uses 0.5 CPUs and 2 GB RAM, suitable for up to 625 messages per second to be processed.
• Standard - Uses 2 CPU and 4 GB RAM, suitable for up to 2500 messages per second to be processed.
• Performance - Uses 8 CPU and 8 GB RAM, suitable for up to 10.000 messages per second to be processed.
• Max - Uses 16 CPU and 16 GB RAM, suitable for up to 20.000 messages per second to be processed.

To avoid costs we would ask you to start with Starter trying out the Dynamic Mapper. You can easily switch by just replacing the archive with one of the other 4 options later.

See related issue #349

Responsive UI

The UI has been improved to act more responsive on small screens. In previous versions we had cases that buttons / tool bars disappeared out of screen due to the fixed size of elements. This should be resolved now. #358

Other Changes

• We renamed Filter Mapping to Filter execution mapping making more clear that this filter defines if a mapping is executed or not. #354
• The limitation to only use one wildcard in topic for outbound mappings has been removed. Before change it was only possible to use something like this device/+/temp. Now it is also supported to use this device/+/+ which allows more dynamic mappings but more comprehensive mapping logic of course #357
• Make source payload editable in test UI to dynamically test different payloads #359
• We removed "c8y_IsDevice" fragment from Dynamic Mapper Managed Object to avoid unintended deletion. #362
• Shrinked the UI archive from 20Mb+ to ~11 MB now.

Full Changelog: v5.5.0...v6.0.0-pre

Release v5.5.0

What's Changed

⚠️ BREAKING CHANGES

• Role Names Changed & Permission Enforcement Added
• Renaming from Dynamic-Mapping to Dynamic Mapper

This release introduces breaking changes to role naming and adds strict permission enforcement.
Also due to the renaming from Dynamic-Mapping to Dynamic-Mapper some migration steps might be neccesary.

1. (Breaking) Role Name Changes

The following roles have been renamed:

Previous Role	New Role
ROLE_MAPPING_ADMIN	ROLE_MAPPER_ADMIN
ROLE_MAPPING_CREATE	ROLE_MAPPER_CREATE
ROLE_MAPPING_HTTP_CONNECTOR_CREATE	ROLE_MAPPER_HTTP_CONNECTOR_CREATE

Permission Enforcement

Permissions are now strictly enforced (previously not enforced):

• Default permissions: Users without assigned roles have read-only access to mappings, service configuration, and connectors
• Mapping permissions: Users with ROLE_MAPPER_CREATE role can create, update delete mappings and do device subscriptions for outbound mappings.
• Administrative functions: Users with ROLE_MAPPER_ADMIN role can change Service Configuration, Create/Update/Delete Connectors, add code-templates and others.
• HTTP connector message ingestion: Requires ROLE_MAPPER_HTTP_CONNECTOR_CREATE role

Migration Required

• ✅ Update existing role assignments to use new role names
• ✅ Review user permissions and assign appropriate roles
• ✅ Users who previously had implicit access will need explicit role assignments

2. (Breaking) Renaming to Dynamic Mapper

In this past the naming of Dynamic Mapper evovled. Starting with MQTT-Mapping it evolved to Dynamic-Mapping for some time. Now we consistently changed it it Dynamic-Mapper / Dynamic Mapper in the microservice and UI.

Migration Required

⚠️Important: In all cases we would like you to delete the old microservice & UI and re-deploy them with the new naming.

In the following cases additional migration is required:

1. You are using extension -> Please remove the internal & external extensions, restart the microservice (unsubscribe / subscribe) and upload them again. This is due to package name changes of the interface of extensions
2. Code-Mappings -> If you are using code-mappings the templates needs to be re-created. You can do that in Configuration --> Code templates --> Init system templates

3. Open API Documentation

With the introduction of permission enforcement we also documented all endpoints of all available controllers.
The API documentation is availavble in the repo: API Documentation.
You can also check & download the Open API Specification to be imported in any REST client of your choice or use the Swagger UI as part of the deployed microservice in your tenant. {YourTenantURL}/service/dynamic-mapper-service/swagger-ui/index.html

Bug Fixes

• Identity Cache is not properly updated when device is deleted #340
• Processing of messages fails when mapping statistic is reset #343

Changes

• Swagger API documentation #347
• Connector Annotations #346
• Refactor to use "dynamic.mapper" #345
• Consistent use of Dyanmic Mapper instead of Dynamic Mapping #344
• Protect API & UI functions with roles #338

Full Changelog: v5.1.2...v5.5.0

Release v5.1.2

What's Changed

• Fixing Bug #335
• HTTP Connection Pool Optimization
• Additional OTLP Metrics to measure processing time and C8Y Request processing time

Full Changelog: v5.1.1...v5.1.2

Release v5.1.1

What's Changed

• Fix for #333
• Fix for Inbound Code Mappings throw NPE for GraalsContext

Full Changelog: v5.1.0...v5.1.1

Release v5.1.0

New Features

• Reliable Messaging for MQTT connectors - We are now handling QoS in mappings & messages correctly. Meaning when creating mappings with QoS 1 or 2 all messages will be only acked at the broker when the processing in Cumulocity was successful. On reconnect of the connector non-acked messages will be retransmitted and again processed if cleanSession is set to false. This includes inbound & outbound mappings #317
• Max Failure Count Error handling - It is now possible to specifiy how many mappings errors should lead to a mapping to be deactivated. This is useful if you have faulty mappings which aren't working at all. They will be automatically deactivated when they reach the max failure count treshold. #326
• MQTT v5 support - It is now possible to select either v.3.1.1 or v5 for MQTT Broker connections #90
• Performance benchmark & script - We created a performance script to perform an end-to-end test, using MQTT Service as broker.
• Robustness Service - Add configuration setting maxCPU time in milliseconds for JavaScript code, to avoid malicious code from starving the CPU
• Robustness WebUI - When testing CodeBased mappings including JavaScript in the browser, this is executed seperatedly in a WebWorker and execution is terminated after 250 ms. This increases the stability of the browser.

Performance Test resuts

We achieved 2500 msg/s for 2 CPUs 4 GB RAM configured in the microservice manifest for graphical mappings using JSONata. Messages were sent with QoS 0. Theoretical linear scaling to 16 CPUs would mean up to 20k msg/s can be processed with a single instance of the dynamic mapper.

Code-mappings are currently 50% slower achieving 1250 msg/s using same setup (2 CPUs, 4 GB RAM) as they use more resources. Here we found things to optimize and are currently planning some improvements.

QoS 1 message could not be tested due to MQTT Service issues but will be repeated when issues are resolved.

The main limitation we saw in all cases are the C8Y HTTP Client Pool Connections of the Microservice SDK which is currently limited to 50 concurrent connections. Here is also space for improvement in later releases, currently we are blocking virtual threads using a semaphore of 50. If we would increase the concurrent connections, we might also have a higher throughput.

Fixes

• Stepping editor shows code editor for graphical mappings #331
• Outbound snooping is incosistent #327
• Performance improvements for code-mappings by re-using same engine per tenant
• Sample mapping don't appear in list #329
• Use concurrentHashMaps in multi-threaded cases #309
• Adding semaphore to limit C8Y HTTP Connection Pool to 50
• Updated dependencies to latest versions
• A lot of other minor bug fixes.

Full Changelog: v5.0.1...v5.1.0

Release v5.0.1

What's Changed

• Fixing requests hanging during microservice startup by @switschel in #323

Full Changelog: v5.0.0...v5.0.1

Release v5.0.0

Breaking Change incoming..

Caution: This release 5.0.0 contains a breaking change. Due to a security fix in Cumulocity the tenant options category must match the one defined in the manifest. Unfortunately the existing category key dynamic.mapper.service must be changed to dynMappingService" as no . are allowed in the key anymore.

As a result please save all your connector details before upgrading. After the upgrade a new service conifguration and connectors must be created manually.

In addition the MQTT_SERVICE_ADMIN role has been removed from the manifest as it is not existing anymore in the latest Cumulocity release. MQTT Service will be enabled by default in the upcoming Public Preview, no additional role is required.

New Features

• It is now possible to do JavaScript code-based mappings for inbound and outbound #305 Check the updated user guide
• Advanced filtering of mappings for inbound and outbound #312 Check the updated user guide
• Snooping of Inventory Update Outbound is now supported
• Introducing a Cumulocity Internal Flag for WebHooks to send data back to Cumulocity
• Enhanced code-templates for Inbound & Outbound code-based mappings
• Inventory Filter for MEAO messages so you can filter if a message should be mapped by fragments of the source device e.g. type
• Add samples for Inbound & Outbound mapping with a new introduced button to quickly get started with Dynamic Mapper
• Overwrite Method and publishTopic for WebHook in Outbound Mappings (before it was defaulted to POST only).
• Removed MQTT-Service role from Cumulocity manifest. There is only once microservice zip now.
• Adding a PATCH-method logic to update fragments in C8Y without overwriting all properties but just updating single properties of that fragment
• Overwrite API for JavaScript code-based mappings you can override the API to dynamically determine if you send an ALARM, EVENT; MEASUREMENT, ...
• Added OpenAPI specification of all REST endpoints
• Events can now be created with attachments from the Mapper UI. #283

Fixes

• A lot of bugs have been identified and fixed. For details check the detail change log.

Full Changelog: v4.8.0...v5.0.0

Pre-Release v4.9.2 (v5.0.0 pre)

New Features

• In addition to define your mappings/substitutions using JSONata, now you can define them using JavaScript code
• Introducing a Cumulocity Internal Flag for WebHooks to send data back to Cumulocity
• Enhanced code-templates for Inbound & Outbound code-based mappings
• Inventory Filter for MEAO messages so you can filter if a message should be mapped by fragments of the source device e.g. type
• Add samples for Inbound & Outbound mapping with a new introduced button to quickly get started with Dynamic Mapper
• Overwrite Method for WebHook in Outbound Mappings (before it was defaulted to POST only).
• Removed MQTT-Service role from Cumulocity manifest. There is only once microservice zip now.
• Adding a PATCH-method logic to update fragments in C8Y without overwriting all properties but just updating single properties of that fragment

Please note: This is a pre-release and it can contain breaking changes. It might be necessary to delete your service configuration, connectors and mappings. You can use the provided dm script to do so: https://github.com/Cumulocity-IoT/cumulocity-dynamic-mapper/blob/develop/resources/script/mgmt/dm.sh

Fixes

• A lot of bugs have been identified and fixed. For details check the detail change log.

Full Changelog: v4.8.0...v4.9.2