JSON Transformation Engine

A flexible and high-performance library for transforming JSON data with rule-based configuration.

Overview

Write transformations as declaritive JSON rules, not code. The engine compiles your rule DSL into fast, production‑grade transformations that you can version, test, and ship independently from your services. Compose operations, target specific customers and data types, and scale to high‑throughput pipelines without changing your application code.

Features

• Declarative JSON rules DSL: Ship transformations as data, not code
• Composable operations: Structural, array, value, conditional, filtering, aggregation
• Multi-tenant ready: Customer‑specific overrides by eventType and customerId
• Production performance: Pre‑compiled rules, multi-level caching
• Library‑agnostic JSON: Works across JSON libraries via JsonEngineNode
• Batteries included: Spring Boot auto‑config, file‑based rule loading, rich test suite

Supported Operations

• Structural: RENAME_FIELD, COPY_FIELD, MOVE_FIELD, REMOVE_FIELD, OBJECT_FLATTEN, OBJECT_UNFLATTEN, OBJECT_MERGE, OBJECT_SPLIT, NEST_FIELDS, WRAP, UNWRAP, PIVOT, UNPIVOT, LOOKUP
• Value & Expression: DATE_FORMAT, DEFAULT_VALUE, SET_DEFAULT, TYPE_CONVERSION, STRING_OPERATION, MATH_OPERATION, EXPRESSION
• Array & Aggregation: ARRAY_MAP, ARRAY_FILTER, ARRAY_SORT, ARRAY_UNIQUE, ARRAY_CHUNK, ARRAY_REDUCE, ARRAY_FLATMAP, COUNT, SUM, AVERAGE, MIN, MAX, FIRST, LAST, DISTINCT, GROUP_BY
• Filtering & Conditional: FILTER, KEEP_FIELDS, REMOVE_FIELDS, CONDITION

60‑second Quick Start (the DSL in action)

1. Define a rule in JSON

{
  "id": "order-normalization",
  "eventType": "ORDER",
  "operations": [
    { "type": "RENAME_FIELD", "sourcePath": "orderNumber", "targetPath": "id" },
    { "type": "DATE_FORMAT",  "path": "timestamp", "sourceFormat": "unix", "targetFormat": "ISO-8601" },
    { "type": "ARRAY_MAP",    "arrayPath": "items", "operations": [
      { "type": "SET_DEFAULT", "path": "discount", "value": 0 }
    ]}
  ]
}

2. Apply it to any JSON string

// Create a facade and compile the rule DSL
TransformationEngineFacade facade = new TransformationEngineFacade();
TransformationEngine engine = facade.createFromJson(ruleJsonString);

// Transform JSON → JSON in one line
String outputJson = facade.createAdapter(engine).transform(inputJsonString);

Example input (before transformation):

{
  "orderNumber": "ORD-42",
  "timestamp": 1625097600,
  "items": [
    {
      "sku": "SKU-001",
      "price": 199.99
    }
  ]
}

Example output (after transformation):

{
  "id": "ORD-42",
  "timestamp": "2021-07-01T00:00:00Z",
  "items": [
    {
      "sku": "SKU-001",
      "price": 199.99,
      "discount": 0
    }
  ]
}

That’s it: no model changes, no custom code. Swap or update rules without touching your services.

Getting Started

Maven Dependency

Add the following dependency to your project:

<dependency>
    <groupId>com.salesforce.jsonengine</groupId>
    <artifactId>json-transformation-engine</artifactId>
    <version>1.0.0</version>
</dependency>

Basic Java Interface Usage

// Get a transformer for a specific type
TransformationEngine transformer = facade.getEngine("ORDER");

// Parse or obtain a JsonEngineNode (library-agnostic JSON abstraction)
JsonEngineNode inputNode = facade.parseString(inputJsonString);

// Apply the transformation to a JsonEngineNode
JsonEngineNode outputNode = transformer.transform(inputNode);

// Or use the adapter to work with POJOs/strings
TransformationEngineAdapter adapter = facade.createAdapter(transformer);
OrderOutput typedOutput = adapter.transform(orderInput, OrderOutput.class);
String transformedJson = adapter.transform(inputJsonString);

Creating and Registering a Simple Transformer

// Create at application startup
List<TransformationEngine> transformers = new ArrayList<>();

// Obtain the parser adapter (default implementation is selected automatically)
JsonParserAdapter parserAdapter = JsonEngineParserFactory.getDefaultAdapter();

// Add multiple transformers (constructors require a JsonParserAdapter)
transformers.add(new FieldRenameTransformer("user.name", "user.fullName", parserAdapter));
transformers.add(DateFormatTransformer.unixToIso("timestamp", parserAdapter));
transformers.add(new DefaultValueTransformer("user.status", parserAdapter.createPrimitive("active"), parserAdapter));

// Create a composite transformer to be cached
TransformationEngine compositeTransformer = new CompositeTransformationEngine(transformers);

// Register with the facade
facade.registerEngine("USER_PROFILE", compositeTransformer);

Rule-based Configuration

For more flexible management, transformation rules can be defined as JSON and loaded at startup:

{
  "id": "premium-customer-order-transform",
  "customerId": "12345-ABC",
  "eventType": "ORDER",
  "operations": [
    {
      "type": "RENAME_FIELD",
      "sourcePath": "order.items",
      "targetPath": "items"
    },
    {
      "type": "DATE_FORMAT",
      "path": "timestamp",
      "sourceFormat": "unix",
      "targetFormat": "ISO-8601"
    },
    {
      "type": "ARRAY_MAP",
      "arrayPath": "items",
      "operations": [
        {
          "type": "SET_DEFAULT",
          "path": "discount",
          "value": 0.1
        }
      ]
    }
  ]
}

High-Performance Transformations with PerformanceOptimizedTransformer

For systems with high throughput requirements, the library provides a specialized PerformanceOptimizedTransformer that significantly improves performance through advanced optimization techniques.

Key Features

• Result Caching: Remembers transformation results to avoid redundant computation
• Value Hashing: Ensures documents with different values get different results
• Path Pre-computation: Prepares frequently accessed paths for faster processing

When to Use

Use the PerformanceOptimizedTransformer when:

• Processing large volumes of events (thousands or millions per second)
• Dealing with similar document structures repeatedly
• CPU utilization is a bottleneck in your application
• You need to minimize transformation latency

Basic Rules Interface Usage Pattern

{
  "id": "premium-customer-order-transform",
  "customerId": "12345-ABC",
  "eventType": "ORDER",
  "cache": {
    "enabled": false
  },
  "operations": [
    {
      "type": "RENAME_FIELD",
      "cache": {
        "enabled": true
      },
      "sourcePath": "order.items",
      "targetPath": "items"
    },
    {
      "type": "DATE_FORMAT",
      "path": "timestamp",
      "sourceFormat": "unix",
      "targetFormat": "ISO-8601"
    },
    {
      "type": "ARRAY_MAP",
      "arrayPath": "items",
      "cache": {
        "enabled": true
      },
      "operations": [
        {
          "type": "SET_DEFAULT",
          "path": "discount",
          "value": 0.1
        }
      ]
    }
  ]
}

Note: Caching can be enabled at the rule level or individual operation level. It is disabled by default.

Basic Java Interface Usage Pattern

// Create your normal transformer
TransformationEngine baseTransformer = new CompositeTransformationEngine(transformers);

// Wrap it with the performance optimizer (requires a JsonParserAdapter)
JsonParserAdapter parserAdapter = JsonEngineParserFactory.getDefaultAdapter();
PerformanceOptimizedTransformer optimizedTransformer =
    new PerformanceOptimizedTransformer(baseTransformer, parserAdapter);

// Apply transformations normally
JsonEngineNode result = optimizedTransformer.transform(inputNode);

Advanced Configuration

// Create with custom settings
JsonParserAdapter parserAdapter = JsonEngineParserFactory.getDefaultAdapter();
PerformanceOptimizedTransformer optimizedTransformer = new PerformanceOptimizedTransformer(
    baseTransformer,      // The underlying transformer to optimize
    parserAdapter,        // Parser adapter
    5000                  // Cache size (number of results to store)
);

// Pre-compute paths that will be accessed frequently
optimizedTransformer.precomputePaths(
    "user.profile.name",
    "user.profile.email",
    "metadata.timestamp"
);

Spring Boot Integration

The library provides automatic integration with Spring Boot:

// Configuration is automatically provided by TransformationEngineAutoConfiguration

@Service
public class DataTransformationService {
    @Autowired
    private TransformationEngineFacade facade;
    
    public <T> T transformData(T data, String type) {
        TransformationEngine transformer = facade.getEngine(type);
        TransformationEngineAdapter adapter = facade.createAdapter(transformer);
        return adapter.transform(data);
    }
}

// Load rules at application startup (construct loader using provided beans)
@Component
public class RuleLoaderInitializer implements ApplicationRunner {
    @Autowired private TransformationRuleParser ruleParser;
    @Autowired private TransformationEngineFacade facade;
    @Autowired private JsonParserAdapter parserAdapter;

    @Override
    public void run(ApplicationArguments args) {
        TransformationRuleLoader loader = new FileBasedTransformationRuleLoader(
            parserAdapter,
            "config/transformation-rules"
        );

        List<TransformationRule> rules = loader.loadRules();
        for (TransformationRule rule : rules) {
            TransformationEngine transformer = ruleParser.parseRule(rule);
            facade.registerEngine(
                rule.getEventType(),
                rule.getCustomerId(),
                transformer
            );
        }
    }
}

Additional Resources

• Detailed Documentation
• Transformation Rules Documentation
• Transformation Engine Components Overview
• JSON Schema for Rules
• Custom Transformer Guide
• Performance Optimized Transformers Guide
• Example Projects