Skip to content

Releases: masesgroup/KEFCore

V4.1.0: updates to KNet 3.3.0 adding projection-aware data retrieve

Choose a tag to compare

@masesdevelopers masesdevelopers released this 28 Aug 16:59
d7a9b7d

Caution

KEFCore 4.1.0 is a complete architectural rewrite. Existing code targeting 2.x will require significant updates. See documentation for the migration guide.

Key breaking changes: DatabaseName renamed to TopicPrefix; full rename of methods and classes to be compliant with project name; singleton/context options separation in IKEFCoreSingletonOptions; PreserveStreamsAcrossContexts removed; ApplicationId, StreamsConfig and ProducerConfig promoted to singleton scope; IEntityTypeProducer replaced by IStreamsChangeManager; EntityState moved from IValueContainer into IKEFCoreRowBag.

Internal benchmarks (100-iteration test suite) show overall query execution time improving roughly 2x without the new value buffer caching, and roughly 7x with caching enabled, compared to the 2.6.7 baseline — with some query patterns improving by 10x or more. This is a combined result of KEFCore's new caching architecture and the underlying JNet/JCOBridge evolution from 2.6.8 to 2.6.9.

This is a major release of Entity Framework Core provider for Apache Kafka, available for Entity Framework Core 8, 9 and 10, based on KNet version 3.3.0, on top of Apache Kafka 4.3.1.

Entity Framework Core provider for Apache Kafka comes with the following changes:

  • MASES.EntityFrameworkCore.KNet:
    • Updates to KNet version 3.3.0 based on Apache Kafka 4.3.1
    • Adds projection-aware data retrieve (#719)
  • MASES.EntityFrameworkCore.KNet.Serialization:
    • Updates IValueContainerMetadata adding FullFlattenedProperties beside FlattenedProperties which contains projection properties
    • Review DefaultValueContainer to support #719
  • MASES.EntityFrameworkCore.KNet.Serialization.Avro:
    • Review AvroValueContainer to support #719
    • Updates Apache.Avro version
    • Adds custom AvroValueContainer codec to reduce boxing (#679)
  • MASES.EntityFrameworkCore.KNet.Serialization.Avro.Compiler: no changes
  • MASES.EntityFrameworkCore.KNet.Serialization.Protobuf:
    • Review ProtobufValueContainer to support #719
    • Updates Google.Protobuf version
  • MASES.EntityFrameworkCore.KNet.Templates: no changes
  • Documentation: aligns to the new architecture; adds backend compatibility reference
  • Tests: general improvements; updates complex type tests

Official NuGet packages (*)

(*) packages are also available on GitHub packages.

What's Changed

Full Changelog: V4.0.0.0...V4.1.0.0

V3.1.0: updates to KNet 2.9.13 adding projection-aware data retrieve

Choose a tag to compare

@masesdevelopers masesdevelopers released this 28 Aug 16:50
a7db945

Caution

KEFCore 3.1.0 is a complete architectural rewrite. Existing code targeting 2.x will require significant updates. See documentation for the migration guide.

Key breaking changes: DatabaseName renamed to TopicPrefix; full rename of methods and classes to be compliant with project name; singleton/context options separation in IKEFCoreSingletonOptions; PreserveStreamsAcrossContexts removed; ApplicationId, StreamsConfig and ProducerConfig promoted to singleton scope; IEntityTypeProducer replaced by IStreamsChangeManager; EntityState moved from IValueContainer into IKEFCoreRowBag.

Internal benchmarks (100-iteration test suite) show overall query execution time improving roughly 2x without the new value buffer caching, and roughly 7x with caching enabled, compared to the 2.6.7 baseline — with some query patterns improving by 10x or more. This is a combined result of KEFCore's new caching architecture and the underlying JNet/JCOBridge evolution from 2.6.8 to 2.6.9.

This is a major release of Entity Framework Core provider for Apache Kafka, available for Entity Framework Core 8, 9 and 10, based on KNet version 2.9.13, on top of Apache Kafka 3.9.2.

Entity Framework Core provider for Apache Kafka comes with the following changes:

  • MASES.EntityFrameworkCore.KNet:
    • Updates to KNet version 2.9.13 based on Apache Kafka 3.9.2
    • Adds projection-aware data retrieve (#719)
  • MASES.EntityFrameworkCore.KNet.Serialization:
    • Updates IValueContainerMetadata adding FullFlattenedProperties beside FlattenedProperties which contains projection properties
    • Review DefaultValueContainer to support #719
  • MASES.EntityFrameworkCore.KNet.Serialization.Avro:
    • Review AvroValueContainer to support #719
    • Updates Apache.Avro version
    • Adds custom AvroValueContainer codec to reduce boxing (#679)
  • MASES.EntityFrameworkCore.KNet.Serialization.Avro.Compiler: no changes
  • MASES.EntityFrameworkCore.KNet.Serialization.Protobuf:
    • Review ProtobufValueContainer to support #719
    • Updates Google.Protobuf version
  • MASES.EntityFrameworkCore.KNet.Templates: no changes
  • Documentation: aligns to the new architecture; adds backend compatibility reference
  • Tests: general improvements; updates complex type tests

Official NuGet packages (*)

(*) packages are also available on GitHub packages.

What's Changed

Full Changelog: V3.0.0.0...V3.1.0.0

V4.0.0: complete reengineering based on KNet 3.2.3 and JNet 2.6.9

Choose a tag to compare

@masesdevelopers masesdevelopers released this 17 Jun 04:39
a88b890

Caution

KEFCore 4.0.0 is a complete architectural rewrite. Existing code targeting 2.x will require significant updates. See documentation for the migration guide.

Key breaking changes: DatabaseName renamed to TopicPrefix; full rename of methods and classes to be compliant with project name; singleton/context options separation in IKEFCoreSingletonOptions; PreserveStreamsAcrossContexts removed; ApplicationId, StreamsConfig and ProducerConfig promoted to singleton scope; IEntityTypeProducer replaced by IStreamsChangeManager; EntityState moved from IValueContainer into IKEFCoreRowBag.

Internal benchmarks (100-iteration test suite) show overall query execution time improving roughly 2x without the new value buffer caching, and roughly 7x with caching enabled, compared to the 2.6.7 baseline — with some query patterns improving by 10x or more. This is a combined result of KEFCore's new caching architecture and the underlying JNet/JCOBridge evolution from 2.6.8 to 2.6.9.

This is a major release of Entity Framework Core provider for Apache Kafka, available for Entity Framework Core 8, 9 and 10, based on KNet version 3.2.3, on top of Apache Kafka 4.2.1.

Entity Framework Core provider for Apache Kafka comes with the following changes:

  • MASES.EntityFrameworkCore.KNet:
    • Updates to KNet version 3.2.3 based on Apache Kafka 4.2.1
    • Complete rewrite of the per-entity convention system: every aspect of Kafka behavior can now be configured via IModelFinalizingConvention, Attribute, or Fluent API, overriding context-level defaults (topic partitions, replication factor, retention, read-only enforcement, store lookup, SerDes selector, producer tuning)
    • Adds complete complex type support: IComplexTypeConverterFactory for complex type management, flattened property handling in query translation and serializers, complex-type projection and second-level extraction; enforces value equality on ComplexType via KEFCoreComplexTypeConverterConvention and attribute, requiring complex types to implement IEquatable or override Equals for correct comparison; requires Binary as the preferred conversion type on Avro and Protobuf serializers (instead of Text) for correct complex type round-tripping
    • Adds RocksDB, SSL and SASL configuration via Attribute and Fluent API, integrating with KNetRocksDBConfigSetter
    • Introduces KafkaClusterAdmin, separating admin operations from KafkaCluster; ClusterId now depends only on cluster info and is used for service provider matching
    • Introduces StreamsManager factory; removes PreserveStreamsAcrossContexts; streams lifecycle now managed via IStreamsChangeManager replacing IEntityTypeProducer
    • Separates singleton and context-scoped options in IKEFCoreSingletonOptions; promotes ApplicationId, StreamsConfig and ProducerConfig to singleton scope
    • Updates the Streams uncaught exception handler to detect StreamsException errors related to TimestampExtractor and attempt thread replacement (StreamThreadExceptionResponse.REPLACE_THREAD) instead of shutting down the application; adds explicit, deterministic handling of all KafkaStreams states in StreamsManager (PENDING_ERROR, PENDING_SHUTDOWN, ERROR, NOT_RUNNING, RUNNING) with a documented state diagram; adds a retrying GetIterator helper in KNetStreamsRetriever and KafkaStreamsBaseRetriever to recover from InvalidStateStoreException while the stream thread is STARTING, with cancellation support
    • Adds Kafka transactional producer with exactly-once semantics, async transactions, commit callbacks and future management
    • Adds in-memory value buffer caching for entity types with background population on partial enumeration
    • Implements Find behavior directly via the backend; updates the EF Core tracker directly when the backend reloads
    • Adds IValueGeneratorSelector support, threaded through cluster creation (IKafkaClusterCache.CreateCluster, KafkaCluster constructor) and exposed on IKafkaCluster; renames IKafkaIntegerValueGenerator to IKafkaValueGenerator and adds DateTime/DateTimeOffset generators; updates change detection in KafkaStateHelper to correctly treat a null value arriving from Kafka against a non-null current row value as a real change when bumping generators
    • Adds ResetStreams API; refactors topic handling to avoid Kafka Streams reset when persistent storage is in use
    • Renames DatabaseName to TopicPrefix; propagates TopicPrefix annotation along the full configuration chain
    • Adds infrastructure logger to KafkaCluster
    • Uses producer Send API with managed callback lifecycle
    • Adds optional RecyclableMemoryStream integration
    • Adds TranslateRightJoin support for EF Core 10
    • Validates topic ACLs and partition rules on startup
    • Removes Activator.CreateInstance from hot path
    • Improves resource disposal throughout: iterators, event data types, disposable object patterns
    • Full rename of methods and classes to be compliant with project name
  • MASES.EntityFrameworkCore.KNet.Serialization:
    • Introduces IValueContainerData, moving EntityState out of IValueContainer and into IKEFCoreRowBag
    • Revises IValueContainer
    • Adds optional parameters to IValueContainerMetadata
    • Adds IsComplex helper method, used across serializers
  • MASES.EntityFrameworkCore.KNet.Serialization.Avro:
    • Updates header serialization to use AddVoid
    • Centralizes type conversion for managed types of Avro SerDes
  • MASES.EntityFrameworkCore.KNet.Serialization.Avro.Compiler: no changes
  • MASES.EntityFrameworkCore.KNet.Serialization.Protobuf:
    • Updates header serialization to use AddVoid
    • Updates Google.Protobuf version
  • MASES.EntityFrameworkCore.KNet.Templates: no changes
  • Documentation: aligns to the new architecture; adds backend compatibility reference
  • Tests: general improvements; adds complex type tests

Official NuGet packages (*)

(*) packages are also available on GitHub packages.

What's Changed

Read more

V3.0.0: complete reengineering based on KNet 2.9.12 and JNet 2.6.9

Choose a tag to compare

@masesdevelopers masesdevelopers released this 17 Jun 04:34
d81eacf

Caution

KEFCore 3.0.0 is a complete architectural rewrite. Existing code targeting 2.x will require significant updates. See documentation for the migration guide.

Key breaking changes: DatabaseName renamed to TopicPrefix; full rename of methods and classes to be compliant with project name; singleton/context options separation in IKEFCoreSingletonOptions; PreserveStreamsAcrossContexts removed; ApplicationId, StreamsConfig and ProducerConfig promoted to singleton scope; IEntityTypeProducer replaced by IStreamsChangeManager; EntityState moved from IValueContainer into IKEFCoreRowBag.

Internal benchmarks (100-iteration test suite) show overall query execution time improving roughly 2x without the new value buffer caching, and roughly 7x with caching enabled, compared to the 2.6.7 baseline — with some query patterns improving by 10x or more. This is a combined result of KEFCore's new caching architecture and the underlying JNet/JCOBridge evolution from 2.6.8 to 2.6.9.

This is a major release of Entity Framework Core provider for Apache Kafka, available for Entity Framework Core 8, 9 and 10, based on KNet version 2.9.12, on top of Apache Kafka 3.9.2.

Entity Framework Core provider for Apache Kafka comes with the following changes:

  • MASES.EntityFrameworkCore.KNet:
    • Updates to KNet version 2.9.12 based on Apache Kafka 3.9.2
    • Complete rewrite of the per-entity convention system: every aspect of Kafka behavior can now be configured via IModelFinalizingConvention, Attribute, or Fluent API, overriding context-level defaults (topic partitions, replication factor, retention, read-only enforcement, store lookup, SerDes selector, producer tuning)
    • Adds complete complex type support: IComplexTypeConverterFactory for complex type management, flattened property handling in query translation and serializers, complex-type projection and second-level extraction; enforces value equality on ComplexType via KEFCoreComplexTypeConverterConvention and attribute, requiring complex types to implement IEquatable or override Equals for correct comparison; requires Binary as the preferred conversion type on Avro and Protobuf serializers (instead of Text) for correct complex type round-tripping
    • Adds RocksDB, SSL and SASL configuration via Attribute and Fluent API, integrating with KNetRocksDBConfigSetter
    • Introduces KafkaClusterAdmin, separating admin operations from KafkaCluster; ClusterId now depends only on cluster info and is used for service provider matching
    • Introduces StreamsManager factory; removes PreserveStreamsAcrossContexts; streams lifecycle now managed via IStreamsChangeManager replacing IEntityTypeProducer
    • Separates singleton and context-scoped options in IKEFCoreSingletonOptions; promotes ApplicationId, StreamsConfig and ProducerConfig to singleton scope
    • Updates the Streams uncaught exception handler to detect StreamsException errors related to TimestampExtractor and attempt thread replacement (StreamThreadExceptionResponse.REPLACE_THREAD) instead of shutting down the application; adds explicit, deterministic handling of all KafkaStreams states in StreamsManager (PENDING_ERROR, PENDING_SHUTDOWN, ERROR, NOT_RUNNING, RUNNING) with a documented state diagram; adds a retrying GetIterator helper in KNetStreamsRetriever and KafkaStreamsBaseRetriever to recover from InvalidStateStoreException while the stream thread is STARTING, with cancellation support
    • Adds Kafka transactional producer with exactly-once semantics, async transactions, commit callbacks and future management
    • Adds in-memory value buffer caching for entity types with background population on partial enumeration
    • Implements Find behavior directly via the backend; updates the EF Core tracker directly when the backend reloads
    • Adds IValueGeneratorSelector support, threaded through cluster creation (IKafkaClusterCache.CreateCluster, KafkaCluster constructor) and exposed on IKafkaCluster; renames IKafkaIntegerValueGenerator to IKafkaValueGenerator and adds DateTime/DateTimeOffset generators; updates change detection in KafkaStateHelper to correctly treat a null value arriving from Kafka against a non-null current row value as a real change when bumping generators
    • Adds ResetStreams API; refactors topic handling to avoid Kafka Streams reset when persistent storage is in use
    • Renames DatabaseName to TopicPrefix; propagates TopicPrefix annotation along the full configuration chain
    • Adds infrastructure logger to KafkaCluster
    • Uses producer Send API with managed callback lifecycle
    • Adds optional RecyclableMemoryStream integration
    • Adds TranslateRightJoin support for EF Core 10
    • Validates topic ACLs and partition rules on startup
    • Removes Activator.CreateInstance from hot path
    • Improves resource disposal throughout: iterators, event data types, disposable object patterns
    • Full rename of methods and classes to be compliant with project name
  • MASES.EntityFrameworkCore.KNet.Serialization:
    • Introduces IValueContainerData, moving EntityState out of IValueContainer and into IKEFCoreRowBag
    • Revises IValueContainer
    • Adds optional parameters to IValueContainerMetadata
    • Adds IsComplex helper method, used across serializers
  • MASES.EntityFrameworkCore.KNet.Serialization.Avro:
    • Updates header serialization to use AddVoid
    • Centralizes type conversion for managed types of Avro SerDes
  • MASES.EntityFrameworkCore.KNet.Serialization.Avro.Compiler: no changes
  • MASES.EntityFrameworkCore.KNet.Serialization.Protobuf:
    • Updates header serialization to use AddVoid
    • Updates Google.Protobuf version
  • MASES.EntityFrameworkCore.KNet.Templates: no changes
  • Documentation: aligns to the new architecture; adds backend compatibility reference
  • Tests: general improvements; adds complex type tests

Official NuGet packages (*)

(*) packages are also available on GitHub packages.

What's Changed

Read more

V3.0.0-rc4: temporary release

Pre-release

Choose a tag to compare

@masesdevelopers masesdevelopers released this 10 Jun 20:18
2fb7e5f

What's Changed

  • Bump MASES.KNet.Serialization.Avro from 3.2.3-rc94 to 3.2.3-rc95 by @dependabot[bot] in #620
  • Bump MASES.KNet from 3.2.3-rc94 to 3.2.3-rc95 by @dependabot[bot] in #619
  • Bump bojieyang/indexnow-action from 2 to 3 by @dependabot[bot] in #621
  • Bump MASES.KNet.Serialization.Avro from 3.2.3-rc95 to 3.2.3-rc96 by @dependabot[bot] in #624
  • Updates tests to be compliant with changes in #531 by @masesdevelopers in #626
  • Dispose objects returned from iterators by @masesdevelopers in #627
  • Dispose the object at its origin avoiding multiple Dispose invocations by @masesdevelopers in #628
  • Bump MASES.KNet from 3.2.3-rc95 to 3.2.3-rc991 by @dependabot[bot] in #623
  • Update version from 3.0.0-rc3 to 3.0.0-rc4 by @github-actions[bot] in #630

Full Changelog: V3.0.0.0-rc3...V3.0.0.0-rc4

V3.0.0-rc3: temporary release

Pre-release

Choose a tag to compare

@masesdevelopers masesdevelopers released this 26 May 09:27
c7cb860

What's Changed

  • Bump MASES.KNet.Serialization.Avro from 3.2.3-rc93 to 3.2.3-rc94 by @dependabot[bot] in #615
  • Bump MASES.KNet from 3.2.3-rc93 to 3.2.3-rc94 by @dependabot[bot] in #614
  • Update version from 3.0.0-rc2 to 3.0.0-rc3 by @github-actions[bot] in #616
  • Revert "Bump MASES.KNet from 3.2.3-rc93 to 3.2.3-rc94" by @masesdevelopers in #617
  • Revert "Revert "Bump MASES.KNet from 3.2.3-rc93 to 3.2.3-rc94"" by @masesdevelopers in #618

Full Changelog: V3.0.0.0-rc2...V3.0.0.0-rc3

V3.0.0-rc2: temporary release

Pre-release

Choose a tag to compare

@masesdevelopers masesdevelopers released this 24 May 00:26
b17da0c

What's Changed

  • Update for per-entity TopicPrefix convention by @masesdevelopers in #595
  • Improve resource disposal and event data types by @masesdevelopers in #598
  • Bump MASES.KNet.Serialization.Avro from 3.2.3-rc8 to 3.2.3-rc9 by @dependabot[bot] in #601
  • Bump MASES.KNet from 3.2.3-rc8 to 3.2.3-rc9 by @dependabot[bot] in #600
  • Switch Register/Unregister of IStreamsManager from IEntityTypeProducer to IStreamsChangeManager with armonization of event source detection by @masesdevelopers in #597
  • Bump Microsoft.SourceLink.GitHub from 10.0.203 to 10.0.300 by @dependabot[bot] in #605
  • Bump MASES.KNet.Serialization.Avro from 3.2.3-rc9 to 3.2.3-rc91 by @dependabot[bot] in #603
  • Bump Microsoft.EntityFrameworkCore from 9.0.15 to 9.0.16 by @dependabot[bot] in #604
  • Bump MASES.KNet from 3.2.3-rc9 to 3.2.3-rc91 by @dependabot[bot] in #602
  • Bump Google.Protobuf.Tools from 3.34.1 to 3.35.0 by @dependabot[bot] in #607
  • Bump MASES.KNet.Serialization.Avro from 3.2.3-rc91 to 3.2.3-rc92 by @dependabot[bot] in #609
  • Bump Google.Protobuf from 3.34.1 to 3.35.0 by @dependabot[bot] in #606
  • Bump MASES.KNet from 3.2.3-rc91 to 3.2.3-rc92 by @dependabot[bot] in #608
  • Removes occurrences of Activator.CreateInstance in hot path by @masesdevelopers in #610
  • Bump MASES.KNet.Serialization.Avro from 3.2.3-rc92 to 3.2.3-rc93 by @dependabot[bot] in #612
  • Bump MASES.KNet from 3.2.3-rc92 to 3.2.3-rc93 by @dependabot[bot] in #611
  • Update version from 3.0.0-rc1 to 3.0.0-rc2 by @github-actions[bot] in #613

Full Changelog: V3.0.0.0-rc1...V3.0.0.0-rc2

V3.0.0-rc1: temporary release

Pre-release

Choose a tag to compare

@masesdevelopers masesdevelopers released this 09 May 01:16
dee2e6f

What's Changed

Full Changelog: V3.0.0.0-rc...V3.0.0.0-rc1

V3.0.0-rc: temporary release

Pre-release

Choose a tag to compare

@masesdevelopers masesdevelopers released this 28 Mar 07:02
68002c9

What's Changed

  • Update documentation after commit 0446e0a by @github-actions[bot] in #548
  • Bump Google.Protobuf.Tools from 3.34.0 to 3.34.1 by @dependabot[bot] in #550
  • Bump Google.Protobuf from 3.34.0 to 3.34.1 by @dependabot[bot] in #549
  • Review documentation adding backend compatibility by @masesdevelopers in #553
  • Update documentation after commit e1edfaf by @github-actions[bot] in #554
  • Adds professional support by @masesdevelopers in #555
  • Bump MASES.KNet.Serialization.Avro from 3.2.1 to 3.2.2 by @dependabot[bot] in #558
  • Bump MASES.KNet from 3.2.1 to 3.2.2 by @dependabot[bot] in #557
  • Implementation of RocksDb, SSL and SASL configuration using Attribute and Fluent API by @masesdevelopers in #552
  • Update version from 3.0.0-beta to 3.0.0-rc by @github-actions[bot] in #556
  • Update documentation after commit 932c514 by @github-actions[bot] in #559

Full Changelog: V3.0.0.0-beta...V3.0.0.0-rc

V3.0.0-beta: temporary release

Pre-release

Choose a tag to compare

@masesdevelopers masesdevelopers released this 20 Mar 18:20
0446e0a

What's Changed

Full Changelog: V3.0.0.0-alpha4...V3.0.0.0-beta