OLake Roadmap Discussion

[zriyanshdz]

Introduction

Hello everyone! We’re thrilled to share our open-source roadmap for OLake, starting from April 2025. This overview highlights upcoming connectors, integrations, and platform enhancements. As this is a living document, changes may occur based on community feedback and evolving project priorities.

We’d love your thoughts on:

• Which features you’d like to see prioritized.
• Areas that might need extra attention or rethinking.
• Any potential contributions you’d be interested in making.

Your input is valuable, so feel free to comment below!

Quarter 2 Roadmap

• Incremental Sync Support

  • MySQL
  • PostgreSQL - Related Issue
  • MongoDB - Related Issue

• Soft Delete Support Enhancements
  Currently we support soft delete via Iceberg equality delete. We will add support for:

  • Append-only use cases
  • Hard delete use cases

• Schema Evolution

  • Handle deletion/addition of new columns
  • Creation of new tables
  • Renaming of tables/columns
  • Changing a column type to nullable
  • Handling data type changes (current implementation involves creating new parquet files)

• Writer Improvments

  • Partitioning in Iceberg Writer
  • Rust based Iceberg Writer (Currently written in Java optimised by moving to Go or Rust based approach) - Related Issue

• Support for Various Iceberg Catalogs

  • REST (Polaris, Nessie, Gravitino, Unity)
  • AWS Glue - Link
  • Hive
  • JDBC - Link

• Source Connectors: Postgres, MySQL, Mongo - Improvements and bug resolutions

• New Sources (based on community requests)

  • MS SQL
  • Kafka - Related Issue
  • S3 - Related Issue
  • REST API
  • db2
  • Oracle

• Heartbeat Feature - Related Issue

• Integration with Tools

  • Airflow
  • Temporal (for Orchestration)

• Ingestion Logs on GUI

Quarter 3 Roadmap

• Integration (Monitoring and logging, upon community request)

  • Prometheus / Grafana / Loki
  • SigNoz

• Further Integration Support

  • DBT

Note: This roadmap may be revised based on community feedback or shifting priorities. Please share your ideas or contributions in the discussion!

How to Contribute

1. Suggest Changes: Share your ideas, concerns, or feature requests in this discussion.
2. Open Issues: If a particular item intrigues you or you’ve got an alternate approach, open a GitHub Issue to start a more focused conversation.
3. Submit PRs: We wholeheartedly welcome pull requests! If you’re ready to code, open a PR referencing the roadmap item you’re tackling.

Read guidelines here before you start contributing.

Feedback & Discussion Points

1. Feature Prioritization: Which of the roadmap items should be at the top of our list, based on your needs or experiences?
2. Technical Feedback: Have suggestions for frameworks, libraries, or best practices we could adopt?
3. Missing Features: Is there anything crucial you’d like to see added to the roadmap?
4. Community Resources: Would supporting documentation, tutorials, or sample projects make OLake easier to adopt?

Closing Thoughts

We’re excited to collaborate with the community to make OLake even more powerful and flexible. Please leave your comments, ideas, or questions below. Thank you for being part of the journey!

[heman026]

Any idea Oracle Support will be available? I saw a PR available for Oracle Source connector- #355.

[zriyanshdz]

cc @rkhameshra

[rkhameshra]

Hey @heman026 !

Yes, Oracle support is coming very soon. We’ll be releasing the JDBC-based Oracle connector (supporting both full load and incremental sync) in the next couple of weeks. The current PR you saw is part of that effort — we're finalizing testing and documentation to make it production-ready.

Let me know if you'd like early access or want to try it out as soon as it's available.

[heman026]

I'd like to have early access to test it. Thanks

[rkhameshra]

Hey @heman026!

We are live with oracle as a source, it currently supports full refresh functionality via JDBC driver and we are working towards adding incremental support.
Let us know how your test goes. Happy to help if required anywhere.
Docs: https://olake.io/docs/connectors/oracle/overview