🗄️ SurrealDB: Why We Use It & Known Challenges
This is a tracking issue that documents our database architecture decision and groups related issues. Unlike other umbrella issues focused on community work, this one explains why we made this choice and tracks challenges we're actively working on.
Why SurrealDB?
Open Notebook chose SurrealDB for several strategic reasons:
1. Multi-model in One
SurrealDB combines document store, graph database, and relational features. This matters for Open Notebook because:
- Sources are documents with metadata
- Notebooks ↔ Sources relationships are graph-like
- Embeddings are built-in — we already use them extensively for semantic search
2. Graph-First Future
We plan to significantly expand the use of graph relationships in the product. SurrealDB's native graph capabilities will enable:
- Complex knowledge connections between sources
- Cross-notebook relationships
- Semantic linking of concepts
3. AI-Oriented Database
SurrealDB is explicitly focused on AI use cases and actively evolving in this direction:
- Native vector embeddings (which we already use)
- ML-friendly query patterns
- Continuous improvements for AI workloads
4. Frontend-Accessible (like Firebase/Supabase)
SurrealDB can be accessed directly from the frontend, similar to Firebase or Supabase. This enables:
- Real-time subscriptions from the UI
- Simplified architecture
- Future possibilities for offline-first features
5. Simplified Infrastructure
Traditional stacks require multiple services:
Typical stack: Open Notebook stack:
├── Postgres └── SurrealDB (does it all)
├── Redis (cache) ├── Data storage
├── Celery (jobs) ├── Background jobs (surreal-commands)
└── Vector DB ├── Vector embeddings
└── Real-time subscriptions
This means:
- Easier self-hosting — one database to manage
- Simpler Docker setup — fewer containers
- Lower resource usage — important for local/privacy-focused users
Known Trade-offs
We're aware of these challenges:
| Challenge |
Impact |
Our Approach |
| Younger ecosystem |
Fewer tutorials, smaller community |
We document more, contribute back |
| Transaction conflicts |
Verbose error logs under concurrency |
✅ Resolved — retries via Tenacity; confirmed log-verbosity only (#362, #373) |
| Performance tuning |
Less established best practices |
Profile and optimize as we go (source listing optimized, #351) |
| Enterprise readiness |
Questions about production scale |
Monitor closely, have fallback plan |
| Version upgrades |
Breaking changes across SurrealDB majors |
Track and migrate deliberately (v3 compat, #378) |
About Transaction Conflicts
This was a known issue that we've since worked through:
Current Decision
Stay with SurrealDB, work through the challenges.
Rationale
- Migration cost is high — Rewriting data layer + losing graph features
- Problems are addressable — The early concurrency/performance concerns are resolved
- Unique value — No other single DB gives us document + graph + jobs
- Aligned with our users — Privacy-focused users prefer simpler infra
When We'd Reconsider
- Transaction conflicts become unworkable despite optimizations
- Performance doesn't improve with query tuning
- Critical security issue without timely fix
- A clear alternative emerges with same benefits + maturity
Issues Being Tracked
Open
Resolved ✅
Alternatives We Considered
| Option |
What We'd Gain |
What We'd Lose |
| PostgreSQL + pgvector |
Maturity, ecosystem, proven scale |
Graph queries, simple jobs (need Celery) |
| SQLite + LiteFS |
Ultimate simplicity, zero config |
Scale, concurrency, graph features |
| MongoDB + Redis + Celery |
Familiar stack, lots of tooling |
Simplicity, our infra advantage |
| Hybrid (Postgres + Neo4j) |
Best of both worlds |
Complexity, ops burden |
For Contributors
If you're experiencing SurrealDB issues:
- Check if it's known — Look at linked issues above
- Provide details — SurrealDB version, query patterns, data volume
- Share workarounds — If you found one, others benefit
If you want to help:
- Performance profiling is welcome
- Query optimization PRs appreciated
- Documentation of patterns that work
🚀 Opportunity: Schema & Feature Optimization
We started using SurrealDB before fully understanding everything it could do.
This means there's significant opportunity to improve our current schema and leverage features we're not yet using. If you have SurrealDB expertise, you could help with:
- Schema optimization — Better table structures, indexes, relations
- Query patterns — More idiomatic SurrealQL
- Feature adoption — Using capabilities we haven't explored yet
- Performance tuning — Identifying bottlenecks and fixes
This is a great way to contribute if you know SurrealDB well — you'd be directly improving the database layer that powers everything.
💬 We Want Your Input
Do you have experience with database infrastructure at scale?
We'd love to hear from you. This is an open discussion — we're not married to any particular solution. If you have:
- Experience running SurrealDB in production
- Suggestions for alternative architectures
- Ideas for optimizations we haven't considered
- War stories from similar migrations
- SurrealDB expertise — schema design, query optimization, best practices
Please comment on this issue. We value practical experience over theoretical debates. Tell us what you've seen work (or fail) in real-world scenarios.
💡 SurrealDB community: We're actively sharing this in SurrealDB forums. If you landed here from there — welcome! We'd especially appreciate your database expertise.
References
Maintainer: @lfnovo
Last updated: 2026-06-21
🗄️ SurrealDB: Why We Use It & Known Challenges
This is a tracking issue that documents our database architecture decision and groups related issues. Unlike other umbrella issues focused on community work, this one explains why we made this choice and tracks challenges we're actively working on.
Why SurrealDB?
Open Notebook chose SurrealDB for several strategic reasons:
1. Multi-model in One
SurrealDB combines document store, graph database, and relational features. This matters for Open Notebook because:
2. Graph-First Future
We plan to significantly expand the use of graph relationships in the product. SurrealDB's native graph capabilities will enable:
3. AI-Oriented Database
SurrealDB is explicitly focused on AI use cases and actively evolving in this direction:
4. Frontend-Accessible (like Firebase/Supabase)
SurrealDB can be accessed directly from the frontend, similar to Firebase or Supabase. This enables:
5. Simplified Infrastructure
Traditional stacks require multiple services:
This means:
Known Trade-offs
We're aware of these challenges:
About Transaction Conflicts
This was a known issue that we've since worked through:
Current Decision
Rationale
When We'd Reconsider
Issues Being Tracked
Open
needs-designResolved ✅
Alternatives We Considered
For Contributors
If you're experiencing SurrealDB issues:
If you want to help:
🚀 Opportunity: Schema & Feature Optimization
We started using SurrealDB before fully understanding everything it could do.
This means there's significant opportunity to improve our current schema and leverage features we're not yet using. If you have SurrealDB expertise, you could help with:
This is a great way to contribute if you know SurrealDB well — you'd be directly improving the database layer that powers everything.
💬 We Want Your Input
Do you have experience with database infrastructure at scale?
We'd love to hear from you. This is an open discussion — we're not married to any particular solution. If you have:
Please comment on this issue. We value practical experience over theoretical debates. Tell us what you've seen work (or fail) in real-world scenarios.
References
Maintainer: @lfnovo
Last updated: 2026-06-21