Skip to content

Commit a923130

Browse files
committed
fix: async channels now actually work (v0.2.1)
Fixed completely broken async send/recv implementation: - Was referencing non-existent queue fields - channels.nim had duplicate type definitions conflicting with SPSC - Removed 100+ lines of duplicate code Improvements: - Added exponential backoff (1ms → 100ms) to reduce CPU usage - Fixed Chronos deprecation warnings - Now properly imports from channel_spsc Testing: - Added comprehensive async channel tests - All tests pass (basic, channel, async) - Added async tests to CI workflow Updated documentation: - KNOWN_ISSUES.md reflects improvements - README.md updated for v0.2.1 - Added detailed CHANGELOG entry
1 parent 16ccce7 commit a923130

File tree

10 files changed

+441
-642
lines changed

10 files changed

+441
-642
lines changed

.github/workflows/ci.yml

Lines changed: 50 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,55 @@
11
name: CI
2-
on: [push, pull_request]
2+
3+
on:
4+
push:
5+
branches: [ main, develop ]
6+
pull_request:
7+
branches: [ main ]
8+
39
jobs:
410
test:
11+
name: Test SPSC Channels
512
runs-on: ubuntu-latest
13+
614
steps:
7-
- uses: actions/checkout@v4
8-
- uses: jiro4989/setup-nim-action@v2
9-
with:
10-
nim-version: '2.2.4'
11-
- run: nimble test
12-
- run: nimble bench
15+
- uses: actions/checkout@v3
16+
17+
- name: Setup Nim
18+
uses: jiro4989/setup-nim-action@v1
19+
with:
20+
nim-version: 'stable'
21+
22+
- name: Install dependencies
23+
run: nimble install -y
24+
25+
- name: Run channel tests
26+
run: nim c -r tests/unit/test_channel.nim
27+
28+
- name: Run async channel tests
29+
run: nim c -r tests/unit/test_async_channel.nim
30+
31+
- name: Run basic tests
32+
run: nim c -r tests/unit/test_basic.nim
33+
34+
benchmark:
35+
name: SPSC Performance Validation
36+
runs-on: ubuntu-latest
37+
38+
steps:
39+
- uses: actions/checkout@v3
40+
41+
- name: Setup Nim
42+
uses: jiro4989/setup-nim-action@v1
43+
with:
44+
nim-version: 'stable'
45+
46+
- name: Install dependencies
47+
run: nimble install -y
48+
49+
- name: Build benchmark
50+
run: nim c -d:danger --opt:speed --threads:on --mm:orc tests/performance/benchmark_spsc.nim
51+
52+
- name: Run benchmark and validate
53+
run: |
54+
echo "Running SPSC benchmark..."
55+
./tests/performance/benchmark_spsc

.gitignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,3 +51,11 @@ src/htmldocs/
5151
# Internal roadmap
5252
ROADMAP_v1.0.md
5353
claude.md
54+
55+
# Claude Code context (local only)
56+
CLAUDE.md
57+
58+
# Development/internal files
59+
STATUS_INTERNAL.md
60+
ROADMAP*.md
61+
SUPPORT.md

CHANGELOG.md

Lines changed: 50 additions & 102 deletions
Original file line numberDiff line numberDiff line change
@@ -1,116 +1,64 @@
11
# Changelog
22

3-
All notable changes to this project will be documented in this file.
4-
5-
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6-
7-
## [1.0.0] - 2025-10-31
8-
9-
Production-ready release with comprehensive stress testing and validated performance.
10-
11-
### Added
12-
- Lock-free SPSC channels with backpressure support
13-
- Structured concurrency with TaskGroup
14-
- Work-stealing scheduler with NUMA awareness
15-
- Connection pool with starvation detection and auto-recovery
16-
- WebSocket stress testing (1,000+ clients, 1M+ messages)
17-
- Streaming pipeline with adaptive backpressure
18-
- Comprehensive fault tolerance and error isolation
19-
- 24-hour endurance testing suite
20-
- Integrated observability with metrics and tracing
21-
- Full CI/CD pipeline with automated validation
3+
## [0.2.1] - 2025-11-01
224

235
### Fixed
24-
- Stack overflow in mixed workload scenarios
25-
- Race condition in memory allocation tracking
26-
- Channel buffer overflow handling
27-
- Database connection leak prevention
28-
29-
### Performance
30-
- **SPSC Throughput**: 213M+ ops/sec
31-
- **Task Spawn**: < 100ns overhead
32-
- **Mixed Workload**: 8,400 tasks/sec sustained
33-
- **GC Pauses**: < 2ms at 1GB allocation pressure
34-
- **Memory Stability**: Zero leaks after 24h continuous operation
35-
36-
### Testing
37-
- Comprehensive unit test coverage
38-
- Integration testing across all modules
39-
- Performance benchmarking and regression testing
40-
- Stress testing under extreme conditions
41-
- 24-hour endurance validation
42-
43-
## [0.1.0] - 2025-10-31
44-
45-
Production-ready lock-free SPSC channels with industry-leading performance.
6+
- **Async channels now actually work** - Previous implementation was completely broken
7+
- Fixed `send()`/`recv()` to use actual SPSC implementation (was referencing non-existent fields)
8+
- Removed duplicate Channel/ChannelMode type definitions
9+
- Added exponential backoff (1ms → 100ms) to reduce CPU usage
10+
- Fixed Chronos deprecation warnings
11+
- **Deduplicated channels.nim** - Removed 100+ lines of duplicate code
12+
- Now properly imports from channel_spsc instead of reimplementing everything
4613

4714
### Added
15+
- Comprehensive async channel tests (tests/unit/test_async_channel.nim)
16+
- Added async tests to CI workflow
4817

49-
- **Lock-Free SPSC Channels** - 213M+ ops/sec single-threaded throughput
50-
- **ORC Memory Management** - Zero GC pressure in hot paths
51-
- **Atomic Operations** - Thread-safe with proper memory barriers
52-
- **Production API** - `newChannel`, `trySend`, `tryReceive`
53-
- **Performance Benchmarks** - Validated against 52M ops/sec target (410% achieved)
54-
- **Unit Tests** - Basic functionality validation
55-
- **CI/CD Pipeline** - Automated testing on GitHub Actions
56-
- **Documentation** - Performance-focused README and architecture guide
57-
58-
### Performance Achievements
18+
### Changed
19+
- Updated KNOWN_ISSUES.md to reflect async improvements
5920

60-
- **213,567,459 ops/sec** - 4.1x above 52M target
61-
- **<1KB memory per channel** - Efficient resource usage
62-
- **Sub-microsecond latency** - For uncontended operations
63-
- **ORC-safe** - Advanced Nim garbage collection
64-
- **Lock-free** - No locks, no blocking, no deadlocks
21+
## [0.2.0] - 2025-11-01
6522

66-
### Technical Details
23+
### Reality Check Release
24+
Downgraded from v1.0.0 to v0.2.0 to accurately reflect what's production-ready.
6725

68-
- **Algorithm**: Atomic sequence numbers with acquire/release semantics
69-
- **Memory Layout**: Cache-aligned SPSCSlot structures
70-
- **Threading Model**: Single producer, single consumer
71-
- **Safety**: ORC GC with zero allocations in hot path
72-
- **Compatibility**: Nim 2.0+ with ORC memory management
26+
### Production-Ready
27+
- **SPSC Channels**: 212M+ ops/sec verified
28+
- Tests passing: tests/unit/test_channel.nim
29+
- Benchmark verified: tests/performance/benchmark_spsc.nim
7330

74-
### Breaking Changes
31+
### Removed from Public API
32+
- TaskGroup (has bugs, removed from exports)
33+
- Cancellation (untested, removed from exports)
34+
- All other features remain internal/experimental
7535

76-
- Initial release - no breaking changes
77-
78-
### Known Limitations
79-
80-
- Select operations not yet implemented (planned for v0.2.0)
81-
- MPMC channels not yet implemented (planned for v0.2.0)
82-
- Structured concurrency not yet implemented (planned for v0.3.0)
83-
84-
- SPSC Channels: 50M+ ops/sec (world-leading performance)
85-
- MPMC Channels: 10M+ ops/sec (optimized for concurrency)
86-
- Task Groups: 500K+ tasks/sec spawn rate
87-
- Cancellation: 100K+ ops/sec with <10ns latency
88-
- Memory Efficiency: <1KB per channel overhead
89-
90-
### Infrastructure
91-
92-
- Complete test suite with performance, stress, and regression tests
93-
- Professional benchmarking framework
94-
- Comprehensive metrics and logging
95-
- Production-ready validation tools
96-
- Multi-platform performance validation
97-
98-
## [0.2.0] - 2025-10-28
99-
100-
Production-ready release with 6 advanced modules for enterprise-grade async programming.
36+
### Changed
37+
- Version: 1.0.0 → 0.2.0
38+
- README.md: Replaced with honest version
39+
- Build: "production" → "experimental"
10140

10241
### Added
103-
104-
- **Adaptive Work-Stealing Scheduler** - Modern task distribution with load balancing
105-
- **NUMA-Aware Optimizations** - Multi-socket performance improvements
106-
- **OpenTelemetry Distributed Tracing** - Production observability
107-
- **Adaptive Backpressure Flow Control** - Smart rate limiting
108-
- **Erlang-Style Supervision Trees** - Fault tolerance and recovery
109-
- **Real-Time Performance Metrics** - Monitoring and analytics
110-
111-
### Infrastructure
112-
113-
- Multi-platform CI/CD (Ubuntu, macOS, Windows)
114-
- Complete test coverage and benchmarks
115-
- Community contribution templates
116-
- Professional GitHub setup
42+
- STATUS.md: Feature audit
43+
- CI workflow: Test + benchmark validation
44+
- CHANGELOG.md: This file
45+
- KNOWN_ISSUES.md: Documented bugs and limitations
46+
47+
### Known Issues
48+
- Async send/recv use polling (exponential backoff)
49+
- TaskGroup: nested async macro bugs (not exported)
50+
- Actors: won't compile, needs MPSC
51+
- Streams: compiles but untested
52+
- Scheduler: fake (just metrics)
53+
- NUMA: broken node detection
54+
55+
See KNOWN_ISSUES.md for complete list.
56+
57+
## [1.0.0] - RETRACTED
58+
Premature release. Use v0.2.0 instead.
59+
60+
Retracted because:
61+
- Claimed features that don't work
62+
- TaskGroup exported but broken
63+
- Examples showed inaccessible API
64+
- No CI validation

KNOWN_ISSUES.md

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# Known Issues
2+
3+
This document tracks known bugs and limitations in nimsync v0.2.1.
4+
5+
## Production Code (SPSC Channels)
6+
7+
### No close() Method
8+
**Status**: By design (not a bug)
9+
**Workaround**: Use sentinel values for shutdown signaling
10+
11+
### Async send/recv Use Polling (IMPROVED)
12+
**Status**: IMPROVED - Now uses exponential backoff (1ms → 100ms)
13+
**Previous Issue**: Used fixed 1ms polling
14+
**Current Implementation**: Exponential backoff (1ms, 2ms, 4ms, ..., up to 100ms cap)
15+
**Impact**: Initial operations have 1ms latency, but reduces CPU usage for longer waits
16+
**Workaround**: Use `trySend`/`tryReceive` for zero-latency operations
17+
**Future**: Could be replaced with event-driven Chronos futures
18+
19+
### Size Rounded to Power of 2
20+
**Status**: By design (performance optimization)
21+
**Example**: `newChannel[int](10, SPSC)` creates 16-slot channel
22+
23+
## Experimental Code (NOT Exported)
24+
25+
### TaskGroup - Nested Async Macro Bug
26+
**Status**: BROKEN - Does not compile
27+
**Location**: `src/nimsync/group.nim:172`
28+
**Error**: `expression has no type (or is ambiguous)`
29+
**Cause**: Nested async macros in `spawn()` implementation
30+
**Impact**: TaskGroup cannot be used at all
31+
**Priority**: HIGH - Blocks v0.3.0 release
32+
33+
### Actors - Won't Compile
34+
**Status**: BROKEN - Depends on unimplemented MPSC
35+
**Location**: `src/nimsync/actors.nim:258`
36+
**Error**: Type mismatch in mailbox.receive()
37+
**Cause**: Uses MPSC channels which don't exist
38+
**Dependencies**: Requires MPSC implementation first
39+
**Priority**: MEDIUM - v0.4.0 feature
40+
41+
### Streams - Completely Untested
42+
**Status**: UNKNOWN - Compiles but zero tests
43+
**Location**: `src/nimsync/streams.nim`
44+
**Issues**:
45+
- Line 334: "Unbounded policy not implemented" raises exception
46+
- Zero test coverage
47+
**Priority**: MEDIUM - Needs testing before v0.5.0
48+
49+
### Scheduler - Not Actually a Scheduler
50+
**Status**: FAKE - Just metrics tracking
51+
**Location**: `src/nimsync/scheduler.nim`
52+
**Missing**:
53+
- No work queues
54+
- No worker threads
55+
- No task stealing logic
56+
**Priority**: LOW - Rewrite or remove
57+
58+
### NUMA - Broken Node Detection
59+
**Status**: BROKEN - Always returns 0
60+
**Location**: `src/nimsync/numa.nim:122-136`
61+
**Issue**: `sched_getcpu()` call commented out
62+
**Impact**: All NUMA "optimizations" disabled
63+
**Priority**: LOW - Fix or remove
64+
65+
### Cancellation - Untested in Production
66+
**Status**: UNKNOWN - Code looks good, unproven
67+
**Location**: `src/nimsync/cancel.nim`
68+
**Issue**: No real-world testing
69+
**Priority**: MEDIUM - Needs validation before export
70+
71+
### Supervision - Disconnected
72+
**Status**: INCOMPLETE - Not integrated with actors
73+
**Location**: `src/nimsync/supervision.nim`
74+
**Missing**: Actor integration, restart logic, tests
75+
**Priority**: LOW - Depends on working actor system
76+
77+
## Not Bugs (By Design)
78+
79+
These are intentional design decisions, not bugs:
80+
81+
1. **SPSC Only**: Only Single Producer Single Consumer implemented
82+
- MPSC/SPMC/MPMC not available (will raise ValueError)
83+
- This is documented and intentional for v0.2.0
84+
85+
2. **Internal Code Not Exported**: Experimental features not in public API
86+
- TaskGroup, Actors, Streams, etc. exist but not exported
87+
- Users can import directly: `import nimsync/group`
88+
- This is intentional until features are production-ready
89+
90+
3. **Version 0.2.0**: Not 1.0.0
91+
- Downgraded from premature 1.0.0 release
92+
- Honest versioning reflecting actual state
93+
94+
## Reporting Issues
95+
96+
Found a bug not listed here? Please report it:
97+
98+
**For SPSC Channels** (production code):
99+
- Check if it's already in "Production Code" section above
100+
- If new, open issue: https://github.com/codenimja/nimsync/issues
101+
- Include: Nim version, OS, minimal reproduction
102+
103+
**For Experimental Features** (internal code):
104+
- These are known to be incomplete/broken
105+
- Check "Experimental Code" section first
106+
- Feature requests welcome, but expectations should be low
107+
108+
## Version History
109+
110+
- **v0.2.1** (2025-11-01): Fixed async channels, added exponential backoff
111+
- **v0.2.0** (2025-11-01): Reality check release, SPSC only
112+
- **v1.0.0** (RETRACTED): Premature, claimed features that didn't work
113+
114+
## Contributing
115+
116+
Want to fix any of these? Contributions welcome!
117+
118+
Priority order for fixes:
119+
1. TaskGroup nested async bug (blocking v0.3.0)
120+
2. MPSC channel implementation (enables actors)
121+
3. Stream testing (validation needed)
122+
4. Scheduler/NUMA (rewrite or remove)

0 commit comments

Comments
 (0)