Implement missing subscription testing and server management features

[milliondreams]

Summary

Low Priority: Implement the 3 features that are actually missing from the codebase and need full implementation.

Background

Analysis revealed that only 3 out of 18 "missing" features actually need implementation. The majority are already implemented but not exposed through YAML. See analysis: cursor_workspace/implemented-vs-missing-analysis.md

Features Actually Missing Implementation

❌ Subscription Testing (NOT IMPLEMENTED)

YAML Properties:

• subscription_expected: true
• update_interval_seconds: 1.0
• min_progress_updates: 3

Requirements:

• Real-time update validation
• Subscription lifecycle management
• Update interval verification
• Progress notification tracking

❌ Server Management (NOT IMPLEMENTED)

YAML Properties:

• restart_on_failure: true
• health_check_interval: 5
• max_restart_attempts: 3

Requirements:

• Automatic server restart on failure
• Health check monitoring
• Process lifecycle management
• Restart attempt limits

❌ Network Simulation (NOT IMPLEMENTED)

YAML Properties:

• network_failure_simulation: true
• stress_test_duration_seconds: 30

Requirements:

• Network failure injection
• Latency simulation
• Bandwidth throttling
• Connection reliability testing

Implementation Plan

1. Subscription Testing Implementation

Phase 1: Core Infrastructure

Files to Create/Modify:

• crates/mandrel-mcp-th/src/subscription/mod.rs (NEW)
• crates/mandrel-mcp-th/src/subscription/manager.rs (NEW)
• crates/mandrel-mcp-th/src/subscription/validator.rs (NEW)

Subscription Manager:

pub struct SubscriptionManager {
    active_subscriptions: HashMap<String, SubscriptionState>,
    update_tracker: UpdateTracker,
}

pub struct SubscriptionState {
    pub id: String,
    pub expected_interval: Duration,
    pub received_updates: Vec<UpdateEvent>,
    pub start_time: Instant,
}

impl SubscriptionManager {
    pub async fn start_subscription(&mut self, subscription_id: String, expected_interval: Duration) -> Result<()>;
    pub async fn handle_update(&mut self, subscription_id: String, update: Value) -> Result<()>;
    pub fn validate_subscription(&self, subscription_id: &str) -> SubscriptionValidationResult;
}

YAML Integration:

// Add to ExpectedOutput struct
pub struct ExpectedOutput {
    // ... existing fields
    pub subscription_expected: Option<bool>,
    pub update_interval_seconds: Option<f64>,
    pub min_progress_updates: Option<u32>,
}

2. Server Management Implementation

Phase 1: Process Management

Files to Create/Modify:

• crates/mandrel-mcp-th/src/server/mod.rs (ENHANCE)
• crates/mandrel-mcp-th/src/server/lifecycle.rs (NEW)
• crates/mandrel-mcp-th/src/server/health.rs (NEW)

Server Lifecycle Manager:

pub struct ServerLifecycleManager {
    config: ServerManagementConfig,
    health_monitor: HealthMonitor,
    restart_tracker: RestartTracker,
}

pub struct ServerManagementConfig {
    pub restart_on_failure: bool,
    pub health_check_interval: Duration,
    pub max_restart_attempts: u32,
}

impl ServerLifecycleManager {
    pub async fn monitor_server(&mut self, server_process: &mut ServerProcess) -> Result<()>;
    pub async fn health_check(&self, server: &ServerProcess) -> HealthStatus;
    pub async fn restart_server(&mut self, server: &mut ServerProcess) -> Result<()>;
}

YAML Integration:

// Add to ServerConfig struct
pub struct ServerConfig {
    // ... existing fields
    pub restart_on_failure: Option<bool>,
    pub health_check_interval: Option<u64>,
    pub max_restart_attempts: Option<u32>,
}

3. Network Simulation Implementation

Phase 1: Network Proxy

Files to Create/Modify:

• crates/mandrel-mcp-th/src/network/mod.rs (NEW)
• crates/mandrel-mcp-th/src/network/simulation.rs (NEW)
• crates/mandrel-mcp-th/src/network/proxy.rs (NEW)

Network Simulator:

pub struct NetworkSimulator {
    failure_config: FailureConfig,
    latency_config: LatencyConfig,
    active_simulations: Vec<SimulationRule>,
}

pub struct FailureConfig {
    pub enable_failures: bool,
    pub failure_rate: f64,      // 0.0 - 1.0
    pub failure_duration: Duration,
}

impl NetworkSimulator {
    pub async fn start_simulation(&mut self, config: NetworkSimulationConfig) -> Result<()>;
    pub async fn inject_failure(&self, connection: &mut Connection) -> Result<()>;
    pub async fn add_latency(&self, request: &mut Request, latency: Duration) -> Result<()>;
}

YAML Integration:

// Add to TestConfig struct
pub struct TestConfig {
    // ... existing fields
    pub network_failure_simulation: Option<bool>,
    pub stress_test_duration_seconds: Option<u64>,
}

Implementation Priority

Priority 1: Subscription Testing (MCP Core Feature)

• Justification: Subscriptions are a core MCP feature
• Effort: Medium (2-3 days)
• Impact: High for MCP server testing

Priority 2: Server Management (Quality of Life)

• Justification: Improves test reliability and debugging
• Effort: Medium (2-3 days)
• Impact: Medium for test stability

Priority 3: Network Simulation (Advanced Testing)

• Justification: Advanced fault tolerance testing
• Effort: High (4-5 days)
• Impact: Low for basic testing, high for production validation

Testing Strategy

Subscription Testing:

1. Create test MCP server with subscription support
2. Implement subscription validation tests
3. Test update interval verification
4. Test progress tracking

Server Management:

1. Test server restart scenarios
2. Validate health check monitoring
3. Test restart attempt limits
4. Test failure detection

Network Simulation:

1. Test network failure injection
2. Validate latency simulation
3. Test stress testing scenarios
4. Test failure recovery

Success Criteria

Subscription Testing:

• Can validate subscription lifecycle
• Update interval verification works
• Progress tracking functional
• Integration with existing test framework

Server Management:

• Automatic restart on failure works
• Health checks monitor server status
• Restart attempts properly limited
• Server lifecycle properly managed

Network Simulation:

• Network failures can be injected
• Latency simulation functional
• Stress testing duration controls work
• Network simulation integrates with tests

Impact

Completes the missing 17% of test harness functionality. Enables comprehensive MCP server testing including advanced scenarios like subscription handling, fault tolerance, and network resilience.

Dependencies

• Should be implemented after Expose implemented functionality through YAML parsing struct fields #347 and Wire up partially implemented features to YAML configuration #348 for complete functionality
• Requires MCP specification knowledge for subscription testing
• May require additional dependencies for network simulation

References

• Analysis: cursor_workspace/implemented-vs-missing-analysis.md
• MCP Subscription Spec: specification/
• Current limitations: cursor_workspace/missing-features-list.md

[milliondreams]

Status Update - Scope Significantly Reduced 🎯

Major Finding: Only 3 out of 18 "missing" features actually need implementation. The rest are already implemented but not exposed through YAML.

Analysis Results ✅

Based on comprehensive codebase review:

• 67% of features: Fully implemented, just needed YAML exposure (completed in Expose implemented functionality through YAML parsing struct fields #347/Wire up partially implemented features to YAML configuration #348)
• 17% of features: Partially implemented, minor wiring needed
• 17% of features: Actually missing and need full implementation

Actually Missing Features (Only 3) ❌

1. Subscription Testing - Priority 1 (Core MCP feature)

• Real-time update validation
• Subscription lifecycle management
• Update interval verification
• Effort: 2-3 days

2. Server Management - Priority 2 (Quality of life)

• Automatic server restart on failure
• Health check monitoring
• Process lifecycle management
• Effort: 2-3 days

3. Network Simulation - Priority 3 (Advanced testing)

• Network failure injection
• Latency simulation
• Connection reliability testing
• Effort: 4-5 days

Current Test Harness Status 📊

• 170+ tests running successfully
• Core functionality: ✅ Complete and working
• Script engines: ✅ Lua, JavaScript, Python support
• Performance monitoring: ✅ Comprehensive metrics system
• YAML configuration: ✅ Most features now exposed

Recommendation

Focus on Subscription Testing first as it's a core MCP feature. Server Management and Network Simulation are advanced features that can be future milestones.

Updated Priority: Low → Medium (only 3 features actually missing vs original 18)