ForcedLogout reconnection loop when connecting multiple plants simultaneously

[waynehoover]

Description

When client.connect() opens all 4 plants (ticker, order, history, pnl), Rithmic's server sometimes sends ForcedLogout (template 77) messages to earlier-connected plants, interpreting the rapid successive logins as exceeding the concurrent session limit. The library's auto-reconnect then creates new sessions, displacing other plants, causing a cascading reconnection loop that eventually stabilizes but produces significant noise and delays startup by 30-60+ seconds.

Environment

• async_rithmic 1.5.9
• Python 3.11
• macOS / Rithmic 01
• Connecting all 4 plants (default behavior)

Steps to Reproduce

1. Create a RithmicClient with default settings (all 4 plants)
2. Call await client.connect() with reconnection enabled (default)
3. Observe logs

Expected Behavior

All 4 plants connect cleanly within a few seconds. No ForcedLogout messages during initial connection.

Actual Behavior

plant.ticker - Connected
plant.order - Connected  
plant.history - Connected
plant.ticker - WARNING - Received a ForcedLogout message from Rithmic
plant.ticker - Reconnection attempt #1
plant.pnl - Connected
plant.order - WARNING - Received a ForcedLogout message from Rithmic
plant.order - Reconnection attempt #1
... (loop continues for 30-60s before stabilizing)

Root Cause Analysis

The connect() method connects plants sequentially with only asyncio.sleep(0.1) between them. When Rithmic's server receives 4 rapid login requests from the same user, it treats them as exceeding the concurrent session limit and sends ForcedLogout to earlier sessions.

Key issues:

1. The 100ms inter-plant delay is insufficient for Rithmic's session tracking
2. ForcedLogout triggers the same reconnection path as a network disconnect — no special handling
3. Each plant reconnects independently, competing for session slots (thundering herd)

Suggested Fixes

1. Increase inter-plant connection delay to 1-2 seconds (configurable via connection_delay parameter)
2. Coordinate reconnection across plants — when ForcedLogout is received during initial connect(), pause all reconnection, then retry sequentially
3. Suppress auto-reconnect during initial connect — raise an exception instead of silently reconnecting, since the caller hasn't finished setup yet
4. Add ForcedLogout-specific backoff — treat it differently from network disconnect, with longer delay before retry

Workaround

Currently working around this by accepting the noisy startup and waiting for the loop to stabilize (~30-60s). Connecting to fewer plants via plants= parameter reduces but doesn't fully eliminate the issue.

[rundef]

Hi, thanks for the detailed report.

I will make improvements to the connection/disconnection/ForcedLogout handling logic in a few days.

[rundef]

@waynehoover Can you try the fix/connection_issues branch ?

You can now pass plant_connect_delay=x to the connect method to override the 0.1 sleep default

Reconnection backoff with jitter algo is already part of the codebase (see here). However the first reconnection was happening instantly instead of taking this into consideration.