Add System architecture and Security Architecture documentations [skip ci] (#3794)

### Description
1. Add system architecture
2. Add cellnet architecture
3. Add Security Architecture

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [ ] New tests added to cover the changes.
- [ ] Quick tests passed locally by running `./runtest.sh`.
- [ ] In-line docstrings updated.
- [ ] Documentation updated.

Author: chesterxgchen

docs/cellnet_architecture.rst

@@ -0,0 +1,215 @@
+.. _cellnet_architecture:
+
+FLARE CellNet Architecture
+--------------------------
+
+.. image:: resources/cellnet.png
+   :alt: FLARE CellNet Architecture
+
+
+Purpose and Scope
+#################
+
+CellNet is FLARE's unified communication layer that provides secure, scalable messaging between distributed federated
+learning components. It abstracts away network transport details and provides a consistent API for both small messages and
+large data transfers.
+
+Position in NVFLARE Architecture: CellNet sits between the application layer (Controllers, Executors, Admin commands) and
+the network transport layer (gRPC, TCP, HTTP drivers). All NVFLARE components communicate through CellNet, including:
+
+- **Server-to-client task distribution**
+- **Client-to-server result submission**
+- **peer-to-peer communication**
+- **Admin command execution**
+- **Cross-site auxiliary communication**
+- **Job deployment and management**
+
+
+Key Design Goals:
+#################
+
+- **Unified API**: Single interface for both small messages and large data streams
+- **Transport Agnostic**: Supports multiple network protocols (gRPC, TCP, HTTP)
+- **Hierarchical Addressing**: FQCN-based routing for multi-level cell hierarchies
+- **Secure Communication**: Built-in encryption and authentication
+- **Flow Control**: Automatic chunking and flow control for large transfers
+
+Three-Layer Architecture:
+#########################
+
+- **CoreCell**: Basic message routing, connection management, security
+- **StreamCell**: Large data streaming with chunking and flow control
+- **Cell**: High-level request/reply patterns with automatic channel detection
+
+Layered Cell Architecture
+#########################
+
+
+Three-Layer Design
+^^^^^^^^^^^^^^^^^^
+
+The CellNet architecture consists of three layers, each extending the previous:
+
+Layer 1: **CoreCell** - Basic Message Infrastructure
+provides fundamental messaging infrastructure:
+
+**Key Responsibilities**:
+
+- **Message Handling** - Routes messages to appropriate handlers based on channel/topic
+- **Connection Management** - Manages listeners (incoming) and connectors (outgoing)
+- **Callback Registry** - Stores message handlers in req_reg: Registry
+- **Agent Tracking** - Maintains agents: Dict[str, CellAgent] for remote cells
+- **Request Tracking** - Tracks pending requests in waiters: Dict[str, _Waiter]
+- **Security** - Delegates to credential_manager: CredentialManager for encryption
+
+**Core Methods**:
+
+- **send_request**(channel, target, topic, request, timeout, ...) - Send message and wait for reply
+- **fire_and_forget**(channel, topic, targets, message, ...) - Send without waiting
+- **broadcast_request**(channel, topic, targets, request, ...) - Send to multiple targets
+- **register_request_cb**(channel, topic, cb, ...) - Register callback for channel/topic
+
+Layer 2: **StreamCell** - Large Data Transfer
+
+The StreamCell adds large data transfer capabilities on top of CoreCell:
+
+**Key Components**:
+
+- **cell**: CoreCell - Wrapped CoreCell for basic messaging
+- **byte_streamer**: ByteStreamer - Sends data as chunked streams
+- **byte_receiver**: ByteReceiver - Receives and reassembles chunks
+- **blob_streamer**: BlobStreamer - Optimized for in-memory BLOBs
+
+**Streaming Methods**:
+
+- **send_stream** (channel, topic, target, message, ...) - Send byte stream with flow control
+- **send_blob** (channel, topic, target, message, ...) - Send BLOB (fits in memory)
+- **register_stream_cb** (channel, topic, stream_cb, ...) - Register stream receiver
+- **register_blob_cb** (channel, topic, blob_cb, ...) - Register BLOB receiver
+
+**Streaming Protocol**:
+
+- Automatic chunking into configurable chunk sizes (default 1MB)
+- Flow control with sliding window and ACKs
+- Progress tracking via StreamFuture
+
+Layer 3: **Cell** - Intelligent Request/Reply
+
+The **Cell** class provides unified interface for streaming and non-streaming messages:
+
+**Key Features**:
+
+1. **Dynamic Method Dispatch**:
+
+- Intercept method calls Checks if channel requires streaming via _is_stream_channel()
+- Routes to appropriate implementation:
+- Stream channels → _broadcast_request(), _send_request(), etc.
+- Non-stream channels → core_cell.broadcast_request(), etc.
+
+2. **Channel Classification**:
+
+**Excluded Channels**:
+
+   - CellChannel.CLIENT_MAIN - Admin commands
+   - CellChannel.SERVER_MAIN** - Task distribution
+   - CellChannel.RETURN_ONLY** - Internal replies
+   - CellChannel.CLIENT_COMMAND** - Client commands
+   - Other internal channels
+
+3. **Request Tracking**:
+
+- Maintains requests_dict: Dict[str, SimpleWaiter] for pending requests
+- SimpleWaiter tracks request state and receiving progress
+- Reply handling via _process_reply()
+
+4. **Callback Adaptation**:
+
+- Adapter class wraps application callbacks for streaming
+- Handles encoding/decoding of stream payloads
+- Sends replies back via RETURN_ONLY channel
+
+5. **FQCN: Fully Qualified Cell Name**:
+
+Every cell is identified by a Fully Qualified Cell Name (FQCN), which is a dot-separated hierarchical name:
+
+<site_name>[.<job_id>[.<rank>]]
+
+6. **End-to-end encryption**
+
+Message Structure and Addressing
+###############################
+
+Channel and Topic Addressing
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+F3 CellNet routes messages using a two-level addressing scheme: channel and topic.
+This is stored in message headers:
+
+Title: Channel/Topic Routing in CoreCell
+
+
+Predefined Channels (CellChannel constants):
+
+.. list-table:: **Predefined Channels**
+   :header-rows: 1
+   :widths: 35 25 40
+
+   * - Constant
+     - Value
+     - Purpose
+   * - CellChannel.CLIENT_MAIN
+     - "admin"
+     - Admin commands
+   * - CellChannel.SERVER_MAIN
+     - "task"
+     - Task distribution
+   * - CellChannel.AUX_COMMUNICATION
+     - "aux_communication"
+     - Application-defined
+   * - CellChannel.RETURN_ONLY
+     - "return_only"
+     - Internal reply routing
+   * - CellChannel.SERVER_COMMAND
+     - "server_command"
+     - Server commands
+
+
+Communication Patterns
+^^^^^^^^^^^^^^^^^^^^^^
+Request-Reply Pattern -- send request and wait for reply
+Fire-and-Forget Pattern -- send message without waiting for reply
+Broadcast Pattern -- send to multiple targets
+
+Streaming Components Overview
+#############################
+
+
+The streaming system is organized into sender components, receiver components,and stream abstractions:
+
+Key Streaming Classes:
+
+.. list-table:: **Key Streaming Classes**
+   :header-rows: 1
+   :widths: 25 40
+
+   * - Class
+     - Purpose
+   * - ByteStreamer
+     - Sends byte streams as chunks
+   * - ByteReceiver
+     - Receives and reassembles chunks
+   * - BlobStreamer
+     - Wraps blobs for streaming
+   * - TxTask
+     - Per-stream sending task
+   * - RxTask
+     - Per-stream receiving task
+
+
+Performance and Statistics
+##########################
+Statistics Collection
+CellNet includes comprehensive statistics collection for monitoring and debugging:
+Statistics are collected via StatsPoolManager with categories for different operation types and cell FQCNs.
+
+

docs/conf.py

@@ -44,12 +44,12 @@ def resolve_xref(self, env, fromdocname, builder, typ, target, node, contnode):
 # -- Project information -----------------------------------------------------
 
 project = "NVIDIA FLARE"
-copyright = "2024, NVIDIA"
+copyright = "2025 NVIDIA"
 author = "NVIDIA"
 
 # The full version, including alpha/beta/rc tags
-release = "2.4.0"
-version = "2.4.0"
+release = "2.7.0"
+version = "2.7.0"
 
 readthedocs_version_name = os.environ.get("READTHEDOCS_VERSION_NAME")
 build_version = readthedocs_version_name if readthedocs_version_name not in (None, "latest", "stable") else "main"

docs/flare_overview.rst

@@ -112,6 +112,11 @@ Adjacent to this central stack are tools facilitating experimentation and simula
 .. image:: resources/flare_overview.png
     :height: 500px
 
+For detailed information on the architecture, please refer to the :ref:`flare_system_architecture` section.
+For detailed information on the security overview, please refer to the :ref:`flare_security_overview` section.
+
+
+
 Design Principles
 =================