docs: Document the Spider-based compression orchestration in the deployment orchestration dev doc.

[LinZhihao-723]

Description

This PR depends on #2441.

The deployment orchestration dev doc still describes CLP as a purely Celery-based deployment. It doesn't mention Spider or compression-coordinator, and its "Deployment types" section only covers the query engine, so there's no documented picture of how compression jobs are orchestrated once Spider is enabled.

This PR brings the doc up to date with the Spider-based compression path:

Architecture (Figure 1 and Table 1)

• Adds spider-service and compression-coordinator to the architecture diagram, under new Spider and Coordinators subgraphs.
• Adds compression-coordinator's dependencies:
  • It depends on db-table-creator completing successfully.
  • It depends on spider-service being healthy.
• Renumbers the affected linkStyle indices, and gives the new Spider dependency its own edge colour so it remains visually distinct from the existing edge categories.
• Adds spider_service and compression_coordinator rows to Table 1.

Deployment types

• Restructures the section from two deployment configurations into four, presented as a new Table 3. The configuration is now determined by two independent choices: the query engine (Celery or Presto), and whether Spider is enabled.
• Documents that enabling Spider adds compression-coordinator, which routes compression jobs to be driven by Spider. This is orthogonal to the query engine: queries continue to be driven by Celery or Presto according to the deployment type.
• Documents the two current constraints on Spider deployments:
  • The Celery-based compression-scheduler remains enabled by default, since compression-coordinator can currently only coordinate compression jobs created by log-ingestor. It will be deprecated in a future release once Spider fully replaces it.
  • Spider is only available for Helm deployments; Docker Compose deployments cannot enable it.

Checklist

• The PR satisfies the contribution guidelines.
• This is a breaking change and that has been indicated in the PR title, OR this isn't a
  breaking change.
• Necessary docs have been updated, OR no docs need to be updated.

Validation performed

• Ensure all workflows pass.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 91f772dc-6aa9-45d7-9f37-f1305a7981f9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}