docs: Update the project-structure dev doc to cover all current components; Add missing README for the Rust components.

[LinZhihao-723]

Description

CLP's [components] directory has accumulated a number of Rust components, but most of them ship no README, and design-project-structure.md still lists only the original set. As a result, someone browsing the repository has no way to tell what clp-rust-utils, clp-tdl-package, or compression-coordinator are for, and the dev docs give an incomplete picture of how the project is laid out.

This PR fills in both gaps:

• Adds a README to each of the components that lacked one:
  • api-server: describes the server and links to the pre-generated [OpenAPI] specification for the full endpoint reference.
  • clp-rust-utils: identifies the crate as the shared utilities used by CLP's other Rust components.
  • clp-tdl-package: describes the crate as CLP's Spider TDL package, and lists the tasks it currently bundles (compression::clp_s_s3_compress and compression::commit).
  • compression-coordinator: describes its role in fetching compression jobs, submitting them to Spider, and monitoring their progress.
• Updates docs/src/dev-docs/design-project-structure.md to list the components that were missing (api-server, clp-rust-utils, clp-tdl-package, compression-coordinator, and log-ingestor), and clarifies a few of the existing entries (e.g. clp-py-utils holds utilities shared by the Python components, and job-orchestration schedules both compression and search jobs).
• Removes a stray blank line from the log-ingestor README.

The new READMEs are intentionally brief: each states what the component is and what it's responsible for, without enumerating implementation details to follow the convention of the existing READMEs.

Checklist

• The PR satisfies the contribution guidelines.
• This is a breaking change and that has been indicated in the PR title, OR this isn't a
  breaking change.
• Necessary docs have been updated, OR no docs need to be updated.

Validation performed

• Ensure all workflows pass.

Summary by CodeRabbit

• Documentation
  • Added documentation for the API server, Rust utilities, TDL package, and compression coordinator.
  • Documented supported compression tasks, Spider compatibility, job processing, and API references.
  • Updated project structure documentation with component descriptions and repository links.
  • Improved formatting in the log ingestor README.

[coderabbitai]

Walkthrough

Added README documentation for several CLP components and updated the project structure documentation with component descriptions and repository links.

Changes

Documentation alignment

Layer / File(s)	Summary
Component README additions components/api-server/README.md, components/clp-rust-utils/README.md, components/clp-tdl-package/README.md, components/compression-coordinator/README.md	Added descriptions for the API Server, Rust Utilities, CLP TDL package, and Compression Coordinator.
Project structure index docs/src/dev-docs/design-project-structure.md	Added component entries and repository links. Updated descriptions for job orchestration, the MCP server, and shared Python utilities.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Suggested reviewers: jackluo923

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main documentation updates and the addition of missing READMEs for Rust components.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@components/api-server/README.md`:
- Line 8: Update the Markdown link to the static API specification so its
visible text describes the target instead of using “here,” while preserving the
existing static-api-reference destination.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 5e9a87a6-d8b5-4272-8f00-55280717ebbb

📥 Commits

Reviewing files that changed from the base of the PR and between 155fbda and cfd675c.

📒 Files selected for processing (6)

• components/api-server/README.md
• components/clp-rust-utils/README.md
• components/clp-tdl-package/README.md
• components/compression-coordinator/README.md
• components/log-ingestor/README.md
• docs/src/dev-docs/design-project-structure.md

💤 Files with no reviewable changes (1)

• components/log-ingestor/README.md

[coderabbitai]

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Use descriptive link text for the generated specification.

Replace here with a label that identifies the target. This improves accessibility and link navigation.

Proposed change

- specification can be found [here][static-api-reference].
+ specification can be found in the [generated API specification][static-api-reference].

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	specification can be found [here][static-api-reference].
	specification can be found in the [generated API specification][static-api-reference].

🧰 Tools

🪛 markdownlint-cli2 (0.23.1)

[warning] 8-8: Link text should be descriptive

(MD059, descriptive-link-text)

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@components/api-server/README.md` at line 8, Update the Markdown link to the
static API specification so its visible text describes the target instead of
using “here,” while preserving the existing static-api-reference destination.

Source: Linters/SAST tools