Add delete route and required code to clean up a vectorize job.

[kawidman3]

Add DELETE endpoint for vectorize jobs

Summary

Adds a DELETE /api/v1/table/{job_name} endpoint to clean up all resources associated with a vectorize job, including database objects, PGMQ messages, and in-memory cache.

Changes

Core (core/)

• src/query.rs: Added SQL generation functions for cleanup operations:
  • drop_embeddings_table(), drop_search_tokens_table(), drop_project_view()
  • drop_trigger_handler(), drop_event_trigger(), drop_search_tokens_trigger()
  • delete_job_record()
• src/init.rs: Added cleanup_job() function to orchestrate resource cleanup:
  • Fetches job metadata
  • Deletes pending PGMQ messages
  • Executes all cleanup SQL statements in a transaction
  • Handles partial failures gracefully

Server (server/)

• src/routes/table.rs: Added delete_table() endpoint handler
  • DELETE /api/v1/table/{job_name} route
  • Returns 200 OK on success, 404 if job not found
  • Removes job from in-memory cache after cleanup
• src/server.rs: Registered the new DELETE route

Workers (worker/ & extension/)

• Added safety checks in both workers to handle messages for deleted jobs
• Returns Ok() instead of error to allow message deletion
• Prevents infinite retry loops and worker crashes

Resources Cleaned Up

• ✅ Embeddings table (vectorize._embeddings_{job_name}) and HNSW index
• ✅ Search tokens table (vectorize._search_tokens_{job_name}) and FTS index
• ✅ Project view (vectorize.{job_name}_view)
• ✅ Triggers on source table (INSERT, UPDATE, search tokens)
• ✅ Trigger handler function (vectorize.handle_update_{job_name})
• ✅ Job record from vectorize.job table
• ✅ Pending PGMQ messages
• ✅ In-memory job cache entry

Note: Source table and its data are preserved (not deleted).

Tests

Unit Tests (8 tests in core/src/query.rs)

• SQL generation for all cleanup operations
• Edge cases with special characters in job names

Integration Tests (5 tests in server/tests/tests.rs)

• test_delete_job - Full lifecycle verification
• test_delete_nonexistent_job - 404 error handling
• test_delete_job_idempotency - Double delete protection
• test_delete_job_preserves_source_table - Data integrity
• test_delete_job_with_pending_messages - PGMQ cleanup

Usage

# Delete a job
curl -X DELETE http://localhost:8080/api/v1/table/my_job_name

# Response (200 OK)
{
  "job_name": "my_job_name",
  "message": "Successfully deleted job 'my_job_name'"
}

# Job not found (404)
{
  "error": "Job 'nonexistent_job' not found"
}