-
Notifications
You must be signed in to change notification settings - Fork 159
Feature/spline 815 export import #1436
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: release/0.8
Are you sure you want to change the base?
Conversation
…o attributes and expressions, but without them as separate components yet).
…tion-plans/{id}` request.
… request (returns plan IDs).
…vents` request (returns evens list).
⛔ Snyk checks have failed. 6 issues have been found so far.
⛔ security/snyk check is complete. 6 issues have been found. (View Details) 💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements import/export CLI commands and extends the Producer API with new endpoints for execution plan retrieval and events.
- Adds
lineage-import
andlineage-export
commands in the Admin CLI and integration tests. - Introduces repository methods (
isDatabaseOk
,fetchExecutionPlanIds
,fetchExecutionPlan
,fetchExecutionEvents
) and updates theExecutionProducerRepository
interface. - Exposes new REST endpoints (
GET /execution-plans
,GET /execution-plans/{id}
,GET /execution-plans/{id}/events
) inExecutionPlansController
.
Reviewed Changes
Copilot reviewed 130 out of 130 changed files in this pull request and generated 1 comment.
Show a summary per file
File | Description |
---|---|
ExecutionProducerRepositoryImpl.scala | Added database health check and new fetch methods with AQL. |
ExecutionProducerRepository.scala | Extended interface for new repository methods. |
ExecutionPlansController.scala | Added GET endpoints and updated POST mapping for execution plans. |
ExecutionPlanApiModelAssembler.scala | New assembler for converting persistent models to API models. |
integration-tests/AdminCLISpec.scala | Integration tests for lineage-import and lineage-export. |
Comments suppressed due to low confidence (4)
producer-rest-core/src/main/scala/za/co/absa/spline/producer/rest/controller/ExecutionPlansController.scala:194
- [nitpick] Method name includes uppercase 'IDs', which is inconsistent with Scala camelCase conventions. Consider renaming to
getExecutionPlanIds
.
def getExecutionPlanIDs: Future[Seq[UUID]] = {
producer-services/src/main/scala/za/co/absa/spline/producer/service/repo/ExecutionProducerRepositoryImpl.scala:140
- Consider adding unit tests for
fetchExecutionPlanIds
to verify correct behavior when the database returns various result sets or errors.
override def fetchExecutionPlanIds()(implicit ec: ExecutionContext): Future[Seq[UUID]] = {
producer-services/src/main/scala/za/co/absa/spline/producer/service/repo/ExecutionProducerRepositoryImpl.scala:56
- NonFatal is not imported in this file, leading to a compilation error. Please add
import scala.util.control.NonFatal
.
case NonFatal(_) => Future.successful(false)
producer-services/src/main/scala/za/co/absa/spline/producer/service/repo/ExecutionProducerRepositoryImpl.scala:141
- [nitpick] This query loads all execution plan IDs into memory at once. For large datasets, consider adding pagination or streaming support to limit memory usage.
db.queryAs[apiModel.ExecutionPlan.Id](
...src/main/scala/za/co/absa/spline/producer/service/model/ExecutionPlanApiModelAssembler.scala
Outdated
Show resolved
Hide resolved
…rvice/model/ExecutionPlanApiModelAssembler.scala Co-authored-by: Copilot <[email protected]>
|
Fixes #815
Adds CLI commands:
lineage-import
lineage-export
Adds Producer REST API endpoitns:
GET /execution-plans
-- returns execution plan IDsGET /execution-plans/{planID}
-- returns the givenExecutionPlan
(Producer API v1.1)GET /execution-plans/{planID}/events
-- returns a list ofEvecutionEvent
s of the givenExecutionPlan