Skip to content
This repository was archived by the owner on Dec 31, 2020. It is now read-only.

Refactor to reduce dependencies #13

@EronWright

Description

@EronWright

The library today has numerous layers of functionality that could benefit from refactoring, to reduce dependencies and clarify concepts. The conceptual layering (from bottom up):

  1. Scala language bindings
    a. focus on using the TensorFlow library in Scala (not Flink specific)
  2. Core Flink integration code
    a. graph/model loaders
    b. tensors in the Flink type system (e.g. TensorValue, type tags)
    c. core lifecycle classes (e.g. ModelAwareFunction)
    d. (future) integration of graph state with Flink managed state
  3. ML specializations
    a. representations of specific ML signatures ('classify', 'regress', 'predict')
    b. TF Serving interoperability
    c. FlinkML integration
    d. converters for specific datatypes (e.g. LabeledVector, Example)

Ideally the core module (2) would shed its dependency on 'twitter-bijection', though it may still be useful in (3).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions