Skip to content

borodark/power_of_three

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

93 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Power of Three

What is Power of Three

Power of Three is the Elixir library that provides macros to define a cube, dimensions and measures along side with Ecto.Schema.

This defenitions are complied to cubes config files on mix compile.

The yaml output only for now.

The cubes config files then can be be shared with the running Cube.

Please see separate project for examples showing working features.

What is Cube[.dev]

Solution for data analytics:

How to use cube:

  • Define cubes as collections of measures aggregated along dimensions: DSL, yaml or JS.
  • Decide how to refresh cube data
  • Profit!

TODO:

The future plans are bellow in the order of priority:

  • hex.pm documentation
  • because the cube can impersonate postgres generate an Ecto.Schema Module for the Cubes defined (full loop): columns are measures and dimensions connecting to the separate Repo where Cube is deployed.
  • because the cube can impersonate postgres generate a Module implementing Table.Reader protocol for Explorer.DataFrame connecting to the separate Repo where Cube is deployed.
  • support @schema_prefix
  • validate on pathtrough all options for the cube, dimensions, measures and pre-aggregations
  • handle sql_table names colisions with keywords
  • validate use of already defined cube members in definitions of other measures and dimensions
  • handle dimension's case
  • CI integration: what to do with generated yams: commit to tree? push to S3? when in CI?
  • CI integration: validate yams by starting a cube and make sure configs are sound.
  • generate default dimensions, measures for all columns of the table if cube() macro is used without anything else declared to mimic the capability of cube dev environment

NOT TODO

Handle of cube's sql will not be done. Only sql_table. If you find yourself thinking adding support for sql, please fork and let the force be with you.

Why inline in Ecto Schema modules?

The names of tables and columns used in definitions of measures and dimensions are verifiable to be present in Ecto.Schema, hence why write/maintain another yaml or even worse json?

DEV environment

For crafting Cubes here is the docker: compose.yaml

Deployment Overview

Four types of containers:

  • API
  • Refresh Workers
  • Cubestore Router
  • Cubestore Workers

Logical deployment

Two need the DB connection: API and Refresh Workers.

Router needs shared storage with Store Workers: S3 is recommended.

Installation

To install the Cube Core and run locally see here:

To use library

Available in Hex, the package can be installed by adding power_of_3 to your list of dependencies in mix.exs:

def deps do
  [
    {:power_of_3, "~> 0.1.2"}
  ]
end

About

Elixir macros for Cube.dev DSL

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages