Skip to content

Metaframe builder#440

Draft
qubixes wants to merge 29 commits into
developfrom
metaframe-builder
Draft

Metaframe builder#440
qubixes wants to merge 29 commits into
developfrom
metaframe-builder

Conversation

@qubixes

@qubixes qubixes commented Jun 9, 2026

Copy link
Copy Markdown
Member

This PR will be the main addition to metasyn 3.0. It is very far from ready. Given that the features are interlinked, this PR will unfortunately be quite big.

Planned features for this PR:

  • Column relations, so that columns can be related to each other.
  • Sorted columns
  • Rework of the MetaFrame.fit_dataframe() method. This method would be growing too much and would have too much logic. MetaFrame.fit_dataframe() might still be offered for compatibility, but will lack some of the new features.
    -Removal of the MetaConfig, VarSpec and DistSpec classes. The functionality of these classes will be transferred to new classes.
  • New VarBuilder and MetaFrameBuilder classes. These classes take the place of the MetaFrame.fit_dataframe() and MetaVar.fit() methods.
  • Composed distributions, such as ContinuousNormalDistribution(0, 1) + ContinuousUniformDistribution(0, 1). Fitters for these composed distributions will not be implemented. Depending on privacy considerations, fitters for a subset of the possible composed distributions might be implemented.
metasyn_overview_new drawio

TODO list:

  • Figure out dtype/var_type intricacies: Because of the new structure, we might not know dtypes before synthesis.
  • Figure out prop_missing situation. Leaning towards making prop_missing part of the distribution finally. This should probably be achieved through the composition of two distributions technically, but let's see how we can make it user friendly
  • Connect builder to the Metaframe.fit_dataframe() method for backward compatibility.
  • Make the new composition distributions serializable for GMF file
  • Add more Arithmetic operations (negation, division, subtraction).
  • Add more logical conditions (not equal, greater than, smaller than)
  • Add interface for col_a > col_b, but with fitting the difference. This will require more work in the VarBuilder, where we might have both a distribution and a fitter supplied.
  • Fix .toml configuration files to work with the builder, with MetaConfig probably having to go.
  • Write/update documentation
  • Update multiframe feature to work with the new structure.
  • Remove dead code
  • Write/update tests
  • Update README
  • Write new tutorials
  • Update CLI
  • Figure out if we can split off some of the operator type distributions in a separate file.
  • Add docstrings and comments to all new functionality

Fixes #434

qubixes added 3 commits June 1, 2026 15:20
The builder is now more or less working as expected, but features are
both missing and broken, which is expected.
@qubixes
qubixes marked this pull request as draft June 9, 2026 09:10
qubixes and others added 26 commits June 10, 2026 20:02
Now more work is needed for the GMF file and the toml input file.
Now you can set the series to be referring to columns. After that you
can fit those new series. This can be used to solve a = b + <random_var>
* Add more tests for the builder

* Add more tests

* Fix test
Fix GMF loading and saving with composed distributions
New operator similar to pd.is_null() or pl.is_null().
Ensure that computation with null is always null
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add some simple relations/constraints between columns

2 participants