v2.3.3

What's Changed

• Fix project program allowed system props by @JoshuaHarris391 in #60

Full Changelog: v2.3.2...v2.3.3

v2.3.2

What's Changed

• Fix dont script program proj rule validate by @JoshuaHarris391 in #59

Full Changelog: v2.3.1...v2.3.2

v2.3.1

Full Changelog: v2.3.0...v2.3.1

• Fix added, rule validator now checks that data_type, data_category, and data_format are props in a data_file node

v2.3.0

What's Changed

• Add property type validation to RuleValidator by @JoshuaHarris391 in #52
• Fix data type required by @JoshuaHarris391 in #55
• Fix array should have items by @JoshuaHarris391 in #56
• Add core metadata required link validation in RuleValidator by @JoshuaHarris391 in #57

Full Changelog: v2.2.0...v2.3.0

v2.2.0

What's Changed

• Fix links must have label by @JoshuaHarris391 in #45
• Add 'label' to system properties in RuleValidator by @JoshuaHarris391 in #46
• Implement multiplicity flipping in link conversion by @JoshuaHarris391 in #47
• Add program template generation and update CLI logic by @JoshuaHarris391 in #48

Full Changelog: v2.1.1...v2.2.0

v2.1.1

What's Changed

• Fixed typo in lipidomics_file.yaml where link name core_metdata_colle… by @JoshuaHarris391 in #38

Full Changelog: v2.1.0...v2.1.1

v2.1.0

Release Notes - Version 2.1.0

New Features

• Added 'init' command to CLI for schema initialisation
• Added project schema generation functionality
• Added rule validation functionality and schema examples

Enhancements

• Added required properties to output schema in populate_template function
• Added additionalProperties field to gen3_metaschema.yml
• Added option to include all schemas in CLI validation
• Enhanced CLI validation process with exclusion list for schemas
• Added script to generate schema examples in YAML format

Bug Fixes

• Fixed data_file category to properly add data_file property reference
• Updated namespace handling in converter and schema templates
• Added core_metadata_collections to data_file properties

Refactoring

• Refactored CLI validation process to improve code clarity
• Refactored resolve_schema function to return dictionary instead of list
• Removed bundled_schema_to_list_dict function

Documentation

• Updated settings template to include versioning information

What's Changed

• category data_file now adds data_file property ref by @JoshuaHarris391 in #32
• Add 'init' command to CLI for schema initialization and update quicks… by @JoshuaHarris391 in #36
• Add project schema generation functionality by @JoshuaHarris391 in #37

Full Changelog: v2.0.11...v2.1.0

v2.0.11

Gen3SchemaDev V2 Release

We're excited to announce a new release of Gen3SchemaDev, a comprehensive toolkit for developing Gen3 data dictionaries.

What's New

This release includes extensive documentation and learning materials to help you create robust data models in Gen3:

Getting Started

• Quickstart Guide: Jump straight into using Gen3SchemaDev as a data modelling tool
• First Dictionary Tutorial: Step-by-step guide to creating your first Gen3 dictionary
• Troubleshooting Resources: Common issues and solutions

Documentation

• Core Concepts: Essential data modelling principles and FAIR data concepts
• Deep Dive Materials: Comprehensive coverage of Gen3 dictionary structure, schemas, descriptors, links, and properties
• Practical Examples: Real-world schema examples to accelerate your learning

For Developers

• Simplified installation via Poetry
• Test suite included
• Clear contributing guidelines
• Apache 2.0 licence

Community Contributions Welcome

We're actively seeking contributions in two key areas:

1. Maintaining the Gen3 metaschema
2. Expanding business rule logic in the rule validator module

Check out our contributing guidelines to get involved.

Special Thanks

Special acknowledgement to Marion Shadbolt for providing the foundational source material.

Get started today: Gen3SchemaDev Repository

What's Changed

• Feature m2 gen3 meta schema validation by @JoshuaHarris391 in #19
• Feature yaml to gen3 compiler by @JoshuaHarris391 in #20
• Feature cli generate by @JoshuaHarris391 in #22
• added bundle command by @JoshuaHarris391 in #23
• Fix consolidate example schema by @JoshuaHarris391 in #24
• Fix change entity to node by @JoshuaHarris391 in #25
• Feature ddvis by @JoshuaHarris391 in #27
• Main version 2 by @JoshuaHarris391 in #26
• Fix version parser by @JoshuaHarris391 in #29

Full Changelog: v1.0.0...v2.0.11

v1.0.0

First version of gen3schema dev. Uses google sheets, and various containers to facilitate dictionary modelling, bundling, and some preliminary validation

v0.1.0

📦 gen3schemadev Release - Archival Milestone

This release archives the current state of the gen3schemadev repository as it transitions to focus exclusively on data modeling and schema development. Previously, the repository supported tasks such as data generation, upload, synthetic file generation, and API-based data manipulation. These functionalities will no longer be the focus.

---

🔧 Release Details

🗂️ Repository Summary

This repository provides tools for automating processes in the Gen3 ecosystem, specifically:

• 📘 Data dictionary creation
• 📊 Data simulation
• ✅ Metadata validation
• 📤 Data submission

---

🔑 Key Features

1️⃣ gen3schemadev: Object-Relational Mapper for Gen3 Schemas

• Converts spreadsheets into YAML files for building Gen3 Data Dictionaries.
• Example tool: sheet2yaml.py

2️⃣ Workflow for Editing Project Dictionaries

• Edits made in Google Sheets.
• YAML schemas generated and validated locally.
• Simulated data created, validated, and uploaded to Gen3.
• Indexing services configured to integrate new dictionaries.

3️⃣ sheet2yaml-CLI.py: Command-Line Tool

• Generates schemas from Google Sheets/tabs formatted according to the provided template.

4️⃣ Plausible Data Generator

• Enhances simulated data by replacing random values with plausible ones based on defined distributions.
• Input: JSON files and a CSV or Google Sheet describing plausible values.
• Output: Edited JSONs and optional dummy sequencing/lipid files.

Example Usage:

python3 plausible_data_gen.py --path <PATH_TO_SIM_DATA> [--values <PATH_TO_CSV> | --gurl <GOOGLE_SHEET_URL>] --generate-files --file-types aligned_reads 

5️⃣ Metadata Validator

• Validates metadata against defined schemas.
• Includes a user guide and Jupyter notebook example.

6️⃣ Gen3 Data Submitter

• Automates data submission to Gen3 with detailed usage instructions.

---

📌 Supported Workflows

• 🛠️ Schema Development: YAML generation from spreadsheets.
• 🎛️ Data Simulation: Plausible dataset creation and refinement.
• 📑 Metadata Validation: Schema compliance checks.
• 🚀 Data Submission: Automated upload and indexing in Gen3.

---

🔮 Moving Forward

The repository will now focus exclusively on data modeling and schema development. Other functionalities will no longer be maintained or supported.

What's Changed

• merge null removal by @mshadbolt in #1
• Patch synth data acdc mar 2024 by @JoshuaHarris391 in #3
• 6 metadata json validator by @JoshuaHarris391 in #7
• pull main into gsheet feature by @JoshuaHarris391 in #8
• 4 download gsheet option by @JoshuaHarris391 in #9
• 11 synthetic data file generator by @JoshuaHarris391 in #12
• Feature validation reporter v2 by @JoshuaHarris391 in #14

Full Changelog: https://github.com/AustralianBioCommons/gen3schemadev/commits/v0.1.0