From 58fe4683c70a35f91b454e5adf853fbbddf8ee9a Mon Sep 17 00:00:00 2001 From: Michel Villanueva Date: Thu, 19 Feb 2026 23:06:11 -0500 Subject: [PATCH 1/4] Migrate from Carpentries/Jekyll to Jupyter Book 2.x (MyST) Replace the Jekyll/Carpentries build system with Jupyter Book 2.x using MyST Markdown. Convert all episode files from Carpentries blockquote syntax ({: .callout}, {: .challenge}, {: .solution}, {: .output}, {: .source}) to MyST directives (:::{note}, :::{admonition} with :class: challenge/dropdown). Add visible admonitions for questions, objectives, and keypoints in each episode. - Create myst.yml with project config, TOC, and HSF logo - Convert all 11 episodes, index, setup, reference, and extras - Replace Makefile with Jupyter Book commands (jb-build, jb-start) - Remove Jekyll files (_config.yml, Gemfile) - Update README with Jupyter Book build instructions - Add CLAUDE.md and requirements.txt Co-Authored-By: Claude Opus 4.6 --- .codespellignore | 1 + .gitignore | 3 +- CLAUDE.md | 62 +++++ Gemfile | 18 -- Makefile | 189 +------------- README.md | 19 +- _config.yml | 107 -------- _episodes/01-introduction.md | 42 +-- _episodes/02-sql-basics.md | 238 +++++++++-------- _episodes/03-break.md | 3 +- _episodes/04-mysql-and-python.md | 160 ++++++------ _episodes/05-mysql-queries.md | 116 ++++----- _episodes/06-conditions-database.md | 38 +-- _episodes/07-break.md | 3 +- _episodes/08-sqllite.md | 164 ++++++------ _episodes/09-opensearch-queires.md | 381 +++++++++++++--------------- _episodes/10-text-based-search.md | 129 +++++----- _extras/about.md | 4 +- _extras/discuss.md | 4 - _extras/guide.md | 4 - fig/hsf-logo.png | Bin 0 -> 23188 bytes index.md | 17 +- myst.yml | 34 +++ reference.md | 2 +- requirements.txt | 1 + setup.md | 66 +++-- 26 files changed, 789 insertions(+), 1016 deletions(-) create mode 100644 .codespellignore create mode 100644 CLAUDE.md delete mode 100644 Gemfile delete mode 100644 _config.yml create mode 100644 fig/hsf-logo.png create mode 100644 myst.yml create mode 100644 requirements.txt diff --git a/.codespellignore b/.codespellignore new file mode 100644 index 0000000..7bfd075 --- /dev/null +++ b/.codespellignore @@ -0,0 +1 @@ +SLAC diff --git a/.gitignore b/.gitignore index c569c7b..bad1635 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,4 @@ _site +_build .ruby-version -Gemfile.lock \ No newline at end of file +Gemfile.lock diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..9da6093 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,62 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +HSF Training lesson: "Introduction to Databases for HEP" — a Jupyter Book 2.x (MyST Markdown) educational curriculum teaching database fundamentals (MySQL, SQLite, OpenSearch) with applications in High Energy Physics and Nuclear Physics. + +**Status:** Pre-alpha. Published at https://hsf-training.github.io/hsf-training-databases-basics/ + +## Build and Development Commands + +Activate the virtual environment first, then: + +```bash +make jb-build # Build Jupyter Book site (HTML output in _build/) +make jb-start # Start Jupyter Book dev server with live reload +make jb-clean # Remove _build/ directory +``` + +Or directly: +```bash +jupyter-book build --html +jupyter-book start +``` + +## Pre-commit Hooks + +Required before committing: +```bash +pip3 install pre-commit && pre-commit install +``` + +Hooks enforce: trailing whitespace removal, end-of-file newlines, codespell (dictionary overrides in `codespell.txt`), and `blacken-docs` for Python code blocks in Markdown. Run manually with `pre-commit run -a`. + +## Architecture + +- **`myst.yml`** — Jupyter Book 2.x configuration: project metadata, table of contents, and site settings. +- **`_episodes/`** — Lesson content as numbered MyST Markdown files (`NN-topic.md`, e.g. `02-sql-basics.md`). Each has YAML front matter with title, teaching time, and exercises. Questions, objectives, and keypoints are rendered as admonition blocks. Episodes are listed in the TOC in `myst.yml`. +- **`_extras/`** — Supplementary pages (about, discussion, instructor guide). +- **`fig/`** — Figures and diagrams referenced by episodes. +- **`Makefile`** — Jupyter Book build commands. +- **`.devcontainer/`** — GitHub Codespaces config with Docker Compose services: MySQL (3306), OpenSearch (9200, 9600), Grafana (3000). +- **`_build/`** — Generated output (gitignored, never edit directly). + +## Content Conventions + +- Episodes use MyST directive syntax for admonitions: + - `:::{note}` for callouts/informational boxes + - `:::{important}` for prerequisites + - `:::::{admonition} Title` with `:class: challenge` for exercises, containing nested `::::{admonition} Solution` with `:class: dropdown` for collapsible solutions +- Code output is shown in `` ```text `` fenced blocks. +- Mark incomplete sections with `FIXME`. +- New episodes must follow the `NN-topic.md` naming pattern with two-digit ordering prefix and be added to the TOC in `myst.yml`. +- Several files are centrally maintained by the HSF Training organization (marked with `CENTRALLY MAINTAINED` comments) — avoid modifying these: `.pre-commit-config.yaml` (partially), `.github/config.yml`, `.github/stale.yml`. + +## Commit and PR Guidelines + +- Short, imperative commit messages with optional scope prefix (`docs:`, `build:`, `setup:`). +- Reference issues with `#123`. +- Run `jupyter-book build --html` to verify the site builds before opening PRs. +- Never commit `_build/` contents. diff --git a/Gemfile b/Gemfile deleted file mode 100644 index 7609dd0..0000000 --- a/Gemfile +++ /dev/null @@ -1,18 +0,0 @@ -# frozen_string_literal: true - -source 'https://rubygems.org' - -git_source(:github) {|repo_name| "https://github.com/#{repo_name}" } - -# Synchronize with https://pages.github.com/versions -ruby '>=2.5.5' - -gem 'github-pages', group: :jekyll_plugins - -gem 'rake' -gem 'hsf-training-theme', git: "https://github.com/hsf-training/hsf-training-theme.git", branch: 'main' - -if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new('3.0.0') - gem 'webrick', '>= 1.6.1' -end - diff --git a/Makefile b/Makefile index 6bf02c6..150910c 100644 --- a/Makefile +++ b/Makefile @@ -1,188 +1,27 @@ ## ======================================== -## Commands for both workshop and lesson websites. +## Jupyter Book 2 commands # Settings -MAKEFILES=Makefile $(wildcard *.mk) -JEKYLL=bundle config set --local path .vendor/bundle && bundle install && bundle update && bundle exec jekyll -PARSER=bin/markdown_ast.rb -DST=_site - -# Find Docker -DOCKER := $(shell which docker 2>/dev/null) - -# Check Python 3 is installed and determine if it's called via python3 or python -# (https://stackoverflow.com/a/4933395) -PYTHON3_EXE := $(shell which python3 2>/dev/null) -ifneq (, $(PYTHON3_EXE)) - ifeq (,$(findstring Microsoft/WindowsApps/python3,$(subst \,/,$(PYTHON3_EXE)))) - PYTHON := $(PYTHON3_EXE) - endif -endif - -ifeq (,$(PYTHON)) - PYTHON_EXE := $(shell which python 2>/dev/null) - ifneq (, $(PYTHON_EXE)) - PYTHON_VERSION_FULL := $(wordlist 2,4,$(subst ., ,$(shell python --version 2>&1))) - PYTHON_VERSION_MAJOR := $(word 1,${PYTHON_VERSION_FULL}) - ifeq (3, ${PYTHON_VERSION_MAJOR}) - PYTHON := $(PYTHON_EXE) - else - PYTHON_NOTE = "Your system does not appear to have Python 3 installed." - endif - else - PYTHON_NOTE = "Your system does not appear to have any Python installed." - endif -endif - +JB = jupyter-book +DST = _build # Default target .DEFAULT_GOAL := commands -## I. Commands for both workshop and lesson websites -## ================================================= +.PHONY: jb-build jb-start jb-clean commands -.PHONY: site docker-serve repo-check clean clean-rmd +## * jb-build : build Jupyter Book site (HTML) +jb-build: + $(JB) build --html -## * serve : render website and run a local server -serve : lesson-md index.md - ${JEKYLL} serve +## * jb-start : start Jupyter Book dev server +jb-start: + $(JB) start -## * site : build website but do not run a server -site : lesson-md index.md - ${JEKYLL} build - -## * docker-serve : use Docker to serve the site -docker-serve : -ifeq (, $(DOCKER)) - $(error Your system does not appear to have Docker installed) -else - @$(DOCKER) pull carpentries/lesson-docker:latest - @$(DOCKER) run --rm -it \ - -v $${PWD}:/home/rstudio \ - -p 4000:4000 \ - -p 8787:8787 \ - -e USERID=$$(id -u) \ - -e GROUPID=$$(id -g) \ - carpentries/lesson-docker:latest -endif - -## * repo-check : check repository settings -repo-check : python - @${PYTHON} bin/repo_check.py -s . - -## * clean : clean up junk files -clean : +## * jb-clean : clean Jupyter Book build artifacts +jb-clean: @rm -rf ${DST} - @rm -rf .sass-cache - @rm -rf bin/__pycache__ - @rm -rf .vendor - @rm -rf .bundle - @rm -f Gemfile.lock - @find . -name .DS_Store -exec rm {} \; - @find . -name '*~' -exec rm {} \; - @find . -name '*.pyc' -exec rm {} \; - -## * clean-rmd : clean intermediate R files (that need to be committed to the repo) -clean-rmd : - @rm -rf ${RMD_DST} - @rm -rf fig/rmd-* - - -## -## II. Commands specific to workshop websites -## ================================================= - -.PHONY : workshop-check - -## * workshop-check : check workshop homepage -workshop-check : python - @${PYTHON} bin/workshop_check.py . - - -## -## III. Commands specific to lesson websites -## ================================================= - -.PHONY : lesson-check lesson-md lesson-files lesson-fixme install-rmd-deps -# RMarkdown files -RMD_SRC = $(wildcard _episodes_rmd/*.Rmd) -RMD_DST = $(patsubst _episodes_rmd/%.Rmd,_episodes/%.md,$(RMD_SRC)) - -# Lesson source files in the order they appear in the navigation menu. -MARKDOWN_SRC = \ - index.md \ - CODE_OF_CONDUCT.md \ - setup.md \ - $(sort $(wildcard _episodes/*.md)) \ - reference.md \ - $(sort $(wildcard _extras/*.md)) \ - LICENSE.md - -# Generated lesson files in the order they appear in the navigation menu. -HTML_DST = \ - ${DST}/index.html \ - ${DST}/conduct/index.html \ - ${DST}/setup/index.html \ - $(patsubst _episodes/%.md,${DST}/%/index.html,$(sort $(wildcard _episodes/*.md))) \ - ${DST}/reference.html \ - $(patsubst _extras/%.md,${DST}/%/index.html,$(sort $(wildcard _extras/*.md))) \ - ${DST}/license/index.html - -## * install-rmd-deps : Install R packages dependencies to build the RMarkdown lesson -install-rmd-deps: - @${SHELL} bin/install_r_deps.sh - -## * lesson-md : convert Rmarkdown files to markdown -lesson-md : ${RMD_DST} - -_episodes/%.md: _episodes_rmd/%.Rmd install-rmd-deps - @mkdir -p _episodes - @$(SHELL) bin/knit_lessons.sh $< $@ - -## * lesson-check : validate lesson Markdown -lesson-check : python lesson-fixme - @${PYTHON} bin/lesson_check.py -s . -p ${PARSER} -r _includes/links.md - -## * lesson-check-all : validate lesson Markdown, checking line lengths and trailing whitespace -lesson-check-all : python - @${PYTHON} bin/lesson_check.py -s . -p ${PARSER} -r _includes/links.md -l -w --permissive - -## * unittest : run unit tests on checking tools -unittest : python - @${PYTHON} bin/test_lesson_check.py - -## * lesson-files : show expected names of generated files for debugging -lesson-files : - @echo 'RMD_SRC:' ${RMD_SRC} - @echo 'RMD_DST:' ${RMD_DST} - @echo 'MARKDOWN_SRC:' ${MARKDOWN_SRC} - @echo 'HTML_DST:' ${HTML_DST} - -## * lesson-fixme : show FIXME markers embedded in source files -lesson-fixme : - @grep --fixed-strings --word-regexp --line-number --no-messages FIXME ${MARKDOWN_SRC} || true - -## -## IV. Auxililary (plumbing) commands -## ================================================= - -.PHONY : commands python - -## * commands : show all commands. -commands : +## * commands : show all commands. +commands: @sed -n -e '/^##/s|^##[[:space:]]*||p' $(MAKEFILE_LIST) - -python : -ifeq (, $(PYTHON)) - $(error $(PYTHON_NOTE)) -else - @: -endif - -index.md : -ifeq (, $(wildcard index.md)) - $(error index.md not found) -else - @: -endif diff --git a/README.md b/README.md index 4e6b451..df68c85 100644 --- a/README.md +++ b/README.md @@ -39,15 +39,20 @@ We'd like to ask you to familiarize yourself with our [Contribution Guide](CONTR the [more detailed guidelines][lesson-example] on proper formatting, ways to render the lesson locally, and even how to write new episodes. -Quick summary of how to get a local preview: Install [jekyll][jekyll] and then run +Quick summary of how to get a local preview: Install [Jupyter Book][jupyter-book] and then run +```bash +pip install -r requirements.txt +jupyter-book start ``` -bundle install -bundle update -bundle exec jekyll serve -``` -Unless we change framework versions, only the last command needs to be typed after the first time. +This will build the site and start a local server (default port 3000). Open `http://localhost:3000` in your browser. + +To build static HTML without starting a server: + +```bash +jupyter-book build --html +``` Before committing anything, we also ask you to install the [pre-commit][pre-commit] hooks of this repository: @@ -108,7 +113,7 @@ Instead, all regular contributors are listed on our [HSF Training Community page [schools]: https://hepsoftwarefoundation.org/Schools/events.html [issues]: https://github.com/hsf-training/hsf-training-databases-basics/issues [progit]: http://git-scm.com/book/en/v2/GitHub-Contributing-to-a-Project -[jekyll]: https://jekyllrb.com/ +[jupyter-book]: https://jupyterbook.org/ [allcontrib-emoji-key]: https://allcontributors.org/docs/en/emoji-key [gfi-badge]: https://img.shields.io/badge/-good%20first%20issue-gold.svg [schools-badge]: https://img.shields.io/badge/upcoming%20events-browse-ff69b4 diff --git a/_config.yml b/_config.yml deleted file mode 100644 index 98f40ee..0000000 --- a/_config.yml +++ /dev/null @@ -1,107 +0,0 @@ -#------------------------------------------------------------ -# Values for this lesson. -#------------------------------------------------------------ - -# Which carpentry is this ("swc", "dc", "lc", or "cp")? -# swc: Software Carpentry -# dc: Data Carpentry -# lc: Library Carpentry -# cp: Carpentries (to use for instructor traning for instance) -# hsf: HEP Software Foundation carpentry -carpentry: "hsf" - -# Overall title for pages. -title: "Introduction to Databases for HEP" - -# Life cycle stage of the lesson -# See this page for more details: https://cdh.carpentries.org/the-lesson-life-cycle.html -# Possible values: "pre-alpha", "alpha", "beta", "stable" -life_cycle: "pre-alpha" - -#------------------------------------------------------------ -# Generic settings (should not need to change). -#------------------------------------------------------------ - -# What kind of thing is this ("workshop" or "lesson")? -kind: "lesson" - -# Magic to make URLs resolve both locally and on GitHub. -# See https://help.github.com/articles/repository-metadata-on-github-pages/. -# Please don't change it: / is correct. -repository: / - -# Email address, no mailto: -email: "hsf-training-wg@googlegroups.com" - -# Sites. -amy_site: "https://amy.carpentries.org/" -carpentries_github: "https://github.com/carpentries" -carpentries_pages: "https://carpentries.github.io" -carpentries_site: "https://carpentries.org/" -dc_site: "https://datacarpentry.org" -hsf_site: "https://hepsoftwarefoundation.org/training/curriculum" -example_repo: "https://github.com/carpentries/lesson-example" -example_site: "https://carpentries.github.io/lesson-example" -lc_site: "https://librarycarpentry.org/" -swc_github: "https://github.com/swcarpentry" -swc_pages: "https://swcarpentry.github.io" -swc_site: "https://software-carpentry.org" -template_repo: "https://github.com/carpentries/styles" -training_site: "https://carpentries.github.io/instructor-training" -workshop_repo: "https://github.com/carpentries/workshop-template" -workshop_site: "https://carpentries.github.io/workshop-template" -cc_by_human: "https://creativecommons.org/licenses/by/4.0/" - -# Surveys. -pre_survey: "https://carpentries.typeform.com/to/wi32rS?slug=" -post_survey: "https://carpentries.typeform.com/to/UgVdRQ?slug=" -instructor_pre_survey: "https://www.surveymonkey.com/r/instructor_training_pre_survey?workshop_id=" -instructor_post_survey: "https://www.surveymonkey.com/r/instructor_training_post_survey?workshop_id=" - - -# Start time in minutes (0 to be clock-independent, 540 to show a start at 09:00 am). -start_time: 0 - -# Specify that things in the episodes collection should be output. -collections: - episodes: - output: true - permalink: /:path/index.html - extras: - output: true - permalink: /:path/index.html - -# Set the default layout for things in the episodes collection. -defaults: - - values: - root: . - layout: page - - scope: - path: "" - type: episodes - values: - root: .. - layout: episode - - scope: - path: "" - type: extras - values: - root: .. - layout: page - -# Files and directories that are not to be copied. -exclude: - - Makefile - - bin/ - - .Rproj.user/ - - .vendor/ - - vendor/ - - .docker-vendor/ - -# Turn on built-in syntax highlighting. -highlighter: rouge - -# show authors managed by allcontributors bot -show_authors: true - -remote_theme: hsf-training/hsf-training-theme diff --git a/_episodes/01-introduction.md b/_episodes/01-introduction.md index 6bb64d9..1fa8382 100644 --- a/_episodes/01-introduction.md +++ b/_episodes/01-introduction.md @@ -2,18 +2,18 @@ title: "Introduction" teaching: 60 exercises: 30 -questions: -- "What is a database management system and why it is a good idea to use one?" -- "What are the differences between relational and non-relational databases?" -objectives: -- "Understand the concepts of a database management system" -- "Learn about common database management systems used in HEP and Nuclear Physics" -keypoints: -- "A database management system (DBMS) is specialized on managing databases in a scalable and reliable way." -- "A relational database organizes data into tables with rows and columns." -- "A non-relational database does not use the tabular schema of columns found in relational databases, having a more flexible schema." --- +:::{admonition} Questions +- What is a database management system and why it is a good idea to use one? +- What are the differences between relational and non-relational databases? +::: + +:::{admonition} Objectives +- Understand the concepts of a database management system +- Learn about common database management systems used in HEP and Nuclear Physics +::: + # What is a database? A database is a collection of data stored in a computer system. For example: @@ -42,10 +42,10 @@ the sizes of the files that match the date you are interested in. Things get muc the information can get very easily corrupted. And what if multiple people need to constantly entry information? What a nightmare to keep everything in sync! -> ## Data organization -> When developing software, how do we organize data is a critical decision that has an effect on the performance -> and scalability of the application. -{: .callout} +:::{note} Data organization +When developing software, how do we organize data is a critical decision that has an effect on the performance +and scalability of the application. +::: In practice, databases are managed by software called database management systems (DBMS). A DBMS is a computer software application that interacts with the user, other applications, and the database itself to capture and analyze data. @@ -114,10 +114,16 @@ Non-relational databases, also known as **NoSQL**, are databases that do not use Instead, they use a storage model optimized for the specific requirements of the type of data being stored. In other words, they don't have a fixed number attributes that each record must have and the schema is more flexible. -> ## Relational vs NoSQL databases -> NoSQL databases have become very popular due to the flexibility they offer. However, many-to-one and many-to-many relationships are more easily represented in a relational database. -> Which one to choose depends on the specific requirements and must be considered carefully. -{: .callout} +:::{note} Relational vs NoSQL databases +NoSQL databases have become very popular due to the flexibility they offer. However, many-to-one and many-to-many relationships are more easily represented in a relational database. +Which one to choose depends on the specific requirements and must be considered carefully. +::: In the last part of this training module, we will discuss more in details NoSQL databases, and we will have a quick look on [OpenSearch](https://opensearch.org/docs/), a distributed search engine used as a NoSQL database. + +:::{admonition} Key Points +- A database management system (DBMS) is specialized on managing databases in a scalable and reliable way. +- A relational database organizes data into tables with rows and columns. +- A non-relational database does not use the tabular schema of columns found in relational databases, having a more flexible schema. +::: diff --git a/_episodes/02-sql-basics.md b/_episodes/02-sql-basics.md index 0740ead..4cba7c2 100644 --- a/_episodes/02-sql-basics.md +++ b/_episodes/02-sql-basics.md @@ -2,19 +2,19 @@ title: "MySQL Basics" teaching: x exercises: x -questions: -- "What are the core SQL commands you need to know to interact with a MySQL database?" -- "How do you create, update, and delete records in a MySQL database?" -objectives: -- "Understand the basic SQL commands used in MySQL." -- "Learn how to create a database and tables in MySQL." -- "Practice inserting, updating, and deleting records in a MySQL table." -keypoints: -- "SQL commands are the building blocks for interacting with a MySQL database." -- "Creating a database and tables is the first step in structuring your data." -- "Inserting, updating, and deleting records allows you to manage data effectively." --- +:::{admonition} Questions +- What are the core SQL commands you need to know to interact with a MySQL database? +- How do you create, update, and delete records in a MySQL database? +::: + +:::{admonition} Objectives +- Understand the basic SQL commands used in MySQL. +- Learn how to create a database and tables in MySQL. +- Practice inserting, updating, and deleting records in a MySQL table. +::: + ## SQL Commands @@ -30,25 +30,25 @@ Here are some of the core SQL commands that you'll use in MySQL: ## Setting up a client for sql commands In the terminal , run the following command to start the mysql client inside the Docker container: -~~~bash +```bash docker exec -it myfirst-sqlserver bash -c "mysql -uroot -pmypassword" -~~~ +``` Then you will see mysql command prompt as ``mysql>`` . All the sql command has to be typed in this command prompt. -> # Using a MySQL client in the host -> If you want to use a MySQL client in the host, you can install it (e.g., `mysql-client` package in Ubuntu) -> and connect to the MySQL server running in the Docker container. -> ```bash -> mysql -uroot -pmypassword -P 3306 --protocol=tcp -> ``` -> Notice that you need to specify the port number and the protocol to connect to the MySQL server running in the Docker container. See the -> details of the Docker container hosting the MySQL server by running the command `docker ps`: -> ```bash -> docker ps -> CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -> f86b66cc36bf mysql "docker-entrypoint.s…" 2 minutes ago Up 2 minutes 0.0.0.0:3306->3306/tcp, 33060/tcp myfirst-sqlserver ->``` -{: .callout} +:::{note} Using a MySQL client in the host +If you want to use a MySQL client in the host, you can install it (e.g., `mysql-client` package in Ubuntu) +and connect to the MySQL server running in the Docker container. +```bash +mysql -uroot -pmypassword -P 3306 --protocol=tcp +``` +Notice that you need to specify the port number and the protocol to connect to the MySQL server running in the Docker container. See the +details of the Docker container hosting the MySQL server by running the command `docker ps`: +```bash + docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +f86b66cc36bf mysql "docker-entrypoint.s…" 2 minutes ago Up 2 minutes 0.0.0.0:3306->3306/tcp, 33060/tcp myfirst-sqlserver +``` +::: ## Create a database. We will first create a database named ``metadata`` in our mysql server. @@ -56,18 +56,18 @@ We will first create a database named ``metadata`` in our mysql server. CREATE DATABASE metadata; ``` -> ## Case sensitivity in MySQL ->In MySQL, SQL commands are case-insensitive. In other words, `CREATE DATABASE metadata` is the same as `create database metadata`. -> ->However, it is a common practice to write SQL commands in uppercase ->to distinguish them from table and column names, which are case-sensitive. -{: .callout} +:::{note} Case sensitivity in MySQL +In MySQL, SQL commands are case-insensitive. In other words, `CREATE DATABASE metadata` is the same as `create database metadata`. + +However, it is a common practice to write SQL commands in uppercase +to distinguish them from table and column names, which are case-sensitive. +::: You can list all the databases by using the command ```sql SHOW DATABASES; ``` -~~~ +```text +--------------------+ | Database | +--------------------+ @@ -77,33 +77,30 @@ SHOW DATABASES; | performance_schema | | sys | +--------------------+ -~~~ -{: .output} +``` It shows that the database "metadata" is created. -> ## What are those other databases? -> ->By default, MySQL comes with several databases that serve specific purposes. We will not go into details of each database, ->but here is a brief overview: -> -> - `mysql`: This database contains user account information and privileges -> - `information_schema`: Contains metadata about all the other databases in the MySQL server -> - `performance_schema`: Contains performance metrics for the MySQL server -> - `sys`: Used for tuning and diagnosis use cases -> -> You can read more about these databases in the [MySQL documentation](https://dev.mysql.com/doc/refman/8.0/en/information-schema.html). -> For now it is not necessary to understand them in detail. -{: .callout} +:::{note} What are those other databases? +By default, MySQL comes with several databases that serve specific purposes. We will not go into details of each database, +but here is a brief overview: + +- `mysql`: This database contains user account information and privileges +- `information_schema`: Contains metadata about all the other databases in the MySQL server +- `performance_schema`: Contains performance metrics for the MySQL server +- `sys`: Used for tuning and diagnosis use cases + +You can read more about these databases in the [MySQL documentation](https://dev.mysql.com/doc/refman/8.0/en/information-schema.html). +For now it is not necessary to understand them in detail. +::: To work with a specific database, you can use the USE command. For instance, to select the "metadata" database: ```sql USE metadata; ``` -~~~ +```text Database changed -~~~ -{: .output} +``` ## Creating a table @@ -151,7 +148,7 @@ You can see the table and corresponding columns by using the command ```sql DESCRIBE dataset; ``` -~~~ +```text +------------------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +------------------+--------------+------+-----+---------+----------------+ @@ -163,8 +160,7 @@ DESCRIBE dataset; | data_type | text | YES | | NULL | | | collision_energy | int | NO | | NULL | | +------------------+--------------+------+-----+---------+----------------+ -~~~ -{: .output} +``` ## INSERT record into table You can use the INSERT INTO command to add records to a table. This command has structure ``INSERT INTO () Values ()``. @@ -186,7 +182,7 @@ The SELECT command allows you to retrieve records from a table. To retrieve all ```sql SELECT * FROM dataset; ``` -~~~ +```text mysql> SELECT * FROM dataset; +----+-------------------+------------+-------------+----------------+-----------+------------------+ | id | filename | run_number | total_event | collision_type | data_type | collision_energy | @@ -194,21 +190,19 @@ mysql> SELECT * FROM dataset; | 1 | expx.myfile1.root | 100 | 1112 | pp | data | 11275 | | 2 | expx.myfile2.root | 55 | 999 | pPb | mc | 1127 | +----+-------------------+------------+-------------+----------------+-----------+------------------+ -~~~ -{: .output} +``` You can select specific columns by listing them after the SELECT statement: ```sql SELECT filename FROM dataset; ``` -~~~ +```text +-------------------+ | filename | +-------------------+ | expx.myfile1.root | | expx.myfile2.root | +-------------------+ -~~~ -{: .output} +``` 2 rows in set (0.00 sec) ### Search with some condition @@ -221,30 +215,28 @@ In addition you can use logical operators such as AND and OR to combine multiple SELECT filename FROM dataset WHERE run_number > 50 AND collision_type='pp'; ``` -{: .source} - -> ## SELECT on different condition -> -> Get the filename of condition total_event > 1000 and data_type is "data". -> -> > ## Solution -> > -> > ```sql -> >SELECT filename FROM dataset WHERE event_number > 1000 AND data_type='mc'; -> > ``` -> > {: .source} -> > -> > ~~~ ->> +-------------------+ ->>| filename | ->> +-------------------+ ->> | expx.myfile1.root | ->> +-------------------+ ->> 1 row in set (0.00 sec) -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} +:::::{admonition} SELECT on different condition +:class: challenge + +Get the filename of condition total_event > 1000 and data_type is "data". + +::::{admonition} Solution +:class: dropdown + +```sql +SELECT filename FROM dataset WHERE event_number > 1000 AND data_type='mc'; +``` + +```text ++-------------------+ +| filename | ++-------------------+ +| expx.myfile1.root | ++-------------------+ +1 row in set (0.00 sec) +``` +:::: +::::: ## UPDATE @@ -258,30 +250,30 @@ SET collision_type = 'PbPb', collision_energy = 300 WHERE filename = 'expx.myfile1.root'; ``` -> ## Update on a condition -> -> update the total_event of file "expx.myfile2.root" to 800. -> -> > ## Solution -> > -> > ```sql -> > UPDATE dataset -> > SET total_event = 800 -> > WHERE filename = 'expx.myfile2.root'; -> > ``` -> > {: .source} -> > -> > ~~~ ->> +-------------------+ ->>| filename | ->> +-------------------+ ->> | expx.myfile1.root | ->> +-------------------+ ->> 1 row in set (0.00 sec) -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} +:::::{admonition} Update on a condition +:class: challenge + +update the total_event of file "expx.myfile2.root" to 800. + +::::{admonition} Solution +:class: dropdown + +```sql +UPDATE dataset +SET total_event = 800 +WHERE filename = 'expx.myfile2.root'; +``` + +```text ++-------------------+ +| filename | ++-------------------+ +| expx.myfile1.root | ++-------------------+ +1 row in set (0.00 sec) +``` +:::: +::::: ## DELETE @@ -293,14 +285,20 @@ DELETE FROM dataset WHERE filename = 'expx.myfile2.root'; ``` ->## Be careful with UPDATE and DELETE without WHERE! ->Very important: if you omit the `WHERE` clause in an `UPDATE` or `DELETE` statement, you will update or delete ALL records in the table! -> -> For example, the following command -> ```sql -> DELETE FROM dataset; -> ``` -> will delete all records in the `dataset` table. -> ->This can have unintended consequences, so be cautious when using these commands. -{: .callout} +:::{note} Be careful with UPDATE and DELETE without WHERE! +Very important: if you omit the `WHERE` clause in an `UPDATE` or `DELETE` statement, you will update or delete ALL records in the table! + +For example, the following command +```sql +DELETE FROM dataset; +``` +will delete all records in the `dataset` table. + +This can have unintended consequences, so be cautious when using these commands. +::: + +:::{admonition} Key Points +- SQL commands are the building blocks for interacting with a MySQL database. +- Creating a database and tables is the first step in structuring your data. +- Inserting, updating, and deleting records allows you to manage data effectively. +::: diff --git a/_episodes/03-break.md b/_episodes/03-break.md index 41f04f9..28843d3 100644 --- a/_episodes/03-break.md +++ b/_episodes/03-break.md @@ -6,5 +6,4 @@ questions: - "Get up, stretch out, and dance!" objectives: - "Refresh your mind!" - -{% include links.md %} +--- diff --git a/_episodes/04-mysql-and-python.md b/_episodes/04-mysql-and-python.md index 1279ecd..8eb150e 100644 --- a/_episodes/04-mysql-and-python.md +++ b/_episodes/04-mysql-and-python.md @@ -2,18 +2,19 @@ title: "SQLAlchemy and MySQL: Exercises" teaching: 5 exercises: 6 -questions: -- "How to perform CRUD operations using SQLAlchemy?" -- "How to query and filter records in SQLAlchemy?" -objectives: -- "Practice inserting records into a MySQL database using SQLAlchemy." -- "Perform queries and filtering on the dataset table." -- "Update and delete records in the dataset table." -keypoints: -- "CRUD operations in SQLAlchemy: Create, Read, Update, Delete." -- "Querying and filtering records based on specific conditions." --- +:::{admonition} Questions +- How to perform CRUD operations using SQLAlchemy? +- How to query and filter records in SQLAlchemy? +::: + +:::{admonition} Objectives +- Practice inserting records into a MySQL database using SQLAlchemy. +- Perform queries and filtering on the dataset table. +- Update and delete records in the dataset table. +::: + ## Why python with SQL? SQL is a perfectly designed language to specify database operations in a declarative, record-centred way, but this very design makes it unfamiliar to programmers used to writing object-oriented, imperative or functional code. Worse, good performance in SQL can depend on the specifics of the database engine we're interacting with - MySQL, PostgreSQL and so on all have their own ideal performance approaches and extensions. Finally, it's easy to write inadvertently insecure SQL queries by forgetting to sanitise and quote inputs. @@ -26,9 +27,9 @@ SQLAlchemy is a powerful library that provides a high-level interface for intera ## Lets create a new database for this chapter. So as not to collide with database created in previous chapter, let's create a different database for this one called `metadata2` In another terminal, run the following command -~~~bash +```bash docker exec -it metadata bash -c "mysql -uroot -pmypassword" -~~~ +``` Then you will see a mysql command prompt as ``mysql>`` . Use following command to create a database named ``metadata2``. ```sql CREATE DATABASE metadata2; @@ -70,7 +71,7 @@ Now, create a new python file, and use it for the subsequent commands. ## Setting Up the Database Connection: -SQLAlchemy facilitates database connections in Python by using an Engine, which acts as the interface between the application and the database. The Engine manages database communication, executing SQL commands, and transaction handling. It requires a connection URL to specify the database type, credentials, and other connection details, which allows SQLAlchemy to translate its functions to whatever database configuration we want to interact with. Sessions, managed by sessionmaker, handle interactions between the application and the database, allowing for transactions, queries, and data manipulation in a structured manner. +SQLAlchemy facilitates database connections in Python by using an Engine, which acts as the interface between the application and the database. The Engine manages database communication, executing SQL commands, and transaction handling. It requires a connection URL to specify the database type, credentials, and other connection details, which allows SQLAranslate its functions to whatever database configuration we want to interact with. Sessions, managed by sessionmaker, handle interactions between the application and the database, allowing for transactions, queries, and data manipulation in a structured manner. Let's import necessary things. ```python @@ -182,28 +183,26 @@ for result in results: print(result.filename) ``` -{: .source} - -> ## Search on different column -> -> Retrieve and display all collision_type -> -> > ## Solution -> > -> > ```python -> >results = session.query(Dataset.collison_type).all() -> >for result in results: - print(result.collison_type) -> > ``` -> > {: .source} -> > -> > ~~~ -> > pp -> > pPb -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} +:::::{admonition} Search on different column +:class: challenge + +Retrieve and display all collision_type + +::::{admonition} Solution +:class: dropdown + +```python +results = session.query(Dataset.collison_type).all() +for result in results: + print(result.collison_type) +``` + +```text +pp +pPb +``` +:::: +::::: # Search the database with condition. In SQLAlchemy, the filter() method is used within a query() to add conditions or criteria to the query. It narrows down the selection by applying specific constraints based on the given criteria. @@ -226,28 +225,30 @@ for result in results2: print(result.filename) ``` -{: .source} - -> ## Search using OR statement -> -> Retrieve and display filenames with "mc" data_type and collision_energy>1000 -> -> > ## Solution -> > -> > ```python -> >results = session.query(Dataset.filename).filter((Dataset.data_type =="mc") | (Dataset.collision_energy>1000 )).all() -> >for result in results: -> > print(result.collison_type) -> > ``` -> > {: .source} -> > -> > ~~~ -> > expx.myfile1.root -> > expx.myfile2.root -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} +:::::{admonition} Search using OR statement +:class: challenge + +Retrieve and display filenames with "mc" data_type and collision_energy>1000 + +::::{admonition} Solution +:class: dropdown + +```python +results = ( + session.query(Dataset.filename) + .filter((Dataset.data_type == "mc") | (Dataset.collision_energy > 1000)) + .all() +) +for result in results: + print(result.collison_type) +``` + +```text +expx.myfile1.root +expx.myfile2.root +``` +:::: +::::: ## Update the database To update a record in a table, you begin by querying for the specific record or records you want to update using query(). The filter() method is used to specify the conditions for the selection. Once you have the record(s) to update, modify the attributes as needed. Finally, calling session.commit() saves the changes made to the database. This ensures that the modifications are persisted permanently. @@ -262,29 +263,23 @@ if record_to_update: record_to_update.collision_energy = 300 session.commit() ``` -{: .source} - -> ## Update -> -> Update `run_number` to 1000 for record with data_type `mc`. -> -> > ## Solution -> > -> > ```python -> >record_to_update = (session.query(Dataset).filter(Dataset.data_type == "mc").first()) -> > if record_to_update: -> > record_to_update.run_number = 1000 -> > session.commit() -> > ``` -> > {: .source} -> > -> > ~~~ -> > -> > -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} + +:::::{admonition} Update +:class: challenge + +Update `run_number` to 1000 for record with data_type `mc`. + +::::{admonition} Solution +:class: dropdown + +```python +record_to_update = session.query(Dataset).filter(Dataset.data_type == "mc").first() +if record_to_update: + record_to_update.run_number = 1000 + session.commit() +``` +:::: +::::: ## Delete the database Basically the same, we need to first get the record to update using query and filter. Then delete the record and commit to see the changes. @@ -303,3 +298,8 @@ It's essential to close the session after you've finished working with it to rel ```python session.close() ``` + +:::{admonition} Key Points +- CRUD operations in SQLAlchemy: Create, Read, Update, Delete. +- Querying and filtering records based on specific conditions. +::: diff --git a/_episodes/05-mysql-queries.md b/_episodes/05-mysql-queries.md index 2b687b1..26cc390 100644 --- a/_episodes/05-mysql-queries.md +++ b/_episodes/05-mysql-queries.md @@ -2,16 +2,17 @@ title: "Relations between Tables" teaching: 60 exercises: 30 -questions: -- "How to perform SQL Join?" -- "How to make use of different SQL joins?" -objectives: -- "To retrieve data from multiple tables Simultaneously when data is related." -keypoints: -- "JOIN operations are used to combine rows from two or more tables based on a common column." -- "Foreign keys are used to maintain data integrity and enforce references." --- +:::{admonition} Questions +- How to perform SQL Join? +- How to make use of different SQL joins? +::: + +:::{admonition} Objectives +- To retrieve data from multiple tables Simultaneously when data is related. +::: + Let's explore how to retrieve data from multiple tables simultaneously when the data is related. Before starting, let's create a new database that will contain metadata from multiple experiments, and two tables to work with. @@ -50,7 +51,7 @@ Remember that you can check the data in the table using: ```sql SELECT * FROM experiments; ``` -~~~ +```text +----+-----------------+------------+----------------+------------+ | id | experiment_name | laboratory | collision_type | start_year | +----+-----------------+------------+----------------+------------+ @@ -61,8 +62,7 @@ SELECT * FROM experiments; | 5 | Belle2 | KEK | e+e- | 2018 | | 6 | BaBar | SLAC | e+e- | 1999 | +----+-----------------+------------+----------------+------------+ -~~~ -{: .output} +``` Let's create another table `files` with the schema below ```sql @@ -99,7 +99,7 @@ Confirm the data in the `files` table: ```sql SELECT * FROM files; ``` -~~~ +```text +----+--------------------+---------------+------------+--------------+-----------+ | id | filename | experiment_id | run_number | total_events | data_type | +----+--------------------+---------------+------------+--------------+-----------+ @@ -110,8 +110,7 @@ SELECT * FROM files; | 5 | belle2.myfile.root | 5 | 505 | 3141 | data | | 7 | alice.myfile2.root | 4 | 404 | 1124 | mc | +----+--------------------+---------------+------------+--------------+-----------+ -~~~ -{: .output} +``` ## SQL JOIN @@ -133,7 +132,7 @@ FROM files f INNER JOIN experiments e ON f.experiment_id = e.id; ``` -~~~ +```text +--------------------+-----------------+ | filename | experiment_name | +--------------------+-----------------+ @@ -144,8 +143,7 @@ ON f.experiment_id = e.id; | belle2.myfile.root | Belle2 | | alice.myfile2.root | ALICE | +--------------------+-----------------+ -~~~ -{: .output} +``` One more example: ```sql @@ -170,7 +168,7 @@ FROM files f LEFT JOIN experiments e ON f.experiment_id = e.id; ``` -~~~ +```text +--------------------+-----------------+----------------+ | filename | experiment_name | collision_type | +--------------------+-----------------+----------------+ @@ -182,8 +180,7 @@ ON f.experiment_id = e.id; | alice.myfile2.root | ALICE | PbPb | | alps.myfile.root | NULL | NULL | +--------------------+-----------------+----------------+ -~~~ -{: .output} +``` Notice that: - Returns all rows from the left table (`files`), including unmatched rows. @@ -202,7 +199,7 @@ FROM files f RIGHT JOIN experiments e ON f.experiment_id = e.id; ``` -~~~ +```text +-----------------+--------------------+ | experiment_name | filename | +-----------------+--------------------+ @@ -214,8 +211,7 @@ ON f.experiment_id = e.id; | CMS | cms.myfile.root | | LHCb | lhcb.myfile.root | +-----------------+--------------------+ -~~~ -{: .output} +``` You can notice: - Returns all rows from the right table (`experiments`), including unmatched rows. @@ -238,7 +234,7 @@ FROM experiments e LEFT JOIN files f ON f.experiment_id = e.id; ``` -~~~ +```text +--------------------+-----------------+----------------+ | filename | experiment_name | collision_type | +--------------------+-----------------+----------------+ @@ -251,8 +247,7 @@ ON f.experiment_id = e.id; | alps.myfile.root | NULL | NULL | | NULL | BaBar | e+e- | +--------------------+-----------------+----------------+ -~~~ -{: .output} +``` Note: The used `SELECT` statements have the same columns to use `UNION` (try with different columns and see what happens). @@ -273,36 +268,41 @@ ADD CONSTRAINT fk_experiment_id FOREIGN KEY (experiment_id) REFERENCES experiments(id); ``` -~~~ +```text ERROR 1452 (23000): Cannot add or update a child row: a foreign key constraint fails (`all_experiments_data`.`#sql-1_9`, CONSTRAINT `fk_experiment_id` FOREIGN KEY (`experiment_id`) REFERENCES `experiments` (`id`)) -~~~ -{: .output} - -> ## Why it didn't work? -> -> What prevents to create a foreign key? How to fix the error message above? -> -> > ## Solution -> > -> > The error message indicates that there is a constraint violation. This is because we have a row in the `files` table with an `experiment_id` that does not exist in the `experiments` table. -> > In this case, we have a row with `experiment_id = 9` in the `files` table, but there is no corresponding `id` in the `experiments` table. -> > -> > Let's add the information of the experiment ALPS -> > -> > ```sql -> > INSERT INTO experiments (experiment_name, laboratory, collision_type, start_year) -> > VALUES -> > ("ALPS II", "DESY", "N/A", 2011); -> > ``` -> > -> > And set the proper `experiment_id` (in this case is id = 7, confirm with `SELECT * FROM experiments`). -> > ```sql -> > UPDATE files -> > SET experiment_id = 7 -> > WHERE filename = "alps.myfile.root"; -> > ``` -> > -> > Now, try again to create the foreign key as before. -> > -> {: .solution} -{: .challenge} +``` + +:::::{admonition} Why it didn't work? +:class: challenge + +What prevents to create a foreign key? How to fix the error message above? + +::::{admonition} Solution +:class: dropdown + +The error message indicates that there is a constraint violation. This is because we have a row in the `files` table with an `experiment_id` that does not exist in the `experiments` table. +In this case, we have a row with `experiment_id = 9` in the `files` table, but there is no corresponding `id` in the `experiments` table. + +Let's add the information of the experiment ALPS + +```sql +INSERT INTO experiments (experiment_name, laboratory, collision_type, start_year) +VALUES +("ALPS II", "DESY", "N/A", 2011); +``` + +And set the proper `experiment_id` (in this case is id = 7, confirm with `SELECT * FROM experiments`). +```sql +UPDATE files +SET experiment_id = 7 +WHERE filename = "alps.myfile.root"; +``` + +Now, try again to create the foreign key as before. +:::: +::::: + +:::{admonition} Key Points +- JOIN operations are used to combine rows from two or more tables based on a common column. +- Foreign keys are used to maintain data integrity and enforce references. +::: diff --git a/_episodes/06-conditions-database.md b/_episodes/06-conditions-database.md index 77173bd..257e2e4 100644 --- a/_episodes/06-conditions-database.md +++ b/_episodes/06-conditions-database.md @@ -2,19 +2,20 @@ title: "Introduction to Conditions Databases in HEP" teaching: 1.5 hours exercises: 2 -questions: -- "What are the key objects in a Conditions Database and how are they related?" -- "How can you use SQLAlchemy to model and query a simple Conditions Database?" -objectives: -- "Understand the role of Conditions Databases in high-energy physics." -- "Learn the key concepts: Global Tags, PayloadTypes, Payloads, and IOVs." -- "Model relationships between these objects using SQLAlchemy." -- "Perform basic queries to retrieve conditions data efficiently." -keypoints: -- "Conditions Databases store metadata for time-dependent data like alignment and calibration." -- "Global Tags group related PayloadTypes, which contain Payloads valid for specific IOVs." --- +:::{admonition} Questions +- What are the key objects in a Conditions Database and how are they related? +- How can you use SQLAlchemy to model and query a simple Conditions Database? +::: + +:::{admonition} Objectives +- Understand the role of Conditions Databases in high-energy physics. +- Learn the key concepts: Global Tags, PayloadTypes, Payloads, and IOVs. +- Model relationships between these objects using SQLAlchemy. +- Perform basic queries to retrieve conditions data efficiently. +::: + # Lesson: Introduction to Conditions Databases in HEP ## Introduction @@ -49,12 +50,12 @@ A **Global Tag** is a label that identifies a consistent set of conditions data. - Each **Payload** represents a specific piece of conditions data and is valid for the **Interval of Validity (IOV)** associated with it. This ensures that the correct payload is applied for a given run or timestamp. - During data processing, the CDB retrieves the appropriate payload by matching the **IOV** to the required run or timestamp, ensuring consistency and accuracy. -
- CDB schema diagram -
CDB schema diagram
-
-

+:::{figure} ../fig/cdb_diagram.png +:alt: CDB schema diagram +:width: 250px +CDB schema diagram +::: For simplification, in the following example, we work with three objects: @@ -258,3 +259,8 @@ for global_tag_name, payload_type_name, payload_url, max_iov in query: GlobalTag: Conditions, PayloadType: Calibrations, PayloadIOV URL: http://example.com/calib_v2.root, IOV: 2 GlobalTag: Conditions, PayloadType: Alignment, PayloadIOV URL: http://example.com/align_v1.root, IOV: 1 + +:::{admonition} Key Points +- Conditions Databases store metadata for time-dependent data like alignment and calibration. +- Global Tags group related PayloadTypes, which contain Payloads valid for specific IOVs. +::: diff --git a/_episodes/07-break.md b/_episodes/07-break.md index 41f04f9..28843d3 100644 --- a/_episodes/07-break.md +++ b/_episodes/07-break.md @@ -6,5 +6,4 @@ questions: - "Get up, stretch out, and dance!" objectives: - "Refresh your mind!" - -{% include links.md %} +--- diff --git a/_episodes/08-sqllite.md b/_episodes/08-sqllite.md index b14749d..91cb13d 100644 --- a/_episodes/08-sqllite.md +++ b/_episodes/08-sqllite.md @@ -2,17 +2,18 @@ title: "SQLite" teaching: 60 exercises: 30 -questions: -- "What is SQLite?" -- "How do I create an SQLite database?" -objectives: -- "Creating SQLite database" -- "Manipulating the database" -keypoints: -- "For lightweight applications, use SQLite." -- "Benefit from integration between sqlite3 and pandas." --- +:::{admonition} Questions +- What is SQLite? +- How do I create an SQLite database? +::: + +:::{admonition} Objectives +- Creating SQLite database +- Manipulating the database +::: + ## Introduction to SQLite @@ -32,47 +33,50 @@ Create a working directory `hsf_sqlite_training`: mkdir hsf_sqlite_training cd hsf_sqlite_trainins ``` -{: .source} Let's create a database, that contains information from the Particle Data Group particle properties table. First, we need to download the [particle properties table](https://pdg.lbl.gov/2023/mcdata/mass_width_2023.txt) from 2023. You can do it manually and place it in your directory `hsf_sqlite_training` under the name `particle_table.txt` or if you use Python 3.7+ you can download the file automatically with a python module `requests`. -> ## Additional: using `requests` -> ->To check the Python version on your machine do the following: -> ->```bash ->python3 --version ->``` ->{: .source} -> ->If the python version is higher than 3.7, you can install `requests` with `python -m pip install requests`. ->Then you can create a script `download_particle_table.py` using your favorite code editor. -> ->```python ->import requests ->particle_table = "https://pdg.lbl.gov/2023/mcdata/mass_width_2023.txt" #url of the file we want ->response = requests.get(particle_table) #getting the response from the url. -> ->if response.status_code == 200: -> with open("particle_table.txt", "wb") as file: #writing the response into the txt file `particle_table.txt` locally -> file.write(response.content) -> print("Particle table is downloaded!") ->else: -> print("Failed to download the particle table.") #it can be that the server is down or you have a typo in the url ->``` ->{: .source} -> ->Save the `download_particle_table.py` script and execute with `python3 download_particle_table.py`. ->You should see the downloaded file `particle_table.txt` in your working directory. -{: .callout} +:::{note} Additional: using `requests` +To check the Python version on your machine do the following: + +```bash +python3 --version +``` + +If the python version is higher than 3.7, you can install `requests` with `python -m pip install requests`. +Then you can create a script `download_particle_table.py` using your favorite code editor. + +```python +import requests + +particle_table = ( + "https://pdg.lbl.gov/2023/mcdata/mass_width_2023.txt" # url of the file we want +) +response = requests.get(particle_table) # getting the response from the url. + +if response.status_code == 200: + with open( + "particle_table.txt", "wb" + ) as file: # writing the response into the txt file `particle_table.txt` locally + file.write(response.content) + print("Particle table is downloaded!") +else: + print( + "Failed to download the particle table." + ) # it can be that the server is down or you have a typo in the url +``` + +Save the `download_particle_table.py` script and execute with `python3 download_particle_table.py`. +You should see the downloaded file `particle_table.txt` in your working directory. +::: Open the `particle_table.txt` in your favorite text editor and study the data structure inside. You should see: -```bash +```text * MASSES, WIDTHS, AND MC ID NUMBERS FROM 2023 EDITION OF RPP * * The following values were generated on 31-May-2023 by the Berkeley Particle @@ -118,7 +122,6 @@ You should see: 25 1.2525E+02 +1.7E-01 -1.7E-01 3.2E-03 +2.4E-03 -1.7E-03 H 0 ... ``` -{: .output} Now that we have obtained the particle table, let's build the SQlite database! @@ -139,7 +142,6 @@ Here one can perform operations on the data base, such as inserting, updating, d connection.close() # close database connection ``` -{: .source} The `Connection` object represents the database that, in this case, is stored in RAM using a special name `:memory:`. If you want to save database locally replace `:memory:` with `.db`. @@ -175,7 +177,6 @@ readout = pd.read_sql("SELECT * FROM particles", connection) # read from databa print(readout) # print the database connection.close() # close database connection ``` -{: .source} In the execute we have create a table called `particles` that contains a particle id (integer), mass (float), upper and lower mass errors (float), width (float), upper and lower width errors (float), particle name (TEXT) and charge (INTEGER). Then, we fill in the information on gluon into our `particles` table. @@ -184,11 +185,10 @@ To read the database, we use pandas `read_sql` function. Now execute `create_database.py` with `python3 create_database.py`. You should see: -```bash +```text id mass masserrlow masserrup width widtherrlow widtherr_up name charge 0 21 0.0 0.0 0.0 0.0 0.0 0.0 g 0 ``` -{: .output} However, we would like to write down the entire list of the PDG particles into our database, not just a gluon! For this, we will use `executemany`, instead of `execute`. @@ -196,7 +196,7 @@ But before that, we need to prepare our input data in the `particle_table.txt`. First of all the first 38 lines are taken with the header information that we do not need in our database, so we will skip these rows when copying. Secondly, the sizes of columns are different in the `particle_table.txt`, but luckily the header specifies the exact sizes of the table columns: -```bash +```text * 3) column * 1 - 8 \ Monte Carlo particle numbers as described in the "Review of * 9 - 16 | Particle Physics". Charge states appear, as appropriate, @@ -221,7 +221,6 @@ Secondly, the sizes of columns are different in the `particle_table.txt`, but lu * should not be taken as a standardized presentation of * particle names. ``` -{: .output} You can check in your favorite text editor that the column definitions are correct. Now `particle_table.txt` contains more precise information than what we want to save in our database. @@ -262,7 +261,6 @@ data[["name", "charge"]] = data["namecharge"].str.extract(r"(.+?)\s+(\S+)$") data = data.drop("namecharge", axis=1) data = data.values.tolist() ``` -{: .source} `executemany` expects an iterable input. Therefore we transform the pandas dataset to nested lists. @@ -275,11 +273,10 @@ readout = pd.read_sql("SELECT * FROM particles", connection) # read from databa print(readout) # print the database connection.close() ``` -{: .source} Save the `create_database.py` and execute with `python3 create_database.py`, you should see the following output: -```bash +```text id mass masserrlow masserrup width widtherrlow \ 0 21 0.00000 0.00000 0.00000 0.000000e+00 0.000000e+00 1 22 0.00000 0.00000 0.00000 0.000000e+00 0.000000e+00 @@ -308,7 +305,6 @@ Save the `create_database.py` and execute with `python3 create_database.py`, you [229 rows x 9 columns] ``` -{: .output} ### Creating sql database directly from pandas dataframe @@ -353,7 +349,6 @@ readout = pd.read_sql("SELECT * FROM particles", connection) # read from databa print(readout) # print the database connection.close() ``` -{: .source} ## Manipulating the database with SQLite @@ -361,31 +356,40 @@ Before learning how to manipulate the database, let's first save the database cr Replace the name of the database `:memory:` with `particles.db` and rerun `create_database.py`. You should see `particles.db` in your directory. -> ## Exercise -> -> Open the `particles.db` file and create a table that contains all neutral particles. -> -{: .challenge} - -> ## Solution -> ->```python ->cursor.execute("CREATE TABLE neutral_particles AS SELECT * from particles WHERE charge = '0'") ->``` -> {: .source} -{: .solution} - - -> ## Exercise -> ->Open the `particles.db` file and select only neutral leptons. -> -{: .challenge} - -> ## Solution -> ->```python ->readout = pd.read_sql("SELECT * FROM neutral_particles WHERE name LIKE 'nu%'", connection) ->``` -> {: .source} -{: .solution} +:::::{admonition} Exercise +:class: challenge + +Open the `particles.db` file and create a table that contains all neutral particles. + +::::{admonition} Solution +:class: dropdown + +```python +cursor.execute( + "CREATE TABLE neutral_particles AS SELECT * from particles WHERE charge = '0'" +) +``` +:::: +::::: + + +:::::{admonition} Exercise +:class: challenge + +Open the `particles.db` file and select only neutral leptons. + +::::{admonition} Solution +:class: dropdown + +```python +readout = pd.read_sql( + "SELECT * FROM neutral_particles WHERE name LIKE 'nu%'", connection +) +``` +:::: +::::: + +:::{admonition} Key Points +- For lightweight applications, use SQLite. +- Benefit from integration between sqlite3 and pandas. +::: diff --git a/_episodes/09-opensearch-queires.md b/_episodes/09-opensearch-queires.md index 0f4ca78..94dc816 100644 --- a/_episodes/09-opensearch-queires.md +++ b/_episodes/09-opensearch-queires.md @@ -2,23 +2,21 @@ title: "Intro to NoSQL and Opensearch Queries" teaching: x exercises: 6 -questions: -- "What is NoSQL database and Opensearch?" -- "How to perform indexing in Opensearch?" -- "How to query and filter records in opensearch?" -objectives: -- "Understand the basic structure of Opensearch queries." -- "Learn how to create and manage indices in Opensearch." -- "Practice using different types of queries such as term queries, range queries, and compound queries." -- "Gain familiarity with updating and deleting documents in Opensearch." -keypoints: -- "Opensearch queries can be used to search, update, and delete documents in an Opensearch index." -- "Indices in Opensearch define the structure and mapping of documents." -- "Term queries match exact terms or values in a specific field." -- "Range queries match documents within a specified range of values." -- "Compound queries combine multiple conditions using boolean logic." --- +:::{admonition} Questions +- What is NoSQL database and Opensearch? +- How to perform indexing in Opensearch? +- How to query and filter records in opensearch? +::: + +:::{admonition} Objectives +- Understand the basic structure of Opensearch queries. +- Learn how to create and manage indices in Opensearch. +- Practice using different types of queries such as term queries, range queries, and compound queries. +- Gain familiarity with updating and deleting documents in Opensearch. +::: + # NOSQL Databases NSQL databases diverge from the traditional table-based structure of RDMS and are designed to handle unstructured or semi-structured data. They offer flexibility in data modeling and storage, supporting various data formats. Types of NoSQL database are : @@ -234,29 +232,27 @@ for hit in search_results["hits"]["hits"]: print(hit["_source"]) ``` -{: .source} - -> ## Search for filename for documents with data_type `mc`. -> -> Retrieve and display filename -> -> > ## Solution -> > -> > ```python -> > search_query = {"query": {"term": {"data_type": "mc"}}} -> > search_results = es.search(index=index_name, body=search_query) -> > for hit in search_results["hits"]["hits"]: - print(hit["_source"]["filename"]) -> > ``` -> > {: .source} -> > -> > ~~~ -> > expx.myfile2.root -> > expx.myfile4.root -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} +:::::{admonition} Search for filename for documents with data_type `mc`. +:class: challenge + +Retrieve and display filename + +::::{admonition} Solution +:class: dropdown + +```python +search_query = {"query": {"term": {"data_type": "mc"}}} +search_results = es.search(index=index_name, body=search_query) +for hit in search_results["hits"]["hits"]: + print(hit["_source"]["filename"]) +``` + +```text +expx.myfile2.root +expx.myfile4.root +``` +:::: +::::: #### Range Query It is also a term level query where we can apply a range of values to a field/metadata. @@ -284,37 +280,26 @@ for hit in search_results["hits"]["hits"]: print(hit["_source"]) ``` -{: .source} - -> ## Search for filename for all the documents whose collision energy ranging from 100 to 200 (both exclusive) . -> -> Retrieve and display filename with range query -> -> > ## Solution -> > -> > ```python -> > search_query = { -> > "query": { -> > "range": { -> > "collision_energy": { -> > "gt": 100, -> > "lt": 200 -> > } -> > } -> > } -> > } -> > search_results = es.search(index=index_name, body=search_query) -> > for hit in search_results["hits"]["hits"]: -> > print(hit["_source"]["filename"]) -> > ``` -> > {: .source} -> > -> > ~~~ -> > expx.myfile3.root -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} +:::::{admonition} Search for filename for all the documents whose collision energy ranging from 100 to 200 (both exclusive) . +:class: challenge + +Retrieve and display filename with range query + +::::{admonition} Solution +:class: dropdown + +```python +search_query = {"query": {"range": {"collision_energy": {"gt": 100, "lt": 200}}}} +search_results = es.search(index=index_name, body=search_query) +for hit in search_results["hits"]["hits"]: + print(hit["_source"]["filename"]) +``` + +```text +expx.myfile3.root +``` +:::: +::::: #### Prefix Query Another term level query is prefix query. As the name suggest it search for terms with sopecific prefix. @@ -360,37 +345,35 @@ for hit in search_results["hits"]["hits"]: print(hit["_source"]) ``` -{: .source} - -> ## Search for filename for documents with data_type `data` and collision_energy `150` . -> -> Retrieve and display filename -> -> > ## Solution -> > -> > ```python -> > search_query = { -> > "query": { -> > "bool": { -> > "must": [ -> > { "term": {"data_type": "data"} }, -> > { "term": { "collision_energy": 150 } } -> > ] -> > } -> > } -> > } -> > search_results = es.search(index=index_name, body=search_query) -> > for hit in search_results["hits"]["hits"]: -> > print(hit["_source"]["filename"]) -> > ``` -> > {: .source} -> > -> > ~~~ -> > expx.myfile3.root -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} +:::::{admonition} Search for filename for documents with data_type `data` and collision_energy `150` . +:class: challenge + +Retrieve and display filename + +::::{admonition} Solution +:class: dropdown + +```python +search_query = { + "query": { + "bool": { + "must": [ + {"term": {"data_type": "data"}}, + {"term": {"collision_energy": 150}}, + ] + } + } +} +search_results = es.search(index=index_name, body=search_query) +for hit in search_results["hits"]["hits"]: + print(hit["_source"]["filename"]) +``` + +```text +expx.myfile3.root +``` +:::: +::::: #### Should Query The should query searches for documents that match any of the specified conditions. This is equivalent to OR operator. @@ -412,38 +395,36 @@ for hit in search_results["hits"]["hits"]: print(hit["_source"]) ``` -{: .source} - -> ## Search for filename for documents with run_number `55` or collision_energy `150` . -> -> Retrieve and display filename -> -> > ## Solution -> > -> > ```python -> > search_query = { -> > "query": { -> > "bool": { -> > "should": [ -> > { "term": {"run_number": 55} }, -> > { "term": { "collision_energy": 150 } } -> > ] -> > } -> > } -> > } -> > search_results = es.search(index=index_name, body=search_query) -> > for hit in search_results["hits"]["hits"]: - print(hit["_source"]["filename"]) -> > ``` -> > {: .source} -> > -> > ~~~ -> > expx.myfile2.root -> > expx.myfile3.root -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} +:::::{admonition} Search for filename for documents with run_number `55` or collision_energy `150` . +:class: challenge + +Retrieve and display filename + +::::{admonition} Solution +:class: dropdown + +```python +search_query = { + "query": { + "bool": { + "should": [ + {"term": {"run_number": 55}}, + {"term": {"collision_energy": 150}}, + ] + } + } +} +search_results = es.search(index=index_name, body=search_query) +for hit in search_results["hits"]["hits"]: + print(hit["_source"]["filename"]) +``` + +```text +expx.myfile2.root +expx.myfile3.root +``` +:::: +::::: #### Must Not Query The must_not query excludes documents that match the specified condition. This is equivalent to NOT operator. @@ -455,78 +436,62 @@ for hit in search_results["hits"]["hits"]: print(hit["_source"]) ``` -{: .source} - -> ## Search for filename for all the documents that is not run_number `55` . -> -> Retrieve and display filename -> -> > ## Solution -> > -> > ```python -> > search_query = { -> > "query": { -> > "bool": { -> > "must_not": [ -> > { "term": {"run_number": 55} } -> > ] -> > } -> > } -> > } -> > search_results = es.search(index=index_name, body=search_query) -> > for hit in search_results["hits"]["hits"]: - print(hit["_source"]["filename"]) -> > ``` -> > {: .source} -> > -> > ~~~ -> > expx.myfile1.root -> > expx.myfile3.root -> > expx.myfile4.root -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} - -{: .source} - -> ## Search for filename for all the documents that must total_event greater than 200 and run_number greater than 50, should have collision_type as PbPb and must NOT have collision_energy 150. . -> -> Retrieve and display filename combing must, should and mustn't queries. -> -> > ## Solution -> > -> > ```python -> > search_query = { -> > "query": { -> > "bool": { -> > "must": [ -> > { "range": { "total_event": { "gt": 200 } } }, -> > { "range": { "run_number": { "gt": 50 } } } -> > ], -> > "should": { -> > "term": { "collision_type": "PbPb" } -> > }, -> > "must_not": { -> > "term": { "collsiion_energy": 150 } -> > } -> > } -> > } -> > } -> > search_results = es.search(index=index_name, body=search_query) -> > for hit in search_results["hits"]["hits"]: -> > print(hit["_source"]["filename"]) -> > ``` -> > {: .source} -> > -> > ~~~ -> > expx.myfile1.root -> > expx.myfile2.root -> > expx.myfile4.root -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} +:::::{admonition} Search for filename for all the documents that is not run_number `55` . +:class: challenge + +Retrieve and display filename + +::::{admonition} Solution +:class: dropdown + +```python +search_query = {"query": {"bool": {"must_not": [{"term": {"run_number": 55}}]}}} +search_results = es.search(index=index_name, body=search_query) +for hit in search_results["hits"]["hits"]: + print(hit["_source"]["filename"]) +``` + +```text +expx.myfile1.root +expx.myfile3.root +expx.myfile4.root +``` +:::: +::::: + +:::::{admonition} Search for filename for all the documents that must total_event greater than 200 and run_number greater than 50, should have collision_type as PbPb and must NOT have collision_energy 150. . +:class: challenge + +Retrieve and display filename combing must, should and mustn't queries. + +::::{admonition} Solution +:class: dropdown + +```python +search_query = { + "query": { + "bool": { + "must": [ + {"range": {"total_event": {"gt": 200}}}, + {"range": {"run_number": {"gt": 50}}}, + ], + "should": {"term": {"collision_type": "PbPb"}}, + "must_not": {"term": {"collsiion_energy": 150}}, + } + } +} +search_results = es.search(index=index_name, body=search_query) +for hit in search_results["hits"]["hits"]: + print(hit["_source"]["filename"]) +``` + +```text +expx.myfile1.root +expx.myfile2.root +expx.myfile4.root +``` +:::: +::::: # Update a document by filename @@ -544,3 +509,11 @@ Lets delete a document by its document ID (which is filename in our case) _id = "expx.myfile1.root" es.delete(index=index_name, id=_id) ``` + +:::{admonition} Key Points +- Opensearch queries can be used to search, update, and delete documents in an Opensearch index. +- Indices in Opensearch define the structure and mapping of documents. +- Term queries match exact terms or values in a specific field. +- Range queries match documents within a specified range of values. +- Compound queries combine multiple conditions using boolean logic. +::: diff --git a/_episodes/10-text-based-search.md b/_episodes/10-text-based-search.md index de318a3..4fbe83f 100644 --- a/_episodes/10-text-based-search.md +++ b/_episodes/10-text-based-search.md @@ -2,20 +2,18 @@ title: "Opensearch Text Based Queries" teaching: x exercises: 2 -questions: -- "How to perform text based search in opensearch?" -- "What are the ways to do text based search in opensearch?" - -objectives: -- "Understand the fundamental query types in Opensearch for text-based searches." -- "Learn how to construct and execute various text-based queries using Opensearch." - -keypoints: -- "Opensearch supports a range of text-based query types, including match, match_phrase, wildcard, prefix, and fuzzy queries." -- "Each query type has specific use cases and parameters that can be customized for tailored search results." -- "Efficient utilization of text-based queries in Opensearch can significantly enhance data retrieval and analysis capabilities" --- +:::{admonition} Questions +- How to perform text based search in opensearch? +- What are the ways to do text based search in opensearch? +::: + +:::{admonition} Objectives +- Understand the fundamental query types in Opensearch for text-based searches. +- Learn how to construct and execute various text-based queries using Opensearch. +::: + # Text Based Queries Lets first understand why Opensearch has advantages on full text-based search compared to mySQL (SQL). @@ -62,34 +60,28 @@ for hit in search_results["hits"]["hits"]: print(hit["_source"]) ``` -{: .source} - -> ## Search for documents with exact phrase "without cherenkov detector" . -> -> Retrieve documents with match phrase query. -> -> > ## Solution -> > -> > ```python -> > search_query = { -> > "query": { -> > "match_phrase": { -> > "description": "without cherenkov detector" -> > } -> > } -> >} -> > search_results = es.search(index=index_name, body=search_query) -> > for hit in search_results["hits"]["hits"]: - print(hit["_source"]["filename"]) -> > ``` -> > {: .source} -> > -> > ~~~ -> > {'filename': 'expx.myfile3.root', 'run_number': 120, 'total_event': 200, 'collision_type': 'PbPb', 'data_type': 'data', 'collision_energy': 150, 'description': 'This file is produced without cherenkov detector'} -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} +:::::{admonition} Search for documents with exact phrase "without cherenkov detector" . +:class: challenge + +Retrieve documents with match phrase query. + +::::{admonition} Solution +:class: dropdown + +```python +search_query = { + "query": {"match_phrase": {"description": "without cherenkov detector"}} +} +search_results = es.search(index=index_name, body=search_query) +for hit in search_results["hits"]["hits"]: + print(hit["_source"]["filename"]) +``` + +```text +{'filename': 'expx.myfile3.root', 'run_number': 120, 'total_event': 200, 'collision_type': 'PbPb', 'data_type': 'data', 'collision_energy': 150, 'description': 'This file is produced without cherenkov detector'} +``` +:::: +::::: ## Match query The match query is a basic query type in opensearch used to search for documents containing specific words or phrases in a specified field, such as the description field. @@ -137,35 +129,28 @@ search_results = es.search(index=index_name, body=search_query) for hit in search_results["hits"]["hits"]: print(hit["_source"]) ``` -{: .source} - -> ## Search for documents with words "cherenkov" or "trigger" . -> -> Retrieve documents with match phrase query. -> -> > ## Solution -> > -> > ```python -> > search_query = { -> > "query": { -> > "match": { -> > "description": "cherenkov trigger" -> > } -> > } -> >} -> > search_results = es.search(index=index_name, body=search_query) -> > for hit in search_results["hits"]["hits"]: - print(hit["_source"]["filename"]) -> > ``` -> > {: .source} -> > -> > ~~~ -> > {'filename': 'expx.myfile3.root', 'run_number': 120, 'total_event': 200, 'collision_type': 'PbPb', 'data_type': 'data', 'collision_energy': 150, 'description': 'This file is produced without cherenkov detector'} -> > {'filename': 'expx.myfile1.root', 'run_number': 100, 'total_event': 1112, 'collision_type': 'pp', 'data_type': 'data', 'collision_energy': 250, 'description': 'This file is produced with L1 and L2 trigger.'} -> > ~~~ -> > {: .output} -> {: .solution} -{: .challenge} + +:::::{admonition} Search for documents with words "cherenkov" or "trigger" . +:class: challenge + +Retrieve documents with match phrase query. + +::::{admonition} Solution +:class: dropdown + +```python +search_query = {"query": {"match": {"description": "cherenkov trigger"}}} +search_results = es.search(index=index_name, body=search_query) +for hit in search_results["hits"]["hits"]: + print(hit["_source"]["filename"]) +``` + +```text +{'filename': 'expx.myfile3.root', 'run_number': 120, 'total_event': 200, 'collision_type': 'PbPb', 'data_type': 'data', 'collision_energy': 150, 'description': 'This file is produced without cherenkov detector'} +{'filename': 'expx.myfile1.root', 'run_number': 100, 'total_event': 1112, 'collision_type': 'pp', 'data_type': 'data', 'collision_energy': 250, 'description': 'This file is produced with L1 and L2 trigger.'} +``` +:::: +::::: # query_string @@ -207,3 +192,9 @@ for hit in search_results["hits"]["hits"]: ``` This query will match documents with terms like "production," "producer," "products," etc., based on the fuzziness parameter specified. + +:::{admonition} Key Points +- Opensearch supports a range of text-based query types, including match, match_phrase, wildcard, prefix, and fuzzy queries. +- Each query type has specific use cases and parameters that can be customized for tailored search results. +- Efficient utilization of text-based queries in Opensearch can significantly enhance data retrieval and analysis capabilities +::: diff --git a/_extras/about.md b/_extras/about.md index a6bb036..4c8e6e8 100644 --- a/_extras/about.md +++ b/_extras/about.md @@ -1,7 +1,5 @@ - --- title: About --- -{% include carpentries.html %} -{% include links.md %} +This lesson is part of the [HSF Training](https://hepsoftwarefoundation.org/training/curriculum.html) curriculum. diff --git a/_extras/discuss.md b/_extras/discuss.md index 7975806..524440a 100644 --- a/_extras/discuss.md +++ b/_extras/discuss.md @@ -1,8 +1,4 @@ - --- title: Discussion --- FIXME - -{% include links.md %} - diff --git a/_extras/guide.md b/_extras/guide.md index 4a6b833..da9be37 100644 --- a/_extras/guide.md +++ b/_extras/guide.md @@ -1,8 +1,4 @@ - --- title: "Instructor Notes" --- FIXME - -{% include links.md %} - diff --git a/fig/hsf-logo.png b/fig/hsf-logo.png new file mode 100644 index 0000000000000000000000000000000000000000..8c1dbc7823c6608c5df034ef80cee970d8ec89d6 GIT binary patch literal 23188 zcmeFZbyO7Y`Y#IOr~`_st_c`nSeb>6{xRy(pdGm?SldtbyC`w^tKEgypL&KJl7FR(-gGiyFp|`>AfKO7M zW;KI<&|Ordo}-lwl5T)sFdU_|UC_|52vC2~(NaH4tA`@FC9$G zSUv0*4~BRLx}3{2!8N0>SH!4_}?L}HbPWd3NPSd4$fw9 z9#(Evb}C^^I2)ylyhj#}5)#KFx~h>8le(ZB!vb52() z^Z(h&-sNA{0vBXMeZ$7V%Fg!hwZT+D)TjJn4t9>tW-cyZePJHKza#(av;W!WpYbm& z9b6p%6`ZY1W$azeoWW#QW7O${IsY~L|9!;&zLt`+l^MA8-`O1hn*E=j{cF4+8|vc! zL5P2-{P$BpW?@W0wturs7_(xTa}Es+fhHp^s^)>dla7&0INsP^>}5`JM-BJcQIRr1 zg7E{j7f^HDd!F9Ou#dO$%QIADL?8NajcqESFi%F2$+8yn_o3nAR`D zPipWFgCqN0CvRRkrXugn3C#(qkE@S62o0sAmhm6^EO;uZAD0jDx%F0Q!O_rR2s8+; zKN|G^{`i0C@c$h++H)>-5(jzWgTkGdb4|%luatu_}J(3A-*$b}~N_ z>@vT9!QQz^KBR!cV^TU2XWT+Ze##QW*J;k^eSflQ{-2@EOYPR`W-kQkzMK z9#a9*-1&n7DOpiqMz5<;P<@(VQeVOWo{@}Gg58{7rj?s1Sn!UN5i{AfYMukr3Uye# zPtJiU4T8nV?r!a`5Md`zskAzXk=By^lUNf8wCCA>p{`#S|udMQ*^Q;g*+LjTxLxg@?1%Z^LPnCz9QE0!z za%njB!~o(vH+i$2Tv{*XYWg5|cUjKtmh<&~Ob1?-PjuXwiK9#7i*FI@{EAk2X(5$F z{1EdN*`nV)r#H4sbh_mV>xf`(Bd~Uqune)t5(8O+r9D^9vBH5D#}Y$2t{l+|`*&RB zeQ&Oe@GW0xjVAO*W3eTEd3hvEOWlEcVSwvCl>e~brfo5fQN1JCt}yfPY~k0y$k2gEUn_&HCQkW%A8O;K(75igun{ z<4(q6gLj&yy;F1ahw;d=UuT5;rEx@-5L5YH!T#8zM?=H29-N?x|A{l@eH&Ru+C{_S0=&;h7>o zWg=77EWX9+_kMO3dIl}7SFxGl53{Z;^0F#8mudQOCcV34ie7v_YM(51!c4mj37|46 z%RwzA-lwed(l*Gy0NT%H%jKCOW58x^KtcVr3Zpz1w=a<%DeTfB6G+x9=(`|Gso zEvc{S#)%*q{aVm>iLWnv-z$El$*lO!MOr0>2C;%A+dSEcQy^_Eq0Ug|r24KGV{SK6 z=aXqgM+AES3rhBHydJQ5BY)tf&HTS8p;MYR;89pS*K(gT@G8^lB1$$zT)t^XIgNns z)}yCl@L{NYczNSOa%CZ13Pm7Yis+R2)kl>Xvx;bfN0l${ETp=rAk1_uP^_x#2I~6P zC?dEqk|~r5v-4tZM#)TrltufDx|Br~v3PrGR8n5i5mB|ob-yGq`tFm@_~G`yjA%e4 z22JzN_ob{?UD7v0DF=B-;VJvmNNTsAs#;%Tu_OkjgB~J{biTx8Gc_t=esbM%MaFg0 zPF@|7h&=h18Lo$s#2V|!Vd61OM{($@&>qvUEcKMCRI6x^6>4sfpns9YGt3+|{P^09D-lIR1e! zfx?UD{(-oZzyY`?-Qh}|^7eKWMw}Y@dmCXF`4{;iB)HED%@L#t)&s9K?GK@9e`N!8 z=)pQRRdO!jh{MTdudkfYPDDq)UgoxBuhfU+ma9XcaCb1O;~CJ~^ki$6s8wvKi{cm= zvpGwvF)C~?B82F=!+S8qIKm{ebTdXV;-2do#MSkuYF-eQ>f~&ZI1$b@<-Od+W1oDN z(wC3>GeM9p4NVRRIQ!`)w%i#Wd!VPPgjA8^cL1KYP2T2d8`&lS2um0|>ba^qREG*G zU!R*&%fna{x21-jFq^1nflOkIp0@v%T^SUO$hXgxU!s}ZxjlqIARHuqFJK#rD>Z#q zK@3r*Gx}!t&{#dRK~~88>ZU2>9)-F_or~F%I{oP7uM_+jHu{~FP3dvl$L!HUYe>xZ z#i1CpAJ~=Kq#1Or@Bx*~TUD)L4U!#=OJP&nZwoRBUoB}`AHDS)_$C=n?Nl>JgF#E3 z+%&*X`Lo7X*W{}1_@ZV2RygL9NxrN#h<&qx`fd#!m95$qe)Up=Cg00vCB2i&pTL)p8&X>F~kAW@2w?|%cL)R>pr}v2@CTY@>u$^GBLE^`KXuDZXt{!5sZ8kOU)7&Fd*VHZiat&pc$AOV zemmUd8I3&g%zG7K2xysA17n9b*KDSZJcrXm6x`f&XB792-+5d<<|A7)&EJcfAM|Q& z?oK^@=jlBEzKfyUG~QtUE59TqT0jRu)(XRKb#Z^(-%0*fEYA4*7NmVC+vRJsEnVEZ z>xa~J6&l}`H3}|Rq#c;U*qC_M9$+_uOETaycyW0WepI1@1;ut;?k`J~=+~b=!|2A) zvr(qYQ1Y&sZMsfBols&(VN@bVro~UzkeogJ#27u*11|bU+CPz+!gv`IB1+3%oNvCc z`=_bhhO_(Dn@6Ti4gDm!UnHYygrmAeyxn&Ax#Y8AUh5P@3FvM-r(Na3hdzMC^P}rT z$zI*dx};S=l%G_jZ56j$@SX6Sw^K#1+aj;i%#tXOQ#1T3`2dA+9Ifs6Yx#ArGny`pu*j)BUsq(H5RagZz z^^cuK{iY2b_tPYp3+Y%jjJ%f3@3Vb3f+gE3`xn-pQu_1GKrtS}aeW{I$EJ@7-hFM| zFd%$*7(AYscb{tD1xFw zL7LE+l4KXRFBxt)GouBg*iU;7{l5S<^#f%|nXb4CiKWUa_`x($qU#nI^drTXDYrR zVK%`cd8VuEy(c_vlPqDL5Vg3W@?G!^Vfd6@yDdQ>}~2l{IH1M8mAap483R z=nlG_gqHe_vD-}aVk6J)0umxRU_sXS4gQtXq%de0Le=iLoTqKQ1#x?vI6^1fe9G*E z4?A1{8Aw9Gfg*}Dtv|19-sc-!7$bTui9Wf!@ym9C@Ge$MOS|@Gxt17@0bq$j0?10V z-?)j!Qog_qSZ%7bC}Q5}eRKb09ezAX&u#d_tui*}M4ZGE`mh@C&;m;0^6R0_J+Y40 z5UwqoHv8^n9Oy5A)kSz%*A#4JlD8 zCJYVINZ{eS^p(X+!$(kwEq0FUG^gczA^IeR)qM8o7Clg#+q4R_yi|~q9#Tl>AxHe;VG1hBxeOnHXx@>{r z558z;x+SuKac^U{?ij*qXzz7?V3;Tzwq&2Kt|`Q`%d1Jmh3S@4>TXi`UgWBBSh@ej zZ)}t@sY34coDi~oqh$Ahn14{FGM+0GoiYoo*GRrSpnoJ;BKR~|W32O!6Cr>fR!C@$ z<2dn;L^ygRSCc50kzc-+L=wBTd&s?Xb~ELRY?Cg3LzMU93uO09=CL7e&wKwAblp99 z0mh*(dVOVOH3G#U_b)ETg0YgC%n8L%E*3-){=qZ~azEiWeDO}e*>ZQZR;&9lw*3@S zk7WO~o0wep8NdmZcPN}dY~W_Cd~)4PjSGqBNUbgQD|!1xl9+m1tN;4**u>x-Wki@L zMJ51UiEI=ja$dIBn(b+4Nno$f0X{?`e(u{CXjE1aNL~VeA0f{F&bl z%O)c_+mr(Rd~0abpo9rNk(Z_EWRYc-N6m#sAMRMwQAB`{;d97!?!@thsR^<5C27 z0N?TWg=NVPwRTJ^WF;3}`_y%j;q~uZKw{wX0QQWFSWF_H1k5+M6&VJHPoFqEp5Z4y z>ZAB)ok)Onwmgz8vD8_wKB=$oUoBrUB-`nR*CVZx_RfA{QJ^l_DGLZTkxISWgR3>y zdM@`Uv^8P!HYb;nZsk-;1lK_-_X$A33UKn^);sS+j-W?_7X+6KL8iK=PwQO!kG~KL z|2=dH*sQ-+YWpMo@H6TY;uh9WJ7n?OuOWKdwG`VZ-i1Pd{zgA%Q3)Bwx7=KxGHlhS zZe{a7tiICKnLn7vAxBxk^Gc4n%58}sY*xdv>n$6a;-S>-(#v-tRv0vG$?ny~k9S@b zGrhf`Q~q(#w zN*R=au?>?|P>CAG^H>=cV`k}_NA`6L5Lid)J2fW>SD@4oZVg@62-k^J$r#4oR{Q(rBe}X<|P=M5#?yb0e=h;}^YdHz{XB%p;%|9zR zlJ+54d4$@^rz;TdOqyg(Cn!FhGUSNF1$8abYLTVF?RS^)Q z1Uif0_sJcyqY(lMXqY%WgW>ScO!ef0Ld%A~SM`FBFJy@gy@en;2q2Z5-CH)%iPTEg z_jAB~7;%4}-_ggs$4Cu5pj$lJEVZ6zRi97o=0`2N4sc7N-kwq0AA`oE5~t;!QtuYm zV2u@d``APuz7YTbf5bkBZ^F9ChiK)qOr|Q#G9D~A^{$m@HH8-lv)@Ot^54X?g>KY!;-o1~~BJl{oC=()Cr%th~ekO-w zK^^2-*qX2-_zD3xK}kpPi!cImh^mgx{N(Z_+v`V_%iCjWx&i~L6eMp^cj5qd3RP$5 z!XQTPuDjlF`x-^m-eE&A_4i9!@Fl}Iof!s2fYvsJ!CgCnL1V7dvWHv$OGPnJN4Ji1 z4vIsp4F9RFTiKDbJeF>^bL+kli1&v*LY*HEMR+u_BwT6yWZv?vTW+NTO1>wycF2Wf zbwmBF6rX#4wH+5M6~y#DJ1E}YZeh*ymlvKL)la>>Mha4(i@2d^;A<#TELBh%qQras z)~1e}hdyR^<$c?t$FFN9eJgNKH%5h`#^s_*I6?s-!B{t?%A!MODltKa;cN7Yvz^3- zydYx-$5jEhiqRl7mJ0XZnL0gJrs{Tf-#vTTHQ@WF^Dau9Du6fz*RF;N=s+c-ybz^Q z6c@@27`e;it?pqIZNJe`TScI@D&aASA`J-9uGZLmccFX5l)TLQjC*o&ZzD!iHRF83!36drFTDgCqJEwCbq^*a)|5P#EztIna$&HZae2Ns`GR<~-a zr|2L^1N*Xrf%5gXZ(Xe&dXc}5v?nH$9h_Ur+JM9aubm}MYl zD`vzO9E`st9m&>8R+~1F>C9z|85MXW!eF&LY5&K@j1&B3_qhWY!)J&YJCK+6RrGBO zxRu&(SS@C1oZEetaZsF&K}o9G7()EK|GIHc!Hp|z&~9*h z5Lrc4n_dd{&SXR8GVC>ok$*h)nmntmtxt}}@V@1@#X_tc>#j^VuyUQG3(BNj*^+BD2MoF*ZY`iLcM13BJqY9! zhO@&U`L9c7S#Q{obrbhG`?GM=nMX4AGyWV}OYlcyP~VLtxCO}O51<;N#}{-bfF|I3 zX{o-K*UsgyHL5aZ+6dn{JpuRC?WQIUiHKX1sKo_=xj4AT5<!Z=FlCYw3>&qWD8N3V~pT1FzcfX#D zmT0om@VTnHoxF&PlC}UUe_l*AH{5&BJ9I|oL#*|^IFkgneRl_Uli}p?t?6WP4|0ufX}gnbu=+I$D;I%}DKtKj94g>4Hh%KEO_5r1;>_V~Ki6f)VB_*`1w&hWjob*!|P(ds;psjB-v8ww;k@PKbtCJTc> z&Il}Rx4ger-eQdrCadv@m?LvYkZy`j%)Q)7p#~`J2h`&7X9O|KL($0EW-TX?KtMKX z!fV$S_x{mKU3nIn?&De-A&%;N^Imwr69ZWMe}Y-UXoTtO3j%j-+tjA1d1eE`M$g^X zmp6`mVb97b8uT@NlEvS9!TyG-gh@cB8ZP(mgW24`(~4BrBA|_#(R;QV`}z1*J;C66 z1vgV<+U7TJo0D7_b@Il}C#ZX%1L?aXE`A3V6x|UYoPJY8($+=D^F@%LXr1S_|N9F2 z4gJbXg-E$3>+3(8J7Pp4pMh4!0uc`r=TU*dY-NpZ8eKHGBT5DI*os@7Mq5P}mTdD_ z>-r{2Y7@C9H5SIzMBGh=sp&w02+j|P{G(7#780NUn;I?XevA|tVhm5%2(CRTQs9Jo z2R0!?Xh_!{D+=y=tL`Kdr*3tBxa|G-R^yJKZ70fGR|H~kfJx&G`}N!i<>;@tpFvwI z>S&WLMf-DzXBDNtmI};!E+5yYT7SFF_4FPX6peVF46gZ`fC0x3uZ1OGhsHitn`>3a zq18y}alp8Z<{?z|DS~e|ZFXkn_`Jr7U_tL>zBY0BL&CDLXn$R;DJ`q}g!WD8L~cQe z*0ch4IPy9heccNU)Bte(I~GyYGy2rLv)XHdFZToXJALiJxxPg|y9+#BsJd;HLW9aT zZGMlVlLgs^bSmLYe`r{@w9oczQoVgbP1C1XRru&sm6%m;^Ys&6pny~o{`R%f>h!?@ z2zzKg2s_m@jH6X;`WBo8sHPgnR_fB;JC;>8PqvwvFW48Ara#yYeRm(&8BTzaF*a|f z;ER;#hYjrp0?rj=;pt4VN3n0qyt(Y|IHpb4lNFq5yEVF54dx!M?LISSUVwDq0c(f> z`AFLL@$tOJKrp8WJ`aKtB7wM-*bHLeqFY4qIG_8G_2x%2Jo^iu(gs?5X{`JB=hn&H zO(sfJb(=H7b$&7o6+ka94ThuNZA6)|z40-I_L>YXtLpII5|UvtjlwL}je3K~g>9cc zzi4I}o=mgp_GbB+>>2yv2Og7A-Z#2*Bia&zvDBkDgPoIRkiZgA)T%Jj4(gRw_NK|2 z_@3hy$xuo79G(%{`f(Y;61A?uoNLZ^2ZBdCJUnqsFQ6gpe)ToBjwo7(gXgK0Qn$>d z$-vCGLt3Rw^`?#R(8X-FMqm70NFcpG?;}C6CkVI-^pUQv2q%HmjN-J%yWD(fqOx-K z@0Bw99+wmfwWfv{+NwT3OU}>jaT`50*IYKK&a9|@m}a=Xc0{E_EEU4+?Nm8|e1d6MvIi(ksWqyDzo&ieKeJDTqj=-w+9t`JyZPk1|dT*ytp!MEz!iJ@${FAm&zJlYeDI= z8N|8H`z0XCPG@Xh>hRj4BuorwP)boQ9QSuLVtJ`7i8d~ZC0spOyY7b2E%s&4rc6D@ zru)S)ed|{Ld;9ullM{-+qj@s`CW!!1zs!wf15T&EfjN>X)>W+}+m?c8XFrsP&CZD? zCWJ9P)zQmvSk>Mo&ALd*3l%htnwwJtA(I8R_Jzh{Q8Bm{H2;aUA$M)-7IU3?U6M}R zZao#Rl#18Vpzo&F#j=1y%XOmo2+zQ>jk{uIbSVJlWB{L?YPw%B1N;fJZvOb`uI-y? zoMmiUH>>|F!}92{53a8jACY+Hwm&(xa`L;siQ{gA8HoVMY~28~x#sGohEgdR$?76+ zp4Poe=d=CPTAYver!d1_m;CH`8iZdXrD?V;Uo)ZqMG`U`UzUE5LZmI5rdN@Y5|UVt5r7zm$!iUtMdhP4c!-ynbZhmWmzhnM-(iv5TXH%{e9`1GFrzMKJ`vUGdXk%} zJb%{735+QnFs5nTmABDRwzn+LOkK=o(&16Gu=`Mz)^@SAOlrzgQ|L46vOuyY^!v+y zdcUR()Q&&n86V$y@YzI#1|}W`WI|Ul6o93Ds=N(yhkSU+Ee)o|rn-}b zUqbFLFq-QoS=)|)%st78yt!VSSvs!L(UXbfTzYfB^ZT!3Ln$E2|B6k?VL^cB3RNW^ zOSkB;O>JIr>D#AbE;`V16z1MBD&*PtZBXzEux-J7JTir_s)$+-CY}h+UZpPg8I^uI zIofa-~{IQ%PH$YJQF=ObcU#2y(%;=`9g5m?>m6Vm3(69Hl^mZ(&9L z@m(1SP6eJ@JsE!e(p&EFC{L;f)oke_BhwES9Wk&Wu^@f=D9f6x?>fs#^tDw)Fp0lJ zrMpc=vT^b9T$gk70u_F;pvB;$#`Z6{+ONmA6V@X$^BRK`V?w7|LXAClb@rStoT+xb zW#;VsDwspAvy6Xs`n}#IybDah9f)hO9pbYH$`Qs$){M7AJJrsqV?=(A(zdYX_a_b{ z-7_+Mj}}qbyu)2!vBL}!@|sVTQt+)rd9@5kanPTE+}AHU1zW(X41!N7Kd#?>NY;Em zJENy7@Y!i;Aufj3U{uv%wegUOmt7U@Q|-31#o60}tt&fXo*K8=yj@?SlsqlyQAdZY zI-;=UQMXS&hrsXASMaFWa`z{|W((t4TH2(sX&#Oj9r##u-R+hW_mtA1=z z(-w0xrS|Nk1Z0rbDan91oq>rYaqj?6w*N$^&~ynZOl;i<)2$Kq6Wy%Xp$B zA+l7zz6fdeCPhz0-~L&e;pxUfh;Q1m{AQ#8+xqu9U2fz_;zu@|+x`eNl$Wc;7z7}d zP_aXvweVrX(0sOS&j|-oSbr2_fO%GjJz5P*MSeq9F}q$Y`kI2MpWz)Qw|;A|AuyC*z4SxxP z#sMKjrR@M1qyc+cTWjh3eeYf|X}r!$`&1pb<%P}G1Pwno`9{1+_ZvT$ zcmSA(Su6bzI1V3R>*}<{kCFVo3sdure%cb#^IrQqEXohlXaZ{T-b-H^luG-hYw9+u zDk6Ho<#JIO`taaLAO>R?xr1qh;a!7qFLX$QIpt`4jME7Wxr^<$HM3uk_o4-u;37zg z0{MkO)}#ZM{g95Q#(g%Me(r=RT>CxG(+$?!#G8D!M@0$B`{GN=ts%dfPI7EQpm&}I zAmB6r+b9beIsv4&(6i^&+C%|U%5A&;n`|f9r&5r0f4=w?IrrLVBZVv7YX5w}s>{6R zXHICg6oCI${ycFYuy71Z+mWho6N8yg$$e%m?25NxwRBesrtgoAykW%M)AVhu*Cwi0 zBm-1~07?FB&_I1urlV~sOdREyS}X83Jkj9LP->j}gh8hnj2)sjx2ibXR9|Smt;9_y z!ygUYq#j9N3nTAe03!lB!b=8@R$jsE;B-rivy{RU&)8LYFJ@auQf4P77rW!XVIrpP z_E)U`F*peL5%k2M#E`MDcEGjRF}zB?-bHUmnL@fK5iq9; z%I)#G@{R+A1Cr>qG|gPuXJo4=Ye~~oT}8=Pu7PFiM{h{!D@8jTKs3_+4izVd3I?FE zgEVo9^*aYb>Qe%5=HhngN#sB4gzI`oi#0(bmb2<(mcNb9QObGK$KUx{x z?(xbh%4O~cg?w@vH|~5BI38~XL}ooO?*QtsZw4@+MAjA#s)q+EV!GXG$S8ey>{zR@ z6yEg@w4LPiioFjazL%j?4c)ZRFyQg25pMIMQkgO7{;Oinj{s;`b$$4rXx5%8RA2uZ zBy^-6Z2aORS?1#9w~~itD+Y%FC=va1+XO5PkD(4= zEKTRy(K_>e^l2`3nX=4W^>k)3ZL9lflQp3!E{p^arh_TS5k;7L;#rlo$F*Stb;-Fh z)E4*RC&RB4shmoec~i$gefS3r2olX(RhWPbFF{W-1zt%^uj-PUPoEfGu@)_BEn7@Z zW^C6RUW#_Gp|k^)w+BGo2c;WIsW!hzcjtQ^SaY*owShj+Mewc|Suk$<)JT#O0S`i9 zAF6u+bzqtbR4t^qi1dU9d&Kui<3MI@_0`)x{I#=XEH=<=0E)kZqPD(Olc=4}@RgH#S(+t2*?xmy0DApg+YXA#=o*q+;t2%1+Prk^SIi^7Usoferax*|h0F#2Ai&*jz0^ZVIRjllNKVvQ z4#j)1BCI{Et|#WCI6jrS=Q?n0w4nvkVNbF#6o(nD4PED#w{f7??7-KoZdfw&+`)ERokI-vW*fwNMA&D zpK#!y4*}T_l!~~8;=pH0Bjm)gf#fpp#8a2k$JGgc%R%K}LMEe)#6u^j4*1| zv5eZvN^~a9j|ez00XZ$PPJmV*9f41xq0FNieO&{xL2dqBwkt(6X>_EOnF`3%&I zLj5yzwthrRlNp5MN}azt8WFCeT;u_?nhdJ^n~$`FlCJTviH*FBiQqc`l@}op;U183 z{4EPWBM|*3YNUJ*mKcn96nQ-XO0Hl;{7s~FDs|{O(Fo^s4u#^!qtXV9lfq^o*)JRC zpI(vVq3dWN=dfc+2AKzQJo|{Z{wCj~Ix2aXdXQZ<7xR+Q$8lgMrZZJEG~nVC;)QDO z|C~>p-aVcWo}gT00Vk~YuKZh2afFigooJBqX(%mXDj;xs0CN~{v|j!#_1uPR9?tZS zI#1nak;;I4Yxpm|;8l=WV1cV$jB9vVQ2@540=C+8{!^1+hhX&0^!3G<)Lqs6Ex!C0 z*+J#9SmA&(H^{7gCC|rJtiuMVL^*~j1yBiu|39raUZ>OK1|fvuw3h!T;oc%ntsQhD zv+U^SS-(%*#0M;DrjJI7`C5`j@l89jUuKhHj)50!qtHCB#Ct(N3`c)Djp3 zoj#VgZ;RK2!zN%-7*sa?>xS@QZu@?6ub>-5xme}S)RS@_BMOKCP!CT9k_!~K-XP%i z>E)lNMx^FhINO|mWoT5tBCojjKWRuHz!KEMYN}j{o$Ibcw<>npC*>XXPG)iL!xuRK z|GNEwxC6DHf|3e2wXSHH8FYV4%%N{Z6)gXNJnXN#X5N3*89yL}XS`cGifi2F6kl?1 zSayrC{0f}L7r3=&q7VO;i^C9z!$;WFwrgwG4Rtqs#-eKi*MB;(pe?{nK1gJnK#_zn zvSZV&*spN6hU^N~5jE}i`+tX^4hzVD8xQ>_fq_HMNcqaKygI77=}?g~J@Hx54lcmr zpGzk(I}{uj7LTvjC}842K;o&0WH^9qjL>DBMX=U<_)6QR0x15s z6h2Nr2wwyRoqw+*2fDOO+w%BG{5JEVuj>YKqgTvMx~}REA3~H-8COsYzXKTa5~Cy& zUBLKi!T~MHZcYH@yR}~};rtif1>wVbhAu&Ny05H1#4em3xDl<0v7Kf~CkG+mus=Sc zx3h<#e+yaLpi8M(NuCBDM<7b|=WA_~iShRjSag<}4N0H>ml4y2a(`R*9xy$UqFpF3 zRhCnq)G+NWb1Hr^6@b76(Dz1*`74Nj5b#?8`Kg;WAA*kM-14mCYudvf29Z_oaM=KJ zbg}i&0o4Gw*Uo>*LOzdjBE$m`7e9E!gx5nCmb3<+g{xC_*Eyt&%}UbFK=6XI;$Wz@O??Qe zFr;5kcOAN-b)))I@J7Oj#RflaRPT0m|4o-AZIMLpmRRY#@)6q&_!p0 zJfX%eC6lr$QvxR)T1f#$7VE(&7f%LD_QoXhL?TEj1vJh)$ z(WHw##?S|ScE<6@$pWdV7fZsxc0u_<7GQjy8Zw>;xMv?aT6}CI%k|&?`AC}1YD_rw z5o`1LMdcks!(BdH(*N>J-$-S7_QnsCu2AU5RHX085n|&Q1Z7I*rnQ+)pPbS}lwny% ze7zlD0j}up0PvcASdd1Dwp-A4=ahmS%QHDAeQ|F`VQX68%4Zn zR@q?4YuU5M{5D>w$Cvf2^wZm5;;l(rAABdRw&VXG^Ox|$Te+15(M zdU61c--ySsZXTCr<@`6Yh(IJfxe?vadM0H1AY5ByW<#XJ=NZ;^@sA1O$-3>6s)MP* z_r&!>F4dP)Z_uE^w77AZ(rSO(cCeul?B@j|l#lBgj+E*RqE1euHMKGeNlkY6x&4*) zw&pBT$M)%SzI{_9xfwcnrpNTu9GE;`pq86=gQ9_)E@I5)z?6P-_R2;IH91WVX}wm% zJ|0mps3RI#s;Zh|d{6x9N&gmt6G*5hLFkGP^e#9(E(3W$_^%mNC)NmW@2KxS;&Z;>u(ndoi2tQQt&jLy#{k0JyX)57^O(=nyROYgq`O`}Mo-H0f+rBd}DHS-U%^Zo5N913$E z`}G$#$*ZD|9#xkzl`1@%IrY49zZf0L+^wG6W9~ElW@qL(QP<063sS!2oK=9D=)i%% zM9|_wpkauPd)YGd;v;r)VF|VuBP-J?!m{V&s%0a7OrL~_RMj6t<1<96cUBzAwlYr5 zU!BS=y)+{;sG?e{~9jXUxi8CHuzBhP+rTYxLrLvT{ z^20j&`DnjX$4LzWm49XA$1#56Qk2yTUbwVW_Ce4mVUEkMuxUfYJ8FEvhSv}}WZTPm zThIQsfrRx3s7`-`(JgeYtv7fw!LZVRD8_5FA7^IGG$O*Bz$Hn7=~JD0yDag@Icw zglx-A9}`^tVX@i3>lUGja%XPgu`gF(5@~{mkh_1}ZtxkJ32t|=(EeosW(Q&MZS zq$%v)^nUR}K>++kDzv}j)HR!t@so<~XXgQnC=Qb{nx=<=tWombk`=i&L^AeU?TU7M z-z_NSo~r%tYIZ@3*g6zSc=4x*^LI>D+z(f((7R`yIp} z3n!sW);im0afk9-Co->akzns(0(PavT+Fz!oA=qdH~hL8t{1{_IQ4W_uchnSvej~a zogVn`wX`D&R=dsU@Di+EtLUY;<=;HK^TH_DQA-A&U8lQ+tRr`l8LRk`Mnd-Eg-Wy`D~0DXj*GYO9Qwna!xh=+m;FBnq9iykI^gnY{D(^h);hWJ zavJ>T2{JEMs^&r)Z5Oa+1|R`Fxa_6UU{&G(Vq(`B_!KMvjez}Xv2cm%%Cd?S zG?t-mbaUMJVRCi&5EI$DDnc?QMT6SqKGcq*GR5Uv_~Ex2Su^YLUbEM9c;-zHtFc#x z?H|DY$aX|n$u_?vkXj8Hgal;aYZZ5>bV+XB$Z(AKrL!nC`7rRl1J}GJ0l6d%9+`07M%=-rgjVGc51| z)n?>fM6NWWtKsH8^9psu-1FP$D_IJ#C*u#ePAn7S&Z>^A8>K@0RulH|Yi)EE``L~o z+9&RQ?z$WvqNzKW%V6r1e@1;cf}L#w@wF6P>hY$m49fI>n3u`K8h3MQXhFdGy=D5q z!|4l_h>z^z?SL*$uTLXiWRV_=i;B~iiFe&OHptLjGP#yFoxZhK5EWkNUp=FiXd72KI?q zyB|roUuOcd+mdCdU2k2dzT%){{|`j1_>j9hn!R!FKG`$ zUzhcT_j=^TdE%oQN=TCZ@ua7rYIQ!-L8%KauUZS7BL>xGfkbm_rx~hlJQzKQB_#cR zjOhG=deuBC+!Q=Aj)u}8SMX-k{P4@JV;+Vqk~!sZ*29HLwGI6|T3J73FO>~!T5z?j zKLA2rQI`PB`EEK&=3skjXfPTwbh40q2_8B>bmP@vOt<6+T=@ICJ1E$H19S1MhL&|W zFK9MHA7vUhTG{I|dBi4_Q{QDzH*Rq5SR~|tYUL+_^^J{b znbR;A+8CxBgNc`CYV*my@=-Fr=BIe0k>(t znA@wvk~jU-uoPo0Z(4XUf2*u{7g8%pI?a}oGt`$b zul#VHobzRmm_??KyKcK+z@-P$u0B$y)$7`my1fg-(NWE4>@scPa+S;Ew;fbNc&rC0)iL|n zeT?r zQ2wYafR0el+^h%U2v2$Rb99${kFVE4I<)M68yIUkr9bo2C>4|1g2?24HPP9#SLZ8J z<=ZuE^FG5kzif`b3b@dm3m~$YESu9=VfeJdp$*4y8})L}7al#259;Knfb4DD{c(Io ziZFovlI<9{ct1E=e0=3S+PNm1)l z#P(Hrj49y&VT1;;j^$XEyw&dLcTgWjHD0>P)`Z?#D+j-=LdsZnN_chOT7xXx5GT;u z?qSSx!#mh$C8G-KSmzDpmuZYN1`N#17u~Pg4$~zXXQY#qZW*^zx#(_vGiD^q9ui3P zo;iv8n*Q*DXED3&(#Lgs*JmF)Hqs%Fyk;d)cmDlWLouL2J#w=!sodJ=I_FXHjc-x< zubYEs$`H5N2fDWVS>$Aj0@vak$xa@+IOZ=}nv_K7MOM+@ILVsJ>lHL`_pN^Uu7#;@ zBRG?!=2u>L&3|#{#80V?E_C@|r+(pj?73e;YNg}m!Bw{HPI-;&J}cqdx4$gdO>bkd zm@h_gDJA$Ge5W_WTk}aMBFeHV^!->>bx_GQBz z<5$+$#>pv+LkQ%+S&m@&)}x&le70wEa+w1J1=vhO$n$oYi`|Dkie(pW<;7lmt*24L zJi^M|+l&3w$qt$aDfYN zej6@9gnUBoAr0>4rm3W`exTUQ=Qi|hXY!Ma?|;v4IDsLo`D`XLnM3US!>NgVkdI+tHSKUAY;*Z*FCXV+TJ_I`ugbYr_#Erqw{kBR1}6F;%CjdtUxC#_12 zI+@hGa5LInC}Rn16>WMT4aH6|+zPx7-n~s+OIow}%_UH=B6I6y@d(>h<^~qaZZO-N zcf`z1oZ?dXM$+Q@LSe$dvje{u&)i*HIG1YV}^xUca5*@!O;iS#R1n!r4e7Q#J*pR_3hMT?ZGE$^mi2W zJlU%JGVX5@MbBN_q;mFm?F^Ncminn?h+h3a&0L2&8;aMiTeNmnwN>l3s#YbnXQ^5( zVz*Y!A{0>-(jfebmYP*F_PAnGjTr>JMq8T@vr)t**N8NU#OKfW&R=kz^PY2_^PcCt zN6{`qVx$-`M=!})N`n>4T#~L|@Z30>hZaZP)+zcrw0NbvW-Q9OFpR1Y-LQ5&6OnJP z3p9$FXjpst>M8v$hn_CrgYV)JiOhD;$vVkWeWdXNq32R_A60$;EvVqb&CH0wm)~82 z_+S-0?1^wQBGx&CiGMtFTPuX7Sa2>F^+_VKROxf9Ei~B|Na=RV;vx^#fejj&gl4*p zqD$+H%P)e>OIt<&irynQADTaU#sDRTyg4g4hi=PQ9C4hq-E&%;pTDRx8$xyC@zivY z`X%k$v;dwjTC5j}Ljt8~b^mRu41&izlO&dy5|GTWSlx7`k_ zE=aAUsOBSe^l>B^EL#l0UD}F$=puEZ?7W-@4f0(egam8a4~_|-8#YZ38qHe1H94D| z)hfCMID)MM|Dd>6f{IUINFZw+$;x_$d?!$!pG-ZP|5bL6BqXf5evsNnf`3h;v95e5 z;|Xqzw5Nk=ei!$p`A1l6-8h5|vR`65Eeme2B0jV-ySJUl=x5%8x-1rJpH=PI$%;-# zMSYCG>JjthXN(i5MlU^Sn5h(jvcY4+GvP*MXT0x@u3Ie@rEXz^RMp^X-V4ecj9&|s z6}+F)x1veiv~6AOy=o-9M5n7W9Vx|)U(#HcuKOe-WOgdP@(ydxb2@ zmwgRF4)a-0|9)2eFZkM&7$&ZB*T~_afHCJWp!sqe|I1NT_Cp637YN4D<@=mlp-_kd zsZxK`N)a@Wy_x7!=Gp_#JyH%QWS_7l%q`{so^`4{z00sK3QSY%*86$7)XV<&ZMVp@ zhneO#B?ER)imp21IB>e!)s-i!$%VGTdLAH`Vozz6jPg4;$b)BkMXwSs|?0R6Jd@r+^W}^HPLV4M!#AU zV-$zbJ9Q;9FL4k+u~sIv=;Nqfc@|7Vgo6(6H8ha-zp}T631iXB%+y*BT%BKi@v^q& zcM`qBk(sguO!{mnKybiRIGA+!)QpB?Y%C{~ z=HLvaNTowQ@X4siAcK@mGqA;6#JA4JG0aKn2iQ*D^w$oR&?a{G{rWc2^vIfa^LiKR zknxS%V|d8k#2fK3d3-R8qmF>`A3bjPfzFG#6(mG6yx5o>@72NSVPJmN&+Uz{9>&Xz znK=)~9gh`{t+!Q7`op}^BLL|oX)#r<0`V(R&@b}I)kpH4Kh3mr$=Mx|)YhHFrfEb| z;(=(N;Q^vFeEbOc*#l5%YDrdkHV7EkPy#x&O9B=~G|JV@m_5LRkmCANm~Ff$r+mD6 z&VJC5+kh)q^Y57;h;uNcU&&spzpXpbec-4csYj)@u<}>YEeDr0e`*iK2w!-9laoO^c)p&dw~g7 zP_8pLOHW$hp2xLgpMgLl{%8<=1Vq1F?l%lZS4#S-HPUNyv>97jy^PG%oNZ|C4E?8d zfpy*FO+-tUypyW!S%Z}<^assP!?+AJ8a_Qg5xf1XA1X++F%t!2cZXE=hoz~{=2BEr zV_tRIWI}d#*dDiZy3a{jsXNIB4hc5!Yf&iui1-=dfoDi508W&iBDSSw;VaoHexEny zm5O!Op1k{U({2u+uuO7v{g~?7J;}Uit$X7rD!5R3R4gQqN8Ihj6NP{8SCg;4kU}^V{n4q_p8KFI)*Z9HxNOBqfsLKd;8rZN(g2o_y_~@=d6kmf{hWwwa(c(s zl7BciwD_L;!m}HBNLT5~CCQ6|?&%DKjYQ6%DW9sA3 z%HDKx*vcbS<$AZM^or{0>g7DzBb|gs*E<_eGYL&14!H%bdgnI-&6^`Q`@6h_6X0|A zs>s8`VJ`q;N;XGpN8P6&8|Lr4KrEF=BScKFg$Zee?#d-hX?t`rMJ_|X7;>6##I^%Y zgnU-aiHS0M6`e8dDLRt}PJ&gHkPd`1`^Zf#i|jAwRo^51p02g>Jy}W8Ky`TM(s3ne zqCGZ0U&gM~%LPESIm!WCT~gkkzQK^Bd1VIW$NS1p@Gulb=}+D)f~rMz&+KYZ^fBl~ z%-8rlew_usPvpX{0giFB)>}xHoij@+40F!MVR+i#3iJy2=eqF=o-cvrtx8>4eY2T- zQ;!AX`;)q@J7KcU-j4Xc_-%WlExGm)E9d|xr-iM#$5SBz>(>@Q@cq#&;DV0mSysee zhD_?fbPlq8M`@35GPK`UEa7HZP}ELTvFFFSJ409&*pE0MK7WBy!NQd#sehA1a!5!I zy|E%KpS20W{mIgYDiA6f0hCRn>>!aCcvQ0Vsfk+7ASc{$r?uh**CiH>fozSb#XTk^N?3c0L$J}`z>!Jm)BR`bxFG#lS2+5o<G00;dPTm6{75Q(fY6 zu;$z#Sl0hdM>~9zeR<&J_owM44m%$XDjnbOTK7=4j9MRc?vM9B1oy@B19Ol@BRrNd zm3&kG`z4-Wm5Kpl9ejQ#RZcd{P1ypYaA9`kL(1#}i<_ds>Dg!E+XK*lc8jX^yk%oc z0Sl0hfb_n3x@-q>iFR$O+I>J^6P5Ac$miP4#dO(K4$Ya)SydYtk{>iEO~6(srgI&m znGZOUySJUeq)Ki|H3W!nXO@P}UNt4J1o(?f>f-$vN90Sa%c+s?0RkK8f_P`g)amx1 z5}{g1y;VRWxu2BN%L{`Gl|xOz2SUm5V9 zcjGp$b~~Hzo_fI!eru81B6hD$-$pLK3`4bFgnc6XYuDg{csDv`>UqX>D40OvEzn}9RNEi7z9%FvUCKh{&snV%{dkw#k9>Kt0)Cv>Om;(SldrJz#f|X0b`_aXr z`oH9yC}se!w434fjH&RU^%fIR%mg5_sQhqFQoKV3G>*FrBYF(&Xx}mjD1>M@C?Fir zPEob$t}`3un2)zu?8Zxh0#i+d%9tVnO4E|pAtmY1Y=So3n_w{IC|Ts1`B13@g+!c! z6kJh?+LCmXR^4;8BdsQ0+#OYmO>KoG6+un4>vE7_om|+3$kN%oW=22=aOqZMF@k&G z^fkYPgh{(RXo|zc5y|+XId@d4s$(>*O7yKcA6#l_Xy|J1+0s7BUwk`gHaxm}|Jf z+D^31bGWZ+1|5WX2Pp@x&Ge3HtXbQ`kyZuufk9S?{Zh={r%lDBfD(ILxeb{~C*kEz z0t8>jL3$c$xFzQ%^Xk9ZwzfDXh2q1Kuzy@9m4$(csp){gk_9u-$~?4@PK(XXL2YkB z{o~cwN9sy7=z2eijIfZ@JiopVc_$3adURM6=MZCQM04=^|MLy ## Prerequisites -> -> * Fixme -{: .prereq} - - -{% include curriculum.html %} - -{% include links.md %} - +:::{important} Prerequisites +* Fixme +::: diff --git a/myst.yml b/myst.yml new file mode 100644 index 0000000..9536eb9 --- /dev/null +++ b/myst.yml @@ -0,0 +1,34 @@ +version: 1 + +project: + title: "Introduction to Databases for HEP" + description: "HSF Training lesson on databases with applications in HEP and Nuclear Physics" + github: https://github.com/hsf-training/hsf-training-databases-basics + license: CC-BY-4.0 + toc: + - file: index.md + - file: setup.md + - title: Episodes + children: + - file: _episodes/01-introduction.md + - file: _episodes/02-sql-basics.md + - file: _episodes/03-break.md + - file: _episodes/04-mysql-and-python.md + - file: _episodes/05-mysql-queries.md + - file: _episodes/06-conditions-database.md + - file: _episodes/07-break.md + - file: _episodes/08-sqllite.md + - file: _episodes/09-opensearch-queires.md + - file: _episodes/10-text-based-search.md + - title: Reference + file: reference.md + - title: Extras + children: + - file: _extras/about.md + - file: _extras/discuss.md + - file: _extras/guide.md + +site: + title: "Introduction to Databases for HEP" + options: + logo: fig/hsf-logo.png diff --git a/reference.md b/reference.md index b4a3103..b137558 100644 --- a/reference.md +++ b/reference.md @@ -1,5 +1,5 @@ --- -layout: reference +title: Reference --- ## Glossary diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..0f2d46d --- /dev/null +++ b/requirements.txt @@ -0,0 +1 @@ +jupyter-book>2.1 diff --git a/setup.md b/setup.md index 6004f65..db99898 100644 --- a/setup.md +++ b/setup.md @@ -6,7 +6,7 @@ title: Setup Simply click on the button below and then click on ``Create codespace``. The only requirement is a GitHub account. -[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/hsf-training/hsf-training-databases-basics){:target="_blank"} +[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/hsf-training/hsf-training-databases-basics) You should have a new tab with VSCode running and everything pre-installed. The passwords for MYSQL and Opensearch will be ``HSFtraining1``. @@ -44,15 +44,15 @@ docker run -d --name=myfirst-sqlserver -p 3306:3306 --env="MYSQL_ROOT_PASSWORD=m Here we named the container as ``myfirst-sqlserver``. It is running on host ``localhost`` and port ``3306``. A user with name ``root`` already exists with the password that you set in the environment variable ``MYSQL_ROOT_PASSWORD``. -> ## Port conflict issues -> If you run into a port conflict issue (because the port is already in use, for example), then you can map the port -> number to a different one. Something like port ``XXXX`` in ``-p XXXX:3306`` in the above Docker command. -{: .callout} +:::{note} Port conflict issues +If you run into a port conflict issue (because the port is already in use, for example), then you can map the port +number to a different one. Something like port ``XXXX`` in ``-p XXXX:3306`` in the above Docker command. +::: -> ## Never use weak passwords in production! -> Probably obvious, but this is a friendly reminder. Use [strong passwords](https://security.harvard.edu/use-strong-passwords) -> when you are working with real data in databases that are accessible from outside your computer. -{: .callout} +:::{note} Never use weak passwords in production! +Probably obvious, but this is a friendly reminder. Use [strong passwords](https://security.harvard.edu/use-strong-passwords) +when you are working with real data in databases that are accessible from outside your computer. +::: To test that if everything is up and running, execute the following command: @@ -63,18 +63,18 @@ you should see the mysql prompt as ``mysql>``. If yes, then everything is workin You can type ``exit;`` in the mysql command prompt to exit. -> ## Stopping and starting the MySQL server -> If you want to stop the Docker container that hosts MySQL server to continue working later, execute the following command: -> ```bash -> docker stop myfirst-sqlserver -> ``` -> And the container will be stopped. To start the container again, execute the following command: -> ```bash -> docker start myfirst-sqlserver -> ``` -> -> You can learn more about Docker commands at the [HSF Introduction to Docker and Podman](https://hsf-training.github.io/hsf-training-docker). -{: .callout} +:::{note} Stopping and starting the MySQL server +If you want to stop the Docker container that hosts MySQL server to continue working later, execute the following command: +```bash +docker stop myfirst-sqlserver +``` +And the container will be stopped. To start the container again, execute the following command: +```bash +docker start myfirst-sqlserver +``` + +You can learn more about Docker commands at the [HSF Introduction to Docker and Podman](https://hsf-training.github.io/hsf-training-docker). +::: ## Option 2: Setup a MySQL server via Apptainer @@ -145,19 +145,17 @@ docker run -d -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" -e "OPEN Replace: `` to a secure password of your choice. -> ## Choosing a safe password -> If you run with `-it` instead of `-d` and the password is not secure you will see the following message and the container will exit immediately. -> -> ``` -> Password failed validation: -> -> < reason for failure > -> -> Please re-try with a minimum 8 character password and must contain at least one uppercase letter, one lowercase letter, one digit, and one special character that is strong. Password strength can be tested here: https://lowe.github.io/tryzxcvbn -> ``` -> {: .output} -> -{: .callout} +:::{note} Choosing a safe password +If you run with `-it` instead of `-d` and the password is not secure you will see the following message and the container will exit immediately. + +```text +Password failed validation: + +< reason for failure > + +Please re-try with a minimum 8 character password and must contain at least one uppercase letter, one lowercase letter, one digit, and one special character that is strong. Password strength can be tested here: https://lowe.github.io/tryzxcvbn +``` +::: To test that if everything is up and running, execute the following command: ```bash From ab172aa07d11047d95884014cd90704c65674ae1 Mon Sep 17 00:00:00 2001 From: Michel Villanueva Date: Thu, 19 Feb 2026 23:25:42 -0500 Subject: [PATCH 2/4] Deployment via GitHub actions --- .github/workflows/deploy.yml | 52 ++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 .github/workflows/deploy.yml diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml new file mode 100644 index 0000000..7ecc432 --- /dev/null +++ b/.github/workflows/deploy.yml @@ -0,0 +1,52 @@ +name: Deploy Jupyter Book to GitHub Pages + +on: + push: + branches: [main] + pull_request: + branches: [main] + +permissions: + contents: read + pages: write + id-token: write + +concurrency: + group: "pages" + cancel-in-progress: false + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - name: Install dependencies + run: pip install -r requirements.txt + + - name: Build static site + run: jupyter-book build --html --ci + env: + BASE_URL: /hsf-training-databases-basics/ + + - name: Upload artifact + uses: actions/upload-pages-artifact@v3 + with: + path: _build/html + + deploy: + if: github.event_name == 'push' + needs: build + runs-on: ubuntu-latest + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - name: Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@v4 From 3090f3de7e68a1775a5bcd376265c78639aa897f Mon Sep 17 00:00:00 2001 From: Michel Villanueva Date: Fri, 20 Feb 2026 12:31:49 -0500 Subject: [PATCH 3/4] Adding episodes to the navigation bar --- myst.yml | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/myst.yml b/myst.yml index 9536eb9..81f6e45 100644 --- a/myst.yml +++ b/myst.yml @@ -8,25 +8,23 @@ project: toc: - file: index.md - file: setup.md - - title: Episodes - children: - - file: _episodes/01-introduction.md - - file: _episodes/02-sql-basics.md - - file: _episodes/03-break.md - - file: _episodes/04-mysql-and-python.md - - file: _episodes/05-mysql-queries.md - - file: _episodes/06-conditions-database.md - - file: _episodes/07-break.md - - file: _episodes/08-sqllite.md - - file: _episodes/09-opensearch-queires.md - - file: _episodes/10-text-based-search.md - - title: Reference - file: reference.md + - file: _episodes/01-introduction.md + - file: _episodes/02-sql-basics.md + - file: _episodes/03-break.md + - file: _episodes/04-mysql-and-python.md + - file: _episodes/05-mysql-queries.md + - file: _episodes/06-conditions-database.md + - file: _episodes/07-break.md + - file: _episodes/08-sqllite.md + - file: _episodes/09-opensearch-queires.md + - file: _episodes/10-text-based-search.md + #- title: Reference + # file: reference.md - title: Extras children: - file: _extras/about.md - - file: _extras/discuss.md - - file: _extras/guide.md + #- file: _extras/discuss.md + #- file: _extras/guide.md site: title: "Introduction to Databases for HEP" From 8db9e961e92fd2ac13e8151259c94cc74d1006c4 Mon Sep 17 00:00:00 2001 From: Michel Villanueva Date: Fri, 20 Feb 2026 12:35:00 -0500 Subject: [PATCH 4/4] Removing unnecesary file --- CLAUDE.md | 62 ------------------------------------------------------- 1 file changed, 62 deletions(-) delete mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index 9da6093..0000000 --- a/CLAUDE.md +++ /dev/null @@ -1,62 +0,0 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## Project Overview - -HSF Training lesson: "Introduction to Databases for HEP" — a Jupyter Book 2.x (MyST Markdown) educational curriculum teaching database fundamentals (MySQL, SQLite, OpenSearch) with applications in High Energy Physics and Nuclear Physics. - -**Status:** Pre-alpha. Published at https://hsf-training.github.io/hsf-training-databases-basics/ - -## Build and Development Commands - -Activate the virtual environment first, then: - -```bash -make jb-build # Build Jupyter Book site (HTML output in _build/) -make jb-start # Start Jupyter Book dev server with live reload -make jb-clean # Remove _build/ directory -``` - -Or directly: -```bash -jupyter-book build --html -jupyter-book start -``` - -## Pre-commit Hooks - -Required before committing: -```bash -pip3 install pre-commit && pre-commit install -``` - -Hooks enforce: trailing whitespace removal, end-of-file newlines, codespell (dictionary overrides in `codespell.txt`), and `blacken-docs` for Python code blocks in Markdown. Run manually with `pre-commit run -a`. - -## Architecture - -- **`myst.yml`** — Jupyter Book 2.x configuration: project metadata, table of contents, and site settings. -- **`_episodes/`** — Lesson content as numbered MyST Markdown files (`NN-topic.md`, e.g. `02-sql-basics.md`). Each has YAML front matter with title, teaching time, and exercises. Questions, objectives, and keypoints are rendered as admonition blocks. Episodes are listed in the TOC in `myst.yml`. -- **`_extras/`** — Supplementary pages (about, discussion, instructor guide). -- **`fig/`** — Figures and diagrams referenced by episodes. -- **`Makefile`** — Jupyter Book build commands. -- **`.devcontainer/`** — GitHub Codespaces config with Docker Compose services: MySQL (3306), OpenSearch (9200, 9600), Grafana (3000). -- **`_build/`** — Generated output (gitignored, never edit directly). - -## Content Conventions - -- Episodes use MyST directive syntax for admonitions: - - `:::{note}` for callouts/informational boxes - - `:::{important}` for prerequisites - - `:::::{admonition} Title` with `:class: challenge` for exercises, containing nested `::::{admonition} Solution` with `:class: dropdown` for collapsible solutions -- Code output is shown in `` ```text `` fenced blocks. -- Mark incomplete sections with `FIXME`. -- New episodes must follow the `NN-topic.md` naming pattern with two-digit ordering prefix and be added to the TOC in `myst.yml`. -- Several files are centrally maintained by the HSF Training organization (marked with `CENTRALLY MAINTAINED` comments) — avoid modifying these: `.pre-commit-config.yaml` (partially), `.github/config.yml`, `.github/stale.yml`. - -## Commit and PR Guidelines - -- Short, imperative commit messages with optional scope prefix (`docs:`, `build:`, `setup:`). -- Reference issues with `#123`. -- Run `jupyter-book build --html` to verify the site builds before opening PRs. -- Never commit `_build/` contents.