Add environment setup files, devcontainer configuration, and GitHub Actions CI workflow #1

Copilot · 2025-10-16T10:44:22Z

This PR adds comprehensive environment setup infrastructure to make it easier for contributors and users to get started with the diffusion-evolution project.

What's Added

Environment Setup Files

environment.yml: Conda environment specification targeting Python 3.10 with dependencies managed via pip
requirements.txt: Core ML and scientific computing packages (numpy, scipy, pandas, matplotlib, seaborn, scikit-learn, tqdm, pillow, pyyaml, omegaconf, hydra-core, einops, tensorboard)
setup.sh: Flexible bash helper script that supports:
- Both conda and pip installation methods (auto-detects conda availability)
- Multiple CUDA targets: cpu, cu118 (CUDA 11.8), cu121 (CUDA 12.1)
- Customizable environment names
- Command-line flags: --method, --cuda, --env-name, --help

Note: PyTorch is intentionally installed separately via the setup script (rather than in requirements.txt) to ensure the correct CUDA-compatible version is fetched based on your target environment.

GitHub Codespaces / devcontainer Support

.devcontainer/devcontainer.json: VS Code devcontainer configuration using the official Microsoft Miniconda image
.devcontainer/postCreate.sh: Automatically runs setup after container creation
Includes Python extension and Pylance for enhanced development experience

CI/CD Infrastructure

.github/workflows/ci.yml: GitHub Actions workflow that:
- Triggers on push and pull_request events
- Sets up Python 3.10
- Installs CPU-only PyTorch (for efficient CI)
- Installs project dependencies
- Runs smoke tests to verify PyTorch installation

Documentation

README_SETUP.md: Comprehensive setup guide covering:
- Local conda installation
- Local pip + venv installation
- Codespaces usage
- PyTorch verification commands
- CI workflow notes

Usage Examples

Quick start with conda (recommended for GPU work):

./setup.sh --method conda --cuda cu121
conda activate diffusion-evolution

Quick start with pip + venv (CPU or explicit CUDA):

./setup.sh --method pip --cuda cpu
source .venv/bin/activate

Verify installation:

python -c "import torch; print('torch', torch.__version__, 'cuda available', torch.cuda.is_available())"

Testing

The CI workflow has been tested via simulation:

✅ PyTorch 2.9.0+cpu successfully installs from wheel repository
✅ All requirements install without conflicts
✅ Smoke test passes (torch import and version validation)

Benefits

Consistency: All contributors use the same environment setup
Flexibility: Support for different CUDA versions and installation methods
Automation: Codespaces auto-setup and CI smoke tests catch issues early
Documentation: Clear instructions reduce setup friction for new users

This infrastructure complements the existing setup.py for package installation while providing a complete development environment setup workflow.

Original prompt

Add a new branch setup/pytorch-cu121 that adds environment setup files, a Codespaces/devcontainer configuration, and a GitHub Actions CI workflow that runs smoke tests on CPU. Files to add (paths and exact contents):

name: diffusion-evolution
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.10
  - pip
  - pip:
    - -r requirements.txt

# Core scientific & ML packages (torch is installed separately via the helper script to match CUDA)
numpy>=1.24
scipy>=1.10
pandas>=2.0
matplotlib>=3.7
seaborn>=0.12
scikit-learn>=1.2
tqdm>=4.65
pillow>=9.5
pyyaml>=6.0
omegaconf>=2.3
hydra-core>=1.3
einops>=0.6
tensorboard>=2.12
# optional: experiment tracking (uncomment if used)
# wandb>=0.15
# Add any repo-specific dependencies below:
# - some-local-package==x.y.z

#!/usr/bin/env bash
set -euo pipefail

usage() {
  cat <<EOF
Usage: ./setup.sh [OPTIONS]

Options:
  --env-name NAME     Conda environment name (default: diffusion-evolution)
  --method METHOD     install method: conda or pip (default: conda if conda exists, else pip)
  --cuda VERSION      CUDA alias: cpu, cu118, cu121 (default: cu121)
  --help              show this help
Example:
  ./setup.sh --method pip --cuda cu121
  ./setup.sh --method conda --cuda cpu
EOF
}

# defaults
ENV_NAME="diffusion-evolution"
METHOD=""
CUDA="cu121"

# parse args
while [[ $# -gt 0 ]]; do
  case "$1" in
    --env-name) ENV_NAME="$2"; shift 2;;
    --method) METHOD="$2"; shift 2;;
    --cuda) CUDA="$2"; shift 2;;
    --help) usage; exit 0;;
    *) echo "Unknown arg: $1"; usage; exit 1;;
  esac
done

# prefer conda if present and method not forced
if [[ -z "$METHOD" ]]; then
  if command -v conda &> /dev/null; then
    METHOD="conda"
  else
    METHOD="pip"
  fi
fi

echo "Install method: $METHOD"
echo "CUDA target: $CUDA"

install_pytorch_pip() {
  # Map CUDA target to PyTorch index url
  case "$CUDA" in
    cpu)
      INDEX_URL="https://download.pytorch.org/whl/cpu"
      ;;
    cu118)
      INDEX_URL="https://download.pytorch.org/whl/cu118"
      ;;
    cu121)
      INDEX_URL="https://download.pytorch.org/whl/cu121"
      ;;
    *)
      echo "Unknown CUDA target: $CUDA"; exit 1;;
  esac

  echo "Installing PyTorch (pip) from $INDEX_URL ..."
  python -m pip install --upgrade pip setuptools wheel
  python -m pip install "torch" "torchvision" "torchaudio" --index-url "${INDEX_URL}"
  echo "Installing remaining requirements..."
  python -m pip install -r requirements.txt
}

install_pytorch_conda() {
  # Using conda env creation with environment.yml, but we'll still ensure torch matches
  if ! command -v conda &> /dev/null; then
    echo "conda not found. Aborting."
    exit 1
  fi

  echo "Creating conda environment '${ENV_NAME}' with environment.yml..."
  conda env create -f environment.yml -n "${ENV_NAME}" || {
    echo "If the env already exists, try: conda env update -f environment.yml -n ${ENV_NAME}"
  }

  echo "Activating environment and ensuring torch is installed correctly..."
  if [[ "${CUDA}" == "cpu" ]]; then
    conda run -n "${ENV_NAME}" python -m pip install --upgrade pip setuptools wheel
    conda run -n "${ENV_NAME}" python -m pip install --index-url https://download.pytorch.org/whl/cpu "torch" "torchvision" "torchaudio"
  else
    case "$CUDA" in
      cu118)
        conda run -n "${ENV_NAME}" conda install -y pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
        ;;
      cu121)
        conda run -n "${ENV_NAME}" conda install -y pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
        ;;
      *)
        echo "Unknown CUDA target: $CUDA"; exit 1;;
    esac
    conda run -n "${ENV_NAME}" python -m pip install -r requirements.txt
  fi

  echo "Conda environment '${ENV_NAME}' created. To use it: conda activate ${ENV_NAME}"
}

if [[ "$METHOD" == "pip" ]]; then
  echo "Creating virtualenv in .venv..."
  python -m venv .venv
  # shellcheck disable=SC1090
  source .venv/bin/activate
  install_pytorch_pip
  echo "Done. Activate with: source .venv/bin/activate"
elif [[ "$METHOD" == "conda" ]]; then
  install_pytorch_conda
else
  echo "Unknown method: $METHOD"
  exit 1
fi

echo "Setup finished."

```markdown
# Setup instructions for diffusion-evolution

This file explains how to set up the Python / PyTorch environment to run experiments locally, on GitHub Codespaces, or via conda/pip.

Quick choices:
- Local conda preferred (GPU support via pytorch-cuda/pytorch channel)
- Local pip + venv works well (use the included setup.sh to fetch an appropriate torch wheel)
- Codespaces: use the included .devcontainer which runs the setup script on post-create

Defaults in these files target CUDA cu121. If you need CPU-only, pass `--cuda cpu` to the setup script or edit `environment.yml`.

1) Using conda (recommended when using GPU)
- Create from template:
  - Edit `environment.yml` to change python version or rem...

</details>

*This pull request was created as a result of the following prompt from Copilot chat.*
> Add a new branch `setup/pytorch-cu121` that adds environment setup files, a Codespaces/devcontainer configuration, and a GitHub Actions CI workflow that runs smoke tests on CPU. Files to add (paths and exact contents):
> 
> ```name=environment.yml
> name: diffusion-evolution
> channels:
>   - conda-forge
>   - defaults
> dependencies:
>   - python=3.10
>   - pip
>   - pip:
>     - -r requirements.txt
> ```
> 
> ```name=requirements.txt
> # Core scientific & ML packages (torch is installed separately via the helper script to match CUDA)
> numpy>=1.24
> scipy>=1.10
> pandas>=2.0
> matplotlib>=3.7
> seaborn>=0.12
> scikit-learn>=1.2
> tqdm>=4.65
> pillow>=9.5
> pyyaml>=6.0
> omegaconf>=2.3
> hydra-core>=1.3
> einops>=0.6
> tensorboard>=2.12
> # optional: experiment tracking (uncomment if used)
> # wandb>=0.15
> # Add any repo-specific dependencies below:
> # - some-local-package==x.y.z
> ```
> 
> ```name=setup.sh
> #!/usr/bin/env bash
> set -euo pipefail
> 
> usage() {
>   cat <<EOF
> Usage: ./setup.sh [OPTIONS]
> 
> Options:
>   --env-name NAME     Conda environment name (default: diffusion-evolution)
>   --method METHOD     install method: conda or pip (default: conda if conda exists, else pip)
>   --cuda VERSION      CUDA alias: cpu, cu118, cu121 (default: cu121)
>   --help              show this help
> Example:
>   ./setup.sh --method pip --cuda cu121
>   ./setup.sh --method conda --cuda cpu
> EOF
> }
> 
> # defaults
> ENV_NAME="diffusion-evolution"
> METHOD=""
> CUDA="cu121"
> 
> # parse args
> while [[ $# -gt 0 ]]; do
>   case "$1" in
>     --env-name) ENV_NAME="$2"; shift 2;;
>     --method) METHOD="$2"; shift 2;;
>     --cuda) CUDA="$2"; shift 2;;
>     --help) usage; exit 0;;
>     *) echo "Unknown arg: $1"; usage; exit 1;;
>   esac
> done
> 
> # prefer conda if present and method not forced
> if [[ -z "$METHOD" ]]; then
>   if command -v conda &> /dev/null; then
>     METHOD="conda"
>   else
>     METHOD="pip"
>   fi
> fi
> 
> echo "Install method: $METHOD"
> echo "CUDA target: $CUDA"
> 
> install_pytorch_pip() {
>   # Map CUDA target to PyTorch index url
>   case "$CUDA" in
>     cpu)
>       INDEX_URL="https://download.pytorch.org/whl/cpu"
>       ;;
>     cu118)
>       INDEX_URL="https://download.pytorch.org/whl/cu118"
>       ;;
>     cu121)
>       INDEX_URL="https://download.pytorch.org/whl/cu121"
>       ;;
>     *)
>       echo "Unknown CUDA target: $CUDA"; exit 1;;
>   esac
> 
>   echo "Installing PyTorch (pip) from $INDEX_URL ..."
>   python -m pip install --upgrade pip setuptools wheel
>   python -m pip install "torch" "torchvision" "torchaudio" --index-url "${INDEX_URL}"
>   echo "Installing remaining requirements..."
>   python -m pip install -r requirements.txt
> }
> 
> install_pytorch_conda() {
>   # Using conda env creation with environment.yml, but we'll still ensure torch matches
>   if ! command -v conda &> /dev/null; then
>     echo "conda not found. Aborting."
>     exit 1
>   fi
> 
>   echo "Creating conda environment '${ENV_NAME}' with environment.yml..."
>   conda env create -f environment.yml -n "${ENV_NAME}" || {
>     echo "If the env already exists, try: conda env update -f environment.yml -n ${ENV_NAME}"
>   }
> 
>   echo "Activating environment and ensuring torch is installed correctly..."
>   if [[ "${CUDA}" == "cpu" ]]; then
>     conda run -n "${ENV_NAME}" python -m pip install --upgrade pip setuptools wheel
>     conda run -n "${ENV_NAME}" python -m pip install --index-url https://download.pytorch.org/whl/cpu "torch" "torchvision" "torchaudio"
>   else
>     case "$CUDA" in
>       cu118)
>         conda run -n "${ENV_NAME}" conda install -y pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
>         ;;
>       cu121)
>         conda run -n "${ENV_NAME}" conda install -y pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
>         ;;
>       *)
>         echo "Unknown CUDA target: $CUDA"; exit 1;;
>     esac
>     conda run -n "${ENV_NAME}" python -m pip install -r requirements.txt
>   fi
> 
>   echo "Conda environment '${ENV_NAME}' created. To use it: conda activate ${ENV_NAME}"
> }
> 
> if [[ "$METHOD" == "pip" ]]; then
>   echo "Creating virtualenv in .venv..."
>   python -m venv .venv
>   # shellcheck disable=SC1090
>   source .venv/bin/activate
>   install_pytorch_pip
>   echo "Done. Activate with: source .venv/bin/activate"
> elif [[ "$METHOD" == "conda" ]]; then
>   install_pytorch_conda
> else
>   echo "Unknown method: $METHOD"
>   exit 1
> fi
> 
> echo "Setup finished."
> ```
> 
> ````markdown name=README_SETUP.md
> ```markdown
> # Setup instructions for diffusion-evolution
> 
> This file explains how to set up the Python / PyTorch environment to run experiments locally, on GitHub Codespaces, or via conda/pip.
> 
> Quick choices:
> - Local conda preferred (GPU support via pytorch-cuda/pytorch channel)
> - Local pip + venv works well (use the included setup.sh to fetch an appropriate torch wheel)
> - Codespaces: use the included .devcontainer which runs the setup script on post-create
> 
> Defaults in these files target CUDA cu121. If you need CPU-only, pass `--cuda cpu` to the setup script or edit `environment.yml`.
> 
> 1) Using conda (recommended when using GPU)
> - Create from template:
>   - Edit `environment.yml` to change python version or remove CUDA-specific steps.
>   - Run:
>     ```bash
>     conda env create -f environment.yml -n diffusion-evolution
>     conda activate diffusion-evolution
>     ```
> 
> - Or use the helper script:
>     ```bash
>     ./setup.sh --method conda --cuda cu121
>     ```
> 
> 2) Using pip + venv
> 
> ```bash
> ./setup.sh --method pip --cuda cpu
> # or for CUDA
> ./setup.sh --method pip --cuda cu121
> source .venv/bin/activate
> ```
> 
> 3) Verifying PyTorch and GPU
> 
> ```bash
> python -c "import torch; print('torch', torch.__version__, 'cuda available', torch.cuda.is_available()); print(torch.cuda.get_device_name(0) if torch.cuda.is_available() else '')"
> ```
> 
> 4) Codespaces / devcontainer
> - The provided `.devcontainer/` runs the setup script after the container is created. It uses a Miniconda-based image so conda installs are available.
> 
> 5) CI notes
> - The included GitHub Actions workflow runs on CPU and performs a smoke test installing the CPU PyTorch wheel and the requirements.
> 
> If you'd like, I can adjust the devcontainer to pre-install GPU-capable CUDA drivers or pin exact package versions.
> ```
> ````
> 
> ```name=.devcontainer/devcontainer.json
> {
>   "name": "diffusion-evolution",
>   "image": "mcr.microsoft.com/devcontainers/miniconda:23-ubuntu-22.04",
>   "customizations": {
>     "vscode": {
>       "extensions": [
>         "ms-python.python",
>         "ms-python.vscode-pylance"
>       ]
>     }
>   },
>   "postCreateCommand": "bash .devcontainer/postCreate.sh",
>   "remoteUser": "vscode"
> }
> ```
> 
> ```name=.devcontainer/postCreate.sh
> #!/usr/bin/env bash
> set -euo pipefail
> 
> # Run the project setup using conda + cu121 by default in Codespaces
> bash ../setup.sh --method conda --cuda cu121
> ```
> 
> ```name=.github/workflows/ci.yml
> name: CI
> on: [push, pull_request]
> 
> jobs:
>   smoke:
>     runs-on: ubuntu-latest
>     steps:
>       - uses: actions/checkout@v4
>       - name: Set up Python
>         uses: actions/setup-python@v4
>         with:
>           python-version: '3.10'
>       - name: Install CPU PyTorch
>         run: |
>           python -m pip install --upgrade pip setuptools wheel
>           python -m pip install --index-url https://download.pytorch.org/whl/cpu torch torchvision torchaudio
>       - name: Install requirements
>         run: python -m pip install -r requirements.txt
>       - name: Run smoke tests
>         run: |
>           python - <<PY
> import torch, sys
> print('torch', torch.__version__, 'cuda available', torch.cuda.is_available())
> if not isinstance(torch.__version__, str):
>     sys.exit(1)
> PY
> ```
> 
> PR description / summary:
> - Adds environment.yml, requirements.txt, setup.sh, README_SETUP.md
> - Adds .devcontainer/ to provision Codespaces with Miniconda and run the setup on post-create
> - Adds a GitHub Actions workflow that installs CPU PyTorch and runs a basic smoke test
> 
> Please create a new branch `setup/pytorch-cu121`, add the files above, and open a pull request against `main` with the title and description provided. The PR should be ready for review and include the files exactly as listed.
> 

<!-- START COPILOT CODING AGENT TIPS -->
---

✨ Let Copilot coding agent [set things up for you](https://github.com/nghiata-uit/diffusion-evolution/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: nghiata-uit <[email protected]>

nghiata-uit · 2025-11-12T16:11:38Z

@copilot https://github.com/nghiata-uit/diffusion-evolution/tree/experiment-nghiata

chạy thực nghiệm benchmark sử dụng gpu

Copilot · 2025-11-12T16:14:39Z

@copilot https://github.com/nghiata-uit/diffusion-evolution/tree/experiment-nghiata

chạy thực nghiệm benchmark sử dụng gpu

I cannot run the GPU benchmarks directly as I don't have access to GPU hardware. However, I can help with the environment setup to support GPU benchmarks.

The current setup already includes GPU support through the --cuda cu121 option in setup.sh which installs PyTorch with CUDA 12.1 support.

Could you clarify what specific changes you'd like to the environment setup files? For example:

Add benchmark-specific dependencies?
Update the setup script for better GPU configuration?
Create a specific setup guide for running benchmarks on GPU?

Initial plan

b027aa4

Copilot AI assigned Copilot and nghiata-uit Oct 16, 2025

Copilot started work on behalf of nghiata-uit October 16, 2025 10:44 View session

Copilot AI and others added 3 commits October 16, 2025 10:49

Add environment setup files and CI workflow

cc8b059

Co-authored-by: nghiata-uit <[email protected]>

Push setup/pytorch-cu121 branch

4d896bd

Co-authored-by: nghiata-uit <[email protected]>

Remove test file - finalize environment setup

2f7f97a

Co-authored-by: nghiata-uit <[email protected]>

nghiata-uit marked this pull request as ready for review October 16, 2025 10:55

Copilot AI changed the title ~~[WIP] Add environment setup files and CI workflow for smoke tests~~ Add environment setup files, devcontainer configuration, and GitHub Actions CI workflow Oct 16, 2025

Copilot AI requested a review from nghiata-uit October 16, 2025 10:56

Copilot finished work on behalf of nghiata-uit October 16, 2025 10:56

Copilot started work on behalf of nghiata-uit November 12, 2025 16:12 View session

Copilot finished work on behalf of nghiata-uit November 12, 2025 16:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add environment setup files, devcontainer configuration, and GitHub Actions CI workflow #1

Add environment setup files, devcontainer configuration, and GitHub Actions CI workflow #1

Uh oh!

Copilot AI commented Oct 16, 2025 •

edited

Loading

Uh oh!

nghiata-uit commented Nov 12, 2025

Uh oh!

Copilot AI commented Nov 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add environment setup files, devcontainer configuration, and GitHub Actions CI workflow #1

Are you sure you want to change the base?

Add environment setup files, devcontainer configuration, and GitHub Actions CI workflow #1

Uh oh!

Conversation

Copilot AI commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What's Added

Environment Setup Files

GitHub Codespaces / devcontainer Support

CI/CD Infrastructure

Documentation

Usage Examples

Testing

Benefits

Uh oh!

nghiata-uit commented Nov 12, 2025

Uh oh!

Copilot AI commented Nov 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Oct 16, 2025 •

edited

Loading