Docker file

[fahadshery]

Hi All,

I tried dockerising this app but having the following error:

NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs:
     query       : shape=(1, 577, 16, 64) (torch.bfloat16)
     key         : shape=(1, 577, 16, 64) (torch.bfloat16)
     value       : shape=(1, 577, 16, 64) (torch.bfloat16)
     attn_bias   : <class 'NoneType'>
     p           : 0.0
`fa2F@v2.5.6-pt` is not supported because:
    requires device with capability > (8, 0) but your GPU has capability (6, 1) (too old)
    bf16 is only supported on A100+ GPUs
`cutlassF-pt` is not supported because:
    bf16 is only supported on A100+ GPUs

. Regardless Here is the Dockerfile and docker-compose...See any of you could make it work for compute capability of 6 atleast!

# Base image with CUDA 12.4
FROM nvidia/cuda:12.4.1-base-ubuntu22.04

# Install pip if not already installed
RUN apt-get update -y && apt-get install -y \
    python3-pip \
    libgl1 \
    libglib2.0-0
    #python3-dev \
    #git \
    #build-essential  # Install dependencies for building extensions

# Define environment variables for UID and GID and local timezone
ENV PUID=${PUID:-1000}
ENV PGID=${PGID:-1000}

# Create a group with the specified GID
RUN groupadd -g "${PGID}" appuser
# Create a user with the specified UID and GID
RUN useradd -m -s /bin/sh -u "${PUID}" -g "${PGID}" appuser

WORKDIR /app

# Install Torch, Torchvision, and Torchaudio for CUDA 12.4
RUN pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# Install main application dependencies because of 24G VRAM using requirements_fp8 but change it to main
COPY ./requirements_fp8.txt ./requirements.txt
RUN pip install --no-cache-dir -r ./requirements.txt

RUN chown -R appuser:appuser /app

# delete redundant requirements.txt
RUN rm ./requirements.txt

#Run application as non-root
#USER appuser

# Copy PuLID application code
COPY . ./PuLID

EXPOSE 8080

ENV GRADIO_SERVER_NAME="0.0.0.0"

WORKDIR /app/PuLID

# Run PuLID-Flux Python application
CMD ["python3", "./app_flux.py","--offload","--fp8"]