Replication Package for ASE2025

This repository contains the code for the paper "When All Close Fails: Roundoff Error Estimation for Deep Learning Programs". The code is designed to be used with the Triton compiler and is intended for research purposes.

Install

We recommend using a conda environment to install the dependencies. You can create and activate a new conda environment with the following command:

conda create -n roundoff python=3.11
conda activate roundoff

Then, install the required packages using pip:

pip install torch==2.4.0
pip install numpy
pip install numba
pip install tabulate 
cd triton
pip install ninja cmake wheel pybind11; # build-time dependencies
pip install -e python

Run

To run the code, you can use the following commands:

cd triton
python run.py # generate interval for each case
python judge_all.py # judge the result

and you will get the results in stdout like this:

╒════════════╤═════════════════╤════════════════╤════════════════╤════════════════════╤════════════════╤═════════════╤═════════════════════════════════════╕
│ Issue ID   │ Triton Output   │ Triton GT      │ Triton Match   │ Reference Output   │ Reference GT   │ Ref Match   │ Out-of-Range Count (Triton / Ref)   │
╞════════════╪═════════════════╪════════════════╪════════════════╪════════════════════╪════════════════╪═════════════╪═════════════════════════════════════╡
│ 1924       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/262144 / 0/262144                 │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 5990       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/261121 / 0/261121                 │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 3017       │ ✗ Out of Range  │ ✗ Expected Out │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 224/256 / 0/256                     │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 5895       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/12 / 0/12                         │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 4551       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/4096 / 0/4096                     │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 1190       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/524288 / 0/524288                 │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 2843       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/4096 / 0/4096                     │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 1960       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/10 / 0/10                         │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 2680b      │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/205797 / 0/205797                 │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 376        │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/8192 / 0/8192                     │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 5065       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/262144 / 0/262144                 │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 1840       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/131072 / 0/131072                 │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 1666       │ ✗ Out of Range  │ ✗ Expected Out │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 261632/262144 / 0/262144            │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 1937       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/131072 / 0/131072                 │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 1671       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/6291456 / 0/6291456               │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 1821       │ ✗ Out of Range  │ ✗ Expected Out │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 5504691/12582912 / 0/12582912       │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 4701       │ ✗ Out of Range  │ ✗ Expected Out │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 32/48 / 0/48                        │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 1808       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/4194304 / 0/4194304               │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 1578       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/10000 / 0/10000                   │
├────────────┼─────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┼─────────────┼─────────────────────────────────────┤
│ 6227       │ ✓ In Range      │ ✓ Expected In  │ ✔              │ ✓ In Range         │ ✓ Expected In  │ ✔           │ 0/1280000 / 0/1280000               │
╘════════════╧═════════════════╧════════════════╧════════════════╧════════════════════╧════════════════╧═════════════╧═════════════════════════════════════╛

If you encounter errors "roundoff/bin/../lib/libstdc++.so.6: version GLIBCXX_3.4.30' not found", you can solve it by running conda install -c conda-forge gcc=12.1.0`