Skip to content

Optimizer rework #29

Description

@Human9000-bit

Big optimizer rework!!!!

Before

Right now, all optimizations are being performed on ONNX model. It is quite simple approach, but we have to restrict ourselves due to risk of invalidating the model. Also it is quite hard there to add new fused kernels.

Now

Structure

This is heavily inspired by compilers, rustc in particular
Now there are 3 levels of representations:

  1. Source -- ONNX model.
  2. (Brand-new) IR, on which all optimizations are being applied. Pass infrastructure has also been reworked, akin to rustc's.
  3. (Brand-new) Execution plan, on which we select kernels w/ graph colouring algo, akin to instruction selection in most of compilers. Also it is much easier now to write new computational kernels.

Progress:

  1. Optimizer rework itself (done in optimizer rework #25)
  2. Tensor allocation reusage. We can reuse allocations for tensors with same shape product and data type. Requires full shape inference coverage and custom arena allocator.
  3. Optimizer correctness testing
  4. Check everything out on x86_64-v4 and ARM
  5. Look into model loading time.

cc @enthropy7

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions