You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Right now, all optimizations are being performed on ONNX model. It is quite simple approach, but we have to restrict ourselves due to risk of invalidating the model. Also it is quite hard there to add new fused kernels.
Now
Structure
This is heavily inspired by compilers, rustc in particular
Now there are 3 levels of representations:
Source -- ONNX model.
(Brand-new) IR, on which all optimizations are being applied. Pass infrastructure has also been reworked, akin to rustc's.
(Brand-new) Execution plan, on which we select kernels w/ graph colouring algo, akin to instruction selection in most of compilers. Also it is much easier now to write new computational kernels.
Tensor allocation reusage. We can reuse allocations for tensors with same shape product and data type. Requires full shape inference coverage and custom arena allocator.
Big optimizer rework!!!!
Before
Right now, all optimizations are being performed on ONNX model. It is quite simple approach, but we have to restrict ourselves due to risk of invalidating the model. Also it is quite hard there to add new fused kernels.
Now
Structure
This is heavily inspired by compilers,
rustcin particularNow there are 3 levels of representations:
rustc's.Progress:
x86_64-v4and ARMcc @enthropy7