Merge pull request #67 from hmorimitsu/update

Update

Author: hmorimitsu

README.md

@@ -24,6 +24,19 @@ This is still under development, so some things may not work as intended. I plan
 
 ## What's new
 
+###  - v0.3.2
+
+- Added new models:
+  - MemFlow [https://arxiv.org/abs/2404.04808](https://arxiv.org/abs/2404.04808)
+  - NeuFlow [https://arxiv.org/abs/2403.10425](https://arxiv.org/abs/2403.10425)
+  - SEA-RAFT [https://arxiv.org/abs/2405.14793](https://arxiv.org/abs/2405.14793)
+  - SplatFlow [https://arxiv.org/abs/2306.08887](https://arxiv.org/abs/2306.08887)
+- Add support for more datasets:
+  - TartanAir [https://theairlab.org/tartanair-dataset/](https://theairlab.org/tartanair-dataset/)
+  - Kubric [https://github.com/google-research/kubric](https://github.com/google-research/kubric)
+- Add ONNX and TensorRT conversion to RAPIDFlow
+- Fix LR scheduler when accumulating gradients
+
 ###  - v0.3.1
 
 - Added new models:
@@ -84,6 +97,7 @@ This is still under development, so some things may not work as intended. I plan
 - LLA-Flow [https://arxiv.org/abs/2304.08101](https://arxiv.org/abs/2304.08101)
 - MaskFlownet [https://arxiv.org/abs/2003.10955](https://arxiv.org/abs/2003.10955)
 - MatchFlow [https://arxiv.org/abs/2303.08384](https://arxiv.org/abs/2303.08384)
+- MemFlow [https://arxiv.org/abs/2404.04808](https://arxiv.org/abs/2404.04808)
 - MS-RAFT+ [https://arxiv.org/abs/2210.16900](https://arxiv.org/abs/2210.16900)
 - NeuFlow [https://arxiv.org/abs/2403.10425](https://arxiv.org/abs/2403.10425)
 - PWCNet [https://arxiv.org/abs/1709.02371](https://arxiv.org/abs/1709.02371)
@@ -92,8 +106,10 @@ This is still under development, so some things may not work as intended. I plan
 - RPKNet [https://hmorimitsu.com/publication/2024-aaai-rpknet](https://hmorimitsu.com/publication/2024-aaai-rpknet)
 - ScopeFlow [https://arxiv.org/abs/2002.10770](https://arxiv.org/abs/2002.10770)
 - SCV [https://arxiv.org/abs/2104.02166](https://arxiv.org/abs/2104.02166)
+- SEA-RAFT [https://arxiv.org/abs/2405.14793](https://arxiv.org/abs/2405.14793)
 - SeparableFlow [https://openaccess.thecvf.com/content/ICCV2021/papers/Zhang_Separable_Flow_Learning_Motion_Cost_Volumes_for_Optical_Flow_Estimation_ICCV_2021_paper.pdf](https://openaccess.thecvf.com/content/ICCV2021/papers/Zhang_Separable_Flow_Learning_Motion_Cost_Volumes_for_Optical_Flow_Estimation_ICCV_2021_paper.pdf)
 - SKFlow [https://arxiv.org/abs/2205.14623](https://arxiv.org/abs/2205.14623)
+- SplatFlow [https://arxiv.org/abs/2306.08887](https://arxiv.org/abs/2306.08887)
 - STaRFlow [https://arxiv.org/abs/2007.05481](https://arxiv.org/abs/2007.05481)
 - VCN [https://papers.nips.cc/paper/2019/file/bbf94b34eb32268ada57a3be5062fe7d-Paper.pdf](https://papers.nips.cc/paper/2019/file/bbf94b34eb32268ada57a3be5062fe7d-Paper.pdf)
 - VideoFlow [https://arxiv.org/abs/2303.08340](https://arxiv.org/abs/2303.08340)

datasets.yml

@@ -7,3 +7,6 @@ mpi_sintel: /path/to/MPI-Sintel
 kitti_2012: /path/to/KITTI/2012
 kitti_2015: /path/to/KITTI/2015
 hd1k: /path/to/HD1K
+tartanair: /path/to/tartanair
+spring: /path/to/spring
+kubric: /path/to/kubric

docs/source/_static/plot-memory(gb)-fp32-sintel-final-occ-val_epe.html

@@ -99,6 +99,14 @@ matchflow_gma-kitti-bc72ce81.ckpt,matchflow-g-kitti.pth,
 matchflow_gma-sintel-683422f4.ckpt,matchflow-g-sintel.pth,
 matchflow_gma-things-49295bd8.ckpt,matchflow-g-things.pth,
 matchflow_raft-things-bf560032.ckpt,matchflow-r-things.pth,
+memflow-things-90d0b74c.ckpt,MemFlowNet_things.pth,https://github.com/DQiaole/MemFlow
+memflow-sintel-38621d84.ckpt,MemFlowNet_sintel.pth,
+memflow-kitti-ee6cbf09.ckpt,MemFlowNet_kitti.pth,
+memflow-spring-7ee1b984.ckpt,MemFlowNet_spring.pth,
+memflow_t-things-6028d89f.ckpt,MemFlowNet_T_things.pth,
+memflow_t-things_kitti-542e0a1c.ckpt,MemFlowNet_T_things_kitti.pth,
+memflow_t-sintel-d2df0424.ckpt,MemFlowNet_T_sintel.pth,
+memflow_t-kitti-9eeabb65.ckpt,MemFlowNet_T_kitti.pth,
 ms_raft_plus-mixed-2bb01f62.ckpt,mixed.pth,https://github.com/cv-stuttgart/MS_RAFT_plus
 neuflow-things-c402aa7a.ckpt,neuflow_things.pth,https://github.com/neufieldrobotics/neuflow
 neuflow-sintel-0d969ea2.ckpt,neuflow_sintel.pth,
@@ -129,13 +137,26 @@ scv-quarter-chairs-4726627e.ckpt,quarter/scv-chairs.pth,
 scv-quarter-kitti-e86c7953.ckpt,quarter/scv-kitti.pth,
 scv-quarter-sintel-2d9b4a05.ckpt,quarter/scv-sintel.pth,
 scv-quarter-things-0dac9b66.ckpt,quarter/scv-things.pth,
+sea_raft_m-tartan-e684ed5f.ckpt,Tartan480x640-M.pth,https://github.com/princeton-vl/SEA-RAFT
+sea_raft_s-tartan-f7e26f21.ckpt,Tartan480x640-S.pth,
+sea_raft_m-chairs-1cb7b11e.ckpt,Tartan-C368x496-M.pth,
+sea_raft_s-chairs-6980249f.ckpt,Tartan-C368x496-S.pth,
+sea_raft_m-things-ac45dd7f.ckpt,Tartan-C-T432x960-M.pth,
+sea_raft_s-things-a15c1713.ckpt,Tartan-C-T432x960-S.pth,
+sea_raft_m-sintel-f8bb7e3f.ckpt,Tartan-C-T-TSKH432x960-M.pth,
+sea_raft_s-sintel-bb63371a.ckpt,Tartan-C-T-TSKH432x960-S.pth,
+sea_raft_m-kitti-e51f7603.ckpt,Tartan-C-T-TSKH-kitti432x960-M.pth,
+sea_raft_s-kitti-3a96c1cc.ckpt,Tartan-C-T-TSKH-kitti432x960-S.pth,
+sea_raft_m-spring-de7c13e2.ckpt,Tartan-C-T-TSKH-spring540x960-M.pth,
+sea_raft_s-spring-4d13c106.ckpt,Tartan-C-T-TSKH-spring540x960-S.pth,
 separableflow-kitti-c9395318.ckpt,sepflow_kitti.pth,https://github.com/feihuzhang/SeparableFlow
 separableflow-sintel-4c9a8c03.ckpt,sepflow_sintel.pth,
 separableflow-things-31fe3b2d.ckpt,sepflow_things.pth,
 separableflow-universal-87350d91.ckpt,sepflow_universal.pth,
 skflow-kitti-4e1f8b63.ckpt,skflow-kitti.pth,https://github.com/littlespray/SKFlow
 skflow-sintel-98fb67cf.ckpt,skflow-sintel.pth,
 skflow-things-f84e6538.ckpt,skflow-things.pth,
+splatflow-kitti-2aa8e145.ckpt,kitti.pth,https://github.com/wwsource/SplatFLow
 starflow-kitti-34b9a3ed.ckpt,StarFlow_kitti,https://github.com/pgodet/star_flow
 starflow-sintel-3c06b844.ckpt,StarFlow_sintel,
 starflow-things-d3966a93.ckpt,StarFlow_things,

docs/source/_static/plot-memory(gb)-fp32-sintel-final-occ-val_outlier.html

@@ -266,6 +266,17 @@ MatchFlow
 
 - Model names: ``matchflow``, ``matchflow_raft``
 
+MemFlow
+-------
+
+`[source code] <https://github.com/hmorimitsu/ptlflow/tree/main/ptlflow/models/memflow>`__
+
+- Paper: **MemFlow: Optical Flow Estimation and Prediction with Memory** - `https://arxiv.org/abs/2404.04808 <https://arxiv.org/abs/2404.04808>`_
+
+- Reference code: `https://github.com/DQiaole/MemFlow <https://github.com/DQiaole/MemFlow>`_
+
+- Model names: ``memflow``, ``memflow_t``
+
 MS-RAFT+
 --------
 
@@ -350,6 +361,17 @@ SCV
 
 - Model names: ``scv4``, ``scv8``
 
+SEA-RAFT
+--------
+
+`[source code] <https://github.com/hmorimitsu/ptlflow/tree/main/ptlflow/models/sea_raft>`__
+
+- Paper: **SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow** - `https://arxiv.org/abs/2405.14793 <https://arxiv.org/abs/2405.14793>`_
+
+- Reference code: `https://github.com/princeton-vl/SEA-RAFT <https://github.com/princeton-vl/SEA-RAFT>`_
+
+- Model names: ``sea_raft``, ``sea_raft_s``, ``sea_raft_m``, ``sea_raft_l``
+
 SeparableFlow
 -------------
 
@@ -372,6 +394,17 @@ SKFlow
 
 - Model names: ``skflow``
 
+SplatFlow
+---------
+
+`[source code] <https://github.com/hmorimitsu/ptlflow/tree/main/ptlflow/models/splatflow>`__
+
+- Paper: **SplatFlow: Learning Multi-frame Optical Flow via Splatting** - `https://arxiv.org/abs/2306.08887 <https://arxiv.org/abs/2306.08887>`_
+
+- Reference code: `https://github.com/wwsource/SplatFLow <https://github.com/wwsource/SplatFLow>`_
+
+- Model names: ``splatflow``
+
 STaRFlow
 --------