[CVPR2024 Extended] ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association
This is the official implementation of the CVPR'24 paper
ADA-Track: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association
Shuxiao Ding, Lukas Schneider, Marius Cordts, Juergen Gall
as well as its extended arxiv paper
ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association
Shuxiao Ding, Lukas Schneider, Marius Cordts, Juergen Gall
Query-based 3D Multi-Object Tracking (MOT) facilitates seamless integration into end-to-end frameworks. Many existing methods adopt the tracking-by-attention paradigm, utilizing track queries for identity-consistent detection and object queries for identity-agnostic track spawning. Tracking-by-attention, however, entangles detection and tracking queries in one embedding for both the detection and tracking task, which is sub-optimal. Other approaches resemble the tracking-by-detection paradigm, detecting objects using decoupled track and detection queries followed by a subsequent association. These methods, however, do not leverage synergies between the detection and association task. Combining the strengths of both paradigms, we introduce ADA-Track, a novel end-to-end framework for 3D MOT from multi-view cameras. We introduce a learnable data association module based on edge-augmented cross-attention, leveraging appearance and geometric features. Furthermore, we integrate this association module into the decoder layer of a DETR-based 3D detector, enabling simultaneous DETR-like query-to-image cross-attention for detection and query-to-query cross-attention for data association. By stacking these decoder layers, queries are refined for the detection and association task alternately, effectively harnessing the task dependencies. We evaluate our method on the nuScenes dataset and demonstrate the advantage of our approach compared to the two previous paradigms.
- [2024/06/02] We release our source code for ADA-Track++.
- [2024/12/13] We release an extended version, ADA-Track++. The arxiv paper can be accessed here.
- [2024/02/26] Our paper is accepted by CVPR 2024! 🎉🎉
- [2024/05/14] We release our source code.
| Detector | Backbone | N_GPU | AMOTA | AMOTP | Logfile | Checkpoint |
|---|---|---|---|---|---|---|
| DETR3D | ResNet-101 | 4 | 38.4 | 1.378 | log | Download |
| PETR | V2-99 | 8 | 48.3 | 1.242 | log | Download |
| Detector | Backbone | N_GPU | AMOTA | AMOTP | Logfile | Checkpoint |
|---|---|---|---|---|---|---|
| DETR3D | ResNet-101 | 4 | 39.8 | 1.336 | log | Download |
| PETR | V2-99 | 8 | 50.4 | 1.197 | log | Download |
See LICENSE for more details.
Our implementation is a plugin to mmdetection3d, and it is highly dependent on MUTR3D. We build our tracker based on the implementation of the detectors DETR3D and PETR. The success of this work is impossible without these aforementionend projects as well as many other high-quality open-source projects. Many thanks to them for enabling our research!