Merge main code - #1
Conversation
Added a diagram of the VLA pipeline architecture to the README.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| "config": dataclasses.asdict(config), | ||
| "timestamp": time.time(), | ||
| } | ||
| torch.save(metadata, tmp_ckpt_dir / "metadata.pt") |
There was a problem hiding this comment.
[Security] Semgrep · rules.python.tob.pickles-in-pytorch - 959ca
Functions reliant on pickle can result in arbitrary code execution. Consider loading from state_dict, using fickling, or switching to a safer serialization method like ONNX
Lines 124–124 in scripts/train.py
- Severity: ERROR | Likelihood: MEDIUM | Impact: HIGH | Confidence: MEDIUM
- References:
- https://blog.trailofbits.com/2021/03/15/never-a-dill-moment-exploiting-machine-learning-pickle-files/
To ignore this finding, comment on PR with/ignore 959ca.
- https://blog.trailofbits.com/2021/03/15/never-a-dill-moment-exploiting-machine-learning-pickle-files/
| optimizer_path = ckpt_dir / "optimizer.pt" | ||
|
|
||
| if optimizer_path.exists(): | ||
| optimizer_state_dict = torch.load(optimizer_path, map_location=device, weights_only=False) |
There was a problem hiding this comment.
[Security] Semgrep · rules.python.tob.pickles-in-pytorch - e3c56
Functions reliant on pickle can result in arbitrary code execution. Consider loading from state_dict, using fickling, or switching to a safer serialization method like ONNX
Lines 183–183 in scripts/train.py
- Severity: ERROR | Likelihood: MEDIUM | Impact: HIGH | Confidence: MEDIUM
- References:
- https://blog.trailofbits.com/2021/03/15/never-a-dill-moment-exploiting-machine-learning-pickle-files/
To ignore this finding, comment on PR with/ignore e3c56.
- https://blog.trailofbits.com/2021/03/15/never-a-dill-moment-exploiting-machine-learning-pickle-files/
|
|
||
| # Load metadata | ||
| logging.info("Loading metadata...") | ||
| metadata = torch.load(ckpt_dir / "metadata.pt", map_location=device, weights_only=False) |
There was a problem hiding this comment.
[Security] Semgrep · rules.python.tob.pickles-in-pytorch - c3634
Functions reliant on pickle can result in arbitrary code execution. Consider loading from state_dict, using fickling, or switching to a safer serialization method like ONNX
Lines 196–196 in scripts/train.py
- Severity: ERROR | Likelihood: MEDIUM | Impact: HIGH | Confidence: MEDIUM
- References:
- https://blog.trailofbits.com/2021/03/15/never-a-dill-moment-exploiting-machine-learning-pickle-files/
To ignore this finding, comment on PR with/ignore c3634.
- https://blog.trailofbits.com/2021/03/15/never-a-dill-moment-exploiting-machine-learning-pickle-files/
| self._data_loader = torch.utils.data.DataLoader( | ||
| typing.cast(torch.utils.data.Dataset, dataset), | ||
| batch_size=local_batch_size, | ||
| shuffle=shuffle, | ||
| num_workers=num_workers, | ||
| multiprocessing_context=mp_context, | ||
| persistent_workers=num_workers > 0, | ||
| collate_fn=_collate_fn, | ||
| worker_init_fn=_worker_init_fn, | ||
| drop_last=drop_last, | ||
| generator=generator, | ||
| ) |
There was a problem hiding this comment.
[Security] Semgrep · rules.python.tob.automatic-memory-pinning - 45623
If possible, it is better to rely on automatic pinning in PyTorch to avoid undefined behavior and for efficiency
Lines 567–578 in src/asset_vla/training/data_loader.py
- Severity: WARNING | Likelihood: LOW | Impact: LOW | Confidence: HIGH
- References:
- https://pytorch.org/docs/stable/data.html#memory-pinning
To ignore this finding, comment on PR with/ignore 45623.
- https://pytorch.org/docs/stable/data.html#memory-pinning
2 participants
No description provided.