Skip to content

Distributed solution - new madengine CLI #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
coketaste wants to merge 155 commits into
base: main
Choose a base branch
from
Open

Distributed solution - new madengine CLI #11

coketaste wants to merge 155 commits into from

Conversation

@coketaste
Copy link
Contributor

@coketaste coketaste commented Jul 3, 2025

DoD:
This solution splits the madengine run_models.py workflow into separate 'build' and 'run' phases to enable distributed execution scenarios.

@coketaste coketaste self-assigned this Jul 3, 2025
coketaste added 28 commits July 3, 2025 15:43
@coketaste
Fixed the test cases for distributed solution
ea2dc0c
@coketaste
Updated the interface and fix the issue due to updating
bb64b73
@coketaste
Reorganize the cli interface of distributed solution
86d1790
@coketaste
Updated the interface of distributed solution and refine the code wit…
dd71dfa 
…h test coverage
@coketaste
Added setup.py for installation with dev
8236a7b
@coketaste
Fix the test case of distributed cli
0c42bbf
@coketaste
Fixed the flow of manifest and run_phase to work properly
f942a45
@coketaste
Updated setup.py for the cases of modern and legacy installation
08ff29b
@coketaste
Fixed and enhanced live log in build and run phases
d82d78e
@coketaste
Fixed the log generate for different phase, and correct log name
c848419
@coketaste
Fix the perf.csv generation in distributed execution
3e2a44c
@coketaste
Fixed the data which update to perf.csv
8a359ac
@coketaste
Fixed the columns in perf.csv due to parsing issue
ac32cbe
@coketaste
Fix the incorrect regex escaping in the container runner that prevent…
bb6d3fc 
…ed proper performance metric extraction
@coketaste
Update the patterns of performance and metric
a255c50
@coketaste
Fixed the issue of docker_image column in perf.csv
8988508
@coketaste
Improve the interface and reduce erro in registry flow
f0a10a7
@coketaste
Updated the flow of run phase, fix the docker pull, fix the creds ver…
8caae5c 
…ify before docker login
@coketaste
updated the tagged name for docker image and add a docker_image_tagge…
942e666 
…d field to the build manifest
@coketaste
Updated the sequence of operations in build phas
a7baa17
@coketaste
Fixed the registry_image
2e613ca
@coketaste
Update the registry_image
b4e7d22
@coketaste
Updated the process of run phase
d1ecb97
@coketaste
Refactored the file structure of package for distributed_cli
799cce7
@coketaste
Fixed the errors in unit tests
9875fda
@coketaste
Fix the error in unit test of distributed cli
168ffe5
@coketaste
Refactored constants to make design as best practices
756d82a
@coketaste
Cleanup:
9431d7f 
Simplified - No confusing multiple configuration files
Modern - Follows current Python packaging standards (PEP 621)
Maintainable - Single source of truth
Compatible - Works with all modern Python tools
@coketaste coketaste changed the title Distributed solution refactoring current run_model flow Distributed solution - new madengine CLI Aug 5, 2025
coketaste and others added 22 commits August 5, 2025 17:26
@coketaste
Updated README
72982f8
@coketaste
Added discover command to mad_cli
b6b79ca
@coketaste
Implemented CLI detect MAD_CONTAINER_IMAGE in additional context, pro…
00f4a5e 
…duction-ready and maintains full backward compatibility with existing madengine workflows
@coketaste
Merge pull request #31 from ROCm/coketaste/refactor-interface
ee5740d 
Ensure Backward Compatibility in New madengine-cli
@coketaste
Implemented the core multi-GPU architectures support for docker image…
364bef4 
… building
@coketaste
Implemented unit tests for the feature of multi-gpu arch
156bcfe
@coketaste
Debug and fix the unit test of multi gpu arch
8457257
@coketaste
Debug the issue of display results table
3a0b4c7
@coketaste
Enhanced the results table, and improved the flow of handle gpu arch …
682bec2 
…surfix at docker image name
@coketaste
Creates architecture-specific images with proper naming and metadata,…
89784ca 
… regardless of the underlying Dockerfile configuration.
@coketaste
Fixed the syntax error
23bbf57
@coketaste
Merge pull request #32 from ROCm/coketaste/refactor-multi-gpu-archs
4e61147 
Feature to build docker images for multigpu archs
@Boss2002n
ported changes from coketaste/amd-smi
5444a67
@Boss2002n
Revert "ported changes from coketaste/amd-smi"
9dfe5d8 
This reverts commit 5444a67.
@coketaste
Resolved merging conflicts
d5c3402
@coketaste
Fixed the tools for distributed mode
e9202c2
@coketaste
Fixed the cleanup
b49ed4b
@coketaste
Merge pull request #52 from ROCm/coketaste/refactor-tools
0ac1855 
Sync up amd-smi and update tools
@coketaste
Fixed the table of resutls
15cbeaa
@coketaste
Fixed the GPU Product Name
026fec3
@coketaste
Fixed the issue in selftest
9b7b347
@coketaste
Enhanced unit tests and cleanup
eca075a
@coketaste coketaste marked this pull request as ready for review November 28, 2025 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@coketaste coketaste

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@coketaste @Boss2002n