Refactor GPU batch processing in Team class to optimize memory usage …#293
Open
thangk wants to merge 1 commit intofani-lab:mainfrom
Open
Refactor GPU batch processing in Team class to optimize memory usage …#293thangk wants to merge 1 commit intofani-lab:mainfrom
thangk wants to merge 1 commit intofani-lab:mainfrom
Conversation
…and improve performance. Adjusted device handling and implemented dynamic batch sizing based on available GPU memory.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Added proper GPU processing unlike previously which was just transferring from CPU to GPU then right back to GPU to CPU without doing any processing.
This change does the processing on the GPU in batches. However, it does not outperform the CPU's processing time as stated in the issue comment #286 (comment)