[https://nvbugs/5508301][feat] Move D->H copies to a worker thread whe…#8463

Open

dhansen-nvidia wants to merge 13 commits intoNVIDIA:mainfrom

dhansen-nvidia:cc_perf_fixes

Commits on Nov 25, 2025

[https:/nvbugs/5508301][feat] Move D->H copies to a worker thread when confidential compute is active
mojombo
committed
Refactor to pull all common code under the AsyncWorkerMixin
mojombo
committed
Better conform to TRT-LLM style guidelines and add an LLM API argument to explicitly enable the async worker for testing purposes
mojombo
committed
Make confidential_compute_enabled() more robust and complete
mojombo
committed
Test accuracy of sampling when the async worker is enabled
mojombo
committed
Refactor to further compartmentalize worker from core sampler code
mojombo
committed
Explicitly track and wait on all futures; makes it possible to scale to > 1 thread correctly in the future
mojombo
committed
Fix some issues discovered in testing
mojombo
committed
Put the synchronization functionality of the new SamplerEvent class into the class itself to improve composability
mojombo
committed
Fix testnames in test_lists
mojombo
committed
Move async_worker_start() into PyExecutor::start_worker() where it more logically belongs
mojombo
committed
Fix api_stability unittest
mojombo
committed
Change name of llmapi argument to better match guidelines in the docs
mojombo
committed