Skip to content

Commit 8d9a017

Browse files
committed
fix IndexError when runners list changes dynamically
Signed-off-by: Shijie Wang <[email protected]>
1 parent 8eb6a21 commit 8d9a017

File tree

1 file changed

+5
-2
lines changed

1 file changed

+5
-2
lines changed

tensorrt_llm/_torch/autotuner.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -350,11 +350,14 @@ def search_cache(
350350
Returns:
351351
A tuple containing:
352352
[is_cache_hit, runner_id, tactic, stored_profile]
353+
runner_id is the index in the current runners list
353354
"""
354-
for r in runners:
355+
for idx, r in enumerate(runners):
355356
if (cache_key := self.get_cache_key(custom_op, r, input_shapes,
356357
tuning_config)) in self.cache:
357-
return True, *self.cache[cache_key]
358+
# Return the current index in runners list, not the cached runner_id
359+
cached_runner_id, tactic, min_time = self.cache[cache_key]
360+
return True, idx, tactic, min_time
358361

359362
return False, *self.fallback_entry()
360363

0 commit comments

Comments
 (0)