Search results

[Bug][Disagg] KvCacheAwareRouter silently degrades to load-only routing (per-server block table not populated; matched_tokens≡0) — ~90% vs 96% prefix-cache hit

Disaggregated serving<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

Status: Open.

#14882 In NVIDIA/TensorRT-LLM;

· lishicheng1996-nv opened on Jun 3, 2026

[Feature]: [AutoDeploy] GPT-OSS-120b perf analsys and opt (tp>=2)

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14874 In NVIDIA/TensorRT-LLM;

· taylor-yb-lee opened on Jun 2, 2026

[Feature]: AutoDeploy: DFlash speculative decoding support

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Speculative Decoding<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter

Status: Open.

#14843 In NVIDIA/TensorRT-LLM;

· govind-ramnarayan opened on Jun 2, 2026

[Feature]: Support encoder - prefill disagg in AutoDeploy for multimodal models

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

MultimodalLabel for issues & PRs regarding Multimodal related objectsLabel for issues & PRs regarding Multimodal related objects

Status: Open.

#14840 In NVIDIA/TensorRT-LLM;

· bmarimuthu-nv opened on Jun 2, 2026

[Bug]: AutoDeploy: Fix race from non-blocking D2H copy in Eagle + make copy finer-grained.

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

Status: Open.

#14838 In NVIDIA/TensorRT-LLM;

· govind-ramnarayan opened on Jun 1, 2026

[Feature Request] AutoDeploy: propagate layer_type hint across pattern-matcher rewrites

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14834 In NVIDIA/TensorRT-LLM;

· greg-kwasniewski1 opened on Jun 1, 2026

[Feature]: AutoDeploy: Support multi kv cache memory pool in trtllm attention

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14828 In NVIDIA/TensorRT-LLM;

· taylor-yb-lee opened on Jun 1, 2026

[Bug]: Using the official tensorrt llm whisper conversion script does not work, due to numpy dependency issue

bugSomething isn't workingSomething isn't working

Triton backend<NV>Related to NVIDIA Triton Inference Server backend<NV>Related to NVIDIA Triton Inference Server backend

Status: Open.

#14826 In NVIDIA/TensorRT-LLM;

· protonicage opened on Jun 1, 2026

[Bug]: kimi 2-6 nvfp4 + SpecD gives low accuracy on GPQA Diamond

bugSomething isn't workingSomething isn't working

Speculative Decoding<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter

Status: Open.

#14825 In NVIDIA/TensorRT-LLM;

· sravan500 opened on Jun 1, 2026

[Feature Request] Make sharding IR the AutoDeploy default with auto-detect fallback to legacy

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14784 In NVIDIA/TensorRT-LLM;

· greg-kwasniewski1 opened on May 30, 2026

[Feature Request] Improved pattern matcher for AR+residual_add+RMSNorm fusion (cast/reshape coverage)

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14781 In NVIDIA/TensorRT-LLM;

· greg-kwasniewski1 opened on May 30, 2026

[FEAT] Wan 2.2: fuse NVFP4 quantization with preceding LayerNorm/AdaLN and GELU-tanh

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

Model optimization<NV>Model-specific performance optimizations and tuning<NV>Model-specific performance optimizations and tuning

Status: Open.

#14772 In NVIDIA/TensorRT-LLM;

· anikaj-eng opened on May 30, 2026