Support speculative decoding #289
Description
Is it on the roadmap for this tool to support inference optimization such as speculative decoding for gpus and neuron cores?
{{ message }}
Is it on the roadmap for this tool to support inference optimization such as speculative decoding for gpus and neuron cores?