Open
Description
System Info
Arch Linux
NVIDIA-SMI 550.90.07 Driver Version: 550.90.07 CUDA Version: 12.4
Information
- Docker
- The CLI directly
Tasks
- An officially supported command
- My own modifications
Reproduction
When running the trtllm backend on 39cfe23 with the dev profile, I found the following panic.
thread 'main' panicked at /home/linuxuser/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/clap_builder-4.5.32/src/builder/debug_asserts.rs:112:17:
Command text-generation-backends-trtllm: Short option names must be unique for each argument, but '-p' is in use by both 'port' and 'prometheus_port'
I tested that removing the short from line 40 here resolves this problem.
text-generation-inference/backends/trtllm/src/main.rs
Lines 38 to 41 in 39cfe23
Reproduction step:
cargo build --profile dev --package text-generation-backends-trtllm --bin text-generation-backends-trtllm
./target/debug/text-generation-backends-trtllmExpected behavior
The panic should not happen and should output this.
error: the following required arguments were not provided:
--tokenizer-name <TOKENIZER_NAME>
--model-id <MODEL_ID>
--executor-worker <EXECUTOR_WORKER>
Usage: text-generation-backends-trtllm --tokenizer-name <TOKENIZER_NAME> --model-id <MODEL_ID> --executor-worker <EXECUTOR_WORKER>
For more information, try '--help'.
Metadata
Metadata
Assignees
Labels
No labels