Skip to content

Unable to assign mmproj value when running docker  #6226

Closed
@antonymevin

Description

@antonymevin

Please include information about your system, the steps to reproduce the bug, and the version of llama.cpp that you are using. If possible, please provide a minimal code example that reproduces the bug.

If the bug concerns the server, please try to reproduce it first using the server test scenario framework.

Command

sudo docker run -p 5000:8000  --gpus all --runtime=nvidia -v /models:/models ghcr.io/ggerganov/llama.cpp:server-cuda -m /models/ggml-model-q4_k.gguf --mmproj /models/mmproj-model-f16.gguf  --port 8000 --host 0.0.0.0 -v  -t 16  -n 512 -c 2048 -ngl 1 -cb -np 4 --n-gpu-layers 33

Error

error: unknown argument: --mmproj

--mmproj option is not supported by docker.

The documentation mentions this option though.
https://github.com/ggerganov/llama.cpp/tree/master/examples/server#llamacpp-http-server

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions