Support start strings, the opposite of stop tokens.

[matteoserva]

This allows setting one or multiple start strings that behave the opposite of stop tokens.
The output is discarded until a start string is reached, then it is sent to client as usual even in streaming mode.
If no start string is found, the entire generated text is sent to client at the end of generation.

Use case:

• Support for broken clients that don't behave well with reasoning models (github copilot) while still allowing the model to perform its reasoning. Example: set start string to "</think>

From command line: llama-server --start-string "</think>" --start-string "</thinking>"

From client:

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "Qwen/Qwen3-8B",
  "messages": [
    {"role": "user", "content": "Hi"}
  ],
  "start_strings": ["</think>","</thinking>"]
}'

[ngxson]

Please also add a test case under server/tests/unit/test_chat_completion.py

[matteoserva]

Please also add a test case under server/tests/unit/test_chat_completion.py

done

[ngxson]

Upon closer inspection, I think manipulating send_text is not an optimum solution here.

If you look at the streamed response, when start string is not yet found, the server will still send an empty string for each token generated, which is very wasteful. See the else branch of if (send_text) below.

Instead, I think this logic should be placed outside this if (!incomplete) block

[ngxson]

Sorry for slow review, this feature is very cool but I'm always not very confident about the code for processing the generated token. The logic has always been a bit fragile, so I may take it slow to think about all of the possible cases.

I think I will take over this PR for now, will push some modification directly here.

[ngxson]

If I understand correctly, the final thing we need is found_pos + found_string.size(), so probably find_first_substring don't even need to return the found_pos and found_string. Instead, it could return the position right after the found start string. This way, we don't even need to care about which start string is selected.

For example, my start string is </think>:

Ok I'm ready to give the answer</think>The final answer is
                                       ^ return this position

[matteoserva]

Sorry for slow review, this feature is very cool but I'm always not very confident about the code for processing the generated token. The logic has always been a bit fragile, so I may take it slow to think about all of the possible cases.

I think I will take over this PR for now, will push some modification directly here.

I took some time to refactor the server code and try to handle all the funny edge cases.
Maybe you could take a look at it.

[ngxson]

I don't have time for this rn, but tbh the new version looks even more hacky and risky than before. I would expect something very simple