Skip to content

Commit 9d3ba0b

Browse files
committed
improvement for the appended 0
1 parent a4b7b4c commit 9d3ba0b

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

llama.cpp

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6425,8 +6425,8 @@ static std::pair<std::vector<uint32_t>, llama_partial_utf8> decode_utf8(
64256425
static const int lookup[] = { 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 2, 2, 3, 4 };
64266426
const char * pos = src;
64276427
std::vector<uint32_t> code_points;
6428-
// common english strings have the same number of codepoints and bytes.
6429-
code_points.reserve(n_src);
6428+
// common english strings have the same number of codepoints and bytes. `+ 1` for the terminating 0.
6429+
code_points.reserve(n_src + 1);
64306430
uint32_t value = partial_start.value;
64316431
int n_remain = partial_start.n_remain;
64326432

0 commit comments

Comments
 (0)