Skip to content

Commit 7e0f7b7

Browse files
TysonAndrenikic
authored andcommitted
Reduce memory used by token_get_all()
Around a quarter of all strings in array tokens would have a string that's one character long (e.g. ` `, `\`, `1`) For parsing a large number of php files, The memory increase dropped from 378374248 to 369535688 (2.5%) Closes GH-4753.
1 parent 1b08ab2 commit 7e0f7b7

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

ext/tokenizer/tokenizer.c

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,11 @@ static void add_token(zval *return_value, int token_type,
110110
zval keyword;
111111
array_init(&keyword);
112112
add_next_index_long(&keyword, token_type);
113-
add_next_index_stringl(&keyword, (char *) text, leng);
113+
if (leng == 1) {
114+
add_next_index_str(&keyword, ZSTR_CHAR(text[0]));
115+
} else {
116+
add_next_index_stringl(&keyword, (char *) text, leng);
117+
}
114118
add_next_index_long(&keyword, lineno);
115119
add_next_index_zval(return_value, &keyword);
116120
} else {

0 commit comments

Comments
 (0)