Skip to content

Commit 2c33914

Browse files
committed
added clone to fix prefill issue
1 parent 30d69b3 commit 2c33914

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

generate.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -172,7 +172,7 @@ def generate(
172172
seq = empty
173173
input_pos = torch.arange(0, T, device=device)
174174

175-
next_token = prefill(model, prompt.view(1, -1), input_pos, **sampling_kwargs)
175+
next_token = prefill(model, prompt.view(1, -1), input_pos, **sampling_kwargs).clone()
176176
if is_speculative:
177177
prefill(draft_model, prompt.view(1, -1), input_pos, **sampling_kwargs)
178178
seq[T] = next_token

0 commit comments

Comments
 (0)