Skip to content

Commit b58351e

Browse files
committed
fix: Fix wrong bool condition for split equal in hybrid cache
Branch: HybridRecurrentCache Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
1 parent 728f514 commit b58351e

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/llama-kv-cache.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3175,7 +3175,7 @@ llama_pos llama_kv_cache_hybrid_recurrent::seq_pos_max(llama_seq_id seq_id) cons
31753175
llama_memory_state_ptr llama_kv_cache_hybrid_recurrent::init_batch(const llama_batch & batch, uint32_t n_ubatch, bool embd_pooled, bool logits_all) {
31763176

31773177
// since this includes a recurrent cache, we cannot use split_simple
3178-
auto sbatch = llama_sbatch(batch, hparams.n_embd, true, logits_all);
3178+
auto sbatch = llama_sbatch(batch, hparams.n_embd, false, logits_all);
31793179

31803180
// follow the recurrent pattern for creating the ubatch splits
31813181
std::vector<llama_ubatch> ubatches;

0 commit comments

Comments
 (0)