Skip to content

Commit f857dc0

Browse files
committed
feat: Add a templated helper to the hybrid cache to retrieve a child
This will be the public interface used by functions that need to access one specific type of child. It's a bit brittle since the rest of the hybrid class intentionally avoids expecting there to be exactly one unified child and one recurrent child, but the idea is that this should only be used from a context where that's known to be true. Branch: GraniteFour Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
1 parent 220456d commit f857dc0

File tree

1 file changed

+15
-0
lines changed

1 file changed

+15
-0
lines changed

src/llama-kv-cache.h

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -418,6 +418,21 @@ class llama_kv_cache_hybrid : public llama_kv_cache {
418418
const llama_hparams & hparams,
419419
std::vector<child_cache> children);
420420

421+
// getters for specific child cache type
422+
// NOTE: This will fail if there are multiple of the given type
423+
template<typename child_t>
424+
const child_t * get_child_cache() const {
425+
const child_t * child = nullptr;
426+
for (const auto & child_cache : m_children) {
427+
const child_t * child_cast = dynamic_cast<const child_t *>(child_cache.get());
428+
if (child_cast) {
429+
GGML_ASSERT(!child);
430+
child = child_cast;
431+
}
432+
}
433+
return child;
434+
}
435+
421436
//
422437
// llama_memory_i
423438
//

0 commit comments

Comments
 (0)