Adding vectors to the codebook

[AlecMRogers]

Im wondering if this VQ can serve as a container for Word2Vec vectors that allows back-propagation, ao that I can use actual words in the error function.
Im planning populate only a small vocabulary from Word2Vec based on usage, but my difficulty in doing so is that it seems impossible to insert vectors in a codebook that has already been initialized.
suggestions?

[lucidrains]

@AlecMRogers there is a setter method on VQ with .codebook = your_codes, if that is what you mean

[AlecMRogers]

Can that be used to add vectors to the codebook (I.e. change its size dynamically)?-alechttps://GnosticModels.comOn Apr 16, 2025, at 12:52, Phil Wang ***@***.***> wrote: @AlecMRogers there is a setter method on VQ with .codebook = your_codes, if that is what you mean —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[lucidrains]

ohh, you mean increasing the codebook size

no it can't do that, but I think it can be added easily (famous last words)

let me think about that

[AlecMRogers]

OK. It’s a distant use case for me, right now I can look up all symbols within the input space ahead of time, but the ability to dynamically add/remove codebook vectors (and alter the size) is still important in use cases where the input space is not well-defined in advance.