Skip to content

Refactor: Existing examples refactoring opportunities #7559

Open
@mofosyne

Description

@mofosyne

GG in this PR was suggesting a refactoring of examples and provided an example of what he would like to refactor. This is a ticket to inform others about opportunities to improve examples

I think a bigger advantage would be to do some refactoring in the existing examples and "hide" some of the state for sampling and KV cache management that we expose behind the common API

If there is any other aspect of the example which is currently a pain point for developer grokking, feel free to also suggest some so it can be added here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions