Skip to content

Logging message "ggml_metal_free: deallocating" is shown when verbose=False #802

Closed
@paschembri

Description

@paschembri

Prerequisites

Please answer the following questions for yourself before submitting an issue.

  • I am running the latest code. Development is very rapid so there are no tagged versions as of now.
  • I carefully followed the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
  • I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

When freeing the model and verbose=False, no logging message should appear on stdout/stderr.

Current Behavior

ggml_metal_free: deallocating is displayed

Environment and Context

Steps to Reproduce

In [1]: from llama_cpp import Llama

In [2]: class A:
   ...:     def __init__(self):
   ...:         self.model = Llama('/Users/pas/.cache/py-llm-core/models/mistral', verbose=False, n_ctx=4000, n_gpu_layers=100, n_threads=4)
   ...:     def generate(self, prompt):
   ...:         return self.model.create_completion(prompt)
   ...:     def free_model(self):
   ...:         del self.model
   ...:

In [3]: instance = A()

In [4]: instance.generate('Write a recipe to make a pizza')
Out[4]:
{'id': 'cmpl-76b2943e-1169-4062-bb6b-8e916279b329',
 'object': 'text_completion',
 'created': 1696768489,
 'model': '/Users/pas/.cache/py-llm-core/models/mistral',
 'choices': [{'text': ' crust that is perfect for a thin and crispy pizza.\n\nIngredients:\n- 3 cups all-purpose flour\n- 1 package active dry yeast\n- 2 tablespoons olive oil\n- 1 teaspoon salt\n- 1 cup warm water\n\nInstructions:\n\n1) In a large mixing bowl, combine the flour and yeast.\n\n2) Add the olive oil and salt to the mixture and stir until well combined.\n\n3) Pour in the warm water and use your hands to knead the dough until it forms a smooth ball.',
   'index': 0,
   'logprobs': None,
   'finish_reason': 'length'}],
 'usage': {'prompt_tokens': 8, 'completion_tokens': 128, 'total_tokens': 136}}

In [5]: instance.free_model()
ggml_metal_free: deallocating

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions