Similar errors from different stacktraces are grouped together

[gmile]

While reviewing errors collected by error-tracker, I stumbled upon the following error:

(Ecto.NoResultsError) expected at least one result but got none in query:

After looking through the error occurrences, I noticed two things:

1. actual Ecto query which results in the exception above - can be different,
2. the error may be coming from different unrelated places in the code

This makes it hard to look for similar errors, since I need to look through all occurrences. I think it would be nice if error-tracker collected errors more precisely, for example taking into account both exception message and stacktrace when calculating fingerprint.

[odarriba]

Hey!

Thanks for reporting this.

Error occurrences are grouped based on their fingerprint, which is calculated using three params:

• the error kind
• the source line
• the source function

If two unrelated occurrences are grouped together is because they are happening in the same point of your application (in your app code, not in any library one).

Please, make sure that you have configured the :otp_app setting in your ErrorTracker's configuration, as it is needed to detect the deepest stack trace belonging to your application, which is the one used for this fingerprint generation.

Hope it helps 🤗

[odarriba]

As for the request to use the whole stacktrace and exception message to group errors: it will make your errors list a complex and unmanageable list.

That is because some exceptions change the message on each exception (a Map exception usually includes the inspect version of the map, for example).

As for the stacktrace, any change in the code that leads to a change in the stacktrace (changing the line number of a fn invokation) will generate a new error groups.

We thought a lot about how to group errors and it is not an easy problem 😅

[gmile]

If two unrelated occurrences are grouped together is because they are happening in the same point of your application (in your app code, not in any library one).

Here's a self-contained script that demonstrates my point, which seems to contradict the above, I think:

Mix.install([
  {:ecto_sql, "~> 3.0"},
  {:postgrex, "~> 0.1"},
  {:error_tracker, "~> 0.5"},
  {:bandit, "~> 1.0"}
])

### ErrorLogger config

Application.put_env(:error_tracker, :repo, Repo)
Application.put_env(:error_tracker, :otp_app, :myapp)
Application.put_env(:error_tracker, :enabled, true)

defmodule AddErrorTracker do
  use Ecto.Migration

  def up, do: ErrorTracker.Migration.up(version: 4)

  def down, do: ErrorTracker.Migration.down(version: 1)
end

### Common config

Application.put_env(:myapp, Repo, database: "mix_install_examples")

defmodule Repo do
  use Ecto.Repo, adapter: Ecto.Adapters.Postgres, otp_app: :myapp
end

defmodule Migration0 do
  use Ecto.Migration

  def change do
    create table("posts") do
      add(:title, :string)
      timestamps(type: :utc_datetime_usec)
    end
  end
end

defmodule Post do
  use Ecto.Schema

  schema "posts" do
    field(:title, :string)
    timestamps(type: :utc_datetime_usec)
  end
end

defmodule Main do
  import Ecto.Query, warn: false

  def migrate do
    Repo.__adapter__().storage_down(Repo.config())
    :ok = Repo.__adapter__().storage_up(Repo.config())
    {:ok, _} = Repo.start_link([])
    Ecto.Migrator.run(Repo, [{0, Migration0}, {1, AddErrorTracker}], :up, all: true, log_migrations_sql: :info)
  end

  def register_app_and_modules do
    :application.load({:application, :myapp, [
      {:description, ~c"My sample application"},
      {:vsn, ~c"0.1.0"},
      {:modules, [Router]},
      {:registered, [Router]},
      {:applications, [:kernel, :stdlib]}
    ]})
  end

  def run_http_server do
    Supervisor.start_link([{Bandit, [plug: Router, port: 1234]}], strategy: :one_for_one)
  end

  def make_requests do
    :inets.start()
    :httpc.request("http://localhost:1234/route1")
    :httpc.request("http://localhost:1234/route2")
  end

  def check_what_was_collected do
    ErrorTracker.Error
    |> Repo.all()
    |> IO.inspect()

    ErrorTracker.Occurrence
    |> Repo.all()
    |> Enum.map(&Enum.take(&1.stacktrace.lines, 2))
    |> IO.inspect()
  end
end

defmodule Router do
  use Plug.Router
  use ErrorTracker.Integrations.Plug

  plug(:match)
  plug(:dispatch)

  get "/route1" do
    Repo.get!(Post, 1)
    send_resp(conn, 200, "Anything")
  end

  get "/route2" do
    Repo.get!(Post, 2)
    send_resp(conn, 200, "Anything")
  end
end

Main.migrate()
Main.register_app_and_modules()
Main.run_http_server()
Main.make_requests()
Main.check_what_was_collected() # <- demonstrates the issue

Process.sleep(:infinity)

The code above will produce two exceptions, then, to remonstrate the issue, will pick two top-most lines from each occurrence and output them on the screen. Here's what I see:

# single exception
[
  %ErrorTracker.Error{
    __meta__: #Ecto.Schema.Metadata<:loaded, "error_tracker_errors">,
    id: 1,
    kind: "Elixir.Ecto.NoResultsError",
    reason: "expected at least one result but got none in query:\n\nfrom p0 in Post,\n  where: p0.id == ^1\n",
    source_line: "lib/ecto/repo/queryable.ex:164",
    source_function: "Ecto.Repo.Queryable.one!/3",
    status: :unresolved,
    fingerprint: "9F7D4DAB20AF81E7E80A62D3B6C9F1DD43B28FE2E7092A2010867D59C939AE5B",
    last_occurrence_at: ~U[2025-02-27 22:35:27.622700Z],
    occurrences: #Ecto.Association.NotLoaded<association :occurrences is not loaded>,
    inserted_at: ~U[2025-02-27 22:35:27.607566Z],
    updated_at: ~U[2025-02-27 22:35:27.607566Z]
  }
]

# two top-most stacktrace lines from each occurence
[
  [
    %ErrorTracker.Stacktrace.Line{
      application: "ecto",
      module: "Ecto.Repo.Queryable",
      function: "one!",
      arity: 3,
      file: "lib/ecto/repo/queryable.ex",
      line: 164
    },
    %ErrorTracker.Stacktrace.Line{
      application: "myapp",
      module: "Router",
      function: "-do_match/4-fun-1-",
      arity: 2,
      file: "error-repro.exs",
      line: 100
    }
  ],
  [
    %ErrorTracker.Stacktrace.Line{
      application: "ecto",
      module: "Ecto.Repo.Queryable",
      function: "one!",
      arity: 3,
      file: "lib/ecto/repo/queryable.ex",
      line: 164
    },
    %ErrorTracker.Stacktrace.Line{
      application: "myapp",
      module: "Router",
      function: "-do_match/4-fun-3-",
      arity: 2,
      file: "error-repro.exs",
      line: 105
    }
  ]
]

Despite both error causing the same Ecto exception, they come from different places in the application, yet are grouped under the same error.

Some categories of errors, like Ecto.NoResultsError above, can occur in many places of a real application, that are not related to each other. And having them grouped like above makes it hard to explore errors, since many different code places that produce the same error end up grouped together.

My original expectation was that they all must not be grouped under merely a Ecto.NoResultsError, but be grouped by the location they come from (e.g. lines 100 and 105 are), so I should have 2 errors with 1 ocurrence each 🤔

[odarriba]

I will check it tomorrow, thanks for the example 🤗

[odarriba]

A PR to fix it has been sent for review and will be released as soon as possible.

Thanks for the bug report 🤗

[gmile]

Awesome, thank you so much for a quick fix @odarriba!