--all-warnings unexpected behavior

[michallepicki]

Elixir and Erlang/OTP versions

Erlang/OTP 27 [erts-15.2] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit:ns]

Elixir 1.18.0 (compiled with Erlang/OTP 27)

Operating system

Ubuntu 22.04 on Windows 11 WSL 2

Current behavior

When trying out 1.18 I saw some warnings, but then I was surprised that they disappeared on subsequent recompiles. I have narrowed it down to using Phoenix LiveView .heex files and reported an issue there. I made an example repository with one warning in a Phoenix component in a .ex file and the same kind of warning in a .heex file.

Later when debugging I learned that there is a new(edit: it's not new, it was added in 1.16, I just didn't know about its presence) flag to opt-into always getting warnings, --all-warnings which is mentioned in the Elixir 1.18 changelog:

Ensure warnings from external resources are emitted with --all-warnings when files do not change

It is also documented in mix compile docs, where "external resources" are not mentioned:

--all-warnings (--no-all-warnings) - prints all warnings, including previous compilations (default is true except on errors)

I am surprised about how it behaves in practice, trying it on my example repository I linked earlier.

1. The mix compile documentation says that "default is true except on errors". When I first read this sentence I thought "Great, then I don't need to worry about this, I should be always gettting all warnings, and it's a bug, right?!". But I think it's more likely that I don't understand what the sentence meant. What kind of errors? When, in the current compilation or previous compilations? What I can see is I'm getting all warnings printed (which I thought intuitively the --all-warnigns "flag" set to true should mean) only one time, on the first compilation:

mix clean
mix compile # prints warnings from `.heex` files, exit code 0
mix compile # doesn't print warnings from `.heex files`, exit code 0

So the default is true but only once?

2. Even if I explicitly pass the flag, it doesn't work on subsequent runs if I am not passing --warnings-as-errors at the same time as well:

mix clean
mix compile --all-warnings # prints warnings from `.heex` files, exit code 0
mix compile --all-warnings # doesn't print warnings from `.heex` files, exit code 0

I get the warnings on subsequent runs with both flags passed:

mix clean
mix compile --warnings-as-errors --all-warnings # prints warnings from `.heex` files, exit code 1
mix compile --warnings-as-errors --all-warnings # prints warnings from `.heex` files, exit code 1

3. The changelog talks about "emitting" warnings from external files. But in practice the --warnings-as-errors flag currently seems to affect whether they are persisted in compilation manifest or not. So to get all warnings, I need to make sure I was always passing the --warnings-as-errors flag in previous compile runs:

mix clean
mix compile # prints warnings from `.heex` files, exit code 0
mix compile --all-warnings # doesn't print warnings from `.heex` files, exit code 0

and:

mix clean
mix compile --all-warnings # prints warnings from `.heex` files, exit code 0
mix compile --warnings-as-errors --all-warnings # doesn't print warnings from `.heex` files, exit code 1 because of another warning

and:

mix clean
mix compile # prints warnings from `.heex` files, exit code 0
mix compile --warnings-as-errors --all-warnings # doesn't print warnings from `.heex` files, exit code 1 because of another warning

but surprisingly

mix clean
mix compile --warnings-as-errors # prints warnings from `.heex` files, exit code 1
mix compile --warnings-as-errors --all-warnings # prints warnings from `.heex` files, exit code 1

Expected behavior

I feel I don't understand how the --all-warnings flag should be working, why it's seemingly dependant on the --warnings-as-errors flag, and how the --no-all-warnings would fit into that either. Some warnings from previous compilations do get reliably printed without it, but what's the rule? Changelog mentions something about external files but the documentation does not.

Since I am clearly lost I don't want to say what the expected behavior should be 😅 But I hope my description above presented what I find surprising with the current behavior. But I'll try to summarize:

• I think the documentation should describe in clearer words which warnings get printed on the subsequent compilation runs with the flag passed in, without it (default), or with the -no flag.
• If the behavior has some dependency on other flags like --warnings-as-errors, I think the documentation should explain that dependency. But IMO ideally there would be no interaction.
• If the behavior of these two flags (plus default) depends on what flags were passed on previous compilations, I think the documentation should explain that. But IMO ideally there would be no interaction, so that I can get all warnings even if prior to that I forgot to pass this flag.

[josevalim]

--all-warnings should be the default, so you should always get it. I will investigate :)

[josevalim]

So we were tracking the source only for errors, not for warnings. I will push a fix shortly.

[michallepicki]

Thank you for the fix! Sorry for not catching it on the RC, I did try it, but this problem was a bit more hidden and I noticed it only when using 1.18 in dev for a period of time.

I wonder about this test. To resemble the issue more closely I changed it locally to have two separate capture_io assertions, like this:

      assert capture_io(:stderr, fn ->
               assert {:ok,
                       [%Mix.Task.Compiler.Diagnostic{source: ^source, file: ^file, position: 13}]} =
                        Mix.Tasks.Compile.Elixir.run([])
             end) =~ "oops"

      assert capture_io(:stderr, fn ->
               assert {:noop,
                       [%Mix.Task.Compiler.Diagnostic{source: ^source, file: ^file, position: 13}]} =
                        Mix.Tasks.Compile.Elixir.run(["--all-warnings"])
             end) =~ "oops"

But even before the fix commit this test is passing. Maybe the @external_resource file + IO.warn(..., path: file "trick" is not truly reproducing the scenario of warnings coming from external files that I had a problem with?

[josevalim]

I changed the test simply for it to be more precise. It indeed was not “fixed” or “changed” in this PR. :)