[partial function] Map.!: given key is not an element in the map

[kokobd]

Your environment

Which OS do you use: NixOS

Which LSP client (editor/plugin) do you use: VSCode Haskell

Describe your project (alternative: link to the project):

Use my fork, but comment out the following section in flake.nix, like this:

                # modules = [{
                #   reinstallableLibGhc = false;
                #   nonReinstallablePkgs = [
                #     "rts" "ghc-heap" "ghc-prim" "integer-gmp" "integer-simple" "base" "deepseq" "array" "ghc-boot-th"
                #     "pretty" "template-haskell" "ghcjs-prim" "ghcjs-th" "ghc-bignum" "exceptions" "stm" "ghc-boot"
                #     "ghc" "Cabal" "Win32" "array" "binary" "bytestring" "containers" "directory" "filepath" "ghc-boot"
                #     "ghc-compact" "ghc-prim" "hpc" "mtl" "parsec" "process" "text" "time" "transformers" "unix" "xhtml"
                #     "terminfo"
                #   ];
                # }];

With the modules config, hls works. Without it, hls fails with Map.!: given key is not an element in the map

Steps to reproduce

1. run nix build to generate a hls executable at result/bin/haskell-language-server
2. enter a develop shell with nix develop
3. run result/bin/haskell-language-server to see it fail
4. (optional) you can point vscode to result/bin/haskell-language-server and use nix environment selector.

Expected behaviour

HLS should work normally

Actual behaviour

Every file reports this error on the first line.

Map.!: given key is not an element in the map
CallStack (from HasCallStack):
  error, called at src/Data/Map/Internal.hs:633:17 in containers-0.6.5.1-1mJ1aiYFxjS4zGVm0SPbaq:Data.Map.Internal

Include debug information

log

Finished: QualifyImportedNames.TypeCheck Took: 0.00s
Finished: changeTypeSignature.GetParsedModule Took: 0.00s
Finished: alternateNumberFormat.CollectLiterals Took: 0.00s
Finished: Hlint.GetModSummary Took: 0.00s
Finished: Pragmas.GhcSession Took: 0.00s
Finished: RefineImports Took: 0.00s
Finished: Pragmas.GetFileContents Took: 0.00s
Finished: GADT.GetParsedModuleWithComments Took: 0.00s
Finished: Pragmas.GetParsedModule Took: 0.00s
Finished: GhcideCodeActions.getParsedModule Took: 0.00s
2022-06-30T02:46:03.130105Z | Debug | Finished: RefineImports Took: 0.00s
2022-06-30T02:46:03.130229Z | Debug | Finished: QualifyImportedNames.TypeCheck Took: 0.00s
2022-06-30T02:46:03.130231Z | Debug | Finished: changeTypeSignature.GetParsedModule Took: 0.00s
2022-06-30T02:46:03.130253Z | Debug | Finished: importLens Took: 0.00s
2022-06-30T02:46:03.130253Z | Debug | Finished: Pragmas.GhcSession Took: 0.00s
2022-06-30T02:46:03.130330Z | Debug | Finished: GhcideCodeActions.GhcSession Took: 0.00s
2022-06-30T02:46:03.130362Z | Debug | Finished: GhcideCodeActions.GhcSession Took: 0.00s
2022-06-30T02:46:03.130374Z | Debug | Finished: GADT.GetParsedModuleWithComments Took: 0.00s
2022-06-30T02:46:03.130375Z | Debug | Finished: GhcideCodeActions.GetFileContents Took: 0.00s
2022-06-30T02:46:03.130395Z | Debug | Finished: Hlint.GetFileContents Took: 0.00s
2022-06-30T02:46:03.130406Z | Debug | Finished: alternateNumberFormat.CollectLiterals Took: 0.00s
2022-06-30T02:46:03.130434Z | Debug | Finished: Hlint.GetModSummary Took: 0.00s
2022-06-30T02:46:03.130441Z | Debug | Finished: Pragmas.GetFileContents Took: 0.00s
2022-06-30T02:46:03.130441Z | Debug | Finished: GhcideCodeActions.GetAnnotatedParsedSource Took: 0.00s
2022-06-30T02:46:03.130448Z | Debug | Finished: GhcideCodeActions.GetGlobalBindingTypeSigs Took: 0.00s
2022-06-30T02:46:03.130460Z | Debug | Finished: GhcideCodeActions.GetAnnotatedParsedSource Took: 0.00s
2022-06-30T02:46:03.130467Z | Debug | Finished: Pragmas.GetParsedModule Took: 0.00s
2022-06-30T02:46:03.130501Z | Debug | Finished: GhcideCodeActions.GetParsedModuleWithComments Took: 0.00s
2022-06-30T02:46:03.130506Z | Debug | Finished: GhcideCodeActions.TypeCheck Took: 0.00s
2022-06-30T02:46:03.130508Z | Debug | Finished: GhcideCodeActions.getIdeOptions Took: 0.00s
2022-06-30T02:46:03.130526Z | Debug | Finished: GhcideCodeActions.GetParsedModuleWithComments Took: 0.00s
2022-06-30T02:46:03.130571Z | Debug | Finished: GhcideCodeActions.GetFileContents Took: 0.00s
2022-06-30T02:46:03.130578Z | Debug | Finished: GhcideCodeActions.GetBindings Took: 0.00s
2022-06-30T02:46:03.130651Z | Debug | Finished: GhcideCodeActions.GetParsedModuleWithComments Took: 0.00s
2022-06-30T02:46:03.130669Z | Debug | Finished: GhcideCodeActions.GetFileContents Took: 0.00s
2022-06-30T02:46:03.130697Z | Debug | Finished: GhcideCodeActions.getParsedModule Took: 0.00s
Finished: RefineImports Took: 0.00s
Finished: QualifyImportedNames.TypeCheck Took: 0.00s
Finished: changeTypeSignature.GetParsedModule Took: 0.00s
Finished: importLens Took: 0.00s
Finished: Pragmas.GhcSession Took: 0.00s
Finished: GhcideCodeActions.GhcSession Took: 0.00s
Finished: GhcideCodeActions.GhcSession Took: 0.00s
Finished: GADT.GetParsedModuleWithComments Took: 0.00s
Finished: GhcideCodeActions.GetFileContents Took: 0.00s
Finished: Hlint.GetFileContents Took: 0.00s
Finished: alternateNumberFormat.CollectLiterals Took: 0.00s
Finished: Hlint.GetModSummary Took: 0.00s
Finished: Pragmas.GetFileContents Took: 0.00s
Finished: GhcideCodeActions.GetAnnotatedParsedSource Took: 0.00s
Finished: GhcideCodeActions.GetGlobalBindingTypeSigs Took: 0.00s
Finished: GhcideCodeActions.GetAnnotatedParsedSource Took: 0.00s
Finished: Pragmas.GetParsedModule Took: 0.00s
Finished: GhcideCodeActions.GetParsedModuleWithComments Took: 0.00s
Finished: GhcideCodeActions.TypeCheck Took: 0.00s
Finished: GhcideCodeActions.getIdeOptions Took: 0.00s
Finished: GhcideCodeActions.GetParsedModuleWithComments Took: 0.00s
Finished: GhcideCodeActions.GetFileContents Took: 0.00s
Finished: GhcideCodeActions.GetBindings Took: 0.00s
Finished: GhcideCodeActions.GetParsedModuleWithComments Took: 0.00s
Finished: GhcideCodeActions.GetFileContents Took: 0.00s
Finished: GhcideCodeActions.getParsedModule Took: 0.00s
2022-06-30T02:46:03.399632Z | Debug | src/Ide/Plugin/Eval/CodeLens.hs:127:15 "fp" "/home/kokobd/work/github.com/kokobd/haskell-language-server/refactor-flake/ghcide/src/Development/IDE/Plugin/CodeAction.hs"
src/Ide/Plugin/Eval/CodeLens.hs:127:15 "fp" "/home/kokobd/work/github.com/kokobd/haskell-language-server/refactor-flake/ghcide/src/Development/IDE/Plugin/CodeAction.hs"
2022-06-30T02:46:03.399808Z | Debug | Finished: codeLens.GhcSession Took: 0.00s
2022-06-30T02:46:03.399822Z | Debug | Finished: RefineImports Took: 0.00s
2022-06-30T02:46:03.399822Z | Debug | Finished:  Took: 0.00s
2022-06-30T02:46:03.399869Z | Debug | Finished: codeLens.TypeCheck Took: 0.00s
Finished: codeLens.GhcSession Took: 0.00s
2022-06-30T02:46:03.399876Z | Debug | Finished: ModuleName.ghcSession Took: 0.00s
2022-06-30T02:46:03.399895Z | Debug | Finished: codeLens.GetBindings Took: 0.00s
2022-06-30T02:46:03.399909Z | Debug | Finished: eval.GetParsedModuleWithComments Took: 0.00s
2022-06-30T02:46:03.399916Z | Debug | Finished: codeLens.GetGlobalBindingTypeSigs Took: 0.00s
Finished: RefineImports Took: 0.00s
Finished:  Took: 0.00s
Finished: codeLens.TypeCheck Took: 0.00s
Finished: ModuleName.ghcSession Took: 0.00s
Finished: codeLens.GetBindings Took: 0.00s
Finished: eval.GetParsedModuleWithComments Took: 0.00s
Finished: codeLens.GetGlobalBindingTypeSigs Took: 0.00s
2022-06-30T02:46:14.475744Z | Info | Live bytes: 26.33MB Heap size: 591.40MB
[Info  - 10:46:14 AM] Live bytes: 26.33MB Heap size: 591.40MB

My thoughts on this

There might be something wrong on haskell.nix side, but HLS shouldn't panic on partial function in any case. So something has to be fixed in HLS too.

[michaelpj]

Hmmm, I actually included Map.! in my hlint partial functions config, and the only occurrence is in Wingman.LanguageServer, I think. Are you using wingman? Does turning it off help?

[PaulJohnson]

I have this bug too, and I'm not using Wingman. It started after our latest upgrade to NixOS. An upgrade to GHC 9.2.3 was part of that.

[michaelpj]

The problem with this is that it could conceivably come from a dependency too :(

[michaelpj]

It's also possible that my hlinting was somehow insufficient and there are more lurking.

[wz1000]

You can compile HLS with profiling and use +RTS -xc to get a backtrace that should allow you to locate the source of the exception. There will be many irrelevant backtraces too though because of how normal HLS operation relies on exceptions, but the desired one should also be in there.

[kokobd]

You can compile HLS with profiling and use +RTS -xc to get a backtrace

The profiling build in my config failed when I tried last time. I'll give it another try when I have more time.

@PaulJohnson May I ask, are you using haskell.nix?

[PaulJohnson]

@kokobd Its complicated. My employer uses their own Nix based on haskell.nix. The workaround in your original report works for us.

[pepeiborra]

Guess:

https://github.com/pepeiborra/ghc-check/blob/master/src/GHC/Check.hs#L119

[pepeiborra]

Guess:

https://github.com/pepeiborra/ghc-check/blob/master/src/GHC/Check.hs#L119

[kokobd]

https://github.com/pepeiborra/ghc-check/blob/master/src/GHC/Check.hs#L119

I can verify it's indeed the case. I forked ghc-check, and added a debug trace here:

checkGhcVersion ::
  [(String, PackageVersion)] ->
  GhcVersionChecker
checkGhcVersion compileTimeVersions runTimeLibdir = do
  let compileTimeVersionsMap = Map.fromList compileTimeVersions
      compileTime = version $ trace ("compileTimeVersionsMap: " <> show compileTimeVersionsMap) (compileTimeVersionsMap Map.! "ghc")

Log:

compileTimeVersionsMap: fromList [("base",PackageVersion {myVersion = MyVersion (Version {versionBranch = [4,16,2,0], versionTags = []}), abi = Just "bb1baa13f19181092e6a4041774cd96d"})]

We can see ghc is not in the compileTimeVersionsMap

[kokobd]

I tried all four combinations of two known configs that will affect this behavior:

1. disable cabal flag ghc-check-use-package-abis, not configure modules in flake.nix.
   • Couldn't find a target code insterpreter. Try with -fexternal-interpreter
2. disable cabal flag ghc-check-use-package-abis, configure modules in flake.nix
   • works
3. enable cabal flag ghc-check-use-package-abis (default), not configure modules in flake.nix
   • Map.!: given key is not an element in the map
4. enable cabal flag ghc-check-use-package-abis (default), configure modules in flake.nix
   • works

Conclusion:

1. something is wrong on haskell.nix side. The -fexternal-interpreter issue is not actually resolved. (I run nix flake update)
2. when ghc-check-use-package-apis is enabled (which is the default), it should not panic.

[kokobd]

related issue: input-output-hk/haskell.nix#1527

[kokobd]

Update: I fixed the problem of haskell.nix input-output-hk/haskell.nix#1536