`additional` argument to `shellFor` can only select from packages that were going to be included anyway

[cumber]

I'm trying to add some packages to my development environment that aren't actually dependencies of my project, they're just handy to have access to during development.

The documentation describes the shellFor function having an argument called additional:

Similar to packages, but the selected packages are built and included in ghc-pkg list (not just their dependencies).

This sounded like it would do what I wanted, but when I tried I always got a missing attribute error. So I stuck in a trace to see what names were available. (e.g. additional = pkgs: builtins.trace (builtins.attrNames pkgs) [ ... ]). And the list of names available seems to be a very small set just containing what my project already depended on.

For example, with this minimal setup:

-- test-additional.cabal
cabal-version:      3.0
name:               test-additional
version:            0.1.0.0
build-type:         Simple

executable test-additional
    main-is:          Main.hs
    build-depends:    base ^>=4.14.3.0
    hs-source-dirs:   app
    default-language: Haskell2010

# default.nix
let
  sources = {
    haskellNix = builtins.fetchTarball "https://github.com/input-output-hk/haskell.nix/archive/986ebab4075d4c1ecac1270a9304e65e7ef715bc.tar.gz";
  };
  haskellNix = import sources.haskellNix {};
  pkgs = import
    haskellNix.sources.nixpkgs-unstable
    haskellNix.nixpkgsArgs;

in pkgs.haskell-nix.project {
  src = pkgs.haskell-nix.haskellLib.cleanGit {
    name = "haskell-nix-project";
    src = ./.;
  };
  compiler-nix-name = "ghc8107";
}

# shell.nix
(import ./default.nix).shellFor {
  additional = pkgs: builtins.trace (builtins.attrNames pkgs) [
    pkgs.neat-interpolation
  ];
}

I see this output:

trace: [ "BNFC" "Cabal" "GLFW-b" "Sit" "Win32" "X11" "array" "base" "binary" "bindings-GLFW" "bitvec" "buildPackages" "bytestring" "cabal-install" "clock" "closed" "conduit" "containers" "cpphs" "cryptonite-openssl" "deepseq" "directory" "discount" "filepath" "ghc" "ghc-boot" "ghc-boot-th" "ghc-heap" "ghc-lib-parser" "ghc-prim" "ghcWithHoogle" "ghcWithPackages" "ghci" "ghcide" "ghcjs-prim" "ghcjs-th" "hnix" "hpc" "hscolour" "integer-gmp" "integer-simple" "iserv" "iserv-proxy" "language-c" "libiserv" "llvm-hs" "makeConfigFiles" "mintty" "network" "odbc" "pcap" "polyparse" "pretty" "remote-iserv" "rts" "shellFor" "streaming-commons" "template-haskell" "terminfo" "test-additional" "time" "transformers" "unix" "x509-system" ]
error: attribute 'neat-interpolation' missing

If I add neat-interpolation as a dependency in my .cabal file, then the list of packages available to additional gets larger and it works. But then of course I wouldn't need additional.

I'm genuinely unsure if I'm misunderstanding what additional is supposed to be for, or if it isn't working correctly.

[michaelpj]

additional can include packages that weren't going to be included in the shell, but not packages that aren't in the build plan. haskell.nix fundamentally cannot know about those because we need cabal to tell us e.g. which version to pick and so on.

If you want to pull an additional package into your build plan, try using extra-packages. As ever, the Haskell.Nix Way is to configure what you want with cabal.

[cumber]

Thanks @michaelpj , that makes sense now!

[michaelpj]

It could still be documented better!

[cumber]

Looks like you need to use extra-packages in cabal.project and additional in Haskell.nix

[cumber]

Looks like you need to use extra-packages in cabal.project and additional in Haskell.nix

Actually, is that part a bug? Cabal's docs say "The motivation for extra-packages is making libraries that are not dependencies of any package in the project available for use in ghci", so it seems like since haskell.nix understands that field it should actually put those packages in the shell by default.

If you do put packages in extra-packages but don't do anything on the nix side, you get a development shell where you just can't start cabal repl:

Warning: The package list for 'hackage.haskell.org' does not exist. Run 'cabal
update' to download it.
Resolving dependencies...
Error: cabal: Could not resolve dependencies:
[__0] unknown package: raw-strings-qq (user goal)
[__0] fail (backjumping, conflict set: raw-strings-qq)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: raw-strings-qq

[michaelpj]

so it seems like since haskell.nix understands that field it should actually put those packages in the shell by default.

I actually don't think that haskell.nix does understand it. haskell.nix just consumes the build plan that cabal creates: with extra-packages, cabal puts more stuff in the build plan, so haskell.nix can see it. But I think you make a reasonable argument that haskell.nix should understand that such packages are meant to be "local" and should go in the shell.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[andreabedini]

The message that @cumber shows above means something different.

Warning: The package list for 'hackage.haskell.org' does not exist. Run 'cabal
update' to download it.
Resolving dependencies...
Error: cabal: Could not resolve dependencies:
[__0] unknown package: raw-strings-qq (user goal)
[__0] fail (backjumping, conflict set: raw-strings-qq)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: raw-strings-qq

What cabal is doing there is 1) reading the project configuration 2) resolve the targets as the local package plus extra-packages (at least according to a simple test I made) and try to build them 3) fail because without the hackage index it does not know anything about those packages.

Nevertheless, the issue in #1955 is valid. shellFor and cabal repl need to agree re what packages to have in scope.

@michaelpj Unless you want to turn this into a documentation issue, I'd close it in favour of #1955.
@cumber Let us know if you have found a way to get it to work.