stackProject git deps fail when rev is not in default branch

[roberth]

Nix builtins.fetchGit only fetches the default branch if no branch is specified.
We don't have the branch name (let alone ref) in stack.yaml so we either need to do another comment hack similar to # nix-sha256 or we need to make Nix fall back to fetching all refs. The latter may be acceptable only if it logs a warning so the user can fix it.

Discussion about fetchGit's ref: NixOS/nix#2431

Example of the error: hercules-ci/haskell-language-server.nix#2 (comment)

[cdepillabout]

We're running into this at work as well.

We're seeing messages in our CI logs that look like the following:

fatal: invalid refspec '*:17mzvwz5d5nwwxz1ffd89vnivzdbc9szi95wqp6a4576miyk2jdp'
error: program 'git' failed with exit code 128

This will hopefully help someone else that runs into this issue.

---

edit: Actually, I'm starting to wonder if this error message is not a separate issue. When calling builtins.fetchGit on a repo and passing a rev that is not in the default branch, you get an error like the following:

fatal: not a tree object: 947154e3d40373bb77841a88c2674c84a73fce80

second edit: It turns out that the invalid refspec was due to running an older version of Nix on one of our CI servers (Nix-2.1.3). However, I can't find any reference to this invalid refspec error from git anywhere on the haskell.nix or nix issue trackers.

[codygman]

I was just bit by this issue today with a recent nix on a fresh ubuntu vm:

cody@cody-VirtualBox:~/smurf$ nix-shell -p nix-info --run "nix-info -m"
these paths will be fetched (0.00 MiB download, 0.00 MiB unpacked):
  /nix/store/pr0x3kqqwwkx1n9mf9sfhls9rsfkraka-nix-info
copying path '/nix/store/pr0x3kqqwwkx1n9mf9sfhls9rsfkraka-nix-info' from 'https://cache.nixos.org'...
 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.0-37-generic, Ubuntu, 20.04 LTS (Focal Fossa)`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.6`
 - channels(cody): `"nixpkgs-20.09pre229671.9d0c3ffe678, home-manager"`
 - nixpkgs: `/home/cody/.nix-defexpr/channels/nixpkgs`

I guess in the meantime I can figure out how to add libraries with niv to haskell.nix since I'm already using niv to pin nixpkgs/haskell.nix. Otherwise my teammates can't use what works for me on NixOS with just nix/nixpkgs on ubuntu.

[codygman]

I'd be interested in fixing this issue too since it soft-blocks my team adopting haskell.nix.

It seems that callStackToNix is the piece of code that also needs to add ref?

[codygman]

I think that if we prepend refs/tags so we end up with `fetchGit { ..., rev = "/refs/tags/${sha-commit-hash}", ...} it will fix this.

[codygman]

I think https://github.com/input-output-hk/nix-tools/blob/master/lib/Stack2nix/Stack.hs is where I want to be.

[domenkozar]

One workaround is to define packages.my-package.src = buitlins.fetchgit ... as an override in the module system.

[roberth]

@domenkozar Last time I checked, that workaround only works if you're using committed nix expressions instead of stackProject.

Also note that when testing this, you need to clear both the store path and ~/.cache/nix/gitv2/ or you may get false positive results.

[balsoft]

Also relevant is NixOS/nix#2409

[purefn]

We've started seeing this issue lately too. It would be great to get some kind of fix.

Since stack doesn't have support for refs, maybe the best approach would be similar to the one for materialization and hashes, e.g. add a #nix-ref <ref> comment?

[balsoft]

@purefn Setting # nix-sha256 fixes the issue (because pkgs.fetchgit fetches all refs). I hope to see commercialhaskell/stack#5335 implemented at some point in the future as a proper fix.

[purefn]

@balsoft That's good to know, thanks!

Adding the nix-sha256 only uses pkgs.fetchgit for public git repos. If the repo is private, then builtins.fetchGit is always used, even if the nix-sha256 is present.

Update: Hmm I can't get it to work for public repos either.

Update 2: It isn't working for me because our custom packages are in a snapshot file, not in our stack.yaml file.