Skip to content

Improve binary caching documentation, make more salient #875

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 13, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,21 @@ its dependencies into Nix code.

- [Introduction](https://input-output-hk.github.io/haskell.nix/)
- [Getting Started](https://input-output-hk.github.io/haskell.nix/tutorials/getting-started/)
- [Troubleshooting](https://input-output-hk.github.io/haskell.nix/troubleshooting/)
- Explore the documentation from there to find further topics.

## Help! Something isn't working

The #1 problem that people have when using `haskell.nix` is that they find themselves building GHC.
This should not happen, but you *must* follow the `haskell.nix` setup instructions properly to avoid it.
If you find this happening to you, please check that you have followed the
[getting started instructions](https://input-output-hk.github.io/haskell.nix/tutorials/getting-started/#setting-up-the-binary-cache) and
consult the corresponding [troubleshooting section](https://input-output-hk.github.io/haskell.nix/troubleshooting/#why-am-i-building-ghc).

The troubleshooting documentation also contains some help for other common issues.
Please give it a read before asking on IRC or opening an issue.

If you're still stuck, join the [#haskell.nix](https://www.irccloud.com/invite?channel=%23haskell.nix&hostname=irc.freenode.net&port=6697&ssl=1) channel on [irc.freenode.net](https://freenode.net/), or open an issue.

## Related repos

Expand All @@ -22,7 +37,3 @@ Haskell packages in Nix. It depends on other repos, which are:

- [`stackage.nix`](https://github.com/input-output-hk/stackage.nix) — all of the [Stackage](https://www.stackage.org/) snapshots, converted to Nix expressions.

## IRC Channel

Join the [#haskell.nix](https://www.irccloud.com/invite?channel=%23haskell.nix&hostname=irc.freenode.net&port=6697&ssl=1) channel on [irc.freenode.net](https://freenode.net/) to get help or discuss
the development of `haskell.nix` and `nix-tools`.
6 changes: 2 additions & 4 deletions docs/reference/supported-ghc-versions.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Supported GHC Versions

The following versions of GHC built on the CI servers and should be included
in the cache (for the default haskell.nix `nixpkgs`).
in the binary cache (for the default haskell.nix `nixpkgs`).

* 8.6.5 `compiler-nix-name = "ghc865";`
* 8.8.3 `compiler-nix-name = "ghc883";`
Expand All @@ -27,6 +27,4 @@ The following GHC versions are not included in CI and will not be cached:
See [overlays/bootstrap.nix](https://github.com/input-output-hk/haskell.nix/blob/master/overlays/bootstrap.nix)
for a list of all the valid `compiler-nix-names`.

See also:

* [Instructions on adding new GHC versions](../dev/adding-new-ghc.md).
See also: [Instructions on adding new GHC versions](../dev/adding-new-ghc.md).
48 changes: 24 additions & 24 deletions docs/troubleshooting.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,29 @@
# Troubleshooting

## Issues with building and garbage-collection

### Why am I building GHC?

It's easier to list the reverse: when will you *not* build GHC?

- You have configured the [binary cache](tutorials/getting-started.md) correctly.
- You are using one of the GHC versions which [we support](reference/supported-ghc-versions.md).
- You are using one of the nixpkgs versions used by our CI (you can access the sources for these [through haskell.nix](tutorials/getting-started.md).

If you think you are doing all of these and you still find you're building GHC, drop us a line.

### Why am I building lots of Haskell packages?

We don't generally cache much of Hackage (there's a lot of it!), except for the parts which are used by our tests.
So this is expected, unfortunately.

### How do I prevent the evaluation-time dependencies of my project from being garbage-collected?

The `haskell-nix.roots "ghc884"` should include all the evaluation-time dependencies
and the main build time dependencies of a project using ghc 8.8.4.
So you can add that to the relevant GC root.
In practice, if you're using a CI system like Hydra/Hercules, this means adding it to a job in `release.nix`/`ci.nix`.

## General troubleshooting when using `cabalProject`/`stackProject`/`project`

### Does the cabal/stack build work?
Expand Down Expand Up @@ -46,27 +70,3 @@ Where possible, try to do the configuration in your cabal/stack configuration, e
This will ensure that the two builds agree.

If you want or need to set some of them in Nix, try bringing the two into sync temporarily for troubleshooting.

## Specific issues

### Why am I building GHC?

It's easier to list the reverse: when will you *not* build GHC?

- You have configured the [binary caches](tutorials/getting-started.md) correctly.
- You are using one of the GHC versions built by our CI (have a look in [ci.nix](../ci.nix) to see what gets built).
- You are using one of the nixpkgs versions used by our CI (you can access the sources for these [through haskell.nix](tutorials/getting-started.md).

If you think you are doing all of these and you still find you're building GHC, drop us a line.

### Why am I building lots of Haskell packages?

We don't generally cache much of Hackage (there's a lot of it!), except for the parts which are used by our tests.
So this is expected, unfortunately.

### How do I prevent the evaluation-time dependencies of my project from being garbage-collected?

The `haskell-nix.roots "ghc884"` should include all the evaluation-time dependencies
and the main build time dependencies of a project using ghc 8.8.4.
So you can add that to the relevant GC root.
In practice, if you're using a CI system like Hydra/Hercules, this means adding it to a job in `release.nix`/`ci.nix`.
20 changes: 6 additions & 14 deletions docs/tutorials/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,29 +8,21 @@ project and its dependencies into Nix code.
Assuming you have [Nix](https://nixos.org/download.html) installed, you can
start setting up your project.

## Setting up the Cachix binary cache
## Setting up the binary cache

You can **avoid compiling GHC and nix-tools** by configuring
[Cachix](https://cachix.org) so you can benefit from the binary cache built by
CI:
IMPORTANT: you *must* do this or you *will* build several copies of GHC!

```bash
$ nix-env -iA cachix -f https://cachix.org/api/v1/install
installing 'cachix-0.3.8'
building '/nix/store/bh176xhpk4wrjm56iahm86wf85jaz23v-user-environment.drv'...
created 42 symlinks in user environment
You can configure Nix to use our binary cache, which is pushed to by CI, so should contain the artifacts that you need.

$ cachix use iohk
Configured https://iohk.cachix.org binary cache in ~/.config/nix/nix.conf
```

Note: `haskell.nix` currently uses multiple CI providers to build derivations and store outputs. To improve your chances of getting a cache hit, you might want to add the following additional substituter to `~/.config/nix/nix.conf`:
You need to add the following sections to `/etc/nix/nix.conf` or, if you are a trusted user, `~/.config/nix/nix.conf` (if you don't know what a "trusted user" is, you probably want to do the former).

```
trusted-public-keys = [...] hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ= [...]
substituters = [...] https://hydra.iohk.io [...]
```

This can be tricky to get setup properly. If you're still having trouble getting cache hits, consult the corresponding [troubleshooting section](../reference/troubleshooting#why-am-i-building-ghc).

## Scaffolding

The following work with `stack.yaml` and `cabal.project` based
Expand Down