Skip to content

Upgrade the Nix build system #639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Dec 2, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 0 additions & 28 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -171,33 +171,6 @@ jobs:
key: cabal-{{ .Environment.HIE_CACHE }}-{{ arch }}-{{ .Environment.CIRCLE_JOB }}
paths:
- ~/.cabal
nix:
working_directory: ~/build
environment:
- NIXPKGS_ALLOW_BROKEN: 1
docker:
- image: nixorg/nix:circleci
steps:
- checkout
- run:
name: Sync submodules
command: git submodule sync --recursive
- run:
name: Update submodules
command: git submodule update --recursive --init
- restore-cache:
keys:
- cabal-{{ .Environment.HIE_CACHE }}-{{ arch }}-{{ .Environment.CIRCLE_JOB }}
- run:
name: Build
command: nix-shell -j4 --run "cabal new-update && (cabal new-build -j1 --enable-tests || cabal new-build -j1 --enable-tests) "
no_output_timeout: 30m
- save_cache:
key: cabal-{{ .Environment.HIE_CACHE }}-{{ arch }}-{{ .Environment.CIRCLE_JOB }}
paths:
- ~/.cabal



workflows:
version: 2
Expand All @@ -212,4 +185,3 @@ workflows:
- ghc-8.10.2
# - ghc-nightly
- cabal
- nix
26 changes: 26 additions & 0 deletions .github/workflows/nix.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Nix

on: [push, pull_request]
jobs:
nix:
runs-on: ${{ matrix.os }}

strategy:
fail-fast: false
matrix:
ghc: ['default', 'ghc8102', 'ghc884', 'ghc865']
os: [ubuntu-latest, macOS-latest]

steps:
- uses: actions/checkout@v2
with:
submodules: true
- uses: cachix/install-nix-action@v12
with:
nix_path: nixpkgs=channel:nixos-20.03
- uses: cachix/cachix-action@v8
with:
name: haskell-language-server
extraPullNames: haskell-ghcide
authToken: '${{ secrets.HLS_CACHIX_AUTH_TOKEN }}'
- run: nix-shell --argstr compiler ${{ matrix.ghc }} --run "cabal update && cabal build"
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -701,6 +701,19 @@ $ cd install
$ stack build
```

#### Using Nix

The instructions below show how to set up a Cachix binary cache and open a nix shell for local development.

```shell
$ cachix use haskell-language-server
$ nix-shell
$ cabal update
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Cabal part is unrelated to Nix, as one can use Nix with Stack. How about removing the 2 Cabal commands?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Nix script is specifically targeted at using with cabal build

$ cabal build
```

If you are looking for a Nix expression to create haskell-language-server binaries, see https://github.com/haskell/haskell-language-server/issues/122

#### Introduction tutorial
Pepeiborra [wrote an tutorial](https://github.com/pepeiborra/hls-tutorial) on writing a plugin in HLS.

Expand Down
37 changes: 37 additions & 0 deletions nix/default.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
{ sources ? import ./sources.nix }:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you guys think about adding nixfmt to shell.nix packages and use it on all Nix files? This solves all formatting concerns.

This may also help, as e.g. here the let .. in .. was not aligned and I had slight trouble understanding 😅

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are welcome to do this, please send a PR!

let
overlay = _self: pkgs:
let sharedOverrides = {
overrides = _self: super: {
mkDerivation = args: super.mkDerivation (args //
{
# skip running tests for Hackage packages
doCheck = args.pname != "ghcide" && args.pname != "haskell-language-server";
# relax upper bounds
jailbreak = args.pname != "jailbreak-cabal";
});
};
};
extended = haskellPackages:
haskellPackages.extend (pkgs.haskell.lib.packageSourceOverrides {
haskell-language-server = ../.;
ghcide = ../ghcide;
hie-compat = ../ghcide/hie-compat;
hls-plugin-api = ../hls-plugin-api;
hls-tactics-plugin = ../plugins/tactics;
hls-hlint-plugin = ../plugins/hls-hlint-plugin;
});
in
{
inherit (import sources.gitignore { inherit (pkgs) lib; }) gitignoreSource;
ourHaskell = pkgs.haskell // {
packages = pkgs.haskell.packages // {
# relax upper bounds on ghc 8.10.x versions (and skip running tests)
ghc8101 = extended (pkgs.haskell.packages.ghc8101.override sharedOverrides);
ghc8102 = extended (pkgs.haskell.packages.ghc8102.override sharedOverrides);
};
};
};

in import sources.nixpkgs
{ overlays = [ overlay ] ; config = {allowBroken = true;}; }
22 changes: 17 additions & 5 deletions nix/sources.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,16 @@
{
"gitignore": {
"branch": "master",
"description": "Nix function for filtering local git sources",
"homepage": "",
"owner": "hercules-ci",
"repo": "gitignore",
"rev": "c4662e662462e7bf3c2a968483478a665d00e717",
"sha256": "1npnx0h6bd0d7ql93ka7azhj40zgjp815fw2r6smg8ch9p7mzdlx",
"type": "tarball",
"url": "https://github.com/hercules-ci/gitignore/archive/c4662e662462e7bf3c2a968483478a665d00e717.tar.gz",
"url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
},
"niv": {
"branch": "master",
"description": "Easy dependency management for Nix projects",
Expand All @@ -12,15 +24,15 @@
"url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
},
"nixpkgs": {
"branch": "nixpkgs-unstable",
"branch": "haskell-updates",
"description": "A read-only mirror of NixOS/nixpkgs tracking the released channels. Send issues and PRs to",
"homepage": "https://github.com/NixOS/nixpkgs",
"owner": "NixOS",
"repo": "nixpkgs-channels",
"rev": "a332da8588aeea4feb9359d23f58d95520899e3c",
"sha256": "18hlja5syv3xpi14c07h9lrn1cchq2azmj06fyalq52vl064nx75",
"repo": "nixpkgs",
"rev": "cbf707b2af4cd66bdcdd84b2bddfb627194b39c0",
"sha256": "1bj55jkgdzx7lv6dvy26l354c7vsxaa1jipm99axgqqax905m8sa",
"type": "tarball",
"url": "https://github.com/NixOS/nixpkgs-channels/archive/a332da8588aeea4feb9359d23f58d95520899e3c.tar.gz",
"url": "https://github.com/NixOS/nixpkgs/archive/cbf707b2af4cd66bdcdd84b2bddfb627194b39c0.tar.gz",
"url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
}
}
135 changes: 31 additions & 104 deletions shell.nix
Original file line number Diff line number Diff line change
@@ -1,125 +1,52 @@
# This shell.nix file is designed for use with cabal build
# It aims to leverage the nix cache in as much as possible
# while reducing Nix maintenance costs.
# It does **not** aim to replace Cabal/Stack with Nix
# It does **not** aim to replace Cabal

# Maintaining this file:
#
# - Dealing with broken nix-shell
# - Bump the nixpkgs version using `niv update nixpkgs`
# - To edit the set of local packages:
# 1. Declare them in nix/default.nix
# 2. Edit the list of packages below
#
# 1. Bump the nixpkgs version using `niv update nixpkgs`
# 2. Comment out any remaining failing packages
#
# - Dealing with broken cabal build inside nix-shell:
#
# If my understanding of cabal new-build is correct this should never happen,
# assuming that cabal new-build does succeed outside nix-shell
# For more details: https://github.com/NixOS/nixpkgs/blob/20.03/pkgs/development/haskell-modules/make-package-set.nix#L256


{ sources ? import nix/sources.nix,
nixpkgs ? import sources.nixpkgs { },
compiler ? "default",
hoogle ? false
{ compiler ? "default",
withHoogle ? false,
nixpkgs ? import ./nix {}
}:

with nixpkgs;

let defaultCompiler = "ghc" + lib.replaceStrings ["."] [""] haskellPackages.ghc.version;
haskellPackagesForProject = p:
if compiler == "default" || compiler == defaultCompiler
then if hoogle
then haskellPackages.ghcWithHoogle p
else haskellPackages.ghcWithPackages p
# for all other compilers there is no Nix cache so dont bother building deps
else if hoogle
then haskell.packages.${compiler}.ghcWithHoogle (_: [])
else haskell.packages.${compiler}.ghcWithPackages (_: []);
haskellPackagesForProject =
if compiler == "default"
then ourHaskell.packages.${defaultCompiler}
else ourHaskell.packages.${compiler};

packages = p: [ p.haskell-language-server
p.ghcide
p.hie-compat
p.hls-plugin-api
p.hls-tactics-plugin
p.hls-hlint-plugin
];

retrie = with haskell.lib; dontCheck(disableLibraryProfiling(haskellPackages.retrie));
compilerWithPackages = haskellPackagesForProject(p:
with p;
[
Diff
Glob
HsYAML-aeson
QuickCheck
aeson
alex
async
base16-bytestring
blaze-builder
blaze-markup
brittany
conduit-extra
conduit-parse
cryptohash-sha1
data-default
data-default-class
data-default-instances-containers
data-default-instances-dlist
data-default-instances-old-locale
extra
floskell
# fourmolu
fuzzy
generic-deriving
ghc-check
gitrev
haddock-library
happy
haskell-lsp
haskell-src-exts
hie-bios
hslogger
hspec
lens
lsp-test
megaparsec
network
opentelemetry
optparse-simple
ormolu
parser-combinators
parsers
prettyprinter
prettyprinter-ansi-terminal
primes
psqueues
regex-tdfa
retrie
rope-utf16-splay
safe-exceptions
shake
sorted-list
strict
stylish-haskell
tasty
tasty-ant-xml
tasty-expected-failure
tasty-golden
tasty-hunit
tasty-rerun
temporary
text
typed-process
unix-compat
unordered-containers
xml
yaml
zlib
]);
isSupported = compiler == "default" || compiler == defaultCompiler;
in
stdenv.mkDerivation {
name = "haskell-language-server";
haskellPackagesForProject.shellFor {
inherit withHoogle;
doBenchmark = true;
packages = p: if isSupported then packages p else [p.ghc-paths];
buildInputs = [
git
gmp
ncurses
zlib
ncurses

haskellPackages.cabal-install
haskellPackages.hlint

compilerWithPackages

haskellPackages.ormolu
haskellPackages.stylish-haskell
];
src = null;
shellHook = ''
Expand Down