Skip to content

Commit cbec10a

Browse files
sevanspowellsevanspowell
committed
Doco
1 parent 3d1bbcd commit cbec10a

File tree

4 files changed

+215
-11
lines changed

4 files changed

+215
-11
lines changed

‎docs/dev/coverage.md

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
# Developer Coverage Overview
2+
3+
## Building
4+
5+
The implementation of coverage starts with the "doCoverage" flag on
6+
the builder in `comp-builder.nix`. The doCoverage flag enables and
7+
disables the Cabal coverage flag and copies any generated coverage
8+
data to "$out/share/hpc".
9+
10+
There also exists a `covered` attribute on each component, it takes
11+
the existing derivation for that component and modifies it to have
12+
coverage enabled.
13+
14+
## Mix and tix files
15+
16+
The coverage information for any derivation consists of "mix" and
17+
"tix" files.
18+
19+
Mix files record static information about a source file and are
20+
generated at build time. They primarily contain a path to the source
21+
file and information about expressions and regions of the source file,
22+
which are later referenced by tix files.
23+
24+
Tix files contain dynamic information about a test run, recording when
25+
a portion of a source file is touched by a test. These are generated
26+
when the test is run.
27+
28+
## Coverage reports
29+
30+
### Package reports
31+
32+
The coverage information generated will look something like this:
33+
34+
```bash
35+
/nix/store/...-my-project-0.1.0.0-coverage-report/
36+
└── share
37+
└── hpc
38+
├── mix
39+
│   ├── my-library-0.1.0.0
40+
│   │   └── my-library-0.1.0.0-ERSaOroBZhe9awsoBkhmcV
41+
│   │   ├── My.Lib.Config.mix
42+
│   │   ├── My.Lib.Types.mix
43+
│   │   └── My.Lib.Util.mix
44+
│   └── my-test-1
45+
│   ├── Spec.mix
46+
│   └── Main.mix
47+
├── tix
48+
│ ├── my-library-0.1.0.0
49+
│ │   └── my-library-0.1.0.0.tix
50+
│ └── my-test-1
51+
│ └── my-test-1.tix
52+
└── html
53+
├── my-library-0.1.0.0
54+
│   ├── hpc_index_alt.html
55+
│   ├── hpc_index_exp.html
56+
│   ├── hpc_index_fun.html
57+
│   └── hpc_index.html
58+
└── my-test-1
59+
├── hpc_index_alt.html
60+
├── hpc_index_exp.html
61+
├── hpc_index_fun.html
62+
└── hpc_index.html
63+
```
64+
65+
- The mix files are copied verbatim from the builds with coverage.
66+
- The tix files for each test are copied from the check run verbatim.
67+
- The tix files for each library are generated by summing the tix
68+
files for each test, but excluding any test modules.
69+
- Test modules are determined by inspecting the plan for the project
70+
(i.e. for the project "my-project" and test-suite "my-test-1", the
71+
test modules are read from:
72+
`my-project.project.pkg-set.config.packages.my-project.components.tests.my-test-1.modules`)
73+
- The hpc reports for each component are generated from their
74+
respective tix files.
75+
- `html/my-library-0.1.0.0/hpc_index.html` contains coverage
76+
information for the library, excluding any test modules.
77+
- `html/my-test-1/hpc_index.html` contains the coverage information
78+
that the test contributes to the library, excluding any test modules.
79+
- The library might have 100% coverage, 50% generated by test-1
80+
and 50% generated by test-2. The HTML reports for each test will
81+
only show 50% coverage, but the library will show 100% coverage.
82+
83+
### Project-wide reports
84+
85+
The coverage information for an entire project will look something
86+
like this:
87+
88+
```bash
89+
/nix/store/...-coverage-report
90+
└── share
91+
└── hpc
92+
├── mix
93+
│   ├── my-library-0.1.0.0
94+
│   │   └── my-library-0.1.0.0-ERSaOroBZhe9awsoBkhmcV
95+
│   │   ├── My.Lib.Config.mix
96+
│   │   ├── My.Lib.Types.mix
97+
│   │   └── My.Lib.Util.mix
98+
│   ├── my-test-1
99+
│   │   ├── Spec.mix
100+
│   │   └── Main.mix
101+
│   ├── other-library-0.1.0.0
102+
│   │   └── other-library-0.1.0.0-48EVZBwW9Kj29VTaRMhBDf
103+
│   │   ├── Other.Lib.A.mix
104+
│   │   └── Other.Lib.B.mix
105+
│   └── other-test-1
106+
│   ├── Spec.mix
107+
│   └── Main.mix
108+
├── tix
109+
│ ├── all
110+
│ │   └── all.tix
111+
│ ├── my-library-0.1.0.0
112+
│ │   └── my-library-0.1.0.0.tix
113+
│ ├── my-test-1
114+
│ │   └── my-test-1.tix
115+
│ ├── other-library-0.1.0.0
116+
│ │   └── other-library-0.1.0.0.tix
117+
│ └── other-test-1
118+
│ └── other-test-1.tix
119+
└── html
120+
├── my-library-0.1.0.0
121+
│   ├── my-library-0.1.0.0-ERSaOroBZhe9awsoBkhmcV
122+
│   │ ├── My.Lib.Config.hs.html
123+
│   │ ├── My.Lib.Types.hs.html
124+
│   │ └── My.Lib.Util.hs.html
125+
│   ├── hpc_index_alt.html
126+
│   ├── hpc_index_exp.html
127+
│   ├── hpc_index_fun.html
128+
│   └── hpc_index.html
129+
├── my-test-1
130+
│   ├── my-library-0.1.0.0-ERSaOroBZhe9awsoBkhmcV
131+
│   │ ├── My.Lib.Config.hs.html
132+
│   │ ├── My.Lib.Types.hs.html
133+
│   │ └── My.Lib.Util.hs.html
134+
│   ├── hpc_index_alt.html
135+
│   ├── hpc_index_exp.html
136+
│   ├── hpc_index_fun.html
137+
│   └── hpc_index.html
138+
├── other-libray-0.1.0.0
139+
│   ├── other-library-0.1.0.0-48EVZBwW9Kj29VTaRMhBDf
140+
│   │   ├── Other.Lib.A.hs.html
141+
│   │   └── Other.Lib.B.hs.html
142+
│   ├── hpc_index_alt.html
143+
│   ├── hpc_index_exp.html
144+
│   ├── hpc_index_fun.html
145+
│   └── hpc_index.html
146+
├── other-test-1
147+
│   ├── other-library-0.1.0.0-48EVZBwW9Kj29VTaRMhBDf
148+
│   │   ├── Other.Lib.A.hs.html
149+
│   │   └── Other.Lib.B.hs.html
150+
│   ├── hpc_index_alt.html
151+
│   ├── hpc_index_exp.html
152+
│   ├── hpc_index_fun.html
153+
│   └── hpc_index.html
154+
└── index.html
155+
```
156+
157+
All of the coverage information is copied verbatim from the coverage
158+
reports for each of the constituent packages. A few additions are
159+
made:
160+
- `tix/all/all.tix` is generated from the union of all the library
161+
tix files.
162+
- We use this file when generating coverage reports for
163+
"coveralls.io".
164+
- An index page (`html/index.html`) is generated which links to the
165+
HTML coverage reports of the constituent packages.

‎docs/tutorials/coverage.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Coverage
2+
3+
haskell.nix can generate coverage information for your package or
4+
project using Cabal's inbuilt hpc support.
5+
6+
## Pre-requisites
7+
8+
It is currently required that you enable coverage for each library you
9+
want coverage for prior to attempting to generate a coverage report. I
10+
hope to fix this before merging this PR:
11+
12+
```nix
13+
haskell-nix.cabalProject ({
14+
src = pkgs.haskell-nix.haskellLib.cleanGit {
15+
name = "haskell-nix-project";
16+
src = ./.;
17+
};
18+
modules = [
19+
{
20+
packages.package-1.components.library.doCoverage = true;
21+
packages.package-2.components.library.doCoverage = true;
22+
}
23+
];
24+
});
25+
```
26+
27+
## Per-package
28+
29+
```bash
30+
nix-build default.nix -A $pkg.coverageReport
31+
```
32+
33+
## Project-wide
34+
35+
```bash
36+
nix-build default.nix -A projectCoverageReport
37+
```
38+
39+
This will generate a coverage report for all the local packages in
40+
your project, the directory tree will look something like this:
41+
42+
```bash
43+
```
44+
45+
"all" is a synthetic test target that sums the test coverage
46+
information from all of your test suites.
47+
48+
all/index.html is the HTML coverage report for the project

‎lib/cover.nix

Lines changed: 0 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,6 @@
22

33
{ name, version, library, tests, plan }:
44

5-
# nix-repl> haskellPackages.cardano-launcher.project.pkg-set.config.packages.cardano-launcher.components.tests.cardano-launcher-test.modules
6-
# plan.components.tests.cardano-launcher-test.modules
7-
85
let
96
buildWithCoverage = builtins.map (d: d.covered);
107
runCheck = builtins.map (d: haskellLib.check d);
@@ -42,7 +39,6 @@ in stdenv.mkDerivation {
4239
for drv in ${lib.concatStringsSep " " ([ libraryCovered ] ++ testsWithCoverage)}; do
4340
# Copy over mix files
4441
local mixDir=$(findMixDir $drv)
45-
echo "MIXDIR IS: $mixDir"
4642
cp -R $mixDir $out/share/hpc/mix/
4743
4844
hpcMarkupCmdBase+=("--hpcdir=$mixDir")
@@ -56,7 +52,6 @@ in stdenv.mkDerivation {
5652
for module in $testModules; do
5753
excludedModules+=("$module")
5854
done
59-
echo "Excluded modules: ''${excludedModules[@]}"
6055
6156
hpcSumCmdBase=("hpc" "sum" "--union" "--output=$out/share/hpc/tix/${identifier}/${identifier}.tix")
6257
for exclude in ''${excludedModules[@]}; do
@@ -73,9 +68,7 @@ in stdenv.mkDerivation {
7368
pushd ${check}/share/hpc/tix
7469
7570
tixFileRel="$(find . -iwholename "*.tix" -type f -print -quit)"
76-
echo "TIXFILEREL: $tixFileRel"
7771
78-
# Output tix file as-is with suffix
7972
mkdir -p $out/share/hpc/tix/$(dirname $tixFileRel)
8073
cp $tixFileRel $out/share/hpc/tix/$tixFileRel
8174
@@ -84,7 +77,6 @@ in stdenv.mkDerivation {
8477
8578
hpcMarkupCmdEachTest+=("$out/share/hpc/tix/$tixFileRel")
8679
87-
echo "''${hpcMarkupCmdEachTest[@]}"
8880
eval "''${hpcMarkupCmdEachTest[@]}"
8981
9082
popd
@@ -93,11 +85,8 @@ in stdenv.mkDerivation {
9385
9486
hpcMarkupCmdAll+=("$out/share/hpc/tix/${identifier}/${identifier}.tix")
9587
96-
echo "''${hpcSumCmd[@]}"
9788
eval "''${hpcSumCmd[@]}"
9889
99-
echo "''${hpcMarkupCmdAll[@]}"
10090
eval "''${hpcMarkupCmdAll[@]}"
101-
10291
'';
10392
}

‎mkdocs.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ pages:
3535
- 'Bumping Hackage and Stackage snapshots': tutorials/hackage-stackage.md
3636
- 'Materialization: Speeding up Nix evaluation': tutorials/materialization.md
3737
- 'Cross-compiling your project': tutorials/cross-compilation.md
38+
- 'Generating coverage information': tutorials/coverage.md
3839
- 'Reference':
3940
- 'Command-line tools': reference/commands.md
4041
- 'Haskell.nix Library': reference/library.md
@@ -50,4 +51,5 @@ pages:
5051
- 'Nixpkgs Pin': dev/nixpkgs-pin.md
5152
- 'Removing withPackage wrapper': dev/removing-with-package-wrapper.md
5253
- 'Test Suite': dev/tests.md
54+
- 'Coverage': dev/coverage.md
5355
- 'ChangeLog': changelog.md

0 commit comments

Comments
 (0)