Use forward slashes in relative paths on Windows

[inkychris]

Current Problem
I'm running coverage for multiple Python versions on both Linux and Windows. They are all run on Gitlab, with the Windows tests using Python install on the host, and with the Linux tests using Docker. The paths of the build directories are not deterministic (may be picked up by multiple runners, existing or yet to exist) so I have relative_files = True .

The Windows paths use \ instead of / which, if the coverage combination and reporting is done on Windows, results in a report with every source file twice, one for each separator, although the end result appears to be correct. If this is done on Linux, it fails entirely when trying to resolve the paths containing backslashes.

Example Report

Name                                           Stmts   Miss Branch BrPart  Cover
 --------------------------------------------------------------------------------
 exmpl\__about__.py                                 6      0      0      0   100%
 exmpl/__about__.py                                 6      0      0      0   100%
 exmpl/__init__.py                                298     11    138      5    95%
 exmpl\__init__.py                                298     11    138      5    95%
 exmpl/_logging.py                                 47      2     16      1    92%
 exmpl\_logging.py                                 47      2     16      1    92%
 exmpl/cli.py                                      55      0     28      0   100%
 exmpl\cli.py                                      55      0     28      0   100%
etc...

Describe the solution you'd like
Source paths on Windows should use forward slashes in relative paths to allow result combination and final reporting to be done on Linux, and to clean up the report output when done on Windows.

Describe alternatives you've considered
Manually modifying the coverage results generated from coverage run.

[nedbat]

Have you tried using "--parallel=True", and then "coverage combine" with a "[paths]" setting to rewrite the paths as you need?

[inkychris]

I've been manually renaming the coverage files in their respective CI jobs which I think covers the --parallel=True option. I'm not sure that the [paths] section addresses this problem though, as the path relative to the root source directory is the bit that need rewriting, unless that is doable with this section of the config?

I've updated the description to include an example report generated on Windows from 4 test runs (2 Linux, 2 Windows). This doesn't work at all trying to report on Linux because it can't find any files with backslashes in, while Windows happily deals with the forward slashes used in the Linux results.

[cjolowicz]

I can confirm that the following paths-based configuration works well for combining coverage data across platforms:

[tool.coverage.paths]
source = ["src", "*/site-packages"]
tests = ["tests", "*/tests"]

[tool.coverage.run]
source = ["yourpackage", "tests"]

This assumes a project where the package under test is located under src/yourpackage, and the test suite under tests. It also assumes that the package is installed into the virtual environment where the tests run. Editable installs are not used.

---

As for the original issue, I can confirm that as well. Since relative_files does not normalize path separators, combining coverage data across platforms using relative_files instead of paths results in errors like this:

No source for code: '/Users/cjolowicz/.../yourpackage/tests\__init__.py'.

Note the mixed forward and backward slashes in the path. This was run on macOS using both the latest release and master of Coverage.py.

I have attached the coverage data from the GitHub Action runners. Use these commands to reproduce:

git clone https://github.com/cjolowicz/cookiecutter-hypermodern-python-instance
cd cookiecutter-hypermodern-python-instance
git switch --detach 11b803bd73c6f702aeaa477f2eba78f4b101950f
unzip coverage-data.zip
coverage combine
coverage report

Below is a full traceback for the No source error.

Full traceback

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│                                                                                                  │
│ /Users/cjolowicz/Code/github.com/nedbat/coveragepy/venv-py39/bin/coverage:33 in <module>         │
│                                                                                                  │
│   30                                                                                             │
│   31 if __name__ == '__main__':                                                                  │
│   32 │   sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])                       │
│ ❱ 33 │   sys.exit(load_entry_point('coverage', 'console_scripts', 'coverage')())                 │
│   34                                                                                             │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │            __annotations__ = {}                                                              │ │
│ │               __builtins__ = <module 'builtins' (built-in)>                                  │ │
│ │                 __cached__ = None                                                            │ │
│ │                    __doc__ = None                                                            │ │
│ │                   __file__ = '/Users/cjolowicz/Code/github.com/nedbat/coveragepy/venv-py39/… │ │
│ │                 __loader__ = <_frozen_importlib_external.SourceFileLoader object at          │ │
│ │                              0x108f6fca0>                                                    │ │
│ │                   __name__ = '__main__'                                                      │ │
│ │                __package__ = None                                                            │ │
│ │               __requires__ = 'coverage'                                                      │ │
│ │                   __spec__ = None                                                            │ │
│ │               distribution = <function distribution at 0x109190160>                          │ │
│ │ importlib_load_entry_point = <function importlib_load_entry_point at 0x108fbb0d0>            │ │
│ │           load_entry_point = <function importlib_load_entry_point at 0x108fbb0d0>            │ │
│ │                         re = <module 're' from                                               │ │
│ │                              '/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framew… │ │
│ │                        sys = <module 'sys' (built-in)>                                       │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ /Users/cjolowicz/Code/github.com/nedbat/coveragepy/coverage/cmdline.py:879 in main               │
│                                                                                                  │
│   876 │   if argv is None:                                                                       │
│   877 │   │   argv = sys.argv[1:]                                                                │
│   878 │   try:                                                                                   │
│ ❱ 879 │   │   status = CoverageScript().command_line(argv)                                       │
│   880 │   except _ExceptionDuringRun as err:                                                     │
│   881 │   │   # An exception was caught while running the product code.  The                     │
│   882 │   │   # sys.exc_info() return tuple is packed into an _ExceptionDuringRun                │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │    argv = ['report']                                                                         │ │
│ │ install = <function install at 0x109e68a60>                                                  │ │
│ │     msg = "No source for code:                                                               │ │
│ │           '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hyp"+44                   │ │
│ │  status = 1                                                                                  │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /Users/cjolowicz/Code/github.com/nedbat/coveragepy/coverage/cmdline.py:625 in command_line       │
│                                                                                                  │
│   622 │   │                                                                                      │
│   623 │   │   total = None                                                                       │
│   624 │   │   if options.action == "report":                                                     │
│ ❱ 625 │   │   │   total = self.coverage.report(                                                  │
│   626 │   │   │   │   show_missing=options.show_missing,                                         │
│   627 │   │   │   │   skip_covered=options.skip_covered,                                         │
│   628 │   │   │   │   skip_empty=options.skip_empty,                                             │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │        args = []                                                                             │ │
│ │        argv = []                                                                             │ │
│ │    contexts = None                                                                           │ │
│ │       debug = None                                                                           │ │
│ │     include = None                                                                           │ │
│ │          ok = True                                                                           │ │
│ │        omit = None                                                                           │ │
│ │     options = <Values at 0x10918bd30: {'action': 'report', 'append': None, 'branch': None,   │ │
│ │               'concurrency': None, 'context': None, 'contexts': None, 'debug': None,         │ │
│ │               'directory': None, 'fail_under': None, 'help': None, 'ignore_errors': None,    │ │
│ │               'include': None, 'keep': None, 'module': None, 'omit': None, 'parallel_mode':  │ │
│ │               None, 'precision': None, 'pylib': None, 'quiet': None, 'rcfile': True,         │ │
│ │               'show_contexts': None, 'show_missing': None, 'skip_covered': None,             │ │
│ │               'skip_empty': None, 'sort': None, 'source': None, 'timid': None, 'title':      │ │
│ │               None, 'version': None}>                                                        │ │
│ │      parser = <coverage.cmdline.CmdOptionParser object at 0x109e603d0>                       │ │
│ │ report_args = {                                                                              │ │
│ │               │   'morfs': [],                                                               │ │
│ │               │   'ignore_errors': None,                                                     │ │
│ │               │   'omit': None,                                                              │ │
│ │               │   'include': None,                                                           │ │
│ │               │   'contexts': None                                                           │ │
│ │               }                                                                              │ │
│ │        self = <coverage.cmdline.CoverageScript object at 0x10918abe0>                        │ │
│ │      source = None                                                                           │ │
│ │       total = None                                                                           │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /Users/cjolowicz/Code/github.com/nedbat/coveragepy/coverage/control.py:934 in report             │
│                                                                                                  │
│    931 │   │   │   sort=sort                                                                     │
│    932 │   │   ):                                                                                │
│    933 │   │   │   reporter = SummaryReporter(self)                                              │
│ ❱  934 │   │   │   return reporter.report(morfs, outfile=file)                                   │
│    935 │                                                                                         │
│    936 │   def annotate(                                                                         │
│    937 │   │   self, morfs=None, directory=None, ignore_errors=None,                             │
│                                                                                                  │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮                     │
│ │      contexts = None                                                     │                     │
│ │          file = None                                                     │                     │
│ │ ignore_errors = None                                                     │                     │
│ │       include = None                                                     │                     │
│ │         morfs = []                                                       │                     │
│ │          omit = None                                                     │                     │
│ │     precision = None                                                     │                     │
│ │      reporter = <coverage.summary.SummaryReporter object at 0x109e606a0> │                     │
│ │          self = <coverage.control.Coverage object at 0x10918bd90>        │                     │
│ │  show_missing = None                                                     │                     │
│ │  skip_covered = None                                                     │                     │
│ │    skip_empty = None                                                     │                     │
│ │          sort = None                                                     │                     │
│ ╰──────────────────────────────────────────────────────────────────────────╯                     │
│                                                                                                  │
│ /Users/cjolowicz/Code/github.com/nedbat/coveragepy/coverage/summary.py:43 in report              │
│                                                                                                  │
│    40 │   │   self.outfile = outfile or sys.stdout                                               │
│    41 │   │                                                                                      │
│    42 │   │   self.coverage.get_data().set_query_contexts(self.config.report_contexts)           │
│ ❱  43 │   │   for fr, analysis in get_analysis_to_report(self.coverage, morfs):                  │
│    44 │   │   │   self.report_one_file(fr, analysis)                                             │
│    45 │   │                                                                                      │
│    46 │   │   # Prepare the formatting strings, header, and column sorting.                      │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ analysis = <coverage.results.Analysis object at 0x109e60640>                                 │ │
│ │       fr = <PythonFileReporter                                                               │ │
│ │            '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-python-inst… │ │
│ │    morfs = []                                                                                │ │
│ │  outfile = None                                                                              │ │
│ │     self = <coverage.summary.SummaryReporter object at 0x109e606a0>                          │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /Users/cjolowicz/Code/github.com/nedbat/coveragepy/coverage/report.py:72 in                      │
│ get_analysis_to_report                                                                           │
│                                                                                                  │
│   69 │                                                                                           │
│   70 │   for fr in sorted(file_reporters):                                                       │
│   71 │   │   try:                                                                                │
│ ❱ 72 │   │   │   analysis = coverage._analyze(fr)                                                │
│   73 │   │   except NotPython:                                                                   │
│   74 │   │   │   # Only report errors for .py files, and only if we didn't                       │
│   75 │   │   │   # explicitly suppress those errors.                                             │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │       analysis = <coverage.results.Analysis object at 0x109e60640>                           │ │
│ │         config = <coverage.config.CoverageConfig object at 0x109e60790>                      │ │
│ │       coverage = <coverage.control.Coverage object at 0x10918bd90>                           │ │
│ │ file_reporters = [                                                                           │ │
│ │                  │   <PythonFileReporter                                                     │ │
│ │                  '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-pytho… │ │
│ │                  │   <PythonFileReporter                                                     │ │
│ │                  '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-pytho… │ │
│ │                  │   <PythonFileReporter                                                     │ │
│ │                  '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-pytho… │ │
│ │                  │   <PythonFileReporter                                                     │ │
│ │                  '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-pytho… │ │
│ │                  ]                                                                           │ │
│ │             fr = <PythonFileReporter                                                         │ │
│ │                  '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-pytho… │ │
│ │          morfs = []                                                                          │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /Users/cjolowicz/Code/github.com/nedbat/coveragepy/coverage/control.py:829 in _analyze           │
│                                                                                                  │
│    826 │   │   if not isinstance(it, FileReporter):                                              │
│    827 │   │   │   it = self._get_file_reporter(it)                                              │
│    828 │   │                                                                                     │
│ ❱  829 │   │   return Analysis(data, self.config.precision, it, self._file_mapper)               │
│    830 │                                                                                         │
│    831 │   def _get_file_reporter(self, morf):                                                   │
│    832 │   │   """Get a FileReporter for a module or file name."""                               │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ data = <CoverageData @0x1096df310 _no_disk=False                                             │ │
│ │        _basename='/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-pytho… │ │
│ │        _suffix=None                                                                          │ │
│ │        _filename='/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-pytho… │ │
│ │        _file_map={'tests\\__init__.py': 1, 'tests\\test_main.py': 2,                         │ │
│ │        'src/cookiecutter_hypermodern_python_instance/__init__.py': 3,                        │ │
│ │        'src/cookiecutter_hypermodern_python_instance/__main__.py': 4} _dbs={4553778688:      │ │
│ │        <SqliteDb @0x1096df670 debug=None                                                     │ │
│ │        filename='/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-python… │ │
│ │        nest=0 con=None>} _pid=2200 _lock=<unlocked _thread.lock object at 0x1096df330>       │ │
│ │        _have_used=True _has_lines=False _has_arcs=True _current_context=None                 │ │
│ │        _current_context_id=None _query_context_ids=None>                                     │ │
│ │   it = <PythonFileReporter                                                                   │ │
│ │        '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-python-instance… │ │
│ │ self = <coverage.control.Coverage object at 0x10918bd90>                                     │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /Users/cjolowicz/Code/github.com/nedbat/coveragepy/coverage/results.py:20 in __init__            │
│                                                                                                  │
│    17 │   │   self.data = data                                                                   │
│    18 │   │   self.file_reporter = file_reporter                                                 │
│    19 │   │   self.filename = file_mapper(self.file_reporter.filename)                           │
│ ❱  20 │   │   self.statements = self.file_reporter.lines()                                       │
│    21 │   │   self.excluded = self.file_reporter.excluded_lines()                                │
│    22 │   │                                                                                      │
│    23 │   │   # Identify missing statements.                                                     │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │          data = <CoverageData @0x1096df310 _no_disk=False                                    │ │
│ │                 _basename='/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermod… │ │
│ │                 _suffix=None                                                                 │ │
│ │                 _filename='/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermod… │ │
│ │                 _file_map={'tests\\__init__.py': 1, 'tests\\test_main.py': 2,                │ │
│ │                 'src/cookiecutter_hypermodern_python_instance/__init__.py': 3,               │ │
│ │                 'src/cookiecutter_hypermodern_python_instance/__main__.py': 4}               │ │
│ │                 _dbs={4553778688: <SqliteDb @0x1096df670 debug=None                          │ │
│ │                 filename='/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermode… │ │
│ │                 nest=0 con=None>} _pid=2200 _lock=<unlocked _thread.lock object at           │ │
│ │                 0x1096df330> _have_used=True _has_lines=False _has_arcs=True                 │ │
│ │                 _current_context=None _current_context_id=None _query_context_ids=None>      │ │
│ │   file_mapper = <function relative_filename at 0x1099229d0>                                  │ │
│ │ file_reporter = <PythonFileReporter                                                          │ │
│ │                 '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-python… │ │
│ │     precision = 0                                                                            │ │
│ │          self = <coverage.results.Analysis object at 0x109e60940>                            │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /Users/cjolowicz/Code/github.com/nedbat/coveragepy/coverage/python.py:189 in lines               │
│                                                                                                  │
│   186 │                                                                                          │
│   187 │   def lines(self):                                                                       │
│   188 │   │   """Return the line numbers of statements in the file."""                           │
│ ❱ 189 │   │   return self.parser.statements                                                      │
│   190 │                                                                                          │
│   191 │   def excluded_lines(self):                                                              │
│   192 │   │   """Return the line numbers of statements in the file."""                           │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ self = <PythonFileReporter                                                                   │ │
│ │        '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-python-instance… │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /Users/cjolowicz/Code/github.com/nedbat/coveragepy/coverage/python.py:180 in parser              │
│                                                                                                  │
│   177 │   def parser(self):                                                                      │
│   178 │   │   """Lazily create a :class:`PythonParser`."""                                       │
│   179 │   │   if self._parser is None:                                                           │
│ ❱ 180 │   │   │   self._parser = PythonParser(                                                   │
│   181 │   │   │   │   filename=self.filename,                                                    │
│   182 │   │   │   │   exclude=self.coverage._exclude_regex('exclude'),                           │
│   183 │   │   │   )                                                                              │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ self = <PythonFileReporter                                                                   │ │
│ │        '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-python-instance… │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /Users/cjolowicz/Code/github.com/nedbat/coveragepy/coverage/parser.py:42 in __init__             │
│                                                                                                  │
│     39 │   │   if not self.text:                                                                 │
│     40 │   │   │   from coverage.python import get_python_source                                 │
│     41 │   │   │   try:                                                                          │
│ ❱   42 │   │   │   │   self.text = get_python_source(self.filename)                              │
│     43 │   │   │   except OSError as err:                                                        │
│     44 │   │   │   │   raise NoSource(f"No source for code: '{self.filename}': {err}") from err  │
│     45                                                                                           │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │           exclude = '(?:#\\s*(pragma|PRAGMA)[:\\s]?\\s*(no|NO)\\s*(cover|COVER))'            │ │
│ │          filename = '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-py… │ │
│ │ get_python_source = <function get_python_source at 0x109b19c10>                              │ │
│ │              self = <coverage.parser.PythonParser object at 0x109e60a30>                     │ │
│ │              text = None                                                                     │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /Users/cjolowicz/Code/github.com/nedbat/coveragepy/coverage/python.py:60 in get_python_source    │
│                                                                                                  │
│    57 │   │   │   break                                                                          │
│    58 │   else:                                                                                  │
│    59 │   │   # Couldn't find source.                                                            │
│ ❱  60 │   │   raise NoSource(f"No source for code: '{filename}'.")                               │
│    61 │                                                                                          │
│    62 │   # Replace \f because of http://bugs.python.org/issue19035                              │
│    63 │   source = source.replace(b'\f', b' ')                                                   │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │         base = '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-python-… │ │
│ │          ext = '.py'                                                                         │ │
│ │         exts = ['.py']                                                                       │ │
│ │     filename = '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-python-… │ │
│ │       source = None                                                                          │ │
│ │ try_filename = '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-python-… │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
NoSource: No source for code: '/Users/cjolowicz/Code/github.com/cjolowicz/cookiecutter-hypermodern-python-instance/tests\__init__.py'.

[cjolowicz]

FWIW the paths in the combined coverage look like this:

❯ sqlite3 .coverage 'select * from file'
1|tests/__init__.py
2|tests/test_main.py
3|src/cookiecutter_hypermodern_python_instance/__init__.py
4|src/cookiecutter_hypermodern_python_instance/__main__.py
5|tests\__init__.py
6|tests\test_main.py

[MarkKoz]

I'm running into a similar problem. I have coverage data generated in a Linux container, but want to generate the report on the Windows host. Since the Linux container wrote / separators, the Windows host cannot find the source files when generating the report.

[freakboy3742]

In case it helps anyone else: I've been able to work around this by re-writing the Windows coverage file prior to coverage combine. As long as you use relative_files = True, it's a 1 liner in sqlite3:

sqlite3 .coverage.* "update file set path = replace(path, '\\', '/')"

If you aren't using relative_files = True, I imagine a very similar command would work; however, you'd also need to modify your [paths] definition to account for the fact that the Windows path that is being merged will be using C:/path/to/src rather than C:\path\to\src.

[nedbat]

Thanks for bring this to back to my attention, and sorry for letting it languish.

Definitely one thing is needed to fix this: if relative_files=True, then an entry in [paths] like foo should remain foo instead of being made absolute (/home/runner/project/foo or whatever). That's this:

--- a/coverage/files.py
+++ b/coverage/files.py
@@ -363,8 +363,9 @@ def add(self, pattern, result):

         # The pattern is meant to match a filepath.  Let's make it absolute
         # unless it already is, or is meant to match any prefix.
-        if not pattern.startswith('*') and not isabs_anywhere(pattern + pattern_sep):
-            pattern = abs_file(pattern)
+        if not self.relative:
+            if not pattern.startswith('*') and not isabs_anywhere(pattern + pattern_sep):
+                pattern = abs_file(pattern)
         if not pattern.endswith(pattern_sep):
             pattern += pattern_sep

Then if you had an entry like */tests (as @cjolowicz does), you could change it to tests instead, and it would work properly. But maybe it should also be that if using relative_files, then a pattern entry like */tests should match tests/foo/bar.etc. Today it doesn't because there's no slash before tests. In fact, maybe it should always match that? Doesn't */tests mean, a "tests" directory anywhere? Opinions?