golangci-lint fails if it can't read the contents of a directory, even if it's in --skip-dirs

[zealws]

• Yes, I'm using a binary release within 2 latest major releases. Only such installations are supported.
• Yes, I've searched similar issues on GitHub and didn't find any.
• Yes, I've included all information below (version, config, etc).
• Yes, I've tried with the standalone linter if available. (https://golangci-lint.run/usage/linters/)

Description of the problem

If there is a directory that is not readable in the current project, golangci-lint fails.

This is expected, as it can't lint potential files inside a directory that is not readable.

However, adding the directory to skip-dirs either on the command line, or via the yaml config does not allow golangci-lint to function against the other directories which it can read from.

Version of golangci-lint

# golangci-lint --version
golangci-lint has version 1.38.0 built from 507703b on 2021-03-10T00:37:58Z

Config file

# cat .golangci.yml
--- 
run:
  skip-dirs:
    - solrdata

Go environment

# go version && go env
go version go1.16 linux/amd64
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/zeal/.cache/go-build"
GOENV="/home/zeal/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/zeal/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/zeal/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/zeal/pantheon/search-gateway/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1751798816=/tmp/go-build -gno-record-gcc-switches"

Verbose output of running

zeal@theos ~/pantheon/search-gateway
# golangci-lint cache clean
zeal@theos ~/pantheon/search-gateway
# golangci-lint run -E goimports -v --skip-dirs 'solrdata'
INFO [config_reader] Config search paths: [./ /home/zeal/pantheon/search-gateway /home/zeal/pantheon /home/zeal /home /] 
INFO [config_reader] Used config file .golangci.yml 
INFO [lintersdb] Active 11 linters: [deadcode errcheck goimports gosimple govet ineffassign staticcheck structcheck typecheck unused varcheck] 
INFO [loader] Go packages loading at mode 575 (types_sizes|compiled_files|exports_file|imports|deps|files|name) took 135.890614ms 
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 2.22762ms 
ERRO [linters context] typechecking error: pattern ./...: open /home/zeal/pantheon/search-gateway/solrdata/data: permission denied 
INFO [linters context/goanalysis] analyzers took 7.887110931s with top 10 stages: buildir: 5.74124395s, inspect: 442.224728ms, fact_deprecated: 301.768098ms, ctrlflow: 244.091513ms, nilness: 203.095967ms, fact_purity: 188.176352ms, printf: 182.168859ms, SA5012: 119.320248ms, typedness: 90.93006ms, goimports: 16.6311ms 
WARN [runner] Can't run linter goanalysis_metalinter: S1036: failed prerequisites: [inspect@./...: analysis skipped: errors in package: [-: pattern ./...: open /home/zeal/pantheon/search-gateway/solrdata/data: permission denied]] 
INFO [linters context/goanalysis] analyzers took 180.930901ms with top 10 stages: buildir: 166.586967ms, unused: 7.355048ms, directives: 5.455527ms, isgenerated: 1.533359ms 
WARN [runner] Can't run linter unused: buildir: analysis skipped: errors in package: [-: pattern ./...: open /home/zeal/pantheon/search-gateway/solrdata/data: permission denied] 
INFO [runner] processing took 3.966µs with stages: max_same_issues: 505ns, nolint: 470ns, max_from_linter: 375ns, skip_dirs: 337ns, source_code: 331ns, autogenerated_exclude: 291ns, cgo: 189ns, sort_results: 157ns, filename_unadjuster: 152ns, path_prettifier: 151ns, diff: 150ns, path_shortener: 149ns, uniq_by_line: 145ns, identifier_marker: 135ns, skip_files: 127ns, severity-rules: 61ns, path_prefixer: 61ns, max_per_file_from_linter: 60ns, exclude: 60ns, exclude-rules: 60ns 
INFO [runner] linters took 2.934005368s with stages: goanalysis_metalinter: 2.742242962s, unused: 191.715397ms 
ERRO Running error: buildir: analysis skipped: errors in package: [-: pattern ./...: open /home/zeal/pantheon/search-gateway/solrdata/data: permission denied] 
INFO Memory: 32 samples, avg is 257.5MB, max is 409.2MB 
INFO Execution took 3.07575772s                   

In that error solrdata/data is owned by root and not readable as the current user.

Code example or link to a public repository

I was not able to come up with a simple test case of this because of a different golangci-lint issue, but you should be able to reproduce this in the following way with any repository:

• Create a directory owned by root.
• Chmod to remove read/execute access from the current user.
• Run golangci-lint run ... with --skip-dirs

[boring-cyborg]

Hey, thank you for opening your first Issue ! 🙂 If you would like to contribute we have a guide for contributors.

[shmsr]

@zealws As far as I know:

Skip Dirs is a post-processor i.e., golangci-lint being a meta linter cannot control every aspect of the linter that it is running. The loader that it uses is also not aware of skip dirs, files, etc. and it shouldn't be either because it makes it super complex then.

So, the loader (for AST parsing, dependency resolution, etc.) is working just fine and the output that you are seeing is from there only, I guess.

Also, as Skip Dirs is a post-processor, golangci-lint will collect issues raised by every linter that it is running and then runs a set of post-processor(s) — like SkipDirs which then excludes the issues for files that are present in those directories.

So, I do not think it's a bug at all. If you see an issue raised by golangci-lint for files present inside inside solrdata then it's a bug; otherwise, it is not.

[roonyh]

@shmsr So how to make it not look in a directory? Currently if we run it on the root of our project it fails because of some issue inside a go file in node_modules folder.

We don't want to lint that file. How can we tell it to not look inside node_modules?

[silverwind]

I also observe golangci-lint run venturing into node_modules with no way documented to stop it from doing that (#301 (comment) does not help at all).

I can only imagine how many CPU cycles are being wasted by traversing node_modules which in some cases can contain millions of files.

Edit: golang/go#42965 would probably one way to solve it.

[azeez010]

of

@zealws As far as I know:

Skip Dirs is a post-processor i.e., golangci-lint being a meta linter cannot control every aspect of the linter that it is running. The loader that it uses is also not aware of skip dirs, files, etc. and it shouldn't be either because it makes it super complex then.

So, the loader (for AST parsing, dependency resolution, etc.) is working just fine and the output that you are seeing is from there only, I guess.

Also, as Skip Dirs is a post-processor, golangci-lint will collect issues raised by every linter that it is running and then runs a set of post-processor(s) — like SkipDirs which then excludes the issues for files that are present in those directories.

So, I do not think it's a bug at all. If you see an issue raised by golangci-lint for files present inside inside solrdata then it's a bug; otherwise, it is not.

of what purpose is of making it post. waste of computing power

[ldez]

Close in favor of #4360 - Feel free to subscribe there for updates.