unparam does not skip autogenerated files

[tamalsaha]

Thank you for creating the issue!

Please include the following information:

1. Version of golangci-lint: golangci-lint --version (or git commit if you don't use binary distribution)
   github.com/golangci/golangci-lint/cmd/golangci-lint@13e3a1a37556421d5dbad97bba07d7ff9a218931

2. Config file: cat .golangci.yml
   There is no such file

3. Go environment: go version && go env
   Go 1.12

4. Verbose output of running: golangci-lint run -v
   https://github.com/kubedb/apimachinery/pull/453/checks#step:3:50

[tamalsaha]

List of autogenerated files that are not ignored:

https://github.com/kubedb/apimachinery/blob/fc97357bc3ec26247d8ca41956b77aa606f317b6/client/clientset/versioned/fake/register.go#L34

https://github.com/kubedb/apimachinery/blob/master/apis/config/v1alpha1/openapi_generated.go

[tpounds]

@tamalsaha Thanks for the report! I don't think there is an easy way to have golangci-lint infer which files were generated versus regular source when it has a standard .go extension and only indicates it was generated via comments. Is there any reason you can't add the generated files to a list of files to skip?

[tamalsaha]

I could add these generated files to a --skip-files. Could golanglint add an option to ignore build tags? These files are generated by tools in the Kubernetes project. They add a tag like below:

// +build !ignore_autogenerated

If I could pass a flag like --skip-tags=ignore_autogenerated that would be easier.

[jirfag]

@tpounds see the autogenerated filter
golangci-lint already parses comments and detects auto-generated files by them.

There are such comments:

// Code generated by openapi-gen. DO NOT EDIT.

// This file was autogenerated by openapi-gen. Do not edit it manually!

[jirfag]

@tamalsaha

1. I strongly recommend using binary releases in CI
2. I have no access to build log, please, attach it here

[tpounds]

@jirfag Thanks for the info! If that's the case then these files should be ignored right?

[jirfag]

@tpounds yep, they should be ignored

[tamalsaha]

Please see the logs.

GitHub repo: https://github.com/kubedb/apimachinery

$ pwd
/home/tamal/go/src/kubedb.dev/apimachinery

$ golangci-lint version
golangci-lint has version v1.19.2-0.20191002180103-13e3a1a37556 built from (unknown, mod sum: "h1:NJKwD+7k5ESPuXtImgNWcFeYwTUAJPijPv0FxHpFsEs=") on (unknown)


$ golangci-lint run -v --enable goconst,gofmt,goimports,unparam --skip-dirs-use-default --deadline=10m
INFO [config_reader] Config search paths: [./ /home/tamal/go/src/kubedb.dev/apimachinery /home/tamal/go/src/kubedb.dev /home/tamal/go/src /home/tamal/go /home/tamal /home /] 
INFO [lintersdb] Active 14 linters: [deadcode errcheck goconst gofmt goimports gosimple govet ineffassign staticcheck structcheck typecheck unparam unused varcheck] 
INFO [loader] Go packages loading at mode 575 (files|imports|types_sizes|compiled_files|deps|exports_file|name) took 1.289502105s 
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 44.695434ms 
INFO [runner/max_from_linter] 363/413 issues from linter unparam were hidden, use --max-issues-per-linter 
INFO [runner] Issues before processing: 438, after processing: 51 
INFO [runner] Processors filtering stat (out/in): diff: 414/414, max_from_linter: 51/414, source_code: 51/51, path_shortener: 51/51, cgo: 438/438, skip_files: 438/438, skip_dirs: 438/438, autogenerated_exclude: 438/438, exclude: 438/438, exclude-rules: 438/438, nolint: 416/438, max_per_file_from_linter: 414/414, path_prettifier: 438/438, max_same_issues: 414/414, identifier_marker: 438/438, uniq_by_line: 414/416, filename_unadjuster: 438/438 
INFO [runner] processing took 102.584198ms with stages: nolint: 76.422147ms, exclude: 21.469035ms, identifier_marker: 1.852402ms, source_code: 895.629µs, path_prettifier: 714.483µs, skip_dirs: 307.494µs, autogenerated_exclude: 211.909µs, max_same_issues: 178.878µs, uniq_by_line: 154.801µs, max_from_linter: 120.707µs, max_per_file_from_linter: 86.052µs, cgo: 81.824µs, filename_unadjuster: 69.15µs, path_shortener: 18.062µs, diff: 704ns, exclude-rules: 487ns, skip_files: 434ns 
INFO [runner] linters took 24.710922647s with stages: goanalysis_metalinter: 21.006303139s, unused: 3.60192343s 
client/clientset/versioned/fake/register.go:34:5: var `parameterCodec` is unused (unused)
var parameterCodec = runtime.NewParameterCodec(scheme)
    ^
apis/config/v1alpha1/openapi_generated.go:361:44: `schema_go_encoding_json_types_IntHash` - `ref` is unused (unparam)
func schema_go_encoding_json_types_IntHash(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                           ^
apis/config/v1alpha1/openapi_generated.go:1434:64: `schema_k8sio_api_apps_v1_RollingUpdateStatefulSetStrategy` - `ref` is unused (unparam)
func schema_k8sio_api_apps_v1_RollingUpdateStatefulSetStrategy(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                               ^
apis/config/v1alpha1/openapi_generated.go:1791:64: `schema_k8sio_api_core_v1_AWSElasticBlockStoreVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_AWSElasticBlockStoreVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                               ^
apis/config/v1alpha1/openapi_generated.go:1866:46: `schema_k8sio_api_core_v1_AttachedVolume` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_AttachedVolume(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                             ^
apis/config/v1alpha1/openapi_generated.go:1922:53: `schema_k8sio_api_core_v1_AzureDiskVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_AzureDiskVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                    ^
apis/config/v1alpha1/openapi_generated.go:1978:63: `schema_k8sio_api_core_v1_AzureFilePersistentVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_AzureFilePersistentVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                              ^
apis/config/v1alpha1/openapi_generated.go:2020:53: `schema_k8sio_api_core_v1_AzureFileVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_AzureFileVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                    ^
apis/config/v1alpha1/openapi_generated.go:2232:44: `schema_k8sio_api_core_v1_Capabilities` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_Capabilities(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                           ^
apis/config/v1alpha1/openapi_generated.go:2487:46: `schema_k8sio_api_core_v1_ClientIPConfig` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_ClientIPConfig(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                             ^
apis/config/v1alpha1/openapi_generated.go:2507:50: `schema_k8sio_api_core_v1_ComponentCondition` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_ComponentCondition(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                 ^
apis/config/v1alpha1/openapi_generated.go:2717:50: `schema_k8sio_api_core_v1_ConfigMapEnvSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_ConfigMapEnvSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                 ^
apis/config/v1alpha1/openapi_generated.go:2744:52: `schema_k8sio_api_core_v1_ConfigMapKeySelector` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_ConfigMapKeySelector(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                   ^
apis/config/v1alpha1/openapi_generated.go:2828:57: `schema_k8sio_api_core_v1_ConfigMapNodeConfigSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_ConfigMapNodeConfigSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                        ^
apis/config/v1alpha1/openapi_generated.go:3199:46: `schema_k8sio_api_core_v1_ContainerImage` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_ContainerImage(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                             ^
apis/config/v1alpha1/openapi_generated.go:3234:45: `schema_k8sio_api_core_v1_ContainerPort` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_ContainerPort(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                            ^
apis/config/v1alpha1/openapi_generated.go:3400:53: `schema_k8sio_api_core_v1_ContainerStateWaiting` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_ContainerStateWaiting(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                    ^
apis/config/v1alpha1/openapi_generated.go:3497:46: `schema_k8sio_api_core_v1_DaemonEndpoint` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_DaemonEndpoint(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                             ^
apis/config/v1alpha1/openapi_generated.go:3694:44: `schema_k8sio_api_core_v1_EndpointPort` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_EndpointPort(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                           ^
apis/config/v1alpha1/openapi_generated.go:4200:43: `schema_k8sio_api_core_v1_EventSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_EventSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                          ^
apis/config/v1alpha1/openapi_generated.go:4227:42: `schema_k8sio_api_core_v1_ExecAction` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_ExecAction(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                         ^
apis/config/v1alpha1/openapi_generated.go:4254:46: `schema_k8sio_api_core_v1_FCVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_FCVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                             ^
apis/config/v1alpha1/openapi_generated.go:4432:51: `schema_k8sio_api_core_v1_FlockerVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_FlockerVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                  ^
apis/config/v1alpha1/openapi_generated.go:4459:61: `schema_k8sio_api_core_v1_GCEPersistentDiskVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_GCEPersistentDiskVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                            ^
apis/config/v1alpha1/openapi_generated.go:4501:51: `schema_k8sio_api_core_v1_GitRepoVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_GitRepoVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                  ^
apis/config/v1alpha1/openapi_generated.go:4536:63: `schema_k8sio_api_core_v1_GlusterfsPersistentVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_GlusterfsPersistentVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                              ^
apis/config/v1alpha1/openapi_generated.go:4578:53: `schema_k8sio_api_core_v1_GlusterfsVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_GlusterfsVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                    ^
apis/config/v1alpha1/openapi_generated.go:4669:42: `schema_k8sio_api_core_v1_HTTPHeader` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_HTTPHeader(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                         ^
apis/config/v1alpha1/openapi_generated.go:4730:41: `schema_k8sio_api_core_v1_HostAlias` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_HostAlias(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                        ^
apis/config/v1alpha1/openapi_generated.go:4764:52: `schema_k8sio_api_core_v1_HostPathVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_HostPathVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                   ^
apis/config/v1alpha1/openapi_generated.go:4990:41: `schema_k8sio_api_core_v1_KeyToPath` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_KeyToPath(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                        ^
apis/config/v1alpha1/openapi_generated.go:5312:51: `schema_k8sio_api_core_v1_LoadBalancerIngress` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_LoadBalancerIngress(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                  ^
apis/config/v1alpha1/openapi_generated.go:5367:52: `schema_k8sio_api_core_v1_LocalObjectReference` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_LocalObjectReference(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                   ^
apis/config/v1alpha1/openapi_generated.go:5387:49: `schema_k8sio_api_core_v1_LocalVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_LocalVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                ^
apis/config/v1alpha1/openapi_generated.go:5415:47: `schema_k8sio_api_core_v1_NFSVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_NFSVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                              ^
apis/config/v1alpha1/openapi_generated.go:5546:45: `schema_k8sio_api_core_v1_NamespaceSpec` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_NamespaceSpec(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                            ^
apis/config/v1alpha1/openapi_generated.go:5573:47: `schema_k8sio_api_core_v1_NamespaceStatus` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_NamespaceStatus(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                              ^
apis/config/v1alpha1/openapi_generated.go:5640:43: `schema_k8sio_api_core_v1_NodeAddress` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_NodeAddress(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                          ^
apis/config/v1alpha1/openapi_generated.go:5889:48: `schema_k8sio_api_core_v1_NodeProxyOptions` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_NodeProxyOptions(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                               ^
apis/config/v1alpha1/openapi_generated.go:5982:55: `schema_k8sio_api_core_v1_NodeSelectorRequirement` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_NodeSelectorRequirement(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                      ^
apis/config/v1alpha1/openapi_generated.go:6273:46: `schema_k8sio_api_core_v1_NodeSystemInfo` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_NodeSystemInfo(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                             ^
apis/config/v1alpha1/openapi_generated.go:6357:51: `schema_k8sio_api_core_v1_ObjectFieldSelector` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_ObjectFieldSelector(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                  ^
apis/config/v1alpha1/openapi_generated.go:6385:47: `schema_k8sio_api_core_v1_ObjectReference` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_ObjectReference(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                              ^
apis/config/v1alpha1/openapi_generated.go:6781:65: `schema_k8sio_api_core_v1_PersistentVolumeClaimVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_PersistentVolumeClaimVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                                ^
apis/config/v1alpha1/openapi_generated.go:7227:54: `schema_k8sio_api_core_v1_PersistentVolumeStatus` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_PersistentVolumeStatus(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                     ^
apis/config/v1alpha1/openapi_generated.go:7261:64: `schema_k8sio_api_core_v1_PhotonPersistentDiskVolumeSource` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_PhotonPersistentDiskVolumeSource(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                               ^
apis/config/v1alpha1/openapi_generated.go:7461:48: `schema_k8sio_api_core_v1_PodAttachOptions` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_PodAttachOptions(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                               ^
apis/config/v1alpha1/openapi_generated.go:7635:50: `schema_k8sio_api_core_v1_PodDNSConfigOption` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_PodDNSConfigOption(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                 ^
apis/config/v1alpha1/openapi_generated.go:7661:46: `schema_k8sio_api_core_v1_PodExecOptions` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_PodExecOptions(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                             ^
apis/config/v1alpha1/openapi_generated.go:7871:53: `schema_k8sio_api_core_v1_PodPortForwardOptions` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_PodPortForwardOptions(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                                    ^
apis/config/v1alpha1/openapi_generated.go:7912:47: `schema_k8sio_api_core_v1_PodProxyOptions` - `ref` is unused (unparam)
func schema_k8sio_api_core_v1_PodProxyOptions(ref common.ReferenceCallback) common.OpenAPIDefinition {
                                              ^
INFO File cache stats: 2 entries of total size 712.8KiB 
INFO Memory: 187 samples, avg is 952.8MB, max is 1155.9MB 
INFO Execution took 26.05545856s     

[jirfag]

thank you, we need to debug it

[Lacklub]

I'm having the same issue. It looks like the bug is in https://github.com/golangci/golangci-lint/blob/master/pkg/result/processors/autogenerated_exclude.go

Func getDoc, around line 133, the scanner.scan for loop exits when it encounters a line that isn't a 1) "//" style comment, 2) an empty line, or 3) a package declaration. This means that in my case and in @tamalsaha's case, the block comment at the beginning of the file causes it to stop scanning, so it never reaches the DO NOT EDIT comment.