Limit concurrency according to Linux container CPU quota by default

[orestisfl]

Welcome

• Yes, I've searched similar issues on GitHub and didn't find any.
• I agree to follow this project's Code of Conduct

Your feature request related to a problem? Please describe

golangci-lint supports automatically limiting the CPU quota if you use the --concurrency=0 flag / run.concurrency: 0 option.

However, the concurrency is set to runtime.NumCPU() and not 0 by default:

golangci-lint/pkg/commands/run.go

Line 597 in 225f7a0

return runtime.NumCPU()

.

As a result, golangci-lint will run by default with the number of detected logical cores instead of the limit. We observed that running golangci-lint on k8s resulted in it using 32 CPUs even though only 4 were available according to the limit set to the pod. That resulted in increased memory usage probably because the different threads were starved for CPU and were allocating memory in parallel. Eventually, the pod was OOM-killed after reaching the 8GB limit. By manually setting -j 4 or -j 0 we confirmed that the memory usage dropped by 50% and execution time was faster as well.

Describe the solution you'd like

Switch the default behavior to be equivalent to today's -j 0 behavior.

Describe alternatives you've considered

Manually setting the desired concurrency.

Additional context

No response

Supporter

• I am a sponsor through GitHub or OpenCollective

[ldez]

Hello,

for now, we don't plan to change the default as it's a breaking change.

[orestisfl]

Thanks @ldez. FMI, what's the usecase that would break? From my perspective, respecting CFS limits is a desired behavior.

[Zxilly]

A better approach is use github.com/uber-go/automaxprocs to detect the limit set by cgroup.

[ldez]

@Zxilly this is already the case, the issue is about changing the default value.

golangci-lint/pkg/commands/run.go

Lines 163 to 173 in 5d5f373

	if c.cfg.Run.Concurrency == 0 {
	backup := runtime.GOMAXPROCS(0)
	// Automatically set GOMAXPROCS to match Linux container CPU quota.
	_, err := maxprocs.Set(maxprocs.Logger(c.log.Infof))
	if err != nil {
	runtime.GOMAXPROCS(backup)
	}
	} else {
	runtime.GOMAXPROCS(c.cfg.Run.Concurrency)
	}

[ldez]

Fixed by #5470