path/filepath: no-op path.filepath.EvalSymLinks on Windows

[ericwj]

What version of Go are Windows users using (go version)?

$ go version
go version * windows/*

Does this issue reproduce with the latest release?

Yes, it is due to architectural differences between Windows and other operating systems.

What did Windows people do?

Nothing.

We include regular end users of Go programs, including system administrators and developers who use software written in Go like Docker or software which is partially written in or makes use of Go components like Visual Studio and Git.

What did you expect to see?

That software written in go continues to work after disaster recovery or changing a sub directory from being physically present to being a reparse point to a large fresh volume, without hunting down configuration issues with go software because it persisted paths after calling EvalSymLinks, assuming that the path would be more stable than the path configured explicitly by the system administrator after calling EvalSymLinks.

Why should EvalSymLinks no-op on Windows?

On Windows, the path that EvalSymLinks returns is not more stable, rather it is in very common situations random due to the GPT volume identifier being random and is hence the path returned by this API is also not under control of the administrator. The administrator might be forced to change reparse points or hard links due to disk failure, disaster recovery, resolving disk full problems, and similar. Hence, the use of EvalSymLinks for the purposes it is designed for is wrong on Windows, even if its use is justified in the same software on other operating systems.

What did you see instead?

Docker services stop being able to start, even though they should have run and should continue to run from C:\Docker, regardless of the fact that this directory resolves now to a Windows path on a different volume with a different GPT unique identifer and hence a different volume path in the form \\?\Volume{guid}.

[networkimprov]

Thanks for raising this. It's a proposal (I think) so doesn't have to follow the standard issue template. It's also rather long and a bit grumpy.

Could you thin it down to a list of specific problems (with examples) in the current API, and possible solutions?

Note that the Windows port has long depended on contributions from outside the team at Google.

[ericwj]

Well sorry for being grumpy, but leaving Windows end users like me with a myriad of problems that they should wholeheartedly own as stewards of go is ... more grumpiness. Notice that I was explicitly mentioning Google and not the well-meaning community, but they are limited to playing whack-a-mole and naturally everybody cares about just their own little issues.

As I said, I have no experience with Go. I just counted them and I only ever wrote 54 lines in 2 files and just 20 lines that actually do something.

So, although I might shorten this post and make it less grumpy, making this specific I feel first of all doesn't truly reflect the fact I find issues in about every function in path_windows.go and file_windows.go and probably other *_windows.go files. I mean from palmface issues like the VolumeName API not returning or intending even to return any kind of volume name but a root path to hacking strings and finding ':' where you might also in fact be looking for '\\?\Volume{GUID}' to stripping '\\?\' from a path after 'GetFinalPathNameByHandle' returns that, breaking long paths and special paths in the process, to suspecting that happens in perhaps every file API in go by looking at the issues in this repository, on Stack Overflow and in the docker repository, etc.

I mean how did they get that name so wrong? Was that API proposal not reviewed or did they like the confusion that would result but which they could hope would go unnoticed? Is go a joke?

[networkimprov]

What Go app has been giving you trouble, Docker? Have you filed an issue there? If so, can you point it out?

It's the app's responsibility to work correctly on platforms it supports. I patch the Go stdlib on Windows where needed.

[ianlancetaylor]

@ericwj I'm sorry that you are frustrated but this issue doesn't seem to give us a useful way forward.

I doubt this will make you feel any better but pretty much all the Windows path support was written and contributed by Windows experts who do not work at Google.

It seems to me that filepath.EvalSymlinks is well defined, and it is clearly useful on Unix systems. Perhaps programs that run on Windows shouldn't be calling it, but that doesn't mean that we should remove it.

CC @alexbrainman @mattn

[ericwj]

Are you saying Google isn't involved? They are at least reviewing every single change - or they appear to be.

It's the app's responsibility to work correctly on platforms it supports.

If you personally wrote a new language, I would agree. Imho Google has a larger responsibility. But hey, the fact the first thing Microsoft does contributing to chromium is to add features for people with disabilities speaks to the company culture at Google refusing to do anything of the sort for 12 whole years.

@ianlancetaylor I am not asking for the API to be removed if it is 'normal' to use that on *nix, although I think overruling an administrator is about always wrong by principle regardless of the operating system. But go knows whether you are on Windows or cross building for it and if it has infrastructure to give warnings or suggest fixes for this API on Windows then that would be step 1.

I didn't personally file issues with Docker, but the are legion links to be found in their issues and on Stack Overflow with people stumbling on error messages like unable to prepare context: unable to evaluate symlinks in context path and similar.

I did file issues with Microsoft about this Visual Studio behavior, but since these errors do not reside in actual Microsoft code they de-prioritized.

About Git I am honestly just unaware of the components that make up the system. Visual Studio Code works quite allright, but Visual Studio has some go somewhere in its Git extension or I don't know where such that these problems magically appear in recent versions of Visual Studio.

[ianlancetaylor]

Are you saying Google isn't involved? They are at least reviewing every single change - or they appear to be.

Google is a huge company. I'm not sure what it means to say that Google is involved. Certainly Google employees are heavily involved with the development of Go. But it's not accurate to say that Google employees are reviewing every single change. Go is an open source project. Many changes to Go are written, reviewed, and submitted by people who are not Google employees, without anybody employed by Google being involved at all.

[networkimprov]

Well, Eric, since you learned Go, maybe you could fix some of those Docker issues (perhaps by replacing EvalSymlinks in their codebase), and then file an issue or two here about how Go could support them better :-)

But we need specific bug reports to make progress.

[ericwj]

Go is an open source project.

So is .NET. And .NET actually works even with Windows-isms on a long list of *nix variants, because Microsoft as primary steward of the ecosystem is taking its proper responsibility. The downside is they have so much tests that it is hard to find any actual code if you look for a definition of something in /dotnet/runtime say.

Look, let's talk solutions. I am fed up hitting brick walls every time someone chooses go over another language and then shoves it onto my system as part of some software that I use - that's why I'm here. I identified very concrete issues and I can probably write code from plain primitives that uses the Windows API properly to fill in go API surface -- I just can't realistically write it in go. Would I help writing plain C# implementing some of the broken API's in go?

I could perhaps even write tests that will show how to break go, but I'd have to do that mostly in PowerShell and C#.

[ericwj]

PowerShell to create .vhdx files and mount/unmount them - system administration stuff, which I don't think go can do today at all.