Static Assets -- what's happening?

[dazinator]

I am really just looking for information around changes to static web assets in .net 6.

In .net 5, there was a relatively simple IFileProvider implementation for static assets. There was basically a manifest file with some root directories in, and when your application ran, the manifest was discovered, and file providers were set up to serve content from these directories. These file providers passed through much logic to PhysicalFileProvider which was quite nice because:

• It was already a proven implementation of file provider
• It has support for Watch and change notifications - which is useful when serving from a local directory for example.

I have recently upgraded to .NET 6 and I have discovered that the old manifest file format has now been replaced by a newer format, and a newer ManifestStaticWebAssetFileProvider.

• This new format of manifest file looks to expanded a great deal in its complexity and size. (I am seeing mine used to be <1kb, in the new model it's 284kb).
• The new file provider seemingly looks a bit more complex and puts more logic in front of the underlying PhysicalFileProvider (which was a proven implementation, so i.e risk of new bugs therefore increased with .net 6 in this area)
• The new file provider doesn't seem to support Watch or change tokens anymore (even though the underlying physical file providers are capable of this) - which seems a degradation in functionality from before.. e.g it does this:
  public IChangeToken Watch(string filter) => NullChangeToken.Singleton;

Therefore given these perceived negatives, may I ask what the reason for the new provider and new manifest file is - i.e there must be reasons why it was necessary to make these changes, it would be very helpful to understand what is accomplished in this new model of dealing with static assets. Also I would be interested to know why Watch change tokens are deprecated (thinking of the scenario in local development where you edit a content file in a locally referenced project I would have though Watch would have been useful for enabling the app to respond - or is this not really the case?

Many Thanks

[dazinator]

Is the goal of the new manifest file provider, to be able to constrain the files that will be visible from the physical directory, to the files either explicitly mentioned in the manifest, or ones that match a pattern in the manifest? i.e if the file on disk isn't explicitly in the manifest, or doesn't match a pattern explicitly in the manifest, then it shouldn't be resolvable? I assume that goes hand in hand with project authors being able to prevent specific files from being included in this manifest, or being able to include patterns in this manifest?

[javiercn]

@dazinator thanks for contacting us.

There are a few changes we did in 6.0 that required us to change the manifest. In broad terms:

• We wanted to support mapping files to a given path independent of their physical location on disk.
• We wanted to limit the set of files that were available during development to avoid accidentally mapping additional files that weren't explicitly mapped and which would not be available in the published app.

In essence, the new manifest format works more like the EmbeddedManifestFileProvider where it creates a virtual file system representation of the static web assets in the project.

[javiercn]

Also I would be interested to know why Watch change tokens are deprecated (thinking of the scenario in local development where you edit a content file in a locally referenced project I would have though Watch would have been useful for enabling the app to respond - or is this not really the case?

This might have been an oversight in our implementation.

[javiercn]

Let me further address some of your points:

I have recently upgraded to .NET 6 and I have discovered that the old manifest file format has now been replaced by a newer format, and a newer ManifestStaticWebAssetFileProvider.

• This new format of manifest file looks to expanded a great deal in its complexity and size. (I am seeing mine used to be <1kb, in the new model it's 284kb).
• The new file provider seemingly looks a bit more complex and puts more logic in front of the underlying PhysicalFileProvider (which was a proven implementation, so i.e risk of new bugs therefore increased with .net 6 in this area)
• The new file provider doesn't seem to support Watch or change tokens anymore (even though the underlying physical file providers are capable of this) - which seems a degradation in functionality from before.. e.g it does this:
  public IChangeToken Watch(string filter) => NullChangeToken.Singleton;

File size is not a concern for us here since this is a development feature.

The provider is more complex because the old provider + manifest didn't allow room to support some of the features that we built in 6.0 and didn't allow us to later things like Blazor on top of it without having to do unnatural things on MSBuild for the pieces to fit.

As to for the risk of bugs, there are always risks when changing software, I don't particularly think this is a risky area, since it's a well understood problem and we have plenty of tests for the provider, the manifest and everything in between. Everything that was tested on the old provider still works in the new provider and we added a bunch of additional tests.

Watch as I mentioned, its something that we overlooked in the new implementation. I've filed a separate issue to track that here

I hope this helps shed some light into the reasons for the changes.

[dazinator]

Thanks @javiercn

We wanted to support mapping files to a given path independent of their physical location on disk.
We wanted to limit the set of files that were available during development to avoid accidentally mapping additional files that weren't explicitly mapped and which would not be available in the published app.

Thanks.
The explicit mappings of files is easy to interpret, but I'm having a hard time understanding how to interpret the new patterns sections.. I wondered if you would be able to shed any light on those?

"Patterns": [
                            {
                                "ContentRootIndex": 0,
                                "Pattern": "**",
                                "Depth": 0
                            }
                        ]

I can see that the server's wwwroot folder for example, is mapped like this on root (i.e /).
However each file in it's wwwroot also has an explicit mapping under root:

"Root": {
        "Children": {
            "foo.txt": {
                "Children": null,
                "Asset": {
                    "ContentRootIndex": 0,
                    "SubPath": "foo.txt"
                },
                "Patterns": null
            },

So if one was to just interpret the explicit file mappings and ignore the patterns completely, one could already set things up to serve /foo.txt in the example above, from [content root]/foo.txt on disk.

Let's assume that there is no more hierarchy in this file, just this single root node with one child for foo.txt. Is this pattern basically saying:

• Any request path that matches ** (under the root path /) - if it does not match an explicit file mapping already, will be resolved via the ContentRoot file provider - in other words if there is no explicit file mapping for "/bar.txt" and I requested "/bar.txt" the pattern would come into effect and try to resolve "bar.txt" from the Content root file provider pointing at the servers wwwroot directory in this case?

If I am interpreting that correctly, then:

We wanted to limit the set of files that were available during development to avoid accidentally mapping additional files that weren't explicitly mapped and which would not be available in the published app.

How is this accomplished given the wildcard patterns added by default to all projects wwwroot folders in a solution?

[javiercn]

The explicit mappings of files is easy to interpret, but I'm having a hard time understanding how to interpret the new patterns sections.. I wondered if you would be able to shed any light on those?

This was going to be part of supporting .razor.js files which are mapped outside the wwwroot folder (For example, /Pages/Index.razor.js). We didn't want that mapping "/Pages" would cause "/Pages/Index.razor" to be mapped as a static asset. The idea was that things like JS modules was going to be just a convention that got mapped, but I don't remember exactly why, we decided at the end switch to an approach similar to the one we use for scoped CSS. What this meant is that even if you add a razor.js file while the app was running, you needed to rebuild the app to be picked up.

The patterns are used to allow discovery of the files while the app is running when there isn't an explicit mapping. For example, if you add a file while the app is running. You see ** for all the wwwroot folders because that's essentially what all the previous static web assets version mapped.

[dazinator]

@javiercn ok thank you!

I quite like the idea of having a IFileProvider that can map files from other providers based on explicit path mappings, or pattern matching. It potentially lets the user take complete control of how files from various sources present to their applications, or the browser etc - i.e taking more granular control over the shape of the virtual directory.

The provider you've created is currently internal, along with the manifest, so if it's ok with you, I am going to create my own take on this provider over here: https://github.com/dazinator/Dazinator.Extensions.FileProviders that others could use.

It will use its own representation of the mappings, and DotNet.Glob for the pattern matching aspect, but it will be similar in nature in terms of the hierarchical way it evaluates paths, with explicit file mappings taking precedence over patterns etc. To make this potentially more broadly useful, I'll add an extension method so that this provider can be loaded from this new static assets manifest file format in .net 6. I will most likely copy the internal manifest model class that you deserialize the json file into, to achieve this extension method the short term..

Thank you for the discussion and for answering the questions I had around this.

[javiercn]

The provider you've created is currently internal, along with the manifest, so if it's ok with you, I am going to create my own take on this provider over here: https://github.com/dazinator/Dazinator.Extensions.FileProviders that others could use.

I'll add an extension method so that this provider can be loaded from this new static assets manifest file format in .net 6.

I want to point out that the provider as well as the format are an implementation detail and that its subject to change at any point in a future release if we need to do so to accommodate new features.

You might be better off creating your own separate manifest and your own separate implementation as ours is very tailored to supporting the scenarios we care about.

[dazinator]

Yep understood, I'm creating my own mapping provider that let's you create / add the mappings via its own api, and has no native manifest file format so its completely independent from static assets. From there, I'll add an extension method that will read your manifest to configure these mappings. This will result in a mapping provider that yield very similar behaviour but is not tied to static assets. I understand if you change the file format I'll have to adjust this extension method accordingly.