Empty HasMany relationship data[] on requests without "includes"

[UpQuark]

Description

Hi folks. I'm seeing some odd behavior setting up / fetching relationships. I have two entities, subscription and pricingTier. pricingTier hasMany subscription and subscription hasOne pricingTier, which seems straightforward enough.

When requesting /api/pricing-tier, I see:

"relationships": {
        "plan": {
            "links": {
                "self": "https://localhost:5000/api/pricing-tier/1/relationships/plan",
                "related": "https://localhost:5000/api/pricing-tier/1/plan"
            },
            "data": {
                "type": "plan",
                "id": "1"
            }
        },
        "subscription": {
            "links": {
                "self": "https://localhost:5000/api/pricing-tier/1/relationships/subscription",
                "related": "https://localhost:5000/api/pricing-tier/1/subscription"
            },
            "data": []
        }
    },

...note the empty array under data for the HasMany, but not the HasOne relationship to plan

Requesting /api/pricing-tier?inclue=subscription instead gets me:

"subscription": {
            "links": {
                "self": "https://localhost:5000/api/pricing-tier/1/relationships/subscription",
                "related": "https://localhost:5000/api/pricing-tier/1/subscription"
            },
            "data": [
                {
                    "type": "subscription",
                    "id": "1"
                }
            ]
        }

...plus the "included" collection that I'd expect.

I'm a little confused as to whether this is an error somewhere in my model setup or this is by design. I think I followed the examples reasonably closely, and to make sure it wasn't a problem with my DB-first approach, I've been using migrations on a fresh DB, but the problem has persisted.

Any help greatly appreciated! If it's not by design and it's a problem with my setup, I can provide the relevant file excerpts.

Thanks!

Environment

• JsonApiDotNetCore Version: 3.1.0
• Other Relevant Package Versions:

[maurei]

It's definitely not by design, but I'm not sure if it's caused by your setup or if its a bug. I can look into it soon, but a small repro case would be helpful!

[UpQuark]

Okay. I'll take a look and see if I can make a reproduction case for you! I wanted to make sure it wasn't by design before I went really nuts on tearing up my database configuration.

Thanks.

[UpQuark]

Hey. Below is some example JSON from an API response. I made a stripped-down version of my project to just be two models with bare bones attributes. I want to to clarify just so I'm expecting the right things:

The relationships links work fine and return data as I'd expect. I'm also used to for a HasMany relationship, seeing the data[] array as well, as I do for the HasOne relationships that are working fine.

Am I right to expect that that data[] array should return by default? In my above example it was always empty, but with a slimmer setup (shown below) it's just missing completely.

API response from the PlanController.cs

{
  "data": [
    {
      "attributes": {
        "interval": "monthly",
        "name": "Example"
      },
      "relationships": {
        "pricing-tiers": {
          "links": {
            "self": "https://localhost:5001/api/plan/1/relationships/pricing-tiers",
            "related": "https://localhost:5001/api/plan/1/pricing-tiers"
          }
        }
      },
      "type": "plan",
      "id": "1"
    },
    {
      "attributes": {
        "interval": "sadfadf",
        "name": "aqweqeqwe"
      },
      "relationships": {
        "pricing-tiers": {
          "links": {
            "self": "https://localhost:5001/api/plan/2/relationships/pricing-tiers",
            "related": "https://localhost:5001/api/plan/2/pricing-tiers"
          }
        }
      },
      "type": "plan",
      "id": "2"
    }
  ]
}

I pasted my models and DB context below, I'm happy to send you a full project if you're interested, I feel like I'm missing something small and critical but I've checked over the documentation a dozen times and I can't seem to spot what. I removed everything but these two models + context, and two simple controllers, and am using model-first scaffolding the DB instead of DB-first.

Plan.cs

namespace MyDataAccess.Entities
{
    [Table("plan")]
    public class Plan : Identifiable
    {
        [Attr("interval")]
        public string Interval { get; set; }
        
        [Attr("name")]
        public string Name { get; set; }
  
        [HasMany("pricing-tiers")]
        public virtual List<PricingTier> PricingTiers { get; set; }
    }
}

PricingTier.cs

namespace MyDataAccess.Entities
{
    [Table("pricingTier")]
    public class PricingTier : Identifiable
    {
        [Attr("description")]
        public string Description { get; set; }
  
        public int PlanId { get; set; }

        [HasOne("plan")]
        public virtual Plan Plan { get; set; }

    }
}

MyDataContext.cs

namespace MyDataAccess.Entities
{
    public partial class MyDataContext : DbContext
    {
        public virtual DbSet<Plan> Plan { get; set; }
        public virtual DbSet<PricingTier> PricingTier { get; set; }

        public MyDataContext() : base()
        {
        }
        
        public MyDataContext(DbContextOptions<MyDataContext> options) : base(options)
        {
        }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
             // connection info
        }

    }
}

Thanks a ton for your help! Tremendously appreciated.

[UpQuark]

So perusing the JSON API spec at https://jsonapi.org/format/#fetching-relationships-responses-200 says:

A GET request to a URL from a to-many relationship link could return:

Content-Type: application/vnd.api+json

{
  "links": {
    "self": "/articles/1/relationships/tags",
    "related": "/articles/1/tags"
  },
  "data": [
    { "type": "tags", "id": "2" },
    { "type": "tags", "id": "3" }
  ]
}

...which is what I'd expect from the above. So I guess I must have a misconfiguration somewhere in the above.

(also here are my startup options for the above examples)

services.AddJsonApi<MyDataContext>(
        opt =>
        {
          opt.Namespace = "api";
          opt.AllowClientGeneratedIds = true;
          opt.DefaultPageSize = 10;
          opt.IncludeTotalRecordCount = true;
        });

[maurei]

Thanks for the elaborate feedback!

So perusing the JSON API spec at https://jsonapi.org/format/#fetching-relationships-responses-200 says:

Indeed, it gets a bit confusing here. Note that in your last two post (I think you're already aware of this), two different endpoints are involved for which the semantics are slightly different:

1. The fetching resources endpoint (section in spec)
   eg. /price-tiers/1
2. The fetching relationship endpoint (section in spec)
   eg. price-tiers/1/relationships/subscriptions

Note that for the latter the specs state:

The primary data in the response document MUST match the appropriate value for resource linkage, as described above for relationship objects.

so the (relationship) data must always be populated with resource identifier objects of every subscription related to the priceTier with id 1 (which is what you're seeing in your last post). This makes sense: it would be odd to have a ... /relationships/ ... endpoint that does not actually reveal the related data

However, concerning the former (fetching resources endpoint): the specs are indifferent about having to include the relationship data. The specs only state that the data must a be valid resource object, for which the specs state that it may or may not include relationships objects.

In conclusion: the specs are indifferent about whether the has-one (or has-many) relationships should be included in the dataset by default for the /subscriptions (or /price-tiers) endpoint. It's up to the framework to choose a default behaviour. Clearly, the default behaviour as implemented by JsonApiDotNetCore is not consistent, as demonstrated by your first and second post. This inconsistency is driven by your setup (in the sense that you can steer the behaviour by playing with custom implementations, I believe), but that doesn't mean your setup is wrong: the inconsistency is undesired and can be considered a bug. This item is on the backlog.

If you have a very specific setup with a particular behaviour that you require to be different for your application to work, I can look into that specific case for you and point you towards a work around until we've fixed this inconsistency. In that case I would need a repo I can checkout to and fiddle around with.

[UpQuark]

@maurei Thanks for the fast and thorough response!

So for context, I initially came across this problem when Ember frontend hooked up to my original (un-trimmed) app seemed to be breaking on the empty data: [] that was returning for every HasMany relationship.

On reading your response, where you point out that data: [] is optional in the first place, it occurs to me that maybe the problem the Ember JsonApiAdapter was having was that the relationship data was present, but empty. So maybe (optimistically) my issue is already fixed with data: [] now appearing populated only on request (which is fine according to the spec, as you pointed out), and the Ember JsonApiAdapter will be smart enough to navigate my API as-is, and fetch relationships via the links.

I'll fiddle around with it a little bit and report back.

Just for my own curiosity, do you have a link to that issue on the backlog, and do you know of any example of a custom setup that you mention above where the inconsistency is resolved?

Thanks much! I really appreciate your help.

[maurei]

If you get an empty data: [] for the has many relationships even though there actually is related data, then this would clearly be a bug. If (optionally) we choose to include relationship data by default, it should be complete, else the field should be left out completely. I would be interested in seeing the exact setup for that if this is what you're experiencing.

Apart from this issue, we didn't have a dedicated issue for this yet. The closest we had related to this is #504. This is because I believe the inconsistent behaviour is caused by the serializer, which in turn is strongly related to the serializer being tightly coupled with the rest of the framework through JsonApiContext. I'm working on decoupling JsonApiContext right now, and as soon as I've reached the (de)serialization layer I will reevaluate this issue.

Let me know what you find by fiddling around!

[UpQuark]

Okay so I stepped through the library code a little bit, through the getAsync() method hierarchy. GetAsync in the EntityResourceService:

    public virtual async Task<TResource> GetAsync(TId id)
    {
        if (ShouldIncludeRelationships())
            return await GetWithRelationshipsAsync(id);

        TEntity entity = await _entities.GetAsync(id);

        return MapOut(entity);
    }

Using the above models (a Plan which HasMany PricingTiers) I noticed that as of executing return MapOut(entity); the entity (a Plan) has a null reference to PricingTiers. This is why I'm not seeing the relationships.pricing-tiers.data node in the JSON i pasted above. I was getting the empty data [] array originally because I had constructor boilerplate in my model from originally having scaffolded from the DB below:

public Plan()
{
    this.PricingTiers = new List<PricingTier>();
}

...which always created an empty list of PricingTiers instead of a null one, which was returned as the empty Data[] in the returned JSON.

I was able to have the HasMany relationship's data array included the way I wanted by adding the following to the constructor of my PlanController.cs:

    public PlanController(
      IJsonApiContext        jsonApiContext,
      IResourceService<Plan> resourceService,
      ILoggerFactory         loggerFactory)
      : base(jsonApiContext, resourceService, loggerFactory)
    {
      jsonApiContext.QuerySet = new QuerySet();
      jsonApiContext.QuerySet.IncludedRelationships = new List<string>();
      jsonApiContext.QuerySet.IncludedRelationships.Add("pricing-tiers");
    }

...which just adds the PricingTiers hasMany to the list of relationships to return, giving me the output:

{
  "data": {
    "attributes": {
      "interval": "month",
      "name": "Tiered"
    },
    "relationships": {
      "pricing-tiers": {
        "links": {
          "self": "https://localhost:5001/api/plan/14/relationships/pricing-tiers",
          "related": "https://localhost:5001/api/plan/14/pricing-tiers"
        },
        "data": [
          {
            "type": "pricing-tier",
            "id": "15"
          },
          {
            "type": "pricing-tier",
            "id": "16"
          },
          {
            "type": "pricing-tier",
            "id": "17"
          },
          {
            "type": "pricing-tier",
            "id": "18"
          },
          {
            "type": "pricing-tier",
            "id": "19"
          },
          {
            "type": "pricing-tier",
            "id": "20"
          }
        ]
      }
    },
    "type": "plan",
    "id": "14"
  }
}

...which is exactly what I wanted.

So I think with that small addendum, I'm basically set. It involves including a little bit of extra boilerplate code in controllers, but since the JsonApiControllers are so lean as-is, that doesn't really strike me as the end of the world.

[maurei]

Thanks for the investigations. I'm leaving this issue open as long as this behaviour has been fixed, but I'm taking that you have a viable work-around for now.

[maurei]

This is fixed in #558