Improve crates endpoint performance

[eth3lbert]

This pr improve the performance by utilizing a count subquery for total retrieval.

P.S. make a separate query to fetch total should also outperforms window functions in this context.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (main@c6c0a16). Click here to learn what that means.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #7941   +/-   ##
=======================================
  Coverage        ?   87.57%           
=======================================
  Files           ?      257           
  Lines           ?    25101           
  Branches        ?        0           
=======================================
  Hits            ?    21982           
  Misses          ?     3119           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[LawnGnome]

I like this! I particularly like the new test assertions.

I experimented with a similar change last year, but trying to fully automatically build the count query instead of having it added separately. Turned out to be really hard (maybe actually impossible, right now?) because of what Diesel does and doesn't expose from its innards. So I like this as a pragmatic middle ground, and I like that the changes to the controller weren't as bad as I thought they'd be last year.

@Turbo87 Let's give this a quick once over on staging before we deploy, but I think this is good.

[LawnGnome]

I'm correct in thinking this isn't a regression, right, because we don't support this today?

[eth3lbert]

Yes, and I also like to make a follow up PR which expands the scope of the seek-based pagination for crates endpoint.

[eth3lbert]

For offset-based pagination, the response will remain unchanged. The only modification pertains to seek-based pagination. Previously, we provided an approximate total count; however, with this PR, the total count will align precisely with the offset-based count.

[Turbo87]

Let's give this a quick once over on staging before we deploy, but I think this is good.

I've just deployed main to production, so we can now merge this and test it out on staging.

thanks a lot for the work, @eth3lbert! :)

[carols10cents]

Cross-referencing here: we had to revert this PR because it broke search queries with spaces in them. New issue for re-landing: #8052