Idea: "Recently 1.0 crates" list on the home page

[carols10cents]

I've been thinking about the purpose of the lists on the homepage and whether the current lists are fulfilling that or any purpose, and I had an idea I wanted to get feedback on.

What if we list the 10 crates who recently released their first >= 1.0 version (either started with 1.0 or had versions < 1.0 and just posted the first >= 1.0 version)?

The goal would be to encourage crates to become mature and supported enough for production use, which I hope would be achieved by showcasing those crates. This would also give production users confidence that the production-ready ecosystem is growing, and help people become aware of stable crates.

The downsides I could see would be:

• Crates start with 1.0 or mark themselves 1.0 without the intention of being stable or being supported, just to get on the list
• Crates that use different version schemes than the usual semver conventions might never show up on this list

What other advantages and disadvantages am I missing?

I haven't thought at all about what the query for this would look like yet (😱) so this conversation might be moot if this is expensive to implement. I'm looking to see if this is an idea worth investigating right now.

[sgrif]

I like this idea. I don't think the query to implement it would be prohibitively expensive, but I'll investigate further

[smarnach]

This sounds like something that could be useful. Let's look at the list a few times manually before implementing this to get a feeling of what the results generally look like. I suggest replacing the "New crates" section with that, since most of the crates published for the first time aren't particularly useful to the general public. In the early days of crates.io, it may still have been possible to follow what's happening on the site, but today with tens of new crates being published every single day, we need to find a way of editorialising what content may be interesting to people.

We may also filter this down to crates that are in active use by some measure, e.g. crates with a certain number of recent downloads reaching 1.0 might give an even more interesting list.

[XAMPPRocky]

I think highlighting stable crates is a great idea. I think it might be nice if was something slightly more general like "recent stable releases", and included any major version bump not just 1.0, as there are also crates that are reaching 2.0 releases and beyond. I don't know enough about crate.io's backend to say if this would make the logic more complex or not than the original suggestion.

[sgrif]

This might be a little harder to implement than just performing the query on the summary page, the query to get this is:

select * from (
  select versions.*, (to_semver_no_prerelease(num)).major as major, 
    lag((to_semver_no_prerelease(num)).major) over (partition by crate_id order by to_semver_no_prerelease(num)) as last_major
  from versions where not yanked order by created_at desc
) t where major > last_major limit 10;

Unfortunately this requires two sorts, which means we can only index one of them, which means this query will grow linearly with the size of the versions table and can't be done as is. We either need to remove one of the sorts or somehow restrict the size of the data we start with. The former is probably easiest, we can have an is_major_version_bump column on versions and do the check on insertion

[smarnach]

I manually looked at the results of the above query, and they are mostly nice. There is one source of noise, though, which is the automatically published crates from the rustc repo, e.g. https://crates.io/crates/rustc-ap-arena/versions. They were published daily until the end of October, and weekly since then. However, even on the weekly schedule they still account for about 40 percent of the major version upgrades, and arguably these upgrades aren't particularly interesting. Maybe we should exclude major version upgrades above version 50 or 100 or so?

[bruceadams]

So long as there exist at least ten crates with exactly one version above 0.*, this query below will pick the ten most recently updated such crates. The miss here is we could pick up a crate which released 1.0.0 two months ago but released 0.4.2 yesterday.

select count(versions), crates.*
from crates
join versions on crates.id = versions.crate_id
where versions.num not like '0.%'
group by crates.id
order by 1, crates.updated desc
limit 10;

With the results from the query above, we'd still need to lookup the relevant version information to show on the crates.io page.

We could deal with the problem of a newly updated 0._ release by upping the limit in the query above and post processing to re-sorting the results based on the non-0._ version updated_at. I'm guessing this cross version update issue is rare enough that a limit of 12 or 15 would be high enough to hide the problem.

[bruceadams]

I had fun looking at the data (once I figured out how to retain updated_at in the crates table, see #1996). Three crates with a single, recent 1.x.x release are well over a year old. bloomfilter is more than five years old ‼️ and released 1.0.1 in early December.

select count(versions), crates.id, crates.name, crates.updated_at, crates.created_at
from crates
join versions on crates.id = versions.crate_id
where versions.num not like '0.%'
group by crates.id
order by 1, crates.updated_at desc
limit 30;
 count |   id   |          name           |         updated_at         |         created_at
-------+--------+-------------------------+----------------------------+----------------------------
     1 | 164944 | broadcaster             | 2019-12-17 16:53:45.430946 | 2019-09-15 20:43:02.429914
     1 | 182951 | libtzfile               | 2019-12-17 15:52:48.672054 | 2019-11-20 10:46:45.222555
     1 | 102959 | goban                   | 2019-12-17 14:44:47.319074 | 2018-12-20 18:18:54.96608
     1 | 131627 | ggrep                   | 2019-12-16 21:33:31.232198 | 2019-05-02 21:44:42.070005
     1 | 189808 | async-ctrlc             | 2019-12-16 18:01:09.871201 | 2019-12-16 18:01:09.871201
     1 | 189695 | nuki-sys                | 2019-12-16 08:11:22.84255  | 2019-12-16 08:11:22.84255
     1 |  48178 | indradb                 | 2019-12-15 14:20:28.646495 | 2018-01-24 19:50:43.362237
     1 |  48176 | indradb-lib             | 2019-12-14 23:12:11.054445 | 2018-01-24 19:32:17.824664
     1 | 175046 | atomic-take             | 2019-12-14 12:11:58.45368  | 2019-10-23 23:28:04.129827
     1 | 189081 | chirpstack_api          | 2019-12-13 15:58:18.988206 | 2019-12-13 15:58:18.988206
     1 | 168545 | httpserv                | 2019-12-13 08:09:27.774862 | 2019-09-29 04:02:49.895571
     1 | 114797 | fluminurs               | 2019-12-10 15:28:14.247949 | 2019-02-14 18:45:22.895285
     1 | 188069 | buffer_io               | 2019-12-09 23:18:05.73565  | 2019-12-09 23:18:05.73565
     1 | 147215 | ro                      | 2019-12-08 21:23:51.852112 | 2019-07-07 05:10:06.690166
     1 | 184482 | bitboard_xo             | 2019-12-08 20:12:56.65458  | 2019-11-26 13:46:08.852656
     1 | 187138 | niffler                 | 2019-12-07 18:02:03.695254 | 2019-12-07 18:02:03.695254
     1 | 186913 | make_singleton          | 2019-12-06 18:08:53.156483 | 2019-12-06 18:08:53.156483
     1 | 186874 | make_singleton_derive   | 2019-12-06 16:01:44.387389 | 2019-12-06 16:01:44.387389
     1 | 186789 | args-to-json            | 2019-12-06 02:27:18.853489 | 2019-12-06 02:27:18.853489
     1 | 186603 | yeslogic-fontconfig-sys | 2019-12-05 02:53:42.380581 | 2019-12-05 02:53:42.380581
     1 | 186538 | dull                    | 2019-12-04 20:54:23.546872 | 2019-12-04 20:54:23.546872
     1 | 186535 | json2cbor               | 2019-12-04 20:39:10.062133 | 2019-12-04 20:39:10.062133
     1 | 186373 | evmc-declare            | 2019-12-04 12:31:10.430193 | 2019-12-04 12:31:10.430193
     1 | 186372 | evmc-vm                 | 2019-12-04 12:26:47.017427 | 2019-12-04 12:26:47.017427
     1 | 186370 | evmc-sys                | 2019-12-04 12:17:55.698961 | 2019-12-04 12:17:55.698961
     1 |    402 | bloomfilter             | 2019-12-03 16:18:12.78814  | 2014-11-26 03:03:29.614564
     1 | 130718 | micromath               | 2019-12-03 01:52:34.916437 | 2019-04-28 15:49:15.125151
     1 | 185604 | ssh_host                | 2019-12-02 18:57:32.917285 | 2019-11-30 21:47:00.077921
     1 | 185688 | serde_mangadex          | 2019-12-01 05:31:28.59979  | 2019-12-01 05:31:28.59979
     1 | 185598 | acpitool                | 2019-11-30 21:10:06.824013 | 2019-11-30 21:10:06.824013
(30 rows)