Skip to content

Add blog article about preventing version conflicts with versionScheme #1210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 17, 2021

Conversation

julienrf
Copy link
Contributor

@julienrf julienrf commented Feb 8, 2021

No description provided.


## What is a version conflict?

However, the library ecosystem is not without problems. A library that you pulled could depend on other libraries, and the transitive dependencies could cause version conflicts. Here's a quick example of a Scala project that uses Akka HTTP, a Postgres database, and JSON. Its build declares two library dependencies, `akka-http-circe` and `doobie-postgres-circe`:
Copy link
Member

@SethTisue SethTisue Feb 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this should explicitly be framed as a general JVM problem, not a Scala-specific problem. As JVM vets that's ingrained in us, but not all readers are #JVM4Life.

Copy link
Member

@SethTisue SethTisue Feb 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, readers might like to know if the solution only works for Scala libraries, or whether Java libraries could use it too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While you're right that this stems from the loose behavior of the dependency resolvers used in JVM, given we have Scala.js and Native, I don't think we should make it too JVM centric either.


Here, we see that sbt successfully detected the conflict for circe-core.

This was a step in the right direction, but it did not work well in reality because I had no way to tell whether two versions of a library would be binary compatible or not. In 2014 what I did was guess that a Java library would adopt [Semantic Versioning][2], and a Scala library would adopt [PVP][3]. (Apparently, using the first two numbers of the version, e.g. 1.2.x, to mean major version has a name, and it's Haskell Package Versioning Policy, or PVP for short).
Copy link
Member

@SethTisue SethTisue Feb 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've never heard of PVP in my life. Apparently it is not common parlance. I'm not sure whether introducing it here is educational, or simply a distraction. I lean towards distraction. Maybe substitute "epoch.major.minor"? Not sure.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some point we should introduce the term PVP since it's one of the supported version schemes - https://www.scala-sbt.org/1.x/docs/Publishing.html#Version+scheme


- When the `major` version is `0`, a minor version increment MAY contain **both source and binary breakages**, but a patch version increment MUST remain **binary compatible**.

We call this Early SemVer, because according to the [Semantic Versioning Spec][2] there are no guarantees between any versions when the major version is `0`. In the Scala library ecosystem, though, we often start guaranteeing binary compatibility for `0.y.z` like sbt 0.13 and Scala.JS 0.6.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"We call this" is ambiguous who "we" is. Is "early semver" standard terminology, or is it sbt-specific?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

early-semver is a version scheme Alex came up as a name in coursier/versions#7, and I've been pushing it as implementation in Coursier, sbt, and elsewhere.


## Summary

If you are a library author, check out [sbt-version-policy][5] to enforce the recommended versioning scheme. Or, at least declare the versioning scheme you use, with the `versionScheme` key. If you are a library user, keep in mind that starting from sbt 1.5.0 you should configure your `libraryDependencySchemes` to get accurate eviction errors.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest putting the recommendation to use versionScheme first. It is a universal recommendation to all library authors that we realllllllly want eeeeeeverybody to take.

The other recommendations are more on a "if you want" basis.

Copy link
Member

@SethTisue SethTisue Feb 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, iiuc, that recommendation is so important, so central, that maybe it ought to be right at the top of the blog post, as well as here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be worth turning this into a list

  • Set ThisBuild / versionScheme key if you're publishing a library
  • Check out [sbt-version-policy][5] to enforce the recommended versioning scheme
  • From sbt 1.5.0, use libraryDependencySchemes to supplement the version scheme information

and also copying it to the head of the post as tl;dr in the spirit of "we're not writing fiction".

@julienrf
Copy link
Contributor Author

julienrf commented Feb 9, 2021

Thank you all for your feedback! I’ve pushed a couple of commits that address most of your points. I have left a few discussions open, although I don’t think I will address them, except this one: #1210 (comment). I think this point needs to be resolved, but it depends on sbt itself.

@SethTisue
Copy link
Member

SethTisue commented Feb 10, 2021

(this is more of a general thought than one about this post in particular, but)

On these blog posts we always put "by: Eugene Yokota, Julien Richard-Foy" at the top, as if everybody knows who that is. Because everybody who's in-the-know does know who Eugene and Julien are. (And that Seth guy, who is he? I know I've seen the name around, but what does he do?)

But not everybody's in-the-know. How about little bios, even if it's only one sentence? "Eugene Yokota is the lead developer of sbt. Julien Richard-Foy makes MOOCs for the Scala Center". That kind of thing. And maybe links to your Twitters? It could go at the bottom? In italics?

Just thinkin'...

@julienrf
Copy link
Contributor Author

julienrf commented Feb 16, 2021

Thank you all for your comments and suggestions.

I think the doubts about the behavior of sbt in 1.5.0 is solved, I’ve pushed a last commit with a few clarifications. I think this is ready to go, please let me know if there is anything left I should address?

@julienrf julienrf changed the title Add blog article about improving Scala library ecosystem Add blog article about preventing version conflicts with versionScheme Feb 17, 2021
@julienrf julienrf merged commit a22eed6 into scala:master Feb 17, 2021
@julienrf julienrf deleted the library-ecosystem branch February 17, 2021 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants