You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _overviews/tutorials/binary-compatibility-for-library-authors.md
+33-30Lines changed: 33 additions & 30 deletions
Original file line number
Diff line number
Diff line change
@@ -26,11 +26,11 @@ Before we start, let's understand how code is compiled and executed on the Java
26
26
Scala is compiled to a platform-independent format called **JVM bytecode** and stored in `.class` files. These class files are collated in JAR files for distribution.
27
27
28
28
When some code depends on a library, its compiled bytecode references the library's bytecode. The library's bytecode is referenced by its class/method signatures and loaded lazily
29
-
by the the JVM classloader during runtime. If a class or method matching the signature is not found, an exception is thrown.
29
+
by the JVM classloader during runtime. If a class or method matching the signature is not found, an exception is thrown.
30
30
31
31
As a result of this execution model:
32
32
33
-
* We need to provide the JARs of every library used in our dependency tree when starting an application, since the library's bytecode is only referenced -- not merged into its user's bytecode
33
+
* We need to provide the JARs of every library used in our dependency tree when starting an application since the library's bytecode is only referenced, not merged into its user's bytecode
34
34
* A missing class/method problem may only surface after the application has been running for a while, due to lazy loading.
35
35
36
36
Common exceptions from classloading failures includes
@@ -42,21 +42,24 @@ Consider an application `App` that depends on `A` which itself depends on librar
42
42
for all of `App`, `A` and `C` (something like `java -cp App.jar:A.jar:C.jar:. MainClass`). If we did not provide `C.jar` or if we provided a `C.jar` that does not contain some classes/methods
43
43
which `A` calls, we will get classloading exceptions when our code attempts to invoke the missing classes/methods.
44
44
45
-
These are what we call **Binary Incompatibility Errors**. An error caused by binary incompatibility happens when the compiled bytecode references a name that cannot be resolved during runtime
45
+
These are what we call **Binary Incompatibility Errors** -- errors that happen when the compiled bytecode references a name that cannot be resolved during runtime.
46
+
47
+
Before we look at how to avoid binary incompatibility errors, let us first
48
+
establish some key terminologies we will be using for the rest of the guide.
46
49
47
50
## What are Evictions, Source Compatibility and Binary Compatibility?
48
51
49
52
### Evictions
50
53
When a class is needed during execution, the JVM classloader loads the first matching class file from the classpath (any other matching class files are ignored).
51
-
Because of this, having multiple versions of the same library in the classpath is generally undesireable:
54
+
Because of this, having multiple versions of the same library in the classpath is generally undesirable:
52
55
53
56
* Unnecessary application size increase
54
-
* Unexpected runtime behaviour if the order of class files changes
57
+
* Unexpected runtime behavior if the order of class files changes
55
58
56
59
Therefore, when resolving JARs to use for compilation and packaging, most build tools will pick only one version of each library and **evict** the rest.
57
60
58
61
### Source Compatibility
59
-
Two library versions are **Source Compatible** if switching one for the other does not incur any compile errors.
62
+
Two library versions are **Source Compatible** if switching one for the other does not incur any compile errors or unintended behavioral changes at runtime.
60
63
For example, If we can upgrade `v1.0.0` of a dependency to `v1.1.0` and recompile our code without any compilation errors, `v1.1.0` is source compatible with `v1.0.0`.
61
64
62
65
### Binary Compatibility
@@ -66,46 +69,45 @@ For example, if we can replace the class files of a library's `v1.0.0` with the
66
69
67
70
**NOTE:** While breaking source compatibility normally results in binary compatibility breakages as well, they are actually orthogonal -- breaking one does not imply breaking the other.
68
71
69
-
### Forwards and Backwards Compatibility
72
+
####Forwards and Backwards Compatibility
70
73
71
74
There are two "directions" when we describe compatibility of a library release:
72
75
73
76
**Backwards Compatible** means that a newer library version can be used in an environment where an older version is expected. When talking about binary and source compatibility,
74
77
this is the common and implied direction.
75
78
76
79
**Forwards Compatible** means that an older library can be used in an environment where a newer version is expected.
77
-
Forward compatibility is generally not upheld for userland libraries. It is only important in situations where an older version of a library is commonly
78
-
used at runtime against code that is compiled with newer version. (e.g. Scala's standard library)
80
+
Forward compatibility is generally not upheld for userland libraries. It is only important in situations where an older version of a library is commonly used at runtime against code that is compiled with newer versions. (for example, [Scala's standard library](http://docs.scala-lang.org/overviews/core/binary-compatibility-of-scala-releases.html))
79
81
80
82
Let's look at an example where library `A v1.0.0` is compiled with library `C v1.1.0`.
`C v1.1.0 ` is **Forwards Binary Compatible** with `v1.0.0` if we can use `v1.0.0`'s JAR at runtime instead of `v1.1.0`'s JAR without any binary compatibility errors.
85
87
86
-
`C v1.2.0 ` is **Backwards Binary Compatible** with `v1.1.0` if we can use `v1.2.0`'s JAR at runtime instaed of `v1.1.0`'s JAR without any binary compatibility errors..
88
+
`C v1.2.0 ` is **Backwards Binary Compatible** with `v1.1.0` if we can use `v1.2.0`'s JAR at runtime instead of `v1.1.0`'s JAR without any binary compatibility errors.
87
89
88
90
## Why binary compatibility matters
89
91
90
-
Binary Compatibility matters because failing to maintain it makes life hard for everyone.
92
+
Binary Compatibility matters because breaking binary compatibility have bad consequences on the ecosystem around the software.
91
93
92
-
* End users has to update all library versions in their whole transitive dependency tree such that they are binary compatible, otherwise binary compatibility errors will happen at runtime
93
-
* Library authors are forced to update the dependencies of their library so users can continue using them, greatly increases the effort required to maintain libraries
94
+
* End users have to update versions transitively in all their dependency tree such that they are binary compatible. This process is time-consuming and error-prone, and it can change the semantics of end program.
95
+
* Library authors need to update their library dependencies to avoid "falling behind" and causing dependency hell for their users. Frequent binary breakages increase the effort required to maintain libraries.
94
96
95
-
Constant binary compatibility breakages in libraries, especially ones that are used by other libraries, is detrimental to our ecosystem as they require a lot of effort
96
-
from users and library authors to resolve.
97
+
Constant binary compatibility breakages in libraries, especially ones that are used by other libraries, is detrimental to our ecosystem as they require time
98
+
and effort from end users and maintainers of dependent libraries to resolve.
97
99
98
100
Let's look at an example where binary incompatibility can cause grief and frustration:
99
101
100
102
### An example of "Dependency Hell"
101
103
102
-
Our application `App` depends on library `A` and `B`. Both `A` and `B` depends on library `C`. Initially both `A` and `B` depends on `C v1.0.0`.
104
+
Our application `App` depends on library `A` and `B`. Both `A` and `B` depends on library `C`. Initially, both `A` and `B` depends on `C v1.0.0`.
Some time later, we see `B v1.1.0` is available and upgrade its version in our build. Our code compiles and seems to work so we push it to production and go home for dinner.
108
+
Sometime later, we see `B v1.1.0` is available and upgrade its version in our build. Our code compiles and seems to work so we push it to production and go home for dinner.
107
109
108
-
Unfortunately at 2am, we got frantic calls from customers saying that our application is broken! Looking at the logs, you find lots of `NoSuchMethodError`is being thrown by some code in `A`!
110
+
Unfortunately at 2am, we get frantic calls from customers saying that our application is broken! Looking at the logs, you find lots of `NoSuchMethodError`are being thrown by some code in `A`!
Library authors use verioning schemes to communicate compatibility guarantees between library releases to their users. Versioning schemes like [Semantic Versioning](http://semver.org/)(SemVer) allow
165
-
users to easily reason about the impact of a updating a library, without needing to read the detailed release note.
166
+
Library authors use versioning schemes to communicate compatibility guarantees between library releases to their users. Versioning schemes like [Semantic Versioning](http://semver.org/)(SemVer) allow
167
+
users to easily reason about the impact of updating a library, without needing to read the detailed release note.
166
168
167
-
In the following section we will outline a versioning scheme based on Semantic Versioning that we **strongly encourage** you to adopt for your libraries. The rules listed below are **in addition** to
169
+
In the following section, we will outline a versioning scheme based on Semantic Versioning that we **strongly encourage** you to adopt for your libraries. The rules listed below are **in addition** to
168
170
Semantic Versioning v2.0.0.
169
171
170
-
### Recommmended Versioning Scheme
172
+
### Recommended Versioning Scheme
171
173
172
-
* If backwards**binary compatibility** is broken, **major version number** must be increased
173
-
* If backwards**source compatibility** is broken, **minor version number** must be increased
174
-
* A change in **patch version number** signals **no binary nor source incompatibility**. According to SemVer, patch versions should contain only bug fixes that fixes incorrect behavior so major behavioral
174
+
* If backward**binary compatibility** is broken, **major version number** must be increased
175
+
* If backward**source compatibility** is broken, **minor version number** must be increased
176
+
* A change in **patch version number** signals **no binary nor source incompatibility**. According to SemVer, patch versions should contain only bug fixes that fix incorrect behavior so major behavioral
175
177
change in method/classes should result in a minor version bump.
176
178
* When major version is `0`, a minor version bump **may contain both source and binary breakages**
177
179
* Some libraries may take a harder stance on maintaining source compatibility, bumping the major version number for ANY source incompatibility even if they are binary compatible
@@ -189,8 +191,8 @@ Many libraries in the Scala ecosystem has adopted this versioning scheme. A few
189
191
190
192
If this version scheme is followed, reasoning about binary compatibility is now very simple:
191
193
192
-
* Ensure major versions of the all versions of a library in the dependency tree are the same
193
-
* Pick latest version and evict the rest (This is the default behavior of SBT).
194
+
* Ensure major versions of all versions of a library in the dependency tree are the same
195
+
* Pick the latest version and evict the rest (This is the default behavior of SBT).
194
196
195
197
### Explanation
196
198
@@ -202,11 +204,12 @@ From our [example](#why-binary-compatibility-matters) above, we have learned two
202
204
* If a new library version is binary compatible but source incompatible, the user can simply fix the compile errors in their application and everything will work
203
205
204
206
Therefore, **binary incompatible releases should be avoided if possible** and be more noticeable when they happen, warranting the use of the major version number. While source compatibility
205
-
is also important, if they are minor breakages that does not require effort to fix, then it is best to let the major number signal just binary compatibility.
207
+
is also important, if they are minor breakages that do not require effort to fix, then it is best to let the major number signal just binary compatibility.
206
208
207
209
## Conclusion
208
210
209
-
In this guide we covered the importance of binary compatibility and showed you a few tricks to avoid breaking binary compatibility. Finally, we laid out a versioning scheme to communicate
211
+
In this guide, we covered the importance of binary compatibility and showed you a few tricks to avoid breaking binary compatibility. Finally, we laid out a versioning scheme to communicate
210
212
binary compatibility breakages clearly to your users.
211
213
212
214
If we follow these guidelines, we as a community can spend less time untangling dependency hell and more time building cool things!
0 commit comments