From 0f11be520e2395a4f9f85125789cfed2a679fefb Mon Sep 17 00:00:00 2001 From: Jacob Wang Date: Sun, 10 Sep 2017 23:49:01 +1000 Subject: [PATCH 01/13] Guide on binary compatibility for library authors --- ...inary-compatibility-for-library-authors.md | 140 ++++++++++++++++++ .../library-author-guide/after_update.png | Bin 0 -> 5030 bytes .../library-author-guide/before_update.png | Bin 0 -> 4892 bytes .../dependency_hell.plantuml | 25 ++++ .../library-author-guide/dependency_hell.png | Bin 0 -> 12927 bytes 5 files changed, 165 insertions(+) create mode 100644 _overviews/tutorials/binary-compatibility-for-library-authors.md create mode 100644 resources/images/library-author-guide/after_update.png create mode 100644 resources/images/library-author-guide/before_update.png create mode 100644 resources/images/library-author-guide/dependency_hell.plantuml create mode 100644 resources/images/library-author-guide/dependency_hell.png diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md new file mode 100644 index 0000000000..58a78a0eaa --- /dev/null +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -0,0 +1,140 @@ +--- +layout: singlepage-overview +title: Binary Compatibility for library authors + +discourse: true +permalink: /tutorials/:title.html +--- + +## Introduction + +A diverse and comprehensive set of libraries is important to any productive software ecosystem. While it is easy to develop and distribute Scala libraries, good library authorship goes far +beyond just writing code and publishing them. + +In this guide, we will cover the important topic of **Binary Compatibility**: + +* How binary incompatibility can cause production failures in your applications +* How library authors can avoid breaking binary compatibility, and/or convey breakages clearly to library users when they happen + +Before we start, first we need to understand how code is compiled and executed on the Java Virtual Machine (JVM). + +## The JVM execution model + +Code compiled to run on the JVM is compiled to a platform-independent format called **JVM bytecode** and stored in **Class File** format (with `.class` extension) and these class files are stored +in JAR files. The bytecode is what we refer to as the **Binary** format. + +When application or library code is compiled, their bytecode invokes named references of classes/methods from their dependencies instead of including the dependencies' actual bytecode +(unless inlining is explicitly requested). During runtime, the JVM classloader will search through the provided class files for classes/methods referenced by and invoke them. + +Let's illustrate with an example: + +We got an application `App` that depends on `A` which itself depends on library `C`. When starting the application we need to provide the class files +for all of `App`, `A` and `C` (something like `java -cp App.jar:A.jar:C.jar:. MainClass`). If we did not provide `C.jar`, or if we provided a `C.jar` that does not contain certain classes/methods +which `A` expected to exist in library `C`, we will get an exception when our code attempt to invoke the missing classes/methods. + +This is what we call **Binary Incompatibility Errors** - The bytecode interface used for compilation differs and is incompatible with the bytecode provided during runtime. + +## What are Evictions, Source Compatibility and Binary Compatibility? + +Since the classloader only loads the first match of a class, having multiple versions of the same library in the classpath is redundant. +Therefore when deciding which JARs to use for compilation, SBT only selects one version from each library (by default the highest), +and all other versions of the same library are **evicted**. When packaging applications, the same versions of libraries that was used for compiling the +application is packaged and used during runtime. + +Two library versions are said to be **Source Compatible** if switching one for the other does not incur any compile errors. For example, If we can switch from `v1.0.0` of a dependency to `v2.0.0` and +recompile our code without causing any compilation errors, `v2.0.0` is said to be source compatible with `v1.0.0`. + +Two library versions are said to be **Binary Compatible** if the compiled bytecode of these versions are compatible. Using the example above, removing a class will render two version +binary incompatible too, as the compiled bytecode for v2.0.0 will no longer contain the removed class. + +When talking about two versions being compatible, the direction matters too. If we can use `v2.0.0` in place of `v1.0.0`, `v2.0.0` is said to be **backwards compatible** with `v1.0.0`. Conversely, +if we say that any library release of `v1.x.x` will be forwards compatible, we can use `v1.0.0` anywhere where `v1.1.0` was originally used. +For the rest of the guide, when the 'compatible' is used we mean backwards compatible, as it is the more common case of compatibility guarantee. + +An important note to make is that while breaking source compatibility normally results in breaking binary compatibility, they are actually orthorgonal +(breaking one does not imply breaking the other). See below for more examples (TODO: make sure we have examples?) +TODO: more facts? + +## Why binary compatibility matters + +Let's look at an example where binary incompatibility between versions of a library can have catastrophic consequences: + +Our application depends on library `A` and `B`. Both `A` and `B` depends on library `C`. Initially both `A` and `B` depends on `C v1.0.0`. + +![Initial dependency graph]({{ site.baseurl }}/resources/images/library-author-guide/before_update.png){: style="width: 280px; margin: auto; display: block;"} + +Some time later, we see `B v1.1.0` is now available and we upgraded the version in our `build.sbt`. Our code compiles and seems to work so we push it to production and goes home for dinner. + +Unfortunately at 2am, we got frantic calls from customers saying that our App is broken! Looking at the logs, you find lots of `NoSuchMethodError` is being thrown by some code in `A`! + +![Binary incompatibility after upgrading]({{ site.baseurl }}/resources/images/library-author-guide/after_update.png){: style="width: 280px; margin: auto; display: block;"} + +Why did we get a `NoSuchMethodError`? Remember that `A v1.0.0` is compiled with `C v1.0.0` and thus calls methods availble in `Cv1.0.0`. While `B` and +our App has been recompiled with available classes/methods in `C v2.0.0`, `A v1.0.0`'s bytecode hasn't changed - it still calls the same method that is now missing in `C v2.0.0`! + +This situation can only be resolved by ensuring that the chosen version of `C` is binary compatible with all other evicted versions of `C`. In this case, we need a new version of `A` that depends +on `C v2.0.0` (or any other future `C` version that is binary compatible with `C v2.0.0`). + +Now imagine if our App is more complex with lots of dependencies themselves depending on `C` (either directly or transitively) - it becomes extremely difficult to upgrade any dependencies because it now +pulls in a version of `C` that is incompatible with the rest of the versions of `C` in our dependency tree! In the example below, we cannot upgrade `D` because it will transitively pull in `C v2.0.0`, causing breakages +due to binary incompatibility. This inability to upgrade any packages without breaking anything is common known as **Dependency Hell**. + +![Dependency Hell]({{ site.baseurl }}/resources/images/library-author-guide/dependency_hell.png) + +How can we, as library authors, spare our users of runtime errors and dependency hell? + +* Use **Migration Manager** (MiMa) to catch unintended binary compatibility breakages before releasing a new library version +* **Avoid breaking binary compatibility** through careful design and evolution of your library interfaces +* Communicate binary compatibility breakages clearly through **versioning** + +## MiMa - Check Binary Compatibility with Previous Library Versions + +The [Migration Manager for Scala](https://github.com/typesafehub/migration-manager) (MiMa) is a tool for diagnosing binary incompatibilities between different library versions. + +When run standalone in the command line, it will compare the .class files in the two provided JARs and report any binary incompatibilities found. Most library authors use the [SBT plugin](https://github.com/typesafehub/migration-manager/wiki/Sbt-plugin) +to help spot binary incompatibility between library releases. (Follow the link for instructions on how to use it in your project) + +## Designing for Evolution - without breaking binary compatibility + +TODO + +## Versioning Scheme - Communicate binary compatiblity breakages + +We recommend using the following schemes to communicate binary and source compatibility to your users: + +* Any release with the same major version are **Binary Backwards Compatible** with each other +* A minor version bump signals new features and **may contain minor source incompatibilities** that can be easily fixed by the end user +* Patch version for bugfixes and minor behavioural changes +* For **expreimental library versions** (where the major version is `0`, such as `v0.1.0`), a minor version bump **may contain both source and binary breakages** +* Some libraries may take a harder stance on maintaining source compatibility, bumping the major version number for ANY source incompatibility even if they are binary compatible + +Some examples: + +* `v1.0.0 -> v2.0.0` is binary incompatible. Cares needs to be taken to make sure no evicted versions are still in the `v1.x.x` range to avoid runtime errors +* `v1.0.0 -> v1.1.0` is binary compatible and maybe source incompatible +* `v1.0.0 -> v1.0.1` is binary compatible and source compatible +* `v0.4.0 -> v0.5.0` is binary incompatible and maybe source incompatible +* `v0.4.0 -> v0.4.1` is binary compatible and source compatible + +Many libraries in the Scala ecosystem has adopted this versioning scheme. A few examples are [Akka](http://doc.akka.io/docs/akka/2.5/scala/common/binary-compatibility-rules.html), +[Cats](https://github.com/typelevel/cats#binary-compatibility-and-versioning) and [Scala.js](https://www.scala-js.org/). + +### Explanation + +Why do we use the major version number to signal binary compatibility releases? + +From our [example](#why-binary-compatibility-matters) above, we have learnt two important lessons: + +* Binary incompatibility releases often leads to dependency hell, rendering your users unable to update any of their libraries without breaking their application +* If a new library version is binary compatible but source incompatible, the user can simply fix the compile errors in their application and everything will work + +Therefore, **binary incompatible releases should be avoided if possible** and be more noticeable when they happen, warranting the use of the major version number. While source compatibility +is also important, if they are minor breakages that does not require effort to fix, then it is best to let the major number signal just binary compatibility. + +## Conclusion + +In this guide we covered the importance of binary compatibility and showed you a few tricks to avoid breaking binary compatibility. Finally, we laid out a versioning scheme to communicate +binary compatibility breakages clearly to your users. + +If we follow these guidelines, we as a community can spend less time untangling dependency hell and more time making cool things! + diff --git a/resources/images/library-author-guide/after_update.png b/resources/images/library-author-guide/after_update.png new file mode 100644 index 0000000000000000000000000000000000000000..0c5becc52cf7286ee55e7b081cfbb77e8ae8eade GIT binary patch literal 5030 zcmY*dc|4Tg+g3@4klhfG?+@I&1=X|bnT_@a7Uz3rZhn|XxicwqZ?tMyMPkHue zPg7b336>5@Cyr7#LD?XXUXHeK6qTkeVC!OyvVA7-)Jwn~g+eOH$RHi90hF7QqqGge ziA7eCmx}6?g7*Ux)L%apm1l}Yj2iG&mRX&cSltrFE_w0W2h{|1^$!*7I@B*-I?W16 zm)&hkwCQ&tUC1iBlN2zvkKZnI{;Fif?^8JNrB|$1jokYV5=Qg-T9xaBHJgUiG}vB0 zjGQsWA*EE(MHXAjsBZU;0`3?t8ZcZc2G<j2^vL`PMMM&D7Xa^DMOTqUb5a zrPNQhr;~2}f%s+ zvfeT6YR&DAPFd{wlc%;^r=O*&e)z{-f8dWdRq~OeHoZz6BD5%7R5i zeMAz=5qE52&skV77gJHO>}cOrd*C^|n#zP?8?0|9zGU`dOE@XzoiW|fj)h283NVJX zQw>4hMl7^^L^pkabSr<`{*++j8Ke18Ud!slI*T1Q`5Du+2B$L}ZDkJ9ebv$DsF!%! zpkqkB@20BBEkU{j;W^bt=i${Fk(OuWHO3@n9Dh7haS&4Hz#e{)7QLK()Spx1L4z4L9_R1_KbyIaB#lDLyTF zem)KqTiq;Ii^>;YT->n~#R`I*U5rw#X0{Jb2dc_)IXXBjudE1xYtFyMoIjK{c2$!u zHWjj1QFwC~FgNwO6w|)EdwA@h!Ac+8f#wQCmVM$6$HaXBt+>~YtkNjuxf#yB0o9-T zdjHvbb0FeIS<%C*Kh%nXXbQ5kg9!__Gh&O)U9jS~p0i+i3~tutLzkM7I?%z0WvNN+ zBAW~FYJ1z~b}&vqb+LKpzmHv`e1GgfU1#r4rt!C|e0-u+{Q7{xnkJx!hgna$+O%5+ z8Mr_Jnr2*2vJqc2x)gcak@}!34TCqsXN!x8 zB6|BZ6_U;O%*@z2v54^@MOUEl-?RtDPL_=)ogF{fuU^V4l#FU(vDUimIFR@D>d9R| zuNfC7JYMBH#vzlY3v{4)!TbT{V0h00vKmsBlbd_W;d9z58`imtVWiL;D+U)H(mJDf zf5T_NFrV#6Kke2v5$QFesEds+@MBZMn5F!pLslSfQ{bf`9C%#TcVE`GZKvhjA!Ro9 zgpzS;g{l2;!W)o_%?041vdnE^+h@Egv7mcbxxP0Ff;W+w^InTH8g(y2wj(OIn=>PIEU{A>yjtyNpBHfiQ}|3fvQxqBdhMy?|*;n@m~&7zU1E} ze{-Z^=Q*kI$j}dXyA+hB}1JA(hAnkQJD=Wy83E z5VW(+?GB6P^2Bj%!sw3bV9ja7Wk=%ql9f8^z$!SN-F3_F<7Mt*W_qwR`<;jhJ9k-< z8-9TQYz!ZhNQXpDE-5WwH{v()v;t!agT6TLg?LcxO8=UHgorf#s><6YzqH$)SzC#r zpXa;hZj>Wkg|>HxeIAPXR+0G4T|GxRTx|5thboP6Rb_0OOp%&K8xd{+yJL+NmGPi6 zkNff4%a7_Z`o4o6I_;>$!hk6FyqW&TvU?>u)yO5uy_n25An5ProgOC3vQOS&Hi{!m zM?N&zYles;lk*ZmY0=WgF?~Lhm!tbw>_xydr5>CRR^M;g&4CbKx1S9GBpDYQYGv$5 zmCsUU5@dC7!bCx5cXiQ;oz#ims02F=rjNS4=ezJ@McYCfXq7it$4Owq27643h4W`b zHj6X{i(*p7j7m10p(LisYTEct^qV2`iC^T2_gTC=d@iW&PPAu;WfiG_=uU2s6W2^y zyOlS(EeZ~=g5nzp)k>n_qCa-FeT7`ESlYTy4*V8MgDMP^F4r8i6}v>W;nuJFkA8k( z(`Rso{nDq~E>}WQ5sHh+l--m7%j(j88l?`<*q!UYK_-7IQc_H&EWu&o2@|%rHubhA zbpqNS?zCrF9k$(U8Vj-*8LKc35c7sU%H3&qVqfn$U_!SScH#si#XZ=I?fY=wRaST6 zK&g*$QL~42w+^Y_6KH?%P7shGGa6PR!Rg_CvJ#~st_AO^JD$Xt(%~7E#tzd*A}<`o z_WbNbTS{W9WD_u`@*BKM0^pj@l*tu_*3#<~I=kt+tXIZi8k2{aG=ztwQs*8*b~Mo@q;;ZSpvdkz0txM62GY7;#%Qf2*F)lOzG^;Pd_FCTUl@u>M7pItb1g zefN^BCX`@4(RA9|WW$044(1NTG}uL-xaB*vt_)q5LESZs5~8DzV(5R8iX7NEc3w6v zHyqE%Td4v39lTb*7t9azn0%-@XYg%;#&&a+yZgf;hSh(6&*M_Sa$;yUxh=Sm<~%Tb zYh9zg#Gjmf0FWIY~w_+)#j^v%0?^>z7kcaUqqpRd+;%f zeFiLdg`k%2@{q8fIJ-el!zAvYVlG}7OR%P3mW^~g*P*bX7#15it4 z$t&?gdl86RFSWRLXxkWT!f4myF&V>SwE^CTsUfddissF$S zyLuHrz3EwR**MNbhJJX%6Zkn6Wpp`JU0YSu{#sm$jA?B%P*pn$?8kfza>+CmkOX#J z)fXkVPNlN()8Syw8kK`Dg2`eOa$xrO@lD#-!9%cc6vsh`>9D0epmW1@vJMA##hQzO z^AK{h_OJg$XF>=JfuMnc-cuGr+!?9&6F|@TjPrW!V?}{0pQpFYW#SdpL2EJv_2J$T3whiM)H+p9e+ zHh*>*NR&MA*XuQ{^8^Ud76Hde2@kU9%jww0U$^XLYR25hV49b5IR3+1?Du(1`}_M} z%9-$cOoTjs^RTpA_julY zb>aQp8WG9jI&paWcqiuGtQg(La~Gd2Y;8f#tl!u;VZ-*4j**#o-ABv*)pT>&t|yH1 z<9Rp7@zvXXgQ@Fw3to#URy_ybC|e3?sWhK`LmF=0L-8ej_gds7PFM3&c#zT@JOgP^ zD0DXWv5O7E=vytjBO>W}C)z8kyyLac;DLH^VI76!Quu6XWUI(xMTOQ#gxo~i%bHFd zQR2zdbo~6LCxrnUpA8~EWR#j*Z@d-`oM|9yw2@T>sJaLMxUxccc%l(@)o9w!4E_VA zp`-{H${ZZ~JDwq!hgUZIY=3V1qHY||P}8X;+RnIt2Y4Bx)P=DRB3l;e1rft}?t{4( zXR6g80wIh)KA&ZE+EB?$m^_+1>l>&!i8OHn7&0+YC~cAz(=pjC7y=WmiTy+2D7^Z^ zUxUJnL;fLlSN(S&acd0tdp&|Mczp}Y*0ah(sUrLeJ~^74v9%SfIv&PK{r2nU>6WHqNPCb zFP}V^%?h=+0#8C~2aVX;%++X&nR@jH2|*P`E9T}hR|m5n;f56tf&Iqi_gE94^x?(x zlc@gp7~S-}#F%=~ErXWY;?z*N);kKLak6V3JHbU_()Ud~gtq0b3w{K&CvtK#yA!VG zu9MX@d%j)|WWHB5L$J%UI=Fm9ynWc1rDixPY4w_?UinKuohIXvW?^n~=^F}GIT&)k z)2T86-Y8%6Z8E56!uySaO@f+C1dg8x~fwB&wOF< zTrZA|@L-{i}Njl!Mv1Ks4DdD^X2^UGP)o^0{I3Qdcd zKKSA#S_}*v8TV43@Kor)s?0rT6!54WeM6OfPF)kD`m3qNbmokbIOO6ec`C}WV33~9 z(A#jcvZ5l7j}~MJj`VGjbdsqdXD4Gf#^L2yb7?ld2iO`A*6*#ONoNHV(tcHgx!neN#2BBr_?43SJzay_Un&ZgZ`Ut!Fo7qL$1A`4C-9;C4<{T944^PvK_o}^ zE{6N$44)UUw5;0}Lw*cxxga_1N~kW;>fW^NmjZXW)sjzYH;gENDliu=%8{Epo(mqG zniNs?kxSuzP%nzd{a@g2)DiE=F$1Q4z`mYmj~PQnhGIs5+Sv{3N2jrBboL0MMnlNI zm`$neybyw#Y6TRE{}iOb%{)m>nuqUiSYLBEl(METUrfX(prDT{9R(#M>K8Qt6b?6s z<37(p$cK)idiDPc#@sonUHAZwdU{kdDi32h^_1G7HxM+`nu_CcE^R4w{+=>|PbV%K zRZ;J>^jf@K4DCj;nuY^{I;uwT(50JgC^mLDYcaocrkRryO9r?7FAb)eZz^lVqiwRH zZ5PbGyw~I-WAUfqTW{if<(c&;i{59?1u5m#l)u+@{B}X!eMRnCO`xL1VjGE}4zEhl zvF^kcm~7+Y1k&W(5PBP-K!%HLA2Zx;eEjXeA3liQj#O-XM}dy-y7P6Ywr^39f5asP zf7_4$$oCFd8J1g&z$&f1nGYwt(*zsUjT_tmdNMKUB7}nPGA_2W35BDEDY0k^+LTK6 z0v!4`D&8lc^skhBL94q$Lg*D^c)=9W_!c_~I6h}3Ts&!zieDCKXIv6)9FK8&iP1&s z?vsxHNt-y2J;k#mRx**ZN4ks4p$zOe*ZTPsPi4jg4d(VE(k@(SSq literal 0 HcmV?d00001 diff --git a/resources/images/library-author-guide/before_update.png b/resources/images/library-author-guide/before_update.png new file mode 100644 index 0000000000000000000000000000000000000000..aa981a7ea2266327b1f6de55218180826e5c2119 GIT binary patch literal 4892 zcmY*dc|26#|5iwgNkWz(A?c$+wy{*m_CaHW2q9|>uE7Y&8Y)c6k{B5~*>^LtjxA;1 zhbu$YWZxUiZ+y1j>-)#O=brcboclVU`e9Zh9?%*61E)QR(iyBq~H~n82 z-Hx3|^JL+mrnUz{xa{ug{S;@uI2u_8L^4uiFecyQZHr?cR(gyqU+>}TV-V{b(iT#O z3)Y?7dmif(a*yX#RzteCEGGMm+^^xD)tk`TU*~fwszO8b2fK1XMMx1SV8~I*oX1?j(}Obw2bEd%GPm+5M*PwE6irpAI}w-RFZu#B~SYf@V%XZHt;E zx9OE^-Z2`)1@zLx9ZDu&*pOCQgm3?A^bN!5scOPiwSwymhPZVXjtBn8V|SS`VS zQRP(2?^C%zVzu|X`QH4!5qKQ1)HCeSeVGw@8RC-ecq8*|PI&~bif{&&(8@8 z(+h^LS;0Uo9pMjcttyi9*_TZVXjxg=%JT9+XbU6J^V7%pTrru~g$W|&NHG=xSk|s> zM2mHW>C#ih)F1AI1_(3=m6OXKgw21jxcRJ>}#6ssEcNC!tg0d59jU$eDw zF+*FU_#r?8rXJ)~h-KrDQAx?J9GW&M>UHSrACBbpF?%=?xVk#2Fg|v;&j1V%JHN(7 zl$K4Th8CC^_m{$W)xwm|`}Mh7Sh!ay1wkL6`g_;*@NQoHA~&ik{bjA2ZxBoAG#wqQ zMds@KxDB$vu_Xjqg9Bv0Q1{usQ@?xo!v_;aM#qbCx0p|bKzO-^{!VvbcXDR4WxbXS zQS?AFUf#FTd6ldJc!fTJ`}ztOq;5H6H(>7Dj@YkFk?ve`#7MjqVyKsqA>Cbtva&9- zaeD2s;U{s>pWX%!Cd;CSPSVRWFl`kt*K7>c`B8eZ5lZ_=+Gsg26wic{Z|MAw!)Pm# z^8!om(qTCRJ=cIBhYg-SvEkYPzXV?Ou~5KUiKjH~ThqqU0Ph+5qeF)Y_9~(gvw46z zdKBQ_p+9;-Aofyd`}c+Ei+)u;h3|FQmsUlv=i**Kb&0kKYpuTAeO->@Pl}!Wt>AA8x*EQkfZZmhG46_et;{Na8~j^=x%jihDnxE{YsanQl8#CC>VG#1fhYbfma2_0NL0XAu@P6H&tH(eUBd;c^vchxl*I zy)g;RGbm^*Hg|%p)Odzfcq6Lh*Y4bZA6a;g1E@A+*Ov<@ra#4R+Is89t>?|$m%s7z zyZAckRFUQx=B4Lt=VL(thPP_3bN&35|IS!N{gqdgTOQ=cjpm5n$s0cPSRv zqJ!0t_I|S&lKwOness$xY#X6@4EBf=G>;b_b`rkbJlVW76A;M={jw!8bNl z@8g3-8G6TIzaGePB9c!{%Hz#D3B!oF&tE4oSaIu#yhoJ+)7dH<7a#bN!yt;50Yph_ zdxvXKL>wED>kPQraaOJEfU?9Kn$!`$SW(FsKo8Tdl43fmZ?%4 zl@k=F1)*6`6Y-tNK@A8lBBU)iy+To5Mrm{SCtN#Obdr-M4=6 zY~;_m=m5r2dxiV3`K#}sDxc}Q+s??ySLlg6B|Vpv=nU`PgT)#7T)AXioAhVhwSat; z*B}j|e|`oRv-C~abKO!)1qdIp@Rw{=Wrr5N;g54j!kV|DN)1#W#UoP~e_Fj;S+D;i zk_k}2$^Y&=Za{5^$Q$*&3%1q{P6=U!L5AE^-RAhKwy|nk5RN!-F&j1=w_Ibdy%xfi zdE5Kz37FvpFT_K&ME_e;?c+PQsqN;7GgORDsrv;`+Vl>`BZ~J(WE#ctOTAxZ31T|= z3um0tREipaGwJ{ckhqk`L z?lzXu#gsaSZcCJLbN)<))J#b(hZf1tUu0LuZni(6C==nK*31KtU7^`?;(0 zJq63-_l1kjXm*QN!+5~W5qJMR37HN4iQg|b+H@`M$Cx`fe69Dpp#D{j#Rk;ld`G18 zT+uI>o{ZOBDq#NlopzqL_g9>KPZ937yuzrvTzGOX^xV6yc<5y5TC8q-`%U?`O5@F$ zi-}mFPg?LyjiLT`v6mAFld7^SF=hxtZ%v`m!9Z0{PLR2!`#PR^s3jg_SZAe;Fx_YiF~v%!lm0as(u8*vd_B>><=& zBFBIKe#U*=D$8?2Qvu`MbltgI8b)%{ny%5!mP)v12;|ho38Yw=(#A|Xe6lz}z1p9A zJwE4|mSQtN)F(3*{G^=N`&ek*59^x%ZACtc<*jE_W5T))r(4xlu0}2R#^}pLPBjwX`VInnHk|Inh>BK9ta8&J=~+8CBPFv zhnmPoT(ZAfGj(uCWT_||)=o%}bBk;&w%XtA74*r1`vzV`v3N764k zh3Nj66x)H>8E$Qz8nz5bChH{veeNFfdb)-K3_;1GrZL|3Zd6XOq<6`6ts62zLU$B? zzYZ)bA!6PVi6^eYVU7e7g8Tia+luAIo{LD+WeCjhcuk&Dd|vEV%?8rL$*k1WS3jY- z#<5x05D<-ieg^=@H?_VM6yPviZV|=2AE#PJD7;tF>Km%~+F zoS;qp)4u{`y6j(la;(jZJ>Q23D8E8>W_I&9tSN|hD`BtJYB|ziN=4?5m@mq@@rS-f z`c-eWl|3s|=B0n5*$+Bed136No-Nu8q6 z45D(dIcVz@@ydT?jIhMI<+XIig9Dgo?^5ncJ?q-UBD)=?FJwb>T~?~Eer#{^TL&Du)+fwB0nseVN%a&U z14bd7>=rIbN-iAi;=VN~O>PWUoDF22MaN*VHe(T`6!@t8yw$qW&giG|M@2X2E_5th zfx+VISH3{kU(MmdJc_UyPJxUb)uE5Um7Y;%QL(!Zy(E_DB^uKJ0diI?VUS%mbAzf* zL8RHy(PF8DZoQISm-UJ5UlS_y>D7+@XJxjQGj!V^Ez?RZ&m?&x^eF^cZc+LN3$x!K zGusT7U`D@Pbd*O9ysD5OADuX0h@8H}r==Ag1KpY*l7W@a%zPW<%`gzwkSE-a0&>Y& zSB1yz%j9{_X7`guMn?Mjjswp=VOvE~@^qHK6q*&Q+|1DFoViDF4M$Wyyl#{?2vo%| zh{a!<4=Nr-_wTaxl$b>_P|J)8`WLroc)Xm_r>d@*to*@dTzcx*$8-E8%K`Sz!A5z) zr*by+TFfjh#XT=|)B-$fVJ0KVp474*(KPt}Wy|LE;;wlrH`M}1?kbo@Lbdb?h^h3t zxHr(jpIPDfhYEmU9yaUX(HPBQB67zZ=6@l8dz)|8AFd#@8!=x!4O;Lxb};8r6&-5nUz>K_NMP`R@By(;=RB9BHcH)PnKOe7Of0Y!^86@n_LU?L6vWt^3Efe zBU)zv20>Ia65)4=>%uy-+d1Fab^<1e+zSg%w=AV3r_$4LC*`o#gh{O`WjwdycHyY0 z!MTI%^F0H$ThBcS+Mk{?B*9C-)ePDI^rT4raylTDX0>+nPtp7oO18zlDduQhB*t_h z|1#)6ziy}=EVH@gKcdVHk5J7tJNJ8%j}I5IDWbl=lb-T)0YOZGyjC ze>LNs<((n3fQz?(pGB5^DoMKeqmfATDw-u!nZsjiYHY2fBkwrM2Ri3RQr1$T%TnY! z^t3N-eVoO(aUZF*{4&oB)=}TMkNw`U^W&L+Maag%KrVEa)5iZMt=V4`_$q>xlSYa7 z#u{`a#R_Dhc{@E1j|x;2;c|0!sayKRalAOUZM5I{K}aAQS3j?YsS-9 z(qD8K9toX_aA$Kc)HLCSeVP!-l^y?omWy!{;{3wu{xZq^^LVs-`ib$o+tf*kEXdK% ejeBAgai^U#5z>trNZRi%IxTfwwZglPgZ~eOrM{{F literal 0 HcmV?d00001 diff --git a/resources/images/library-author-guide/dependency_hell.plantuml b/resources/images/library-author-guide/dependency_hell.plantuml new file mode 100644 index 0000000000..30938a1ee6 --- /dev/null +++ b/resources/images/library-author-guide/dependency_hell.plantuml @@ -0,0 +1,25 @@ +component "App" as app +component "A 1.0.0" as a1 +component "B 1.0.0" as b1 +component "C 1.0.0" as c_1 +component "D 1.1.0" as d +component "E 1.5.0" as e +component "C 1.0.0" as c_2 +component "C 1.0.0" as c_3 + +app -down-> a1 +app -down-> b1 +app -down-> d +a1 -down-> c_1 +b1 -down-> c_2 +d -down-> e +e -down-> c_3 + +cloud "Available Updates" { + component "D 1.1.1" as d2 + component "E 1.5.1" as e_new + component "C 2.0.0" as c_new_1 + + d2 -down-> e_new + e_new -down-> c_new_1 +} diff --git a/resources/images/library-author-guide/dependency_hell.png b/resources/images/library-author-guide/dependency_hell.png new file mode 100644 index 0000000000000000000000000000000000000000..139803c11d1b17f9d96e478ee19c07d4ca4a7afd GIT binary patch literal 12927 zcmcJ$c|6qL8#k;Xq{x&grtFj~B@9E7eUB{3o-JYQV~rvq8rgTszHh_WX-H$qP_m3A z*|)J|-`!{W{=U!ezMtp$=e}RB`wx9SKF+z$bUEQsS=Et zE^EoMUgH^!LKJ6p42T-iK&I+_h8~Hj)r^d<7R?=0E87~F8>e}3_Sb$_;iXA6w2=6$ zV)nJN=k<1=-y`AShaMMVD!*60{-K^c#*xYM)zI>-r6>Dj%gYH0(X|l8GV)!c)mZ<> z%3`iAaaymxkLz@BcZYYu4y}JedO!Bp-Rnb5>{#c#LM#|rM)%D}bUFR!;^O52scRbxY zJB{R=)W)i<=q3q0Y~OxrTzNV(6#8wj%bO~#^8I^E|Jr5@`1-F*%G%0b$3DEZBeT*5cTH?_$e#`2sPtbe?Fh&(wlH+N^iEAg zf`QWV&feKOdozvAZIE>`G9g!GIT;<#;pNl|ZaV$7JNXwMt8$n{u6dsk%rkn|KO-ie z9aE}8-|%YQIK=3kQDN`!NQ7a0O%g;-#^^yt-WG==r!SW!c>s0LW#%)dzkgU~`_x7* z;v{19u#)M)rPJA~vK~GXvlcG1-%^ARh)pg7qRmN<>-Mi6BCS6K*(;{+ zr!J9^v0}-{l!X4zpOEwa^#A{VzKUdlIk5^03-j~yb8uv4WCWm_;9=xvcB29*>1d%6 zm5*~ZpaC@tImtR4b4ABCk!EB!mK@XXzs!=_Arf3WV{SssyA$rnLKlXs?EL)h+lWXr zmz?9kl3imfkFA`JvF->bR+~v9Z4X;LuPA&_>pu>s8d*+Yyz;)&2gT4IFy#U`!1z z9v&MTn^gi@QKzP<={oh<-lRASc(F-bTpV}qbSye4UDwFHUw3bB4?+c0db(69f{Bib zf@F9F9>>JQkVoLTxVRV?8NuK7e=P1Y??Ua|G@eZ)C0z>*T3T8%TxL`VIQ2{+WO& zW(WjAR~$bL`RTT{HeFp^C81X)ubp;Q#)|h2+4h5^_ok!YydkeuFgZD#lTPNu79X2) z-n|w(3FCmlGc9l^O{uNRknpcHVCcJsCRMy?2BYC*i zTBu)YQs;IPN>7&b%nDbUxWP=|^@Km|`)NgxqD8eNE1yXu%H&3B2P*U$Xh-N19$9tWC5E*3Gk1kCZkadsCl* zsyfY|@U2btsRvT6_Iq;O2yXJHiUO~$@F9& ztlUjDp;l+0nlOB&xcXOAm%{sNTnFp74qW6?z4vC~XO?T6r;5U?Oi}I5e7?uO1L-Uk zPu~$yuQH~lJ>tn!jJU#3SXz2R*gCu+mbYmpxP9Cac@ndM~1rke-p{>Qa>XdTl+u8NAO{q1GGYRX#el~kW? zqPh09R|-Sv^mn}0Emhrga6xa(7Gs{jCbwc|!J(kh)j&-M9cDjqF_>mJ_II!UI!^jn zt1%Yp1@~P09d{!G!LPFWrY~2sYsBwpU!EQ7a5T;)`<6VtI4_T?5XQe6;_z}AZvz}LXt}uH6CGVSjK}J)0eV6_hl*>^LSJszoTa5@ds9Pa6N^@^occ<` zh6X+KJIb1F%wc)t@X}IT=`&8N zOp>b0Dwq~pfok_d%ZQ6SsdkxB@q!AOJwTUfEZ>my>_w4RAMgxBdxoGwCR&5Lj&szL zi9<3Ml&(v8|CC@qz&4xQ*gtu@5>jUSx~yqCQP@Vu zvDm&UyyyNw(cy&D&S+-Z(v0r*ObAQ$hJW!y&3^MI{P0cGUH!7n;N#(S3b1nSdn`*G ztx3oq1!~by%8&46Yh3>8nmFM)&a$YfXl`|F$9~=vZ_n!e5Q<2M7bHHg(Pu|b#-koG zx{_{bV-u#NOy&u!IXEGPTc(iL7Mz7TF||~Qs+r3rzbEtZ%?g5Lwr7;knj}lza*X~l z{V^qq#&X=++2rL`zkqSw(8+KGM?af?#%y)Rl&axNtR?Q$F84)DB9yUbK6-ytfb{!$ zu#-w<5ly$i3LSs;LmEj&?~3^{QvB7p2x`MgYDDHIIAqVj4X1rz_e7-Cz8!U@_u2p| zEO?&_P755~vn!x^E`>Ty^Jg6K#|yjt@M9d)RE^*hS&tT;+V!Tu&(uOh&~$Ahx7Q-J9ftP z3ghL=?X8iXZ#{jx7ViEwFmdZ#flc<4g&#GOM!Z_**MX7^h%}SUFykDGAer!;&9WtKfvB--*>!AMG37fT5?+rJw!|e)M_c0dOakI> zdsP}%hs`i}L{J)aqtN$7@BC@GE+vo%bN=#R6ZBGR@!K4gM!dpJW}iA2v9evexihS>sm*ORL`qzuY&%*b^=xHQ2EG9^~+YMl(u? zy~8E$^FD`v5+{yhN8+!Fa|94!C(3Ia5r?dTtF{EMg=`%H-L?h20aqu<=YFsBIY&Lf zf-AI^W5?Ee%=s-D>a$|6KR~ARaPATb(ndbZ&7Fnb_G-$?dIy~msw#Jj`Kl4!*CcoA zkeY1Rrg<^9GNchF-M|Tg21iGh=x#|bi7ntLo_}O=qzmR7% zju~3H$(g`gqf4!!)^L$}`!;EP#2~&sQsYZ&R@`b#Z^`=@)fC@o9s~eE(d7R73vpRl zc&u!{y84WZ9IB7rTqv>#I_C*n3FYw%C$BGV2K%B$+84L|P3LafPI?iUC+};Q$C7iu zj_&WQ_SwTq`X4Km37$5`+2Mk?u(=ws?<~$kz5DXCs@un;jK14@2}O0a1EZ>n{{8w) zX&f1bAS0f^O{MDIw5h!~6Mv$LLov47p_OeNQXdAqeCQrRpND{3S6rDHwS5+@VWYT_rz^zh~w zZ%?CQ>T@REbO06DT!CZ(eo6=oaF8L$M7<5YaN$BnM+ZALKR;hhCCpfnyvyUEQp-~~ zY6ku?tJ&gfcx2LYf1$pf-j)2TccRVSPJJAEFr~qcJmCU^mHF}veE}c3JoYJo z`4dRy&u@u~gi?ZGSf-lHDyD#+6XYDarm29H zSd%yhj>`fs_NjxC)3;Ek{_`86Kx_V@iI{h$N8F_w9~x}YdXl34dqrN8uI3J%>Z$h9 zo^D%%O=$1UaR2KDbu82apT>K!!_L@DSNVm+*WzKRwbAtgb7sO_JRPV$l(bZ=@sQ1E zoc_kSbLU)KTq;@>hh;I3csv^`K`9S-rX zZ|~e!My8m=^gJnBSoG1?*^W23DKvKFK|o;WmsCloTBkc-c?{qh9C*_d93fN}~i>O=X+HeY4{tlzyMW51?qcAs*}PGcDNj>6Xi| z#?5NW>DH02(k`>pjEf3+=3BL+km7`Fc;D0@?+GE(pr~8%=_dSyY~3VQXNxM zO{+G}i%4(?g(7RBJp^o+Zw8gVj3UAspG){nzL)9i%ITK!m@RUD#G;B=oitz9(PrTO zalIhpOLCv}+{rk+P}DJAki#^;ygYLM9X0KLbQ+o=vSxp%*c~LTc1VwO!I;yGF@tR> z`I6%sC7+N;BXk2PSH zLXuh>bPUv-;-JCgcC^4_D@=ACu2k25vO`n`4@Vg<`Jm!L65eejseCj!-gE2Mj=B5ZYc>XHF-^WG{Co!@ld_QzRA`=evc;yu>~m&3^wnLv!5 zqkxF7_?o57$E!50Ir_jxoV;;k=yxT_Kh~Pl9~C2HP0O$!pmX`hD$_L%ha4HE)|J6P z89|~jzH#MpS%@T@mU`f^Kk!#nUY?9c{HdFKben@++m8|>lgE#?!31^6M_xODidTZc4p2{FoJW?cgzrW>M-S2x$I9nZ@=@_GY z`%q@qyI_Q?9raH{k=I!t0SolrkxpA%%RFNB-k&oBXN8Bzl%?`dJ{^qLFu`BzrmpW) zf1&;sB4ilUfpQI^i@);j*zf6aZp{n27eC7hnoTlqgaCkPM}^%78*6_!+l+I<=$#)4 z3|Ynei3qtE#xg~%^_p1YnFj)4##^xt#-!?C51YnELpCXeS6|6tmCBS!@mK4?On7X# z?@)3_GiU&YxsnxLJc41wg|_^d6~AzDpxhN3{G3iDDDf^!?1Ka%EE>4Mq`cVRTf#0|iDy6R z`Z!rw<-p#6KTwr)9;`N}6Jc|C(==lll4HYxc~0OYCKSywRZM+D1?;O8&5H^Ho_;b?|cQrH?8XpzKt|b)xqh-oz#X8h~i^c8-S}HGHHO`5NftJQv zE}^*P796onBq%vC=}0m5dKr79j>yUn^D&n+EbXSLG_-SnqY-7Um`VvzGEaBTEr{s7 zp7JRUAEvvFSp`~OCs|-Ua5W!K0;Kk<|IU1erMvX}B?_=%NO6x@dqQO`7srhOe}al> zC8bE>^bzCWCVY$dvs3u3=|P zNJl55^N;J(aoYn!rVUbD-HbaJrn3NSC6W`I&7}o>n%CtID}&J&a#eYb-gUhA#0JN6 zk|c`+jZ~%{+CA|ZI_R>I4{}elfyn+fF94fFwOn%#YaZ5Xj>uXmAkN2M`+l`zD*R2= zRCw%}6Ab0aukF#mtMt}|LUy>ShhR0D{CKzBeRg5~UBe|D33u(W4Q6OI_=;-Y_??iw zPFd$)*YA7PXO#88a?bMsdm|y_70JcH2IQ!eYH+OJWP?=Rcy$bpK8aV$Nm33($Rpvs zmRI*q2@eMrM6O?@nUr)6Y%&W3B3Dxi?mzp{N&B^uG9TqIbt6(S8m2k=*NR9o({&q6 zfq5z~C0`x5jy$<-e3Gu<0>20_24_3!fX97gdtYFWpO!SO63LwQ?z;$Peu6W%403c_ zd|x5)OY>~^Ly-!p!ydaf;FM53H(P<7uuvqYLkIQdHz$5P{BA0Ix^VwYKijY@8htzy zUCsAUvVy4r68x6n>^$jG0udxQTlLLiBzJK<6>|RXony|>=%1atOI}S$%{+7S7gH%i zYWJ$2Phs=Zo$Z)=9Lxo2!t9f{nNdCSR`k^UlRV4}hxT4LMe~hA+32?(I(G`wXztn3 zK^D52%$r|k=9E?*#+q-a6!f~9I2qNZ6f&hjcUFoR1Gt_X2|Vj)5AHrN{^CBSprov8 zmhsMa^Zi{y#}cgl^U%aJEcCqda|KBS-<)GrDbC^5n>?~4e-HLfitWqpbxH*BU| z*_p1^%OAn$s}v-?X;88F5XTD7L76IZsW^06kEv&u*$G@9a62T|5)zoJ=NyTZW9h0+2+vKZw&j5ZPb61eGaO2n1}x|8;>o}X}(*uACvX7)Kv6q zgNafqdTO$JCgTQoQYut`$i~P7=NRUiHy!peUog=)HCXS;jCS>`r7zvf zW>~Nz1FI*Sf#0?iN^~&(h&Va+7fM;|%T^obcJ~ykT*O}55e1(X6E+~DN897$+poMR z21Vg`&v@6s-vue+t~sv>&gpR{H$Qle#9y1E8239e{Yu;~ZL`*lx0Uc?|KNH2^^#Ie zTf^&ovuWD>W4@*F-R&c<)U(PnX9ue!8U5FhLCQh>;&1^!BYY#TC!ou6tZ>UUK5^Gl zVh6WQ=1}XwY5S9z!?66ZiUmz2tm^8KdrFLcny=ltU*1c2)1pStp~cda!Dn9kkwS#5 zyDyrMMh&$_mCv6)XM!V(e%h<3ypzM4tEoi$7&~+bI8<{mtM*j4e!-5cdT;i2juZBx zb9x%q>>Y_nmK1>(qZ;K|N@oYqOU#79Z=tC5^AONBRH0(0t*x3q(@ zt>uGkdeWe8$-~%&7ehA|koIAW!dJV|$znOeKM>pMj2#VTNd?ES4%FkVeUq;l!%_Wy zM2tZudQZ1`lO{MGn!BhQ?y@q9Xm(T_H(LH*T1{&Ssq3Eb z!WxLMjm0WnDRgC}&Wbc(|Fn-mVw)v*qYtFDqUwywrl6fG(HbYR^t1FE%rUGpT>jZ* zJNQfAVE-i%Asc5I85w8i^3i~-7l!w#dFa zrFX9GR^4Rzp18O;pj5T}YOxsKi%yO#9nJLbd~$?G5VUGG(r{GHV&?IFq9h4#UoS44+D(Kd6$n!KEZ6i(sdV0JDj6gt zy{iuqZqN4rZ6%a?L~6B~VPhOo%?5qeOW`TK=}Lo371{{D5!C)&A9Rae*C zrV7ZJZN=y~hji+L?CQ#_<$85q=C)u4P{K_S@C#HjWdcz}`x zbTke_tiW{Ixb*b6d0Cq~X*QSI)Ya&eo*n7d?Mrm1c2Z~5L$avH5@CxXP1?eeLaF)r zR}pg9d%6`T;d|C)_ zw!e^k!)hF3y+Z*vfD7EFz7AVk937<)`vqhDBGMW>emj=Fj=Qr{&j?A#o|!4MG&PY= z{@H}?a*O-@yIMpf`Q)JaM0Qk1tbXApcHlVl(un+};PbWS)9sF8qH*FF6Swy)6X8^p zb~te%(t|=DiVL3Gdi(* zw^^xdn6s#_85x>I4%CvE~4ScKTm?a%uq{n-de1-y}Izzwfy zN-rcBdbw2wSB5+J>jGap)H)N^m2L+<#LmQVt8bJ#Q`6thBZ=Ry)2wz<^LK zqcoYbU#@wg>=Z9xs;M*|6Z6w+@(cZX$SZQiB}Q8U=#Tp1FVhUj9+x0`Z{ORo|(J@&s$j z%Zps+!raizQ2I2lR54-p(P%S5#yTrxkl;+axYegw&B^6Xdp#WS;#NF|%jlmFMZJNP z-}H3ile>3sD6i!|&Zeiu)tZalxPX)T^Cd!x3FX?eB$>cSEFfPUVVoZ<6@##No0LUY zO+6f3isFjE`1;0iuyIpWKfN@7U0>IQ7$`l;oFx-U-m%!YJFW~U_~fiRnyC_^Ms&y# z(sqtk4-^|py3MO+DvYu3_~8b^-bFToSn-M6ka7l7?J!bUAXt;d`FK;zvo1t^Bnrxy zPfo|;xiwI10BDFvJszrHCQ2$lzV`+GC-+}$*AwA2-%MMbp>Pw{_j_~F&6?zGc;0vM z$Uboq*>}B-K7%CFe=0J5%%o#z9Xi_H6cXS-izAdPXnm(48F>Vv;=)yzz@$qIB@P{^ zu%|E18bl}zzbChn^1t~$XR*G?Y!E$_r_ae$Z<_`UZf;ZY2h{4-X6}!M%`sOfyFBXF zRc-&2`{pu}-0OHZ4Xwp8%)R4rj}QErJdf2_{rconN9s{4bGZSW8^Y{B z>|^tM1FZ=H+6L{*)8+AEfF8S^I?+Rulb@6bRlFhHs&XFJ*6C($h5N44{S9TI{vbHI z@JZ99dM{>*KAjF!ripU*X9S5{A^j+Uoa3IZaK%*1;`+{D&RO1u7&BqwZ1%!=bn=-# z`5NG*`ged0)g2yHdELY&(AI-v&1r|5aFnpYi30~x)78u?B`jp~=N)cVFkM8m=+cj$us8%RiQD-i~=8(Qco(uiM4Deyei z^A{2z+0FU%iA`MGfD9I^Qr%PYP_wQ>ZM)q#RqnMcm9$De_tb^ zHlmiePaJ?c>1rcU+dzoAx+365p2tJ={Xa^`HHyM%hWItqa?Oz}0*nkt{Q~fn&n>Ya z*?d|&2$+!AtF=1J$S~PH+XS#S@&Qc%C;RL0xf^U5eKctF1B}i#VB|inml^sdX;c$o zMgovZJj!s;;f>CxI&6Wr6v)zu)9x&C-R9N1_fpdc&NW6l8D`5BQ;V48*rz(We5Km? z+L;v`nqog$kYSpA3yGw0Lclo8`|9U8N9(w4^K|Btvdmx8IF9ssNHV+~o9uIw=n45e z!>(@*MNiQd-|k#;>#|F2=G&^KNV+=BRHE0H=!G364OX@MNA6}B04XU?67CF8PPDva*27RC0Rpq5vsPFcCeX&L8~YYX3KCAuQ=w=3?e`PnQ*zk%!uHD3%KPr zEpqJmGzz4P2*Tlp1$v0G^@Ki`M|T&2X#V@^j3r|&?||%d1uUre1g&UkBmej`&SvwU zlM)eJluG!w4awK}4l>%?Et6cEOn}96u>-J^Db1UM|1<7R{Npy2Ms6Xd#`tNBNb-hF z$&B`VfB@Rx9g8+2zFlCiP6kNd2oF-`oF0wqBNbi4x6(VLr2A$PgRoLZe(4L73XRV| ze};Mh>|1BrLDZYFOKT8Y#y0#sPf02pBn~nX5y(+S(}w7{?MElaFCh>KV>hiyP{^2G1bfSVnwHJ!gJ7jwi8JL zM)e?ODYzLUz9p=-=c8>-SSJrYpShHyR@-V8-v>5VrV*dKQ>LT+MDyw=DZlmU?}>@% z;%Wl}*QZ-#$^YoiOpS4E?AD|ibekCF2f@t9MLhc_gqE4;;%$W!&l|KWsdWJ${qjiQ zD5srWSaG$xJC_TMyKf=}7;|Nmnb%|-ZW@%5F7co~c8R!~uKk-?%PlQpWF z%qRtTMt?`*{u13w+QR*n#|>BA*;6qw;S?xYjRvg;P^A~s+uP=YZGGrf7;;I z^(i(WjVOiA9hGU5z%bqifZ>XoYp9L-v3rqb-}C*BycH1=`sk@qWGzUL0x3;B7IpIf z2!P-Ta8FU9fKAUa+n4Eri_2JvCnilQh^Td;yd3nsvJcsX=w~Q5gyQpRR1&{_Qwz*xi#u40V z5n}=fYq3bj$Hw^NCbIz)hfE^5RVFBf#c!mZtB(a3;m38MRKZ!`fF-`skPi6u0dlvQ z_#b7=IUAzD!)@3$mlan{?b2)8+@S)FFHIg>lcJePN|+NHxj%*#TAPWf&FPwwh^r=BnG&hwTjWk*d2fHII0xtnKd<+Z>{=(M`4kafw;uUbpYf@|p&PCKr-}~+X z$tEN()6|8kKRG@a0XcX?`W>#fQ_ue67j9&d05T>Tic5M$Yolil26X_Ta1KYTiHW%! z8lc?GDiZkfJb*5>LzcMiG%03k1yIl+K^cDZUr0Na^OaW;_>3!aB%c;p3IfC4eQ-qL zn^#490J=Aqh{!ZE!UKrQR(A+W`k+1!D(}Z6lHuO{UvUB+^(N)X$2pJrjri*2e*nz1 z2MT))RZ{^s!4)@N-gZzR6g_=2c-{p(Lm^yqVv>>e?aQds1XoMEI-$*fp&he2x3t)r^DlMg+V*|4w3?Q+{7cV=d1 z@hReuTM8+!d;Sh$GOr^W7ifND-L@|YcrvxKTN6p`S$)!;m( zVEbcThp+Sno;cvodXUn;Q6$3EFt#V8B*Y4e@$z}!#Y2O#?-U{hN?(}alA7DV7`kqdV$N$6bUq_SrQ2dv&XP`Fm`6EOq#s7L`0J^OO zxZ2=L01NM~n4&)+&^kk9eG1%cCoKHv;>b^HCZ zq(+ZBbEMfIM&7HXcPPN(o1W9MQ$^M`Lh)m_A->P8A}LQ)F#u0#qhFB+(p|(>eC-S; zpd8G9-C@Q4zCrCLXrblM@!||-=LaLnIsZt^r3~fkw**aa|Dr zuCt^h=hrFfv+p@;_#<^8vmVrVRYOilgFT_*0W0fkCtY0zQh!#!dXQ|!gK5`fKmO}@ zQFl0sij(PsS?|+;?(5+03J{@gyHA`h=dUj+UQZm|i2jjgz$D2#x1(%J>iRqP<51bp z^Q109Dgn*Alc|-J!QgsK^oQPsj4+7e`#P)Ew_L?0wHAuoA7uvJ=6l&TDN78iYfPJe zt=q0A+6~-(w|4h$cfEjrVBv83q2-T*RmRSdoSgOUWK$L~#PdQ%()Fc0r{~lHe1^^h ztN6J1(x{hS%Y+T)7vlsgk|t&q-SxrjK>(Wo=aXz}$>ZH`r_Styt3N7g4NjZ$Bccf3CjLJca`=r+ zsuATXn;-NE9!^(D2l`7GdGyD>2Roc(WPaS&<5uNJn^gSweh(S-+XUq@LYD{0l?4t! zz70G7bvC5+xfl$kzdcUw=ItjpsxFh~7IYA??I^+11zNGGork(jGj zzrgcXN;>CI*#*+l9m=fD8)$1f-bn3GIJVozK%ni>s01*J$HBa*q%ssC7CRk ucHY;o3bz Date: Tue, 12 Sep 2017 22:13:27 +1000 Subject: [PATCH 02/13] fix typos and spelling mistakes --- .../binary-compatibility-for-library-authors.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md index 58a78a0eaa..1a95fd5fc7 100644 --- a/_overviews/tutorials/binary-compatibility-for-library-authors.md +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -51,7 +51,7 @@ When talking about two versions being compatible, the direction matters too. If if we say that any library release of `v1.x.x` will be forwards compatible, we can use `v1.0.0` anywhere where `v1.1.0` was originally used. For the rest of the guide, when the 'compatible' is used we mean backwards compatible, as it is the more common case of compatibility guarantee. -An important note to make is that while breaking source compatibility normally results in breaking binary compatibility, they are actually orthorgonal +An important note to make is that while breaking source compatibility normally results in breaking binary compatibility, they are actually orthogonal (breaking one does not imply breaking the other). See below for more examples (TODO: make sure we have examples?) TODO: more facts? @@ -69,7 +69,7 @@ Unfortunately at 2am, we got frantic calls from customers saying that our App is ![Binary incompatibility after upgrading]({{ site.baseurl }}/resources/images/library-author-guide/after_update.png){: style="width: 280px; margin: auto; display: block;"} -Why did we get a `NoSuchMethodError`? Remember that `A v1.0.0` is compiled with `C v1.0.0` and thus calls methods availble in `Cv1.0.0`. While `B` and +Why did we get a `NoSuchMethodError`? Remember that `A v1.0.0` is compiled with `C v1.0.0` and thus calls methods available in `Cv1.0.0`. While `B` and our App has been recompiled with available classes/methods in `C v2.0.0`, `A v1.0.0`'s bytecode hasn't changed - it still calls the same method that is now missing in `C v2.0.0`! This situation can only be resolved by ensuring that the chosen version of `C` is binary compatible with all other evicted versions of `C`. In this case, we need a new version of `A` that depends @@ -98,14 +98,14 @@ to help spot binary incompatibility between library releases. (Follow the link f TODO -## Versioning Scheme - Communicate binary compatiblity breakages +## Versioning Scheme - Communicate binary compatibility breakages We recommend using the following schemes to communicate binary and source compatibility to your users: * Any release with the same major version are **Binary Backwards Compatible** with each other * A minor version bump signals new features and **may contain minor source incompatibilities** that can be easily fixed by the end user -* Patch version for bugfixes and minor behavioural changes -* For **expreimental library versions** (where the major version is `0`, such as `v0.1.0`), a minor version bump **may contain both source and binary breakages** +* Patch version for bug fixes and minor behavioral changes +* For **experimental library versions** (where the major version is `0`, such as `v0.1.0`), a minor version bump **may contain both source and binary breakages** * Some libraries may take a harder stance on maintaining source compatibility, bumping the major version number for ANY source incompatibility even if they are binary compatible Some examples: @@ -123,7 +123,7 @@ Many libraries in the Scala ecosystem has adopted this versioning scheme. A few Why do we use the major version number to signal binary compatibility releases? -From our [example](#why-binary-compatibility-matters) above, we have learnt two important lessons: +From our [example](#why-binary-compatibility-matters) above, we have learned two important lessons: * Binary incompatibility releases often leads to dependency hell, rendering your users unable to update any of their libraries without breaking their application * If a new library version is binary compatible but source incompatible, the user can simply fix the compile errors in their application and everything will work From dbe3fbcb4ddd1c7feb401d24cf4a93a40ee52861 Mon Sep 17 00:00:00 2001 From: Jacob Wang Date: Tue, 12 Sep 2017 22:23:21 +1000 Subject: [PATCH 03/13] various rewordings and grammar fixes --- .../binary-compatibility-for-library-authors.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md index 1a95fd5fc7..661fb4c9ec 100644 --- a/_overviews/tutorials/binary-compatibility-for-library-authors.md +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -41,15 +41,16 @@ Therefore when deciding which JARs to use for compilation, SBT only selects one and all other versions of the same library are **evicted**. When packaging applications, the same versions of libraries that was used for compiling the application is packaged and used during runtime. -Two library versions are said to be **Source Compatible** if switching one for the other does not incur any compile errors. For example, If we can switch from `v1.0.0` of a dependency to `v2.0.0` and -recompile our code without causing any compilation errors, `v2.0.0` is said to be source compatible with `v1.0.0`. +Two library versions are said to be **Source Compatible** if switching one for the other does not incur any compile errors. For example, If we can switch from `v1.0.0` of a dependency to `v1.1.0` and +recompile our code without causing any compilation errors, `v1.1.0` is said to be source compatible with `v1.0.0`. Two library versions are said to be **Binary Compatible** if the compiled bytecode of these versions are compatible. Using the example above, removing a class will render two version binary incompatible too, as the compiled bytecode for v2.0.0 will no longer contain the removed class. When talking about two versions being compatible, the direction matters too. If we can use `v2.0.0` in place of `v1.0.0`, `v2.0.0` is said to be **backwards compatible** with `v1.0.0`. Conversely, if we say that any library release of `v1.x.x` will be forwards compatible, we can use `v1.0.0` anywhere where `v1.1.0` was originally used. -For the rest of the guide, when the 'compatible' is used we mean backwards compatible, as it is the more common case of compatibility guarantee. + +(For the rest of this guide, when you see the word "compatible" assume backwards compatibility, as it is the more common case of compatibility guarantee) An important note to make is that while breaking source compatibility normally results in breaking binary compatibility, they are actually orthogonal (breaking one does not imply breaking the other). See below for more examples (TODO: make sure we have examples?) @@ -63,7 +64,7 @@ Our application depends on library `A` and `B`. Both `A` and `B` depends on libr ![Initial dependency graph]({{ site.baseurl }}/resources/images/library-author-guide/before_update.png){: style="width: 280px; margin: auto; display: block;"} -Some time later, we see `B v1.1.0` is now available and we upgraded the version in our `build.sbt`. Our code compiles and seems to work so we push it to production and goes home for dinner. +Some time later, we see `B v1.1.0` is available and upgrade its version in our build.sbt. Our code compiles and seems to work so we push it to production and go home for dinner. Unfortunately at 2am, we got frantic calls from customers saying that our App is broken! Looking at the logs, you find lots of `NoSuchMethodError` is being thrown by some code in `A`! @@ -77,7 +78,7 @@ on `C v2.0.0` (or any other future `C` version that is binary compatible with `C Now imagine if our App is more complex with lots of dependencies themselves depending on `C` (either directly or transitively) - it becomes extremely difficult to upgrade any dependencies because it now pulls in a version of `C` that is incompatible with the rest of the versions of `C` in our dependency tree! In the example below, we cannot upgrade `D` because it will transitively pull in `C v2.0.0`, causing breakages -due to binary incompatibility. This inability to upgrade any packages without breaking anything is common known as **Dependency Hell**. +due to binary incompatibility. This inability to upgrade any packages without breaking anything is commonly known as **Dependency Hell**. ![Dependency Hell]({{ site.baseurl }}/resources/images/library-author-guide/dependency_hell.png) From 61695cf4a346f12f5baa326de0d198c6de22222a Mon Sep 17 00:00:00 2001 From: Jacob Wang Date: Wed, 27 Sep 2017 23:14:50 +1000 Subject: [PATCH 04/13] Added "Evolving code without breaking binary compatibility" section + minor updates/rewording --- ...inary-compatibility-for-library-authors.md | 37 ++++++++++++++----- 1 file changed, 28 insertions(+), 9 deletions(-) diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md index 661fb4c9ec..c08f29c3c7 100644 --- a/_overviews/tutorials/binary-compatibility-for-library-authors.md +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -52,9 +52,8 @@ if we say that any library release of `v1.x.x` will be forwards compatible, we c (For the rest of this guide, when you see the word "compatible" assume backwards compatibility, as it is the more common case of compatibility guarantee) -An important note to make is that while breaking source compatibility normally results in breaking binary compatibility, they are actually orthogonal -(breaking one does not imply breaking the other). See below for more examples (TODO: make sure we have examples?) -TODO: more facts? +An important note to make is that while source compatibility breakages normally results in binary compatibility breakages as well, they are actually orthogonal +(breaking one does not imply breaking the other). ## Why binary compatibility matters @@ -88,19 +87,39 @@ How can we, as library authors, spare our users of runtime errors and dependency * **Avoid breaking binary compatibility** through careful design and evolution of your library interfaces * Communicate binary compatibility breakages clearly through **versioning** -## MiMa - Check Binary Compatibility with Previous Library Versions +## MiMa - Check binary compatibility with previous library versions -The [Migration Manager for Scala](https://github.com/typesafehub/migration-manager) (MiMa) is a tool for diagnosing binary incompatibilities between different library versions. +The [Migration Manager for Scala](https://github.com/typesafehub/migration-manager) (MiMa) is a tool for diagnosing binary incompatibilities between different library versions. +It works by comparing the class files of two provided JARs and report any binary incompatibilities found. -When run standalone in the command line, it will compare the .class files in the two provided JARs and report any binary incompatibilities found. Most library authors use the [SBT plugin](https://github.com/typesafehub/migration-manager/wiki/Sbt-plugin) -to help spot binary incompatibility between library releases. (Follow the link for instructions on how to use it in your project) +By incorporating [MiMa SBT plugin](https://github.com/typesafehub/migration-manager/wiki/Sbt-plugin) into your SBT build, you can easily check whether +you have accidentally introduced binary incompatible changes. Detailed instruction on how to use the SBT plugin can be found in the link. -## Designing for Evolution - without breaking binary compatibility +We strongly encourage every library author to incorporate MiMa into their library release workflow. -TODO +## Evolving code without breaking binary compatibility + +Binary compatibility breakages can often be avoided through careful use of certain Scala features as well as some techniques you can apply when modifying code. + +Some language features may break binary compatibility: + +* Default parameter values for methods or classes +* Case classes +* Default methods on traits (doesn't cause breakages since 2.12) + +Techniques you can use to avoid breaking binary compatibility: + +* Annotate public methods's return type explicitly +* Mark methods as package private when you want to remove a method or modify its signature +* Don't use inlining (for libraries) + +For brevity of this guide, detailed explanation and runnable code examples can be found in [Binary Compatibility Code Examples & Explanation](https://github.com/jatcwang/binary-compatibility-guide). + +Again, we recommend using MiMa to double check that you have not broken binary compatibility after making changes. ## Versioning Scheme - Communicate binary compatibility breakages +### Recommmended Versioning Scheme We recommend using the following schemes to communicate binary and source compatibility to your users: * Any release with the same major version are **Binary Backwards Compatible** with each other From 8aeac02f1c7b79638ba6b4ab53c85ec6e9a280c7 Mon Sep 17 00:00:00 2001 From: Jacob Wang Date: Sun, 1 Oct 2017 23:13:05 +1000 Subject: [PATCH 05/13] Addressing feedback from jvican --- ...inary-compatibility-for-library-authors.md | 143 ++++++++++++------ .../backwards_forwards_compatibility.plantuml | 10 ++ .../fowards_backwards_compatibility.png | Bin 0 -> 3275 bytes 3 files changed, 107 insertions(+), 46 deletions(-) create mode 100644 resources/images/library-author-guide/backwards_forwards_compatibility.plantuml create mode 100644 resources/images/library-author-guide/fowards_backwards_compatibility.png diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md index c08f29c3c7..1a1c8bbdf8 100644 --- a/_overviews/tutorials/binary-compatibility-for-library-authors.md +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -8,75 +8,115 @@ permalink: /tutorials/:title.html ## Introduction -A diverse and comprehensive set of libraries is important to any productive software ecosystem. While it is easy to develop and distribute Scala libraries, good library authorship goes far -beyond just writing code and publishing them. +A diverse and comprehensive set of libraries is important to any productive software ecosystem. While it is easy to develop and distribute Scala libraries, good library authorship goes +beyond just writing code and publishing it. In this guide, we will cover the important topic of **Binary Compatibility**: * How binary incompatibility can cause production failures in your applications -* How library authors can avoid breaking binary compatibility, and/or convey breakages clearly to library users when they happen +* How to avoid breaking binary compatibility +* How to reason about and communicate the impact of their code changes -Before we start, first we need to understand how code is compiled and executed on the Java Virtual Machine (JVM). +Before we start, let's understand how code is compiled and executed on the Java Virtual Machine (JVM). ## The JVM execution model -Code compiled to run on the JVM is compiled to a platform-independent format called **JVM bytecode** and stored in **Class File** format (with `.class` extension) and these class files are stored -in JAR files. The bytecode is what we refer to as the **Binary** format. +Scala is compiled to a platform-independent format called **JVM bytecode** and stored in `.class` files. These class files are collated in JAR files for distribution. -When application or library code is compiled, their bytecode invokes named references of classes/methods from their dependencies instead of including the dependencies' actual bytecode -(unless inlining is explicitly requested). During runtime, the JVM classloader will search through the provided class files for classes/methods referenced by and invoke them. +When some code depends on a library, its compiled bytecode references the library's bytecode. The library's bytecode is referenced by its class/method signatures and loaded lazily +by the the JVM classloader during runtime. If a class or method matching the signature is not found, an exception is thrown. -Let's illustrate with an example: +As a result of this execution model: -We got an application `App` that depends on `A` which itself depends on library `C`. When starting the application we need to provide the class files -for all of `App`, `A` and `C` (something like `java -cp App.jar:A.jar:C.jar:. MainClass`). If we did not provide `C.jar`, or if we provided a `C.jar` that does not contain certain classes/methods -which `A` expected to exist in library `C`, we will get an exception when our code attempt to invoke the missing classes/methods. +* We need to provide the JARs of every library used in our dependency tree when starting an application, since the library's bytecode is only referenced -- not merged into its user's bytecode +* A missing class/method problem may only surface after the application has been running for a while, due to lazy loading. -This is what we call **Binary Incompatibility Errors** - The bytecode interface used for compilation differs and is incompatible with the bytecode provided during runtime. +Common exceptions from classloading failures includes +`InvocationTargetException`, `ClassNotFoundException`, `MethodNotFoundException`, and `AbstractMethodError`. + +Let's illustrate this with an example: + +Consider an application `App` that depends on `A` which itself depends on library `C`. When starting the application we need to provide the class files +for all of `App`, `A` and `C` (something like `java -cp App.jar:A.jar:C.jar:. MainClass`). If we did not provide `C.jar` or if we provided a `C.jar` that does not contain some classes/methods +which `A` calls, we will get classloading exceptions when our code attempts to invoke the missing classes/methods. + +These are what we call **Binary Incompatibility Errors**. An error caused by binary incompatibility happens when the compiled bytecode references a name that cannot be resolved during runtime ## What are Evictions, Source Compatibility and Binary Compatibility? -Since the classloader only loads the first match of a class, having multiple versions of the same library in the classpath is redundant. -Therefore when deciding which JARs to use for compilation, SBT only selects one version from each library (by default the highest), -and all other versions of the same library are **evicted**. When packaging applications, the same versions of libraries that was used for compiling the -application is packaged and used during runtime. +### Evictions +When a class is needed during execution, the JVM classloader loads the first matching class file from the classpath (any other matching class files are ignored). +Because of this, having multiple versions of the same library in the classpath is generally undesireable: + +* Unnecessary application size increase +* Unexpected runtime behaviour if the order of class files changes + +Therefore, when resolving JARs to use for compilation and packaging, most build tools will pick only one version of each library and **evict** the rest. + +### Source Compatibility +Two library versions are **Source Compatible** if switching one for the other does not incur any compile errors. +For example, If we can upgrade `v1.0.0` of a dependency to `v1.1.0` and recompile our code without any compilation errors, `v1.1.0` is source compatible with `v1.0.0`. + +### Binary Compatibility +Two library versions are **Binary Compatible** if the compiled bytecode of these versions can be interchanged without causing binary compatibility errors. +For example, if we can replace the class files of a library's `v1.0.0` with the class files of `v1.1.0` without any binary compatibility errors during runtime, +`v1.1.0` is binary compatible with `v1.0.0`. + +**NOTE:** While breaking source compatibility normally results in binary compatibility breakages as well, they are actually orthogonal -- breaking one does not imply breaking the other. + +### Forwards and Backwards Compatibility + +There are two "directions" when we describe compatibility of a library release: -Two library versions are said to be **Source Compatible** if switching one for the other does not incur any compile errors. For example, If we can switch from `v1.0.0` of a dependency to `v1.1.0` and -recompile our code without causing any compilation errors, `v1.1.0` is said to be source compatible with `v1.0.0`. +**Backwards Compatible** means that a newer library version can be used in an environment where an older version is expected. When talking about binary and source compatibility, +this is the common and implied direction. -Two library versions are said to be **Binary Compatible** if the compiled bytecode of these versions are compatible. Using the example above, removing a class will render two version -binary incompatible too, as the compiled bytecode for v2.0.0 will no longer contain the removed class. +**Forwards Compatible** means that an older library can be used in an environment where a newer version is expected. +Forward compatibility is generally not upheld for userland libraries. It is only important in situations where an older version of a library is commonly +used at runtime against code that is compiled with newer version. (e.g. Scala's standard library) -When talking about two versions being compatible, the direction matters too. If we can use `v2.0.0` in place of `v1.0.0`, `v2.0.0` is said to be **backwards compatible** with `v1.0.0`. Conversely, -if we say that any library release of `v1.x.x` will be forwards compatible, we can use `v1.0.0` anywhere where `v1.1.0` was originally used. +Let's look at an example where library `A v1.0.0` is compiled with library `C v1.1.0`. -(For the rest of this guide, when you see the word "compatible" assume backwards compatibility, as it is the more common case of compatibility guarantee) +![Forwards and Backwards Compatibility]({{ site.baseurl }}/resources/images/library-author-guide/fowards_backwards_compatibility.png){: style="width: 65%; margin: auto; display: block"} -An important note to make is that while source compatibility breakages normally results in binary compatibility breakages as well, they are actually orthogonal -(breaking one does not imply breaking the other). +`C v1.1.0 ` is **Forwards Binary Compatible** with `v1.0.0` if we can use `v1.0.0`'s JAR at runtime instead of `v1.1.0`'s JAR without any binary compatibility errors. + +`C v1.2.0 ` is **Backwards Binary Compatible** with `v1.1.0` if we can use `v1.2.0`'s JAR at runtime instaed of `v1.1.0`'s JAR without any binary compatibility errors.. ## Why binary compatibility matters -Let's look at an example where binary incompatibility between versions of a library can have catastrophic consequences: +Binary Compatibility matters because failing to maintain it makes life hard for everyone. + +* End users has to update all library versions in their whole transitive dependency tree such that they are binary compatible, otherwise binary compatibility errors will happen at runtime +* Library authors are forced to update the dependencies of their library so users can continue using them, greatly increases the effort required to maintain libraries + +Constant binary compatibility breakages in libraries, especially ones that are used by other libraries, is detrimental to our ecosystem as they require a lot of effort +from users and library authors to resolve. + +Let's look at an example where binary incompatibility can cause grief and frustration: + +### An example of "Dependency Hell" -Our application depends on library `A` and `B`. Both `A` and `B` depends on library `C`. Initially both `A` and `B` depends on `C v1.0.0`. +Our application `App` depends on library `A` and `B`. Both `A` and `B` depends on library `C`. Initially both `A` and `B` depends on `C v1.0.0`. -![Initial dependency graph]({{ site.baseurl }}/resources/images/library-author-guide/before_update.png){: style="width: 280px; margin: auto; display: block;"} +![Initial dependency graph]({{ site.baseurl }}/resources/images/library-author-guide/before_update.png){: style="width: 50%; margin: auto; display: block;"} -Some time later, we see `B v1.1.0` is available and upgrade its version in our build.sbt. Our code compiles and seems to work so we push it to production and go home for dinner. +Some time later, we see `B v1.1.0` is available and upgrade its version in our build. Our code compiles and seems to work so we push it to production and go home for dinner. -Unfortunately at 2am, we got frantic calls from customers saying that our App is broken! Looking at the logs, you find lots of `NoSuchMethodError` is being thrown by some code in `A`! +Unfortunately at 2am, we got frantic calls from customers saying that our application is broken! Looking at the logs, you find lots of `NoSuchMethodError` is being thrown by some code in `A`! -![Binary incompatibility after upgrading]({{ site.baseurl }}/resources/images/library-author-guide/after_update.png){: style="width: 280px; margin: auto; display: block;"} +![Binary incompatibility after upgrading]({{ site.baseurl }}/resources/images/library-author-guide/after_update.png){: style="width: 50%; margin: auto; display: block;"} -Why did we get a `NoSuchMethodError`? Remember that `A v1.0.0` is compiled with `C v1.0.0` and thus calls methods available in `Cv1.0.0`. While `B` and -our App has been recompiled with available classes/methods in `C v2.0.0`, `A v1.0.0`'s bytecode hasn't changed - it still calls the same method that is now missing in `C v2.0.0`! +Why did we get a `NoSuchMethodError`? Remember that `A v1.0.0` is compiled with `C v1.0.0` and thus calls methods available in `C v1.0.0`. +While `B v1.1.0` and `App` has been recompiled with `C v2.0.0`, `A v1.0.0`'s bytecode hasn't changed - it still calls the method that is now missing in `C v2.0.0`! -This situation can only be resolved by ensuring that the chosen version of `C` is binary compatible with all other evicted versions of `C`. In this case, we need a new version of `A` that depends +This situation can only be resolved by ensuring that the chosen version of `C` is binary compatible with all other evicted versions of `C` in your dependency tree. In this case, we need a new version of `A` that depends on `C v2.0.0` (or any other future `C` version that is binary compatible with `C v2.0.0`). -Now imagine if our App is more complex with lots of dependencies themselves depending on `C` (either directly or transitively) - it becomes extremely difficult to upgrade any dependencies because it now -pulls in a version of `C` that is incompatible with the rest of the versions of `C` in our dependency tree! In the example below, we cannot upgrade `D` because it will transitively pull in `C v2.0.0`, causing breakages +Now imagine if `App` is more complex with lots of dependencies themselves depending on `C` (either directly or transitively) - it becomes extremely difficult to upgrade any dependencies because it now +pulls in a version of `C` that is incompatible with the rest of `C` versions in our dependency tree! + +In the example below, we cannot upgrade to `D v1.1.1` because it will transitively pull in `C v2.0.0`, causing breakages due to binary incompatibility. This inability to upgrade any packages without breaking anything is commonly known as **Dependency Hell**. ![Dependency Hell]({{ site.baseurl }}/resources/images/library-author-guide/dependency_hell.png) @@ -87,7 +127,7 @@ How can we, as library authors, spare our users of runtime errors and dependency * **Avoid breaking binary compatibility** through careful design and evolution of your library interfaces * Communicate binary compatibility breakages clearly through **versioning** -## MiMa - Check binary compatibility with previous library versions +## MiMa - Checking binary compatibility against previous library versions The [Migration Manager for Scala](https://github.com/typesafehub/migration-manager) (MiMa) is a tool for diagnosing binary incompatibilities between different library versions. It works by comparing the class files of two provided JARs and report any binary incompatibilities found. @@ -117,15 +157,21 @@ For brevity of this guide, detailed explanation and runnable code examples can b Again, we recommend using MiMa to double check that you have not broken binary compatibility after making changes. -## Versioning Scheme - Communicate binary compatibility breakages +## Versioning Scheme - Communicating compatibility breakages + +Library authors use verioning schemes to communicate compatibility guarantees between library releases to their users. Versioning schemes like [Semantic Versioning](http://semver.org/)(SemVer) allow +users to easily reason about the impact of a updating a library, without needing to read the detailed release note. + +In the following section we will outline a versioning scheme based on Semantic Versioning that we **strongly encourage** you to adopt for your libraries. The rules listed below are **in addition** to +Semantic Versioning v2.0.0. ### Recommmended Versioning Scheme -We recommend using the following schemes to communicate binary and source compatibility to your users: -* Any release with the same major version are **Binary Backwards Compatible** with each other -* A minor version bump signals new features and **may contain minor source incompatibilities** that can be easily fixed by the end user -* Patch version for bug fixes and minor behavioral changes -* For **experimental library versions** (where the major version is `0`, such as `v0.1.0`), a minor version bump **may contain both source and binary breakages** +* If backwards **binary compatibility** is broken, **major version number** must be increased +* If backwards **source compatibility** is broken, **minor version number** must be increased +* A change in **patch version number** signals **no binary nor source incompatibility**. According to SemVer, patch versions should contain only bug fixes that fixes incorrect behavior so major behavioral +change in method/classes should result in a minor version bump. +* When major version is `0`, a minor version bump **may contain both source and binary breakages** * Some libraries may take a harder stance on maintaining source compatibility, bumping the major version number for ANY source incompatibility even if they are binary compatible Some examples: @@ -139,9 +185,14 @@ Some examples: Many libraries in the Scala ecosystem has adopted this versioning scheme. A few examples are [Akka](http://doc.akka.io/docs/akka/2.5/scala/common/binary-compatibility-rules.html), [Cats](https://github.com/typelevel/cats#binary-compatibility-and-versioning) and [Scala.js](https://www.scala-js.org/). +If this version scheme is followed, reasoning about binary compatibility is now very simple: + +* Ensure major versions of the all versions of a library in the dependency tree are the same +* Pick latest version and evict the rest (This is the default behavior of SBT). + ### Explanation -Why do we use the major version number to signal binary compatibility releases? +Why do we use the major version number to signal binary incompatible releases? From our [example](#why-binary-compatibility-matters) above, we have learned two important lessons: @@ -156,5 +207,5 @@ is also important, if they are minor breakages that does not require effort to f In this guide we covered the importance of binary compatibility and showed you a few tricks to avoid breaking binary compatibility. Finally, we laid out a versioning scheme to communicate binary compatibility breakages clearly to your users. -If we follow these guidelines, we as a community can spend less time untangling dependency hell and more time making cool things! +If we follow these guidelines, we as a community can spend less time untangling dependency hell and more time building cool things! diff --git a/resources/images/library-author-guide/backwards_forwards_compatibility.plantuml b/resources/images/library-author-guide/backwards_forwards_compatibility.plantuml new file mode 100644 index 0000000000..23ee43c891 --- /dev/null +++ b/resources/images/library-author-guide/backwards_forwards_compatibility.plantuml @@ -0,0 +1,10 @@ +top to bottom direction + +component "A v1.0.0" as a +component "C v1.1.0" as c1 +component "C v1.0.0" as c0 +component "C v1.2.0" as c2 + +a -down-> c1 +c0 -[hidden]> c1 +c1 -[hidden]> c2 diff --git a/resources/images/library-author-guide/fowards_backwards_compatibility.png b/resources/images/library-author-guide/fowards_backwards_compatibility.png new file mode 100644 index 0000000000000000000000000000000000000000..908edc54a03983adc6cc95651b32e68ad4808c8c GIT binary patch literal 3275 zcmcInc{p3!7S|bV)o@iQRnO3BinN59YYeSWHKajkkPve@RD!ED_3}suQxP?lUPW3% zN@A*n_O*tDND)J;lc|v^(Tbrj?S1!t_j~`mzuvd^cg{M$-`;Dleb!ogos)jS5xQSo zL0m*cWWT+g^+gepZAL;lPjtKR+Lg{aA-rhWe!X z2mldAIyw;{s4yTpG(_7kJX8v7s3_E8iFdsO{E3T*#1hdrEu3g)B~y7tpOrO;ZP@Mo4U3ge%zOnYmvH>9v7I(NHE)pNNzjZE0HeqvKwMtg5D^E4TKDt=DhjvQLepp z>PGU{t-58e>E~EgZT&?|8QA>Q}aLXk<(7|vKqM@9wPoIm;#m|EhiOd2kISV;) zyZoR&TSk#&5Np(d@&$q@)<=Fn<=N{_^Gq(*z80nsT5+noW?9;OG${Bn5It&?j+bgX zlmW7WE^)KImWuhJktgiRr}XAzc5A=wn<>ez&d<<{tINr~f91eZnTFeL$)J#}(F(yG zn8l(eCi2FP(a@4M!@cT@hbdRberc$@Qt+v({_f_E-R0TCqickE5s|$I?5!#U7 z#K4@HZO_BEtRG^qgXs%K=NFO!Bi=OCo7?t9__n%jR2;SN(ySbRI=pqNn#TO4a8|GL zUn$Wkmw%mAr}PkszCBl98U2G^g^G3s8aX|uPiI2G`Dg51>etic2n+Iz9_4tnvWoL!$Yy9c)Z>LS9sthXsTs|IMOQ<(`dRm$Zc{h@mnJPas#{o)f%gQXC%^#{k zEHRmq!dbdo{_H~&11vH(A{7V?30a9XF?t$g)@OHCXb~LP#X3fJd+q6&aqxj-+Mp~F z5o8HJ3kM0e*fPeJ)7k;iV#!4Y5uR(GG2~fFP|ItUozR1_Oo&T#U0s5?!PV*`%It#0 z#k8cP`261776cUfVV=$CD>2v633rQ&>p59e2BKofq$2&gV5}h(MK6T1$-f6yWWs8L zot&0m=Jw*_oN7qVWkL5&&sRgN;6Q&6vU8)M{qbkHv?e$y_HBoeehgyIRCV>#>_K)^ z%u0hG`fB z74uilRe-2TO4N>}XYM0k(~D*m(62k*HZ-(9R5bh&X8(fJ@<`*A8~&JXFRbB*<$3c) zMjW#L3z_U})B0XJj&y6Lf6mrc9whjl8=A|RS@e0FFniG@Ap!eI;m(u$6Z?>Zkn=x$ z@V%}%raN>vPhRKgx#7aX&9ybg>+hBF(OF?RQ$~*GV&4V9CS(iMe!8Lcha1*cSb~-; z&gpl%F2T}-?^D@$)ti`IZ}P8c0^wy_o7=M)Wml&%p-rF}QoN{?Fpz~|p|WiZ7Yq@G z@IEu*m#?JEok0H~WFz-g&};@t!UXZb8+MH(6lcXXozf^|g;LOYnDFtk!i2|uLs@s&AJe?OH)9p!@F^+?>E0-!?6_7 z?ELZV*RL^VqFN{SA5=viKC)*c5e`I4lwGic9(goUgPIIfA0iLNeB;dthw9nv+nuby zG}?))wWi`jpv4zQhApCX6(yEqDuOW;euT6J39~|NL7g>K5c(%8o&NbJYf(!l*fu(H zcMDkw4#B9BRGmEk@%?+Xa09C4MLW3VSz%)Q(STilHJiUcVLts2`2RI_7Z5FZyT(ck z$XY*93Nmpd9WjT*FMi3S5`fkHuFaBCR7>jT&hMFagI%5Yx4`Dt7f}yi9}(zCm4WU{ z1?2tm(&62q;92~;64QrG{o@X2X2s`v$QdqoSYY>?EYbkZLKc=Lwg;)g{r0>Z9&f(J zEl}P%mxn*tHr&zjc__`aJBf6Qu+rh8?QGn<2$5);h_BddUyPjiHB7(p3o5Ky8gTj_2L!(wpVv!Bs@1*x$vZ z7PM2nYaj02xxMT{U_?YjZ0s57bTCa+x1SF56n7nPb?UraO0|5HnHL@wZLIT)6Sk5u zv%J39b@>7%g^pt7TZv|a`}ebQ`{di`F<9H<{d;;R!wtshDZ8_Glp~#yWv~@Ju5thHuk* z7S_8GiR)h)W#OUP`JSg}LNn{wkYF9#KV+8ki1Af3^Z0LHak!d~jFRTSL0agzeu=qH z(Z(b8tWpQW&SZA2Ij(QG(4xofDjvi0LFE zu}59K_1SatMy)>=|D5L>&iYvrBr5+_66B-o8xR}7y48LuF;|PW;v9oneNbnD+YP~jZI3Z)f8;l#}s+3xYZ5Y8^AcN=>ii6kq?6oJaBsV>bH{xebRG5{qn}t`l|8(EPZc_)r8&m8vx2H< z=;qrsAgUOzG4fTE0-36FZtRkb4E-s}GNl;hK_33E(r_?tw&BLWfFEO+to{Kb-)?kz z9o*kf1bccAIySyA!e2{_DprZfMe5zQd+bFz6BM)>+?@ZOnI7i)8|Sd5yC2LXOv&U0 zM*b?ds~ik_K?G-=3kWFoN??9)KTmhq^pjSO{|x1p08dKgDVWc4+Qe9BpC=-qhCT`AkU zDcCKa_x=m}YfrAJO4HOFwx)8;2h|*Cee<3*h}a!Fh0r3qo0}?{9S4yT7e=(OqFmjf z>Ok|aPG?Ke-oFZ(c$(3AT}nea`^`%;l(Yu*G)+_nsT}(&FVna7;;8lbOmKCgPd~2wu~yV7N@K{) zqlUckN@|SvC#!ns%B7 literal 0 HcmV?d00001 From 4b236225461a89a8cc6bd3132ee5eb32dd83d63a Mon Sep 17 00:00:00 2001 From: Jacob Wang Date: Sat, 21 Oct 2017 21:53:55 +1000 Subject: [PATCH 06/13] Update headers & page location --- _data/overviews.yml | 4 ++++ .../tutorials/binary-compatibility-for-library-authors.md | 7 ++++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/_data/overviews.yml b/_data/overviews.yml index f7ac783f48..05fa535e3f 100644 --- a/_data/overviews.yml +++ b/_data/overviews.yml @@ -63,6 +63,10 @@ description: "When two versions of Scala are binary compatible, it is safe to compile your project on one Scala version and link against another Scala version at run time. Safe run-time linkage (only!) means that the JVM does not throw a (subclass of) LinkageError when executing your program in the mixed scenario, assuming that none arise when compiling and running on the same version of Scala. Concretely, this means you may have external dependencies on your run-time classpath that use a different version of Scala than the one you’re compiling with, as long as they’re binary compatible. In other words, separate compilation on different binary compatible versions does not introduce problems compared to compiling and running everything on the same version of Scala." icon: puzzle-piece url: "core/binary-compatibility-of-scala-releases.html" + - title: Binary Compatibility for library authors + description: "A diverse and comprehensive set of libraries is important to any productive software ecosystem. While it is easy to develop and distribute Scala libraries, good library authorship goes beyond just writing code and publishing it. In this guide, we cover the important topic of Binary Compatibility." + icon: puzzle-piece + url: "core/binary-compatibility-for-library-authors.html" - category: "Reference/Documentation" description: "Reference material on core Scala tools like Scaladoc and the Scala REPL." diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md index 1a1c8bbdf8..1d90822ce7 100644 --- a/_overviews/tutorials/binary-compatibility-for-library-authors.md +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -2,10 +2,12 @@ layout: singlepage-overview title: Binary Compatibility for library authors -discourse: true -permalink: /tutorials/:title.html +partof: binary-compatibility +permalink: /overviews/core/:title.html --- +**By Jacob Wang** + ## Introduction A diverse and comprehensive set of libraries is important to any productive software ecosystem. While it is easy to develop and distribute Scala libraries, good library authorship goes @@ -208,4 +210,3 @@ In this guide we covered the importance of binary compatibility and showed you a binary compatibility breakages clearly to your users. If we follow these guidelines, we as a community can spend less time untangling dependency hell and more time building cool things! - From 3cbd13d6d888010e0973ea3d9d1795d00777d12b Mon Sep 17 00:00:00 2001 From: Jacob Wang Date: Sat, 21 Oct 2017 23:19:29 +1000 Subject: [PATCH 07/13] Various rewording and grammar fixes as suggested by jvican --- ...inary-compatibility-for-library-authors.md | 63 ++++++++++--------- 1 file changed, 33 insertions(+), 30 deletions(-) diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md index 1d90822ce7..5599ff3534 100644 --- a/_overviews/tutorials/binary-compatibility-for-library-authors.md +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -26,11 +26,11 @@ Before we start, let's understand how code is compiled and executed on the Java Scala is compiled to a platform-independent format called **JVM bytecode** and stored in `.class` files. These class files are collated in JAR files for distribution. When some code depends on a library, its compiled bytecode references the library's bytecode. The library's bytecode is referenced by its class/method signatures and loaded lazily -by the the JVM classloader during runtime. If a class or method matching the signature is not found, an exception is thrown. +by the JVM classloader during runtime. If a class or method matching the signature is not found, an exception is thrown. As a result of this execution model: -* We need to provide the JARs of every library used in our dependency tree when starting an application, since the library's bytecode is only referenced -- not merged into its user's bytecode +* We need to provide the JARs of every library used in our dependency tree when starting an application since the library's bytecode is only referenced, not merged into its user's bytecode * A missing class/method problem may only surface after the application has been running for a while, due to lazy loading. Common exceptions from classloading failures includes @@ -42,21 +42,24 @@ Consider an application `App` that depends on `A` which itself depends on librar for all of `App`, `A` and `C` (something like `java -cp App.jar:A.jar:C.jar:. MainClass`). If we did not provide `C.jar` or if we provided a `C.jar` that does not contain some classes/methods which `A` calls, we will get classloading exceptions when our code attempts to invoke the missing classes/methods. -These are what we call **Binary Incompatibility Errors**. An error caused by binary incompatibility happens when the compiled bytecode references a name that cannot be resolved during runtime +These are what we call **Binary Incompatibility Errors** -- errors that happen when the compiled bytecode references a name that cannot be resolved during runtime. + +Before we look at how to avoid binary incompatibility errors, let us first +establish some key terminologies we will be using for the rest of the guide. ## What are Evictions, Source Compatibility and Binary Compatibility? ### Evictions When a class is needed during execution, the JVM classloader loads the first matching class file from the classpath (any other matching class files are ignored). -Because of this, having multiple versions of the same library in the classpath is generally undesireable: +Because of this, having multiple versions of the same library in the classpath is generally undesirable: * Unnecessary application size increase -* Unexpected runtime behaviour if the order of class files changes +* Unexpected runtime behavior if the order of class files changes Therefore, when resolving JARs to use for compilation and packaging, most build tools will pick only one version of each library and **evict** the rest. ### Source Compatibility -Two library versions are **Source Compatible** if switching one for the other does not incur any compile errors. +Two library versions are **Source Compatible** if switching one for the other does not incur any compile errors or unintended behavioral changes at runtime. For example, If we can upgrade `v1.0.0` of a dependency to `v1.1.0` and recompile our code without any compilation errors, `v1.1.0` is source compatible with `v1.0.0`. ### Binary Compatibility @@ -66,7 +69,7 @@ For example, if we can replace the class files of a library's `v1.0.0` with the **NOTE:** While breaking source compatibility normally results in binary compatibility breakages as well, they are actually orthogonal -- breaking one does not imply breaking the other. -### Forwards and Backwards Compatibility +#### Forwards and Backwards Compatibility There are two "directions" when we describe compatibility of a library release: @@ -74,8 +77,7 @@ There are two "directions" when we describe compatibility of a library release: this is the common and implied direction. **Forwards Compatible** means that an older library can be used in an environment where a newer version is expected. -Forward compatibility is generally not upheld for userland libraries. It is only important in situations where an older version of a library is commonly -used at runtime against code that is compiled with newer version. (e.g. Scala's standard library) +Forward compatibility is generally not upheld for userland libraries. It is only important in situations where an older version of a library is commonly used at runtime against code that is compiled with newer versions. (for example, [Scala's standard library](http://docs.scala-lang.org/overviews/core/binary-compatibility-of-scala-releases.html)) Let's look at an example where library `A v1.0.0` is compiled with library `C v1.1.0`. @@ -83,29 +85,29 @@ Let's look at an example where library `A v1.0.0` is compiled with library `C v1 `C v1.1.0 ` is **Forwards Binary Compatible** with `v1.0.0` if we can use `v1.0.0`'s JAR at runtime instead of `v1.1.0`'s JAR without any binary compatibility errors. -`C v1.2.0 ` is **Backwards Binary Compatible** with `v1.1.0` if we can use `v1.2.0`'s JAR at runtime instaed of `v1.1.0`'s JAR without any binary compatibility errors.. +`C v1.2.0 ` is **Backwards Binary Compatible** with `v1.1.0` if we can use `v1.2.0`'s JAR at runtime instead of `v1.1.0`'s JAR without any binary compatibility errors. ## Why binary compatibility matters -Binary Compatibility matters because failing to maintain it makes life hard for everyone. +Binary Compatibility matters because breaking binary compatibility have bad consequences on the ecosystem around the software. -* End users has to update all library versions in their whole transitive dependency tree such that they are binary compatible, otherwise binary compatibility errors will happen at runtime -* Library authors are forced to update the dependencies of their library so users can continue using them, greatly increases the effort required to maintain libraries +* End users have to update versions transitively in all their dependency tree such that they are binary compatible. This process is time-consuming and error-prone, and it can change the semantics of end program. +* Library authors need to update their library dependencies to avoid "falling behind" and causing dependency hell for their users. Frequent binary breakages increase the effort required to maintain libraries. -Constant binary compatibility breakages in libraries, especially ones that are used by other libraries, is detrimental to our ecosystem as they require a lot of effort -from users and library authors to resolve. +Constant binary compatibility breakages in libraries, especially ones that are used by other libraries, is detrimental to our ecosystem as they require time +and effort from end users and maintainers of dependent libraries to resolve. Let's look at an example where binary incompatibility can cause grief and frustration: ### An example of "Dependency Hell" -Our application `App` depends on library `A` and `B`. Both `A` and `B` depends on library `C`. Initially both `A` and `B` depends on `C v1.0.0`. +Our application `App` depends on library `A` and `B`. Both `A` and `B` depends on library `C`. Initially, both `A` and `B` depends on `C v1.0.0`. ![Initial dependency graph]({{ site.baseurl }}/resources/images/library-author-guide/before_update.png){: style="width: 50%; margin: auto; display: block;"} -Some time later, we see `B v1.1.0` is available and upgrade its version in our build. Our code compiles and seems to work so we push it to production and go home for dinner. +Sometime later, we see `B v1.1.0` is available and upgrade its version in our build. Our code compiles and seems to work so we push it to production and go home for dinner. -Unfortunately at 2am, we got frantic calls from customers saying that our application is broken! Looking at the logs, you find lots of `NoSuchMethodError` is being thrown by some code in `A`! +Unfortunately at 2am, we get frantic calls from customers saying that our application is broken! Looking at the logs, you find lots of `NoSuchMethodError` are being thrown by some code in `A`! ![Binary incompatibility after upgrading]({{ site.baseurl }}/resources/images/library-author-guide/after_update.png){: style="width: 50%; margin: auto; display: block;"} @@ -151,7 +153,7 @@ Some language features may break binary compatibility: Techniques you can use to avoid breaking binary compatibility: -* Annotate public methods's return type explicitly +* Annotate public method's return type explicitly * Mark methods as package private when you want to remove a method or modify its signature * Don't use inlining (for libraries) @@ -161,17 +163,17 @@ Again, we recommend using MiMa to double check that you have not broken binary c ## Versioning Scheme - Communicating compatibility breakages -Library authors use verioning schemes to communicate compatibility guarantees between library releases to their users. Versioning schemes like [Semantic Versioning](http://semver.org/)(SemVer) allow -users to easily reason about the impact of a updating a library, without needing to read the detailed release note. +Library authors use versioning schemes to communicate compatibility guarantees between library releases to their users. Versioning schemes like [Semantic Versioning](http://semver.org/)(SemVer) allow +users to easily reason about the impact of updating a library, without needing to read the detailed release note. -In the following section we will outline a versioning scheme based on Semantic Versioning that we **strongly encourage** you to adopt for your libraries. The rules listed below are **in addition** to +In the following section, we will outline a versioning scheme based on Semantic Versioning that we **strongly encourage** you to adopt for your libraries. The rules listed below are **in addition** to Semantic Versioning v2.0.0. -### Recommmended Versioning Scheme +### Recommended Versioning Scheme -* If backwards **binary compatibility** is broken, **major version number** must be increased -* If backwards **source compatibility** is broken, **minor version number** must be increased -* A change in **patch version number** signals **no binary nor source incompatibility**. According to SemVer, patch versions should contain only bug fixes that fixes incorrect behavior so major behavioral +* If backward **binary compatibility** is broken, **major version number** must be increased +* If backward **source compatibility** is broken, **minor version number** must be increased +* A change in **patch version number** signals **no binary nor source incompatibility**. According to SemVer, patch versions should contain only bug fixes that fix incorrect behavior so major behavioral change in method/classes should result in a minor version bump. * When major version is `0`, a minor version bump **may contain both source and binary breakages** * Some libraries may take a harder stance on maintaining source compatibility, bumping the major version number for ANY source incompatibility even if they are binary compatible @@ -189,8 +191,8 @@ Many libraries in the Scala ecosystem has adopted this versioning scheme. A few If this version scheme is followed, reasoning about binary compatibility is now very simple: -* Ensure major versions of the all versions of a library in the dependency tree are the same -* Pick latest version and evict the rest (This is the default behavior of SBT). +* Ensure major versions of all versions of a library in the dependency tree are the same +* Pick the latest version and evict the rest (This is the default behavior of SBT). ### Explanation @@ -202,11 +204,12 @@ From our [example](#why-binary-compatibility-matters) above, we have learned two * If a new library version is binary compatible but source incompatible, the user can simply fix the compile errors in their application and everything will work Therefore, **binary incompatible releases should be avoided if possible** and be more noticeable when they happen, warranting the use of the major version number. While source compatibility -is also important, if they are minor breakages that does not require effort to fix, then it is best to let the major number signal just binary compatibility. +is also important, if they are minor breakages that do not require effort to fix, then it is best to let the major number signal just binary compatibility. ## Conclusion -In this guide we covered the importance of binary compatibility and showed you a few tricks to avoid breaking binary compatibility. Finally, we laid out a versioning scheme to communicate +In this guide, we covered the importance of binary compatibility and showed you a few tricks to avoid breaking binary compatibility. Finally, we laid out a versioning scheme to communicate binary compatibility breakages clearly to your users. If we follow these guidelines, we as a community can spend less time untangling dependency hell and more time building cool things! + From e5f10194f1fb072ecf671a4a9f0a42c13f9bcf95 Mon Sep 17 00:00:00 2001 From: Jacob Wang Date: Thu, 26 Oct 2017 22:36:14 +1000 Subject: [PATCH 08/13] reword reason why evictions are performed by build tools --- .../tutorials/binary-compatibility-for-library-authors.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md index 5599ff3534..61ea41bbb8 100644 --- a/_overviews/tutorials/binary-compatibility-for-library-authors.md +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -53,7 +53,7 @@ establish some key terminologies we will be using for the rest of the guide. When a class is needed during execution, the JVM classloader loads the first matching class file from the classpath (any other matching class files are ignored). Because of this, having multiple versions of the same library in the classpath is generally undesirable: -* Unnecessary application size increase +* Need to fetch and bundle multiple library versions when only one is actually used * Unexpected runtime behavior if the order of class files changes Therefore, when resolving JARs to use for compilation and packaging, most build tools will pick only one version of each library and **evict** the rest. From 1d1c480d3dabc9c04aca38fc079d1fa252da2383 Mon Sep 17 00:00:00 2001 From: Jacob Wang Date: Thu, 26 Oct 2017 23:00:25 +1000 Subject: [PATCH 09/13] correct point about not inlining, as inlining from source is safe --- .../tutorials/binary-compatibility-for-library-authors.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md index 61ea41bbb8..aae32bb527 100644 --- a/_overviews/tutorials/binary-compatibility-for-library-authors.md +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -155,7 +155,7 @@ Techniques you can use to avoid breaking binary compatibility: * Annotate public method's return type explicitly * Mark methods as package private when you want to remove a method or modify its signature -* Don't use inlining (for libraries) +* Don't turn on inlining for methods from dependencies For brevity of this guide, detailed explanation and runnable code examples can be found in [Binary Compatibility Code Examples & Explanation](https://github.com/jatcwang/binary-compatibility-guide). From 371090897094bb1ec00dcdb3ddd40848d2e9eb35 Mon Sep 17 00:00:00 2001 From: Jacob Wang Date: Sun, 12 Nov 2017 12:07:57 +1000 Subject: [PATCH 10/13] - Add scala.js/native section - merge conclusion and Explanation for versioning scheme - adjust colors so they don't hurt my eye - various fixes for clarity and accuracy --- ...inary-compatibility-for-library-authors.md | 112 +++++++++--------- 1 file changed, 59 insertions(+), 53 deletions(-) diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md index aae32bb527..a629008a44 100644 --- a/_overviews/tutorials/binary-compatibility-for-library-authors.md +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -23,10 +23,13 @@ Before we start, let's understand how code is compiled and executed on the Java ## The JVM execution model -Scala is compiled to a platform-independent format called **JVM bytecode** and stored in `.class` files. These class files are collated in JAR files for distribution. +Scala is compiled to a platform-independent format called **JVM bytecode** and stored in `.class` files. +These class files are collated in JAR files for distribution. -When some code depends on a library, its compiled bytecode references the library's bytecode. The library's bytecode is referenced by its class/method signatures and loaded lazily -by the JVM classloader during runtime. If a class or method matching the signature is not found, an exception is thrown. +When some code depends on a library, its compiled bytecode references the library's bytecode. +The library's bytecode is referenced by its class/method signatures and loaded lazily +by the JVM classloader during runtime. If a class or method matching the signature is not found, +an exception is thrown. As a result of this execution model: @@ -42,7 +45,14 @@ Consider an application `App` that depends on `A` which itself depends on librar for all of `App`, `A` and `C` (something like `java -cp App.jar:A.jar:C.jar:. MainClass`). If we did not provide `C.jar` or if we provided a `C.jar` that does not contain some classes/methods which `A` calls, we will get classloading exceptions when our code attempts to invoke the missing classes/methods. -These are what we call **Binary Incompatibility Errors** -- errors that happen when the compiled bytecode references a name that cannot be resolved during runtime. +These are what we call **Linkage Errors** -- errors that happen when the compiled bytecode references a name that cannot be resolved during runtime. + +### What about Scala.js and Scala Native? +Similarly to the JVM, Scala.js and Scala Native have their respective equivalents of .class files, namely .sjsir files and .nir files. Similarly to .class files, they are distributed in .jars and linked together at the end. + +However, contrary to the JVM, Scala.js and Scala Native link their respective IR files at link time, so eagerly, instead of lazily at run-time. Failure to correctly link the entire program results in linking errors reported while trying to invoke `fastOptJS`/`fullOptJS` or `nativeLink`. + +Besides that difference in the timing of linkage errors, the models are extremely similar. **Unless otherwise noted, the contents of this guide apply equally to the JVM, Scala.js and Scala Native.** Before we look at how to avoid binary incompatibility errors, let us first establish some key terminologies we will be using for the rest of the guide. @@ -59,15 +69,14 @@ Because of this, having multiple versions of the same library in the classpath i Therefore, when resolving JARs to use for compilation and packaging, most build tools will pick only one version of each library and **evict** the rest. ### Source Compatibility -Two library versions are **Source Compatible** if switching one for the other does not incur any compile errors or unintended behavioral changes at runtime. -For example, If we can upgrade `v1.0.0` of a dependency to `v1.1.0` and recompile our code without any compilation errors, `v1.1.0` is source compatible with `v1.0.0`. +Two library versions are **Source Compatible** with each other if switching one for the other does not incur any compile errors or unintended behavioral changes (semantic errors). +For example, If we can upgrade `v1.0.0` of a dependency to `v1.1.0` and recompile our code without any compilation errors or semantic errors, `v1.1.0` is source compatible with `v1.0.0`. ### Binary Compatibility -Two library versions are **Binary Compatible** if the compiled bytecode of these versions can be interchanged without causing binary compatibility errors. -For example, if we can replace the class files of a library's `v1.0.0` with the class files of `v1.1.0` without any binary compatibility errors during runtime, -`v1.1.0` is binary compatible with `v1.0.0`. +Two library versions are **Binary Compatible** with each other if the compiled bytecode of these versions can be interchanged without causing Linkage Errors. -**NOTE:** While breaking source compatibility normally results in binary compatibility breakages as well, they are actually orthogonal -- breaking one does not imply breaking the other. +### Relationship between source and binary compatibility +While breaking source compatibility often results in binary incompatibilities as well, they are actually orthogonal -- breaking one does not imply breaking the other. #### Forwards and Backwards Compatibility @@ -76,20 +85,20 @@ There are two "directions" when we describe compatibility of a library release: **Backwards Compatible** means that a newer library version can be used in an environment where an older version is expected. When talking about binary and source compatibility, this is the common and implied direction. -**Forwards Compatible** means that an older library can be used in an environment where a newer version is expected. -Forward compatibility is generally not upheld for userland libraries. It is only important in situations where an older version of a library is commonly used at runtime against code that is compiled with newer versions. (for example, [Scala's standard library](http://docs.scala-lang.org/overviews/core/binary-compatibility-of-scala-releases.html)) +**Forwards Compatible** means that an older library can be used in an environment where a newer version is expected. +Forward compatibility is generally not upheld for libraries. Let's look at an example where library `A v1.0.0` is compiled with library `C v1.1.0`. ![Forwards and Backwards Compatibility]({{ site.baseurl }}/resources/images/library-author-guide/fowards_backwards_compatibility.png){: style="width: 65%; margin: auto; display: block"} -`C v1.1.0 ` is **Forwards Binary Compatible** with `v1.0.0` if we can use `v1.0.0`'s JAR at runtime instead of `v1.1.0`'s JAR without any binary compatibility errors. +`C v1.1.0 ` is **Forwards Binary Compatible** with `v1.0.0` if we can use `v1.0.0`'s JAR at runtime instead of `v1.1.0`'s JAR without any linkage errors. -`C v1.2.0 ` is **Backwards Binary Compatible** with `v1.1.0` if we can use `v1.2.0`'s JAR at runtime instead of `v1.1.0`'s JAR without any binary compatibility errors. +`C v1.2.0 ` is **Backwards Binary Compatible** with `v1.1.0` if we can use `v1.2.0`'s JAR at runtime instead of `v1.1.0`'s JAR without any linkage errors. ## Why binary compatibility matters -Binary Compatibility matters because breaking binary compatibility have bad consequences on the ecosystem around the software. +Binary Compatibility matters because breaking binary compatibility has bad consequences on the ecosystem around the software. * End users have to update versions transitively in all their dependency tree such that they are binary compatible. This process is time-consuming and error-prone, and it can change the semantics of end program. * Library authors need to update their library dependencies to avoid "falling behind" and causing dependency hell for their users. Frequent binary breakages increase the effort required to maintain libraries. @@ -134,30 +143,32 @@ How can we, as library authors, spare our users of runtime errors and dependency ## MiMa - Checking binary compatibility against previous library versions The [Migration Manager for Scala](https://github.com/typesafehub/migration-manager) (MiMa) is a tool for diagnosing binary incompatibilities between different library versions. -It works by comparing the class files of two provided JARs and report any binary incompatibilities found. +It works by comparing the class files of two provided JARs and report any binary incompatibilities found. +Both backwards and forwards binary incompatibility can be detected by swapping input order of the JARs. By incorporating [MiMa SBT plugin](https://github.com/typesafehub/migration-manager/wiki/Sbt-plugin) into your SBT build, you can easily check whether you have accidentally introduced binary incompatible changes. Detailed instruction on how to use the SBT plugin can be found in the link. -We strongly encourage every library author to incorporate MiMa into their library release workflow. +We strongly encourage every library author to incorporate MiMa into their continuous integration and release workflow. + +Detecting backwards source compatibility is difficult with Scala due to language features like implicit +and named parameters. The best approximation to checking backwards source compatibility is running +both forwards and backwards binary compatibility check, as this can detect most cases +of source-incompatible changes. For example, adding/removing public class members is a source +incompatible change, and will be caught through forward + backward binary compatibility check. ## Evolving code without breaking binary compatibility -Binary compatibility breakages can often be avoided through careful use of certain Scala features as well as some techniques you can apply when modifying code. +Binary compatibility breakages can often be avoided through the careful use of certain Scala features +as well as some techniques you can apply when modifying code. -Some language features may break binary compatibility: +For example, the use of these language features are a common source of binary compatibility breakages +in library releases: * Default parameter values for methods or classes * Case classes -* Default methods on traits (doesn't cause breakages since 2.12) - -Techniques you can use to avoid breaking binary compatibility: - -* Annotate public method's return type explicitly -* Mark methods as package private when you want to remove a method or modify its signature -* Don't turn on inlining for methods from dependencies -For brevity of this guide, detailed explanation and runnable code examples can be found in [Binary Compatibility Code Examples & Explanation](https://github.com/jatcwang/binary-compatibility-guide). +You can find detailed explanations, runnable examples and tips to maintain binary compatibility in [Binary Compatibility Code Examples & Explanation](https://github.com/jatcwang/binary-compatibility-guide). Again, we recommend using MiMa to double check that you have not broken binary compatibility after making changes. @@ -171,45 +182,40 @@ Semantic Versioning v2.0.0. ### Recommended Versioning Scheme -* If backward **binary compatibility** is broken, **major version number** must be increased -* If backward **source compatibility** is broken, **minor version number** must be increased -* A change in **patch version number** signals **no binary nor source incompatibility**. According to SemVer, patch versions should contain only bug fixes that fix incorrect behavior so major behavioral +* If backward **binary compatibility** is broken, **major version number** must be increased. +* If backward **source compatibility** is broken, **minor version number** must be increased. +* A change in **patch version number** signals **no binary nor source incompatibility**. +According to SemVer, patch versions should contain only bug fixes that fix incorrect behavior so major behavioral change in method/classes should result in a minor version bump. -* When major version is `0`, a minor version bump **may contain both source and binary breakages** -* Some libraries may take a harder stance on maintaining source compatibility, bumping the major version number for ANY source incompatibility even if they are binary compatible +* When major version is `0`, a minor version bump **may contain both source and binary breakages**. Some examples: -* `v1.0.0 -> v2.0.0` is binary incompatible. Cares needs to be taken to make sure no evicted versions are still in the `v1.x.x` range to avoid runtime errors -* `v1.0.0 -> v1.1.0` is binary compatible and maybe source incompatible -* `v1.0.0 -> v1.0.1` is binary compatible and source compatible -* `v0.4.0 -> v0.5.0` is binary incompatible and maybe source incompatible -* `v0.4.0 -> v0.4.1` is binary compatible and source compatible +* `v1.0.0 -> v2.0.0` is binary incompatible. + End users and library maintainers need to update all their dependency graph to remove all dependency on `v1.0.0`. +* `v1.0.0 -> v1.1.0` is binary compatible. Classpath can safely contain both `v1.0.0` and `v1.1.0`. End user may need to fix minor source breaking changes introduced +* `v1.0.0 -> v1.0.1` is binary compatible. This is a safe upgrade that does not introduce binary or source incompatibilities. +* `v0.4.0 -> v0.5.0` is binary incompatible. + End users and library maintainers need to update all their dependency graph to remove all dependency on `v0.4.0`. +* `v0.4.0 -> v0.4.1` is binary compatible. Classpath can safely contain both `v1.0.0` and `v1.1.0`. End user may need to fix minor source breaking changes introduced Many libraries in the Scala ecosystem has adopted this versioning scheme. A few examples are [Akka](http://doc.akka.io/docs/akka/2.5/scala/common/binary-compatibility-rules.html), [Cats](https://github.com/typelevel/cats#binary-compatibility-and-versioning) and [Scala.js](https://www.scala-js.org/). -If this version scheme is followed, reasoning about binary compatibility is now very simple: - -* Ensure major versions of all versions of a library in the dependency tree are the same -* Pick the latest version and evict the rest (This is the default behavior of SBT). - -### Explanation +## Conclusion -Why do we use the major version number to signal binary incompatible releases? +Why is binary compatibility so important such that we recommend using the major version number to track it? From our [example](#why-binary-compatibility-matters) above, we have learned two important lessons: -* Binary incompatibility releases often leads to dependency hell, rendering your users unable to update any of their libraries without breaking their application -* If a new library version is binary compatible but source incompatible, the user can simply fix the compile errors in their application and everything will work +* Binary incompatibility releases often lead to dependency hell, rendering your users unable to update any of their libraries without breaking their application. +* If a new library version is binary compatible but source incompatible, the user can fix the compile errors and their application should work. -Therefore, **binary incompatible releases should be avoided if possible** and be more noticeable when they happen, warranting the use of the major version number. While source compatibility -is also important, if they are minor breakages that do not require effort to fix, then it is best to let the major number signal just binary compatibility. - -## Conclusion +Therefore, **binary incompatible releases should be avoided if possible** and unambiguously documented +when they happen, warranting the use of the major version number. Users of your library can then enjoy +simple version upgrades and have clear warnings when they need to align library versions in their dependency tree +due to a binary incompatible release. -In this guide, we covered the importance of binary compatibility and showed you a few tricks to avoid breaking binary compatibility. Finally, we laid out a versioning scheme to communicate -binary compatibility breakages clearly to your users. +If we follow all recommendations laid out in this guide, we as a community can spend less time untangling dependency hell and more time building cool things! -If we follow these guidelines, we as a community can spend less time untangling dependency hell and more time building cool things! From 756fab83a004b4c45865655c2c0d036d555d3c36 Mon Sep 17 00:00:00 2001 From: Jacob Wang Date: Thu, 28 Dec 2017 00:07:25 +1000 Subject: [PATCH 11/13] Address @gmethvin's feedback: fix Mima link, wording and typos --- .../tutorials/binary-compatibility-for-library-authors.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md index a629008a44..51a0f92a37 100644 --- a/_overviews/tutorials/binary-compatibility-for-library-authors.md +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -142,11 +142,11 @@ How can we, as library authors, spare our users of runtime errors and dependency ## MiMa - Checking binary compatibility against previous library versions -The [Migration Manager for Scala](https://github.com/typesafehub/migration-manager) (MiMa) is a tool for diagnosing binary incompatibilities between different library versions. +The [Migration Manager for Scala](https://github.com/lightbend/migration-manager) (MiMa) is a tool for diagnosing binary incompatibilities between different library versions. It works by comparing the class files of two provided JARs and report any binary incompatibilities found. Both backwards and forwards binary incompatibility can be detected by swapping input order of the JARs. -By incorporating [MiMa SBT plugin](https://github.com/typesafehub/migration-manager/wiki/Sbt-plugin) into your SBT build, you can easily check whether +By incorporating [MiMa SBT plugin](https://github.com/lightbend/migration-manager/wiki/Sbt-plugin) into your SBT build, you can easily check whether you have accidentally introduced binary incompatible changes. Detailed instruction on how to use the SBT plugin can be found in the link. We strongly encourage every library author to incorporate MiMa into their continuous integration and release workflow. @@ -175,7 +175,7 @@ Again, we recommend using MiMa to double check that you have not broken binary c ## Versioning Scheme - Communicating compatibility breakages Library authors use versioning schemes to communicate compatibility guarantees between library releases to their users. Versioning schemes like [Semantic Versioning](http://semver.org/)(SemVer) allow -users to easily reason about the impact of updating a library, without needing to read the detailed release note. +users to easily reason about the impact of updating a library, without needing to read the detailed release notes. In the following section, we will outline a versioning scheme based on Semantic Versioning that we **strongly encourage** you to adopt for your libraries. The rules listed below are **in addition** to Semantic Versioning v2.0.0. @@ -184,7 +184,7 @@ Semantic Versioning v2.0.0. * If backward **binary compatibility** is broken, **major version number** must be increased. * If backward **source compatibility** is broken, **minor version number** must be increased. -* A change in **patch version number** signals **no binary nor source incompatibility**. +* A change in **patch version number** signals **neither binary nor source incompatibility**. According to SemVer, patch versions should contain only bug fixes that fix incorrect behavior so major behavioral change in method/classes should result in a minor version bump. * When major version is `0`, a minor version bump **may contain both source and binary breakages**. From a41fbfc577fd48efd191a97b2d12e28fe606db55 Mon Sep 17 00:00:00 2001 From: Jacob Wang Date: Sat, 6 Jan 2018 18:33:30 +1000 Subject: [PATCH 12/13] mention default eviction strategy of SBT and gradle --- .../tutorials/binary-compatibility-for-library-authors.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md index 51a0f92a37..96326e6c88 100644 --- a/_overviews/tutorials/binary-compatibility-for-library-authors.md +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -66,7 +66,8 @@ Because of this, having multiple versions of the same library in the classpath i * Need to fetch and bundle multiple library versions when only one is actually used * Unexpected runtime behavior if the order of class files changes -Therefore, when resolving JARs to use for compilation and packaging, most build tools will pick only one version of each library and **evict** the rest. +Therefore, build tools like SBT and Gradle will pick one version and **evict** the rest when resolving JARs to use for compilation and packaging. +By default they pick the latest version of each library, but it is possible to specify another version if required. ### Source Compatibility Two library versions are **Source Compatible** with each other if switching one for the other does not incur any compile errors or unintended behavioral changes (semantic errors). From 6acae7c3cb3eace435aaf21f3f8295c025a9276b Mon Sep 17 00:00:00 2001 From: Jacob Wang Date: Tue, 9 Jan 2018 22:31:22 +1000 Subject: [PATCH 13/13] minor edit --- .../tutorials/binary-compatibility-for-library-authors.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_overviews/tutorials/binary-compatibility-for-library-authors.md b/_overviews/tutorials/binary-compatibility-for-library-authors.md index 96326e6c88..f88e740e94 100644 --- a/_overviews/tutorials/binary-compatibility-for-library-authors.md +++ b/_overviews/tutorials/binary-compatibility-for-library-authors.md @@ -195,7 +195,7 @@ Some examples: * `v1.0.0 -> v2.0.0` is binary incompatible. End users and library maintainers need to update all their dependency graph to remove all dependency on `v1.0.0`. * `v1.0.0 -> v1.1.0` is binary compatible. Classpath can safely contain both `v1.0.0` and `v1.1.0`. End user may need to fix minor source breaking changes introduced -* `v1.0.0 -> v1.0.1` is binary compatible. This is a safe upgrade that does not introduce binary or source incompatibilities. +* `v1.0.0 -> v1.0.1` is source and binary compatible. This is a safe upgrade that does not introduce binary or source incompatibilities. * `v0.4.0 -> v0.5.0` is binary incompatible. End users and library maintainers need to update all their dependency graph to remove all dependency on `v0.4.0`. * `v0.4.0 -> v0.4.1` is binary compatible. Classpath can safely contain both `v1.0.0` and `v1.1.0`. End user may need to fix minor source breaking changes introduced