Skip to content

Rework tag driven release infrastructure #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 14, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
language: scala
env:
global:
- PUBLISH_JDK=oraclejdk8 # admin/build.sh only publishes when running on this jdk
# Don't commit sensitive files, instead commit a version encrypted with $SECRET,
# this environment variable is encrypted with this repo's private key and stored below:
# (See http://docs.travis-ci.com/user/environment-variables/#Secure-Variables.)
secure: "sGB53QddmPmQ4ftCGYxT0gaJcFt0bpMJoGxJRJCFTxdzg6nNMqJ9qDWbyJo7vDFx30axNQlyBH928pUiS5KfsmvzVdoVHUBEUJlF1lBurlpx06tGLuBdcFDwUF5ybi7SGRNdUPuX/6uLdgK5clpcW16/pcfT5Qr5vo/0mvPY85s="
- PUBLISH_JDK=oraclejdk8
# PGP_PASSPHRASE
- secure: "BzgzRZLYa52rS/hBfzf43b++CfDhdcd3Mmu8tsyBHgThSQOd2YBLbV5kWD8aYVFKVHfW7XX0PTe3F+rR/fFZqGItE6o8Px0Y7Vzb5pqjlaQdxFEJ+WrsnshS0xuAKZ7OwVHRp+d+jznaCwRxEo2vpW3ko1OPAJ8cxfhVL/4C1I0="
# SONA_USER
- secure: "lx2qFeFxh9AFmyHR7hH4Qf9flIEx8VgYj6ebzuxp1cc1ZZiXHC1256x0bHFDUH9bhJACOazOrco/+v6MBAriBkWxLBc98FrC6OkVeQMFW2ffWSBuHRclilKsQA/Lsgc81Wg+WV105hOqUNAkTXgroblInNt+KS+DhC/8FVoh9ZY="
# SONA_PASS
- secure: "FZC+FZnBNeklA150vW5QDZJ5J7t+DExJrgyXWM46Wh0MobjH8cvydgC3qatItb0rDBV8l7zO1LDwl2KEi92aefw2a8E49z6qVOHgUXiI3SAx7M0UO0FFeKPmTXCLcBlbnGLcUqNjIZfuIEufQvPblKTl8qN4eMmcMn9jsNzJr28="
script:
- admin/build.sh
scala:
Expand Down
58 changes: 58 additions & 0 deletions admin/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
## Tag Driven Releasing

### Background Reading

- http://docs.travis-ci.com/user/environment-variables/
- http://docs.travis-ci.com/user/encryption-keys/
- http://docs.travis-ci.com/user/encrypting-files/

### Initial setup for the repository

To configure tag driven releases from Travis CI.

1. Generate a key pair for this repository with `./admin/genKeyPair.sh`.
Edit `.travis.yml` and `admin/build.sh` as prompted.
2. Publish the public key to https://pgp.mit.edu
3. Store other secrets as encrypted environment variables with `admin/encryptEnvVars.sh`.
Edit `.travis.yml` as prompted.
4. Edit `.travis.yml` to use `./admin/build.sh` as the build script,
and edit that script to use the tasks required for this project.
5. Edit `.travis.yml` to select which JDK will be used for publishing.

It is important to add comments in .travis.yml to identify the name
of each environment variable encoded in a `:secure` section.

After all of these steps, your .travis.yml should contain config of the
form:

language: scala
env:
global:
- PUBLISH_JDK=openjdk6
# PGP_PASSPHRASE
- secure: "XXXXXX"
# SONA_USER
- secure: "XXXXXX"
# SONA_PASS
- secure: "XXXXXX"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does encrypted_1ce132863fa7_key and companion go?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are referred to admin/build.sh. Are they provided automagically by Travis? I don't quite understand this part.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I meant, why don't they have to be listed in .travis.yml

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the log of running genKeyPair.sh

sh -x ./admin/genKeyPair.sh
+ set -e
+ for f in admin/secring.asc.enc admin/secring.asc admin/pubring.asc
+ promptDelete admin/secring.asc.enc
+ [[ -f admin/secring.asc.enc ]]
+ echo About to delete admin/secring.asc.enc, Enter for okay / CTRL-C to cancel
About to delete admin/secring.asc.enc, Enter for okay / CTRL-C to cancel
+ read

+ rm admin/secring.asc.enc
+ for f in admin/secring.asc.enc admin/secring.asc admin/pubring.asc
+ promptDelete admin/secring.asc
+ [[ -f admin/secring.asc ]]
+ for f in admin/secring.asc.enc admin/secring.asc admin/pubring.asc
+ promptDelete admin/pubring.asc
+ [[ -f admin/pubring.asc ]]
+ echo About to delete admin/pubring.asc, Enter for okay / CTRL-C to cancel
About to delete admin/pubring.asc, Enter for okay / CTRL-C to cancel
+ read

+ rm admin/pubring.asc
+ echo Generating key pair. Please enter 1. repo name 2. scala-internals@googlegroups.com, 3. a new passphrase
Generating key pair. Please enter 1. repo name 2. scala-internals@googlegroups.com, 3. a new passphrase
+ cp admin/gpg.sbt project
+ sbt 'set pgpReadOnly := false' 'set pgpPublicRing := file("admin/pubring.asc")' 'set pgpSecretRing := file("admin/secring.asc")' 'pgp-cmd gen-key'
[info] Loading global plugins from /Users/jason/.sbt/0.13/plugins
[info] Loading project definition from /Users/jason/code/scala-java8-compat/project/project
[info] Loading project definition from /Users/jason/code/scala-java8-compat/project
[warn] There may be incompatibilities among your library dependencies.
[warn] Here are some of the libraries that were evicted:
[warn]  * com.typesafe.sbt:sbt-pgp:0.8.1 -> 0.8.3
[warn] Run 'evicted' to see detailed eviction warnings
[info] Set current project to scala-java8-compat (in build file:/Users/jason/code/scala-java8-compat/)
[info] Defining */*:pgpReadOnly
[info] The new value will be used by no settings or tasks.
[info] Reapplying settings...
[info] Set current project to scala-java8-compat (in build file:/Users/jason/code/scala-java8-compat/)
[info] Defining */*:pgpPublicRing
[info] The new value will be used by */*:pgpStaticContext
[info] Reapplying settings...
[info] Set current project to scala-java8-compat (in build file:/Users/jason/code/scala-java8-compat/)
[info] Defining */*:pgpSecretRing
[info] The new value will be used by */*:pgpStaticContext
[info] Reapplying settings...
[info] Set current project to scala-java8-compat (in build file:/Users/jason/code/scala-java8-compat/)
Please enter the name associated with the key: scala-java8-compat
Please enter the email associated with the key: scala-internals@googlegroups.com
Please enter the passphrase for the key: ***************
Please re-enter the passphrase for the key: ***************
[info] Creating a new PGP key, this could take a long time.
[info] Public key := /Users/jason/code/scala-java8-compat/admin/pubring.asc
[info] Secret key := /Users/jason/code/scala-java8-compat/admin/secring.asc
[info] Please do not share your secret key.   Your public key is free to share.
+ rm project/gpg.sbt
+ echo ============================================================================================
============================================================================================
+ echo Encrypting admin/secring.asc. Update K and IV variables in admin/build.sh accordingly.
Encrypting admin/secring.asc. Update K and IV variables in admin/build.sh accordingly.
+ echo ============================================================================================
============================================================================================
+ travis encrypt-file admin/secring.asc
encrypting admin/secring.asc for scala/scala-java8-compat
storing result as secring.asc.enc
storing secure env variables for decryption

Please add the following to your build script (before_install stage in your .travis.yml, for instance):

    openssl aes-256-cbc -K $encrypted_1ce132863fa7_key -iv $encrypted_1ce132863fa7_iv -in secring.asc.enc -out admin/secring.asc -d

Pro Tip: You can add it automatically by running with --add.

Make sure to add secring.asc.enc to the git repository.
Make sure not to add admin/secring.asc to the git repository.
Commit all changes to your .travis.yml.
+ rm admin/secring.asc
+ mv secring.asc.enc admin
+ echo ============================================================================================
============================================================================================
+ echo Encrypting environment variables. Add each to a line in .travis.yml. Include a comment
Encrypting environment variables. Add each to a line in .travis.yml. Include a comment
+ echo with the name of the corresponding variable
with the name of the corresponding variable
+ echo ============================================================================================
============================================================================================
+ read -s -p 'PGP_PASSPHRASE: ' PGP_PASSPHRASE
PGP_PASSPHRASE: + travis encrypt PGP_PASSPHRASE=D1cXBjsw15dLUA4
Please add the following to your .travis.yml file:

  secure: "YwLY+gZ1TQvRYEGMNKIHR6RAinI4UftTpXGEESQqOGIZvfKW3/rZqMGVU1vnEtgBQENH7jrL+ujFsH0K4t36WF3jgnGd/fYhiODuOofi0+ZWG0hcvt0EtM9XguvT7BrmWn1+zTdVuZyz8IYxacVpk6sToDyLwxmjUdGMEMu5qV8="

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, so Travis does more magic behind the scenes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I suspect this won't work :(

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've noticed the --add option didn't work with travis encrypt for some reason.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see. It has stored them on the Travis server as a build setting.

image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! Don't underestimate the magic of Travis! :-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

travis encrypt FOO=BAR --add worked for me. Maybe you tried travis encrypt --add FOO=BAR ?

In the end I decided not to use --add in this script so we could hand-curate the comments.

script:
- admin/build.sh

If Sonatype credentials change in the future, step 3 can be repeated
without generating a new key.

### Testing

1. Follow the release process below to create a dummy release (e.g. 0.1.0-TEST1).
Confirm that the release was staged to Sonatype but do not release it to Maven
central. Instead, drop the staging repository.

### Performing a release

1. Create a GitHub "Release" (with a corresponding tag) via the GitHub
web interface.
2. Travis CI will schedule a build for this release. Review the build logs.
3. Log into https://oss.sonatype.org/ and identify the staging repository.
4. Sanity check its contents
5. Release staging repository to Maven and send out release announcement.

10 changes: 7 additions & 3 deletions admin/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,14 @@ if [ "$TRAVIS_JDK_VERSION" == "$PUBLISH_JDK" ] && [[ "$TRAVIS_TAG" =~ ^v[0-9]+\.
myVer=$(echo $TRAVIS_TAG | sed -e s/^v// | sed -e 's/_[0-9]*\.[0-9]*//')
publishVersion='set every version := "'$myVer'"'
extraTarget="publish-signed"

cat admin/gpg.sbt >> project/plugins.sbt
admin/decrypt.sh sensitive.sbt
(cd admin/ && ./decrypt.sh secring.asc)
cp admin/publish-settings.sbt .

# Copied from the output of genKeyPair.sh
K=$encrypted_1ce132863fa7_key
IV=$encrypted_1ce132863fa7_iv

aes-256-cbc -K $K -iv $IV -in admin/secring.asc.enc -out admin/secring.asc -d
fi

sbt ++$TRAVIS_SCALA_VERSION "$publishVersion" clean update test publishLocal $extraTarget
2 changes: 0 additions & 2 deletions admin/decrypt.sh

This file was deleted.

2 changes: 0 additions & 2 deletions admin/encrypt.sh

This file was deleted.

19 changes: 0 additions & 19 deletions admin/encryptAll.sh

This file was deleted.

11 changes: 11 additions & 0 deletions admin/encryptEnvVars.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash
#
# Encrypt sonatype credentials so that they can be
# decrypted in trusted builds on Travis CI.
#
set -e

read -s -p 'SONA_USER: ' SONA_USER
travis encrypt SONA_USER="$SONA_USER"
read -s -p 'SONA_PASS: ' SONA_PASS
travis encrypt SONA_PASS="$SONA_PASS"
40 changes: 40 additions & 0 deletions admin/genKeyPair.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
#!/bin/bash
#
# Generates a key pair for this repository to sign artifacts.
# Encrypt the private key and its passphrase in trusted builds
# on Travis CI.
#
set -e

# Based on https://gist.github.com/kzap/5819745:
function promptDelete() {
if [[ -f "$1" ]]; then
echo About to delete $1, Enter for okay / CTRL-C to cancel
read
rm "$1"
fi
}
for f in admin/secring.asc.enc admin/secring.asc admin/pubring.asc; do promptDelete "$f"; done

echo Generating key pair. Please enter 1. repo name 2. scala-internals@googlegroups.com, 3. a new passphrase
cp admin/gpg.sbt project
sbt 'set pgpReadOnly := false' \
'set pgpPublicRing := file("admin/pubring.asc")' \
'set pgpSecretRing := file("admin/secring.asc")' \
'pgp-cmd gen-key'
rm project/gpg.sbt

echo ============================================================================================
echo Encrypting admin/secring.asc. Update K and IV variables in admin/build.sh accordingly.
echo ============================================================================================
travis encrypt-file admin/secring.asc
rm admin/secring.asc
mv secring.asc.enc admin

echo ============================================================================================
echo Encrypting environment variables. Add each to a line in .travis.yml. Include a comment
echo with the name of the corresponding variable
echo ============================================================================================
read -s -p 'PGP_PASSPHRASE: ' PGP_PASSPHRASE
travis encrypt PGP_PASSPHRASE="$PGP_PASSPHRASE"

24 changes: 0 additions & 24 deletions admin/gpg.sbt
Original file line number Diff line number Diff line change
@@ -1,26 +1,2 @@

addSbtPlugin("com.typesafe.sbt" % "sbt-pgp" % "0.8.3") // only added when publishing:

// There's a companion sensitive.sbt, which was created like this:
//
// 1. in an sbt shell that has the sbt-pgp plugin, create pgp key in admin/:
//
// sbt
// set pgpReadOnly := false
// set pgpPublicRing := file("admin/pubring.asc")
// set pgpSecretRing := file("admin/secring.asc")
// pgp-cmd gen-key // use $passPhrase
// Please enter the name associated with the key: $repoName
// Please enter the email associated with the key: scala-internals@googlegroups.com
// Please enter the passphrase for the key: $passphrase
//
// 2. create sensitive.sbt with contents:
//
// pgpPassphrase := Some($passPhrase.toArray)
//
// pgpPublicRing := file("admin/pubring.asc")
//
// pgpSecretRing := file("admin/secring.asc")
//
// credentials += Credentials("Sonatype Nexus Repository Manager", "oss.sonatype.org", $sonaUser, $sonaPass)

7 changes: 7 additions & 0 deletions admin/publish-settings.sbt
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
pgpPassphrase := Some(sys.prop("PGP_PASSPHRASE").toArray)

pgpPublicRing := file("admin/pubring.asc")

pgpSecretRing := file("admin/secring.asc")

credentials += Credentials("Sonatype Nexus Repository Manager", "oss.sonatype.org", sys.prop("SONA_USER"), sys.prop("SONA_PASS"))
26 changes: 13 additions & 13 deletions admin/pubring.asc
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: BCPG v1.49

mQENBFSzVhYBCADp261YcgfWCNZ/IrSpvk1LaqnomeDjktUuCy3LD0WQ/B750vpV
gograxIWDfGK3TaHHdiRTV0OmfhhA6Dv/E8wFiHc1psn8mtaL+tQQcFHCLqBnTEe
/VQXZLrHoFsMSBRrFY0rHZGXtCS0DKIxSeqlba4RH9eS4Q35LzBjMBLKewBCwben
mDMOMxLgTS21xqe7OoyrcQGg3nFPLBMM8hgrqmVH9lYc5c2NuTMSHC4/wUozTwMm
SxmEQ1Ga2lEpVAcaJ6r7bz0+QwX62cMs57nkGuf3SP2D5/+igDkkoVb447wESHHG
s3BZw9ThblHXJOZ5Xb64fvQ3/vCjivLqZIepABEBAAG0NXNjYWxhLWphdmE4LWNv
mQENBFS1xA0BCAC0t2c5MhkWyUbkWsZM4DmIN+/pDjNCr2DNmbIG3gB8i4MI71q/
fj+Ob0lemjJNnNc4ii6+s9RrOcwR1EU4IA8mO79NN+i2yVUhe0LmOWgyfXvG8Qpg
hLmdMrkgOHK0hpWbXJ0i2NGPch4gI6YRJF95yLojz2KENmiYGmSD8p1It06O2824
Xhqc5Cm72/qXvonHP1+MugjiPxmyZN3ajSol0P7tZlgB7ikqpyL3kZXkc162bJ+H
U6y6qUCcQqS5VQ7Fv9bIbTNOjN4ELLJn2ffLVe3ujRG6seioL0MfuQ/gV9IpGcGO
Dew8Xu79QdDyVHQKgDy9N/J276JZ4j9nYCCxABEBAAG0NXNjYWxhLWphdmE4LWNv
bXBhdCA8c2NhbGEtaW50ZXJuYWxzQGdvb2dsZWdyb3Vwcy5jb20+iQEcBBMBAgAG
BQJUs1YWAAoJEF7zF/88US8Xdw4IAJmPcOka4Tc5s5eYAdwZuNOqUiuNO3/9+Za6
tdGZQfQxUVN5PdgXhAGiKfRxrtSTjfzN+O/wiF/7NDqOQXBHNEx53Rzucq770WvL
G5hUwr8MJB577OIyU2CQquslva3h2LbOt8lEHplLy0tI00zm6ueJNmxq36C4Mu3h
l6QMs0zd29OqtUjWpkUNRnz+1HSdhRCPZNhX1bjhRaJARrhUtP24+g3wKgjg3H95
yjPh4951r21w/x7msu+w0vSpdA7j/VJIzql6+2exh14YeLx9AFVDgvkJE6McHXX3
ccr1eQ0FjYpWWUrBMXpS1Pz4SiwXEOOhs1xtsM7fHuikqhkXfHg=
=oZnQ
BQJUtcQNAAoJEGQWNEmlKase8pAH/Rb45Px88u7DDT53DU68zh84oDZLv9i46g7g
16KI97nz17F9OEHdkzNEUA3EgCD1d2k+c/GIdQKg3avVdpNM7krK5SSNgHKcwe/F
0YGMxvh+LgeK1JDuXFbwLJKR+7VIGVKkjw+Z2TC8hZfnD6Qy6c4xkukoBs6yfWQO
tf8gSH6oQox4UIOB/+ADyypl9mnRxgdi1uPvd6UJnL/n9UDE8v1k+8WzO34nTVZr
xWN28pAun5VpLuEq4GAr2JRfRiF+N0hGuS+htiU6hnO81BBK+NusWxI9Aitu8Zyh
eulWpROXvUOw1eJequutgyGwEEQkRi+Yu+2eSM2/EPCWiLXkODk=
=Qro7
-----END PGP PUBLIC KEY BLOCK-----
Binary file modified admin/secring.asc.enc
Binary file not shown.
7 changes: 0 additions & 7 deletions sensitive.sbt.enc

This file was deleted.