Skip to content

Update cross build, Travis and Appveyor-CI #456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ env:
- TESTOPTS="-v"
- TINYTDS_UNIT_HOST=localhost
rvm:
- 2.3.8
- 2.4.5
- 2.5.3
- 2.6.1
- 2.7.0
before_install:
- docker info
- sudo ./test/bin/install-openssl.sh
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
## (unreleased)

* Removed old/unused appveyor config
* Remove old Rubies from CI & cross compile list
* Add Ruby 2.6 and 2.7 to the cross compile list

## 2.1.2

* Use Kernel.BigDecimal vs BigDecimal.new. Fixes #409.
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,9 @@ $ apt-get install wget
$ apt-get install build-essential
$ apt-get install libc6-dev

$ wget http://www.freetds.org/files/stable/freetds-1.1.6.tar.gz
$ tar -xzf freetds-1.1.6.tar.gz
$ cd freetds-1.1.6
$ wget http://www.freetds.org/files/stable/freetds-1.1.24.tar.gz
$ tar -xzf freetds-1.1.24.tar.gz
$ cd freetds-1.1.24
$ ./configure --prefix=/usr/local --with-tdsver=7.3
$ make
$ make install
Expand Down Expand Up @@ -401,7 +401,7 @@ The default is true and since FreeTDS v1.0 would do this as well.

## Compiling Gems for Windows

For the convenience of Windows users, TinyTDS ships pre-compiled gems for Ruby 2.0, 2.1, 2.2, and 2.3 on Windows. In order to generate these gems, [rake-compiler-dock](https://github.com/rake-compiler/rake-compiler-dock) is used. This project provides a [Docker image](https://registry.hub.docker.com/u/larskanis/rake-compiler-dock/) with rvm, cross-compilers and a number of different target versions of Ruby.
For the convenience of Windows users, TinyTDS ships pre-compiled gems for supported versions of Ruby on Windows. In order to generate these gems, [rake-compiler-dock](https://github.com/rake-compiler/rake-compiler-dock) is used. This project provides several [Docker images](https://registry.hub.docker.com/u/larskanis/) with rvm, cross-compilers and a number of different target versions of Ruby.

Run the following rake task to compile the gems for Windows. This will check the availability of [Docker](https://www.docker.com/) (and boot2docker on Windows or OS-X) and will give some advice for download and installation. When docker is running, it will download the docker image (once-only) and start the build:

Expand Down
19 changes: 11 additions & 8 deletions appveyor.yml
Original file line number Diff line number Diff line change
@@ -1,21 +1,27 @@
init:
- SET PATH=C:\Ruby%ruby_version%\bin;%PATH%
- SET PATH=C:\MinGW\msys\1.0\bin;%PATH%
- SET RAKEOPT=-rdevkit
- SET TESTOPTS='-v'
- SET MAKE=make V=1
- SET MAKE=make V=1 -j2
clone_depth: 5
skip_tags: true
skip_branch_with_pr: true
matrix:
fast_finish: true
fast_finish: false
install:
# Output debugging info
- ps: Update-AppveyorBuild -Version "$(Get-Content $env:appveyor_build_folder\VERSION).$env:appveyor_build_number"
- ps: |
if ($env:ruby_version -like "*head*") {
$(new-object net.webclient).DownloadFile("https://github.com/oneclick/rubyinstaller2/releases/download/rubyinstaller-head/rubyinstaller-$env:ruby_version.exe", "$pwd/ruby-setup.exe")
cmd /c ruby-setup.exe /verysilent /dir=C:/Ruby$env:ruby_version
}
- perl --version
- ruby --version
- gem --version

# prevent freetds to link to wrong ws2_32 lib on i686-w64-mingw32
- c:/msys64/usr/bin/rm /usr/lib/w32api/libws2_32.a
# Set up project prerequisits
- bundle install
- bundle exec rake ports
Expand Down Expand Up @@ -43,13 +49,10 @@ environment:
TINYTDS_UNIT_AZURE_PASS:
secure: fYKSKV4v+36OFQp2nZdX4DfUpgmy5cm0wuR73cgdmEk=
matrix:
- ruby_version: "23-x64"
- ruby_version: "23"
- ruby_version: "24-x64"
- ruby_version: "24"
- ruby_version: "25-x64"
- ruby_version: "25"
- ruby_version: "26-x64"
- ruby_version: "26"
- ruby_version: "head-x64"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of the changes to this matrix list are surprising:

  • why remove 24-x64, 25, and 26-x64? If it's for overall speed concerns, I'd think that the ongoing verification is preferable to saving a few minutes per PR
  • any limit against adding ruby 2.7?
  • personally, I don't think testing against head builds is necessary, but I'm not opposed to it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed some Ruby versions because one CI run takes almost two hours otherwise and testing all Ruby versions on add architectures has little value. Errors that affect only a single Ruby version on a single architecture are very rare.

Ruby-2.7 is not yet available in appveyor.

Testing on Ruby head is good to get early hints about incompatibilities or deprecations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI run takes almost two hours

Yeouch, I didn't know it was that bad.... I'm on board with slimming down the list then. Also this & the fact that it's so slow is making me want to revisit our overall CI config (later....)

on_failure:
- find -name config.log | xargs cat
- find -name compile.log | xargs cat
30 changes: 17 additions & 13 deletions ext/tiny_tds/extconf.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,35 +22,39 @@ def do_help

# Make sure to check the ports path for the configured host
host = RbConfig::CONFIG['host']
project_dir = File.join(['..']*4)
project_dir = File.expand_path("../../..", __FILE__)
freetds_ports_dir = File.join(project_dir, 'ports', host, 'freetds', FREETDS_VERSION)
freetds_ports_dir = File.expand_path(freetds_ports_dir)

# Add all the special path searching from the original tiny_tds build
# order is important here! First in, last searched.
# order is important here! First in, first searched.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious for my own understanding, why was it this order worked in the past but is now a problem?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously we used a side effect of calling dir_config multiple times. This doesn't work any longer on Ruby-2.7.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks for explaining. Is it odd in retrospect that we were calling dir_config in a loop rather than setting up the directories in the right order?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so.

DIRS = %w(
/usr/local
/opt/local
/usr/local
)

# Add the ports directory if it exists for local developer builds
DIRS.unshift(freetds_ports_dir) if File.directory?(freetds_ports_dir)

# Grab freetds environment variable for use by people on services like
# Heroku who they can't easily use bundler config to set directories
DIRS.push(ENV['FREETDS_DIR']) if ENV.has_key?('FREETDS_DIR')

# Add the ports directory if it exists for local developer builds
DIRS.push(freetds_ports_dir) if File.directory?(freetds_ports_dir)
DIRS.unshift(ENV['FREETDS_DIR']) if ENV.has_key?('FREETDS_DIR')

# Add the search paths for freetds configured above
DIRS.each do |path|
idir = "#{path}/include"
ldirs = DIRS.flat_map do |path|
ldir = "#{path}/lib"
[ldir, "#{ldir}/freetds"]
end

dir_config('freetds',
[idir, "#{idir}/freetds"],
[ldir, "#{ldir}/freetds"]
)
idirs = DIRS.flat_map do |path|
idir = "#{path}/include"
[idir, "#{idir}/freetds"]
end

puts "looking for freetds headers in the following directories:\n#{idirs.map{|a| " - #{a}\n"}.join}"
puts "looking for freetds library in the following directories:\n#{ldirs.map{|a| " - #{a}\n"}.join}"
dir_config('freetds', idirs, ldirs)

have_dependencies = [
find_header('sybfront.h'),
find_header('sybdb.h'),
Expand Down
4 changes: 2 additions & 2 deletions ext/tiny_tds/extconsts.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
ICONV_VERSION = ENV['TINYTDS_ICONV_VERSION'] || "1.15"
ICONV_SOURCE_URI = "http://ftp.gnu.org/pub/gnu/libiconv/libiconv-#{ICONV_VERSION}.tar.gz"

OPENSSL_VERSION = ENV['TINYTDS_OPENSSL_VERSION'] || '1.1.0j'
OPENSSL_VERSION = ENV['TINYTDS_OPENSSL_VERSION'] || '1.1.1d'
OPENSSL_SOURCE_URI = "https://www.openssl.org/source/openssl-#{OPENSSL_VERSION}.tar.gz"

FREETDS_VERSION = ENV['TINYTDS_FREETDS_VERSION'] || "1.00.27"
FREETDS_VERSION = ENV['TINYTDS_FREETDS_VERSION'] || "1.1.24"
FREETDS_VERSION_INFO = Hash.new { |h,k|
h[k] = {files: "http://www.freetds.org/files/stable/freetds-#{k}.tar.bz2"}
}
Expand Down
2 changes: 1 addition & 1 deletion tasks/native_gem.rake
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ task 'gem:windows' => ['ports:cross'] do
build = ['bundle']

# and finally build the native gem
build << 'rake cross native gem RUBY_CC_VERSION=2.5.0:2.4.0:2.3.0:2.2.2:2.1.6:2.0.0 CFLAGS="-Wall" MAKE="make -j`nproc`"'
build << 'rake cross native gem RUBY_CC_VERSION=2.7.0:2.6.0:2.5.0:2.4.0 CFLAGS="-Wall" MAKE="make -j`nproc`"'

RakeCompilerDock.sh build.join(' && ')
end
12 changes: 5 additions & 7 deletions tasks/ports.rake
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@ require_relative 'ports/openssl'
require_relative 'ports/freetds'
require_relative '../ext/tiny_tds/extconsts'

OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE if defined? OpenSSL

namespace :ports do
openssl = Ports::Openssl.new(OPENSSL_VERSION)
libiconv = Ports::Libiconv.new(ICONV_VERSION)
freetds = Ports::Freetds.new(FREETDS_VERSION)

directory "ports"
CLEAN.include "ports/*mingw32*"
CLEAN.include "ports/*.installed"

task :openssl, [:host] do |task, args|
args.with_defaults(host: RbConfig::CONFIG['host'])
Expand Down Expand Up @@ -71,15 +71,13 @@ namespace :ports do
task 'cross' do
require 'rake_compiler_dock'

# make sure to install our bundle
build = ['bundle']

# build the ports for all our cross compile hosts
GEM_PLATFORM_HOSTS.each do |gem_platform, host|
# make sure to install our bundle
build = ['bundle']
build << "rake ports:compile[#{host}] MAKE='make -j`nproc`'"
RakeCompilerDock.sh build.join(' && '), platform: gem_platform
end

RakeCompilerDock.sh build.join(' && ')
end
end

Expand Down
20 changes: 2 additions & 18 deletions tasks/ports/openssl.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,27 +27,11 @@ def install

private

def execute(action, command, options={})
# OpenSSL Requires Perl >= 5.10, while the Ruby devkit uses MSYS1 with Perl 5.8.8.
# To overcome this, prepend Git's usr/bin to the PATH.
# It has MSYS2 with a recent version of perl.
prev_path = ENV['PATH']
if host =~ /mingw/ && IO.popen(["perl", "-e", "print($])"], &:read).to_f < 5.010
git_perl = 'C:/Program Files/Git/usr/bin'
if File.directory?(git_perl)
ENV['PATH'] = "#{git_perl}#{File::PATH_SEPARATOR}#{ENV['PATH']}"
ENV['PERL'] = 'perl'
end
end

super
ENV['PATH'] = prev_path
end

def configure_defaults
opts = [
'shared',
target_arch
target_arch,
"--openssldir=#{path}",
]

if cross_build?
Expand Down
4 changes: 2 additions & 2 deletions tiny_tds.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ Gem::Specification.new do |s|
s.required_ruby_version = '>= 2.0.0'
s.metadata['msys2_mingw_dependencies'] = 'freetds'
s.add_development_dependency 'mini_portile2', '~> 2.0'
s.add_development_dependency 'rake', '~> 10.4'
s.add_development_dependency 'rake', '~> 13.0'
s.add_development_dependency 'rake-compiler', '~> 1.0'
s.add_development_dependency 'rake-compiler-dock', '~> 0.6.3'
s.add_development_dependency 'rake-compiler-dock', '~> 1.0'
s.add_development_dependency 'minitest', '~> 5.6'
s.add_development_dependency 'connection_pool', '~> 2.2'
end