Remove MiniPortile from extension build

The extension build currently downloads and installs sources for
libiconv, openssl, and freetds regardless of architecture. This is at
odds with the correct procedure of installing these sources via the OS
package manager for the non-binary gem build.

This commit begins to solve this problem in a few ways:

1. It moves the responsibiliy of managing the MiniPortile builds to a
   ports rake task
2. It removes all ports specific logic from the extension build save the
   ability to search the ports directories if they exist (to facilitate
   the binary gem)
3. It adds cross compilation targets for the ports rake tasks to assist
   in the creation of the binary gems.
4. It enhances the rake clobber list to include the ports directories.
5. It removes custom logic for ports directory names and moves back
   towards relying on the MiniPortile defaults as much as possible.

Relates to issue #342

Author: coderjoe

.travis.yml

@@ -4,22 +4,19 @@ services:
   - docker
 env:
   global:
+    - PATH=/opt/local/bin:$PATH
     - TESTOPTS="-v"
     - TINYTDS_UNIT_HOST=localhost
 rvm:
   - 2.1.9
   - 2.2.5
   - 2.3.1
 before_install:
-  - export PATH=/opt/local/bin:$PATH
   - docker info
-  - sudo ./test/bin/setup.sh
   - sudo ./test/bin/install-openssl.sh
-  - openssl version
   - sudo ./test/bin/install-freetds.sh
-  - tsql -C
+  - sudo ./test/bin/setup.sh
 install:
-  - export PATH=/opt/local/bin:$PATH
   - gem install bundler
   - bundle --version
   - bundle install

Gemfile

@@ -3,5 +3,8 @@ gemspec
 
 group :development do
   gem 'rake-compiler-dock', github: 'rake-compiler/rake-compiler-dock'
-  gem 'byebug'
+end
+
+group :test do
+  gem 'minitest'
 end

README.md

@@ -34,7 +34,10 @@ Installing with rubygems should just work. TinyTDS is currently tested on Ruby v
 $ gem install tiny_tds
 ```
 
-If you use Windows, we pre-compile TinyTDS with static versions of FreeTDS and supporting libraries. On all other platforms, we will find these dependencies. It is recommended that you install the latest FreeTDS via your method of choice. For example, here is how to install FreeTDS on Ubuntu. You might also need the `build-essential` and possibly the `libc6-dev` packages.
+If you use Windows, we pre-compile TinyTDS with static versions of FreeTDS and supporting libraries.
+If you're using RubyInstaller the binary gem will require that devkit is installed and in your path to operate properly.
+
+On all other platforms, we will find these dependencies. It is recommended that you install the latest FreeTDS via your method of choice. For example, here is how to install FreeTDS on Ubuntu. You might also need the `build-essential` and possibly the `libc6-dev` packages.
 
 ```shell
 $ apt-get wget
@@ -52,20 +55,8 @@ $ make install
 If none exist, our native extension will use MiniPortile to install any missing dependencies listed above for your specific platform. These dependencies will be built and linked within the installed TinyTDS gem. Please read the MiniPortile and/or Windows sections at the end of this file for advanced configuration options past the following:
 
 ```
---enable-system-freetds / --disable-system-freetds
---enable-system-iconv   / --disable-system-iconv
---enable-system-openssl / --disable-system-openssl
-  Force use of system or builtin freetds/iconv/openssl library.
-  Default is to prefer system libraries and fallback to builtin.
-
 --with-freetds-dir=DIR
   Use the freetds library placed under DIR.
-
---enable-lookup
-  Search for freetds through all paths in the PATH environment variable.
-
---enable-cross-build
-  Do cross-build.
 ```
 
 
@@ -403,19 +394,6 @@ This is possible using FreeTDS version 0.95 or higher. You must use the `use_utf
 The default is true and since FreeTDS v1.0 would do this as well.
 
 
-## Using MiniPortile
-
-MiniPortile is a minimalistic implementation of a port/recipe system. <https://github.com/luislavena/mini_portile>
-
-The TinyTDS project uses MiniPortile so that we can easily install a local version of FreeTDS and supporting libraries to link against when building a test version of TinyTDS. This same system is also used when installing TinyTDS with Rubygems and building native extensions. It is possible to build TinyTDS with a specific version of FreeTDS using the `TINYTDS_FREETDS_VERSION` environment variable. Here are some exampbles of possible values.
-
-```
-$ rake TDSVER='7.1' TINYTDS_FREETDS_VERSION='0.95' -- --disable-system-freetds --disable-system-iconv
-```
-
-To find out more about the FreeTDS release system [visit this thread](http://lists.ibiblio.org/pipermail/freetds/2012q1/027756.html) on their mailing list. You can also browse thier FTP server [ftp://ftp.astron.com/pub/freetds/](ftp://ftp.astron.com/pub/freetds/) for version number strings.
-
-
 ## Compiling Gems for Windows
 
 For the convenience of Windows users, TinyTDS ships pre-compiled gems for Ruby 2.0, 2.1, 2.2, and 2.3 on Windows. In order to generate these gems, [rake-compiler-dock](https://github.com/rake-compiler/rake-compiler-dock) is used. This project provides a [Docker image](https://registry.hub.docker.com/u/larskanis/rake-compiler-dock/) with rvm, cross-compilers and a number of different target versions of Ruby.
@@ -463,7 +441,7 @@ CREATE USER [tinytds] FOR LOGIN [tinytds];
 EXEC sp_addrolemember N'db_owner', N'tinytds';
 ```
 
-From here you can now run the tests. This assumes you have both Ruby & the needed FreeTDS installed.
+From here you can build and run tests against an installed version of FreeTDS.
 
 ```shell
 $ bundle install
@@ -479,13 +457,6 @@ $ rake TINYTDS_UNIT_HOST=mydb.host.net TINYTDS_SCHEMA=sqlserver_azure
 $ rake TINYTDS_UNIT_HOST=mydb.host.net TINYTDS_UNIT_PORT=5000 TINYTDS_SCHEMA=sybase_ase
 ```
 
-If you do not want to use MiniPortile to compile a local project version of FreeTDS and instead use your local system version, use the `TINYTDS_SKIP_PORTS` environment variable. This will ignore any port tasks and will instead build and link to your system's FreeTDS installation as a normal gem install would.
-
-```
-$ rake TINYTDS_SKIP_PORTS=1
-```
-
-
 ## Help & Support
 
 * Github Source: http://github.com/rails-sqlserver/tiny_tds

Rakefile

@@ -1,107 +1,50 @@
 # encoding: UTF-8
+require 'rbconfig'
 require 'rake'
 require 'rake/clean'
-require 'rbconfig'
-require 'rake/testtask'
 require 'rake/extensiontask'
-require 'rubygems/package_task'
 require_relative './ext/tiny_tds/extconsts'
 
-def test_libs
-  %w(lib test)
-end
-
-def test_files
-  if ENV['TEST_FILES']
-    ENV['TEST_FILES'].split(',').map(&:strip).sort
-  else
-    Dir.glob('test/**/*_test.rb').sort
-  end
-end
-
-def add_file_to_gem(spec, relative_path)
-  target_path = File.join gem_build_path(spec), relative_path
-  target_dir = File.dirname(target_path)
-  mkdir_p target_dir
-  rm_f target_path
-  safe_ln relative_path, target_path
-  spec.files += [relative_path]
-end
-
-def gem_build_path(spec)
-  File.join 'pkg', spec.full_name
-end
-
-gemspec = Gem::Specification.load(File.expand_path('../tiny_tds.gemspec', __FILE__))
-
-Rake::TestTask.new do |t|
-  t.libs = test_libs
-  t.test_files = test_files
-  t.verbose = true
-end
-
-Gem::PackageTask.new(gemspec) do |pkg|
-  pkg.need_tar = false
-  pkg.need_zip = false
-end
-
-task :compile
+SPEC = Gem::Specification.load(File.expand_path('../tiny_tds.gemspec', __FILE__))
+GEM_PLATFORM_HOSTS = {
+  'x86-mingw32' => 'i686-w64-mingw32',
+  'x64-mingw32' => 'x86_64-w64-mingw32'
+}
 
-task build: [:clean, :compile]
-task(:build_quietly) { capture_stds { Rake::Task[:build].invoke } }
+# Add our project specific files to clean for a rebuild
+CLEAN.include FileList["{ext,lib}/**/*.{so,#{RbConfig::CONFIG['DLEXT']},o}"],
+  FileList["exe/*"]
 
-task default: [:build, :test]
+# Clobber all our temp files and ports files including .install files
+# and archives
+CLOBBER.include FileList["tmp/**/*"],
+  FileList["ports/**/*"].exclude(%r{^ports/archives})
 
 Dir['tasks/*.rake'].sort.each { |f| load f }
 
-Rake::ExtensionTask.new('tiny_tds', gemspec) do |ext|
+Rake::ExtensionTask.new('tiny_tds', SPEC) do |ext|
   ext.lib_dir = 'lib/tiny_tds'
   ext.cross_compile = true
-  ext.cross_platform = ['x86-mingw32', 'x64-mingw32']
-  ext.cross_config_options += %w(--disable-lookup --enable-cross-build)
+  ext.cross_platform = GEM_PLATFORM_HOSTS.keys
+
   # Add dependent DLLs to the cross gems
   ext.cross_compiling do |spec|
-    platform_host_map =  {
-      'x86-mingw32' => 'i686-w64-mingw32',
-      'x64-mingw32' => 'x86_64-w64-mingw32'
-    }
+    platform_host_map = GEM_PLATFORM_HOSTS
     gemplat = spec.platform.to_s
     host = platform_host_map[gemplat]
-    dlls = [
-      "libcrypto32-#{OPENSSL_VERSION}-#{host}.dll",
-      "libssl32-#{OPENSSL_VERSION}-#{host}.dll",
-      'libiconv-2.dll',
-      'libsybdb-5.dll'
-    ]
+
     # We don't need the sources in a fat binary gem
     spec.files = spec.files.reject { |f| f =~ %r{^ports\/archives/} }
-    spec.files += dlls.map { |dll| "ports/#{host}/bin/#{File.basename(dll)}" }
+
+    # Make sure to include the ports binaries and libraries
+    spec.files += FileList["ports/#{host}/**/**/{bin,lib}/*"].exclude do |f|
+      File.directory? f
+    end
+
     spec.files += Dir.glob('exe/*')
   end
 end
 
-# Bundle the freetds sources to avoid download while gem install.
-task gem_build_path(gemspec) do
-  add_file_to_gem gemspec, "ports/archives/freetds-#{FREETDS_VERSION}.tar.bz2"
-end
-
-desc 'Build the windows binary gems per rake-compiler-dock'
-task 'gem:windows' do
-  require 'rake_compiler_dock'
-  RakeCompilerDock.sh <<-EOT
-    bundle && rake cross native gem RUBY_CC_VERSION=2.4.0:2.3.0:2.2.2:2.1.6:2.0.0 CFLAGS="-Wall"
-  EOT
-end
+task build: [:clean, :compile]
+task default: [:build, :test]
 
-def capture_stds
-  pstdout = $stdout
-  $stdout = StringIO.new
-  pstderr = $stderr
-  $stderr = StringIO.new
-  yield
-  $stdout.string
-  $stderr.string
-ensure
-  $stdout = pstdout
-  $stderr = pstderr
-end

appveyor.yml

@@ -9,12 +9,15 @@ skip_branch_with_pr: true
 matrix:
   fast_finish: true
 install:
+  # Output debugging info
   - ps: Update-AppveyorBuild -Version "$(Get-Content $env:appveyor_build_folder\VERSION).$env:appveyor_build_number"
   - perl --version
   - ruby --version
   - gem --version
+
+  # Set up project prerequisits
   - bundle install
-  - bundle exec rake build
+  - bundle exec rake ports
 build: off
 branches:
   except:
@@ -26,13 +29,13 @@ test_script:
   - ps: Start-Service 'MSSQL$SQL2014'
   - timeout /t 4 /nobreak > NUL
   - sqlcmd -S ".\SQL2014" -U sa -P Password12! -i %APPVEYOR_BUILD_FOLDER%\test\appveyor\dbsetup.sql
-  - bundle exec rake TINYTDS_UNIT_HOST_TEST=localhost TINYTDS_UNIT_DATASERVER="localhost\SQL2014" TINYTDS_SCHEMA=sqlserver_2014 TDSVER=7.1
-  - bundle exec rake TINYTDS_UNIT_HOST_TEST=localhost TINYTDS_UNIT_DATASERVER="localhost\SQL2014" TINYTDS_SCHEMA=sqlserver_2014
+  - bundle exec rake ports build test TINYTDS_UNIT_HOST_TEST=localhost TINYTDS_UNIT_DATASERVER="localhost\SQL2014" TINYTDS_SCHEMA=sqlserver_2014 TDSVER=7.1
+  - bundle exec rake ports build test TINYTDS_UNIT_HOST_TEST=localhost TINYTDS_UNIT_DATASERVER="localhost\SQL2014" TINYTDS_SCHEMA=sqlserver_2014
   - ps: Stop-Service 'MSSQL$SQL2014'
   - ps: Start-Service 'MSSQL$SQL2012SP1'
   - timeout /t 4 /nobreak > NUL
   - sqlcmd -S ".\SQL2012SP1" -U sa -P Password12! -i %APPVEYOR_BUILD_FOLDER%\test\appveyor\dbsetup.sql
-  - bundle exec rake TINYTDS_UNIT_HOST_TEST=localhost TINYTDS_UNIT_DATASERVER="localhost\SQL2012SP1" TINYTDS_SCHEMA=sqlserver_2014
+  - bundle exec rake ports build test TINYTDS_UNIT_HOST_TEST=localhost TINYTDS_UNIT_DATASERVER="localhost\SQL2012SP1" TINYTDS_SCHEMA=sqlserver_2014
 environment:
   CI_AZURE_HOST:
     secure: 8ydpYysZYKEBKvp6plKlYfepH98/zAuT27FFCaJ9Sss=

circle.yml

@@ -13,13 +13,11 @@ machine:
 
 dependencies:
   override:
+    - sudo ./test/bin/install-openssl.sh
+    - sudo ./test/bin/install-freetds.sh
     - rvm-exec 2.1.9 bundle install
     - rvm-exec 2.2.5 bundle install
     - rvm-exec 2.3.1 bundle install
-    - sudo ./test/bin/install-openssl.sh
-    - openssl version
-    - sudo ./test/bin/install-freetds.sh
-    - tsql -C
 
 database:
   post: