TinyTds with Docker multistage builds

[mlh758]

Environment

Operating System

Please describe your operating system and version here.
If unsure please try the following from the command line:

Linux 4124334e150d 4.9.125-linuxkit #1 SMP Fri Sep 7 08:20:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Ubuntu 18.04

TinyTDS Version and Information

tiny_tds (2.1.2)

FreeTDS Version

freetds-1.00.92

Description

I am building a Rails application with a multi-stage Docker build. My Dockerfile looks something like this, although we have a base image that isn't quite ubuntu 18.04 that adds some Oracle libraries we need, Passenger, and sets up nginx to run as a non-root user:

# Build free tds and put it somewhere easy to get to
FROM ubuntu:18.04 as tds_builder
RUN apt-get update && apt-get install -y wget gcc make && \
  wget ftp://ftp.freetds.org/pub/freetds/stable/freetds-1.00.92.tar.gz && \
  tar -xzf freetds-1.00.92.tar.gz && \
  cd freetds-1.00.92 && \
  ./configure --prefix=/home/deploy/tds && \
  make && make install && make clean

# Build all our gems
FROM ubuntu:18.04 as builder
ENV PATH="${PATH}:/home/.gem/bin"
COPY --from=tds_builder /home/deploy/tds/lib /usr/local/lib
COPY --from=tds_builder /home/deploy/tds/include/* /usr/local/include/
COPY --from=tds_builder /home/deploy/tds/etc/* /etc/freetds/
RUN apt-get update && apt-get install -y curl links g++ git ruby-dev zlib1g-dev make && \
  chown deploy:www-data /var/www
USER deploy
RUN mkdir /var/www/app && \
  git clone <ruby-app-git-url> --depth=1 --branch stable --single-branch /var/www/app && \
  cd /var/www/app && \
  gem install rake --no-document --no-ri && \
  gem install bundler --no-document --no-ri &&\
  bundle install --without development test

# Set up final container
FROM  ubuntu:18.04 as final
ENV PATH="${PATH}:${ORACLE_BASE}:/home/.gem/bin"
COPY --from=builder --chown=deploy:www-data /home/.gem /home/.gem
COPY --from=builder --chown=deploy:www-data /var/www/app /var/www/app
COPY --from=builder --chown=deploy:www-data /etc/default/nginx /etc/default/nginx
COPY --from=builder /usr/local/lib /usr/local/lib
COPY --from=builder /usr/local/include/* /usr/local/include/
COPY --from=builder /etc/freetds /etc/freetds

EXPOSE 8080
ENTRYPOINT [ "start.sh" ]

We have several applications we do this with, but the ones using TinyTDS encounter the following error: Error: The application encountered the following error: libsybdb.so.5: cannot open shared object file: No such file or directory - /home/.gem/gems/tiny_tds-2.1.2/lib/tiny_tds/tiny_tds.so (LoadError)

It seems like TinyTDS relies on something outside of the gem directory, and in addition to the free tds lib/include files when it builds its native extension that gets lost in the copy. None of the other gems we include have had this problem since we use this builder pattern successfully with our other apps.

If I add make and gcc to the final container and do a gem install tiny_tds everything starts to work again so there's definitely something extra being built that I'm not finding. Any advice?

[coderjoe]

RUN apt-get update && apt-get install -y wget gcc make &&
wget ftp://ftp.freetds.org/pub/freetds/stable/freetds-1.00.92.tar.gz &&
tar -xzf freetds-1.00.92.tar.gz &&
cd freetds-1.00.92 &&
./configure --prefix=/home/deploy/tds &&
make && make install && make clean

It looks like you are installing FreeTDS to a nonstandard library location. libsybdb.so.5 is one of FreeTDS's shared libraries that TinyTDS relies upon.

If I add make and gcc to the final container and do a gem install tiny_tds everything starts to work again so there's definitely something extra being built that I'm not finding.

Yes. That's because TinyTDS will download and build its own copy of FreeTDS if it can't find it on your system. It can't find it on your system because /home/deploy/tds is not a known library directory.

Any advice?

Try it without the --prefix=/home/deploy/tds on the FreeTDS build. This is not a tiny-tds problem. It just can't find the library you installed because it is not installed to a valid library prefix that your container is using.

[mlh758]

It looks like you are installing FreeTDS to a nonstandard library location.

That's deliberate. If you look at the Dockerfile this is built in 3 stages. The first stage builds only FreeTDS and places the output into a friendly location. The second stage copies all the FreeTDS libraries and include files into its container (and places them in /usr/local which is the standard location) and builds all the gems. Here, TinyTDS builds successfully and if I stop the build at stage 2 I can launch a rails console and connect to a database with TinyTDS.

The problem is that when I copy all the built gems and app code into the final container, along with the FreeTDS libraries, TinyTDS seems unable to locate them and execute. Reinstalling the gem causes it to find them and work from that point on.

Yes. That's because TinyTDS will download and build its own copy of FreeTDS if it can't find it on your system

Newer versions of TinyTDS no longer seem to do this. If I don't copy the FreeTDS libraries into the final container TinyTDS fails to reinstall at all.

[mlh758]

ldd tiny_tds.so

linux-vdso.so.1 (0x00007ffe692b6000)
	libruby-2.5.so.2.5 => /usr/lib/x86_64-linux-gnu/libruby-2.5.so.2.5 (0x00007f20ba93a000)
	libsybdb.so.5 => not found
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f20ba549000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f20ba32a000)
	libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f20ba0a9000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f20b9ea5000)
	libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f20b9c6d000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f20b98cf000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f20baff2000)

It's definitely not finding the library

ls -l /usr/local/lib

-rw-r--r-- 1 root root 3066946 Oct 30 12:33 libct.a
-rwxr-xr-x 1 root root     927 Oct 30 12:33 libct.la
lrwxrwxrwx 1 root root      14 Oct 30 12:33 libct.so -> libct.so.4.0.0
lrwxrwxrwx 1 root root      14 Oct 30 12:33 libct.so.4 -> libct.so.4.0.0
-rwxr-xr-x 1 root root 1538032 Oct 30 12:33 libct.so.4.0.0
-rw-r--r-- 1 root root 3544828 Oct 30 12:33 libsybdb.a
-rwxr-xr-x 1 root root     948 Oct 30 12:33 libsybdb.la
lrwxrwxrwx 1 root root      17 Oct 30 12:33 libsybdb.so -> libsybdb.so.5.1.0
lrwxrwxrwx 1 root root      17 Oct 30 12:33 libsybdb.so.5 -> libsybdb.so.5.1.0
-rwxr-xr-x 1 root root 1800704 Oct 30 12:33 libsybdb.so.5.1.0

But it's definitely there, and in the same place as it was before in the second stage build container.

I'm confident that /usr/local is a valid place for the FreeTDS libraries since that's where they live in our Chef configured servers after installing FreeTDS with a plain ./configure && make... and it's where TinyTDS is looking here

[coderjoe]

No, you are absolutely correct. I skimmed the Dockerfile too quickly.
It also won't download FreeTDS, I was confusing it with the the fat gem build process.

Please do investigate. If you submit a PR please mention this issue number and I'll review it as soon as I can.

[mlh758]

I added /usr/local/lib to LD_LIBRARY_PATH and made sure to run ldconfig in the build process and now ldd shows it all linked up:

ldd tiny_tds.so
	linux-vdso.so.1 (0x00007fffbc3e3000)
	libruby-2.5.so.2.5 => /usr/lib/x86_64-linux-gnu/libruby-2.5.so.2.5 (0x00007fdac02e6000)
	libsybdb.so.5 => /usr/local/lib/libsybdb.so.5 (0x00007fdac007c000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fdabfc8b000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fdabfa6c000)
	libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007fdabf7eb000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fdabf5e7000)
	libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fdabf3af000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fdabf011000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fdac099e000)

This allowed me to launch a Rails console. To get it working in nginx with passenger I also had to copy that environment variable into /etc/nginx/default but any nginx config location where you can set environment variables for your web server should work.

I think the reason this happened is that the MakeMakeFile command stuff lets TinyTDS look wherever it wants to do the building and linking for native extensions, but FreeTDS plops itself into /usr/local/lib which isn't a place ldconfig would normally look. It works great in the second stage, but in the final stage the links are broken since ldconfig didn't know to look in this extra location.

[mlh758]

I'm good to close this now, would you be open to me opening a PR to the README that explains this a little bit somewhere?

[metaskills]

README PRs always appreciated!