ruby 3.3.0 -> 3.3.1 new error "machine stack overflow in critical region (fatal)"

[bf4]

I'm not positive what this is related to, but figured I'd log an issue in case it means something to someone.

Ubuntu Linux, Heroku-20 image

Ruby 3.3.1

Tiny Tds 2.1.7

The only backtrace I have is

gems/tiny_tds-2.1.7/lib/tiny_tds/client.rb:60:in `connect': machine stack overflow in critical region (fatal)

and it's not reproducible every time.

RbConfig::CONFIG

{"DESTDIR"=>"",
 "MAJOR"=>"3",
 "MINOR"=>"3",
 "TEENY"=>"1",
 "PATCHLEVEL"=>"55",
 "INSTALL"=>"/usr/bin/install -c",
 "EXEEXT"=>"",
 "prefix"=>"/app/vendor/ruby-3.3.1",
 "ruby_install_name"=>"ruby",
 "RUBY_INSTALL_NAME"=>"ruby",
 "RUBY_SO_NAME"=>"ruby",
 "exec"=>"exec",
 "ruby_pc"=>"ruby-3.3.pc",
 "CC_WRAPPER"=>"",
 "PACKAGE"=>"ruby",
 "BUILTIN_TRANSSRCS"=>" enc/trans/newline.c",
 "MKMF_VERBOSE"=>"0",
 "MANTYPE"=>"man",
 "vendorarchhdrdir"=>"/app/vendor/ruby-3.3.1/include/ruby-3.3.0/vendor_ruby/x86_64-linux",
 "sitearchhdrdir"=>"/app/vendor/ruby-3.3.1/include/ruby-3.3.0/site_ruby/x86_64-linux",
 "rubyarchhdrdir"=>"/app/vendor/ruby-3.3.1/include/ruby-3.3.0/x86_64-linux",
 "vendorhdrdir"=>"/app/vendor/ruby-3.3.1/include/ruby-3.3.0/vendor_ruby",
 "sitehdrdir"=>"/app/vendor/ruby-3.3.1/include/ruby-3.3.0/site_ruby",
 "rubyhdrdir"=>"/app/vendor/ruby-3.3.1/include/ruby-3.3.0",
 "RUBY_SEARCH_PATH"=>"",
 "UNIVERSAL_INTS"=>"",
 "UNIVERSAL_ARCHNAMES"=>"",
 "configure_args"=>
  " '--disable-install-doc' '--prefix' '/tmp/d20240423-1-qthlgt/prefix' '--enable-load-relative' '--enable-shared' '--enable-yjit'"
,
 "CONFIGURE"=>"configure",
 "vendorarchdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/vendor_ruby/3.3.0/x86_64-linux",
 "vendorlibdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/vendor_ruby/3.3.0",
 "vendordir"=>"/app/vendor/ruby-3.3.1/lib/ruby/vendor_ruby",
 "sitearchdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/site_ruby/3.3.0/x86_64-linux",
 "sitelibdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/site_ruby/3.3.0",
 "sitedir"=>"/app/vendor/ruby-3.3.1/lib/ruby/site_ruby",
 "rubyarchdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/3.3.0/x86_64-linux",
 "rubylibdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/3.3.0",
 "ruby_version"=>"3.3.0",
 "sitearch"=>"x86_64-linux",
 "arch"=>"x86_64-linux",
 "sitearchincludedir"=>"/app/vendor/ruby-3.3.1/include/x86_64-linux",
 "archincludedir"=>"/app/vendor/ruby-3.3.1/include/x86_64-linux",
 "sitearchlibdir"=>"/app/vendor/ruby-3.3.1/lib/x86_64-linux",
 "archlibdir"=>"/app/vendor/ruby-3.3.1/lib/x86_64-linux",
 "libdirname"=>"libdir",
 "RUBY_EXEC_PREFIX"=>"",
 "RUBY_LIB_VERSION"=>"",
 "RUBY_LIB_VERSION_STYLE"=>"3\t/* full */",
 "RI_BASE_NAME"=>"ri",
 "ridir"=>"/app/vendor/ruby-3.3.1/share/ri",
 "rubysitearchprefix"=>"/app/vendor/ruby-3.3.1/lib/ruby/x86_64-linux",
 "rubyarchprefix"=>"/app/vendor/ruby-3.3.1/lib/ruby/x86_64-linux",
 "MAKEFILES"=>"Makefile GNUmakefile",
 "USE_LLVM_WINDRES"=>"",
 "PLATFORM_DIR"=>"",
 "COROUTINE_TYPE"=>"amd64",
 "THREAD_MODEL"=>"pthread",
 "SYMBOL_PREFIX"=>"",
 "EXPORT_PREFIX"=>"",
 "COMMON_HEADERS"=>"",
 "COMMON_MACROS"=>"",
 "COMMON_LIBS"=>"",
 "MAINLIBS"=>"-lz -lrt -lrt -lgmp -ldl -lcrypt -lm -lpthread ",
 "ENABLE_SHARED"=>"yes",
 "DLDSHARED"=>"gcc -shared",
 "DLDLIBS"=>"-lc",
 "SOLIBS"=>"-lz -lrt -lrt -lgmp -ldl -lcrypt -lm -lpthread ",
 "LIBRUBYARG_SHARED"=>"-Wl,-rpath,'${ORIGIN}/../lib' -Wl,-rpath,'${ORIGIN}/../lib' -lruby",
 "LIBRUBYARG_STATIC"=>
  "-Wl,-rpath,'${ORIGIN}/../lib' -Wl,-rpath,'${ORIGIN}/../lib' -lruby-static -lz -lrt -lrt -lgmp -ldl -lcrypt -lm -lpthread ",
 "LIBRUBYARG"=>"-Wl,-rpath,'${ORIGIN}/../lib' -Wl,-rpath,'${ORIGIN}/../lib' -lruby",
 "LIBRUBY"=>"libruby.so.3.3.1",
 "LIBRUBY_ALIASES"=>"libruby.so.3.3 libruby.so",
 "LIBRUBY_SONAME"=>"libruby.so.3.3",
 "LIBRUBY_SO"=>"libruby.so.3.3.1",
 "LIBRUBY_A"=>"libruby-static.a",
 "RUBYW_INSTALL_NAME"=>"",
 "rubyw_install_name"=>"",
 "EXTDLDFLAGS"=>"",
 "EXTLDFLAGS"=>"",
 "strict_warnflags"=>"",
 "warnflags"=>
  "-Wall -Wextra -Wdeprecated-declarations -Wdiv-by-zero -Wduplicated-cond -Wimplicit-function-declaration -Wimplicit-int -Wpointer
-arith -Wwrite-strings -Wold-style-definition -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-log
ical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -Wno-parentheses-eq
uality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-attr
ibute=noreturn -Wunused-variable -Wmisleading-indentation -Wundef",
 "debugflags"=>"-g",
 "optflags"=>"-O3 -fno-fast-math",
 "NULLCMD"=>":",
 "ENABLE_DEBUG_ENV"=>"",
 "DLNOBJ"=>"dln.o",
 "RJIT_SUPPORT"=>"yes",
 "YJIT_OBJ"=>"yjit.o",
 "YJIT_LIBS"=>"yjit/target/release/libyjit.a",
 "CARGO_BUILD_ARGS"=>"",
 "YJIT_SUPPORT"=>"yes",
 "CARGO"=>"",
 "RUSTC"=>"rustc",
 "INSTALL_STATIC_LIBRARY"=>"no",
 "EXECUTABLE_EXTS"=>"",
 "ARCHFILE"=>"",
 "LIBRUBY_RELATIVE"=>"yes",
 "EXTOUT"=>".ext",
 "PREP"=>"miniruby",
 "CROSS_COMPILING"=>"no",
 "TEST_RUNNABLE"=>"yes",
 "rubylibprefix"=>"/app/vendor/ruby-3.3.1/lib/ruby",
 "setup"=>"Setup",
 "SOEXT"=>"so",
 "TRY_LINK"=>"",
 "PRELOADENV"=>"LD_PRELOAD",
 "LIBPATHENV"=>"LD_LIBRARY_PATH",
 "RPATHFLAG"=>" -Wl,-rpath,%1$-s",
 "LIBPATHFLAG"=>" -L%1$-s",
 "LINK_SO"=>"",
 "ADDITIONAL_DLDFLAGS"=>"",
 "ENCSTATIC"=>"",
 "EXTSTATIC"=>"",
 "ASMEXT"=>"S",
 "LIBEXT"=>"a",
 "DLEXT"=>"so",
 "LDSHAREDXX"=>"g++ -shared",
 "LDSHARED"=>"gcc -shared",
 "CCDLFLAGS"=>"-fPIC",
 "STATIC"=>"",
 "ARCH_FLAG"=>"",
 "DLDFLAGS"=>"-Wl,--compress-debug-sections=zlib",
 "ALLOCA"=>"",
 "EGREP"=>"/usr/bin/grep -E",
 "GREP"=>"/usr/bin/grep",
 "dsymutil"=>"",
 "codesign"=>"",
 "cleanlibs"=>"",
 "POSTLINK"=>":",
 "incflags"=>"",
 "WERRORFLAG"=>"-Werror",
 "RUBY_DEVEL"=>"",
 "CHDIR"=>"cd -P",
 "RMALL"=>"rm -fr",
 "RMDIRS"=>"rmdir --ignore-fail-on-non-empty -p",
 "RMDIR"=>"rmdir --ignore-fail-on-non-empty",
 "CP"=>"cp",
 "RM"=>"rm -f",
 "PKG_CONFIG"=>"pkg-config",
 "DOXYGEN"=>"",
 "DOT"=>"",
 "MKDIR_P"=>"/usr/bin/mkdir -p",
 "INSTALL_DATA"=>"/usr/bin/install -c -m 644",
 "INSTALL_SCRIPT"=>"/usr/bin/install -c",
 "INSTALL_PROGRAM"=>"/usr/bin/install -c",
 "SET_MAKE"=>"",
 "LN_S"=>"ln -s",
 "DLLWRAP"=>"",
 "WINDRES"=>"",
 "ASFLAGS"=>"",
 "ARFLAGS"=>"rcD ",
 "try_header"=>"",
 "CC_VERSION_MESSAGE"=>
  "gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0\nCopyright (C) 2019 Free Software Foundation, Inc.\nThis is free software; see the sou
rce for copying conditions.  There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.",
 "CC_VERSION"=>"gcc --version",
 "CSRCFLAG"=>"",
 "COUTFLAG"=>"-o ",
 "OUTFLAG"=>"-o ",
 "CPPOUTFILE"=>"-o conftest.i",
 "GNU_LD"=>"yes",
 "GCC"=>"yes",
 "CPP"=>"gcc -E",
 "CXXFLAGS"=>"",
 "OBJEXT"=>"o",
 "CPPFLAGS"=>"  ",
 "LDFLAGS"=>"-L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,--no-as-needed",
 "CFLAGS"=>
  "-O3 -fno-fast-math -g -Wall -Wextra -Wdeprecated-declarations -Wdiv-by-zero -Wduplicated-cond -Wimplicit-function-declaration -W
implicit-int -Wpointer-arith -Wwrite-strings -Wold-style-definition -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-
type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-comp
at -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute
=format -Wsuggest-attribute=noreturn -Wunused-variable -Wmisleading-indentation -Wundef  -fPIC",
 "STRIP"=>"strip -S -x",
 "RANLIB"=>"gcc-ranlib",
 "OBJDUMP"=>"objdump",
 "OBJCOPY"=>"objcopy",
 "NM"=>"gcc-nm",
 "LD"=>"ld",
 "CXX"=>"g++",
 "AS"=>"as",
 "AR"=>"gcc-ar",
 "CC"=>"gcc",
 "wasmoptflags"=>"",
 "WASMOPT"=>"",
 "target_os"=>"linux",
 "target_vendor"=>"pc",
 "target_cpu"=>"x86_64",
 "target"=>"x86_64-pc-linux",
 "host_os"=>"linux",
 "host_vendor"=>"pc",
 "host_cpu"=>"x86_64",
 "host"=>"x86_64-pc-linux",
 "build_os"=>"linux-gnu",
 "build_vendor"=>"pc",
 "build_cpu"=>"x86_64",
 "build"=>"x86_64-pc-linux-gnu",
 "RUBY_VERSION_NAME"=>"ruby-3.3.0",
 "RUBYW_BASE_NAME"=>"rubyw",
 "RUBY_BASE_NAME"=>"ruby",
 "RUBY_PROGRAM_VERSION"=>"3.3.1",
 "RUBY_API_VERSION"=>"3.3",
 "HAVE_GIT"=>"yes",
 "GIT"=>"git",
 "cxxflags"=>"",
 "cppflags"=>"",
 "cflags"=>
  "-O3 -fno-fast-math -g -Wall -Wextra -Wdeprecated-declarations -Wdiv-by-zero -Wduplicated-cond -Wimplicit-function-declaration -W
implicit-int -Wpointer-arith -Wwrite-strings -Wold-style-definition -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-
type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-comp
at -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute
=format -Wsuggest-attribute=noreturn -Wunused-variable -Wmisleading-indentation -Wundef",
 "MAKEDIRS"=>"/usr/bin/mkdir -p",
 "target_alias"=>"",
 "host_alias"=>"",
 "build_alias"=>"",
 "LIBS"=>"-lm -lpthread ",
 "ECHO_T"=>"",
 "ECHO_N"=>"-n",
 "ECHO_C"=>"",
 "DEFS"=>"",
 "mandir"=>"/app/vendor/ruby-3.3.1/share/man",
 "localedir"=>"/app/vendor/ruby-3.3.1/share/locale",
 "libdir"=>"/app/vendor/ruby-3.3.1/lib",
:
 "psdir"=>"/app/vendor/ruby-3.3.1/share/doc/ruby",
 "pdfdir"=>"/app/vendor/ruby-3.3.1/share/doc/ruby",
 "dvidir"=>"/app/vendor/ruby-3.3.1/share/doc/ruby",
 "htmldir"=>"/app/vendor/ruby-3.3.1/share/doc/ruby",
 "infodir"=>"/app/vendor/ruby-3.3.1/share/info",
 "docdir"=>"/app/vendor/ruby-3.3.1/share/doc/ruby",
 "oldincludedir"=>"/usr/include",
 "includedir"=>"/app/vendor/ruby-3.3.1/include",
 "runstatedir"=>"/app/vendor/ruby-3.3.1/var/run",
 "localstatedir"=>"/app/vendor/ruby-3.3.1/var",
 "sharedstatedir"=>"/app/vendor/ruby-3.3.1/com",
 "sysconfdir"=>"/app/vendor/ruby-3.3.1/etc",
 "datadir"=>"/app/vendor/ruby-3.3.1/share",
 "datarootdir"=>"/app/vendor/ruby-3.3.1/share",
 "libexecdir"=>"/app/vendor/ruby-3.3.1/libexec",
 "sbindir"=>"/app/vendor/ruby-3.3.1/sbin",
 "bindir"=>"/app/vendor/ruby-3.3.1/bin",
 "exec_prefix"=>"/app/vendor/ruby-3.3.1",
 "PACKAGE_URL"=>"",
 "PACKAGE_BUGREPORT"=>"",
 "PACKAGE_STRING"=>"",
 "PACKAGE_VERSION"=>"",
 "PACKAGE_TARNAME"=>"",
 "PACKAGE_NAME"=>"",
 "PATH_SEPARATOR"=>":",
 "SHELL"=>"/bin/bash",
 "UNICODE_VERSION"=>"15.0.0",
 "UNICODE_EMOJI_VERSION"=>"15.0",
 "platform"=>"x86_64-linux",
 "archdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/3.3.0/x86_64-linux",
 "topdir"=>"/app/vendor/ruby-3.3.1/lib/ruby/3.3.0/x86_64-linux"}

[bf4]

possibly related to grpc/grpc#36125

[andyundso]

I updated a couple of apps now to Ruby 3.2.4 and freetds v1.4.12. will let you know if I run into similar issues.

[bf4]

Thanks @andyundso I pointed someone (byroot) to take a quick look at the c-code and they don't think it's related to the same grpc bug. In fact, even though this was a new error for us, we've now seen it again, though only once, since downgrading back to 3.3.0. So, it appears unlikely it's related to the Ruby 3.3.0 change, but it's definitely a failure which points here.. and unfortunately I haven't been able to reproduce on demand. My guess would be it has to do with connection handling and either 1) having the remote close the connection without and tiny_tds not handling it gracefully or 2) the remote server process died and tiny_tds did not handle it gracefully or 3) having the server sent a kill signal and it not being handled gracefully, though this seems less likely in this case.

[andyundso]

judging from the tests, we simulate a slow_close and a timeout using Toxiproxy. more details are documented in #481 and in #445.

Toxiproxy would also support to simulate a TCPRESET - this is something we could add tests for and see if this would trigger the error mentioned by you. and even if not, it is probably still a good idea to test it.

[bf4]

Looking at it a little more, I'm wondering if the overflow may be in result.c in rb_tinytds_result_exec_helper where it loops and checks if not failed, which perhaps continues on in the TDS_DEAD state and should check for dbdead. Does that seem possible? I don't really know C, so it's hard for me to be confident enough to consider a PR or how to test