A large `RUST_THREADS` value causes an abort

[huonw]

$ cat hello-world.rs
fn main() { println("hello world"); }
$ rustc hello-world.rs
$ RUST_THREADS=254 hello-world
hello world
$ RUST_THREADS=255 hello-world
Aborted

Backtrace (this appears to change occasionally, as does the exact RUST_THREADS value at which the crash happens):

#0  0x00007ffff67541e5 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff6757398 in __GI_abort () at abort.c:90
#2  0x00007ffff7431da7 in uv__loop_init (loop=loop@entry=0x7ffff5412900, default_loop=default_loop@entry=0)
    at ../../../../src/libuv/src/unix/loop.c:75
#3  0x00007ffff742e97d in uv_loop_new () at ../../../../src/libuv/src/unix/core.c:249
#4  0x00007ffff77f8bf9 in rt::run_::_82e8c355ab8d949f::_0$x2e8$x2dpre ()
   from /home/huon/projects/test-rust/../../rust/x86_64-unknown-linux-gnu/stage2/lib/rustc/x86_64-unknown-linux-gnu/lib/libstd-6c65cf4b443341b1-0.8-pre.so
#5  0x00007ffff779b0ed in unstable::lang::start::_76d6c774aa357c7a::_0$x2e8$x2dpre ()
   from /home/huon/projects/test-rust/../../rust/x86_64-unknown-linux-gnu/stage2/lib/rustc/x86_64-unknown-linux-gnu/lib/libstd-6c65cf4b443341b1-0.8-pre.so
#6  0x0000000000400a7b in main ()

There's only one thread.

(RUST_THREADS=<large> rustc works too, but gives a less-nice backtrace.)

[alexcrichton]

I'm not entirely sure that there's a whole lot we can do about this. I believe that the reason for this is that libuv opens a bunch of file descriptors (for communication and such), and then we create even more by making lots of async handles.

We might be able to reduce the number of async handles we make, but otherwise I think this is just an issue of raising the file descriptor limits.

I'm not closing this though, because I would like to investigate if we can reduce the number of file descriptors we force libuv to generate, this is just a hunch of mine.

[alexcrichton]

OK, I looked a bit more into this. This all from OSX's point of view, the reality is probably different for other platforms.

For me, the maximum file descriptor limit is 256. I was able to create ~128 pipes (2 per syscall) and then pipe started failing. It also turns out that libuv starts up 5 file descriptors per loop. Each event loop looks like it has one "event file descriptor", one pair of pipes for a signal handler, and one pair of pipes for an async watcher. All of this initialization always happens, it's not configurable. What this means is that you can create ~50 threads, at this rate (at least on my system), which is we we get an abort after creating the 52nd thread.

With this information in mind, I don't think that there's really anything we can do to fix this. The best action we can take is to alert the libuv folks to see if they can optimize the number of file descriptors they make per loop. In reality, I think this just means that the file descriptor limits need to be raised if you want this many threads. I suppose the other thing we could do is fail a bit more gracefully, but again this is kinda up to libuv b/c the abort is happening inside of libuv, not inside of our code.

I'm closing this because I don't think that there's enough actionable here by us. It'd be nice if we could spawn a million threads, but rust is optimized for spawning millions of tasks, not threads.