Could not start fs_event for path {PATH} :EMFILE

[milkias17]

Description

If filesystem watchers is enables upon startup, and toggling nvim-tree, nvim-tree sends warning notifications saying Could not start the fs_event watcher.
This does not happen when filesystem_watchers is disabled.

Since a log might be helpful, here is the log file generated:

[2023-05-27 14:40:03] [config] default config + user
{
  actions = {
    change_dir = {
      enable = true,
      global = false,
      restrict_above_cwd = false
    },
    expand_all = {
      exclude = {},
      max_folder_discovery = 300
    },
    file_popup = {
      open_win_config = {
        border = "shadow",
        col = 1,
        relative = "cursor",
        row = 1,
        style = "minimal"
      }
    },
    open_file = {
      quit_on_open = false,
      resize_window = true,
      window_picker = {
        chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890",
        enable = true,
        exclude = {
          buftype = { "nofile", "terminal", "help" },
          filetype = { "notify", "packer", "qf", "diff", "fugitive", "fugitiveblame" }
        },
        picker = "default"
      }
    },
    remove_file = {
      close_window = true
    },
    use_system_clipboard = true
  },
  auto_reload_on_write = true,
  diagnostics = {
    debounce_delay = 50,
    enable = true,
    icons = {
      error = "",
      hint = "",
      info = "",
      warning = ""
    },
    severity = {
      max = 1,
      min = 4
    },
    show_on_dirs = false,
    show_on_open_dirs = true
  },
  disable_netrw = false,
  experimental = {},
  filesystem_watchers = {
    debounce_delay = 50,
    enable = true,
    ignore_dirs = {}
  },
  filters = {
    custom = {},
    dotfiles = false,
    exclude = {},
    git_clean = false,
    no_buffer = false
  },
  git = {
    enable = true,
    ignore = true,
    show_on_dirs = true,
    show_on_open_dirs = true,
    timeout = 400
  },
  hijack_cursor = true,
  hijack_directories = {
    auto_open = true,
    enable = true
  },
  hijack_netrw = true,
  hijack_unnamed_buffer_when_opening = false,
  live_filter = {
    always_show_folders = true,
    prefix = "[FILTER]: "
  },
  log = {
    enable = true,
    truncate = false,
    types = {
      all = true,
      config = false,
      copy_paste = false,
      dev = false,
      diagnostics = false,
      git = false,
      profile = false,
      watcher = false
    }
  },
  modified = {
    enable = true,
    show_on_dirs = true,
    show_on_open_dirs = true
  },
  notify = {
    threshold = 2
  },
  on_attach = "default",
  prefer_startup_root = false,
  reload_on_bufenter = false,
  remove_keymaps = false,
  renderer = {
    add_trailing = false,
    full_name = false,
    group_empty = false,
    highlight_git = false,
    highlight_modified = "none",
    highlight_opened_files = "none",
    icons = {
      git_placement = "before",
      glyphs = {
        bookmark = "󰆤",
        default = "",
        folder = {
          arrow_closed = "",
          arrow_open = "",
          default = "",
          empty = "",
          empty_open = "",
          open = "",
          symlink = "",
          symlink_open = ""
        },
        git = {
          deleted = "",
          ignored = "◌",
          renamed = "➜",
          staged = "✓",
          unmerged = "",
          unstaged = "✗",
          untracked = "★"
        },
        modified = "●",
        symlink = ""
      },
      modified_placement = "after",
      padding = " ",
      show = {
        file = true,
        folder = true,
        folder_arrow = true,
        git = true,
        modified = true
      },
      symlink_arrow = " ➛ ",
      webdev_colors = true
    },
    indent_markers = {
      enable = true,
      icons = {
        bottom = "─",
        corner = "└",
        edge = "│",
        item = "│",
        none = " "
      },
      inline_arrows = true
    },
    indent_width = 2,
    root_folder_label = ":~:s?$?/..?",
    special_files = { "Cargo.toml", "Makefile", "README.md", "readme.md" },
    symlink_destination = true
  },
  respect_buf_cwd = false,
  root_dirs = {},
  select_prompts = false,
  sort_by = "name",
  sync_root_with_cwd = true,
  system_open = {
    args = {},
    cmd = ""
  },
  tab = {
    sync = {
      close = false,
      ignore = {},
      open = false
    }
  },
  trash = {
    cmd = "gio trash"
  },
  ui = {
    confirm = {
      remove = true,
      trash = true
    }
  },
  update_focused_file = {
    enable = false,
    ignore_list = {},
    update_root = false
  },
  view = {
    centralize_selection = false,
    cursorline = true,
    debounce_delay = 15,
    float = {
      enable = false,
      open_win_config = {
        border = "rounded",
        col = 1,
        height = 30,
        relative = "editor",
        row = 1,
        width = 30
      },
      quit_on_focus_loss = true
    },
    hide_root_folder = false,
    mappings = {
      custom_only = false,
      list = {}
    },
    number = true,
    preserve_window_proportions = false,
    relativenumber = true,
    side = "left",
    signcolumn = "yes",
    width = "25%"
  }
}
[2023-05-27 14:40:03] [diagnostics] setup
[2023-05-27 14:40:03] [profile] START core init /home/mike/Dev/projects/anony_counseling
[2023-05-27 14:40:03] [watcher] Watcher:new '/home/mike/Dev/projects/anony_counseling' nil
[2023-05-27 14:40:03] [watcher] Event:new '/home/mike/Dev/projects/anony_counseling'
[2023-05-27 14:40:03] [watcher] Event:start '/home/mike/Dev/projects/anony_counseling'
[2023-05-27 14:40:03] [profile] START git toplevel /home/mike/Dev/projects/anony_counseling
[2023-05-27 14:40:03] [git] { "git", "-C", "/home/mike/Dev/projects/anony_counseling", "rev-parse", "--show-toplevel" }
/home/mike/Dev/projects/anony_counseling
[2023-05-27 14:40:03] [profile] END   git toplevel /home/mike/Dev/projects/anony_counseling 2ms
[2023-05-27 14:40:03] [profile] START git untracked /home/mike/Dev/projects/anony_counseling
[2023-05-27 14:40:03] [git] { "git", "-C", "/home/mike/Dev/projects/anony_counseling", "config", "status.showUntrackedFiles" }
[2023-05-27 14:40:03] [profile] END   git untracked /home/mike/Dev/projects/anony_counseling 1ms
[2023-05-27 14:40:03] [profile] START git sync job /home/mike/Dev/projects/anony_counseling nil
[2023-05-27 14:40:03] [git] running job with timeout 400ms
[2023-05-27 14:40:03] [git] git --no-optional-locks status --porcelain=v1 -z --ignored=matching -u
 M app/db.py
 M tests/test_utils.py
!! .env
!! .mypy_cache/.gitignore
!! .mypy_cache/3.11/
!! .mypy_cache/3.9/
!! .mypy_cache/CACHEDIR.TAG
!! Session.vim
!! app.log
!! app/__pycache__/
!! db.log
!! info.log
!! tests/__pycache__/
[2023-05-27 14:40:03] [git] done
[2023-05-27 14:40:03] [profile] END   git sync job /home/mike/Dev/projects/anony_counseling nil 10ms
[2023-05-27 14:40:03] [git] job success    /home/mike/Dev/projects/anony_counseling nil
[2023-05-27 14:40:03] [watcher] git start
[2023-05-27 14:40:03] [watcher] Watcher:new '/home/mike/Dev/projects/anony_counseling/.git' { "FETCH_HEAD", "HEAD", "HEAD.lock", "config", "index" }
[2023-05-27 14:40:03] [watcher] Event:new '/home/mike/Dev/projects/anony_counseling/.git'
[2023-05-27 14:40:03] [watcher] Event:start '/home/mike/Dev/projects/anony_counseling/.git'
[2023-05-27 14:40:03] [profile] START explore init /home/mike/Dev/projects/anony_counseling
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/.env
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/.env 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/.git
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/.git 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/.gitignore
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/.gitignore 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/.mypy_cache
[2023-05-27 14:40:03] [watcher] Watcher:new '/home/mike/Dev/projects/anony_counseling/.mypy_cache' nil
[2023-05-27 14:40:03] [watcher] Event:new '/home/mike/Dev/projects/anony_counseling/.mypy_cache'
[2023-05-27 14:40:03] [watcher] Event:start '/home/mike/Dev/projects/anony_counseling/.mypy_cache'
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/.mypy_cache 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/Dockerfile
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/Dockerfile 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/Pipfile
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/Pipfile 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/Pipfile.lock
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/Pipfile.lock 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/Session.vim
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/Session.vim 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/admin
[2023-05-27 14:40:03] [watcher] Watcher:new '/home/mike/Dev/projects/anony_counseling/admin' nil
[2023-05-27 14:40:03] [watcher] Event:new '/home/mike/Dev/projects/anony_counseling/admin'
[2023-05-27 14:40:03] [watcher] Event:start '/home/mike/Dev/projects/anony_counseling/admin'
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/admin 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/app
[2023-05-27 14:40:03] [watcher] Watcher:new '/home/mike/Dev/projects/anony_counseling/app' nil
[2023-05-27 14:40:03] [watcher] Event:new '/home/mike/Dev/projects/anony_counseling/app'
[2023-05-27 14:40:03] [watcher] Event:start '/home/mike/Dev/projects/anony_counseling/app'
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/app 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/app.log
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/app.log 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/common
[2023-05-27 14:40:03] [watcher] Watcher:new '/home/mike/Dev/projects/anony_counseling/common' nil
[2023-05-27 14:40:03] [watcher] Event:new '/home/mike/Dev/projects/anony_counseling/common'
[2023-05-27 14:40:03] [watcher] Event:start '/home/mike/Dev/projects/anony_counseling/common'
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/common 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/counselor
[2023-05-27 14:40:03] [watcher] Watcher:new '/home/mike/Dev/projects/anony_counseling/counselor' nil
[2023-05-27 14:40:03] [watcher] Event:new '/home/mike/Dev/projects/anony_counseling/counselor'
[2023-05-27 14:40:03] [watcher] Event:start '/home/mike/Dev/projects/anony_counseling/counselor'
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/counselor 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/db.log
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/db.log 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/info.log
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/info.log 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/requirements.txt
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/requirements.txt 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/tests
[2023-05-27 14:40:03] [watcher] Watcher:new '/home/mike/Dev/projects/anony_counseling/tests' nil
[2023-05-27 14:40:03] [watcher] Event:new '/home/mike/Dev/projects/anony_counseling/tests'
[2023-05-27 14:40:03] [watcher] Event:start '/home/mike/Dev/projects/anony_counseling/tests'
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/tests 0ms
[2023-05-27 14:40:03] [profile] START explore populate_children /home/mike/Dev/projects/anony_counseling/user
[2023-05-27 14:40:03] [watcher] Watcher:new '/home/mike/Dev/projects/anony_counseling/user' nil
[2023-05-27 14:40:03] [watcher] Event:new '/home/mike/Dev/projects/anony_counseling/user'
[2023-05-27 14:40:03] [watcher] Event:start '/home/mike/Dev/projects/anony_counseling/user'
[2023-05-27 14:40:03] [profile] END   explore populate_children /home/mike/Dev/projects/anony_counseling/user 0ms
[2023-05-27 14:40:03] [profile] END   explore init /home/mike/Dev/projects/anony_counseling 1ms
[2023-05-27 14:40:03] [profile] END   core init /home/mike/Dev/projects/anony_counseling 16ms
[2023-05-27 14:40:03] [profile] START view open
[2023-05-27 14:40:03] [profile] END   view open 6ms
[2023-05-27 14:40:03] [profile] START draw
[2023-05-27 14:40:03] [profile] END   draw 1ms
[2023-05-27 14:40:03] [profile] START git sync job /home/mike/Dev/projects/anony_counseling nil
[2023-05-27 14:40:03] [git] running job with timeout 400ms
[2023-05-27 14:40:03] [git] git --no-optional-locks status --porcelain=v1 -z --ignored=matching -u
 M app/db.py
 M tests/test_utils.py
!! .env
!! .mypy_cache/.gitignore
!! .mypy_cache/3.11/
!! .mypy_cache/3.9/
!! .mypy_cache/CACHEDIR.TAG
!! Session.vim
!! app.log
!! app/__pycache__/
!! db.log
!! info.log
!! tests/__pycache__/
[2023-05-27 14:40:03] [git] done
[2023-05-27 14:40:03] [profile] END   git sync job /home/mike/Dev/projects/anony_counseling nil 3ms
[2023-05-27 14:40:03] [git] job success    /home/mike/Dev/projects/anony_counseling nil
[2023-05-27 14:40:03] [profile] START reload /home/mike/Dev/projects/anony_counseling
[2023-05-27 14:40:03] [profile] END   reload /home/mike/Dev/projects/anony_counseling 1ms
[2023-05-27 14:40:03] [profile] START draw
[2023-05-27 14:40:03] [profile] END   draw 0ms
[2023-05-27 14:40:03] [profile] START diagnostics update
[2023-05-27 14:40:03] [diagnostics] update
[2023-05-27 14:40:03] [profile] END   diagnostics update 0ms

Neovim version

NVIM v0.9.0
Build type: RelWithDebInfo
LuaJIT 2.1.0-beta3
Compilation: /usr/bin/gcc -O2 -g -Og -g -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wshadow -Wconversion -Wvla -Wdouble-promotion -Wmissing-noreturn -Wmissing-format-attribute -Wmissing-prototypes -fno-common -Wno-unused-result -Wimplicit-fallthrough -fdiagnostics-color=auto -fstack-protector-strong -DUNIT_TESTING -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -I/usr/include/luajit-2.1 -I/usr/include -I/usr/include/luajit-2.1 -I/builddir/build/BUILD/neovim-0.9.0/redhat-linux-build/src/nvim/auto -I/builddir/build/BUILD/neovim-0.9.0/redhat-linux-build/include -I/builddir/build/BUILD/neovim-0.9.0/redhat-linux-build/cmake.config -I/builddir/build/BUILD/neovim-0.9.0/src -I/usr/include -I/usr/include -I/usr/include -I/usr/include -I/usr/include -I/usr/include -I/usr/include

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/usr/share/nvim"

Operating system and version

Linux fedora 6.2.15-200.fc37.x86_64 x86_64 GNU/Linux

nvim-tree version

e2a4c9d

Minimal config

vim.g.loaded_netrw = 1
vim.g.loaded_netrwPlugin = 1

vim.cmd [[set runtimepath=$VIMRUNTIME]]
vim.cmd [[set packpath=/tmp/nvt-min/site]]
local package_root = "/tmp/nvt-min/site/pack"
local install_path = package_root .. "/packer/start/packer.nvim"
local function load_plugins()
  require("packer").startup {
    {
      "wbthomason/packer.nvim",
      "nvim-tree/nvim-tree.lua",
      "nvim-tree/nvim-web-devicons",
      -- ADD PLUGINS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
    },
    config = {
      package_root = package_root,
      compile_path = install_path .. "/plugin/packer_compiled.lua",
      display = { non_interactive = true },
    },
  }
end
if vim.fn.isdirectory(install_path) == 0 then
  print "Installing nvim-tree and dependencies."
  vim.fn.system { "git", "clone", "--depth=1", "https://github.com/wbthomason/packer.nvim", install_path }
end
load_plugins()
require("packer").sync()
vim.cmd [[autocmd User PackerComplete ++once echo "Ready!" | lua setup()]]
vim.opt.termguicolors = true
vim.opt.cursorline = true

-- MODIFY NVIM-TREE SETTINGS THAT ARE _NECESSARY_ FOR REPRODUCING THE ISSUE
_G.setup = function()
  require("nvim-tree").setup {}
end

Steps to reproduce

1. nvim -nu nvt-min.lua
2. NvimTreeToggle

Expected behavior

No response

Actual behavior

No response

[alex-courtis]

Which exact file was the error reported for? Was it "Could not start the fs_event watcher for path /home/mike/Dev/projects/anony_counseling/user : EMFILE" ?

EMFILE (see man 2 open) indicates that you've reached your limit of open files.

1. What does ulimit -a report?
2. What does cat /proc/sys/fs/inotify/max_user_watches show?

When using intellij idea I increase max_user_watches from 524288 to 1048576.

[alex-courtis]

Thank you for the detailed report - we will resolve this a lot faster!

[milkias17]

ulimit -a
Maximum size of core files created                              (kB, -c) unlimited
Maximum size of a process’s data segment                        (kB, -d) unlimited
Control of maximum nice priority                                    (-e) 0
Maximum size of files created by the shell                      (kB, -f) unlimited
Maximum number of pending signals                                   (-i) 62845
Maximum size that may be locked into memory                     (kB, -l) 8192
Maximum resident set size                                       (kB, -m) unlimited
Maximum number of open file descriptors                             (-n) 1024
Maximum bytes in POSIX message queues                           (kB, -q) 800
Maximum realtime scheduling priority                                (-r) 0
Maximum stack size                                              (kB, -s) 8192
Maximum amount of CPU time in seconds                      (seconds, -t) unlimited
Maximum number of processes available to current user               (-u) 62845
Maximum amount of virtual memory available to each process      (kB, -v) unlimited
Maximum contiguous realtime CPU time                                (-y) unlimited

2. 524288

About which file, the error was reported for all directories in the project.

[alex-courtis]

Those look about the same as my machine:

-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             16384
-c: core file size (blocks)         unlimited
-m: resident set size (kbytes)      unlimited
-u: processes                       256275
-n: file descriptors                1024
-l: locked-in-memory size (kbytes)  8192
-v: address space (kbytes)          unlimited
-x: file locks                      unlimited
-i: pending signals                 256275
-q: bytes in POSIX msg queues       819200
-e: max nice                        30
-r: max rt priority                 0
-N 15: rt cpu time (microseconds)   unlimited

Can you please try doubling the inotify watchers? https://youtrack.jetbrains.com/articles/IDEA-A-2/Inotify-Watches-Limit-Linux

Are there a lot of other processes running on your system?

Did this issue start occurring recently? Have watchers worked successfullly in the past?

[milkias17]

I had like ~300 processes running just from a quick ps aux.

Yes, after increasing the inotify watchers, the errors stop showing up.

Yes in the past, it used to work without any problems. I would say maybe 1 month ago or so was when these errors start popping up,

[alex-courtis]

Yes, after increasing the inotify watchers, the errors stop showing up.

That's great. I'll add to the wiki. This problem is not uncommon.

[alex-courtis]

Wiki: https://github.com/nvim-tree/nvim-tree.lua/wiki/Troubleshooting#could-not-start-fs_event-for-path--emfile

Handling: #2240