In order to support the high IO load of small transactions from a database, the underlying file system should be tuned. Symptoms for this are low CPU load with high iowait. In this case, there are a couple of tweaks possible on Linux systems:
noatime,nodiratime flags for disk mount command or in the /etc/fstab for the database disk volume mount.
Linux platforms impose an upper limit on the number of concurrent files a user may have open. This number is reported for the current user and session with the command
user@localhost:~$ ulimit -n 1024
The usual default of 1024 is often not enough, especially when many indexes are used or a server installation sees too many connections (network sockets count against that limit as well).
Users are therefore encouraged to increase that limit to a healthy value of 40000 or more, depending on usage patterns.
Setting this value via the ulimit command is possible only for the root user and that for that session only.
To set the value system wide you have to follow the instructions for your platform.
What follows is the procedure to set the open file descriptor limit to 40k for user neo4j under Ubuntu 10.04 and later. If you opted to run the neo4j service as a different user, change the first field in step 2 accordingly.
Become root since all operations that follow require editing protected system files.
user@localhost:~$ sudo su - Password: root@localhost:~$
Edit /etc/security/limits.conf and add these two lines:
neo4j soft nofile 40000 neo4j hard nofile 40000
Edit /etc/pam.d/su and uncomment or add the following line:
session required pam_limits.so
A restart is required for the settings to take effect.
After the above procedure, the neo4j user will have a limit of 40000 simultaneous open files. If you continue experiencing exceptions on Too many open files or Could not stat() directory then you may have to raise that limit further.
Copyright © 2012 Neo Technology