I've been seeing an increasing number of instances (among both the community at large and our support customer base) where the default value passed to 'ulimit -n" (per-process max open file descriptors) in the OpenNMS start script is insufficient, resulting in wacky problems that can be very difficult to diagnose. Ultimately messages start appearing in the logs about "Too many open files". I think the primary culprit here is that Provisiond's massively parallel, asynchronous way of scanning for services eats up lots of socket FDs.
We should bump the default setting in bin/opennms and in the package maintainer scripts where applicable (thinking Debian). Currently this is:
So far I've had good luck with doubling that value to 20480.