Details
-
Bug
-
Status: Resolved (View Workflow)
-
Major
-
Resolution: Fixed
-
18.0.0
-
Security Level: Default (Default Security Scheme)
-
None
Description
I was doing an upgrade and got a "too many open files" error:
Processing DS /opt/opennms/share/rrd/snmp/5939/jmxJVMIndex/GlassFish-GRP370/ds.properties
Warning: there are no .meta files on /opt/opennms/share/rrd/snmp/5939/jmxJVMIndex/GlassFish-GRP370
Warning: there are no .rrd files on /opt/opennms/share/rrd/snmp/5939/jmxJVMIndex/GlassFish-GRP370
Processing DS /opt/opennms/share/rrd/snmp/5939/jmxJVMIndex/GlassFish-GRP290/ds.properties
Warning: can't perform the upgrade operation because: Can't upgrade the JRBs because /opt/opennms/share/rrd/snmp/5939/jmxJVMIndex/GlassFish-GRP290/ds.properties (Too many open files)
- Executing rollback phase
Warning: can't rollback the upgrade because: Can't restore the backup files because Failed to list contents of /opt/opennms/share/rrd/snmp/7208/jmxJVMIndex/Tomcat8-GRP471
org.opennms.upgrade.api.OnmsUpgradeException: Can't restore the backup files because Failed to list contents of /opt/opennms/share/rrd/snmp/7208/jmxJVMIndex/Tomcat8-GRP471
at org.opennms.upgrade.implementations.JmxRrdMigratorOffline.rollback(JmxRrdMigratorOffline.java:186)
at org.opennms.upgrade.support.Upgrade.executeUpgrade(Upgrade.java:155)
at org.opennms.upgrade.support.Upgrade.execute(Upgrade.java:247)
at org.opennms.upgrade.support.Upgrade.main(Upgrade.java:266)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.opennms.bootstrap.Bootstrap$4.run(Bootstrap.java:460)
at java.lang.Thread.run(Thread.java:745)- Running post-execution phase
After this occurred, the upgrade process still attempted to run addtitional bits that continued to fail with "too many open files"
Removing backup /opt/opennms/share/rrd/snmp/5041/jmxJVMIndex/GlassFish-GRP200.zip
Removing backup /opt/opennms/share/rrd/snmp/4521/jmxJVMIndex/GlassFish-GRP259.zipFinished in 43 minutes
Processing JettyConfigMigratorOffline: Adds jetty.xml if necessary:
NMS-6629
- Running pre-execution phase
- Running execution phase
SSL Enabled ? true
AJP Enabled ? false
Copying /opt/opennms/etc/examples/jetty.xml into /opt/opennms/etc/jetty.xml
Warning: can't perform the upgrade operation because: Can't fix Jetty configuration because /opt/opennms/etc/examples/jetty.xml (Too many open files)- Executing rollback phase
- Running post-execution phase
Finished in 0 seconds
Processing ServiceConfigMigratorOffline: Fixes service-configuration.xml if necessary when upgrading from 1.12:
NMS-6970
- Running pre-execution phase
Ignoring: Can't load /opt/opennms/jetty-webapps/opennms/WEB-INF/version.properties
Processing KscReportsMigrator: Fixes the KSC Reports because of SNMP Interface directories changes- Running pre-execution phase
Backing up /opt/opennms/etc/ksc-performance-reports.xml
Ignoring: Can't initialize ksc-performance-reports.xml because Cannot ZIP files because /opt/opennms/etc/ksc-performance-reports.xml.zip (Too many open files)
Processing ServiceConfig1701MigratorOffline: Fixes service-configuration.xml if necessary when upgrading from 1.12:NMS-6970- Running pre-execution phase
Backing up /opt/opennms/etc/service-configuration.xml
Ignoring: Can't backup service-configurations.xml because Cannot ZIP files because /opt/opennms/etc/service-configuration.xml.zip (Too many open files)
Processing DataCollectionConfigMigratorOffline: Modifies the packages names used in etc/datacollection/*.xml. SeeNMS-7612.- Running pre-execution phase
Backing up /opt/opennms/etc/datacollection
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.opennms.bootstrap.Bootstrap$4.run(Bootstrap.java:460)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at org.opennms.upgrade.api.AbstractOnmsUpgrade.populateFilesList(AbstractOnmsUpgrade.java:342)
at org.opennms.upgrade.api.AbstractOnmsUpgrade.zipDir(AbstractOnmsUpgrade.java:393)
at org.opennms.upgrade.implementations.DataCollectionConfigMigratorOffline.preExecute(DataCollectionConfigMigratorOffline.java:72)
at org.opennms.upgrade.support.Upgrade.executeUpgrade(Upgrade.java:141)
at org.opennms.upgrade.support.Upgrade.execute(Upgrade.java:247)
at org.opennms.upgrade.support.Upgrade.main(Upgrade.java:266)
... 6 more
There are 725,560 files under /opt/opennms/share/rrd/snmp/.