Maven surefire doesn't properly report some test failures

Description

See for some more examples and discussion.

dj-gregors-computer:~/opennms/trunk/opennms/opennms-services dgregor$ ../build.sh -
Dmaven.test.skip=false -Dtest=SpringLoaderTest test
[INFO] Scanning for projects...
[INFO] ---------------------------------------------------------------------------- [INFO] Building OpenNMS Services
[INFO] task-segment: [test]
[INFO] ---------------------------------------------------------------------------- [INFO] [resources:resources]
[INFO] Using default encoding to copy filtered resources.
[WARNING]
Artifact junit:junit:jar:3.8.1:test retains local scope 'test' overriding broader scope 'compile'
given by a dependency. If this is not intended, modify or remove the local scope.

[WARNING]
Artifact oro:oro:jar:2.0.8:runtime retains local scope 'runtime' overriding broader scope 'compile'
given by a dependency. If this is not intended, modify or remove the local scope.

[WARNING]
Artifact xerces:xercesImpl:jar:2.6.2:runtime retains local scope 'runtime' overriding broader scope
'compile'
given by a dependency. If this is not intended, modify or remove the local scope.

[WARNING]
Artifact javax.servlet:servlet-api:jar:2.3:provided retains local scope 'provided' overriding broader
scope 'compile'
given by a dependency. If this is not intended, modify or remove the local scope.

[INFO] [compiler:compile]
[INFO] Nothing to compile - all classes are up to date
[INFO] [resources:testResources]
[INFO] Using default encoding to copy filtered resources.
[INFO] [compiler:testCompile]
[INFO] Nothing to compile - all classes are up to date
[INFO] [surefire:test]
[INFO] Surefire report directory: /Users/.dgregor/opennms/trunk/opennms/opennms-services/target/
surefire-reports

------------------------------------------------------- T E S T S
------------------------------------------------------- Running org.opennms.netmgt.vmmgr.SpringLoaderTest
log4j:WARN No appenders could be found for logger (org.springframework.core.CollectionFactory).
log4j:WARN Please initialize the log4j system properly.
Creating: TestDaemon
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'init-
datasource-factory' defined in URL [jar:file:/Users/dgregor/.m2/repository/org/opennms/opennms-
dao/1.3.2-SNAPSHOT/opennms-dao-1.3.2-SNAPSHOT.jar!/META-INF/opennms/applicationContext-
dao.xml]: Invocation of init method failed; nested exception is java.io.FileNotFoundException: The
OpenNMS home directory "/opt/OpenNMS" does not exist
Caused by: java.io.FileNotFoundException: The OpenNMS home directory "/opt/OpenNMS" does not
exist
at org.opennms.netmgt.ConfigFileConstants.getConfigFileByName(ConfigFileConstants.java:704)
at org.opennms.netmgt.config.DataSourceFactory.init(DataSourceFactory.java:125)
at org.opennms.netmgt.config.DataSourceFactory.init(DataSourceFactory.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:258) at
org.springframework.beans.factory.config.MethodInvokingFactoryBean.doInvoke
(MethodInvokingFactoryBean.java:124)
at org.springframework.beans.factory.config.MethodInvokingFactoryBean.afterPropertiesSet
(MethodInvokingFactoryBean.java:113)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods
(AbstractAutowireCapableBeanFactory.java:877)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean
(AbstractAutowireCapableBeanFactory.java:846)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean
(AbstractAutowireCapableBeanFactory.java:419)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean
(AbstractBeanFactory.java:241)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean
(AbstractBeanFactory.java:152)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons
(DefaultListableBeanFactory.java:247)
at org.springframework.context.support.AbstractApplicationContext.refresh
(AbstractApplicationContext.java:331)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>
(ClassPathXmlApplicationContext.java:92)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>
(ClassPathXmlApplicationContext.java:77)
at org.opennms.netmgt.vmmgr.SpringLoader.<init>(SpringLoader.java:59)
at org.opennms.netmgt.vmmgr.SpringLoader.main(SpringLoader.java:145)
at org.opennms.netmgt.vmmgr.SpringLoaderTest.testStart(SpringLoaderTest.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.maven.surefire.junit.JUnitTestSet.execute(JUnitTestSet.java:210)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet
(AbstractDirectoryTestSuite.java:135)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
(AbstractDirectoryTestSuite.java:160)
at org.apache.maven.surefire.Surefire.run(Surefire.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:182)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:743)
ROOT CAUSE is java.io.FileNotFoundException: The OpenNMS home directory "/opt/OpenNMS" does not
exist
Exception occurred: java.io.FileNotFoundException: The OpenNMS home directory "/opt/OpenNMS"
does not exist
java.io.FileNotFoundException: The OpenNMS home directory "/opt/OpenNMS" does not exist
at org.opennms.netmgt.ConfigFileConstants.getConfigFileByName(ConfigFileConstants.java:704)
at org.opennms.netmgt.config.DataSourceFactory.init(DataSourceFactory.java:125)
at org.opennms.netmgt.config.DataSourceFactory.init(DataSourceFactory.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:258) at
org.springframework.beans.factory.config.MethodInvokingFactoryBean.doInvoke
(MethodInvokingFactoryBean.java:124)
at org.springframework.beans.factory.config.MethodInvokingFactoryBean.afterPropertiesSet
(MethodInvokingFactoryBean.java:113)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods
(AbstractAutowireCapableBeanFactory.java:877)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean
(AbstractAutowireCapableBeanFactory.java:846)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean
(AbstractAutowireCapableBeanFactory.java:419)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean
(AbstractBeanFactory.java:241)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean
(AbstractBeanFactory.java:152)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons
(DefaultListableBeanFactory.java:247)
at org.springframework.context.support.AbstractApplicationContext.refresh
(AbstractApplicationContext.java:331)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>
(ClassPathXmlApplicationContext.java:92)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>
(ClassPathXmlApplicationContext.java:77)
at org.opennms.netmgt.vmmgr.SpringLoader.<init>(SpringLoader.java:59)
at org.opennms.netmgt.vmmgr.SpringLoader.main(SpringLoader.java:145)
at org.opennms.netmgt.vmmgr.SpringLoaderTest.testStart(SpringLoaderTest.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.maven.surefire.junit.JUnitTestSet.execute(JUnitTestSet.java:210)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet
(AbstractDirectoryTestSuite.java:135)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute
(AbstractDirectoryTestSuite.java:160)
at org.apache.maven.surefire.Surefire.run(Surefire.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:182)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:743)
[INFO] ------------------------------------------------------------------------ [ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------ [INFO] There are test failures.
[INFO] ------------------------------------------------------------------------ [INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------ [INFO] Total time: 11 seconds
[INFO] Finished at: Wed Sep 20 20:05:37 EDT 2006
[INFO] Final Memory: 8M/15M
[INFO] ------------------------------------------------------------------------

Environment

Operating System: All Platform: All

Acceptance / Success Criteria

None

Lucidchart Diagrams

Activity

Seth Leger November 25, 2013 at 4:06 PM

It is possible to trap System.exit() calls inside JUnit by using a SecurityManager. I added the use of the system-rules library (http://www.stefan-birkner.de/system-rules/) which uses this technique and provides a nifty JUnit API for trapping the System.exit() calls and return codes. Marking as fixed.

commit e3ea272c4bb3ba841072ab4e9186d2b5224e5a5b

Benjamin Reed December 9, 2008 at 10:06 PM

yeah, this is still busted; I at least changed the test to JUnit4 so they're properly @Ignore'd

DJ Gregor September 20, 2006 at 8:33 PM

I figured out what is going on. In the SpringLoaderTest, it calls SpringLoader.main(....), and
SpringLoader.main() calls System.exit if there are any errors. Matt suggests that it wouldn't be hard to add
a shutdownHook and mark it as a failure or something and that's probably a junit problem though not
surefire directly. It doesn't deal with it well in Eclipse, either, so it's most likely JUnit. And, well, you
probably can't expect the test framework to deal nicely with its VM being shutdown abruptly.

Fixed

Details

Assignee

Reporter

Components

Fix versions

Affects versions

Priority

PagerDuty

Created September 20, 2006 at 8:25 PM
Updated January 27, 2017 at 4:32 PM
Resolved November 25, 2013 at 4:06 PM