dap4 test failing on travis

[lesserwhirls]

See http:https://unidata-tds.s3.amazonaws.com/Travis/5.0.0-4483/4483.1/classes/dap4.test.TestServlet.html

dap4.core.util.DapException: CDMDSP: cannot process: /home/travis/build/Unidata/thredds/dap4/d4tests/src/test/data/resources/testfiles/test_atomic_array.nc
	at dap4.cdm.dsp.CDMDSP.open(CDMDSP.java:142)
	at dap4.cdm.dsp.CDMDSP.open(CDMDSP.java:35)
	at dap4.servlet.DapCache.open(DapCache.java:94)
	at dap4.servlet.DapController.doDMR(DapController.java:307)
	at dap4.servlet.DapController.handleRequest(DapController.java:230)
	at thredds.server.dap4.Dap4Controller.handleRequest(Dap4Controller.java:73)
	at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
	at org.springframework.test.web.servlet.TestDispatcherServlet.service(TestDispatcherServlet.java:65)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.springframework.mock.web.MockFilterChain$ServletFilterProxy.doFilter(MockFilterChain.java:167)
	at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
	at org.springframework.test.web.servlet.MockMvc.perform(MockMvc.java:155)
	at dap4.test.DapTestCommon.perform(DapTestCommon.java:529)
	at dap4.test.TestServlet.dodmr(TestServlet.java:225)
	at dap4.test.TestServlet.doOneTest(TestServlet.java:207)
	at dap4.test.TestServlet.testServlet(TestServlet.java:188)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:114)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:57)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:66)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
	at com.sun.proxy.$Proxy3.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:109)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:147)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:129)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
	at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:46)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused by: dap4.core.util.DapException: java.io.IOException: java.io.IOException: -101: NetCDF: HDF error
	at dap4.cdm.dsp.CDMDSP.createNetcdfFile(CDMDSP.java:1109)
	at dap4.cdm.dsp.CDMDSP.open(CDMDSP.java:138)
	... 75 more
Caused by: java.io.IOException: java.io.IOException: -101: NetCDF: HDF error
	at ucar.nc2.NetcdfFile.open(NetcdfFile.java:512)
	at dap4.cdm.dsp.CDMDSP.createNetcdfFile(CDMDSP.java:1100)
	... 76 more
Caused by: java.io.IOException: -101: NetCDF: HDF error
	at ucar.nc2.jni.netcdf.Nc4Iosp._open(Nc4Iosp.java:315)
	at ucar.nc2.jni.netcdf.Nc4Iosp.open(Nc4Iosp.java:292)
	at ucar.nc2.NetcdfFile.<init>(NetcdfFile.java:1649)
	at ucar.nc2.NetcdfFile.open(NetcdfFile.java:919)
	at ucar.nc2.NetcdfFile.open(NetcdfFile.java:509)
	... 77 more
Jul 19, 2017 4:47:16 PM ucar.unidata.util.test.UnitTestCommon$STDIO printf
INFO: DMR Comparison

>>>> EMPTY TEST RESULT

[DennisHeimbigner]

This test is also failing in the same way on Jenkins.
It works on my machine under intellij, so my speculation
is that we need to upgrade the netcdf-c jna installation
on travis and jenkins. We probably need to enable logging support
for netcdf-4 when it is installed.

[lesserwhirls]

We are using 4.1.1, but could probably bump to 4.1.1.1. Or, should we wait until the next release of netCDF-C?

[lesserwhirls]

...my PR just passed on travis without issue...

https://travis-ci.org/lesserwhirls/thredds/builds/255838524?utm_source=email&utm_medium=notification

[lesserwhirls]

wait...it failed on the Unidata travis run

https://travis-ci.org/Unidata/thredds/builds/255838712?utm_source=github_status&utm_medium=notification

[lesserwhirls]

Ok, one ran on Ubuntu precise (Passed), the other ran on Trusty (failed). See #898. I just made a PR to pin our travis version to precise to keep thing going for now.

[DennisHeimbigner]

This is not good when the underlying OS [version] is involved in the failure.

[lesserwhirls]

I think it's because some of the os libraries that netCDF-C is linked against changed with the OS bump.

[cwardgar]

I can spin up a Vagrant box and quickly generate the new binaries. But weren't we waiting on a new version of netcdf-c? And didn't we want to build with certain flags enabled? Can I get a summary of all that?

[lesserwhirls]

Yeah, I thought we were waiting for the next release, so let's hold off until that happens. Travis is working again now that the version of Ubuntu is pinned. @DennisHeimbigner mentioned building with logging enabled, which would be the --enable-logging flag. Except for the CPPFLAGS and LDFLAGS in-line env variables, I don't think we need to use any other special flags on our test machines (no more --enable-threadsafe on the HDF build). Although, on our production machines, I turn off dap simply because it's not needed, so that one would be --disable-dap.

So, I would say, for netCDF-C specifically:

• CPPFLAGS=<path to /include dir with HDF headers>
• LDFLAGS=<path to /lib dir with HDF libs>
• --disable-dap
• --enable-logging

Everything else should be ok with vanilla or whatever the OS provides.

@DennisHeimbigner - where do the netCDF-C generated logs get dumped? It would be good to include those in the S3 Store (with a link in the testing output) so we can examine them after the tests run, much like we do on travis currently for output from our test suite.

[DennisHeimbigner]

That set of flags looks good.
By default, the log output gets sent to stdout (or maybe stderr). However nothing
is generated unless logging is turned on. The reason I would like it enabled
is so we can turn it on as needed to help debug problems like the one in this issue.
If it turns out that, for some reason, that log output is causing problems, then we
can disable it again.