InternalErrorException apprers ramdomly when performing Search

[subigre]

Steps to reproduce

When performing search operation using the FHIR API, the following exception occurs randomly.

PS: @ppazos has also experienced this error yesterday.

Actual result

SQL Error: 50000, SQLState: HY000
2021-06-24 18:57:00.375 ERROR 2381 --- [io-8888-exec-10] o.h.engine.jdbc.spi.SqlExceptionHelper   : General error: "java.lang.NullPointerException"; SQL statement:
select search0_.PID as pid1_17_0_, myincludes1_.PID as pid1_18_1_, search0_.CREATED as created2_17_0_, search0_.SEARCH_DELETED as search_d3_17_0_, search0_.EXPIRY_OR_NULL as expiry_o4_17_0_, search0_.FAILURE_CODE as failure_5_17_0_, search0_.FAILURE_MESSAGE as failure_6_17_0_, search0_.LAST_UPDATED_HIGH as last_upd7_17_0_, search0_.LAST_UPDATED_LOW as last_upd8_17_0_, search0_.NUM_BLOCKED as num_bloc9_17_0_, search0_.NUM_FOUND as num_fou10_17_0_, search0_.PREFERRED_PAGE_SIZE as preferr11_17_0_, search0_.RESOURCE_ID as resourc12_17_0_, search0_.RESOURCE_TYPE as resourc13_17_0_, search0_.SEARCH_PARAM_MAP as search_14_17_0_, search0_.SEARCH_QUERY_STRING as search_15_17_0_, search0_.SEARCH_QUERY_STRING_HASH as search_16_17_0_, search0_.SEARCH_TYPE as search_17_17_0_, search0_.SEARCH_STATUS as search_18_17_0_, search0_.TOTAL_COUNT as total_c19_17_0_, search0_.SEARCH_UUID as search_20_17_0_, search0_.OPTLOCK_VERSION as optlock21_17_0_, myincludes1_.SEARCH_INCLUDE as search_i2_18_1_, myincludes1_.INC_RECURSE as inc_recu3_18_1_, myincludes1_.REVINCLUDE as revinclu4_18_1_, myincludes1_.SEARCH_PID as search_p5_18_1_, myincludes1_.SEARCH_PID as search_p5_18_0__, myincludes1_.PID as pid1_18_0__ from HFJ_SEARCH search0_ left outer join HFJ_SEARCH_INCLUDE myincludes1_ on search0_.PID=myincludes1_.SEARCH_PID where search0_.SEARCH_UUID=? [50000-200]
2021-06-24 18:57:00.613 ERROR 2381 --- [io-8888-exec-10] o.a.c.p.e.DefaultErrorHandler            : Failed delivery for (MessageId: 4CA8B2436E30CD4-0000000000000042 on ExchangeId: 4CA8B2436E30CD4-0000000000000042). Exhausted after delivery attempt: 1 caught: org.springframework.orm.jpa.JpaSystemException: org.hibernate.exception.GenericJDBCException: could not extract ResultSet; nested exception is javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: could not extract ResultSet. Processed by failure processor: FatalFallbackErrorHandler[Channel[DelegateSync[org.ehrbase.fhirbridge.camel.processor.DefaultExceptionHandler@4a6ac443]]]
2021-06-24 18:57:00.700 ERROR 2381 --- [io-8888-exec-10] c.u.f.r.s.i.ExceptionHandlingInterceptor : Failure during REST processing
ca.uhn.fhir.rest.server.exceptions.InternalErrorException: Unexpected server error
    at org.openehealth.ipf.platform.camel.ihe.fhir.core.FhirConsumer.runRoute(FhirConsumer.java:203) ~[ipf-platform-camel-ihe-fhir-core-4.0.1.jar!/:na]
    at org.openehealth.ipf.platform.camel.ihe.fhir.core.FhirConsumer.handleInRoute(FhirConsumer.java:165) ~[ipf-platform-camel-ihe-fhir-core-4.0.1.jar!/:na]
    at org.openehealth.ipf.platform.camel.ihe.fhir.core.FhirConsumer.handleBundleRequest(FhirConsumer.java:122) ~[ipf-platform-camel-ihe-fhir-core-4.0.1.jar!/:na]
    at org.openehealth.ipf.commons.ihe.fhir.AbstractBundleProvider.obtainResources(AbstractBundleProvider.java:53) ~[ipf-commons-ihe-fhir-core-4.0.1.jar!/:na]
    at org.openehealth.ipf.commons.ihe.fhir.LazyBundleProvider.getPartialResult(LazyBundleProvider.java:102) ~[ipf-commons-ihe-fhir-core-4.0.1.jar!/:na]
    at org.openehealth.ipf.commons.ihe.fhir.LazyBundleProvider.getResources(LazyBundleProvider.java:84) ~[ipf-commons-ihe-fhir-core-4.0.1.jar!/:na]
    at ca.uhn.fhir.rest.server.method.BaseResourceReturningMethodBinding.createBundleFromBundleProvider(BaseResourceReturningMethodBinding.java:166) ~[hapi-fhir-server-5.3.3.jar!/:na]
    at ca.uhn.fhir.rest.server.method.BaseResourceReturningMethodBinding.doInvokeServer(BaseResourceReturningMethodBinding.java:335) ~[hapi-fhir-server-5.3.3.jar!/:na]
    at ca.uhn.fhir.rest.server.method.BaseResourceReturningMethodBinding.invokeServer(BaseResourceReturningMethodBinding.java:380) ~[hapi-fhir-server-5.3.3.jar!/:na]
    at ca.uhn.fhir.rest.server.RestfulServer.handleRequest(RestfulServer.java:1084) ~[hapi-fhir-server-5.3.3.jar!/:na]
    at ca.uhn.fhir.rest.server.RestfulServer.doGet(RestfulServer.java:380) ~[hapi-fhir-server-5.3.3.jar!/:na]
    at ca.uhn.fhir.rest.server.RestfulServer.service(RestfulServer.java:1794) ~[hapi-fhir-server-5.3.3.jar!/:na]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) ~[tomcat-embed-core-9.0.46.jar!/:4.0.1]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) ~[tomcat-embed-websocket-9.0.46.jar!/:na]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-5.3.7.jar!/:5.3.7]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.7.jar!/:5.3.7]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-5.3.7.jar!/:5.3.7]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.7.jar!/:5.3.7]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) ~[spring-boot-actuator-2.5.0.jar!/:2.5.0]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.7.jar!/:5.3.7]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-5.3.7.jar!/:5.3.7]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.7.jar!/:5.3.7]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91) ~[spring-web-5.3.7.jar!/:5.3.7]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.3.7.jar!/:5.3.7]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1707) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-embed-core-9.0.46.jar!/:na]
    at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: org.springframework.orm.jpa.JpaSystemException: org.hibernate.exception.GenericJDBCException: could not extract ResultSet; nested exception is javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: could not extract ResultSet
    at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:408) ~[spring-orm-5.3.7.jar!/:5.3.7]
    at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:552) ~[spring-orm-5.3.7.jar!/:5.3.7]
    at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:61) ~[spring-tx-5.3.7.jar!/:5.3.7]
    at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:242) ~[spring-tx-5.3.7.jar!/:5.3.7]
    at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:152) ~[spring-tx-5.3.7.jar!/:5.3.7]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.7.jar!/:5.3.7]
    at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:145) ~[spring-data-jpa-2.5.1.jar!/:2.5.1]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.7.jar!/:5.3.7]
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) ~[spring-aop-5.3.7.jar!/:5.3.7]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.7.jar!/:5.3.7]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.7.jar!/:5.3.7]
    at com.sun.proxy.$Proxy204.findByUuidAndFetchIncludes(Unknown Source) ~[na:na]
    at ca.uhn.fhir.jpa.search.cache.DatabaseSearchCacheSvcImpl.fetchByUuid(DatabaseSearchCacheSvcImpl.java:108) ~[hapi-fhir-jpaserver-base-5.3.3.jar!/:na]
    at ca.uhn.fhir.jpa.search.cache.DatabaseSearchCacheSvcImpl$$FastClassBySpringCGLIB$$f9ac20c0.invoke(<generated>) ~[hapi-fhir-jpaserver-base-5.3.3.jar!/:na]
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.3.7.jar!/:5.3.7]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) ~[spring-aop-5.3.7.jar!/:5.3.7]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.7.jar!/:5.3.7]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) ~[spring-aop-5.3.7.jar!/:5.3.7]
    at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123) ~[spring-tx-5.3.7.jar!/...

Expected result

No exception should appear.

Success criteria

• The defect has been fixed
• The defect is checked by an unit or an integration test (Robot)
• Merge Request approved
• Unit tests passed
• Build without errors
• Release notes prepared
• Runtime warnings