Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

    12.0p Strange DataSourceLoader exception

    Hi Isomorphic,

    out of nothing we started to get this error today (v12.0p_2020-03-11). I assume it's some race condition or something similar, because we did not change anything and it was only happening for one customer. Does this stack trace mean something to you?
    I had a look at the .ds.xml and the only apparent thing is that it uses many (~70) fields with value map (2 of them with fmt tags (<value ID="W"><fmt:message key="won" /></value>)).

    Two stacktraces, always appearing together:
    Code:
    javax.servlet.ServletException: DataSource 'T_STATUS_CHANGE' failed to load due to an exception on the server: Instantiation of class: java.util.LinkedHashMap$LinkedKeySet threw an InstantiationException - most likely cause is the class represents an abstract class, an interface, an array class, a primitive type, or void; or the class has no zero-argument constructor. See the server-side log for additional details. at com.isomorphic.servlet.DataSourceLoader.processRequest(DataSourceLoader.java:295) at com.isomorphic.servlet.DataSourceLoader.doGet(DataSourceLoader.java:110) at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:176) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lmscompany.lms.server.MimeTypeFilter.doFilter(MimeTypeFilter.java:26) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lmscompany.lms.server.RequestLoggingFilter.doFilter(RequestLoggingFilter.java:41) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lmscompany.lms.server.ExceptionLoggingFilter.doFilter(ExceptionLoggingFilter.java:28) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260) at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:93) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:666) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:330) at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)
    Code:
    java.lang.InstantiationException: Instantiation of class: java.util.LinkedHashMap$LinkedKeySet threw an InstantiationException - most likely cause is the class represents an abstract class, an interface, an array class, a primitive type, or void; or the class has no zero-argument constructor. at com.isomorphic.base.Reflection.newInstance(Reflection.java:230) at com.isomorphic.base.Reflection.newInstance(Reflection.java:193) at com.isomorphic.util.DataTools.deepClone(DataTools.java:490) at com.isomorphic.util.DataTools.deepClone(DataTools.java:486) at com.isomorphic.util.DataTools.deepClone(DataTools.java:493) at com.isomorphic.util.DataTools.deepClone(DataTools.java:486) at com.isomorphic.util.DataTools.deepClone(DataTools.java:486) at com.isomorphic.util.DataTools.deepClone(DataTools.java:486) at com.isomorphic.datasource.DataSource._cloneConfigForSecurityAnnotations(DataSource.java:1762) at com.isomorphic.servlet.DataSourceLoader.processRequest(DataSourceLoader.java:232) at com.isomorphic.servlet.DataSourceLoader.doGet(DataSourceLoader.java:110) at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:176) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lmscompany.lms.server.MimeTypeFilter.doFilter(MimeTypeFilter.java:26) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lmscompany.lms.server.RequestLoggingFilter.doFilter(RequestLoggingFilter.java:41) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lmscompany.lms.server.ExceptionLoggingFilter.doFilter(ExceptionLoggingFilter.java:28) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260) at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:93) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:666) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:330) at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)
    Thank you & Best regards
    Blama
    Last edited by Blama; 29th Apr 2020, 05:00.

    #2
    Sorry for the broken linefeeds, they appear OK in edit-mode.

    Code:
    Test1
    Test2
    Test3

    Comment


      #3
      Trace 1:
      Code:
      java.lang.InstantiationException: Instantiation of class: java.util.LinkedHashMap$LinkedKeySet threw an InstantiationException - most likely cause is the class represents an abstract class, an interface, an array class, a primitive type, or void; or the class has no zero-argument constructor. at com.isomorphic.base.Reflection.newInstance(Reflection.java:230) at com.isomorphic.base.Reflection.newInstance(Reflection.java:193) at com.isomorphic.util.DataTools.deepClone(DataTools.java:490) at com.isomorphic.util.DataTools.deepClone(DataTools.java:486) at com.isomorphic.util.DataTools.deepClone(DataTools.java:493) at com.isomorphic.util.DataTools.deepClone(DataTools.java:486) at com.isomorphic.util.DataTools.deepClone(DataTools.java:486) at com.isomorphic.util.DataTools.deepClone(DataTools.java:486) at com.isomorphic.datasource.DataSource._cloneConfigForSecurityAnnotations(DataSource.java:1762) at com.isomorphic.servlet.DataSourceLoader.processRequest(DataSourceLoader.java:232) at com.isomorphic.servlet.DataSourceLoader.doGet(DataSourceLoader.java:110) at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:176) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lmscompany.lms.server.MimeTypeFilter.doFilter(MimeTypeFilter.java:26) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lmscompany.lms.server.RequestLoggingFilter.doFilter(RequestLoggingFilter.java:41) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lmscompany.lms.server.ExceptionLoggingFilter.doFilter(ExceptionLoggingFilter.java:28) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260) at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:93) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:666) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:330) at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)

      Comment


        #4
        Trace 2 shortened:
        Code:
        javax.servlet.ServletException: DataSource 'T_STATUS_CHANGE' failed to load due to an exception on the server:
        Instantiation of class: java.util.LinkedHashMap$LinkedKeySet threw an InstantiationException - most likely cause is the class represents an abstract class, an interface, an array class, a primitive type, or void; or the class has no zero-argument constructor.
        See the server-side log for additional details.
            at com.isomorphic.servlet.DataSourceLoader.processRequest(DataSourceLoader.java:295)
            at com.isomorphic.servlet.DataSourceLoader.doGet(DataSourceLoader.java:110)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
            at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:176)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at com.lmscompany.lms.server.MimeTypeFilter.doFilter(MimeTypeFilter.java:26)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at com.lmscompany.lms.server.RequestLoggingFilter.doFilter(RequestLoggingFilter.java:41)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at com.lmscompany.lms.server.ExceptionLoggingFilter.doFilter(ExceptionLoggingFilter.java:28)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        Trace 1 shortened:
        Code:
        java.lang.InstantiationException: Instantiation of class: java.util.LinkedHashMap$LinkedKeySet threw an InstantiationException - most likely cause is the class represents an abstract class, an interface, an array class, a primitive type, or void; or the class has no zero-argument constructor.
            at com.isomorphic.base.Reflection.newInstance(Reflection.java:230)
            at com.isomorphic.base.Reflection.newInstance(Reflection.java:193)
            at com.isomorphic.util.DataTools.deepClone(DataTools.java:490)
            at com.isomorphic.util.DataTools.deepClone(DataTools.java:486)
            at com.isomorphic.util.DataTools.deepClone(DataTools.java:493)
            at com.isomorphic.util.DataTools.deepClone(DataTools.java:486)
            at com.isomorphic.util.DataTools.deepClone(DataTools.java:486)
            at com.isomorphic.util.DataTools.deepClone(DataTools.java:486)
            at com.isomorphic.datasource.DataSource._cloneConfigForSecurityAnnotations(DataSource.java:1762)
            at com.isomorphic.servlet.DataSourceLoader.processRequest(DataSourceLoader.java:232)
            at com.isomorphic.servlet.DataSourceLoader.doGet(DataSourceLoader.java:110)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
        Last edited by Blama; 29th Apr 2020, 05:05.

        Comment


          #5
          This doesn't look like a concurrency exception, as this is code that is going through and copying a data structure while not modifying it.

          This actually looks more like the JVM getting internally confused about types. This kind of thing can happen if you do something that causes on-the-fly application reloads, where the JVM is trying to garbage collect actual Java class definitions and then re-load them. If you are reloading your application without restarting the JVM, you might find that this happens only after a reload.

          Finally, you should look at the surrounding logs to see which DataSource was being processed (or if you don't have enough logging to tell this, turn it on for next time). If this error recurs for a single DataSource, there could be something unique the .ds.xml usage in that case, maybe something like custom properties and subelements.

          Comment


            #6
            Hi Isomorphic,

            yes, I find it also very strange - looks to me like a JVM thing as well.

            W.r.t. the .ds.xml: I wrote in #1 what I find special about this DataSource: The only apparent thing is that it uses many (~70) fields with value map (2 of them with fmt tags (<value ID="W"><fmt:message key="won" /></value>)).

            The application wasn't restarted IMHO, everything mysterious.

            If you say that there is DataTools.java:486 3 times in a row in the trace is correct (perhaps because of recursion(?)), then everything is fine for me. This was a one time thing so far here.

            Thank you & Best regards
            Blama

            Comment


              #7
              Yes, that's just recursion, that part looks normal.

              Have you seen this error more than once, and is it always with the same DataSource?

              Comment


                #8
                Hi Isomorphic,

                I saw it only once in production. I think I also saw it locally before (which could involve stop/start-cycles), but not 100% sure.
                I'll report back here if it happens again.

                Thank you & Best regards
                Blama

                Comment

                Working...
                X