[Isomorphic]

The error message is from our code, but we are reporting an Instantiation Exception returned from the Java Core Reflection API, and it makes no sense for Reflection to throw this exception in this circumstance. So again the cause appears to be:

This actually looks more like the JVM getting internally confused about types. This kind of thing can happen if you do something that causes on-the-fly application reloads, where the JVM is trying to garbage collect actual Java class definitions and then re-load them. If you are reloading your application without restarting the JVM, you might find that this happens only after a reload.

Whatever servlet engine you're using, or perhaps other frameworks your using (Spring maybe?), there appear to be bugs, probably related to class loading. It might be trying to reload classes on the fly. You might be triggering that by touching files that aren't meant to be changed (perhaps something touches .jar files and the servlet engine thinks it needs to reload them).

One thing you could try: turn on the (very verbose) JVM class loading logs. This may show you that basic classes (such as the commons-collection library) are being reloaded by the JVM for some reason. Then once this happens, sometime later, this exception might arise.

[Blama]

Isomorphic,

googleing for just "most likely cause is the class represents an abstract class" (with the quotes) I also found this thread of mine (same issue as in this thread) and this thread from claudiobosticco which again has another class throwing the exception. In the latter case you could find some bug introduced between these versions. I think that although this exact issue is most likely unrelated to my problem, perhaps the same type of coding used there does eventually trigger my issue?

In total I find it very confusing that the only search hits for this - which seems to be text from Java and not from the Framework code (this that true?) - are from this forum.

I know that this all seems a bit like reading coffee grounds, but I'm really confused here.

Best regards
Blama

[Blama]

Very unlikely shot in the dark:
Googleing for "most likely cause is the class represents an abstract class, an interface, an array class, a primitive type, or void; or the class has no zero-argument constructor." I got only 2 hits(?!), one of them being this thread. The other one is this. From the visible part of the email, the GWT-topic, and the "schueler"-in the code, I assume that this post is by you, edulid?!
If so, could you tell what caused the issue back then in 2014 and what solved it? With the initial class being RPCManager in your case I don't think it's related, but who knows.

I'm pretty confused by the fact that the error message does not appear more often and that the results also don't include any source code throwing an exception with this text.

[Isomorphic]

We're sorry this is happening for you, but if you continue to ping us just saying "it happened again", there's really nothing more we can do. You've already got our best guesses, above.

[Blama]

Hi Isomorphic,

this happened again, and does so quite regularly every few weeks and is of course bad, as right now I can't do anything about it.
Any idea how I could analyse or fix the issue is highly appreciated.
Touching the .ds.xml or making it uncached by making it dynamic seem like possible solutions, but far from ideal. It would be great if it was possible to find the root cause here.

Best regards
Blama

[Blama]

Hi Isomorphic,

another observation (it does happen here from time to time, but always with the same DS, so here seems to be a triggering bit here somehow):
DataSourceLoader caches DataSources until the .ds.xml file changes. Here the problem was gone by adding a meaningless space and saving the file again, breaking the cache.
So the DataSourceLoader processing seems to be OK all the time - the problem arises with some cache lookup.
I do assume though that this still does not help you very much.

Best regards
Blama

[Blama]

Hi Isomorphic,

it happend once again just now, again with the same DataSource and on a yesterday restarted Tomcat (we don't do application stop/starts in a running Tomcat in production). The application itself was not much used (it's still being prepared for the customer).
There seems to be something special about the DataSource indeed.
The next time it happens I'll take the VM out of load balancing and try to remote debug the JVM instead of just restarting it.

Best regards
Blama

[Isomorphic]

We don't have any new data other than that it happened again, so no. Same speculations as before.

[pavo123]

Hi Isomorphic,

this problem happened again today in our production system. Any new ideas about it?
SmartClient Version: v12.0p_2020-07-24/PowerEdition Deployment (built 2020-07-24)

Best,
Pavo

[Blama]

Hi Isomorphic,

I saw it only once in production. I think I also saw it locally before (which could involve stop/start-cycles), but not 100% sure.
I'll report back here if it happens again.

Thank you & Best regards
Blama

[Isomorphic]

Yes, that's just recursion, that part looks normal.

Have you seen this error more than once, and is it always with the same DataSource?

[Blama]

Hi Isomorphic,

yes, I find it also very strange - looks to me like a JVM thing as well.

W.r.t. the .ds.xml: I wrote in #1 what I find special about this DataSource: The only apparent thing is that it uses many (~70) fields with value map (2 of them with fmt tags (<value ID="W"><fmt:message key="won" /></value>)).

The application wasn't restarted IMHO, everything mysterious.

If you say that there is DataTools.java:486 3 times in a row in the trace is correct (perhaps because of recursion(?)), then everything is fine for me. This was a one time thing so far here.

Thank you & Best regards
Blama

[Isomorphic]

This doesn't look like a concurrency exception, as this is code that is going through and copying a data structure while not modifying it.

This actually looks more like the JVM getting internally confused about types. This kind of thing can happen if you do something that causes on-the-fly application reloads, where the JVM is trying to garbage collect actual Java class definitions and then re-load them. If you are reloading your application without restarting the JVM, you might find that this happens only after a reload.

Finally, you should look at the surrounding logs to see which DataSource was being processed (or if you don't have enough logging to tell this, turn it on for next time). If this error recurs for a single DataSource, there could be something unique the .ds.xml usage in that case, maybe something like custom properties and subelements.

[Blama]

Trace 2 shortened:

javax.servlet.ServletException: DataSource 'T_STATUS_CHANGE' failed to load due to an exception on the server:
Instantiation of class: java.util.LinkedHashMap$LinkedKeySet threw an InstantiationException - most likely cause is the class represents an abstract class, an interface, an array class, a primitive type, or void; or the class has no zero-argument constructor.
See the server-side log for additional details.
    at com.isomorphic.servlet.DataSourceLoader.processRequest(DataSourceLoader.java:295)
    at com.isomorphic.servlet.DataSourceLoader.doGet(DataSourceLoader.java:110)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
    at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:176)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at com.lmscompany.lms.server.MimeTypeFilter.doFilter(MimeTypeFilter.java:26)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at com.lmscompany.lms.server.RequestLoggingFilter.doFilter(RequestLoggingFilter.java:41)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at com.lmscompany.lms.server.ExceptionLoggingFilter.doFilter(ExceptionLoggingFilter.java:28)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

Trace 1 shortened:

java.lang.InstantiationException: Instantiation of class: java.util.LinkedHashMap$LinkedKeySet threw an InstantiationException - most likely cause is the class represents an abstract class, an interface, an array class, a primitive type, or void; or the class has no zero-argument constructor.
    at com.isomorphic.base.Reflection.newInstance(Reflection.java:230)
    at com.isomorphic.base.Reflection.newInstance(Reflection.java:193)
    at com.isomorphic.util.DataTools.deepClone(DataTools.java:490)
    at com.isomorphic.util.DataTools.deepClone(DataTools.java:486)
    at com.isomorphic.util.DataTools.deepClone(DataTools.java:493)
    at com.isomorphic.util.DataTools.deepClone(DataTools.java:486)
    at com.isomorphic.util.DataTools.deepClone(DataTools.java:486)
    at com.isomorphic.util.DataTools.deepClone(DataTools.java:486)
    at com.isomorphic.datasource.DataSource._cloneConfigForSecurityAnnotations(DataSource.java:1762)
    at com.isomorphic.servlet.DataSourceLoader.processRequest(DataSourceLoader.java:232)
    at com.isomorphic.servlet.DataSourceLoader.doGet(DataSourceLoader.java:110)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)

[Blama]

Trace 1:

java.lang.InstantiationException: Instantiation of class: java.util.LinkedHashMap$LinkedKeySet threw an InstantiationException - most likely cause is the class represents an abstract class, an interface, an array class, a primitive type, or void; or the class has no zero-argument constructor. at com.isomorphic.base.Reflection.newInstance(Reflection.java:230) at com.isomorphic.base.Reflection.newInstance(Reflection.java:193) at com.isomorphic.util.DataTools.deepClone(DataTools.java:490) at com.isomorphic.util.DataTools.deepClone(DataTools.java:486) at com.isomorphic.util.DataTools.deepClone(DataTools.java:493) at com.isomorphic.util.DataTools.deepClone(DataTools.java:486) at com.isomorphic.util.DataTools.deepClone(DataTools.java:486) at com.isomorphic.util.DataTools.deepClone(DataTools.java:486) at com.isomorphic.datasource.DataSource._cloneConfigForSecurityAnnotations(DataSource.java:1762) at com.isomorphic.servlet.DataSourceLoader.processRequest(DataSourceLoader.java:232) at com.isomorphic.servlet.DataSourceLoader.doGet(DataSourceLoader.java:110) at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:176) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lmscompany.lms.server.MimeTypeFilter.doFilter(MimeTypeFilter.java:26) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lmscompany.lms.server.RequestLoggingFilter.doFilter(RequestLoggingFilter.java:41) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.lmscompany.lms.server.ExceptionLoggingFilter.doFilter(ExceptionLoggingFilter.java:28) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260) at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:93) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:666) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:330) at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)