Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

    12.0p strange issue: exportData() (and exportClientData()) sometimes resulting in valid, but empty XLSX file

    Hi Isomorphic,

    we changed from exportClientData() to exportData() a few month ago (on 12.0p, now v12.0p_2020-03-11).
    Reason for this were the amount of data possible to export with exportClientData() as well as the fact that we got empty XLSX exports occasionally (which I attributed to network problems and many data back then (was wrong)).

    In this case the file is downloaded correctly and opens in LibreOffice or Excel, but is just empty. If you open it as zip file and navigate to xl\worksheets\sheet1.xml and look into it, it becomes clear that it is correct that LibreOffice doesn't display anything:
    Code:
    <?xml version="1.0" encoding="UTF-8"?>
    <worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">
      <dimension ref="A1"/>
      <sheetViews>
        <sheetView workbookViewId="0" tabSelected="true"/>
      </sheetViews>
      <sheetFormatPr defaultRowHeight="15.0"/>
      <sheetData/>
      <pageMargins bottom="0.75" footer="0.3" header="0.3" left="0.7" right="0.7" top="0.75"/>
    </worksheet>
    My question is now why this is happening. I set the log level to trace, but unfortunately this does not help. The relevant part of the logs after the fetch is always:
    Code:
    2020-05-06 11:37:31,937 DEBUG c.i.t.Timing [https-openssl-nio-443-exec-4] [builtinApplication.T_RESELLER_fetch] SQLTransform (62 rows): 30ms
    2020-05-06 11:37:31,937 INFO c.i.d.DSResponse [https-openssl-nio-443-exec-4] DSResponse: List with 62 items
    2020-05-06 11:37:31,937 DEBUG c.i.d.DSRequest [https-openssl-nio-443-exec-4] freeOnExecute is false for request of type fetch on DataSource T_RESELLER - not freeing resources!
    2020-05-06 11:37:31,938 DEBUG DSRequestLogger [https-openssl-nio-443-exec-4] DSRequest successful.
    2020-05-06 11:37:32,543 DEBUG c.i.r.ExcelDataExport [https-openssl-nio-443-exec-4] Export streaming mode: true
    2020-05-06 11:37:32,861 DEBUG c.i.s.SQLTransaction [https-openssl-nio-443-exec-4] getConnection() looked for transactional connection for Leadtributor:  hashcode "610595744"
    2020-05-06 11:37:32,861 WARN c.i.s.SQLTransaction [https-openssl-nio-443-exec-4] Ending Leadtributor transaction "610595744"
    So logs resulting in a good file and logs resulting in a bad file look the same (there is always a fetch with rows returned also in the case of a bad file).
    Again, this also happened occasionally before when we were using exportClientData(), so I think it's something about XLSX generation.
    Do you have any idea what might be happening here? Could you add more logging in the area of ExcelDataExport?
    Can I enable more logging here somehow already? Perhaps stop the framework from suppressing 3rd party logs via a server.properties setting (if such a thing happens)?

    Any advice is appreciated, because it's very frustrating to get such reports and not be able to find out anything about it.

    Thank you & Best regards
    Blama

    #2
    Hi Isomorphic,

    some important finding:
    When I compared good and bad exports I got my logs from graylog. The logs in #1 are from console locally.
    Now, if the issue happens on the server, I get this exception in catalina.out, but not in graylog (will investigate why this is the case, might be bad config on our part - or can you to see any special treatment of exceptions that are caused by ExcelDataExport.java:404?).

    I think the exception itself might help to find the root cause (I will also look in sun.awt.FontConfiguration.getVersion(FontConfiguration.java:1264) to see if this is somehow related to the environment
    Code:
    java.lang.NullPointerException
            at sun.awt.FontConfiguration.getVersion(FontConfiguration.java:1264)
            at sun.awt.FontConfiguration.readFontConfigFile(FontConfiguration.java:219)
            at sun.awt.FontConfiguration.init(FontConfiguration.java:107)
            at sun.awt.X11FontManager.createFontConfiguration(X11FontManager.java:774)
            at sun.font.SunFontManager$2.run(SunFontManager.java:431)
            at java.security.AccessController.doPrivileged(Native Method)
            at sun.font.SunFontManager.<init>(SunFontManager.java:376)
            at sun.awt.FcFontManager.<init>(FcFontManager.java:35)
            at sun.awt.X11FontManager.<init>(X11FontManager.java:57)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
            at java.lang.Class.newInstance(Class.java:442)
            at sun.font.FontManagerFactory$1.run(FontManagerFactory.java:83)
            at java.security.AccessController.doPrivileged(Native Method)
            at sun.font.FontManagerFactory.getInstance(FontManagerFactory.java:74)
            at java.awt.Font.getFont2D(Font.java:491)
            at java.awt.Font.canDisplayUpTo(Font.java:2064)
            at java.awt.font.TextLayout.singleFont(TextLayout.java:470)
            at java.awt.font.TextLayout.<init>(TextLayout.java:531)
            at org.apache.poi.ss.util.SheetUtil.getDefaultCharWidth(SheetUtil.java:275)
            at org.apache.poi.xssf.streaming.AutoSizeColumnTracker.<init>(AutoSizeColumnTracker.java:117)
            at org.apache.poi.xssf.streaming.SXSSFSheet.<init>(SXSSFSheet.java:82)
            at org.apache.poi.xssf.streaming.SXSSFWorkbook.createAndRegisterSXSSFSheet(SXSSFWorkbook.java:658)
            at org.apache.poi.xssf.streaming.SXSSFWorkbook.createSheet(SXSSFWorkbook.java:650)
            at org.apache.poi.xssf.streaming.SXSSFWorkbook.createSheet(SXSSFWorkbook.java:90)
            at com.isomorphic.rpc.ExcelDataExport.getExportObject(ExcelDataExport.java:404)
            at com.isomorphic.rpc.DataExport.exportResultSet(DataExport.java:647)
            at com.isomorphic.rpc.DataExport.exportResultSet(DataExport.java:452)
            at com.isomorphic.rpc.RPCManager.completeResponse(RPCManager.java:1290)
            at com.isomorphic.rpc.RPCManager.send(RPCManager.java:723)
            at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:183)
            at com.lmscompany.lms.server.LMSIDACall.processRequest(LMSIDACall.java:54)
            at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:119)
            at com.isomorphic.servlet.IDACall.doPost(IDACall.java:79)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
            at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:176)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at com.lmscompany.lms.server.RequestLoggingFilter.doFilter(RequestLoggingFilter.java:41)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at com.lmscompany.lms.server.ExceptionLoggingFilter.doFilter(ExceptionLoggingFilter.java:28)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at com.lmscompany.lms.server.CurrentUserFilterBind.doFilter(CurrentUserFilterBind.java:63)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:247)
            at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:93)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
            at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:666)
            at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
            at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
            at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:555)
            at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
            at org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:330)
            at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747)
            at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
            at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
            at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
            at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
            at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639)
            at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
            at java.lang.Thread.run(Thread.java:748)
    Best regards
    Blama

    Comment


      #3
      Although people make fun of developers that do it, sometimes Googling the error message is the best approach - take a look at this thread as well as other Google hits for the top of that stack.

      If you need more help, let us know:

      1. what JDK you are running

      2. whether this is intermittent or consistently reproducible

      3. whether it occurs under high load (possible concurrency issue) or can occur at any time
      Last edited by Isomorphic; 6 May 2020, 03:20.

      Comment


        #4
        Hi Isomorphic,

        I just wanted to update the thread. I also found that github post. Yes, it's solved with yum install fontconfig.
        It's not clear to me why this is necessary, but that's the way it is.

        Thanks for the fast answer.

        Best regards
        Blama

        Comment

        Working...
        X