Announcement

Collapse
No announcement yet.
X
Collapse
  • Page of 1
  • Filter
  • Time
Clear All
new posts
Previous template Next
    #1

    Serverside exceptions using exportClientData (leaking memory?)

    Hi Isomorphic,

    I got a server exception for a exportClientData request. It *seems* that the exceptions lead to further Java RAM related exceptions in the progress.
    Please see these server logs (using v10.0p_2015-04-10):

    1st request (null pointer exception in com.isomorphic class):
    Code:
    === 2015-04-24 17:39:01,245 [c-39] INFO  RequestContext - URL: '/test/lms/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0': Moz (Gecko) with Accept-Encoding header
=== 2015-04-24 17:39:01,245 [c-39] DEBUG LMSIDACall - Header Name:Value pair: host:REMOVED
=== 2015-04-24 17:39:01,245 [c-39] DEBUG LMSIDACall - Header Name:Value pair: user-agent:Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
=== 2015-04-24 17:39:01,245 [c-39] DEBUG LMSIDACall - Header Name:Value pair: accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
=== 2015-04-24 17:39:01,245 [c-39] DEBUG LMSIDACall - Header Name:Value pair: Accept-Language:de,en-US;q=0.7,en;q=0.3
=== 2015-04-24 17:39:01,246 [c-39] DEBUG LMSIDACall - Header Name:Value pair: Accept-Encoding:gzip, deflate
=== 2015-04-24 17:39:01,246 [c-39] DEBUG LMSIDACall - Header Name:Value pair: referer:REMOVED
=== 2015-04-24 17:39:01,246 [c-39] DEBUG LMSIDACall - Header Name:Value pair: cookie:JSESSIONID=REMOVED; isc_cState=ready; GLog=REMOVED
=== 2015-04-24 17:39:01,246 [c-39] DEBUG LMSIDACall - Header Name:Value pair: connection:keep-alive
=== 2015-04-24 17:39:01,246 [c-39] DEBUG LMSIDACall - Header Name:Value pair: content-type:application/x-www-form-urlencoded
=== 2015-04-24 17:39:01,246 [c-39] DEBUG LMSIDACall - Header Name:Value pair: content-length:12274
=== 2015-04-24 17:39:01,246 [c-39] DEBUG LMSIDACall - session exists: REMOVED
=== 2015-04-24 17:39:01,246 [c-39] DEBUG LMSIDACall - remote user: csuperuser
=== 2015-04-24 17:39:01,249 [c-39] DEBUG RPCManager - Processing 1 requests.
=== 2015-04-24 17:39:01,249 [c-39] DEBUG RPCManager - Request #1 (RPCRequest) data: {
    appID:"isc_builtin",
    className:"builtin",
    methodName:"downloadClientExport",
    arguments:[
        [
            {
                STATUS_SHORTNAME:"Gepickt",
                leadAction:" ",
                STATUS_RESELLER_ID:"ABC-Softwarehaus GmbH (B)",
                STATUS_RESELLER_ID$style:{
                    rawValue:31
                },
                STATUS_USER_ID:"M. Unmüssig",
                STATUS_USER_ID$style:{
                    rawValue:1094
                },
                PICKED_DATE:"08.04.2015",
                PICKED_DATE$style:{
                    rawValue:new Date(1428504613000),
                    dateFormatter:"toEuropeanShortDate"
                },
                STATUS_STATUSDATE:" ",
                CUST_NAME_CONTACTPERSFULLNAME:"Hopsassa Gmbh (Günther Hopsassa)",
                CAMPAIGN_ID:"Hannover Messe 2015",
                CAMPAIGN_ID$style:{
                    rawValue:59
                },
                LEADTEMPERATURE_ID:"Hot Lead",
                LEADTEMPERATURE_ID$style:{
                    rawValue:17
                },
                LEADTYPE_ID:"Event",
                LEADTYPE_ID$style:{
                    rawValue:22
                },
                STATUS_POTREV_COMPANY:" ",
                STATUS_POTREV_PROJECT:" ",
                CREATED_BY:"K. Kowalski",
                CREATED_BY$style:{
                    rawValue:1083
                },
                CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
            },
            {
                STATUS_SHORTNAME:"Gepickt",
                leadAction:" ",
                STATUS_RESELLER_ID:"ABC-Softwarehaus GmbH (B)",
                STATUS_RESELLER_ID$style:{
                    rawValue:31
                },
                STATUS_USER_ID:"M. Unmüssig",
                STATUS_USER_ID$style:{
                    rawValue:1094
                },
                PICKED_DATE:"21.01.2015",
                PICKED_DATE$style:{
                    rawValue:new Date(1421830376000),
                    dateFormatter:"toEuropeanShortDate"
                },
                STATUS_STATUSDATE:" ",
                CUST_NAME_CONTACTPERSFULLNAME:"Test gmbh (Test)",
                CAMPAIGN_ID:"Kündiger-Reaktivierung",
                CAMPAIGN_ID$style:{
                    rawValue:50
                },
                LEADTEMPERATURE_ID:"Marketing qualified Lead",
                LEADTEMPERATURE_ID$style:{
                    rawValue:19
                },
                LEADTYPE_ID:" ",
                STATUS_POTREV_COMPANY:" ",
                STATUS_POTREV_PROJECT:" ",
                CREATED_BY:"K. Kowalski",
                CREATED_BY$style:{
                    rawValue:1083
                },
                CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
            },
            {
                STATUS_SHORTNAME:"Gepickt",
                leadAction:" ",
                STATUS_RESELLER_ID:"Top-Software Reseller UG (MS)",
                STATUS_RESELLER_ID$style:{
                    rawValue:26
                },
                STATUS_USER_ID:"S. Resch",
                STATUS_USER_ID$style:{
                    rawValue:1098
                },
                PICKED_DATE:"26.03.2015",
                PICKED_DATE$style:{
                    rawValue:new Date(1427368899000),
                    dateFormatter:"toEuropeanShortDate"
                },
                STATUS_STATUSDATE:" ",
                CUST_NAME_CONTACTPERSFULLNAME:"tmp Gmbh (Müller)",
                CAMPAIGN_ID:"Cebit Stand 2014",
                CAMPAIGN_ID$style:{
                    rawValue:51
                },
                LEADTEMPERATURE_ID:"Hot Lead",
                LEADTEMPERATURE_ID$style:{
                    rawValue:17
                },
                LEADTYPE_ID:"Trial DL",
                LEADTYPE_ID$style:{
                    rawValue:25
                },
                STATUS_POTREV_COMPANY:" ",
                STATUS_POTREV_PROJECT:" ",
                CREATED_BY:"K. Kowalski",
                CREATED_BY$style:{
                    rawValue:1083
                },
                CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
            },
            {
                STATUS_SHORTNAME:"Angebot",
                leadAction:" ",
                STATUS_RESELLER_ID:"ABC-Softwarehaus GmbH (B)",
                STATUS_RESELLER_ID$style:{
                    rawValue:31
                },
                STATUS_USER_ID:"M. Unmüssig",
                STATUS_USER_ID$style:{
                    rawValue:1094
                },
                PICKED_DATE:"30.03.2015",
                PICKED_DATE$style:{
                    rawValue:new Date(1427724435000),
                    dateFormatter:"toEuropeanShortDate"
                },
                STATUS_STATUSDATE:"30.04.2015",
                STATUS_STATUSDATE$style:{
                    rawValue:new Date(1430344800000),
                    dateFormatter:null
                },
                CUST_NAME_CONTACTPERSFULLNAME:"tmp gmbh (Heiko Wedemeyer)",
                CAMPAIGN_ID:"Cebit Stand 2014",
                CAMPAIGN_ID$style:{
                    rawValue:51
                },
                LEADTEMPERATURE_ID:"Hot Lead",
                LEADTEMPERATURE_ID$style:{
                    rawValue:17
                },
                LEADTYPE_ID:"Event",
                LEADTYPE_ID$style:{
                    rawValue:22
                },
                STATUS_POTREV_COMPANY:"2.000 €",
                STATUS_POTREV_COMPANY$style:{
                    rawValue:2000
                },
                STATUS_POTREV_PROJECT:" ",
                CREATED_BY:"K. Kowalski",
                CREATED_BY$style:{
                    rawValue:1083
                },
                CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
            }
        ],
        "ooxml",
        "picked_leads.xlsx",
        "download",
        {
            exportAs:"ooxml",
            exportDisplay:"download",
            exportFilename:"picked_leads.xlsx",
            exportPath:null,
            exportToClient:null,
            exportToFilesystem:null,
            exportDelimiter:null,
            exportFields:[
                "leadAction",
                "STATUS_RESELLER_ID",
                "STATUS_USER_ID",
                "PICKED_DATE",
                "STATUS_STATUSDATE",
                "CUST_NAME_CONTACTPERSFULLNAME",
                "CAMPAIGN_ID",
                "LEADTEMPERATURE_ID",
                "LEADTYPE_ID",
                "STATUS_POTREV_COMPANY",
                "STATUS_POTREV_PROJECT",
                "CREATED_BY",
                "CREATED_BY_LEGALENTITYNAME"
            ],
            exportHeader:null,
            exportHeaderless:null,
            exportFooter:null,
            exportTitleSeparatorChar:null,
            exportDefaultBGColor:null,
            exportAlternateRowBGColor:null,
            exportRowBGColors:{
            },
            exportColumnBGColors:{
            },
            exportRawValues:false,
            exportCurrencySymbol:"€",
            lineBreakStyle:null,
            exportOtherFields:{
                leadAction:"Aktionen",
                STATUS_RESELLER_ID:"Gepickt von Firma",
                STATUS_USER_ID:"Gepickt von Benutzer",
                PICKED_DATE:"Gepickt am",
                STATUS_STATUSDATE:"Statusdatum",
                CUST_NAME_CONTACTPERSFULLNAME:"Ansprechpartner",
                CAMPAIGN_ID:"Kampagne",
                LEADTEMPERATURE_ID:"Leadtemperatur",
                LEADTYPE_ID:"Leadtyp",
                STATUS_POTREV_COMPANY:"Anteil Kunde (netto)",
                STATUS_POTREV_PROJECT:"Umfang Gesamtprojekt (netto)",
                CREATED_BY:"Erst. v. (User)",
                CREATED_BY_LEGALENTITYNAME:"Erst. v. (Firma)"
            },
            exportWrapHeaderTitles:null
        }
    ],
    is_ISC_RPC_DMI:true
}
=== 2015-04-24 17:39:01,250 [c-39] INFO  LMSIDACall - Performing 1 operation(s)
=== 2015-04-24 17:39:01,258 [c-39] DEBUG RPCDMI - appConfig: isc.Application.create({
    rpcBindings:[
        {
            ID:"builtin",
            className:"com.isomorphic.rpc.BuiltinRPC",
            visibleMethods:[
                {
                    name:"downloadWSDL"
                },
                {
                    name:"downloadClientContent"
                },
                {
                    name:"downloadClientExport"
                },
                {
                    name:"xmlToJS"
                },
                {
                    name:"uploadProgressCheck"
                },
                {
                    name:"saveFile"
                },
                {
                    name:"appendToFile"
                },
                {
                    name:"loadFile"
                },
                {
                    name:"deleteFile"
                },
                {
                    name:"loadSharedXML"
                },
                {
                    name:"saveSharedXML"
                },
                {
                    name:"getAvailableScriptEngines"
                },
                {
                    name:"devConsoleEvalServerScript"
                },
                {
                    name:"evalJava"
                },
                {
                    name:"getLogNames"
                },
                {
                    name:"getLogEntries"
                },
                {
                    name:"clearLogEntries"
                },
                {
                    name:"getLogThresholds"
                },
                {
                    name:"setLogThreshold"
                },
                {
                    name:"setTemporaryLogThreshold"
                },
                {
                    name:"revertTemporaryLogThresholds"
                },
                {
                    name:"getPdfObject"
                },
                {
                    name:"exportImage"
                },
                {
                    name:"areServerTimingsTracked"
                },
                {
                    name:"trackServerTimings"
                }
            ]
        },
        {
            ID:"builtin_tools",
            className:"com.isomorphic.tools.BuiltinRPC",
            visibleMethods:[
                {
                    name:"getDataSourceFromTable"
                },
                {
                    name:"getDataSourceJSONFromTable"
                },
                {
                    name:"getDataSourceFromHibernateMapping"
                },
                {
                    name:"getDataSourceJSONFromHibernateMapping"
                },
                {
                    name:"getTables"
                },
                {
                    name:"getFieldsFromTable"
                },
                {
                    name:"getBeanFields"
                },
                {
                    name:"getHibernateBeans"
                },
                {
                    name:"getDatabaseProductNameAndVersion"
                },
                {
                    name:"getDatabaseTableTypes"
                },
                {
                    name:"setAttributes"
                },
                {
                    name:"clearAttributes"
                },
                {
                    name:"getAttributes"
                },
                {
                    name:"getAttribute"
                },
                {
                    name:"getDataSourceConfigFromJavaClass"
                },
                {
                    args:"cName",
                    language:"groovy",
                    name:"getJavaSource",
                    script:"\n                    if (!com.isomorphic.auth.DevModeAuthFilter.devModeAuthorized(request)) throw new Exception(\"Not Authorized\");                    \n                    //import org.apache.bcel.Repository;\n\n                    try {\n                        return org.apache.bcel.Repository.lookupClass(cName).toString();\n                    } catch (Throwable e) {\n                        return \"Unable to reverse engineer class \"+cName+\": \"+e.getMessage();\n                    }\n                "
                },
                {
                    name:"loadDataSource"
                },
                {
                    name:"dsFromXML"
                },
                {
                    name:"dsConfigFromXML"
                },
                {
                    name:"getDefinedDataSources"
                }
            ]
        },
        {
            ID:"builtin_adminconsole",
            className:"com.isomorphic.tools.AdminConsole",
            visibleMethods:[
                {
                    name:"getDefinedDatabases"
                },
                {
                    name:"testDB"
                },
                {
                    name:"saveDBConfig"
                },
                {
                    name:"setDefaultDB"
                },
                {
                    name:"importDataSources"
                },
                {
                    name:"discoverJNDIDatabases"
                }
            ]
        }
    ]
})

=== 2015-04-24 17:39:01,261 [c-39] INFO  BuiltinRPC - Export data passed to server:
[
    {
        STATUS_SHORTNAME:"Gepickt",
        leadAction:" ",
        STATUS_RESELLER_ID:"ABC-Softwarehaus GmbH (B)",
        STATUS_RESELLER_ID$style:{
            rawValue:31
        },
        STATUS_USER_ID:"M. Unmüssig",
        STATUS_USER_ID$style:{
            rawValue:1094
        },
        PICKED_DATE:"08.04.2015",
        PICKED_DATE$style:{
            rawValue:new Date(1428504613000),
            dateFormatter:"toEuropeanShortDate"
        },
        STATUS_STATUSDATE:" ",
        CUST_NAME_CONTACTPERSFULLNAME:"Hopsassa Gmbh (Günther Hopsassa)",
        CAMPAIGN_ID:"Hannover Messe 2015",
        CAMPAIGN_ID$style:{
            rawValue:59
        },
        LEADTEMPERATURE_ID:"Hot Lead",
        LEADTEMPERATURE_ID$style:{
            rawValue:17
        },
        LEADTYPE_ID:"Event",
        LEADTYPE_ID$style:{
            rawValue:22
        },
        STATUS_POTREV_COMPANY:" ",
        STATUS_POTREV_PROJECT:" ",
        CREATED_BY:"K. Kowalski",
        CREATED_BY$style:{
            rawValue:1083
        },
        CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
    },
    {
        STATUS_SHORTNAME:"Gepickt",
        leadAction:" ",
        STATUS_RESELLER_ID:"ABC-Softwarehaus GmbH (B)",
        STATUS_RESELLER_ID$style:{
            rawValue:31
        },
        STATUS_USER_ID:"M. Unmüssig",
        STATUS_USER_ID$style:{
            rawValue:1094
        },
        PICKED_DATE:"21.01.2015",
        PICKED_DATE$style:{
            rawValue:new Date(1421830376000),
            dateFormatter:"toEuropeanShortDate"
        },
        STATUS_STATUSDATE:" ",
        CUST_NAME_CONTACTPERSFULLNAME:"Test gmbh (Test)",
        CAMPAIGN_ID:"Kündiger-Reaktivierung",
        CAMPAIGN_ID$style:{
            rawValue:50
        },
        LEADTEMPERATURE_ID:"Marketing qualified Lead",
        LEADTEMPERATURE_ID$style:{
            rawValue:19
        },
        LEADTYPE_ID:" ",
        STATUS_POTREV_COMPANY:" ",
        STATUS_POTREV_PROJECT:" ",
        CREATED_BY:"K. Kowalski",
        CREATED_BY$style:{
            rawValue:1083
        },
        CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
    },
    {
        STATUS_SHORTNAME:"Gepickt",
        leadAction:" ",
        STATUS_RESELLER_ID:"Top-Software Reseller UG (MS)",
        STATUS_RESELLER_ID$style:{
            rawValue:26
        },
        STATUS_USER_ID:"S. Resch",
        STATUS_USER_ID$style:{
            rawValue:1098
        },
        PICKED_DATE:"26.03.2015",
        PICKED_DATE$style:{
            rawValue:new Date(1427368899000),
            dateFormatter:"toEuropeanShortDate"
        },
        STATUS_STATUSDATE:" ",
        CUST_NAME_CONTACTPERSFULLNAME:"tmp Gmbh (Müller)",
        CAMPAIGN_ID:"Cebit Stand 2014",
        CAMPAIGN_ID$style:{
            rawValue:51
        },
        LEADTEMPERATURE_ID:"Hot Lead",
        LEADTEMPERATURE_ID$style:{
            rawValue:17
        },
        LEADTYPE_ID:"Trial DL",
        LEADTYPE_ID$style:{
            rawValue:25
        },
        STATUS_POTREV_COMPANY:" ",
        STATUS_POTREV_PROJECT:" ",
        CREATED_BY:"K. Kowalski",
        CREATED_BY$style:{
            rawValue:1083
        },
        CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
    },
    {
        STATUS_SHORTNAME:"Angebot",
        leadAction:" ",
        STATUS_RESELLER_ID:"ABC-Softwarehaus GmbH (B)",
        STATUS_RESELLER_ID$style:{
            rawValue:31
        },
        STATUS_USER_ID:"M. Unmüssig",
        STATUS_USER_ID$style:{
            rawValue:1094
        },
        PICKED_DATE:"30.03.2015",
        PICKED_DATE$style:{
            rawValue:new Date(1427724435000),
            dateFormatter:"toEuropeanShortDate"
        },
        STATUS_STATUSDATE:"30.04.2015",
        STATUS_STATUSDATE$style:{
            rawValue:new Date(1430344800000),
            dateFormatter:null
        },
        CUST_NAME_CONTACTPERSFULLNAME:"tmp gmbh (Heiko Wedemeyer)",
        CAMPAIGN_ID:"Cebit Stand 2014",
        CAMPAIGN_ID$style:{
            rawValue:51
        },
        LEADTEMPERATURE_ID:"Hot Lead",
        LEADTEMPERATURE_ID$style:{
            rawValue:17
        },
        LEADTYPE_ID:"Event",
        LEADTYPE_ID$style:{
            rawValue:22
        },
        STATUS_POTREV_COMPANY:"2.000 €",
        STATUS_POTREV_COMPANY$style:{
            rawValue:2000
        },
        STATUS_POTREV_PROJECT:" ",
        CREATED_BY:"K. Kowalski",
        CREATED_BY$style:{
            rawValue:1083
        },
        CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
    }
]
=== 2015-04-24 17:39:01,263 [c-39] INFO  ProxyHttpServletResponse - content-disposition set to: attachment; fileName=picked_leads.xlsx - unwrapping response output stream (CompressionFilter)
=== 2015-04-24 17:39:01,263 [c-39] DEBUG ConfigParser - Matched var name: webRoot
=== 2015-04-24 17:39:01,264 [c-39] DEBUG ConfigParser - value after substitution: /usr/share/apache-tomcat-7.0.53/lms_test/test
=== 2015-04-24 17:39:01,264 [c-39] INFO  BuiltinRPC - Generating and streaming ooxml file...
=== 2015-04-24 17:39:02,952 [c-39] WARN  BuiltinRPC - Exception during export - continuing anyway to avoid sending the client to a blank screen
java.lang.NullPointerException
	at com.isomorphic.rpc.ExcelDataExport.writeExportObject(ExcelDataExport.java:1313)
	at com.isomorphic.rpc.DataExport.exportResultSet(DataExport.java:640)
	at com.isomorphic.rpc.DataExport.exportResultSet(DataExport.java:460)
	at com.isomorphic.rpc.BuiltinRPC.downloadClientExport(BuiltinRPC.java:850)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.isomorphic.base.Reflection.adaptArgsAndInvoke(Reflection.java:975)
	at com.isomorphic.base.Reflection.adaptArgsAndInvoke(Reflection.java:805)
	at com.isomorphic.rpc.RPCDMI.execute(RPCDMI.java:309)
	at com.isomorphic.rpc.RPCDMI.execute(RPCDMI.java:66)
	at com.isomorphic.rpc.RPCRequest.execute(RPCRequest.java:231)
	at com.isomorphic.servlet.IDACall.handleRPCRequest(IDACall.java:328)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:182)
	at com.lmscompany.lms.server.LMSIDACall.processRequest(LMSIDACall.java:40)
	at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:117)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:76)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:156)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
	at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:83)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:193)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
=== 2015-04-24 17:39:02,953 [c-39] DEBUG RPCDMI - rpc returned data
    2nd request (null pointer exception in com.isomorphic class):
    Code:
    === 2015-04-24 17:39:03,616 [c-56] INFO  RequestContext - URL: '/test/lms/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0': Moz (Gecko) with Accept-Encoding header
=== 2015-04-24 17:39:03,616 [c-56] DEBUG LMSIDACall - Header Name:Value pair: host:REMOVED
=== 2015-04-24 17:39:03,616 [c-56] DEBUG LMSIDACall - Header Name:Value pair: user-agent:Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
=== 2015-04-24 17:39:03,616 [c-56] DEBUG LMSIDACall - Header Name:Value pair: accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
=== 2015-04-24 17:39:03,616 [c-56] DEBUG LMSIDACall - Header Name:Value pair: Accept-Language:de,en-US;q=0.7,en;q=0.3
=== 2015-04-24 17:39:03,616 [c-56] DEBUG LMSIDACall - Header Name:Value pair: Accept-Encoding:gzip, deflate
=== 2015-04-24 17:39:03,616 [c-56] DEBUG LMSIDACall - Header Name:Value pair: referer:REMOVED
=== 2015-04-24 17:39:03,616 [c-56] DEBUG LMSIDACall - Header Name:Value pair: cookie:JSESSIONID=REMOVED; isc_cState=ready; GLog=REMOVED
=== 2015-04-24 17:39:03,616 [c-56] DEBUG LMSIDACall - Header Name:Value pair: connection:keep-alive
=== 2015-04-24 17:39:03,616 [c-56] DEBUG LMSIDACall - Header Name:Value pair: content-type:application/x-www-form-urlencoded
=== 2015-04-24 17:39:03,616 [c-56] DEBUG LMSIDACall - Header Name:Value pair: content-length:12274
=== 2015-04-24 17:39:03,616 [c-56] DEBUG LMSIDACall - session exists: REMOVED
=== 2015-04-24 17:39:03,616 [c-56] DEBUG LMSIDACall - remote user: csuperuser
=== 2015-04-24 17:39:03,620 [c-56] DEBUG RPCManager - Processing 1 requests.
=== 2015-04-24 17:39:03,620 [c-56] DEBUG RPCManager - Request #1 (RPCRequest) data: {
    appID:"isc_builtin",
    className:"builtin",
    methodName:"downloadClientExport",
    arguments:[
        [
            {
                STATUS_SHORTNAME:"Gepickt",
                leadAction:" ",
                STATUS_RESELLER_ID:"ABC-Softwarehaus GmbH (B)",
                STATUS_RESELLER_ID$style:{
                    rawValue:31
                },
                STATUS_USER_ID:"M. Unmüssig",
                STATUS_USER_ID$style:{
                    rawValue:1094
                },
                PICKED_DATE:"08.04.2015",
                PICKED_DATE$style:{
                    rawValue:new Date(1428504613000),
                    dateFormatter:"toEuropeanShortDate"
                },
                STATUS_STATUSDATE:" ",
                CUST_NAME_CONTACTPERSFULLNAME:"Hopsassa Gmbh (Günther Hopsassa)",
                CAMPAIGN_ID:"Hannover Messe 2015",
                CAMPAIGN_ID$style:{
                    rawValue:59
                },
                LEADTEMPERATURE_ID:"Hot Lead",
                LEADTEMPERATURE_ID$style:{
                    rawValue:17
                },
                LEADTYPE_ID:"Event",
                LEADTYPE_ID$style:{
                    rawValue:22
                },
                STATUS_POTREV_COMPANY:" ",
                STATUS_POTREV_PROJECT:" ",
                CREATED_BY:"K. Kowalski",
                CREATED_BY$style:{
                    rawValue:1083
                },
                CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
            },
            {
                STATUS_SHORTNAME:"Gepickt",
                leadAction:" ",
                STATUS_RESELLER_ID:"ABC-Softwarehaus GmbH (B)",
                STATUS_RESELLER_ID$style:{
                    rawValue:31
                },
                STATUS_USER_ID:"M. Unmüssig",
                STATUS_USER_ID$style:{
                    rawValue:1094
                },
                PICKED_DATE:"21.01.2015",
                PICKED_DATE$style:{
                    rawValue:new Date(1421830376000),
                    dateFormatter:"toEuropeanShortDate"
                },
                STATUS_STATUSDATE:" ",
                CUST_NAME_CONTACTPERSFULLNAME:"Test gmbh (Test)",
                CAMPAIGN_ID:"Kündiger-Reaktivierung",
                CAMPAIGN_ID$style:{
                    rawValue:50
                },
                LEADTEMPERATURE_ID:"Marketing qualified Lead",
                LEADTEMPERATURE_ID$style:{
                    rawValue:19
                },
                LEADTYPE_ID:" ",
                STATUS_POTREV_COMPANY:" ",
                STATUS_POTREV_PROJECT:" ",
                CREATED_BY:"K. Kowalski",
                CREATED_BY$style:{
                    rawValue:1083
                },
                CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
            },
            {
                STATUS_SHORTNAME:"Gepickt",
                leadAction:" ",
                STATUS_RESELLER_ID:"Top-Software Reseller UG (MS)",
                STATUS_RESELLER_ID$style:{
                    rawValue:26
                },
                STATUS_USER_ID:"S. Resch",
                STATUS_USER_ID$style:{
                    rawValue:1098
                },
                PICKED_DATE:"26.03.2015",
                PICKED_DATE$style:{
                    rawValue:new Date(1427368899000),
                    dateFormatter:"toEuropeanShortDate"
                },
                STATUS_STATUSDATE:" ",
                CUST_NAME_CONTACTPERSFULLNAME:"tmp Gmbh (Müller)",
                CAMPAIGN_ID:"Cebit Stand 2014",
                CAMPAIGN_ID$style:{
                    rawValue:51
                },
                LEADTEMPERATURE_ID:"Hot Lead",
                LEADTEMPERATURE_ID$style:{
                    rawValue:17
                },
                LEADTYPE_ID:"Trial DL",
                LEADTYPE_ID$style:{
                    rawValue:25
                },
                STATUS_POTREV_COMPANY:" ",
                STATUS_POTREV_PROJECT:" ",
                CREATED_BY:"K. Kowalski",
                CREATED_BY$style:{
                    rawValue:1083
                },
                CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
            },
            {
                STATUS_SHORTNAME:"Angebot",
                leadAction:" ",
                STATUS_RESELLER_ID:"ABC-Softwarehaus GmbH (B)",
                STATUS_RESELLER_ID$style:{
                    rawValue:31
                },
                STATUS_USER_ID:"M. Unmüssig",
                STATUS_USER_ID$style:{
                    rawValue:1094
                },
                PICKED_DATE:"30.03.2015",
                PICKED_DATE$style:{
                    rawValue:new Date(1427724435000),
                    dateFormatter:"toEuropeanShortDate"
                },
                STATUS_STATUSDATE:"30.04.2015",
                STATUS_STATUSDATE$style:{
                    rawValue:new Date(1430344800000),
                    dateFormatter:null
                },
                CUST_NAME_CONTACTPERSFULLNAME:"tmp gmbh (Heiko Wedemeyer)",
                CAMPAIGN_ID:"Cebit Stand 2014",
                CAMPAIGN_ID$style:{
                    rawValue:51
                },
                LEADTEMPERATURE_ID:"Hot Lead",
                LEADTEMPERATURE_ID$style:{
                    rawValue:17
                },
                LEADTYPE_ID:"Event",
                LEADTYPE_ID$style:{
                    rawValue:22
                },
                STATUS_POTREV_COMPANY:"2.000 €",
                STATUS_POTREV_COMPANY$style:{
                    rawValue:2000
                },
                STATUS_POTREV_PROJECT:" ",
                CREATED_BY:"K. Kowalski",
                CREATED_BY$style:{
                    rawValue:1083
                },
                CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
            }
        ],
        "ooxml",
        "picked_leads.xlsx",
        "download",
        {
            exportAs:"ooxml",
            exportDisplay:"download",
            exportFilename:"picked_leads.xlsx",
            exportPath:null,
            exportToClient:null,
            exportToFilesystem:null,
            exportDelimiter:null,
            exportFields:[
                "leadAction",
                "STATUS_RESELLER_ID",
                "STATUS_USER_ID",
                "PICKED_DATE",
                "STATUS_STATUSDATE",
                "CUST_NAME_CONTACTPERSFULLNAME",
                "CAMPAIGN_ID",
                "LEADTEMPERATURE_ID",
                "LEADTYPE_ID",
                "STATUS_POTREV_COMPANY",
                "STATUS_POTREV_PROJECT",
                "CREATED_BY",
                "CREATED_BY_LEGALENTITYNAME"
            ],
            exportHeader:null,
            exportHeaderless:null,
            exportFooter:null,
            exportTitleSeparatorChar:null,
            exportDefaultBGColor:null,
            exportAlternateRowBGColor:null,
            exportRowBGColors:{
            },
            exportColumnBGColors:{
            },
            exportRawValues:false,
            exportCurrencySymbol:"€",
            lineBreakStyle:null,
            exportOtherFields:{
                leadAction:"Aktionen",
                STATUS_RESELLER_ID:"Gepickt von Firma",
                STATUS_USER_ID:"Gepickt von Benutzer",
                PICKED_DATE:"Gepickt am",
                STATUS_STATUSDATE:"Statusdatum",
                CUST_NAME_CONTACTPERSFULLNAME:"Ansprechpartner",
                CAMPAIGN_ID:"Kampagne",
                LEADTEMPERATURE_ID:"Leadtemperatur",
                LEADTYPE_ID:"Leadtyp",
                STATUS_POTREV_COMPANY:"Anteil Kunde (netto)",
                STATUS_POTREV_PROJECT:"Umfang Gesamtprojekt (netto)",
                CREATED_BY:"Erst. v. (User)",
                CREATED_BY_LEGALENTITYNAME:"Erst. v. (Firma)"
            },
            exportWrapHeaderTitles:null
        }
    ],
    is_ISC_RPC_DMI:true
}
=== 2015-04-24 17:39:03,620 [c-56] INFO  LMSIDACall - Performing 1 operation(s)
=== 2015-04-24 17:39:03,621 [c-56] DEBUG RPCDMI - appConfig: isc.Application.create({
    rpcBindings:[
        {
            ID:"builtin",
            className:"com.isomorphic.rpc.BuiltinRPC",
            visibleMethods:[
                {
                    name:"downloadWSDL"
                },
                {
                    name:"downloadClientContent"
                },
                {
                    name:"downloadClientExport"
                },
                {
                    name:"xmlToJS"
                },
                {
                    name:"uploadProgressCheck"
                },
                {
                    name:"saveFile"
                },
                {
                    name:"appendToFile"
                },
                {
                    name:"loadFile"
                },
                {
                    name:"deleteFile"
                },
                {
                    name:"loadSharedXML"
                },
                {
                    name:"saveSharedXML"
                },
                {
                    name:"getAvailableScriptEngines"
                },
                {
                    name:"devConsoleEvalServerScript"
                },
                {
                    name:"evalJava"
                },
                {
                    name:"getLogNames"
                },
                {
                    name:"getLogEntries"
                },
                {
                    name:"clearLogEntries"
                },
                {
                    name:"getLogThresholds"
                },
                {
                    name:"setLogThreshold"
                },
                {
                    name:"setTemporaryLogThreshold"
                },
                {
                    name:"revertTemporaryLogThresholds"
                },
                {
                    name:"getPdfObject"
                },
                {
                    name:"exportImage"
                },
                {
                    name:"areServerTimingsTracked"
                },
                {
                    name:"trackServerTimings"
                }
            ]
        },
        {
            ID:"builtin_tools",
            className:"com.isomorphic.tools.BuiltinRPC",
            visibleMethods:[
                {
                    name:"getDataSourceFromTable"
                },
                {
                    name:"getDataSourceJSONFromTable"
                },
                {
                    name:"getDataSourceFromHibernateMapping"
                },
                {
                    name:"getDataSourceJSONFromHibernateMapping"
                },
                {
                    name:"getTables"
                },
                {
                    name:"getFieldsFromTable"
                },
                {
                    name:"getBeanFields"
                },
                {
                    name:"getHibernateBeans"
                },
                {
                    name:"getDatabaseProductNameAndVersion"
                },
                {
                    name:"getDatabaseTableTypes"
                },
                {
                    name:"setAttributes"
                },
                {
                    name:"clearAttributes"
                },
                {
                    name:"getAttributes"
                },
                {
                    name:"getAttribute"
                },
                {
                    name:"getDataSourceConfigFromJavaClass"
                },
                {
                    args:"cName",
                    language:"groovy",
                    name:"getJavaSource",
                    script:"\n                    if (!com.isomorphic.auth.DevModeAuthFilter.devModeAuthorized(request)) throw new Exception(\"Not Authorized\");                    \n                    //import org.apache.bcel.Repository;\n\n                    try {\n                        return org.apache.bcel.Repository.lookupClass(cName).toString();\n                    } catch (Throwable e) {\n                        return \"Unable to reverse engineer class \"+cName+\": \"+e.getMessage();\n                    }\n                "
                },
                {
                    name:"loadDataSource"
                },
                {
                    name:"dsFromXML"
                },
                {
                    name:"dsConfigFromXML"
                },
                {
                    name:"getDefinedDataSources"
                }
            ]
        },
        {
            ID:"builtin_adminconsole",
            className:"com.isomorphic.tools.AdminConsole",
            visibleMethods:[
                {
                    name:"getDefinedDatabases"
                },
                {
                    name:"testDB"
                },
                {
                    name:"saveDBConfig"
                },
                {
                    name:"setDefaultDB"
                },
                {
                    name:"importDataSources"
                },
                {
                    name:"discoverJNDIDatabases"
                }
            ]
        }
    ]
})

=== 2015-04-24 17:39:03,622 [c-56] INFO  BuiltinRPC - Export data passed to server:
[
    {
        STATUS_SHORTNAME:"Gepickt",
        leadAction:" ",
        STATUS_RESELLER_ID:"ABC-Softwarehaus GmbH (B)",
        STATUS_RESELLER_ID$style:{
            rawValue:31
        },
        STATUS_USER_ID:"M. Unmüssig",
        STATUS_USER_ID$style:{
            rawValue:1094
        },
        PICKED_DATE:"08.04.2015",
        PICKED_DATE$style:{
            rawValue:new Date(1428504613000),
            dateFormatter:"toEuropeanShortDate"
        },
        STATUS_STATUSDATE:" ",
        CUST_NAME_CONTACTPERSFULLNAME:"Hopsassa Gmbh (Günther Hopsassa)",
        CAMPAIGN_ID:"Hannover Messe 2015",
        CAMPAIGN_ID$style:{
            rawValue:59
        },
        LEADTEMPERATURE_ID:"Hot Lead",
        LEADTEMPERATURE_ID$style:{
            rawValue:17
        },
        LEADTYPE_ID:"Event",
        LEADTYPE_ID$style:{
            rawValue:22
        },
        STATUS_POTREV_COMPANY:" ",
        STATUS_POTREV_PROJECT:" ",
        CREATED_BY:"K. Kowalski",
        CREATED_BY$style:{
            rawValue:1083
        },
        CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
    },
    {
        STATUS_SHORTNAME:"Gepickt",
        leadAction:" ",
        STATUS_RESELLER_ID:"ABC-Softwarehaus GmbH (B)",
        STATUS_RESELLER_ID$style:{
            rawValue:31
        },
        STATUS_USER_ID:"M. Unmüssig",
        STATUS_USER_ID$style:{
            rawValue:1094
        },
        PICKED_DATE:"21.01.2015",
        PICKED_DATE$style:{
            rawValue:new Date(1421830376000),
            dateFormatter:"toEuropeanShortDate"
        },
        STATUS_STATUSDATE:" ",
        CUST_NAME_CONTACTPERSFULLNAME:"Test gmbh (Test)",
        CAMPAIGN_ID:"Kündiger-Reaktivierung",
        CAMPAIGN_ID$style:{
            rawValue:50
        },
        LEADTEMPERATURE_ID:"Marketing qualified Lead",
        LEADTEMPERATURE_ID$style:{
            rawValue:19
        },
        LEADTYPE_ID:" ",
        STATUS_POTREV_COMPANY:" ",
        STATUS_POTREV_PROJECT:" ",
        CREATED_BY:"K. Kowalski",
        CREATED_BY$style:{
            rawValue:1083
        },
        CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
    },
    {
        STATUS_SHORTNAME:"Gepickt",
        leadAction:" ",
        STATUS_RESELLER_ID:"Top-Software Reseller UG (MS)",
        STATUS_RESELLER_ID$style:{
            rawValue:26
        },
        STATUS_USER_ID:"S. Resch",
        STATUS_USER_ID$style:{
            rawValue:1098
        },
        PICKED_DATE:"26.03.2015",
        PICKED_DATE$style:{
            rawValue:new Date(1427368899000),
            dateFormatter:"toEuropeanShortDate"
        },
        STATUS_STATUSDATE:" ",
        CUST_NAME_CONTACTPERSFULLNAME:"tmp Gmbh (Müller)",
        CAMPAIGN_ID:"Cebit Stand 2014",
        CAMPAIGN_ID$style:{
            rawValue:51
        },
        LEADTEMPERATURE_ID:"Hot Lead",
        LEADTEMPERATURE_ID$style:{
            rawValue:17
        },
        LEADTYPE_ID:"Trial DL",
        LEADTYPE_ID$style:{
            rawValue:25
        },
        STATUS_POTREV_COMPANY:" ",
        STATUS_POTREV_PROJECT:" ",
        CREATED_BY:"K. Kowalski",
        CREATED_BY$style:{
            rawValue:1083
        },
        CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
    },
    {
        STATUS_SHORTNAME:"Angebot",
        leadAction:" ",
        STATUS_RESELLER_ID:"ABC-Softwarehaus GmbH (B)",
        STATUS_RESELLER_ID$style:{
            rawValue:31
        },
        STATUS_USER_ID:"M. Unmüssig",
        STATUS_USER_ID$style:{
            rawValue:1094
        },
        PICKED_DATE:"30.03.2015",
        PICKED_DATE$style:{
            rawValue:new Date(1427724435000),
            dateFormatter:"toEuropeanShortDate"
        },
        STATUS_STATUSDATE:"30.04.2015",
        STATUS_STATUSDATE$style:{
            rawValue:new Date(1430344800000),
            dateFormatter:null
        },
        CUST_NAME_CONTACTPERSFULLNAME:"tmp gmbh (Heiko Wedemeyer)",
        CAMPAIGN_ID:"Cebit Stand 2014",
        CAMPAIGN_ID$style:{
            rawValue:51
        },
        LEADTEMPERATURE_ID:"Hot Lead",
        LEADTEMPERATURE_ID$style:{
            rawValue:17
        },
        LEADTYPE_ID:"Event",
        LEADTYPE_ID$style:{
            rawValue:22
        },
        STATUS_POTREV_COMPANY:"2.000 €",
        STATUS_POTREV_COMPANY$style:{
            rawValue:2000
        },
        STATUS_POTREV_PROJECT:" ",
        CREATED_BY:"K. Kowalski",
        CREATED_BY$style:{
            rawValue:1083
        },
        CREATED_BY_LEGALENTITYNAME:"OURCOMPANY TESTACCOUNT"
    }
]
=== 2015-04-24 17:39:03,622 [c-56] INFO  ProxyHttpServletResponse - content-disposition set to: attachment; fileName=picked_leads.xlsx - unwrapping response output stream (CompressionFilter)
=== 2015-04-24 17:39:03,622 [c-56] INFO  BuiltinRPC - Generating and streaming ooxml file...
=== 2015-04-24 17:39:03,623 [c-56] WARN  BuiltinRPC - Exception during export - continuing anyway to avoid sending the client to a blank screen
java.lang.NullPointerException
	at com.isomorphic.rpc.ExcelDataExport.writeExportObject(ExcelDataExport.java:1313)
	at com.isomorphic.rpc.DataExport.exportResultSet(DataExport.java:640)
	at com.isomorphic.rpc.DataExport.exportResultSet(DataExport.java:460)
	at com.isomorphic.rpc.BuiltinRPC.downloadClientExport(BuiltinRPC.java:850)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.isomorphic.base.Reflection.adaptArgsAndInvoke(Reflection.java:975)
	at com.isomorphic.base.Reflection.adaptArgsAndInvoke(Reflection.java:805)
	at com.isomorphic.rpc.RPCDMI.execute(RPCDMI.java:309)
	at com.isomorphic.rpc.RPCDMI.execute(RPCDMI.java:66)
	at com.isomorphic.rpc.RPCRequest.execute(RPCRequest.java:231)
	at com.isomorphic.servlet.IDACall.handleRPCRequest(IDACall.java:328)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:182)
	at com.lmscompany.lms.server.LMSIDACall.processRequest(LMSIDACall.java:40)
	at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:117)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:76)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:156)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
	at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:83)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:193)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
=== 2015-04-24 17:39:03,624 [c-56] DEBUG RPCDMI - rpc returned data
    I put both request logs in WinMerge - they are the same besides the timestamp. I don't know what happened here. For some seconds, other requests are processed normally, but then the logs are full of "java.lang.Exception: java.lang.OutOfMemoryError: PermGen space".
    The two requests I have shown are the first ones in today's logs (after 17 hours) resulting in a Java exceptions - operations were normal before.

    1st out of memory request:
    Code:
    === 2015-04-24 17:40:08,380 [c-31] INFO  RequestContext - URL: '/test/lms/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0': Moz (Gecko) with Accept-Encoding header
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - Header Name:Value pair: host:REMOVED
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - Header Name:Value pair: user-agent:Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - Header Name:Value pair: accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - Header Name:Value pair: Accept-Language:de,en-US;q=0.7,en;q=0.3
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - Header Name:Value pair: Accept-Encoding:gzip, deflate
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - Header Name:Value pair: content-type:application/x-www-form-urlencoded; charset=UTF-8
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - Header Name:Value pair: referer:REMOVED
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - Header Name:Value pair: content-length:1059
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - Header Name:Value pair: cookie:JSESSIONID=REMOVED; isc_cState=ready; GLog=REMOVED
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - Header Name:Value pair: connection:keep-alive
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - Header Name:Value pair: pragma:no-cache
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - Header Name:Value pair: Cache-Control:no-cache
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - session exists: REMOVED
=== 2015-04-24 17:40:08,380 [c-31] DEBUG LMSIDACall - remote user: csuperuser
=== 2015-04-24 17:40:08,381 [c-31] DEBUG RPCManager - Processing 1 requests.
=== 2015-04-24 17:40:08,381 [c-31] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
        ID:51
    },
    operationConfig:{
        dataSource:"T_CAMPAIGN",
        repo:null,
        operationType:"fetch",
        textMatchStyle:"exact"
    },
    componentId:"isc_AddEditLeadMasterdata_2_0",
    appID:"builtinApplication",
    operation:"fetchOwnAndOtherLECampaigns",
    oldValues:{
        ID:51
    }
}
=== 2015-04-24 17:40:08,381 [c-31] INFO  LMSIDACall - Performing 1 operation(s)
=== 2015-04-24 17:40:08,805 [c-31] WARN  DataSourceDMI - Failed to create serverObject using operation level config. Will try to use DataSource level config instead. Actual error: PermGen space
=== 2015-04-24 17:40:09,180 [c-31] WARN  RequestContext - dsRequest.execute() failed: 
java.lang.Exception: java.lang.OutOfMemoryError: PermGen space
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:246)
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:64)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2544)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:220)
	at com.lmscompany.lms.server.LMSIDACall.handleDSRequest(LMSIDACall.java:75)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:185)
	at com.lmscompany.lms.server.LMSIDACall.processRequest(LMSIDACall.java:40)
	at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:117)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:76)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:156)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
	at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:83)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:193)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.OutOfMemoryError: PermGen space
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:2957)
	at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:1210)
	at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1690)
	at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:270)
	at com.isomorphic.base.Reflection.classForName(Reflection.java:139)
	at com.isomorphic.rpc.ServerObject.<init>(ServerObject.java:157)
	at com.isomorphic.rpc.ServerObject.<init>(ServerObject.java:70)
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:236)
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:64)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2544)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:220)
	at com.lmscompany.lms.server.LMSIDACall.handleDSRequest(LMSIDACall.java:75)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:185)
	at com.lmscompany.lms.server.LMSIDACall.processRequest(LMSIDACall.java:40)
	at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:117)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:76)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:156)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
	at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:83)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    2nd out of memory request:
    Code:
    === 2015-04-24 17:40:08,381 [c-50] INFO  RequestContext - URL: '/test/lms/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0': Moz (Gecko) with Accept-Encoding header
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - Header Name:Value pair: host:REMOVED
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - Header Name:Value pair: user-agent:Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - Header Name:Value pair: accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - Header Name:Value pair: Accept-Language:de,en-US;q=0.7,en;q=0.3
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - Header Name:Value pair: Accept-Encoding:gzip, deflate
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - Header Name:Value pair: content-type:application/x-www-form-urlencoded; charset=UTF-8
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - Header Name:Value pair: referer:REMOVED
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - Header Name:Value pair: content-length:1282
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - Header Name:Value pair: cookie:JSESSIONID=REMOVED; isc_cState=ready; GLog=REMOVED
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - Header Name:Value pair: connection:keep-alive
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - Header Name:Value pair: pragma:no-cache
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - Header Name:Value pair: Cache-Control:no-cache
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - session exists: REMOVED
=== 2015-04-24 17:40:08,381 [c-50] DEBUG LMSIDACall - remote user: csuperuser
=== 2015-04-24 17:40:09,181 [c-50] DEBUG RPCManager - Processing 1 requests.
=== 2015-04-24 17:40:09,181 [c-50] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
        operator:"equals",
        fieldName:"LEAD_ID",
        value:85,
        _constructor:"AdvancedCriteria"
    },
    operationConfig:{
        dataSource:"V_LEAD_RESELLER_MATCH_ALL_RD",
        repo:null,
        operationType:"fetch",
        textMatchStyle:"exact"
    },
    appID:"builtinApplication",
    operation:"countPossibleResellers",
    oldValues:{
        operator:"equals",
        fieldName:"LEAD_ID",
        value:85,
        _constructor:"AdvancedCriteria"
    }
}
=== 2015-04-24 17:40:09,181 [c-50] INFO  LMSIDACall - Performing 1 operation(s)
=== 2015-04-24 17:40:09,688 [c-50] WARN  RequestContext - dsRequest.execute() failed: 
java.lang.OutOfMemoryError: PermGen space
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:2957)
	at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:1210)
	at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1690)
	at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
	at com.isomorphic.datasource.DSRequest.convertMapOfStringToMapOfSummaryFunctionType(DSRequest.java:642)
	at com.isomorphic.datasource.DSRequest.buildSummaryFields(DSRequest.java:1648)
	at com.isomorphic.datasource.DSRequest.buildFieldData(DSRequest.java:3053)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2392)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:220)
	at com.lmscompany.lms.server.LMSIDACall.handleDSRequest(LMSIDACall.java:75)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:185)
	at com.lmscompany.lms.server.LMSIDACall.processRequest(LMSIDACall.java:40)
	at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:117)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:76)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:156)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
	at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:83)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610)
    3rd out of memory request:
    Code:
    === 2015-04-24 17:40:19,249 [c-54] INFO  RequestContext - URL: '/test/lms/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0': Moz (Gecko) with Accept-Encoding header
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - Header Name:Value pair: host:REMOVED
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - Header Name:Value pair: user-agent:Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - Header Name:Value pair: accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - Header Name:Value pair: Accept-Language:de,en-US;q=0.7,en;q=0.3
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - Header Name:Value pair: Accept-Encoding:gzip, deflate
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - Header Name:Value pair: content-type:application/x-www-form-urlencoded; charset=UTF-8
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - Header Name:Value pair: referer:REMOVED
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - Header Name:Value pair: content-length:1566
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - Header Name:Value pair: cookie:JSESSIONID=REMOVED; isc_cState=ready; GLog=REMOVED
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - Header Name:Value pair: connection:keep-alive
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - Header Name:Value pair: pragma:no-cache
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - Header Name:Value pair: Cache-Control:no-cache
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - session exists: REMOVED
=== 2015-04-24 17:40:19,249 [c-54] DEBUG LMSIDACall - remote user: csuperuser
=== 2015-04-24 17:40:19,251 [c-54] DEBUG RPCManager - Processing 1 requests.
=== 2015-04-24 17:40:19,251 [c-54] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
        operator:"equals",
        fieldName:"LEAD_ID",
        value:85,
        _constructor:"AdvancedCriteria"
    },
    operationConfig:{
        dataSource:"V_LEAD_STATUS_HISTORY",
        repo:null,
        operationType:"fetch",
        textMatchStyle:"exact"
    },
    startRow:0,
    endRow:75,
    sortBy:[
        "-CHANGENUMBER"
    ],
    componentId:"isc_ShowLeadHistoryStatus_1_0",
    appID:"builtinApplication",
    operation:"V_LEAD_STATUS_HISTORY_fetch",
    oldValues:{
        operator:"equals",
        fieldName:"LEAD_ID",
        value:85,
        _constructor:"AdvancedCriteria"
    }
}
=== 2015-04-24 17:40:19,251 [c-54] INFO  LMSIDACall - Performing 1 operation(s)
=== 2015-04-24 17:40:19,770 [c-54] WARN  DataSourceDMI - Failed to create serverObject using operation level config. Will try to use DataSource level config instead. Actual error: PermGen space
=== 2015-04-24 17:40:20,161 [c-54] WARN  RequestContext - dsRequest.execute() failed: 
java.lang.Exception: java.lang.OutOfMemoryError: PermGen space
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:246)
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:64)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2544)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:220)
	at com.lmscompany.lms.server.LMSIDACall.handleDSRequest(LMSIDACall.java:75)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:185)
	at com.lmscompany.lms.server.LMSIDACall.processRequest(LMSIDACall.java:40)
	at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:117)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:76)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:156)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
	at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:83)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:193)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.OutOfMemoryError: PermGen space
=== 2015-04-24 17:40:20,289 [c-54] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
=== 2015-04-24 17:40:20,289 [c-54] DEBUG RPCManager - non-DMI response, dropExtraFields: false
=== 2015-04-24 17:40:20,290 [c-54] INFO  Compression - /test/lms/sc/IDACall: 177 -> 158 bytes
    and so on.

    I restarted Tomcat after noticing, which stopped the exceptions for me. Also, I'm running many instances of the application under different Tomcat server.xml <Host>-tags. It seems, the other hosts were not affected by the error.
    I can provide full logs via email, if needed. I don't have any linux memory listings from before the exception but can provide all needed current RAM usage and configurations parameters.

    Does this (java.lang.NullPointerException at com.isomorphic.rpc.ExcelDataExport.writeExportObject(ExcelDataExport.java:1313)) look like a possible bug?

    Best regards
    Blama
    Tags: None
    #2
    The exception from the first log looks impossible from a code analysis point of view. Is this something you can consistently reproduce? Because an explanation that fits both kinds of crashes is that Tomcat has for some reason loaded multiple copies of classes in memory, or is perhaps repeatedly reloading class definitions because it doesn't realize it already has them loaded - this would explain both the seemingly impossible crash, and running out of memory.

    Comment

      #3
      Hi Isomorphic,

      I can't reproduce. It just happened for the first time and only once and not again for the same export after the Tomcat restart.

      Hopefully it is related to some weird Tomcat Bug then and will not reoccur.

      Best regards
      Blama
      Last edited by Blama; 25 Apr 2015, 00:50.

      Comment

        #4
        Hi Isomorphic,

        something similar happened again today:
        One exception:
        Code:
        === 2015-04-30 11:08:14,503 [c-72] INFO  RequestContext - URL: '/REMOVED/lms/sc/DataSourceLoader', User-Agent: 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36': Safari with Accept-Encoding header
=== 2015-04-30 11:08:14,504 [c-72] DEBUG DataSourceLoader - For DataSource 'MV_ZIPCODE', got instanceId 598
=== 2015-04-30 11:08:14,752 [c-72] ERROR DataSourceLoader - Exception while attempting to load a DataSource
java.lang.OutOfMemoryError: PermGen space
=== 2015-04-30 11:08:14,753 [c-72] ERROR DataSourceLoader - BaseServlet Global Exception
javax.servlet.ServletException: DataSource 'MV_ZIPCODE' failed to load due to an exception on the server:
PermGen space
See the server-side log for additional details.
	at com.isomorphic.servlet.DataSourceLoader.processRequest(DataSourceLoader.java:258)
	at com.isomorphic.servlet.DataSourceLoader.doGet(DataSourceLoader.java:98)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:156)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at com.lmscompany.lms.server.MimeTypeFilter.doFilter(MimeTypeFilter.java:23)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
	at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:83)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:193)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
        Next exception (very similar):
        Code:
        === 2015-04-30 11:26:18,949 [c-43] INFO  RequestContext - URL: '/REMOVED/lms/sc/DataSourceLoader', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0': Moz (Gecko) with Accept-Encoding header
=== 2015-04-30 11:26:18,950 [c-43] DEBUG DataSourceLoader - For DataSource 'MV_ZIPCODE', got instanceId 599
=== 2015-04-30 11:26:19,199 [c-43] ERROR DataSourceLoader - Exception while attempting to load a DataSource
java.lang.OutOfMemoryError: PermGen space
=== 2015-04-30 11:26:19,199 [c-43] ERROR DataSourceLoader - BaseServlet Global Exception
javax.servlet.ServletException: DataSource 'MV_ZIPCODE' failed to load due to an exception on the server:
PermGen space
See the server-side log for additional details.
	at com.isomorphic.servlet.DataSourceLoader.processRequest(DataSourceLoader.java:258)
	at com.isomorphic.servlet.DataSourceLoader.doGet(DataSourceLoader.java:98)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:156)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at com.lmscompany.lms.server.MimeTypeFilter.doFilter(MimeTypeFilter.java:23)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
	at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:83)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
	at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:193)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
        Interesting observation: The two logs are almost the same besides the timestamps and the instanceId.
        It happened during the login process while no other users were logged in.
        Besides the normal users I have a send-email-servlet that only uses SmartGWT server classes and is hit on a regular basis via cron-job+wget. This servlet, which also makes use of DataSources, worked like a charm all the time. In fact I have 8 exceptions in total, this time all like the above (although not reproducible after restart). I then restarted Tomcat and everything is normal now, again.

        Does the log above give pointers about what might be happening?
        Again, I'd have the full logs available.

        Best regards
        Blama

        Comment

          #5
          If your server runs out of memory, whatever happens to be running right then is going to get an exception and generally crash.

          The particular bit of code that happens to be running when memory is exhausted is basically random; seeing a particular piece of code run into an out of memory error doesn't mean that it is involved in the leak, it just means that that code allocates memory.

          We've got a lot of sites under high load that aren't showing any errors of this kind, so, this is likely to be either an application bug or some unusual usage pattern in your code.

          The best way to discover the problem is probably going to be load testing, which is a useful exercise in its own right (for the other kinds of problems it can uncover).

          You might also try a heap inspector, but that can get very technical and might be a useless endeavor. At least with load testing, you will have a set of load tests you can reuse in the future.

          Comment

            #6
            Originally posted by Isomorphic View Post
            If your server runs out of memory, whatever happens to be running right then is going to get an exception and generally crash.

            The particular bit of code that happens to be running when memory is exhausted is basically random; seeing a particular piece of code run into an out of memory error doesn't mean that it is involved in the leak, it just means that that code allocates memory.
            That is clear but does not comply with the observation: All the serverside requests in between the first exception and the server restart went trough smoothly (over a timespan of more than three hours!). Also, the exception 8 times at the exact same location seems at least a bit weird - like there might be a correlation.

            Nevertheless I'll have to approach the load testing.

            Best regards
            Blama

            Comment

              #7
              We missed the fact that a bunch of other requests were handled normally in between the out of memory errors. That's suggestive of some kind of background process going on and temporarily allocating a lot of memory. If you don't have anything implemented along the lines of Quartz timers, it may be some kind of Tomcat housekeeping function.

              Aside from just researching whether Tomcat has such functions (and perhaps experimenting with turning them off to see if it fixes the problem), you might also run a CPU load monitor to see if Tomcat starts doing a bunch of work right around the time this exception occurs.

              As far as the problem happening in the same spot - that bit of code is probably the single largest memory allocation that happens during normal request processing, so it makes sense that that would be where the exception occurs.

              Comment

                #8
                Hi Isomorphic,

                thanks for the pointers. I'll have to search in the Tomcat area, then. Googleing gave hint:
                http://wiki.apache.org/tomcat/OutOfMemory:
                Originally posted by Tomcat Docs
                A webapp that uses lots of libraries with many dependencies, or a server maintaining lots of webapps could exhauste the JVM PermGen space. This space is where the VM stores the classes and methods data. In those cases, the fix is to increase this size. The Sun VM has the flag -XX:MaxPermSize that allows to set its size (the default value is 64M)
                This sounds like a possible reason to me as I'm deploying basically the same project with just different server.properties and different static files (images, skin_styles.css) many times (right now 10 times).
                I'll try to analyse the PermGen usage and increase the value.

                Best regards,
                Blama

                Comment

                Previous template Next
                Working...
                X