[Isomorphic]

This is server-side functionality, so you want the server-side log. The default settings for the framework are to log via log4j, and an error such as malformed XML would be logged. If you are totally unaware of where the server logs are, please talk to whomever set up your deployment.

[satya_paul]

We have huge traces that is ouput in server ? what is the log statements to look for xml entity expansion disabled ?

[satya_paul]

Hi Isomorphic, I tested in 10.0 I see that the request goes to our code and Smartclient does not disable the xml entity expansion. This is found in our logs :

Request Parameters: [protocolVersion="1.0" | isc_rpc="1" | isc_v="v10.0p_2017-03-06" | isc_tnum="19" | _transaction="<!DOCTYPE xxes[<!ENTITY xxe "xxe"><!ELEMENT xxes (#PCDATA)>
<!ENTITY xxe1 "&xxe;&xxe;&xxe;&xxe;&xxe;&xxe;&xxe;&xxe;&xxe;&xxe;">
<!ENTITY xxe2 "&xxe1;&xxe1;&xxe1;&xxe1;&xxe1;&xxe1;&xxe1;&xxe1;&xxe1;&xxe1;">
<!ENTITY xxe3 "&xxe2;&xxe2;&xxe2;&xxe2;&xxe2;&xxe2;&xxe2;&xxe2;&xxe2;&xxe2;">
<!ENTITY xxe4 "&xxe3;&xxe3;&xxe3;&xxe3;&xxe3;&xxe3;&xxe3;&xxe3;&xxe3;&xxe3;">
<!ENTITY xxe5 "&xxe4;&xxe4;&xxe4;&xxe4;&xxe1;&xxe4;&xxe4;&xxe4;&xxe4;&xxe4;">
<!ENTITY xxe6 "&xxe5;&xxe5;&xxe5;&xxe5;&xxe5;&xxe5;&xxe5;&xxe5;&xxe5;&xxe5;">
<!ENTITY xxe7 "&xxe6;&xxe6;&xxe6;&xxe6;&xxe6;&xxe6;&xxe6;&xxe6;&xxe6;&xxe6;">]><xxes>&xxe7;</xxes><transaction xmlns:xsi="http://www.w3.org/2000/10/XMLSchema-instance" xsi:type="xsd:Object"><transactionNum xsi:type="xsd:long">49</transactionNum><operations xsi:type="xsd:List"><elem xsi:type="xsd:Object"><criteria xsi:type="xsd:Object"></criteria><values xsi:type="xsd:Object"><changePassword>N</changePassword><userName>asdf</userName><password>********</password></values><operationConfig xsi:type="xsd:Object"><dataSource>login_DS</dataSource><repo xsi:nil="true"/><operationType>update</operationType><textMatchStyle>exact</textMatchStyle></operationConfig><componentId>LoginForm</componentId><appID>builtinApplication</appID><operation>login_DS_update</operation><oldValues xsi:type="xsd:Object"><changePassword>N</changePassword></oldValues></elem></operations></transaction>" | isc_xhr="1" | uniqueId="qiejtgaqlprx"]

We see that the issue is not resolved. Also, the request is getting timed out. After this our server goes down.

[satya_paul]

Please help us in resolving this issue.

[Isomorphic]

Of course you see the original, unexpanded XML entity declarations if you access request parameters. We disable entity expansion in the XML processor, we don't modify the request parameters (which you can't do anyway).

You've managed to send the logs using a proprietary Windows compression tool. If it's actually necessary for us to look at the logs, please re-send using any standard format.

However, what you actually need to do first is a valid test - a test of the SmartClient server without any of your application code involved. We do not disable XML entity expansion for all code running on the server (which is impossible), we disable it specifically for our own XML processing, and specifically of requests to the IDACall servlet. If your code or any library you are using parses the request as well and uses an XML parser instance where entity expansion is still allowed, you will still be vulnerable to this exploit and there's nothing the SmartClient framework can do to help. This is why the only possible valid test is to try to use the exploit against the stock SDK server environment.

Also, please do not simply try to assert that you are not parsing the XML as a way of claiming you don't need to do a valid test. You've already tried to file this as a Sev 1 issue bogusly several times, without fulfilling the basic requirements for reporting issues, and this is a serious problem (we will wake people in the middle of the night, as needed, for a Sev 1). Please be careful to file a valid issue this time.

[satya_paul]

Hi Isomorphic, I have recreated the issue using smartclientSDK. I have attached the recreation steps and logs for the same in email. Now, in the logs I see that there is an OOM error thrown from XML parser :
------------------------------------------------------------------------------------------------------------------
Real FileSystem Path: 'C:\smartclientSDK\isomorphic\IDACall'
=== 2017-03-13 06:46:14,620 [ec-8] ERROR IDACall - com.isomorphic.servlet.IDACall top-level exception
java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.apache.xerces.xni.XMLString.toString(Unknown Source)
....
at com.isomorphic.xml.XML.parseXML(XML.java:260)
at com.isomorphic.xml.XML.parseRestrictedXML(XML.java:125)
at com.isomorphic.xml.XML.toDSRecords(XML.java:408)
at com.isomorphic.xml.XML.toDSRecords(XML.java:404)
at com.isomorphic.rpc.RPCManager.parseRequest(RPCManager.java:2341)
at com.isomorphic.rpc.RPCManager.<init>(RPCManager.java:333)
at com.isomorphic.rpc.RPCManager.<init>(RPCManager.java:313)
at com.isomorphic.servlet.IDACall.processRequest(IDACall.java:147)
at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:117)
at com.isomorphic.servlet.IDACall.doPost(IDACall.java:76)
------------------------------------------------------------------------------------------------------------------

Please let us know what else you need from us. Thanks.

[satya_paul]

Hi Isomorphic,

The above reply is based on my testing using SmartClient SDK 10.0 Enterprise edition.

I tried the same use case scenario using Smart Client SDK 10.1 Enterprise edition. Using Burp suite, I sent the XML entity in _transaction param and forwarded the request. The server returned the response within few seconds with a warning message :"The server failed to return a formatted response at all." In the logs, I see the OutOfMemoryError but the server response proceeded further and then it returned in the UI : "The server failed to return a formatted response at all.". Is it possible to back port the changes to smartclient 10.0 enterprise edition ? Please let us know. Thanks.

[Isomorphic]

10.1 and 10.0 have the same code for preventing Billion Laughs, and we can see that it is active in your build: note the call through parseRestrictedXML() - this is where we turn off the entity expansion feature.

Even though we explicitly turn off entity expansion, it's obviously still on, seemingly related to DTD parsing. Since we already verified our workaround for Billion Laughs and had it confirmed by multiple customers, it looks like this problem is to due to a new bug in Xerces - we are still looking into this.

[satya_paul]

Please provide us an ETA for this issue in SmartClient 10.0 Enterprise Edition.

[Isomorphic]

There appears to be an issue in Xerces where even if entity expansion is disabled, and even if DTD loading is disabled, Xerces still expands entities when doing some kind of internal validation. This problem doesn't happen in all circumstances - as stated above, our previous workaround for Billion Laughs did fix the issue for other customers - but it was happening in the example case you provided. We have now worked around this (again) by applying a security manager to cap the number of expansions that are allowed; this has been successfully tested specifically against your example.

The fix will be present in all builds of SmartClient and SmartGWT from 9.0 / 4.0 onwards, as of the builds of today, March 14th. Builds will be available later than usual, because we deferred the normal build process in order to get this workaround into today's builds.

Regards,
Isomorphic Software Support

[satya_paul]

Hi Isomorphic,

Many thanks. We tested the SDK from 10_0 Enterprise edition, its resolving the issue.