Performance Degradation in JSON validation routine in IE 11

[bbaker]

SmartClient Version: v11.0p_2016-10-23/LGPL Development Only (built 2016-10-23)
Browser: IE
We're experiencing a significant performance degradation in IE after our roll from SmartGWT 5.0p.20151210 to 6.0p.20161023. Using the IE 11 Developer Tools Profile utility, we found that SmartGWT is spending more time in the isc_DataSource_validateJSONRecord than before. These metrics were recorded performing the same action, on both versions, with identical record counts (I've attached a picture of the call stack from 6.0p.20161023 for better context):

Version	Function	Count	Inclusive time (ms)	Inclusive time %	Exclusive time (ms)	Exclusive time %	Avg time (ms)	Max time (ms)	Min time (ms)
5.0p.20151210	isc_DataSource_validateJSONRecord	15,000	13,176.73	30.21	1,504.08	3.45	0.88	41
6.0p.20161023	isc_DataSource_validateJSONRecord	15,000	25,896.00	35.14	2,464.19	3.34	1.73	48	1

Do you have any idea how we could improve our performance in this function? Any configurations that might help us out here?
Do you know offhand if any performance improvements have been implemented in this area since the 6.0p.20161023 build? We could try to roll if so.

Let me know if you need more information,

Thanks

[Isomorphic]

There's one relatively trivial change in the right timeframe, which introduced a check for whether the value is an Array for every field value. You're pushing so much data through this system that maybe it actually had this impact. We can do a quick optimization here that you can try out - we'll let you know when it's in.

You can make a larger impact by getting rid of valueXPath declarations, which is what causes the calls to selectObjects which remain a majority of the processing time. Perhaps you have some valueXPath declarations that just recapitulate the field name and are no-ops, if so, remove those. For the rest, consider having the server restructure the response so they are not required, or if that's not feasible, let us know what some of your more common valueXPath declarations are and there might be room to optimize.

Finally, we'll add documentation for an internal flag that allows you to skip parts of JSON validation. We need to traverse the whole returned structure to convert Strings such as "2017-2-5" into true Date instances, but we don't need to do this for String, numeric or boolean types if your server is returning those as the correct JSON type, and skipping this process should provide another chunky speedup.

[bbaker]

Thank you for the fast response. We'll investigate if we can remove or reduce our valueXPath usage, and we look forward to the optimization, and the flags to skip parts of the JSON validation.

[bbaker]

We are setting recordXPath on our DataSource to "Jobs/job" and the valueXPath is being set like "properties/[propertyname]". Unfortunately refactoring this structure would be non-trivial, and high risk since we are nearing release.

[bbaker]

Just an update, we've decided to attempt a refactor of our xpath. We'll back with results.

[Isomorphic]

We've made some optimizations to JSON validation and also added the property DataSource.skipJSONValidation as discussed above. The changes have been applied to SGWT 6.0p/SC 11.0p and newer releases are are in the nightly builds dated 2017-02-27 and beyond.

[bbaker]

Thanks for the JSON validation optimizations. We'll likely roll SmartGWT and take advantage of these in our next release.

By refactoring our xPath definition and JSON response structure, we were able to restore IE performance in par with our previous version.
Chrome received an impressively good boost in performance as well. Our Chrome user's will be happy :)