[pgrever]

Just to give you an update. We have found and fixed the issue with the slower sort times on longs. It was a problem in our code. We're not certain how this got missed for so long, but we had a custom sort normalizer on our long fields and this was adding the overhead.

Originally we operated in some environments that had trouble dealing with big ints. So we were passing this data as a string. However, to get correct sorting behavior we needed to treat the content as a long. So we added a custom sort normalizer that did something like:

return Long.parseLong(value.toString());

When the original data was of type String, the toString wasn't too bad, and we needed the parseLong. However, over time we converted the actual data to a Long. But somehow the code that was adding the custom sort normalizer did not get removed. This then became horribly inefficient because it was now first converting the Long to a String and then parsing it back to a Long again. Yuk! Just removing the sort normalizer fixed the issue.

Thank you for the time you spent trying to help us track this down.

[Isomorphic]

It's hard to theorize at this point. The most likely culprit still seems to be something to do with the large integer record values, coupled with the field configuration, that we are (for some reason) unable to reproduce outside the app. But whether this is a result of the GWT wrapper code, some subtle configuration setting within the field or grid, or some native oddity is pretty much up for speculation, it seems.

Of course if the user experience is acceptable, your workaround of converting the values to dates might be a viable (if mysterious) workaround for the problem, or you might be able to come up with something similar (like changing the field type to string and working with string rather than numeric values, etc). Obviously no one wants mysterious workarounds within application code, but if spending more time on this is not an option, that might at least get you going.

[pgrever]

OK, we'll talk to our management about our ability to pay for consulting.

By the way, we thought that perhaps the distribution of the values or the values themselves were influencing the time of the sort algorithm, so we tried the following...

In transformResponse, we did this:

// loop through each of the records
Long data = record.getAttributeAsLong("joblet_queueOrderTime");
if (data == null)
  record.setAttribute("joblet_queueOrderTime", (Date)null);
else
  record.setAttribute("joblet_queueOrderTime", new Date(data));

After this the sort started taking the same time for the joblet_queueOrderTime field as all of the others. We did not change the field types (we left them set to INTEGER). In the grid, the field displayed as a date.

We also built a sample setting up a grid using the same data values as in our applicatoin, but again the performance was just fine in the sample.

Based upon these tests, we don't think it is the actual data that is causing the issue.

[Isomorphic]

Hi Pat
We've spent some time going over this information and it's not obvious to us what is causing the poor performance for you from the configuration you've shared.
There's nothing obviously surprising about the data or the field definitions we can see that explains why sorting would be slower on this column than other columns.
There may be a native slow-down dealing with large numbers like this, but we've spent some time building a local test case using your field definition information, and data and so on, and the performance we're seeing doing updateCaches with 50 rows in a group of 4500, with similar sorts applied, etc, isn't matching up with your descriptions, so there isn't an obvious indication that this is the case.

At this stage I think we either need you guys to come up with a way to reproduce this outside the app and show us a runnable test case, or we'll have to have one of our engineers attempt to live-debug your running application on an hourly basis.
If you'd like to take the latter approach, please contact us offline at support@isomorphic.com and we can figure out details.

Regards
Isomorphic Software

[pgrever]

Here's the data you asked for...

listgridfield.txt: ListGridField definition for the field in question.

viewstate.txt: View state of the grid.

fielddata.txt: Data for the field in question. I used a smaller record set size for this because otherwise the output is huge. I did do it with the actual sample of 4500 records as well, and the data looks the same, just a large number of values.

datasourcefield.txt DataSourceField definition for the field in question.

perfdata.png: Shows where the time is being spend in IE11. The actual CSV export of the data is too large to upload so I am e-mailing that as a zip to support@isomorphic.com. You can see it took about 1300ms. Usually when we run it it is close to 1500ms. If we change the third level or sort from our problem field to one of our integer fields, the time drops to 700-800ms. We see similar values even if we drop the first two levels of sort and only sort on the Long field vs sorting on some Integer field. Timings are faster in Chrome, but the gap between a Long vs and Integer is about 4x instead of 2x.

[Isomorphic]

A couple of thoughts on how to proceed
- Can you show us the field definition for the field in question? You can get this out of the developer console by doing isc.JSON.encode(<listGridID>.getField(<fieldName>));
If that command doesn't work (which is possible), can you show us the field definition in application code as applied to the ListGrid (via setFields or similar, if there is one), and also the definition supplied for the field in the dataSource. We want to be sure we know as much as we can about the field so we can guess/determine exactly what sorting logic is being used for this field.

- Can you evaluate isc.JSON.encode(<listGridID>.data.getProperty(<fieldName>)) -- and show us the result. It is possible that the value is truly numeric in some cases and not in others, so this may give us enough to work on

- Can you evaluate <listGridID>.getViewState() and show us that [this should let us know as much as possible about the current sort, group state etc of the grid, which could be helpful]

- Other than that - you could use the native IE developer tools to run a performance snapshot and see where the time is actually being spent. If there's a way to export that dump and show it to us we may be able to glean something from it.

If none of these give us a way forward we'll need to come up with a way to reproduce the problem. Either keep bashing away at a standalone test case or possibly have one of our developers attempt some live debugging of your application on an hourly basis.
Hopefully the above suggestions will yield up some answers though!

Regards
Isomorphic Software

[Isomorphic]

Ok thanks. The relevant field value does indeed look like a "real" JS number.

We'll have a think about how best to proceed and get back to you.

[pgrever]

The item in question is called joblet_queueOrderTime.

Here's the output from the selected record:

Evaluator: result of 'pvJobletsListGrid.getSelectedRecord();' (0ms):
{linkedResource: Array[0],
jobletID: "973",
internalJobID: "26c61715adbf49f0b58487f88f72f5f3",
id: null,
propertySet: Array[0],
properties: Obj,
link: Array[4],
expand: null,
joblet_canStart: undef,
joblet_condition: "OK",
joblet_queueOrderTime: 1448483741889,
job_activeWorkCells: "Lamination",
joblet_status: "processing",
joblet_impressionsSideA: 4,
joblet_dfeFolder: undef,
joblet_impressionsSideB: 4,
joblet_jdfJobId: undef,
joblet_priority: 50,
joblet_startedLate: undef,
joblet_printCopies: undef,
joblet_canFinish: undef,
joblet_procSpecificStatus: undef,
job_inputConfiguration: "PrintCutLamPack",
joblet_skip: undef,
joblet_descriptiveName: "Job 1149",
job_customerJobName: "Job 1149",
job_jdfJobPartId: undef,
job_completedTime: undef,
joblet_id: 973,
joblet_estimatedNDocOut: undef,
job_external_Tracking_Id: undef,
job_priority: 50,
joblet_reworkRequested: "false",
joblet_endTime: undef,
job_condition: "OK",
job_hasRework: "false",
joblet_processorId: 9,
joblet_impressions: undef,
nPage: undef,
job_orderQuantity: 10,
joblet_estimatedNDocIn: undef,
job_status: "InProgress",
joblet_aspectName: undef,
joblet_jdfJobPartId: undef,
joblet_trackingId: undef,
joblet_workTimeEstimate: 600,
joblet_adjustedQuantity: 10,
job_dueDay: "5",
joblet_autoStart: undef,
joblet_planLaneName: undef,
job_modifiedTime: Date(11/25/2015),
planDescriptor_name: "PrintCutLamPack",
job_customerOrderId: "52600",
job_submittedTime: Date(11/25/2015),
joblet_startTime: Date(01/06/2016),
job_rootJobletId: 975,
joblet_substrates: undef,
joblet_productQuantity: 10,
joblet_sheetsIn: undef,
job_hasLateTask: undef,
joblet_downStreamWorkTimeEstimate: 600,
joblet_estimatedNPageOut: undef,
job_dueDate: Date(10/16/2015),
joblet_nextDropDeadDate: undef,
job_jdfJobId: undef,
job_rootJobletStatus: "processing",
joblet_sheetsOut: undef,
joblet_startDropDeadDate: undef,
job_batchName: undef,
nDoc: undef,
joblet_autoFinish: undef,
joblet_planStepName: undef,
joblet_dfeTaskType: undef,
job_nextDropDeadDate: undef,
joblet_endDropDeadDate: undef,
planDescriptor_id: "Printcutlampack",
joblet_isRework: "false",
job_customerOrgName: "DAW",
joblet_estimatedNPageIn: undef,
c_MyLong: 1448484736561,
dfeAdjust: 1,
job_name: "Job 1149",
job_id: "26c61715adbf49f0b58487f88f72f5f3",
sides: undef,
job_reworkRequested: "false",
job_deleting: "false",
joblet_jdfDeviceId: undef,
joblet_isReworkIcon: "false",
joblet_latenessIndicator: undef,
joblet_procSpecificStatusIcon: "OK",
__grid: ListGrid{dragTrackerStyle: "pm-lightGridDragTrack",
headerButtonProperties: Obj,
headerBackgroundColor: "#f7f7f7",
headerMenuButtonIcon: "[SKINIMG]Product/Menu/light/menu_button...."[43],
scrollbarConstructor: "PMLightScrollbar",
sortAscendingImage: Obj,
sortDescendingImage: Obj,
styleName: "pm-lightListGrid",
bodyStyleName: "pm-lightGridBody",
baseStyle: "pm-primaryLightCell",
editPendingBaseStyle: "pm-primaryLightCell",
emptyMessageStyle: "pm-lightEmptyMessage",
groupNodeStyle: "pm-lightGroupNode",
groupSummaryStyle: "pm-lightSummaryCell",
headerBarStyle: "pm-lightHeaderBar",
headerBaseStyle: "pm-lightHeaderButton",
loadingDataMessageStyle: "pm-lightLoadingDataMessage",
recordSummaryBaseStyle: "pm-lightRecordSummaryCell",
sortNumeralStyle: "pm-lightSortNumeral",
summaryRowStyle: "pm-lightSummaryStyle",
checkboxFieldFalseImage: "[SKINIMG]Product/Checkbox/light/unchecke..."[45],
checkboxFieldTrueImage: "[SKINIMG]Product/Checkbox/light/checked...."[43],
booleanFalseImage: "[SKINIMG]Product/Checkbox/light/unchecke..."[45],
booleanTrueImage: "[SKINIMG]Product/Checkbox/light/checked...."[43],
booleanBaseStyle: "pm-lightCheckbox",
headerMenuButtonProperties: Obj,
sorterProperties: Obj,
filterEditorProperties: Obj,
alternateRecordStyles: true,
ID: "pvJobletsListGrid",
loadingMessage: "Loading data...",
groupByText: "Group by: ${title}",
freezeFieldText: "Freeze: ${title}",
unfreezeFieldText: "Unfreeze: ${title}",
advancedFieldPickerThreshold: 0,
fieldPickerWindowProperties: Obj,
groupStartOpen: "none",
showAsynchGroupingPrompt: false,
width: 1277,
height: 750,
gridComponents: Array[2],
showFilterEditor: false,
minFieldWidth: 2,
wrapCells: false,
showHeaderMenuButton: false,
showSortArrow: "field",
showSortNumerals: true,
canAutoFitFields: false,
canEdit: false,
canHover: true,
hoverAutoDestroy: true,
showHoverComponents: true,
hiliteIcons: Array[0],
showClippedValuesOnHover: true,
sortByGroupFirst: true,
groupSortDirection: "ascending",
reselectOnUpdateNotifications: "none",
canAcceptDrop: true,
canDrop: true,
canDrag: true,
canDragRecordsOut: true,
canAcceptDroppedRecords: true,
groupByMaxRecords: 100000,
dataPageSize: 100000,
dataFetchMode: "paged",
drawAheadRatio: 1.2999999523162841,
instantScrollTrackRedraw: true,
showAllColumns: true,
showAllRecords: false,
selectionType: "multiple",
autoFetchData: true,
dataProperties: Obj,
canFocus: true,
dragTrackerMode: "icon",
canReorderRecords: true,
titleField: "job_name",
dataSource: [DataSource ID:PMDataSource$ProcessorQueue$Lamination$3],
fields: Array[8],
members: Array[2],
position: "absolute",
className: "pm-lightListGrid",
vertical: true,
children: Array[6],
sortDirection: true,
data: [ResultSet ID:isc_ResultSet_0 (dataSource: PMDataSource$ProcessorQueue$Lamination$3, created by: pvJobletsListGrid)],
selection: [Selection ID:pvJobletsListGrid_selection],
wrapHeaderTitles: false,
currentGroupState: "",
sortSpecifiers: Array[3],
initialCriteria: Obj{_constructor:AdvancedCriteria},
showDropLines: false,
tabIndex: 21,
parentElement: [HLayout ID:isc_HLayout_21],
topElement: [VLayout ID:mainAppVLayout],
originalFields: Array[0],
completeFields: Array[83],
canFreezeFields: true,
defaultFieldState: "[{name:"joblet_latenessIndicator",autoFi..."[4353],
header: [Toolbar ID:isc_Toolbar_0],
sorter: [Button ID:pvJobletsListGrid_sorter],
headers: Array[1],
body: [GridBody ID:pvJobletsListGrid_body],
bodies: Array[1],
dragScrollTarget: [GridBody ID:pvJobletsListGrid_body],
cacheOffsetCoords: true,
zIndex: 203150,
innerWidth: 1262,
memberSizes: Array[2],
isGrouped: false,
},
_selection_11: true}

Here's the JSON version:

Evaluator: result of 'isc.JSON.encode(pvJobletsListGrid.getSelectedRecord());' (283ms):
"{
    "linkedResource":[
    ],
    "jobletID":"973",
    "internalJobID":"26c61715adbf49f0b58487f88f72f5f3",
    "id":null,
    "propertySet":[
    ],
    "properties":{
        "joblet_canStart":null,
        "joblet_condition":"OK",
        "joblet_queueOrderTime":"1448483741889",
        "job_activeWorkCells":"Lamination",
        "joblet_status":"processing",
        "joblet_impressionsSideA":"4",
        "joblet_dfeFolder":null,
        "joblet_impressionsSideB":"4",
        "joblet_jdfJobId":null,
        "joblet_priority":"50",
        "joblet_startedLate":null,
        "joblet_printCopies":null,
        "joblet_canFinish":null,
        "joblet_procSpecificStatus":null,
        "job_inputConfiguration":"PrintCutLamPack",
        "joblet_skip":null,
        "joblet_descriptiveName":"Job 1149",
        "job_customerJobName":"Job 1149",
        "job_jdfJobPartId":null,
        "job_completedTime":null,
        "joblet_id":"973",
        "joblet_estimatedNDocOut":null,
        "job_external_Tracking_Id":null,
        "job_priority":"50",
        "joblet_reworkRequested":"false",
        "joblet_endTime":null,
        "job_condition":"OK",
        "job_hasRework":"false",
        "joblet_processorId":"9",
        "joblet_impressions":null,
        "nPage":null,
        "job_orderQuantity":"10",
        "joblet_estimatedNDocIn":null,
        "job_status":"InProgress",
        "joblet_aspectName":null,
        "joblet_jdfJobPartId":null,
        "joblet_trackingId":null,
        "joblet_workTimeEstimate":"600",
        "joblet_adjustedQuantity":"10.0",
        "job_dueDay":"5",
        "joblet_autoStart":null,
        "joblet_planLaneName":null,
        "job_modifiedTime":"2015-11-25T13:35:43-07:00",
        "planDescriptor_name":"PrintCutLamPack",
        "job_customerOrderId":"52600",
        "job_submittedTime":"2015-11-25T13:35:41-07:00",
        "joblet_startTime":"2016-01-06T14:11:30-07:00",
        "job_rootJobletId":"975",
        "joblet_substrates":null,
        "joblet_productQuantity":"10.0",
        "joblet_sheetsIn":null,
        "job_hasLateTask":null,
        "joblet_downStreamWorkTimeEstimate":"600",
        "joblet_estimatedNPageOut":null,
        "job_dueDate":"2015-10-16T06:02:55-06:00",
        "joblet_nextDropDeadDate":null,
        "job_jdfJobId":null,
        "job_rootJobletStatus":"processing",
        "joblet_sheetsOut":null,
        "joblet_startDropDeadDate":null,
        "job_batchName":null,
        "nDoc":null,
        "joblet_autoFinish":null,
        "joblet_planStepName":null,
        "joblet_dfeTaskType":null,
        "job_nextDropDeadDate":null,
        "joblet_endDropDeadDate":null,
        "planDescriptor_id":"Printcutlampack",
        "joblet_isRework":"false",
        "job_customerOrgName":"DAW",
        "joblet_estimatedNPageIn":null,
        "c_MyLong":"1448484736561",
        "dfeAdjust":"1.0",
        "job_name":"Job 1149",
        "job_id":"26c61715adbf49f0b58487f88f72f5f3",
        "sides":null,
        "job_reworkRequested":"false",
        "job_deleting":"false",
        "joblet_jdfDeviceId":null
    },
    "link":[
        {
            "type":null,
            "desc":null,
            "href":"http://localhost:8080/rest/joblets/973",
            "op":null,
            "rel":"self"
        },
        {
            "type":null,
            "desc":null,
            "href":"http://localhost:8080/rest/processors/Lamination/ext/action/973?action=finish",
            "op":null,
            "rel":"finish"
        },
        {
            "type":null,
            "desc":null,
            "href":"http://localhost:8080/rest/processors/Lamination/ext/action/973?action=fail",
            "op":null,
            "rel":"fail"
        },
        {
            "type":null,
            "desc":null,
            "href":"http://localhost:8080/rest/processors/Lamination/ext/action/973?action=suspend",
            "op":null,
            "rel":"suspend"
        }
    ],
    "expand":null,
    "joblet_canStart":null,
    "joblet_condition":"OK",
    "joblet_queueOrderTime":1448483741889,
    "job_activeWorkCells":"Lamination",
    "joblet_status":"processing",
    "joblet_impressionsSideA":4,
    "joblet_dfeFolder":null,
    "joblet_impressionsSideB":4,
    "joblet_jdfJobId":null,
    "joblet_priority":50,
    "joblet_startedLate":null,
    "joblet_printCopies":null,
    "joblet_canFinish":null,
    "joblet_procSpecificStatus":null,
    "job_inputConfiguration":"PrintCutLamPack",
    "joblet_skip":null,
    "joblet_descriptiveName":"Job 1149",
    "job_customerJobName":"Job 1149",
    "job_jdfJobPartId":null,
    "job_completedTime":null,
    "joblet_id":973,
    "joblet_estimatedNDocOut":null,
    "job_external_Tracking_Id":null,
    "job_priority":50,
    "joblet_reworkRequested":"false",
    "joblet_endTime":null,
    "job_condition":"OK",
    "job_hasRework":"false",
    "joblet_processorId":9,
    "joblet_impressions":null,
    "nPage":null,
    "job_orderQuantity":10,
    "joblet_estimatedNDocIn":null,
    "job_status":"InProgress",
    "joblet_aspectName":null,
    "joblet_jdfJobPartId":null,
    "joblet_trackingId":null,
    "joblet_workTimeEstimate":600,
    "joblet_adjustedQuantity":10,
    "job_dueDay":"5",
    "joblet_autoStart":null,
    "joblet_planLaneName":null,
    "job_modifiedTime":"2015-11-25T20:35:43.000",
    "planDescriptor_name":"PrintCutLamPack",
    "job_customerOrderId":"52600",
    "job_submittedTime":"2015-11-25T20:35:41.000",
    "joblet_startTime":"2016-01-06T21:11:30.000",
    "job_rootJobletId":975,
    "joblet_substrates":null,
    "joblet_productQuantity":10,
    "joblet_sheetsIn":null,
    "job_hasLateTask":null,
    "joblet_downStreamWorkTimeEstimate":600,
    "joblet_estimatedNPageOut":null,
    "job_dueDate":"2015-10-16T12:02:55.000",
    "joblet_nextDropDeadDate":null,
    "job_jdfJobId":null,
    "job_rootJobletStatus":"processing",
    "joblet_sheetsOut":null,
    "joblet_startDropDeadDate":null,
    "job_batchName":null,
    "nDoc":null,
    "joblet_autoFinish":null,
    "joblet_planStepName":null,
    "joblet_dfeTaskType":null,
    "job_nextDropDeadDate":null,
    "joblet_endDropDeadDate":null,
    "planDescriptor_id":"Printcutlampack",
    "joblet_isRework":"false",
    "job_customerOrgName":"DAW",
    "joblet_estimatedNPageIn":null,
    "c_MyLong":1448484736561,
    "dfeAdjust":1,
    "job_name":"Job 1149",
    "job_id":"26c61715adbf49f0b58487f88f72f5f3",
    "sides":null,
    "job_reworkRequested":"false",
    "job_deleting":"false",
    "joblet_jdfDeviceId":null,
    "joblet_isReworkIcon":"false",
    "joblet_latenessIndicator":null,
    "joblet_procSpecificStatusIcon":"OK",
    "__grid":"ListGrid{dragTrackerStyle: \"pm-lightGridDragTrack\",\rheaderButtonProperties: Obj,\rheaderBackgroundColor: \"#f7f7f7\",\rheaderMenuButtonIcon: \"[SKINIMG]Product/Menu/light/menu_button....\"[43],\rscrollbarConstructor: \"PMLightScrollbar\",\rsortAscendingImage: Obj,\rsortDescendingImage: Obj,\rstyleName: \"pm-lightListGrid\",\rbodyStyleName: \"pm-lightGridBody\",\rbaseStyle: \"pm-primaryLightCell\",\reditPendingBaseStyle: \"pm-primaryLightCell\",\remptyMessageStyle: \"pm-lightEmptyMessage\",\rgroupNodeStyle: \"pm-lightGroupNode\",\rgroupSummaryStyle: \"pm-lightSummaryCell\",\rheaderBarStyle: \"pm-lightHeaderBar\",\rheaderBaseStyle: \"pm-lightHeaderButton\",\rloadingDataMessageStyle: \"pm-lightLoadingDataMessage\",\rrecordSummaryBaseStyle: \"pm-lightRecordSummaryCell\",\rsortNumeralStyle: \"pm-lightSortNumeral\",\rsummaryRowStyle: \"pm-lightSummaryStyle\",\rcheckboxFieldFalseImage: \"[SKINIMG]Product/Checkbox/light/unchecke...\"[45],\rcheckboxFieldTrueImage: \"[SKINIMG]Product/Checkbox/light/checked....\"[43],\rbooleanFalseImage: \"[SKINIMG]Product/Checkbox/light/unchecke...\"[45],\rbooleanTrueImage: \"[SKINIMG]Product/Checkbox/light/checked....\"[43],\rbooleanBaseStyle: \"pm-lightCheckbox\",\rheaderMenuButtonProperties: Obj,\rsorterProperties: Obj,\rfilterEditorProperties: Obj,\ralternateRecordStyles: true,\rID: \"pvJobletsListGrid\",\rloadingMessage: \"Loading data...\",\rgroupByText: \"Group by: ${title}\",\rfreezeFieldText: \"Freeze: ${title}\",\runfreezeFieldText: \"Unfreeze: ${title}\",\radvancedFieldPickerThreshold: 0,\rfieldPickerWindowProperties: Obj,\rgroupStartOpen: \"none\",\rshowAsynchGroupingPrompt: false,\rwidth: 1277,\rheight: 750,\rgridComponents: Array[2],\rshowFilterEditor: false,\rminFieldWidth: 2,\rwrapCells: false,\rshowHeaderMenuButton: false,\rshowSortArrow: \"field\",\rshowSortNumerals: true,\rcanAutoFitFields: false,\rcanEdit: false,\rcanHover: true,\rhoverAutoDestroy: true,\rshowHoverComponents: true,\rhiliteIcons: Array[0],\rshowClippedValuesOnHover: true,\rsortByGroupFirst: true,\rgroupSortDirection: \"ascending\",\rreselectOnUpdateNotifications: \"none\",\rcanAcceptDrop: true,\rcanDrop: true,\rcanDrag: true,\rcanDragRecordsOut: true,\rcanAcceptDroppedRecords: true,\rgroupByMaxRecords: 100000,\rdataPageSize: 100000,\rdataFetchMode: \"paged\",\rdrawAheadRatio: 1.2999999523162841,\rinstantScrollTrackRedraw: true,\rshowAllColumns: true,\rshowAllRecords: false,\rselectionType: \"multiple\",\rautoFetchData: true,\rdataProperties: Obj,\rcanFocus: true,\rdragTrackerMode: \"icon\",\rcanReorderRecords: true,\rtitleField: \"job_name\",\rdataSource: [DataSource ID:PMDataSource$ProcessorQueue$Lamination$3],\rfields: Array[8],\rmembers: Array[2],\rposition: \"absolute\",\rclassName: \"pm-lightListGrid\",\rvertical: true,\rchildren: Array[6],\rsortDirection: true,\rdata: [ResultSet ID:isc_ResultSet_0 (dataSource: PMDataSource$ProcessorQueue$Lamination$3, created by: pvJobletsListGrid)],\rselection: [Selection ID:pvJobletsListGrid_selection],\rwrapHeaderTitles: false,\rcurrentGroupState: \"\",\rsortSpecifiers: Array[3],\rinitialCriteria: Obj{_constructor:AdvancedCriteria},\rshowDropLines: false,\rtabIndex: 21,\rparentElement: [HLayout ID:isc_HLayout_21],\rtopElement: [VLayout ID:mainAppVLayout],\roriginalFields: Array[0],\rcompleteFields: Array[83],\rcanFreezeFields: true,\rdefaultFieldState: \"[{name:\"joblet_latenessIndicator\",autoFi...\"[4353],\rheader: [Toolbar ID:isc_Toolbar_0],\rsorter: [Button ID:pvJobletsListGrid_sorter],\rheaders: Array[1],\rbody: [GridBody ID:pvJobletsListGrid_body],\rbodies: Array[1],\rdragScrollTarget: [GridBody ID:pvJobletsListGrid_body],\rcacheOffsetCoords: true,\rzIndex: 203150,\rinnerWidth: 1262,\rmemberSizes: Array[2],\risGrouped: false,\r}",
    "_selection_11":true
}"

[Isomorphic]

We'll keep using this thread for this sort performance issue.

One quick test that is probably worth performing.
Can you select a row in the grid and evaluate <listGridID>.getSelectedRecord(); in the developer console and show us the result?
Actually, also please evaluate isc.JSON.encode(<listGridID>.getSelectedRecord(); and show us both results. They will likely be the same, but it's worth checking.

We're interested in what the value for this long data type field actually is in JavaScript within the record in the running app. From the JSON snippet you've shown us and your earlier descriptions it seemed likely that this is actually a string and that the extra time is being spent converting this to a JS number.
The fact that your logic to explicitly convert to long values in transformRequest has no effect would seem to negate that, which is surprising - but it's possible that we're looking at a case where this GWT logic is not behaving exactly as you might expect - data type conversion from GWT Java to JS can have some unintuitive behaviors.

Beyond that - we understand what you're saying about the difficulties of getting us a test case, and we'll continue to think about that if the simple JS test doesn't get us any information.

Regards
Isomorphic Software

[pgrever]

We can send you the JSON sample, but not client sample code. As I said, the problem has been that in our sample the performance issue does not appear, only in our full blown app. We have been unsuccessful in trying to narrow it down any further so we obviously do not know what the key differences are between a small sample and the actual application that cause the difference in behavior. Also building up a sample of the creation of our DS is also non-trivial. To build the DS we make multiple REST calls to a our server that returns complex structures describing the data and we auto generate the field definitions for both the DS and the ListGrid from this. There are 78 columns right now. Coding up sample field definitions for all of this without the actual server and client code is quite a bit of work. We are not doing any of the JSON processing ourselves. We are configuring the DS with a URL and XPATH information about the record layout and fields and the DS itself is doing the processing. Would it still be helpful to see the raw JSON? Would you like us to move this topic to a new thread as well?

[Isomorphic]

We probably need to take a look at what's happening in the sorting code that's taking so long.
Perhaps you could capture a raw JSON response, plus a runnable minimal test case of client side code (the DS definition, the ListGrid and the logic you're running to pick up and process the JSON) and show it to us (email to support@isomorphic.com) so we can try this out on our end and debug what's special about the sort process on this particular field.

And yes - please start a new thread for the memory leak issue if it seems to be a framework problem rather than making this thread any longer -it's already extremely long! :)

Regards
Isomorphic Software

[pgrever]

Unfortunately changing the server side is not an option. So in our transformResponse method we call response.getData() to loop through each of the records and for each Long item in each record we do the following:

Object fieldData = record.getAttributeAsObject(fieldName);
if (fieldData == null) {
    record.setAttribute(fieldName, (Long)null);
} else {
    Long longValue = Long.parseLong(fieldData.toString());
    record.setAttribute(fieldName, longValue);
}

This makes no difference. As a note, when looking in the debugger, the object in fieldData is already of type Long, so I am not certain this routine is actually doing anything. We tried several variations on this. The only one that had any effect was to change the last line to:

record.setAttribute(fieldName, new Date(longValue));

(We did NOT change the field types - we left them set as INTEGER).

After doing this, the updates then take the same time sorting on this field as they do with any other Integer data. We tried in our sample to create a ResultSet from a list for Record objects. The long fields were placed in the Record object using Record.setAttribute(String, Long) - this seems like it should be equivalent to what our application is doing. When we run this, the updates take the same amount of time whether we sort on a Integer field or a Long field. We still cannot explain what is different with these Long values when used in the grid in our application.

We could start treating this field as a Date for better performance, but that seems strange since Date is just a long under the covers (seems like the performance should be the same either way). Also we have other Long fields that are not actually dates so converting them to one would not be a good options since doing this causes them to be displayed as dates in the grid. We could obviously get around this using custom cell formatting, but at that point we are hacking a lot just to get good performance when sorting on a Long field.

As a side note, we still are struggling with the memory issues and cannot find a cause. Since these are not really related to the performance discussion on this thread we'll move that to a new thread in the forum when we're ready to deal with that.

[Isomorphic]

Ah right - you're using custom dataSources etc of course. It is indeed a server side feature - you'd typically set it in your ds.xml file for the DataSource.

For a non SmartClient back end: basically whatever is serializing the server side data for delivery to the client should deliver the JSON as unquoted number literals rather than strings.
If you can't control that, you could change the field types for the DataSources created in client side code to be text rather than integer, or you could possibly perform a one-time conversion somewhere in your application code to loop through the records (once) and do essentially record[fieldName] = parseInt(record[fieldName]), before handing the list to the DataSource.

Does that give you enough to get something working?

[pgrever]

Looks like the stringInBrowser property is on the server side from what we are seeing and we do not use your classes for our server side. Is there a way to control this from the client side?

[Isomorphic]

Try setting stringInBrowser="false" on the field in your dataSource definition.
In order to do a numeric sort, it is likely the sort logic is taking the string type data and converting it to an integer on the fly of each record during the sort (which would take a while).
By disabling the stringInBrowser feature, the numbers should be served as JavaScript numbers instead of strings, getting rid of this.
Alternatively, you could possibly set the field type to "text" on the DataSource / grid and allow a string-sort instead of a numeric-sort to occur.

Let us know if this doesn't get it fixed for you.

On the memory leak - sounds like an application coding issue, but let us know if you think it turns out to be a framework issue.

Regards
Isomorphic Software