Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

    #46
    OK, we're back from vacation and we've been working all week to tweak stuff. At this point the app seems to be working very well in Chrome with >14,000 records in the grid. IE takes about twice as long as Chrome, but still very acceptable. Firefox can take over a minute to load a grid with only 4500 records in it, which is way too slow (not certain yet whether we care enough about FF to want to work on this).

    We have run into two issues.

    In IE if we move around in our UI we keep consuming more memory and after a little while crash the browser. We're working on this and it may just be a simple matter of needing to clean up our DataSources manually.

    The other issue is that it takes a lot longer to perform updates on our grid if it is sorted on one of our columns in particular. We are testing in IE11 with 4500 records in the grid and updating 50 of them after changing one field in each record to a new value (the field changed is not a sort field). We are running in Production mode and are measuring the time using the IE development console profiler. Most of the update time is spent in the sort routine. Normally the updates take 200ms or less. However when we set the sort to this particular field the update takes 1500ms.

    We've tried to repro this in a simple sample with no success. We have not been able to determine what it is about this field that causes the problem. We think it has something to do with the data type. With most of our fields the underlying data is a String, Integer, or Date. In this case it is a Long (if we add other Long fields we see the same problem, but not on any String, Integer, or Date fields).

    The data in the field originates as a Date object based upon when the record was created. The server takes the long value out of the Date object and stores it in our SQL database as a bigint. The client retrieves this data from the server via the DataSource FETCH which makes a REST call and receives JSON back from the server. The JSON for this field looks like:

    Code:
    "joblet_queueOrderTime":"1452200748275"
    The corresponding DataSource and ListGrid fields are of type INTEGER.

    We tried creating a sample with dummy records containing fields like this and record data that contains Integer, Date, and Long values. In the sample we can sort on any of these fields and the times are all very similar and very fast.

    We are not certain what is happening. Maybe the DataSource is treating the information from the server as a String and needing to convert it to a Number each time in order to do the compare? Or maybe even doing String compares on the data? Can you think of anything that might be causing the updates to take longer when sorted on this field vs another? We are at a loss as to what might be causing this or how to fix it.

    Comment


      #47
      Try setting stringInBrowser="false" on the field in your dataSource definition.
      In order to do a numeric sort, it is likely the sort logic is taking the string type data and converting it to an integer on the fly of each record during the sort (which would take a while).
      By disabling the stringInBrowser feature, the numbers should be served as JavaScript numbers instead of strings, getting rid of this.
      Alternatively, you could possibly set the field type to "text" on the DataSource / grid and allow a string-sort instead of a numeric-sort to occur.

      Let us know if this doesn't get it fixed for you.

      On the memory leak - sounds like an application coding issue, but let us know if you think it turns out to be a framework issue.

      Regards
      Isomorphic Software

      Comment


        #48
        Looks like the stringInBrowser property is on the server side from what we are seeing and we do not use your classes for our server side. Is there a way to control this from the client side?

        Comment


          #49
          Ah right - you're using custom dataSources etc of course. It is indeed a server side feature - you'd typically set it in your ds.xml file for the DataSource.

          For a non SmartClient back end: basically whatever is serializing the server side data for delivery to the client should deliver the JSON as unquoted number literals rather than strings.
          If you can't control that, you could change the field types for the DataSources created in client side code to be text rather than integer, or you could possibly perform a one-time conversion somewhere in your application code to loop through the records (once) and do essentially record[fieldName] = parseInt(record[fieldName]), before handing the list to the DataSource.

          Does that give you enough to get something working?

          Comment


            #50
            Unfortunately changing the server side is not an option. So in our transformResponse method we call response.getData() to loop through each of the records and for each Long item in each record we do the following:

            Code:
            Object fieldData = record.getAttributeAsObject(fieldName);
            if (fieldData == null) {
                record.setAttribute(fieldName, (Long)null);
            } else {
                Long longValue = Long.parseLong(fieldData.toString());
                record.setAttribute(fieldName, longValue);
            }
            This makes no difference. As a note, when looking in the debugger, the object in fieldData is already of type Long, so I am not certain this routine is actually doing anything. We tried several variations on this. The only one that had any effect was to change the last line to:

            Code:
            record.setAttribute(fieldName, new Date(longValue));
            (We did NOT change the field types - we left them set as INTEGER).

            After doing this, the updates then take the same time sorting on this field as they do with any other Integer data. We tried in our sample to create a ResultSet from a list for Record objects. The long fields were placed in the Record object using Record.setAttribute(String, Long) - this seems like it should be equivalent to what our application is doing. When we run this, the updates take the same amount of time whether we sort on a Integer field or a Long field. We still cannot explain what is different with these Long values when used in the grid in our application.

            We could start treating this field as a Date for better performance, but that seems strange since Date is just a long under the covers (seems like the performance should be the same either way). Also we have other Long fields that are not actually dates so converting them to one would not be a good options since doing this causes them to be displayed as dates in the grid. We could obviously get around this using custom cell formatting, but at that point we are hacking a lot just to get good performance when sorting on a Long field.

            As a side note, we still are struggling with the memory issues and cannot find a cause. Since these are not really related to the performance discussion on this thread we'll move that to a new thread in the forum when we're ready to deal with that.

            Comment


              #51
              We probably need to take a look at what's happening in the sorting code that's taking so long.
              Perhaps you could capture a raw JSON response, plus a runnable minimal test case of client side code (the DS definition, the ListGrid and the logic you're running to pick up and process the JSON) and show it to us (email to support@isomorphic.com) so we can try this out on our end and debug what's special about the sort process on this particular field.

              And yes - please start a new thread for the memory leak issue if it seems to be a framework problem rather than making this thread any longer -it's already extremely long! :)

              Regards
              Isomorphic Software

              Comment


                #52
                We can send you the JSON sample, but not client sample code. As I said, the problem has been that in our sample the performance issue does not appear, only in our full blown app. We have been unsuccessful in trying to narrow it down any further so we obviously do not know what the key differences are between a small sample and the actual application that cause the difference in behavior. Also building up a sample of the creation of our DS is also non-trivial. To build the DS we make multiple REST calls to a our server that returns complex structures describing the data and we auto generate the field definitions for both the DS and the ListGrid from this. There are 78 columns right now. Coding up sample field definitions for all of this without the actual server and client code is quite a bit of work. We are not doing any of the JSON processing ourselves. We are configuring the DS with a URL and XPATH information about the record layout and fields and the DS itself is doing the processing. Would it still be helpful to see the raw JSON? Would you like us to move this topic to a new thread as well?

                Comment


                  #53
                  We'll keep using this thread for this sort performance issue.

                  One quick test that is probably worth performing.
                  Can you select a row in the grid and evaluate <listGridID>.getSelectedRecord(); in the developer console and show us the result?
                  Actually, also please evaluate isc.JSON.encode(<listGridID>.getSelectedRecord(); and show us both results. They will likely be the same, but it's worth checking.

                  We're interested in what the value for this long data type field actually is in JavaScript within the record in the running app. From the JSON snippet you've shown us and your earlier descriptions it seemed likely that this is actually a string and that the extra time is being spent converting this to a JS number.
                  The fact that your logic to explicitly convert to long values in transformRequest has no effect would seem to negate that, which is surprising - but it's possible that we're looking at a case where this GWT logic is not behaving exactly as you might expect - data type conversion from GWT Java to JS can have some unintuitive behaviors.

                  Beyond that - we understand what you're saying about the difficulties of getting us a test case, and we'll continue to think about that if the simple JS test doesn't get us any information.

                  Regards
                  Isomorphic Software

                  Comment


                    #54
                    The item in question is called joblet_queueOrderTime.

                    Here's the output from the selected record:

                    Code:
                    Evaluator: result of 'pvJobletsListGrid.getSelectedRecord();' (0ms):
                    {linkedResource: Array[0],
                    jobletID: "973",
                    internalJobID: "26c61715adbf49f0b58487f88f72f5f3",
                    id: null,
                    propertySet: Array[0],
                    properties: Obj,
                    link: Array[4],
                    expand: null,
                    joblet_canStart: undef,
                    joblet_condition: "OK",
                    joblet_queueOrderTime: 1448483741889,
                    job_activeWorkCells: "Lamination",
                    joblet_status: "processing",
                    joblet_impressionsSideA: 4,
                    joblet_dfeFolder: undef,
                    joblet_impressionsSideB: 4,
                    joblet_jdfJobId: undef,
                    joblet_priority: 50,
                    joblet_startedLate: undef,
                    joblet_printCopies: undef,
                    joblet_canFinish: undef,
                    joblet_procSpecificStatus: undef,
                    job_inputConfiguration: "PrintCutLamPack",
                    joblet_skip: undef,
                    joblet_descriptiveName: "Job 1149",
                    job_customerJobName: "Job 1149",
                    job_jdfJobPartId: undef,
                    job_completedTime: undef,
                    joblet_id: 973,
                    joblet_estimatedNDocOut: undef,
                    job_external_Tracking_Id: undef,
                    job_priority: 50,
                    joblet_reworkRequested: "false",
                    joblet_endTime: undef,
                    job_condition: "OK",
                    job_hasRework: "false",
                    joblet_processorId: 9,
                    joblet_impressions: undef,
                    nPage: undef,
                    job_orderQuantity: 10,
                    joblet_estimatedNDocIn: undef,
                    job_status: "InProgress",
                    joblet_aspectName: undef,
                    joblet_jdfJobPartId: undef,
                    joblet_trackingId: undef,
                    joblet_workTimeEstimate: 600,
                    joblet_adjustedQuantity: 10,
                    job_dueDay: "5",
                    joblet_autoStart: undef,
                    joblet_planLaneName: undef,
                    job_modifiedTime: Date(11/25/2015),
                    planDescriptor_name: "PrintCutLamPack",
                    job_customerOrderId: "52600",
                    job_submittedTime: Date(11/25/2015),
                    joblet_startTime: Date(01/06/2016),
                    job_rootJobletId: 975,
                    joblet_substrates: undef,
                    joblet_productQuantity: 10,
                    joblet_sheetsIn: undef,
                    job_hasLateTask: undef,
                    joblet_downStreamWorkTimeEstimate: 600,
                    joblet_estimatedNPageOut: undef,
                    job_dueDate: Date(10/16/2015),
                    joblet_nextDropDeadDate: undef,
                    job_jdfJobId: undef,
                    job_rootJobletStatus: "processing",
                    joblet_sheetsOut: undef,
                    joblet_startDropDeadDate: undef,
                    job_batchName: undef,
                    nDoc: undef,
                    joblet_autoFinish: undef,
                    joblet_planStepName: undef,
                    joblet_dfeTaskType: undef,
                    job_nextDropDeadDate: undef,
                    joblet_endDropDeadDate: undef,
                    planDescriptor_id: "Printcutlampack",
                    joblet_isRework: "false",
                    job_customerOrgName: "DAW",
                    joblet_estimatedNPageIn: undef,
                    c_MyLong: 1448484736561,
                    dfeAdjust: 1,
                    job_name: "Job 1149",
                    job_id: "26c61715adbf49f0b58487f88f72f5f3",
                    sides: undef,
                    job_reworkRequested: "false",
                    job_deleting: "false",
                    joblet_jdfDeviceId: undef,
                    joblet_isReworkIcon: "false",
                    joblet_latenessIndicator: undef,
                    joblet_procSpecificStatusIcon: "OK",
                    __grid: ListGrid{dragTrackerStyle: "pm-lightGridDragTrack",
                    headerButtonProperties: Obj,
                    headerBackgroundColor: "#f7f7f7",
                    headerMenuButtonIcon: "[SKINIMG]Product/Menu/light/menu_button...."[43],
                    scrollbarConstructor: "PMLightScrollbar",
                    sortAscendingImage: Obj,
                    sortDescendingImage: Obj,
                    styleName: "pm-lightListGrid",
                    bodyStyleName: "pm-lightGridBody",
                    baseStyle: "pm-primaryLightCell",
                    editPendingBaseStyle: "pm-primaryLightCell",
                    emptyMessageStyle: "pm-lightEmptyMessage",
                    groupNodeStyle: "pm-lightGroupNode",
                    groupSummaryStyle: "pm-lightSummaryCell",
                    headerBarStyle: "pm-lightHeaderBar",
                    headerBaseStyle: "pm-lightHeaderButton",
                    loadingDataMessageStyle: "pm-lightLoadingDataMessage",
                    recordSummaryBaseStyle: "pm-lightRecordSummaryCell",
                    sortNumeralStyle: "pm-lightSortNumeral",
                    summaryRowStyle: "pm-lightSummaryStyle",
                    checkboxFieldFalseImage: "[SKINIMG]Product/Checkbox/light/unchecke..."[45],
                    checkboxFieldTrueImage: "[SKINIMG]Product/Checkbox/light/checked...."[43],
                    booleanFalseImage: "[SKINIMG]Product/Checkbox/light/unchecke..."[45],
                    booleanTrueImage: "[SKINIMG]Product/Checkbox/light/checked...."[43],
                    booleanBaseStyle: "pm-lightCheckbox",
                    headerMenuButtonProperties: Obj,
                    sorterProperties: Obj,
                    filterEditorProperties: Obj,
                    alternateRecordStyles: true,
                    ID: "pvJobletsListGrid",
                    loadingMessage: "Loading data...",
                    groupByText: "Group by: ${title}",
                    freezeFieldText: "Freeze: ${title}",
                    unfreezeFieldText: "Unfreeze: ${title}",
                    advancedFieldPickerThreshold: 0,
                    fieldPickerWindowProperties: Obj,
                    groupStartOpen: "none",
                    showAsynchGroupingPrompt: false,
                    width: 1277,
                    height: 750,
                    gridComponents: Array[2],
                    showFilterEditor: false,
                    minFieldWidth: 2,
                    wrapCells: false,
                    showHeaderMenuButton: false,
                    showSortArrow: "field",
                    showSortNumerals: true,
                    canAutoFitFields: false,
                    canEdit: false,
                    canHover: true,
                    hoverAutoDestroy: true,
                    showHoverComponents: true,
                    hiliteIcons: Array[0],
                    showClippedValuesOnHover: true,
                    sortByGroupFirst: true,
                    groupSortDirection: "ascending",
                    reselectOnUpdateNotifications: "none",
                    canAcceptDrop: true,
                    canDrop: true,
                    canDrag: true,
                    canDragRecordsOut: true,
                    canAcceptDroppedRecords: true,
                    groupByMaxRecords: 100000,
                    dataPageSize: 100000,
                    dataFetchMode: "paged",
                    drawAheadRatio: 1.2999999523162841,
                    instantScrollTrackRedraw: true,
                    showAllColumns: true,
                    showAllRecords: false,
                    selectionType: "multiple",
                    autoFetchData: true,
                    dataProperties: Obj,
                    canFocus: true,
                    dragTrackerMode: "icon",
                    canReorderRecords: true,
                    titleField: "job_name",
                    dataSource: [DataSource ID:PMDataSource$ProcessorQueue$Lamination$3],
                    fields: Array[8],
                    members: Array[2],
                    position: "absolute",
                    className: "pm-lightListGrid",
                    vertical: true,
                    children: Array[6],
                    sortDirection: true,
                    data: [ResultSet ID:isc_ResultSet_0 (dataSource: PMDataSource$ProcessorQueue$Lamination$3, created by: pvJobletsListGrid)],
                    selection: [Selection ID:pvJobletsListGrid_selection],
                    wrapHeaderTitles: false,
                    currentGroupState: "",
                    sortSpecifiers: Array[3],
                    initialCriteria: Obj{_constructor:AdvancedCriteria},
                    showDropLines: false,
                    tabIndex: 21,
                    parentElement: [HLayout ID:isc_HLayout_21],
                    topElement: [VLayout ID:mainAppVLayout],
                    originalFields: Array[0],
                    completeFields: Array[83],
                    canFreezeFields: true,
                    defaultFieldState: "[{name:"joblet_latenessIndicator",autoFi..."[4353],
                    header: [Toolbar ID:isc_Toolbar_0],
                    sorter: [Button ID:pvJobletsListGrid_sorter],
                    headers: Array[1],
                    body: [GridBody ID:pvJobletsListGrid_body],
                    bodies: Array[1],
                    dragScrollTarget: [GridBody ID:pvJobletsListGrid_body],
                    cacheOffsetCoords: true,
                    zIndex: 203150,
                    innerWidth: 1262,
                    memberSizes: Array[2],
                    isGrouped: false,
                    },
                    _selection_11: true}
                    Here's the JSON version:

                    Code:
                    Evaluator: result of 'isc.JSON.encode(pvJobletsListGrid.getSelectedRecord());' (283ms):
                    "{
                        "linkedResource":[
                        ],
                        "jobletID":"973",
                        "internalJobID":"26c61715adbf49f0b58487f88f72f5f3",
                        "id":null,
                        "propertySet":[
                        ],
                        "properties":{
                            "joblet_canStart":null,
                            "joblet_condition":"OK",
                            "joblet_queueOrderTime":"1448483741889",
                            "job_activeWorkCells":"Lamination",
                            "joblet_status":"processing",
                            "joblet_impressionsSideA":"4",
                            "joblet_dfeFolder":null,
                            "joblet_impressionsSideB":"4",
                            "joblet_jdfJobId":null,
                            "joblet_priority":"50",
                            "joblet_startedLate":null,
                            "joblet_printCopies":null,
                            "joblet_canFinish":null,
                            "joblet_procSpecificStatus":null,
                            "job_inputConfiguration":"PrintCutLamPack",
                            "joblet_skip":null,
                            "joblet_descriptiveName":"Job 1149",
                            "job_customerJobName":"Job 1149",
                            "job_jdfJobPartId":null,
                            "job_completedTime":null,
                            "joblet_id":"973",
                            "joblet_estimatedNDocOut":null,
                            "job_external_Tracking_Id":null,
                            "job_priority":"50",
                            "joblet_reworkRequested":"false",
                            "joblet_endTime":null,
                            "job_condition":"OK",
                            "job_hasRework":"false",
                            "joblet_processorId":"9",
                            "joblet_impressions":null,
                            "nPage":null,
                            "job_orderQuantity":"10",
                            "joblet_estimatedNDocIn":null,
                            "job_status":"InProgress",
                            "joblet_aspectName":null,
                            "joblet_jdfJobPartId":null,
                            "joblet_trackingId":null,
                            "joblet_workTimeEstimate":"600",
                            "joblet_adjustedQuantity":"10.0",
                            "job_dueDay":"5",
                            "joblet_autoStart":null,
                            "joblet_planLaneName":null,
                            "job_modifiedTime":"2015-11-25T13:35:43-07:00",
                            "planDescriptor_name":"PrintCutLamPack",
                            "job_customerOrderId":"52600",
                            "job_submittedTime":"2015-11-25T13:35:41-07:00",
                            "joblet_startTime":"2016-01-06T14:11:30-07:00",
                            "job_rootJobletId":"975",
                            "joblet_substrates":null,
                            "joblet_productQuantity":"10.0",
                            "joblet_sheetsIn":null,
                            "job_hasLateTask":null,
                            "joblet_downStreamWorkTimeEstimate":"600",
                            "joblet_estimatedNPageOut":null,
                            "job_dueDate":"2015-10-16T06:02:55-06:00",
                            "joblet_nextDropDeadDate":null,
                            "job_jdfJobId":null,
                            "job_rootJobletStatus":"processing",
                            "joblet_sheetsOut":null,
                            "joblet_startDropDeadDate":null,
                            "job_batchName":null,
                            "nDoc":null,
                            "joblet_autoFinish":null,
                            "joblet_planStepName":null,
                            "joblet_dfeTaskType":null,
                            "job_nextDropDeadDate":null,
                            "joblet_endDropDeadDate":null,
                            "planDescriptor_id":"Printcutlampack",
                            "joblet_isRework":"false",
                            "job_customerOrgName":"DAW",
                            "joblet_estimatedNPageIn":null,
                            "c_MyLong":"1448484736561",
                            "dfeAdjust":"1.0",
                            "job_name":"Job 1149",
                            "job_id":"26c61715adbf49f0b58487f88f72f5f3",
                            "sides":null,
                            "job_reworkRequested":"false",
                            "job_deleting":"false",
                            "joblet_jdfDeviceId":null
                        },
                        "link":[
                            {
                                "type":null,
                                "desc":null,
                                "href":"http://localhost:8080/rest/joblets/973",
                                "op":null,
                                "rel":"self"
                            },
                            {
                                "type":null,
                                "desc":null,
                                "href":"http://localhost:8080/rest/processors/Lamination/ext/action/973?action=finish",
                                "op":null,
                                "rel":"finish"
                            },
                            {
                                "type":null,
                                "desc":null,
                                "href":"http://localhost:8080/rest/processors/Lamination/ext/action/973?action=fail",
                                "op":null,
                                "rel":"fail"
                            },
                            {
                                "type":null,
                                "desc":null,
                                "href":"http://localhost:8080/rest/processors/Lamination/ext/action/973?action=suspend",
                                "op":null,
                                "rel":"suspend"
                            }
                        ],
                        "expand":null,
                        "joblet_canStart":null,
                        "joblet_condition":"OK",
                        "joblet_queueOrderTime":1448483741889,
                        "job_activeWorkCells":"Lamination",
                        "joblet_status":"processing",
                        "joblet_impressionsSideA":4,
                        "joblet_dfeFolder":null,
                        "joblet_impressionsSideB":4,
                        "joblet_jdfJobId":null,
                        "joblet_priority":50,
                        "joblet_startedLate":null,
                        "joblet_printCopies":null,
                        "joblet_canFinish":null,
                        "joblet_procSpecificStatus":null,
                        "job_inputConfiguration":"PrintCutLamPack",
                        "joblet_skip":null,
                        "joblet_descriptiveName":"Job 1149",
                        "job_customerJobName":"Job 1149",
                        "job_jdfJobPartId":null,
                        "job_completedTime":null,
                        "joblet_id":973,
                        "joblet_estimatedNDocOut":null,
                        "job_external_Tracking_Id":null,
                        "job_priority":50,
                        "joblet_reworkRequested":"false",
                        "joblet_endTime":null,
                        "job_condition":"OK",
                        "job_hasRework":"false",
                        "joblet_processorId":9,
                        "joblet_impressions":null,
                        "nPage":null,
                        "job_orderQuantity":10,
                        "joblet_estimatedNDocIn":null,
                        "job_status":"InProgress",
                        "joblet_aspectName":null,
                        "joblet_jdfJobPartId":null,
                        "joblet_trackingId":null,
                        "joblet_workTimeEstimate":600,
                        "joblet_adjustedQuantity":10,
                        "job_dueDay":"5",
                        "joblet_autoStart":null,
                        "joblet_planLaneName":null,
                        "job_modifiedTime":"2015-11-25T20:35:43.000",
                        "planDescriptor_name":"PrintCutLamPack",
                        "job_customerOrderId":"52600",
                        "job_submittedTime":"2015-11-25T20:35:41.000",
                        "joblet_startTime":"2016-01-06T21:11:30.000",
                        "job_rootJobletId":975,
                        "joblet_substrates":null,
                        "joblet_productQuantity":10,
                        "joblet_sheetsIn":null,
                        "job_hasLateTask":null,
                        "joblet_downStreamWorkTimeEstimate":600,
                        "joblet_estimatedNPageOut":null,
                        "job_dueDate":"2015-10-16T12:02:55.000",
                        "joblet_nextDropDeadDate":null,
                        "job_jdfJobId":null,
                        "job_rootJobletStatus":"processing",
                        "joblet_sheetsOut":null,
                        "joblet_startDropDeadDate":null,
                        "job_batchName":null,
                        "nDoc":null,
                        "joblet_autoFinish":null,
                        "joblet_planStepName":null,
                        "joblet_dfeTaskType":null,
                        "job_nextDropDeadDate":null,
                        "joblet_endDropDeadDate":null,
                        "planDescriptor_id":"Printcutlampack",
                        "joblet_isRework":"false",
                        "job_customerOrgName":"DAW",
                        "joblet_estimatedNPageIn":null,
                        "c_MyLong":1448484736561,
                        "dfeAdjust":1,
                        "job_name":"Job 1149",
                        "job_id":"26c61715adbf49f0b58487f88f72f5f3",
                        "sides":null,
                        "job_reworkRequested":"false",
                        "job_deleting":"false",
                        "joblet_jdfDeviceId":null,
                        "joblet_isReworkIcon":"false",
                        "joblet_latenessIndicator":null,
                        "joblet_procSpecificStatusIcon":"OK",
                        "__grid":"ListGrid{dragTrackerStyle: \"pm-lightGridDragTrack\",\rheaderButtonProperties: Obj,\rheaderBackgroundColor: \"#f7f7f7\",\rheaderMenuButtonIcon: \"[SKINIMG]Product/Menu/light/menu_button....\"[43],\rscrollbarConstructor: \"PMLightScrollbar\",\rsortAscendingImage: Obj,\rsortDescendingImage: Obj,\rstyleName: \"pm-lightListGrid\",\rbodyStyleName: \"pm-lightGridBody\",\rbaseStyle: \"pm-primaryLightCell\",\reditPendingBaseStyle: \"pm-primaryLightCell\",\remptyMessageStyle: \"pm-lightEmptyMessage\",\rgroupNodeStyle: \"pm-lightGroupNode\",\rgroupSummaryStyle: \"pm-lightSummaryCell\",\rheaderBarStyle: \"pm-lightHeaderBar\",\rheaderBaseStyle: \"pm-lightHeaderButton\",\rloadingDataMessageStyle: \"pm-lightLoadingDataMessage\",\rrecordSummaryBaseStyle: \"pm-lightRecordSummaryCell\",\rsortNumeralStyle: \"pm-lightSortNumeral\",\rsummaryRowStyle: \"pm-lightSummaryStyle\",\rcheckboxFieldFalseImage: \"[SKINIMG]Product/Checkbox/light/unchecke...\"[45],\rcheckboxFieldTrueImage: \"[SKINIMG]Product/Checkbox/light/checked....\"[43],\rbooleanFalseImage: \"[SKINIMG]Product/Checkbox/light/unchecke...\"[45],\rbooleanTrueImage: \"[SKINIMG]Product/Checkbox/light/checked....\"[43],\rbooleanBaseStyle: \"pm-lightCheckbox\",\rheaderMenuButtonProperties: Obj,\rsorterProperties: Obj,\rfilterEditorProperties: Obj,\ralternateRecordStyles: true,\rID: \"pvJobletsListGrid\",\rloadingMessage: \"Loading data...\",\rgroupByText: \"Group by: ${title}\",\rfreezeFieldText: \"Freeze: ${title}\",\runfreezeFieldText: \"Unfreeze: ${title}\",\radvancedFieldPickerThreshold: 0,\rfieldPickerWindowProperties: Obj,\rgroupStartOpen: \"none\",\rshowAsynchGroupingPrompt: false,\rwidth: 1277,\rheight: 750,\rgridComponents: Array[2],\rshowFilterEditor: false,\rminFieldWidth: 2,\rwrapCells: false,\rshowHeaderMenuButton: false,\rshowSortArrow: \"field\",\rshowSortNumerals: true,\rcanAutoFitFields: false,\rcanEdit: false,\rcanHover: true,\rhoverAutoDestroy: true,\rshowHoverComponents: true,\rhiliteIcons: Array[0],\rshowClippedValuesOnHover: true,\rsortByGroupFirst: true,\rgroupSortDirection: \"ascending\",\rreselectOnUpdateNotifications: \"none\",\rcanAcceptDrop: true,\rcanDrop: true,\rcanDrag: true,\rcanDragRecordsOut: true,\rcanAcceptDroppedRecords: true,\rgroupByMaxRecords: 100000,\rdataPageSize: 100000,\rdataFetchMode: \"paged\",\rdrawAheadRatio: 1.2999999523162841,\rinstantScrollTrackRedraw: true,\rshowAllColumns: true,\rshowAllRecords: false,\rselectionType: \"multiple\",\rautoFetchData: true,\rdataProperties: Obj,\rcanFocus: true,\rdragTrackerMode: \"icon\",\rcanReorderRecords: true,\rtitleField: \"job_name\",\rdataSource: [DataSource ID:PMDataSource$ProcessorQueue$Lamination$3],\rfields: Array[8],\rmembers: Array[2],\rposition: \"absolute\",\rclassName: \"pm-lightListGrid\",\rvertical: true,\rchildren: Array[6],\rsortDirection: true,\rdata: [ResultSet ID:isc_ResultSet_0 (dataSource: PMDataSource$ProcessorQueue$Lamination$3, created by: pvJobletsListGrid)],\rselection: [Selection ID:pvJobletsListGrid_selection],\rwrapHeaderTitles: false,\rcurrentGroupState: \"\",\rsortSpecifiers: Array[3],\rinitialCriteria: Obj{_constructor:AdvancedCriteria},\rshowDropLines: false,\rtabIndex: 21,\rparentElement: [HLayout ID:isc_HLayout_21],\rtopElement: [VLayout ID:mainAppVLayout],\roriginalFields: Array[0],\rcompleteFields: Array[83],\rcanFreezeFields: true,\rdefaultFieldState: \"[{name:\"joblet_latenessIndicator\",autoFi...\"[4353],\rheader: [Toolbar ID:isc_Toolbar_0],\rsorter: [Button ID:pvJobletsListGrid_sorter],\rheaders: Array[1],\rbody: [GridBody ID:pvJobletsListGrid_body],\rbodies: Array[1],\rdragScrollTarget: [GridBody ID:pvJobletsListGrid_body],\rcacheOffsetCoords: true,\rzIndex: 203150,\rinnerWidth: 1262,\rmemberSizes: Array[2],\risGrouped: false,\r}",
                        "_selection_11":true
                    }"

                    Comment


                      #55
                      Ok thanks. The relevant field value does indeed look like a "real" JS number.

                      We'll have a think about how best to proceed and get back to you.

                      Comment


                        #56
                        A couple of thoughts on how to proceed
                        - Can you show us the field definition for the field in question? You can get this out of the developer console by doing isc.JSON.encode(<listGridID>.getField(<fieldName>));
                        If that command doesn't work (which is possible), can you show us the field definition in application code as applied to the ListGrid (via setFields or similar, if there is one), and also the definition supplied for the field in the dataSource. We want to be sure we know as much as we can about the field so we can guess/determine exactly what sorting logic is being used for this field.

                        - Can you evaluate isc.JSON.encode(<listGridID>.data.getProperty(<fieldName>)) -- and show us the result. It is possible that the value is truly numeric in some cases and not in others, so this may give us enough to work on

                        - Can you evaluate <listGridID>.getViewState() and show us that [this should let us know as much as possible about the current sort, group state etc of the grid, which could be helpful]

                        - Other than that - you could use the native IE developer tools to run a performance snapshot and see where the time is actually being spent. If there's a way to export that dump and show it to us we may be able to glean something from it.

                        If none of these give us a way forward we'll need to come up with a way to reproduce the problem. Either keep bashing away at a standalone test case or possibly have one of our developers attempt some live debugging of your application on an hourly basis.
                        Hopefully the above suggestions will yield up some answers though!

                        Regards
                        Isomorphic Software

                        Comment


                          #57
                          Here's the data you asked for...

                          listgridfield.txt: ListGridField definition for the field in question.

                          viewstate.txt: View state of the grid.

                          fielddata.txt: Data for the field in question. I used a smaller record set size for this because otherwise the output is huge. I did do it with the actual sample of 4500 records as well, and the data looks the same, just a large number of values.

                          datasourcefield.txt DataSourceField definition for the field in question.

                          perfdata.png: Shows where the time is being spend in IE11. The actual CSV export of the data is too large to upload so I am e-mailing that as a zip to support@isomorphic.com. You can see it took about 1300ms. Usually when we run it it is close to 1500ms. If we change the third level or sort from our problem field to one of our integer fields, the time drops to 700-800ms. We see similar values even if we drop the first two levels of sort and only sort on the Long field vs sorting on some Integer field. Timings are faster in Chrome, but the gap between a Long vs and Integer is about 4x instead of 2x.
                          Attached Files
                          Last edited by pgrever; 12 Jan 2016, 17:20.

                          Comment


                            #58
                            Hi Pat
                            We've spent some time going over this information and it's not obvious to us what is causing the poor performance for you from the configuration you've shared.
                            There's nothing obviously surprising about the data or the field definitions we can see that explains why sorting would be slower on this column than other columns.
                            There may be a native slow-down dealing with large numbers like this, but we've spent some time building a local test case using your field definition information, and data and so on, and the performance we're seeing doing updateCaches with 50 rows in a group of 4500, with similar sorts applied, etc, isn't matching up with your descriptions, so there isn't an obvious indication that this is the case.

                            At this stage I think we either need you guys to come up with a way to reproduce this outside the app and show us a runnable test case, or we'll have to have one of our engineers attempt to live-debug your running application on an hourly basis.
                            If you'd like to take the latter approach, please contact us offline at support@isomorphic.com and we can figure out details.

                            Regards
                            Isomorphic Software


                            Comment


                              #59
                              OK, we'll talk to our management about our ability to pay for consulting.

                              By the way, we thought that perhaps the distribution of the values or the values themselves were influencing the time of the sort algorithm, so we tried the following...

                              In transformResponse, we did this:

                              Code:
                              // loop through each of the records
                              Long data = record.getAttributeAsLong("joblet_queueOrderTime");
                              if (data == null)
                                record.setAttribute("joblet_queueOrderTime", (Date)null);
                              else
                                record.setAttribute("joblet_queueOrderTime", new Date(data));
                              After this the sort started taking the same time for the joblet_queueOrderTime field as all of the others. We did not change the field types (we left them set to INTEGER). In the grid, the field displayed as a date.

                              We also built a sample setting up a grid using the same data values as in our applicatoin, but again the performance was just fine in the sample.

                              Based upon these tests, we don't think it is the actual data that is causing the issue.

                              Comment


                                #60
                                It's hard to theorize at this point. The most likely culprit still seems to be something to do with the large integer record values, coupled with the field configuration, that we are (for some reason) unable to reproduce outside the app. But whether this is a result of the GWT wrapper code, some subtle configuration setting within the field or grid, or some native oddity is pretty much up for speculation, it seems.

                                Of course if the user experience is acceptable, your workaround of converting the values to dates might be a viable (if mysterious) workaround for the problem, or you might be able to come up with something similar (like changing the field type to string and working with string rather than numeric values, etc). Obviously no one wants mysterious workarounds within application code, but if spending more time on this is not an option, that might at least get you going.

                                Comment

                                Working...
                                X