Rogue Fetch

[stonebranch2]

Hi Isomorphic,

We have a ListGrid with autoFetchData=false. All fetches are initiated when needed by our application.

On changing the ViewState, however, we are seeing a rogue fetch, that does not include any of the criteria, or any data page size limits as configured, which is having some performance consequences.

Our proper paged and filtered queries look like this:

{
    dataSource:"exec", 
    operationType:"fetch", 
    data:{
        operator:"and", 
        criteria:[
            {
                fieldName:"name", 
                operator:"iContains", 
                value:"xxx"
            }
        ]
    }, 
    startRow:0, 
    endRow:25, 
    sortBy:[
        "type"
    ], 
    textMatchStyle:"exact", 
    willHandleError:true, 
    showPrompt:false, 
    oldValues:{
        operator:"and", 
        criteria:[
            {
                fieldName:"name", 
                operator:"iContains", 
                value:"xxx"
            }
        ]
    }, 
    requestId:"exec$62717", 
    fallbackToEval:false, 
    outputs:"type,..., name,..., statusCode", 
    lastClientEventThreadCode:"MDN2", 
    bypassCache:true, 
    dataProtocol:"getParams"
}

The rogue queries without the criteria and page size constraints, which appear to come about with a ViewState change, look like this:

{
    dataSource:"exec", 
    operationType:"fetch", 
    operationId:"exec_fetch", 
    componentId:"(created directly)", 
    sortBy:[
        "-statusCode"
    ], 
    textMatchStyle:"exact", 
    resultSet:[ResultSet ID:isc_ResultSet_17(dataSource: exec, created directly)], 
    callback:{
        caller:[ResultSet ID:isc_ResultSet_17(dataSource: exec, created directly)], 
        methodName:"fetchRemoteDataReply"
    }, 
    willHandleError:true, 
    showPrompt:true, 
    requestId:"exec$62719", 
    internalClientContext:{
        requestIndex:1
    }, 
    fallbackToEval:false, 
    lastClientEventThreadCode:"TMR8", 
    bypassCache:true, 
    dataProtocol:"getParams"
}

I assume somehow, when changing the ViewState, which changes the sort, this rogue query is being generated by the framework, but we do not feel it should be. We need to prevent it, as it is not the proper fetch. Can you please determine the source of rogue fetch and help us with preventing it?

What is also interesting is that the rogue fetch oddly contains: operationId:"exec_fetch" and componentId:"(created directly)"

Thank you

SmartClient Version: v10.1p_2017-03-11/Pro Deployment (built 2017-03-11)

[Isomorphic]

From a quick check, just applying a different sort via ViewState does not (and could not) have the described result.

The "(created directly)" componentId generally means that a ResultSet was created directly by application code rather automatically by calls to fetchData() or by setting autoFerchData. If your code is doing this, it could explain why a non-paged request is issued and why there is a seemingly unnecessary request.

[stonebranch2]

Yes, the application is using fetchData to set the ResultSet directly in this particular use case.

All subsequent fetches are handled by the application too, using fetchData.

In this particular use case, the application only has a single location where it performs the fetchData, and this breakpoint is not hit during the rogue fetch.

Upon setting a new ViewState, the rogue fetch is sent by the framework ***only*** if the sort changes.

When setting the ViewState, the framework fires an onSortChanged event, as shown below, and the rogue fetch occurs.

ActivityPanel$5.onSortChanged(SortEvent) line: 492
SortEvent.dispatch(SortChangedHandler) line: 120
SortEvent.dispatch(EventHandler) line: 1
SortEvent(GwtEvent<H>).dispatch(Object) line: 1
EventBus.dispatchEvent(Event<H>, H) line: 40
HandlerManager$Bus(SimpleEventBus).doFire(Event<H>, Object) line: 193
HandlerManager$Bus(SimpleEventBus).fireEvent(Event<?>) line: 88
HandlerManager.fireEvent(GwtEvent<?>) line: 127
ActivityPanel$1(Widget).fireEvent(GwtEvent<?>) line: 129
...
JavaScriptHost.invokeNativeVoid(String, Object, Class<?>[], Object[]) line: 107
ActivityPanel$1(BaseWidget).setProperty(String, String) line: not available
ActivityPanel$1(BaseWidget).setAttribute(String, String, boolean) line: 855
ActivityPanel$1(ListGrid).setViewState(String) line: 12717
ActivityPanel$1.setViewState(String) line: 210
...

Note, the ListGrid is already populated with data and the initial sort is *already* set at the time the ViewState is updated.

This sounds very similar to another issue you recommended a workaround for to avoid an unexpected fetch, specifically, when changing the sort manually, we had to apply the following workaround to prevent the framework from sending its own fetch.

lg.setData(new RecordList());
lg.setSort(sortSpecifiers);

I am confident the fetch is not initiated by our application directly.

Is there a parser in the client I can use to parse out and separate the SortSpecifier from the other components of the ViewState? Perhaps that would give me a SortSpecifier[]?

If so, I can pursue a workaround on my end to avoid the problem.

Thanks

[Isomorphic]

Yes, the application is using fetchData to set the ResultSet directly in this particular use case.

Calling fetchData would be a way of creating the ResultSet *indirectly*. Doing it directly would mean using "new ResultSet()" and then ultimately calling setData(). Are you doing this or not?

As far as this being a "rogue" fetch, if the grid has a ResultSet and you change the sort, a fetch would be expected and correct in many cases, because setting sort necessarily invalidates a partial cache. Setting data to empty in order to avoid this is not a workaround, it's just a correct approach.

With this in mind, can you explain why (aside from the lack of paging) you are considering this fetch as unnecessary? After all, above you say you are actually changing the sort, so..

As far as parsing the ViewState, this format is intentionally undocumented and any code that tries to parse would be invalid.

[stonebranch2]

Hi Isomorphic,

For context, here is how the data is populated.

Record[] recordList = response.getData();
int length = (recordList == null) ? 0 : recordList.length;
ResultSet resultSet = new ResultSet(dataSource);
resultSet.setInitialLength(length);
resultSet.setInitialSort(sortSpecifiers);
resultSet.setInitialData(recordList);
resultSet.setCriteria(criteria);
resultSet.setDisableCacheSync(Boolean.TRUE);
resultSet.setUseClientSorting(Boolean.FALSE);
resultSet.setAllRows(recordList);
...
listGrid.setData(resultSet);

The reason we feel the fetch is unnecessary is:

A) We use setAllRows.
"If the complete set of records for a resultSet is available when the resultSet is created, it can be made available to the resultSet via this property at initialization time. This data will then be considered cached meaning sorting and filtering can occur on the client (no need for server fetch)."

2) We need need a way to handle the fetch our way, with the correct criteria, among some other specifics, etc. This fetch happens beyond our control, causing incorrect data to be fetched.

You have confirmed, that even if you have autoFetchData=false, along with allRows, the framework will still indeed make a server-side fetch if you change the sort on the ListGrid while it contains data. So I will have to come up with a way to prevent the fetch happening, which I think I can do now, and apply our own appropriate fetch in its place, as we don't want client-side sorting engaging here either.

Thank you

[Isomorphic]

The framework will indeed make a server fetch with a full cache if the sort is changed *and you have disabled client-side sorting*, because there is no other way to perform the sort. So that fetch is clearly necessary.

The only mystery is that the fetch is not paged. We don't see how that can happen and it's not clear if it's a bug (we have no test case). But at this point it's moot, because you definitely have to rearrange your code to avoid the framework issuing this (again necessary) fetch at all.