ListGrid: paging behaves differently when filtering

[csillag]

I have ListGrid.

When the data is loaded initially, only the first 75 records are fetched. This is exactly what I want.

However, if the user enters something into the filter row above the grid, presses enter and stars filtering, all the matching records are loaded to the ListGrid. (in groups of 75.) What could be the cause of this?

The ListGrid is heavily customized, and does lots of other things, so it can easily related to something I have done to it...
...how can one misconfigure it to achieve this behavior?

UPDATE: the excess loading also happens if configure the criteria for the initial fetch. Filtering with an empty criteria yields only the first 75 records, but filtering with some other criteria loads all matching records. Why?

[bftanase]

- Wrong totalRows value returned after filtering?
- setShowAllRecords set to true after filtering, somehow?

Post the RPC request/response from the RPC tab in Devel. Console.

[csillag]

Here are the request and the responses.
They look completely fine to me; the question is why are the second and the third requests sent at all?

[csillag]

I have done some more experimenting, and I can change between the two behaviors just by changing the criteria I pass.

A shorter text (with more results) triggers the load-only-75 behavior, and a longet text (with fewer results) triggers the load-all-records behavior.

This suggests that this might be intended: the number of total matches is included in the first response, and if the client sees that there are only a few more results, it might decide to start a few more requests to fetch them all.

Is this what is happening?
I can understand that this might work well for a few cases, but generally, I am not fond of this sort of automatic decision making...
... where is this documented?

And, more importantly, where do I switch this heuristics off?

[Isomorphic]

There are no such heuristics. The onlything that's close is listGrid.drawAllMaxCells, but that would cause a single, larger fetch. Most likely, you have code tat is accessing all rows of the dataset.

[csillag]

I don't think I am accessing all rows, but I will start cutting down things, just to be sure.

But how do you explain that passing a shorter text to the same criteria, which yields more results (substring match), stops this behavior?

[csillag]

Is there a way to track down the reason why a given autofetch request is generated? If could take a look at the stack, it would understand what happening in a few seconds...

[Isomorphic]

There is, actually: the log category "fetchTrace". Set it to DEBUG level.

[csillag]

Thank you for the tip.
BTW, this log category is not listed in the menu, but after manually adding it (under logging perefences, more, add), it worked.

Here is what I got for the 3 calls:

19:14:26.287:KPR1:WARN:ResultSet:isc_ResultSet_0 (created by: isc_OID_237):ResultSet server fetch with server criteria: {
    "item2_name":"23"
}
    ResultSet.fetchRemoteData(_1=>Obj,  _2=>0,  _3=>75)
    ResultSet.$39w()
    ResultSet.getRangePaged(_1=>0,  _2=>25,  _3=>undef,  _4=>undef)
    ResultSet.$39u(_1=>0,  _2=>25,  _3=>undef,  _4=>undef)
    ResultSet.getRange(_1=>0,  _2=>25,  _3=>undef,  _4=>undef)
    ListGrid.requestVisibleRows()
    Canvas.filterWithCriteria(_1=>Obj,  _2=>Obj{ID:tds1_auto_fetch},  _3=>Obj)
    Canvas.$wo(_1=>"fetch",  _2=>Obj,  _3=>null,  _4=>Obj)
    Class.invokeSuper(_1=>null,  _2=>"$wo",  _3=>undef,  _4=>undef,  _5=>undef,  _6=>undef,  _7=>undef,  _8=>undef,  _9=>undef,  _10=>undef)
    Class.Super(_1=>"$wo",  _2=>Array[4],  _3=>Obj{length:4})
    ListGrid.$wo(_1=>"fetch",  _2=>Obj,  _3=>null,  _4=>Obj,  _5=>undef)
    Canvas.fetchData(_1=>Obj,  _2=>null,  _3=>Obj)
    RecordEditor.performFilter(_1=>undef)
    RecordEditor.performAction(_1=>undef)
    RecordEditor.cellEditEnd(_1=>"enter",  _2=>undef)
    ListGrid.editorKeyPress(_1=>[TextItem ID:isc_TextItem_0 name:item2_name],  _2=>"Enter",  _3=>13)
    [a]DynamicForm.itemKeyPress(_1=>[TextItem ID:isc_TextItem_0 name:item2_name],  _2=>"Enter",  _3=>13)
    FormItem.$18d(_1=>[TextItem ID:isc_TextItem_0 name:item2_name],  _2=>[DynamicForm ID:isc_DynamicForm_0],  _3=>"Enter",  _4=>13)
    FormItem.handleKeyPress(_1=>Obj,  _2=>Obj)
    EventHandler.bubbleEvent(_1=>[TextItem ID:isc_TextItem_0 name:item2_name],  _2=>"keyPress",  _3=>Obj,  _4=>undef)
    EventHandler.handleKeyPress(_1=>[object KeyboardEvent],  _2=>undef)
    EventHandler.$kd(_1=>[object KeyboardEvent])
    EventHandler.dispatch(_1=>EventHandler.$kd(),  _2=>[object KeyboardEvent])
    anonymous(event=>[object KeyboardEvent])

19:14:28.019:TMR2:WARN:ResultSet:isc_ResultSet_0 (created by: isc_OID_237):ResultSet server fetch with server criteria: {
    "item2_name":"23"
}

   ResultSet.fetchRemoteData(_1=>Obj,  _2=>75,  _3=>174)
    ResultSet.$39w()
    Class.fireCallback(_1=>"$39w",  _2=>undef,  _3=>Array[0],  _4=>[ResultSet ID:isc_ResultSet_0 (created by: isc_OID_237)],  _5=>undef) on [Class ResultSet]
    Class.$cm()
    ** recursed on Class.fireCallback

19:14:28.993:TMR6:WARN:ResultSet:isc_ResultSet_0 (created by: isc_OID_237):ResultSet server fetch with server criteria: {
    "item2_name":"23"
}
    ResultSet.fetchRemoteData(_1=>Obj,  _2=>174,  _3=>249)
    ResultSet.$39w()
    Class.fireCallback(_1=>"$39w",  _2=>undef,  _3=>Array[0],  _4=>[ResultSet ID:isc_ResultSet_0 (created by: isc_OID_237)],  _5=>undef) on [Class ResultSet]
    Class.$cm()
    ** recursed on Class.fireCallback

The first call is completly OK, I have left it in just for the sake of completeness.

I am curious about the 2nd and the 3rd call, but I can not make sense of this data.

Can you?

[Isomorphic]

Use the *Debug versions of the GWT module inherits to see unobfuscated stack traces.

However the Support team already has the secret decoder ring, and, as previously indicated, this suggests that you are causing the extra requests via direct calls to getRange() or get() in custom logic. Perhaps the difference in behavior for different criteria rings a bell in terms of what your code does?

[Isomorphic]

Note that setting the "ResultSet" log category to DEBUG will show you all the getRange() calls (unfortunately, no stacks).

[csillag]

[...] as previously indicated, this suggests that you are causing the extra requests via direct calls to getRange() or get() in custom logic. Perhaps the difference in behavior for different criteria rings a bell in terms of what your code does?

No, I don't think my code is doing it. After some experimenting, I could reproduce it in a standalone example, with just a few lines of code:

final ListGrid grid = new ListGrid();
grid.setShowFilterEditor(true);
grid.setWidth(500);
grid.setHeight(300);
DataSource.load("fetchtest", new Function() {
  @Override public void execute() {
    grid.setDataSource(DataSource.get("fetchtest"));
    grid.hideField("some_string");
    grid.hideField("item2_number");
    final DSRequest req = new DSRequest();
    req.setTextMatchStyle(TextMatchStyle.SUBSTRING);
    grid.fetchData(new Criteria("item2_name", "23"), null, req);
  }}, false
);

Attached full source code and DataSource XML definition.

Please fetch the test data from here: http://www.nolmecolindor.com/isomorp...htest_data.sql

Short explanation:

The page contains a button to show the Developer Console, and a ListGrid.
The ListGrid has DataSource attached, which is loaded using the DataSourceLoaded servlet.
When we have the data, we hide two fields, and start a fetch with a substring search for the remaining field (like when the user types something into the filter row.)

Please check the RPC calls on the RPC tab in the Dev console: instead of one fetch, you will find that three fetches are done. (In other terms, all the matching records
are loaded, not only the first 75.)

Interesting notes:
- The same effect can be achieved by directly entering search queries into the filter row.

- The same effect can be achieved by manually hiding the columns (from the context menu), and then searching.

- If you change the search string "23" to just "2", thus change the number of resulting matches from 175 to 1736, this is not happening.

- Hiding the fields is needed to trigger this; if I don't hide both of them, this is not happening.

Could you please take a look at this? I am very interested in finding out what's causing this, and this is totally blocking my path forward.

[Isomorphic]

What version does this apply to (and browser, etc)?

[Isomorphic]

Never mind, we tried it out.

This is due to drawAllMaxCells as we indicated in post #5 of this thread. setDrawAllMaxCells(0) avoids the fetches.

It's unclear why this result in inefficiently sending 3 requests - something to do with hiding fields immediately before calling fetchData, we'll look into that.

[csillag]

What version does this apply to (and browser, etc)?

SmartGwt Power 2.1 licensed edition
GWT 2.0.3
Debian GNU/Linux
Iceweasel (ie. Firefox) 3.5.8

Sorry I omitted this information in the first place.