[csillag]

I am not saying that this is a release blocker for the framework; I am only saying that this makes what I want to do very hard (if not impossible.)

Specifically, I need to be able to tell apart at least three kinds of data arrive events:
- ones generated because of the user is scrolling
- ones generated because the user changes the filter
- ones generated because the data itself has changed.

Since the API does not exposes this information, all I can do is to use various roundabout methods to detect what happened - like comparing the current filter to a previous, saved value. But I can not do this, when there are multiple data arrives caused by the same filter change, because in this case, only the first one is recognized as a filter change.

Anyway, for now, I am trying to work around this an other way.

[Isomorphic]

Taking a step back, it looks like you are tying correct behavior of your application to getting exactly one fetch per filter, and this isn't a good idea. From the perspective of general use of the framework, eliminating these extra fetches is a minor optimization for an edge case. An issue like this would not be considered a release blocker.

[csillag]

setDrawAllMaxCells(0) avoids the fetches.

Well, it certainly helps, but still, there are situations where a filtering request might generate more then one fetch requests.
(Although this might be a separate issue.)

Please see the attached (modified) test case. (The DS XML and dataset I have sent previously still apply.)

Short explanation:

I load a DS, bind it to the ListGrid, fetch the data. Then
- Press "execute test step 1" to scroll down into the data, to record #3000. (Or you can do it manually.)
- Press "execute test step 2" to set a criteria. (Or you can do it manually, by entering "23" in the filter box.)

Result:

Two fetch requests are generated:
- One for "startRow":953, "endRow":1000. (Response: totalRows:176, startRow:176, endRow:176)
- One for "startRow":0, "endRow":75. (Response: totalRows:175, startRow:0, endRow:75)

If I would have to guess, I would say that this is because I was well into the data before doing the search, so the grid figures that after filtering, it should be somewhere in the middle of the data, and not at the start, therefore asks for data from ... where? at 1/3 of the original record number? This is a heuristics for the expected record numbers, after filtering?

Anyway, this is still no good; how can I make it do only one fetch for each filtering?

Thank you!

UPDATE: since this only happens when the user is not at the start of the dataset when doing the filtering, I can avoid this when doing the filtering from code simply by going to the start first. However, I can not do this when the user is doing the search manually. One relatively easy solution would be for you to add a hook/callback function/event handler to when a filtering action is going to start. (Just like when editing is going to start, etc.) Then I could prepare for it, for example by going to the start. Or, if I have an other hook/callback function/event handler at the end of the filtering action, too, then this would make the whole auto-detection thing unnecessary.

UPDATE: not need to go to 3000: the effect already appear if I scroll down to row #40 before the search.
REQ1: [9,84] RESP1: [9,84], total is 175
REQ2: [0, 42] RESP2: [0,42]

So, the second request happen even if the first request _could_ get all the 75 rows it wanted - I guess it figures out from the total number of rows that it should fetch record from the start instead.

What I do not understand is why the two requests ask for intersecting regions? (Both contain [9,42] )

And, the most important bit, how do I avoid, or at least detect this effect? (I was hoping that the second request only happens f the first response does not contain enough information (because trying to read past the end of records), but this does not seem to be the case.)

[csillag]

Never mind, we tried it out.

This is due to drawAllMaxCells as we indicated in post #5 of this thread.

(...and you have also ruled it out in the same post, so instead of looking into this, I started inspecting my code, as you have suggested.)

setDrawAllMaxCells(0) avoids the fetches.

Thank you, great!

It's unclear why this result in inefficiently sending 3 requests

My real problem here is not inefficiency, but the fact that my code can not understand (and react to) what is happening.
The first fetch can be recognized to be a result of a filtering, but the successive ones can not be, as far as I know.

But now this is solved.

something to do with hiding fields immediately before calling fetchData

,
The same thing happens if I manually execute searches later on.

we'll look into that.

OK. In these cases (when you say you will look into something behind the scenes) what is the established process? Will you notify me when it is resolved, or should I check the nightlies periodically?

Thank you for your help!

[csillag]

What version does this apply to (and browser, etc)?

SmartGwt Power 2.1 licensed edition
GWT 2.0.3
Debian GNU/Linux
Iceweasel (ie. Firefox) 3.5.8

Sorry I omitted this information in the first place.

[Isomorphic]

Never mind, we tried it out.

This is due to drawAllMaxCells as we indicated in post #5 of this thread. setDrawAllMaxCells(0) avoids the fetches.

It's unclear why this result in inefficiently sending 3 requests - something to do with hiding fields immediately before calling fetchData, we'll look into that.

[Isomorphic]

What version does this apply to (and browser, etc)?

[csillag]

[...] as previously indicated, this suggests that you are causing the extra requests via direct calls to getRange() or get() in custom logic. Perhaps the difference in behavior for different criteria rings a bell in terms of what your code does?

No, I don't think my code is doing it. After some experimenting, I could reproduce it in a standalone example, with just a few lines of code:

final ListGrid grid = new ListGrid();
grid.setShowFilterEditor(true);
grid.setWidth(500);
grid.setHeight(300);
DataSource.load("fetchtest", new Function() {
  @Override public void execute() {
    grid.setDataSource(DataSource.get("fetchtest"));
    grid.hideField("some_string");
    grid.hideField("item2_number");
    final DSRequest req = new DSRequest();
    req.setTextMatchStyle(TextMatchStyle.SUBSTRING);
    grid.fetchData(new Criteria("item2_name", "23"), null, req);
  }}, false
);

Attached full source code and DataSource XML definition.

Please fetch the test data from here: http://www.nolmecolindor.com/isomorp...htest_data.sql

Short explanation:

The page contains a button to show the Developer Console, and a ListGrid.
The ListGrid has DataSource attached, which is loaded using the DataSourceLoaded servlet.
When we have the data, we hide two fields, and start a fetch with a substring search for the remaining field (like when the user types something into the filter row.)

Please check the RPC calls on the RPC tab in the Dev console: instead of one fetch, you will find that three fetches are done. (In other terms, all the matching records
are loaded, not only the first 75.)

Interesting notes:
- The same effect can be achieved by directly entering search queries into the filter row.

- The same effect can be achieved by manually hiding the columns (from the context menu), and then searching.

- If you change the search string "23" to just "2", thus change the number of resulting matches from 175 to 1736, this is not happening.

- Hiding the fields is needed to trigger this; if I don't hide both of them, this is not happening.

Could you please take a look at this? I am very interested in finding out what's causing this, and this is totally blocking my path forward.

[Isomorphic]

Note that setting the "ResultSet" log category to DEBUG will show you all the getRange() calls (unfortunately, no stacks).

[Isomorphic]

Use the *Debug versions of the GWT module inherits to see unobfuscated stack traces.

However the Support team already has the secret decoder ring, and, as previously indicated, this suggests that you are causing the extra requests via direct calls to getRange() or get() in custom logic. Perhaps the difference in behavior for different criteria rings a bell in terms of what your code does?

[csillag]

Thank you for the tip.
BTW, this log category is not listed in the menu, but after manually adding it (under logging perefences, more, add), it worked.

Here is what I got for the 3 calls:

19:14:26.287:KPR1:WARN:ResultSet:isc_ResultSet_0 (created by: isc_OID_237):ResultSet server fetch with server criteria: {
    "item2_name":"23"
}
    ResultSet.fetchRemoteData(_1=>Obj,  _2=>0,  _3=>75)
    ResultSet.$39w()
    ResultSet.getRangePaged(_1=>0,  _2=>25,  _3=>undef,  _4=>undef)
    ResultSet.$39u(_1=>0,  _2=>25,  _3=>undef,  _4=>undef)
    ResultSet.getRange(_1=>0,  _2=>25,  _3=>undef,  _4=>undef)
    ListGrid.requestVisibleRows()
    Canvas.filterWithCriteria(_1=>Obj,  _2=>Obj{ID:tds1_auto_fetch},  _3=>Obj)
    Canvas.$wo(_1=>"fetch",  _2=>Obj,  _3=>null,  _4=>Obj)
    Class.invokeSuper(_1=>null,  _2=>"$wo",  _3=>undef,  _4=>undef,  _5=>undef,  _6=>undef,  _7=>undef,  _8=>undef,  _9=>undef,  _10=>undef)
    Class.Super(_1=>"$wo",  _2=>Array[4],  _3=>Obj{length:4})
    ListGrid.$wo(_1=>"fetch",  _2=>Obj,  _3=>null,  _4=>Obj,  _5=>undef)
    Canvas.fetchData(_1=>Obj,  _2=>null,  _3=>Obj)
    RecordEditor.performFilter(_1=>undef)
    RecordEditor.performAction(_1=>undef)
    RecordEditor.cellEditEnd(_1=>"enter",  _2=>undef)
    ListGrid.editorKeyPress(_1=>[TextItem ID:isc_TextItem_0 name:item2_name],  _2=>"Enter",  _3=>13)
    [a]DynamicForm.itemKeyPress(_1=>[TextItem ID:isc_TextItem_0 name:item2_name],  _2=>"Enter",  _3=>13)
    FormItem.$18d(_1=>[TextItem ID:isc_TextItem_0 name:item2_name],  _2=>[DynamicForm ID:isc_DynamicForm_0],  _3=>"Enter",  _4=>13)
    FormItem.handleKeyPress(_1=>Obj,  _2=>Obj)
    EventHandler.bubbleEvent(_1=>[TextItem ID:isc_TextItem_0 name:item2_name],  _2=>"keyPress",  _3=>Obj,  _4=>undef)
    EventHandler.handleKeyPress(_1=>[object KeyboardEvent],  _2=>undef)
    EventHandler.$kd(_1=>[object KeyboardEvent])
    EventHandler.dispatch(_1=>EventHandler.$kd(),  _2=>[object KeyboardEvent])
    anonymous(event=>[object KeyboardEvent])

19:14:28.019:TMR2:WARN:ResultSet:isc_ResultSet_0 (created by: isc_OID_237):ResultSet server fetch with server criteria: {
    "item2_name":"23"
}

   ResultSet.fetchRemoteData(_1=>Obj,  _2=>75,  _3=>174)
    ResultSet.$39w()
    Class.fireCallback(_1=>"$39w",  _2=>undef,  _3=>Array[0],  _4=>[ResultSet ID:isc_ResultSet_0 (created by: isc_OID_237)],  _5=>undef) on [Class ResultSet]
    Class.$cm()
    ** recursed on Class.fireCallback

19:14:28.993:TMR6:WARN:ResultSet:isc_ResultSet_0 (created by: isc_OID_237):ResultSet server fetch with server criteria: {
    "item2_name":"23"
}
    ResultSet.fetchRemoteData(_1=>Obj,  _2=>174,  _3=>249)
    ResultSet.$39w()
    Class.fireCallback(_1=>"$39w",  _2=>undef,  _3=>Array[0],  _4=>[ResultSet ID:isc_ResultSet_0 (created by: isc_OID_237)],  _5=>undef) on [Class ResultSet]
    Class.$cm()
    ** recursed on Class.fireCallback

The first call is completly OK, I have left it in just for the sake of completeness.

I am curious about the 2nd and the 3rd call, but I can not make sense of this data.

Can you?

[Isomorphic]

There is, actually: the log category "fetchTrace". Set it to DEBUG level.

[csillag]

Is there a way to track down the reason why a given autofetch request is generated? If could take a look at the stack, it would understand what happening in a few seconds...

[csillag]

I don't think I am accessing all rows, but I will start cutting down things, just to be sure.

But how do you explain that passing a shorter text to the same criteria, which yields more results (substring match), stops this behavior?

[Isomorphic]

There are no such heuristics. The onlything that's close is listGrid.drawAllMaxCells, but that would cause a single, larger fetch. Most likely, you have code tat is accessing all rows of the dataset.