5.1d Grouped ListGrid double fetch when no. of records > setGroupByMaxRecords

[Blama]

Hi Isomorphic,

please see this BuiltInDS-based sample using current SNAPSHOT_v10.1d_2015-10-13/PowerEdition Deployment and the screenshot.
As you can see, two fetch requests are issued (0-200 with grouped field, 0-75 without grouped field).

BuiltInDS.java:

package com.smartgwt.sample.client;

import com.google.gwt.core.client.EntryPoint;
import com.smartgwt.client.core.KeyIdentifier;
import com.smartgwt.client.data.Criterion;
import com.smartgwt.client.data.DataSource;
import com.smartgwt.client.data.SortSpecifier;
import com.smartgwt.client.types.GroupStartOpen;
import com.smartgwt.client.types.OperatorId;
import com.smartgwt.client.types.SortDirection;
import com.smartgwt.client.util.Page;
import com.smartgwt.client.util.PageKeyHandler;
import com.smartgwt.client.util.SC;
import com.smartgwt.client.widgets.IButton;
import com.smartgwt.client.widgets.Window;
import com.smartgwt.client.widgets.events.ClickEvent;
import com.smartgwt.client.widgets.events.ClickHandler;
import com.smartgwt.client.widgets.grid.ListGrid;
import com.smartgwt.client.widgets.grid.ListGridField;

public class BuiltInDS implements EntryPoint {
    private IButton recreateBtn;
    private DataSource supplyItemDS = DataSource.get("supplyItem");

    public void onModuleLoad() {
        KeyIdentifier debugKey = new KeyIdentifier();
        debugKey.setCtrlKey(true);
        debugKey.setKeyName("D");

        Page.registerKey(debugKey, new PageKeyHandler() {
            public void execute(String keyName) {
                SC.showConsole();
            }
        });

        recreateBtn = new IButton("Recreate");
        recreateBtn.addClickHandler(new ClickHandler() {
            @Override
            public void onClick(ClickEvent event) {
                recreate();
            }
        });
        recreate();
        recreateBtn.draw();
    }

    private void recreate() {
        Window w = new Window();
        w.setWidth("95%");
        w.setHeight("95%");
        w.setMembersMargin(0);
        w.setModalMaskOpacity(70);
        w.setTitle("Grouping test");
        w.setShowMinimizeButton(false);
        w.setIsModal(true);
        w.setShowModalMask(true);
        w.centerInPage();

        ListGrid lg = new ListGrid(supplyItemDS);
        lg.setCanEdit(false);
        lg.setCanGroupBy(false);
[B]        lg.setGroupByMaxRecords(200);[/B]
        lg.setAllowFilterExpressions(true);
        lg.setShowFilterEditor(true);
        lg.setAutoFetchData(false);
        lg.setGroupStartOpen(GroupStartOpen.ALL);
        lg.setCanGroupBy(false);
        lg.setCanReorderFields(true);

        ListGridField itemIdLGF = new ListGridField("itemID");
        ListGridField itemNameLGF = new ListGridField("itemName");
        ListGridField skuLGF = new ListGridField("SKU");
        ListGridField descriptionLGF = new ListGridField("description");
        ListGridField categoryLGF = new ListGridField("category");
        categoryLGF.setHidden(true);
        categoryLGF.setCanHide(false);
        ListGridField unitsLGF = new ListGridField("units");

[B]        // Relevant row:
        lg.setSortByGroupFirst(true);[/B]
        lg.setGroupByField(categoryLGF.getName());

        lg.setFields(itemIdLGF, itemNameLGF, skuLGF, descriptionLGF, categoryLGF, unitsLGF);

[B]        lg.setSort(new SortSpecifier[] { new SortSpecifier(itemIdLGF.getName(), SortDirection.ASCENDING),
                new SortSpecifier(itemNameLGF.getName(), SortDirection.ASCENDING) });

        // lg.setSort(new SortSpecifier[] { new SortSpecifier(categoryLGF.getName(), SortDirection.ASCENDING),
        // new SortSpecifier(itemIdLGF.getName(), SortDirection.ASCENDING), new SortSpecifier(itemNameLGF.getName(), SortDirection.ASCENDING) });[/B]

        lg.fetchData(new Criterion(itemIdLGF.getName(), OperatorId.LESS_OR_EQUAL, 300));

        w.addItem(lg);
        w.show();
    }
}

The 2nd fetch goes away if I remove setSortByGroupFirst(). I don't think this is on purpose but a bug. If it is on purpose, I'd say that using the data from the 1st fetch is the better way to handle the situation.
I programmed the ListGrid to be sorted by the GroupBy-Field first. If the grid can't be grouped because of the amount of records, I'll still want the data to be sorted by that column, at least until I manually change the sort.

Other solution: If I want the behavior described, I'll do

lg.setSort(new SortSpecifier[] { new SortSpecifier(categoryLGF.getName(), SortDirection.ASCENDING),
        new SortSpecifier(itemIdLGF.getName(), SortDirection.ASCENDING), new SortSpecifier(itemNameLGF.getName(), SortDirection.ASCENDING) });

which seems to work for me (only one request for the first 200 rows, not a double sort by category in front). But this approach has the downside of small "2" and "3" at the itemIdLGF and itemNameLGF fields, if the categoryLGF is hidden, which is the case in my application.

How should this be solved? IMHO the correct way is: Sort by groupColumn(s), then by setSort-column(s), display sort indicator on all columns, where groupColumn(s) has the sortIndicator-"1" (if visible).
When the user re-sorts, remove the sorting by groupColumn-column(s).

I did not test 5.0p.

Best regards
Blama

[Isomorphic]

It seems this is a legitimate behavior.
The initial request asks for a range of rows from the server sorted by category, then itemID, then itemName. This is as expected since the category is the groupBy field you're sorting by first.

It realizes that there are more total rows in the dataSet (so it doesn't have a full local cache of data), and it ungroups as the max grouping threshold has been exceeded
The ungrouped grid now needs to display the data sorted by the specified sort fields, which are just itemID, itemName.

It has to issue a new fetch request against the server as it doesn't have a complete cache on the client side and the sorting is different so some of the rows that we were not passed in the first response may legitimately be present in the second response (as the order of records will be different).

The proposed fix of applying explicit sort-specifiers to put 'category' at the front seems appropriate.

You can suppress the sortNumerals by setting showSortNumerals to false on your grid. One option if you don't like the defaults is to simply suppress them. You could also customize the title HTML for the columns to embed different sort indicator information in your titles.

Regards
Isomorphic Software

[Isomorphic]

Some time later, we've just added ListGrid.setSortNumeralCustomizer() to 5.1 - you can try it out in builds dated December 18 and later.

If you want to show different numbers (for example starting with the first *visible* field rather than the first sorted field, which may not be visible), then you could make use of this customizer and look at the current sort state and currently visible fields, returning appropriate indices.