ListGrid setData from ResultSet causes grid scrolling to ignore fetchOperation

[dczech]

When I setData with a ResultSet from a DataSource.fetchData call, subsequent scrolling on the ListGrid causes the fetch operations to ignore fetchOperation set on the ListGrid (see attached png for DeveloperConsole RPC tab). I have tested this with Firefox 3.6 and SC_SNAPSHOT-2011-06-13/PowerEdition in hosted mode.

Here is reproducing code example:

package com.smartgwt.sample.client;

import com.smartgwt.client.data.DSCallback;
import com.smartgwt.client.data.DSRequest;
import com.smartgwt.client.data.DSResponse;
import com.smartgwt.client.data.DataSource;
import com.smartgwt.client.data.ResultSet;
import com.smartgwt.client.types.FetchMode;
import com.smartgwt.client.types.TextMatchStyle;
import com.smartgwt.client.widgets.IButton;
import com.smartgwt.client.widgets.Label;
import com.smartgwt.client.widgets.events.ClickEvent;
import com.smartgwt.client.widgets.events.ClickHandler;
import com.smartgwt.client.widgets.grid.ListGrid;
import com.smartgwt.client.widgets.layout.VLayout;

public class RefreshGridTest 
	extends VLayout
{
	private ListGrid grid;
	private static final int REFRESH_ROW_OFFSET = 15;
	
	private static String DATASOURCE = "PING";
	private static String FETCH_OP = "isOne";
	
	private Label label;
	
	public RefreshGridTest()
	{
		setMargin(10);
		setWidth(400);
		setHeight(500);
		setMembersMargin(5);
		
		grid = new ListGrid();
		grid.setDataSource(DataSource.get(DATASOURCE));
		grid.setFetchOperation(FETCH_OP);

		grid.setWidth100();
		grid.setHeight100();
		grid.setShowFilterEditor(true);
		grid.setFetchDelay(1000);
		grid.setFilterOnKeypress(true);
		grid.setAutoFetchData(true);
				
		addMember(grid);
		
		label = new Label();
		label.setHeight(16);
		label.setWidth(400);
		
		addMember(label);
		updateLabel();
		
		final IButton button = new IButton("refresh");
		button.addClickHandler(new ClickHandler() {
			@Override
			public void onClick(ClickEvent event) {
				refresh();
				updateLabel();
			}
		});

		addMember(button);

	}

	private void updateLabel()
	{
		label.setContents(grid.getFetchOperation());
	}
	
	private void refresh()
	{
		final Integer row[] = grid.getVisibleRows();
		
		final DSRequest request = new DSRequest();
		request.setStartRow(Math.max(0, row[0] - REFRESH_ROW_OFFSET));
		request.setEndRow(row[1] + REFRESH_ROW_OFFSET);
		request.setSortBy(grid.getSort());
		request.setOperationId(FETCH_OP);
		request.setTextMatchStyle(TextMatchStyle.SUBSTRING);

		final DataSource ds = grid.getDataSource();
		ds.setShowPrompt(false);
		ds.fetchData(grid.getCriteria(), new DSCallback() {

			@Override
			public void execute(DSResponse response, Object rawData, DSRequest request)
			{
				final ResultSet resultset = new ResultSet();
				resultset.setDataSource(ds);
				resultset.setCriteria(grid.getCriteria());
				resultset.setFetchMode(FetchMode.PAGED);
				resultset.setInitialLength(response.getTotalRows());
				resultset.setInitialData(response.getData());
				resultset.setInitialSort(request.getSortBy());

				grid.setData(resultset);
			}
		}, request);
	}
}

Here is the ds.xml:

<DataSource schema="PING" dbName="Oracle" tableName="PING" ID="PING" serverType="sql" >
	<fields>
		<field name="id" sequenceName="PING_SEQ" primaryKey="true" type="sequence" hidden="true"></field>
		<field name="name" length="51" type="text">
			<validators>
				<validator type="isUnique" requiresServer="true" />
			</validators>
		</field>
		<field name="other" length="21" type="text"></field>
		<field name="type" type="number"></field>
	</fields>
	
	<operationBindings>
		<operationBinding operationType="fetch"	operationId="isOne">
			<whereClause>$defaultWhereClause and type = 1</whereClause>
		</operationBinding>
		<operationBinding operationType="fetch"	operationId="notOne">
			<whereClause>$defaultWhereClause and type != 1</whereClause>
		</operationBinding>
	</operationBindings>
	
</DataSource>

The refresh call would normally be run on a scheduled Timer rather than via button. I'm not sure how I'm messing up the scrolling with my manual fetch and set?

[dczech]

Adding

resultset.setFetchOperation(FETCH_OP);

seems to keep the fetchOperation in place for the results with my preliminary tests, but the RPC log shows the fetch without the OperationId.

Also, if the grid is scrolled down past the record 0 to midway or the end, for example, the

grid.setData(resultset);

causes a second fetch. Is there a way to setInitialScrollPosition or something similar to prevent the second fetch?

[Isomorphic]

It's expected that a manually created ResultSet will not use ListGrid.fetchOperation, or in fact any settings from the ListGrid related to the ResultSet, since the ListGrid is no longer creating the ResultSet. You can use ResultSet.setFetchOperation() to supply the desired operationId.

As far as preserving scroll position without a fetch, note that ResultSet.setInitialData() accepts a gappy array (nulls in some slots). This allows you to initialize a ResultSet such that it has data for the row range currently showing in a ListGrid.

[dczech]

Is there anyway I can prevent the immediate second fetch after I call grid.setData(resultset) when the scrolling is anywhere other than the top (0 row) position? In some cases I am seeing duplicate records being displayed in the ListGrid that seem to be added by the second fetch.

[Isomorphic]

What are the two fetches you're seeing?

Can you reproduce an unnecessary fetch by adding code to a Showcase sample to create and apply a ResultSet?

[dczech]

Here is a modified Live Grid example from the 6/13 LGPL build that causes a second fetch after the setData(resultSet) is called when the scrollbar thumb is in the bottom-most position. The second fetch is not called when the scrollbar thumb is in the top-most position.

Attached is a screenshot of the RPC tab from the Developer Console. Transaction #3 is caused by my manually invoked ds.FetchData. Transactions #4 and #5 seem to be triggered by the setData. When setData is not called, there is no subsequent transactions after the manual ds.FetchData.

(Transaction #1 is the initial fetch data. Transaction #2 is caused by quickly dragging the scrollbar thumb to the bottom most position.)

package com.smartgwt.sample.showcase.client.grid;

import com.smartgwt.client.data.DSCallback;
import com.smartgwt.client.data.DSRequest;
import com.smartgwt.client.data.DSResponse;
import com.smartgwt.client.data.DataSource;
import com.smartgwt.client.data.ResultSet;
import com.smartgwt.client.rpc.RPCManager;
import com.smartgwt.client.types.FetchMode;
import com.smartgwt.client.types.ListGridFieldType;
import com.smartgwt.client.types.PromptStyle;
import com.smartgwt.client.types.TextMatchStyle;
import com.smartgwt.client.widgets.Canvas;
import com.smartgwt.client.widgets.IButton;
import com.smartgwt.client.widgets.events.ClickEvent;
import com.smartgwt.client.widgets.events.ClickHandler;
import com.smartgwt.client.widgets.grid.CellFormatter;
import com.smartgwt.client.widgets.grid.ListGrid;
import com.smartgwt.client.widgets.grid.ListGridField;
import com.smartgwt.client.widgets.grid.ListGridRecord;
import com.smartgwt.client.widgets.layout.VLayout;
import com.smartgwt.sample.showcase.client.PanelFactory;
import com.smartgwt.sample.showcase.client.ShowcasePanel;
import com.smartgwt.sample.showcase.client.data.ItemSupplyXmlDS;

public class LiveGridFetchSample extends ShowcasePanel {

    private static final String DESCRIPTION =
            "<p>Rows are fetched automatically as the user drags the scrollbar. Drag the scrollbar quickly to the bottom to " +
                    "fetch a range near the end (a prompt will appear during server fetch).</p><p>Scroll slowly back up to fill in the middle.</p>" +
                    "<p>Another key unique feature of Smart GWT is lazy rendering of <b>columns</b>. Most browsers cannot handle displaying a large number of column and have serious performance issues." +
                    "Smart GWT however does not render all columns outside the visible area by default and only renders them as you scroll horizontally. You can however disable this feature if desired.</p>" +
                    "<p>You can control how far ahead of the currently visible area rows should be rendered. This is expressed as a ratio from viewport size to rendered area size. The default is 1.3.</p>" +
                    "<p>Tweaking drawAheadRatio allows you to make tradeoffs between continuous scrolling speed vs initial render time and render time when scrolling by large amounts.</p>";

    private ListGrid listGrid;
	private static final int REFRESH_ROW_OFFSET = 15;

    public static class Factory implements PanelFactory {
        private String id;

        public Canvas create() {
            LiveGridFetchSample panel = new LiveGridFetchSample();
            id = panel.getID();
            return panel;
        }

        public String getID() {
            return id;
        }

        public String getDescription() {
            return DESCRIPTION;
        }
    }

    public Canvas getViewPanel() {

    	VLayout vlayout = new VLayout();
    	vlayout.setWidth100();
    	vlayout.setHeight100();
    	vlayout.setMembersMargin(5);
    	
        RPCManager.setPromptStyle(PromptStyle.CURSOR);
        DataSource dataSource = ItemSupplyXmlDS.getInstance();

        ListGridField rowNum = new ListGridField("itemNum", "Item No.");
        rowNum.setWidth(65);
        rowNum.setCellFormatter(new CellFormatter() {
            public String format(Object value, ListGridRecord record, int rowNum, int colNum) {
                return rowNum +"";
            }
        });

        ListGridField itemName = new ListGridField("itemName", 100);
        ListGridField sku = new ListGridField("SKU", 100);
        ListGridField description = new ListGridField("description", 150);
        ListGridField category = new ListGridField("category", 100);
        ListGridField units = new ListGridField("units", 100);

        ListGridField unitCost = new ListGridField("unitCost", 100);
        unitCost.setType(ListGridFieldType.FLOAT);

        ListGridField inStock = new ListGridField("inStock", 100);
        inStock.setType(ListGridFieldType.BOOLEAN);

        ListGridField nextShipment = new ListGridField("nextShipment", 100);
        nextShipment.setType(ListGridFieldType.DATE);



        listGrid = new ListGrid();
        listGrid.setWidth100();
        listGrid.setHeight100();
        listGrid.setAutoFetchData(true);
        listGrid.setDataSource(dataSource);
        listGrid.setShowRowNumbers(true);

        listGrid.setFields(rowNum, itemName, sku, description, category, units, unitCost,
                            itemName, sku, description, category, units, unitCost,
                            itemName, sku, description, category, units, unitCost);

        vlayout.addMember(listGrid);
        
		final IButton button = new IButton("refresh");
		button.addClickHandler(new ClickHandler() {
			@Override
			public void onClick(ClickEvent event) {
				refresh();
			}
		});

		vlayout.addMember(button);

        
        return vlayout;
    }

	private void refresh()
	{
		final Integer row[] = listGrid.getVisibleRows();
		
		final DSRequest request = new DSRequest();
		request.setStartRow(Math.max(0, row[0] - REFRESH_ROW_OFFSET));
		request.setEndRow(row[1] + REFRESH_ROW_OFFSET);
		request.setSortBy(listGrid.getSort());
		request.setTextMatchStyle(TextMatchStyle.SUBSTRING);

		final DataSource ds = listGrid.getDataSource();
		ds.setShowPrompt(false);
		ds.fetchData(listGrid.getCriteria(), new DSCallback() {

			@Override
			public void execute(DSResponse response, Object rawData, DSRequest request)
			{
				final ResultSet resultset = new ResultSet();
				resultset.setDataSource(ds);
				resultset.setCriteria(listGrid.getCriteria());
				resultset.setFetchMode(FetchMode.PAGED);
				resultset.setInitialLength(response.getTotalRows());
				resultset.setInitialData(response.getData());
				resultset.setInitialSort(request.getSortBy());
				listGrid.setData(resultset);

				
			}
		}, request);
	}

    

    public String getIntro() {
        return DESCRIPTION;
    }
}

DSRequest for #3:

{
    "dataSource":"supplyItemDS", 
    "operationType":"fetch", 
    "data":{
    }, 
    "startRow":641, 
    "endRow":700, 
    "sortBy":null, 
    "textMatchStyle":"substring", 
    "showPrompt":false, 
    "requestId":"supplyItemDS$6272"
}

DSRequest for #4:

{
    "dataSource":"supplyItemDS", 
    "operationType":"fetch", 
    "data":{
    }, 
    "startRow":609, 
    "endRow":683, 
    "resultSet":[ResultSet ID:isc_ResultSet_1 (created by: undefined)], 
    "callback":{
        "caller":[ResultSet ID:isc_ResultSet_1 (created by: undefined)], 
        "methodName":"fetchRemoteDataReply"
    }, 
    "willHandleError":true, 
    "showPrompt":false, 
    "clientContext":{
        "requestIndex":1
    }, 
    "requestId":"supplyItemDS$6273"
}

DSRequest for #5:

{
    "dataSource":"supplyItemDS", 
    "operationType":"fetch", 
    "data":{
    }, 
    "startRow":648, 
    "endRow":686, 
    "resultSet":[ResultSet ID:isc_ResultSet_1 (created by: undefined)], 
    "callback":{
        "caller":[ResultSet ID:isc_ResultSet_1 (created by: undefined)], 
        "methodName":"fetchRemoteDataReply"
    }, 
    "willHandleError":true, 
    "showPrompt":false, 
    "clientContext":{
        "requestIndex":2
    }, 
    "requestId":"supplyItemDS$6274"
}

[Isomorphic]

In your attached code, you're applying the result of response.getData() directly to the resultSet as initialData. This isn't working when the request startRow > 0 because the initialData is expected to be a sparse (gappy) array of essentially the full data set, but dsResponse.data is an array containing only the records in the requested startRow --> endRow range.

You can get this working by setting resultSet.initialData to an array containing null entries for rows zero -> requested startRow, and then the returned data -- something like this:

		ds.fetchData(listGrid.getCriteria(), new DSCallback() {
		    
			@Override
			public void execute(DSResponse response, Object rawData, DSRequest request)
			{	    
				final ResultSet resultset = new ResultSet();
				resultset.setDataSource(ds);
				resultset.setCriteria(listGrid.getCriteria());
				resultset.setFetchMode(FetchMode.PAGED);
				resultset.setInitialLength(response.getTotalRows());

				Record[] responseData = response.getData();
				Record[] initialData = new Record[response.getEndRow()+1];
				for (int i = 0; i <= response.getEndRow(); i++) {
				    if (i < response.getStartRow()) initialData[i] = null;
				    else initialData[i] = responseData[i-response.getStartRow()];
				}
				resultset.setInitialData(initialData);
				resultset.setInitialSort(request.getSortBy());
				listGrid.setData(resultset);
			}
		}, request);

Thanks
Isomorphic Software

[dczech]

Thank you! That is a very helpful example. I was not understanding the sparse array in the ResultSet.

[lakshmipathi.d]

Not retaining the scroll position after refresh

After using the code given by ismorphic, still we are facing the scroll issue. Please help me out.

Here is the sample code


DSRequest req = new DSRequest();

Integer[] visibleRows = getVisibleRows();
Integer startRow = visibleRows[0];
Integer endRow = visibleRows[1];
req.setStartRow(startRow);
req.setEndRow(endRow);
req.setSortBy(getSort());

restDataSource.fetchData(this.getCriteria(),new DSCallback() {
@Override
public void execute(DSResponse response, Object rawData,
DSRequest request) {
final ResultSet resultset = new ResultSet();
resultset.setDataSource(restDataSource);
resultset.setCriteria(getCriteria());
resultset.setFetchMode(FetchMode.PAGED);
resultset.setInitialLength(response.getTotalRows());

Record[] responseData = response.getData();
Record[] initialData = new Record[response.getEndRow()];
for (int i = 0; i <= response.getEndRow() - 1; i++) {
if (i < response.getStartRow()) {
initialData[i] = null;
}
else {
initialData[i] = responseData[i-response.getStartRow()];
}
}
resultset.setInitialData(initialData);
if(request.getSortBy() != null){
resultset.setInitialSort(request.getSortBy());

}
setData(resultset);

}
}, req);

Regards
Lakshmipathi