ListGrid - limit fetch result data to ListGrid columns

[daniel_h]

Hello Isomorphic,

There is a certain requirement described below with a possible solution
that would need a patch in the isomorphic JS library.

Could you please verify the solution from your perspective and in case it makes sense, may be consider to apply the patch.

Version: SmartClient Version: v9.1p_2014-11-19/Pro Deployment (built 2014-11-19)

Requirement:
- ListGrid data fetch should return a resultSet that only contains the columns of the ListGrid, not the full object specified by the datasource.
(But still support the original behavior as needed)

- Usage of a specific operationBinding with limited "outputs" is considered to be to "customized". A more general approach is requested.

Possible solution:
- Define the "outputs" limitation as part of the ListGrid definition:

<ListGrid ...
   <dataProperties >
      <context>
	<outputs>Col1, Col2, Col3</outputs>
      </context>
   </dataProperties>
</ListGrid>

This already works for the initial data fetch.
Problem:
Subsequent fetches (filter, sort, ...) operate on the ResultSet and do override the context settings.
The override happens in DataBoundComponents.js
updateDataModel : function (filterCriteria, operation, context)
...
resultSet.setContext(context);
...

The patch would be to merge the context.
In the code below, merge the "outputs" property back in.

updateDataModel : function (filterCriteria, operation, context) {
	// tell the ResultSet/ResultTree the filter changed
	if (this.logIsDebugEnabled()) {
		this.logDebug("Setting filter to: " + this.echoFull(filterCriteria));
	}

	// update the context - this allows requestProperties like "showPrompt" / textMatchStyle
	// to change
	var resultSet = this.getData();
	// Handle the grid being grouped
	if (!this.dataObjectSupportsFilter(resultSet)) resultSet = this.originalData;

	if (!this.dataObjectSupportsFilter(resultSet)) {
		return;
	}
	///////////////////////////////////////////////////////////////////////////////////////
	// Patch to keep non-changeable requestProperties
	if(resultSet.context) {
		// apply non-changeable requestProperties
		if(resultSet.context.outputs) context.outputs = resultSet.context.outputs;		
	}
	///////////////////////////////////////////////////////////////////////////////////////
	resultSet.setContext(context);
	
	// if the ResultSet won't kick off an immediate fetch, kill the afterFlowCallback
	// This is the callback passed into fetchData(...) and would normally be cleared by
	// ResultSet.fetchDataReply()
	// If we don't clear it here, the next time a fetch occurs (EG via 'invalidateCache()') the
	// callback will occur (once) when that fetch completes.
	if (!resultSet.willFetchData(filterCriteria)) delete context.afterFlowCallback;
	resultSet.setCriteria(filterCriteria);

	return resultSet;
}

May be overriding the context regardless, as in the original source code,
might have already introduced problems in the past.

If merging the context is ok, the ListGrid class could even be extended to request only columns or full data objects by using a flag, so the configuration part in <dataproperties> section would not be needed but is done automatically within the ListGrid class.
Of course Master/Detail features need to be reworked too.

What do you think?

Thank you,
Daniel

[Isomorphic]

resultSet.context is not a documented API, so this patch is definitely not something we'd apply.

It's not clear what you think is wrong with the documented ways of doing this (listGrid.fetchOperation, or passing requestProperties to fetchData) - you simply said it was 'considered to be to "customized"' which doesn't make much sense.

As far as requesting only the visible columns, if that's what you're attempting, note that this is a false performance gain: it causes new trips to the server as columns are shown and hidden, or as data from hidden fields is display in various ways (hovers, expandable records, etc). It's much more expensive to go to the server for a new set of records than to deliver extra columns in the server's first response.

[daniel_h]

Thank you for your reply.

First, I meant all columns specified in the ListGrid including invisible ones.
So show/hide columns is supported without any server round trip.

Second, I agree about not using context since it is not documented.

So I changed my code to use requestProperties
and set requestProperties.outputs to limit the columns.

It does again work on the first fetch,
but subsequent fetch from ResultSet get the full object again.

As far as I can see, it is the same root cause,
since the requestproperties are set as context internally and thus are overriden in subsequent ResultSet fetch.

Or may be I'm doing something wrong in MyListGrid class override of fetchData method.
Is is as follows:

isc.defineClass("MyListGrid", "ListGrid");

isc.MyListGrid.addProperties({
	// Flag to indicate whether only the columns defined in the ListGrid
	// should be fetched and returned in the ResultSet
	fetchColsOnly: false,
	
	// override fetchData to apply column restriction if specified
	fetchData : function(criteria, callback, requestProperties) 
	{
		if(this.fetchColsOnly) {
			var outputStr = "";
			var allFields = this.getAllFields();
			var distinctTracker = new Object();
			for(var i=0; i < allFields.length; i++) {
				if(allFields[i].dataPath) {
					// get the top level part of the data path
					var topLevel = allFields[i].dataPath.substring(0, allFields[i].dataPath.indexOf("/"));
					if(!distinctTracker.hasOwnProperty(topLevel)) {
						distinctTracker[topLevel] = 1;
						outputStr += topLevel + ","; 
					}
					outputStr += allFields[i].dataPath; 
				}
				else { 
					outputStr += allFields[i].name; 
				}
				if(i < allFields.length-1) {
					outputStr += ","; 
				}
			}
			requestProperties.outputs = outputStr;
			
		}
		return this.Super("fetchData", arguments);
	}

});

There is some extra logic to support dataPath setting which I think can be ignored for our topic.

Thank you.

[Isomorphic]

You didn't address this point:

It's not clear what you think is wrong with the documented ways of doing this (listGrid.fetchOperation, or passing requestProperties to fetchData) - you simply said it was 'considered to be to "customized"' which doesn't make much sense.

You're asking for a parallel mechanism to an existing mechanism, and it's not clear to us why you don't like the existing mechanism.

[daniel_h]

Well, the point of passing requestProperties has been addressed but shows the same issue that is on subsequent fetch from ResultSet the output limitation is ignored.

The point about ListGrid fetchOperation is that it would need for each DataSource an own operation that limits the output and further more for ListGrids that do use the same DataSource but should fetch a different column set it needs again another operation.

The major use case to be supported (and I believe it has been requested a few times in the forum) is to have in addition to the current ListGrid behavior, another ListGrid that has the common behavior of only fetching the fields that are defined as ListGrid columns. (Or the same ListGrid with a parameter to drive that behavior)

Having the restriction on the ListGrid requestProperties workable would allow a general reusable ListGrid component that does exactly that.

And given the point that isomorphic tells me to look at requestProperties could mean that it actually should work by
setting requestProperties.outputs and the fact that is does not is just because it is a bug.

May be we are talking about a bug fix thing here? ;-)

[Isomorphic]

This isn't making sense - you're showing an override of fetchData() that would *always* pass outputs in requestProperties. But you're claiming that when outputs are configured, they can be dropped when a fetch is performed again. This wouldn't matter if you are always passing outputs.

So we need to see what sequence of calls you are doing where outputs work, and then they do not work in a surprising way (specifically - there were no new requestProperties passed, implying there should have been no change in outputs).