Interaction with paged TreeGrid

[SimonF]

Hi there,

I'm working with the paged treegrid and I’m trying to get a solution to a usecase for the paged treegrid.

I have a scenario that I want to show a hierarchy of a company. Like shown in the examples. After opening multiple nodes, the scrollbar appears and the user can scroll within the treegrid.

Now I have the option that in the last depth (there are ony leaves returned after opening the node) the user can interact with this elements. For example the user can approve some of the shown elements in the last depth.
Through interaction, the user could now remove some nodes.

Now there are two possibilities:
1. To reflect the changes the user might have done, the tree could be set new.
This would result into a new datasource-request for the root-node. The problem is, that the treegrid does scoll up to the top, so the last seen nodes are not visible anymore. If you're lucky the node which was opened, is already visible and the datasource could return the request with "isOpne: true". If the user scrolled down (for the second set of child-nodes of the root), has opened a node and made an interaction, it is not possible to go there by now. It's not possible to use scrollToRow, because you dont know the row, where the parent-node of the interaction-node might be.
This results to: Everytime the user does any interaction with a treenode, the tree has to be created new and the tree will always jump to the top. Even if the datasource responds with "isOpen:true" at the right nodes which leads to the dedicated node, which should be opened.

Jumping to the top at every interaction is quite obstructively. This was already realized in the not paged treegrid. If you have a large tree, and open multiple nodes and scroll to the bottom you can interact with these nodes. If you now work with "tree.setRoot({...})" the scroll-bar stays at the same position and you can see the same nodes, as before.

2. The other possibility could be, to remove one specific node.
The API for that operation exists. Trying to work with this, does result in nothing, because the node will not be removed.
There is also a possibility that you might achieve a move-operation between the nodes (so one child gets a new parent), so you remove and add the node at a different position. I have only tested the remove()-operation, you can take a look at the example code below.

isc.VLayout.create({
	"ID" : "rootLayout_5",
	"width" : "100%",
	"height" : "100%",
	"autoDraw" : true,
	"hideUsingDisplayNone" : false,
	"leaveScrollbarGap" : false,
	"members" :
	[
		isc.Button.create({
			title: "remove node 'root-1-3'",
			width: 200,
			click:function(){
				theTreeGrid.data.remove(theTreeGrid.data.findById("root-1-3"));
			}}
		),
		isc.TreeGrid.create({
			"ID" : "theTreeGrid",
			"width" : "100%",
			"height" : "100%",
			"selectionType" : "multiple",
			"canEdit" : false,
			"showFilterEditor" : false,
			dataSource : isc.DataSource.create({
				"fields" :
				[{
						"name" : "treeGridGeneratedIndex",
						"primaryKey" : true,
						"hidden" : true,
						"canView" : false
					}, {
						"name" : "nameField",
						"title" : "Reason",
						"type" : "text"
					}, {
						"name" : "parentId",
						"rootValue" : "root",
						"foreignKey" : "treeGridGeneratedIndex",
						"hidden" : true
					}
				],
				"dataFormat" : "json",
				"dataURL" : "http://devset.de/treegrid2.php",
				"transformRequest" : requestTransformer,
				"transformResponse" : responseTransformer,
				"recordXPath" : "\/resultData",
				useHttpProxy : false
			}),
			dataProperties : {
				openProperty : "isOpen",
				childrenProperty : "children",
				canReturnOpenFolders: true,
				progressiveLoading : false
			},
			"autoFetchData" : true,
			"dataPageSize" : 50,
			"dataFetchMode" : "paged",
			"selectionProperty" : "isSelected",
			"fields" :
			[{
					"name" : "nameField",
					"title" : "Reason",
					"type" : "text",
					"canEdit" : true,
					"canSort" : true
				}, {
					"name" : "lastField",
					"title" : "Name",
					"type" : "text",
					"canEdit" : true,
					"canSort" : true
				}, {
					"name" : "randomField",
					"title" : "random",
					"type" : "text",
					"canEdit" : true,
					"canSort" : true
				}
			],
			"selectionProperty" : "isSelected",
			"members" :
			[]
		})
	]
});
function requestTransformer(dataSourceRequest) {
	var operationType = {
		operationType : dataSourceRequest.operationType
	};
	if (dataSourceRequest.operationType == "fetch") {
		var params = {
			delay: 300,
			sortBy : dataSourceRequest.sortBy,
			start : dataSourceRequest.startRow,
			end : dataSourceRequest.endRow
		};
	}
	return isc.addProperties({}, operationType, dataSourceRequest.data, params);
}
function responseTransformer(dataSourceResponse, dataSourceRequest, jsonData) {
	if (dataSourceRequest.operationType == "fetch") {
		dataSourceResponse.totalRows = jsonData.totalRows;
		dataSourceResponse.endRow = jsonData.endRow;
		dataSourceResponse.startRow = jsonData.startRow;
	};
}

I think, there would be a third possibility, which sounds to me as the best solution.
If there would be the possibilty to reload the children of a specific node, it would solve the scrolling issue, and would also solve the removing issue. If the user tries to interact with the tree and you have to update some children you simply could start an "invalidateCache" on a specific (parent)node, and the tree would reload all of it's children. Because the rest of the tree stays untouched, there would be no need to scroll to the top again.
The only change, the tree would have to do, is to call the datasource for this specific node and copy the number of the attribute "totalRows" to the childCount of the specific node. Even the scoll-bar position could stay at the same position, because there would be no need to scroll to another position.
By now an "invalidateCache"-method is only implemented in the TreeGrid, so every time you call this, the whole tree gets reloaded. In other words, it's a invalidateCache on the root-node ;-)

I'm really interested in, what you think about this, and how to properly solve a change in the paged tree. For a "read-only" tree, the current implemented solution is quite good - and maybe the treegrid was created for a "read-only" use-case.
If you want to interact in a paged treegrid with the displayed treenode the scrolling to the top is really annoying (I have tested it out). Also I can understand why remove() is not functional with a paged treegrid, because you could not guarantee the synchronicity between the client and the server.

Best Regards

[Isomorphic]

To remove a specific node, use DataSource.removeData(). A direct call to tree.remove() is basically an attempt to directly modify the client-side cache structure and is disallowed. Calling removeData() will cause the specified node to be removed from view once the operation completes.

To reload the children of a node, use Tree.reloadChildren(node).

[SimonF]

Thanks, that I was searching for.

Seems there is some bug(?) with the reloadChildren(node).

At first, I want to remove the node "root-1-3" from the treegrid (this is only to show you that data from the datasource-is loaded).
Removing node "root-1-3" has no effect on the server-side, so the next request does include "root-1-3".
After removing the node I want to reload the children of node "root-1".
It seems that are some empty rows added. In fact there are 12 rows added, which is also the childCount of "root-1" and the number of totalRows in the answer of the datasource-request for root-1.
Same happens if i don't delete "root-1-3" at the beginning.


The datasource-response seems to be normal and like in the other 2 posts, the datasource only sets childCount if the folder is closed or there are any children set.

Code for reproduction:

isc.VLayout.create({
	"ID" : "rootLayout_5",
	"width" : "100%",
	"height" : "100%",
	"autoDraw" : true,
	"hideUsingDisplayNone" : false,
	"leaveScrollbarGap" : false,
	"members" :
	[
		isc.Button.create({
			title: "(non-ds-only) remove node 'root-1-3'",
			width: 200,
			click:function(){
				theTreeGrid.data.remove(theTreeGrid.data.findById("root-1-3"));
			}}
		),
		isc.Button.create({
			title: "remove node 'root-1-3'",
			width: 200,
			click:function(){
				theTreeGrid.dataSource.removeData(theTreeGrid.data.findById("root-1-3"));
			}}
		),
		isc.Button.create({
			title: "reload node 'root-1'",
			width: 200,
			click:function(){
				theTreeGrid.data.reloadChildren(theTreeGrid.data.findById("root-1"));
			}}
		),
		isc.TreeGrid.create({
			"ID" : "theTreeGrid",
			"width" : "100%",
			"height" : "100%",
			"selectionType" : "multiple",
			"canEdit" : false,
			"showFilterEditor" : false,
			dataSource : isc.DataSource.create({
				"fields" :
				[{
						"name" : "treeGridGeneratedIndex",
						"primaryKey" : true,
						"hidden" : true,
						"canView" : false
					}, {
						"name" : "nameField",
						"title" : "Reason",
						"type" : "text"
					}, {
						"name" : "parentId",
						"rootValue" : "root",
						"foreignKey" : "treeGridGeneratedIndex",
						"hidden" : true
					}
				],
				"dataFormat" : "json",
				"dataURL" : "http://devset.de/treegrid2.php",
				"transformRequest" : requestTransformer,
				"transformResponse" : responseTransformer,
				"recordXPath" : "\/resultData",
				useHttpProxy : false
			}),
			dataProperties : {
				openProperty : "isOpen",
				childrenProperty : "children",
				canReturnOpenFolders: true,
				progressiveLoading : false
			},
			"autoFetchData" : true,
			"dataPageSize" : 50,
			"dataFetchMode" : "paged",
			"selectionProperty" : "isSelected",
			"fields" :
			[{
					"name" : "nameField",
					"title" : "Reason",
					"type" : "text",
					"canEdit" : true,
					"canSort" : true
				}, {
					"name" : "lastField",
					"title" : "Name",
					"type" : "text",
					"canEdit" : true,
					"canSort" : true
				}, {
					"name" : "randomField",
					"title" : "random",
					"type" : "text",
					"canEdit" : true,
					"canSort" : true
				}
			],
			"selectionProperty" : "isSelected",
			"members" :
			[]
		})
	]
});
function requestTransformer(dataSourceRequest) {
	var operationType = {
		operationType : dataSourceRequest.operationType
	};
	if (dataSourceRequest.operationType == "fetch") {
		var params = {
			delay: 300,
			sortBy : dataSourceRequest.sortBy,
			start : dataSourceRequest.startRow,
			end : dataSourceRequest.endRow
		};
	}
	return isc.addProperties({}, operationType, dataSourceRequest.data, params);
}
function responseTransformer(dataSourceResponse, dataSourceRequest, jsonData) {
	if (dataSourceRequest.operationType == "fetch") {
		dataSourceResponse.totalRows = jsonData.totalRows;
		dataSourceResponse.endRow = jsonData.endRow;
		dataSourceResponse.startRow = jsonData.startRow;
	};
}

Best

[Isomorphic]

We've fixed the "reload" operation that your sample uses, and it will be ported back to SC 10.0p for the nightly builds available 5-13-2015.

However, some notes:
- the server data for this sample still contains violations of the rules we mentioned in your other thread, though it should not cause a failure
- note the warning in the dev console when the top button, "non-ds-only" is pressed; this is because ResultSets are read only

[SimonF]

Thanks the reload operation is now fixed and doesn't add empty rows to the treegrid. (tested in SmartClient_v100p_2015-05-13_Pro)

Like in the other thread there seems to be a redraw after the reload. In this case I can understand the hiding and showing of all nodes beneath root-1, but it doesn't look very "nice", will the fix from the other post also impact this behaviour?


Sorry for not removing the button "non-ds-only" it wasn't usable for the example, therefore i hadn't included it into the gif.
Also i have modified the datasource once again, so the childCount will be only set if there are children set in the response, sorry.

Best Regards

[SimonF]

I have tested this against the latest nightly (SmartClient_v100p_2015-05-15_Pro) and this issue is - like assumed - already fixed. Thank you for that.

I know that the repsonse of the server return "isOpen: true" although there are no children given.
But this requirement is not so easy to fulfill.

The information about "there is any data" could be retrieved very fast. But the information "which data" is chilren of a node could take some time. So instead of waiting for the calculations on which data should be displayed (and blocking the whole first request), it would be better to set the "isOpen: true" to get the response for the first request of the treegrid back asap. For the second request which gather all needed informations, it could take longer and indeed, there could be multiple requests at once, but it seems reasonable me to to use the fix that "isOpen:true" without children are fobidden, but children gets requested with a warning you have implemented.

Some example:
- Get the children of root node: 200ms
- Getting informations of root-x if there are any childs and it should be opened: 10ms
- Getting the data for root-x nodes 200 ms
- Also there is no time-difference in getting the first child or all childs

Assuming the root node has 10 children and 3 of them should be opened at the beginning.
With the requirements you have stated out its:
200ms for all children of the root-node
+3*200 ms for the first child of every open children
+ 10*10ms for all childs there is one call if there are any children to return
So the initial request for the treegrid needs 900 ms

If I'm using only the "isOpen: true" attribute without sending the childCount or any open children its this calculation:
200ms for all children of the root-node
+ 10*10ms for checking the children has to be open because there are data.
The first request only need 300ms. There are also 3 more nodes afterwards (3*200 ms), but the user has a faster feedback.

This increases much more, if the initial assumed 200 ms for retrieving all childs gets higher. Assuming with 1 second for the children this is 4,1 seconds vs. 1,1 seconds.

Getting back to the usecase of the paged treegrid it really makes sense to use paged treegrids if there is a need to display large datasets. So I think it should be feasible to set "isOpen: true" without any children to increase the response-time of the first request.

Best

[Isomorphic]

The vast majority of the time, it is far more efficient, a better user experience, and a simpler programming model for application developers to assume that there will not be serial requests to the server to load additional data and put the tree in a consistent state.

For your edge case, if you really think it's worthwhile to address, send back closed folders, then call openFolder().

[SimonF]

Hmm, sending back closed folders and open the folders afterwards with openFolder() would mean, that I have to work with the dataArrived()-method?

Otherwise, the tree would be rendered in the client and only after a button-click(or another user interaction) the treenode would get opened. This doesn't seems to look as nice as setting "isOpen: true", does it?

Best

[Isomorphic]

Of course we're not suggesting that the user clicks something to make the second load happen. There are lots of ways to have the second load happen automatically; one would be using dataArrived, another would be to use using queuing to add a second DSRequest that loads the list of open folders, and when that callback fires, start opening folders.