Announcement

**soveste** · 24 May 2011, 10:28

I've also noticed that request.getStartRow() and request.getEndRow() are always null inside DataSource.transformRequest.

**soveste** · 24 May 2011, 11:56

It looks like TreeGrid does not support pagination, only ListGrid does.

**ibrandt** · 11 Jun 2012, 17:15

Is it still the case that this is not implemented? Per the API documentation we expected this functionality:

A TreeGrid works just like a ListGrid, except one column (specified by TreeGridField.treeField) shows a hierarchical Tree.

Except where explicitly overridden, ListGrid methods, callbacks, and properties apply to TreeGrids as well.

We have a need to display trees that may have an upper bound of around 100 nodes under the root, but fully expanded can have up to 100,000 nodes. We planned to use live scrolling (i.e. "pagination") to facilitate the much requested "expand all".

**Isomorphic** · 11 Jun 2012, 17:46

ListGrid supports *rendering* only the rows that are on-screen and so does TreeGrid. ListGrid does not support Tree load on demand so there is nothing to carry forward to TreeGrid here. Indeed ListGrid is not even doing the paging or generating startRow/endRow (it's ResultSet).

If you analyze how tree load on demand would need to work, you'll realize it's a rather tricky problem that cannot really be solved completely from the client-side at all, at least not without generating a flurry of requests to the server with very poor performance (hint: consider the case of nodes arriving from the server that are already open and hence should have children - recursively).

If it's absolutely required for you to page through large open trees, one approach is to make the client side ignorant of the tree structure - use ListGrid data paging with startRow and endRow, form the tree structure for the requested rows server-side and send back information in the rows that will allow you to render correct indentation / open/close controls, etc

**ibrandt** · 13 Jun 2012, 10:51

ListGrid supports *rendering* only the rows that are on-screen and so does TreeGrid. ListGrid does not support Tree load on demand so there is nothing to carry forward to TreeGrid here. Indeed ListGrid is not even doing the paging or generating startRow/endRow (it's ResultSet).

Okay, understood. There is rendering a subset of the records that are loaded on the client side, and then there is loading a subset of the total records available on the server. I've seen "client-side pagination" and "server-side pagination" used by other frameworks to distinguish between the two approaches respectively. I'll borrow those terms notwithstanding that SmartClient presents both with live scrolling as opposed to an explicit pagination widget. Also I'll use "visible tree" to refer to nodes viewable in the current scroll pane, "rendered tree" for nodes that are in the DOM but may be hidden, "loaded tree" for nodes cached client-side that may or may not be in the DOM, and "total tree" for nodes that exist server-side but haven't been sent to, or have been flushed from, the client-side cache.

Going back over the documentation I'd still say it's not overly clear that ListGrid uses ResultSet, TreeGrid uses ResultTree, and ResultTree doesn't support server-side pagination as ResultSet does. Being that I'm not the first to miss this might I propose some documentation additions:

Expand on the ListGrid overview to directly reference ResultSet, and summarize its role in paging, caching, and adaptive filtering and sorting.
Expand on the ResultTree overview, and link back to the Tree DataBinding overview.
Expand on the TreeGrid overview by summarizing the features available for the ListGrid, by way of ResultSet, that are not available for the TreeGrid, by way of ResultTree.

If you analyze how tree load on demand would need to work, you'll realize it's a rather tricky problem that cannot really be solved completely from the client-side at all, at least not without generating a flurry of requests to the server with very poor performance (hint: consider the case of nodes arriving from the server that are already open and hence should have children - recursively).

We currently do tree load on demand with an in-house developed "TreeTable" extension of the YUI 2 DataTable, which I'm looking to migrate to the SmartClient TreeGrid. Before that we did a similar implementation on top of the ExtJs Grid. Neither implements pagination nor filtering client or server-side. Hierarchical sort is only implemented client-side. It sorts over the loaded nodes. Subsequent children are sorted on load. We rely on the user's lack of willingness to expand all that many nodes to keep the number of rendered nodes to a viable number. We get frequent requests for expand-all, and autocomplete searching and seeking to a given node over the total tree. As such I've done analysis for adding server-side pagination, filtering, sorting, and searching/seeking to our current implementation. While certainly not an expert, I believe I'm somewhat familiar with the overall problem.

Bear with me if there is something I'm missing, but I'm not seeing how driving these features server-side is all that much different from the perspective of the client whether it's for a tree or a list. (The server-side implementation is another story of course.) Granted our trees are read-only. Are there any added cache synchronization concerns for adds, updates and deletes for trees over lists? Otherwise there are a couple notable client-side gotchas that I'm aware of:

Assuming the server is stateless it needs some way to know what the expansion state of the total tree is when it receives relevant requests (e.g. resort hierarchically by X and send me nodes 500-550).
The client needs to accept changes to its tree expansion and pagination state when the server sends a response that augments the loaded tree (e.g. here is the new selected node you searched for, and its expanded ancestry you didn't already have loaded).

I believe we have a partial solution to the first concern in our current implementation. We keep a cookie containing a delimited list of the deepest expanded node IDs. All their ancestors are implicitly loaded, as are all their children. The children are either collapsed, or are leaves of the total tree. This allows us to retain the expanded state of the tree across page reloads, or when the user switches between different reports over the same hierarchy.

The expanded nodes list is maintained client-side as a FIFO queue of a bounded number of nodes. The server-side consumes it to infer the subtree to return for a reload or report change. Our current queue size is 100, and we use low integer node IDs. That keeps the cookie size reasonable for our purposes. Others may want to set a lower or higher queue size depending on their needs. A better implementation would probably be to bound the queue by a number of bytes rather than a quantity of IDs, since the cookie size limit is really the upper bound. Of course there could be other ways of maintaining and communicating the queue between the client and server sides, but a cookie is the best fit for our state preservation requirements.

When a node drops off the end of the queue that lineage of the tree is considered collapsed up to the next common ancestor of any other lineages that may have also been expanded. It's been our experience that this is not something users notice, and it seems a very reasonable limitation that can be tuned to suit. With the addition of pagination, where the collapsed nodes may be off the current page, the client side could decide to flush the collapsed nodes from the rendered tree, and furthermore the loaded tree. If they were currently part of the visible tree they would of course be hidden or flushed from the rendered tree. How does the TreeGrid currently handle collapsed nodes, does it hide them or remove them from the DOM? Does ListGrid and friends handle flushing of rendered or loaded records at some point after they are scrolled out of view?

I believe the addition of this queue would give the server-side all it needs to answer requests related to the aforementioned features. Expand-all would complicate things a bit however, as you may overrun your expanded nodes queue size, resulting in "expand-some". We envisioned that the expand-all action would flush the expanded nodes queue, and revert it to a "collapsed nodes" queue instead. This would be distinguished by prepending the node IDs list with a minus sign, or some configurable character that is known to not be the first character of any node IDs. When in this mode any nodes that are collapsed are then pushed onto the queue. The server can then infer that any descendants of those nodes are not to be included in a given response, but all other nodes should be. The queue would revert to an expanded nodes list when either collapse-all is invoked, or perhaps when the collapsed queue size reaches some percentage of the total number of loaded non-leaf nodes. There may be another way to determine or approximate when the queue should optimally be flipped, but I believe a basic implementation here would suit our needs.

With the addition of the expanded/collapsed nodes queue by way of a cookie I don't see how any more requests to the server would be required for a tree over a list. I hadn't considered your example of the client receiving nodes that have an expanded flag set, yet with no children simultaneously provided. What would the scenario be that would result in a server providing such a response? I suppose I could see it coming up for a naive implementation of an action to expand all below a given node. A better implementation would be for the server to send the full subtree in response to that request. Otherwise I agree the recursive request penalty would have to be expected.

Another case I've considered, but not thoroughly analyzed, would be the opposite. The server sends a node flagged as collapsed, yet also sends some or all of its descendants as well. This would be a "pre-fetch" optimization to prime the client-side cache. It could be tuned in numerous ways. Some simple examples might be that the client could be configured to always request an extra level, or the server could be configured to proactively send an extra level if it's below a certain number of nodes. I see this as more of a nice to have feature.

If it's absolutely required for you to page through large open trees, one approach is to make the client side ignorant of the tree structure - use ListGrid data paging with startRow and endRow, form the tree structure for the requested rows server-side and send back information in the rows that will allow you to render correct indentation / open/close controls, etc

I'm hoping for adaptive sorting, filtering, searching/seeking that would respect the hierarchy client-side when the entire tree is loaded. Would it not be better to try to extend the ResultTree as summarized above?

**Isomorphic** · 13 Jun 2012, 12:23

There are lots of possible divisions of responsibility between the client and server.

The approach we mentioned - making the client ignorant of the tree structure - can work with a stateless server with the cookie approach to keeping track of open nodes which you appear to be already using.

Features such as client-side sort and adaptive filter are in some cases inherently not possible, or in other cases would require reading the ResultTree source and heavily modifying it.

Your other option is to consider Feature Sponsorship - we don't currently have plans to build this mode into the product as our experience has been that even when end users directly ask for pagination through large trees, they don't actually use it, because it's just not a very good UI.

Announcement

Cannot enable pagination on a TreeGrid

Comment

Comment

Comment

Comment

Comment

Comment