ListGrid with no datasource

SmartClient Version: v10.0p_2014-09-30/LGPL Development Only (built 2014-09-30)

Based upon other issues we have been experiencing (see thread http://forums.smartclient.com/showthread.php?p=127059&posted=1#post127059 for an example), it felt like you were really pushing us toward not using a DataSource for our grids where we are not using a remote data set.

I tried this, and have run in to a significant set of issues where it appears to me that the grid functionality is quite different when you do not have a data source. Maybe I am missing something, but here's what I have run into...

1. If I do not have a DataSource associated with the grid, I cannot do updates on the grid by calling ListGrid.updateData() - it exceptions saying this is not supported and I should update the record directly.

2. If I update the record directly (as outlined in item #1), many things behave differently than they do when I have a DataSource and use updateData(). They are:

A. The display does not automatically update. I have dealt with this by getting the record index and then calling ListGrid.refreshRow(). This is unnecessary when using updateData() and a DataSource.

B. If the fields changed include sort fields, the record is not automatically moved to it's correct new position. I have dealt with this by calling ListGrid.resort(). This is unnecessary when using updateData() and a DataSource.

C. The grid does not appear to be paying any attention to filter criteria on the grid at all. It does not appear to apply the filter during the initial setData(), or on any changes to the record. I have not found a way to get the grid to filter records when not using a DataSource. This is automatic when using updateData() and a DataSource.

D. If the fields changed include groupBy fields, the record is not automatically moved to the correct group. I have tried to deal with this by calling ListGrid.setGroupBy(ListGrid.getGroupBy()). This is unnecessary when using updateData() and a DataSource. In addition, regrouping causes expanded groups to be collapsed where updateData() on a DataSource does not. Also, on a large set of data, regrouping the grid seems to take noticeably longer than the automatic repositioning of the modified record when updating on a DataSource.

I've attached sample code that is similar to our application using both techniques.

At this point I don't know if I am doing something wrong when not using a DataSource or not. If I am not, issues C & D above make using the grid without a DataSource unacceptable. But using a DataSource in these scenarios appears to be at odds with what you are recommending we should be doing because of other issues we have been having.

Am I doing something wrong here? If not, please recommend a direction that provides the functionality we need from the grid.

[Isomorphic]

There are 3 relevant modes to consider:

1. ListGrid with DataSource and using a ResultSet (as a result of a call to fetchData())

2. ListGrid with DataSource and using a RecordList passed to setData()

3. ListGrid with no DataSource, using a RecordList.

With mode 3, the ListGrid doesn't know the primaryKey of the records, so there are a number of limitations, including the fact that there is no way to implement updateData() (we don't have a way of figuring out which record you mean).

Since updateData() cannot be used, you then have to tell the grid about the updates you've performed (as you've discovered) and in this scenario, the grid is not able to be as smart / incremental about changes.

You probably wanted mode #2.

Based upon earlier communications with you I thought that if I use a DataSource, then operations like addData, updateData, removeData become asynchronous (which is the source of other issues I have been contacting you about). If I use option #2, will these operations be synchronous or asynchronous?

[Isomorphic]

Synchronously. See also listGrid.saveLocally.

OK. Is the saveLocally=true necessary if I've already set clientOnly on the data source?

[Isomorphic]

Yes. Again a clientOnly="true" DataSource fully simulates a remote DataSource. When you use listGrid.fetchData(), the ListGrid has no idea it's even talking to a clientOnly DataSource - it does exactly the same thing it would do with a SQL DataSource.

To put it another way, clientOnly="true" has no impact on whether you are in mode 1, 2 or 3 as described above.

OK, I thought this made sense and that I understood, but this is not working for me. I modified my code to associate a DataSource with the grid, and then initialize the data calling ListGrid.setData(RecordList).

When I have updates I do the following:
1. Find the existing record using RecordList.find()
2. Create a copy of the record using DataSource.copyRecord()
3. Modify the copy with the changes
4. Update the record using ListGrid.updateData()

When I do this, the record changes and the changes show in the grid, but unlike the case where I am using a remote DataSource, the changes are not automatically adjusting for the changes based upon sort order, filtering, or groupBy. The record stays in exactly the same position in the grid (regardless of sort or groupBy field data changes) and does not filter out/in when the change affects the filter). Also, it does not seem to apply the filter on the initial call to setData() either.

Modified code is attached.

[Isomorphic]

We'll check into whether there's an issue with automatic updates in saveLocally mode.

To enable local filtering when you providing data via setData(), set listGrid.filterLocalData.

Note your test case has a bunch of setWidth()/setHeight() calls with Strings like "150px" - when setting pixel values, pass numbers, not Strings.

Something else to clarify:

Based upon other issues we have been experiencing (see thread http://forums.smartclient.com/showthread.php?p=127059&posted=1#post127059 for an example), it felt like you were really pushing us toward not using a DataSource for our grids where we are not using a remote data set.

Other than a report of getRecordList() returning null (for which we have no test case, and which didn't impact correct usage anyway), we can't see any issues being reported in the thread you linked to.

We'd just like to emphasize that using a DataSource with listGrid.fetchData() and a ResultSet (what we called mode #1 in this thread) is by far the most common way to work with data in typical apps. We certainly did not and do not recommend using the specialized RecordList + saveLocally mode as your default approach to data binding. We brought it up in the narrow context that it allows you to reorder data without the need to handle the details of how the reorder would be permanently saved.

One very important consequence of using a RecordList and saveLocally mode that may not have occurred to you: it turns off propagation of changes to other components bound to the same DataSource. Based on previous threads of yours, you probably depend on this.

1. I appreciate your looking into the automatic update issues I'm seeing.

2. Thanks for the reminder about setting height/width using ints.

3. The issue I was referring to is the complexity in your mode #1 where the addData is asynchronous and while processing it I may have events coming in for the same record ID and I have to realize that this is a pending update on the DataSource and queue them up so the addData and each update completes before the next one is processed.

4. Your assumption from previous posts is incorrect. Most of our cases today have a one-to-one relationship between a DataSource and the component bound to it, so the propagation issue is not a problem for us.

----

The getRecordList() returning null is a little (not majorly) more problematic. We have a service that keeps a DataSource up-to-date (based upon events from a server) independent of being associated with any grid. The main call that I have found to fill the DataSource with the initial data is setCacheData which takes a list of records, not a RecordList. So, later on when a grid is associated with that DataSource, if you try to call getRecordList() you get null. The only call in this case that seems to return anything useful is getOriginalResultSet(). Therefore in this case there is no easy way to programmatically move a record to a new position.

In this same grid we support record reordering via drag and drop, which is working just fine. Based upon the drag & drop working it seems that it is actually possible to move a record up/down in this scenario, I just can't find a way to programmatically accomplish the same thing when the DataSource is initialized in this way.

By the way, I tried passing numbers to the setHeight/setWidth functions and there is no overload that takes an int, only String. I can drop the "px" from the string, but I cannot follow your directions to pass numbers not Strings.

Apparently I am doing it correctly. Here's the information from your own documentation:

Sets the object's width. This width does not include decorations such as border, margin, and padding.

Parameters:
width the object's new width, in CSS units (e.g. "10px", "1em")

[Isomorphic]

About the setWidth() API: looks like you are using a GWT (not SmartGWT) Button, and your usage is correct for that class. Although note that mixing GWT and SmartGWT widgets in a real app is a bad idea for reasons discussed in the FAQ.

Looks like we're finally getting some use case information about what this grid is doing - remember to share such information early and often as we can only provide good advice if we understand the big picture of what you're trying to accomplish.

In a seemingly similar situation (real-time updates), we show the use of setData() with an Array, which is essentially similar to RecordList usage.

But what's odd here is that you mention drag and drop reorder being allowed in the same interface, and your complaints about saveLocally mode talk about filtering and grouping - so what's going on here? If the user reorders things, are those order changes just dropped if they apply filtering?

Finally, just to reiterate a third time - in order to investigate the claimed issue with getRecordList() returning null inappropriately, we need a test case.

Thanks for pointing out the use of GWT in my sample code. Good old Eclipse had been too helpful with its auto complete and I didn't notice that it has pulled the class from the wrong namespace. This has been fixed in my sample code.

As far as giving a complete picture up front, you need to remember that we are both dealing with and trying to describe large complex systems to each other. In situations like this it is very difficult to tell up front what is relevant and what is not. Just like you don't bring up all of the details about how data sources and grids interact based upon all of the possible fetch, client-only, localSave, etc. settings in your first response, I try not to complicate my initial description with a lot of detail (most of which will probably turn out to be irrelevant). There are literally thousands of little things that we do differently in our product grid management than what is in the simple Showcase examples. It is not practical to provide samples to you with all of this complexity because it quickly exceeds your upload limits, and it probably would be a waste of your time anyway since most of the differences probably don't matter. So please realize that I am trying to simplify the conversation by eliminating some of the details when I am not really familiar with your internals and I'm trying to make my best guess as to what will be helpful. It's a two way street, and requires some back and forth to focus the conversation in on what's really important. Written communication is a hard medium to use as a way to do that.

With regard to your sample use of setData. I got the impression from other threads that the correct way to initialize a standalone DataSource (one not associated with any visual components yet) was to build a record list and use DataSource.setCacheDate. There is no setData on this class and that is not what your sample is doing. Your sample is using setData on a DSResponse and initializing the data source with a call to updateCaches using that response object. So just to clarify, are you saying that using updateCaches is preferred to setCacheData?

With regard to drag & drop reorder versus filtering and grouping. We disable the drag/drop reorder when the data is filtered or grouped, but when it is not then we allow the drag & drop. Also, because you've said that more info is better...

One of our grids represent items in a print queue for large industrial printers. These printers have different modes they can operate in. In one mode they operate in a strict order (so the user can drag and drop and reorder the jobs to print in a different order). However, they can also order on other things like the priority of the jobs (so drag and drop doesn't make sense and is not supported in that mode). In both cases we use the same grid, the same data source, but enable and disable certain grid features dynamically (e.g., drag & drop record recording) based upon the mode in which the print device is configured. So "yes" the same interface allows both (just not at the same time).

As I said the getRecordList() returning null is not a major problem (especially since the direction/changes we are making based upon these conversions cause the issue to go away). Although I can still see it happening running in our full product, I cannot repro it in a small sample. Given that the other changes will probably make this a moot point, and since it is not in the scope of our support contract to send you our whole application, it's not worth my time to try to narrow down what we are doing differently between the small sample code and our full blown app. So I suggest we just forget about this issue.

[Isomorphic]

We don't need a complete picture and didn't ask for one. We just need some basic information about your use case. In this case, what was important to know is why you need to allow reordering even though that's clearly inconsistent with features like filtering and grouping. And the answer was just: the grid has two modes.

However, what's unexplained now is: presumably when the user is looking at manually ordered print jobs, and changing the order, you are indeed working with a remote dataset and hence should be using DataSources in the most common way (mode 1), and the implementation needs to persist the changes with an asynchronous server call, which jumps us aaaaallll the way back to our first response to your prior thread:

When working with a DataSource, the only meaningful way to change order of records is to have a field that tracks the order, sort by that field, and modify that field when the user reorders records. This is demonstrated here.

This seems to still be the right advice.

You've said something about tricky issues with order-of-operations, but if you need to persist end-user reorders to a server, that's an asynchronous operation, additions to the print queue also come in asynchronously, and a change of data binding approach can't simplify the problem away - it's inherent to the application.

Finally, that Messaging sample we linked to *used to* use ListGrid.setData() in prior versions and was switched over in order to demonstrate updateCaches(). Sorry for pointing out the wrong sample code, but no, nothing about that sample should be taken as reversing any previously given advice or the advice in the docs, which is all correct.

From what I understand of mode #1, our situation does not fit that model very well. The information displayed in the grid comes from a single server call that contains a lot of information other than what is in the grid, it is retrieved long before any choice is made to display a view that includes this information in a grid, and the remote calls to update the data are not record based. This is why we are managing the remoting aspects of this separately and attempting to build the DataSource locally in the client without the reomting. Now maybe your DataSource mode #1 scenario is extensible enough to create a derivation that can handle this type of interaction, but if it is I would need to have it explained to me how to accomplish this.

Our app has hundreds of grids. The use cases vary significantly. We are trying to narrow down to a few patterns that can handle all of these variations so that we can write common routines for working with the grids. When we discuss changes/options for a use case I am presenting to you, I also have to keep in mind whether these changes will negatively impact other use cases in our app that are using the same classes and already working for us (hence my questions sometimes seem inconsistent to you). I really do not want to take the time to explain all of the variations in our application and I don't believe that this should be necessary in order to make progress here. Our app is not limited to a single grid with two modes. That would be an extreme oversimplification.

I believe that you are incorrect that changing the binding approach can't simplify the problem. Since the browser environment is inherently single threaded. If I want to check if a record already exists and if not add it (and if so update it), and I use mode #2, then I can safely find/test/add without concern for an event being handled in the middle of this and finding, testing, and deciding to do and add of the same record because the first add was still in progress when the event got handled (because supposedly in mode #2 the add is synchronous). Because of this, no additional synchronization code is necessary. If I use mode #1, I cannot safely find/test/add because while the add is in progress an event can get handled and do the find/test as well and also think it needs to add and end up with two or more add and get duplicate key errors (because in mode #1 the add is asynchronous). So, in this case I have to add additional synchronization and queuing of events that must be delayed until some later time to be processed. Therefore, mode #2 is less complex because of the change in the data binding approach.

Finally, even if the example you pointed me to had still used ListGrid.setData(), that would not have been relevant to my use case. As I explained earlier, the DataSource in that case is managed by a background service and is initialized at a point in time when no grid exists. It does not have a ListGrid object to use when initializing the content of the DataSource. That is why the code uses setCacheData (which does not support a RecordList as a parameter type).

Now, can we stop focusing on why I have use cases for mode #2 and get back to the actual issues I am running into using this mode? When in mode #2, an update does not take sort or groupBy into account, and it does not seem to pay attention to filter criteria at all. Mode #1 handles all of these cases. It appears that there are three possibilities:

1. Mode #2 is not expected to support these behaviors,
2. I am doing something wrong in my code that is interfering, or
3. There are defects in the SmartGWT code that are interfering.

I just need to know which of these is true, and if it is #3, I would appreciate a fix.