Dynamic DataSource/Dynamic Columns in a ListGrid

[ryan.hollom]

Greetings,

I'm in the evaluation phase of SmartGWT for an enterprise application. There is a requirement for an excel-like editor where the columns are dynamic based on the "resource" that comes from the server. The resource contains both the metadata (the column info) and the actual data (the rows). These resources could have tens of thousands of rows, which makes ListGrid attractive as a potential solution.

However, it seems that all of the discussions I have found require a more "static" datasource that is either pre-defined in xml, or is similarly "pre-defined" programmatically in java.

I am looking for a way to essentially create a ListGrid with dynamic columns. Moreover, I need the ability for the user to add and remove columns as well as "rename" existing ones. I've not found any existing example for this type of use case, nor have I found an elegant solution to allow a dynamic, "editable" data source.

Can anyone suggest a potential solution or point me to a relevant example for this? Or is this something that SmartGWT is not designed to handle? Any and all help would be greatly appreciated!

Thanks,
Ryan

[Isomorphic]

Have you seen DataSource.addDynamicDSGenerator()? Server-side DataSources can be generated dynamically based on totally arbitrary Java code that accesses databases, web services or whatever data in order to form the DataSource.

The system is entirely dynamic.

[ryan.hollom]

No, I had not seen that API. It appears that it is in the (yet to be released) 2.4 version, correct? I will try to download the latest nightly build and give it a try.

Thanks!
-Ryan

[Isomorphic]

That's correct. The capability has been there a long time (via DataSource.fromXML()) but the new API makes it more convenient. We'd recommend going with a 2.4 nightly regardless (lots of new docs, especially the QuickStart Guide).

[ryan.hollom]

Thanks for the info. After reading the documentation, there are still a couple of things that are unclear to me.

1) How is the server side DataSource tied to a client side UI widget, say, a ListGrid? Is the process to still create a myCustomDS.ds.xml file (with no fields), and then register a DynamicDSGenerator with the prefix "myCustomDS", and then set that as the data source for the ListGrid?

2) Is the DynamicDSGenerator/server side DataSource responsible for creating the DSFields and setting the individual properties of these fields? Or is it typically better to dynamically build the data source xml and call DataSource.fromXML?

3) Where should one register these dynamic DS generators, as it cannot be done from the client side?

Thanks again!
-Ryan

[Isomorphic]

1 & 2) the DynamicDSGenerator allows you to provide a DataSource instance, which you get by calling DataSource.fromXML(). There's no static XML file required.

3) generally, in application startup code (eg in a servlet that has <load-on-startup>).

[ryan.hollom]

Ok, if I am understanding correctly, it should be sufficient to simply create an empty DataSource on the client side:

DataSource customDS = new DataSource();
customDS.setID("myPrefix_suffix");
ListGrid grid = new ListGrid();
grid.setDataSource(customDS);

When I do so, I get an error/warning stating that the data source does not have any fields:
16:46:50.405 [ERROR] [webstudio] 16:46:50.397:MUP7:WARN:ListGrid:isc_DecisionTableDataGrid_0:ListGrid.setFields() : neither this ListGrid nor its dataSource have fields

However, I do see that my DynamicDSGenerator is getting called, and the server side DataSource is getting created. I am extending BasicDataSource in this case, and can see that the executeFetch method is getting called. In this method, if I call getFields(), I see that there are the 4 dynamic fields (for this example).

However, on the UI, I see a ListGrid with no headers. I can see the blank rows get created (there is the "remove" button for each row), but I do not see the data.

If I create a static data source (.ds.xml) with the exact same structure as the dynamic one, everything seems to work fine.

Am I missing a step here? Thanks!

[smartgwt.dev]

Use DataSource.get("myPrefix_suffix")

[ryan.hollom]

DataSource.get("myPrefix_suffix") returns null, and does not call my DynamicDSGenerator. That is why I was wondering whether a static .ds.xml file was still required, or simply creating a new DataSource() and setting the appropriate ID would suffice.

[Isomorphic]

Loading a dynamic server-side DataSource works identically to loading a server-side DataSource based on a .ds.xml file - use the DataSourceLoader servlet in your .html bootstrap file, and call DataSource.get() to get the DataSource instance.

[ryan.hollom]

If I try to use the DataSourceLoader servlet as follows in my bootstrap html file:

<script src="webstudio/sc/DataSourceLoader?dataSource=myPrefix_suffix"></script>

Then I get a JspException upon startup:

=== 2010-12-22 21:47:10,796 [l0-2] ERROR DataSourceLoader - Exception while attempting to load a DataSource
javax.servlet.jsp.JspException: Unable to load DataSource for ID: myPrefix_suffix
at com.isomorphic.servlet.DataSourceLoader.processRequest(DataSourceLoader.java:83)
at com.isomorphic.servlet.DataSourceLoader.doGet(DataSourceLoader.java:64)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)

Further, if I must specify the ID of all of the DataSources in the html file first, then this limits the dynamic nature of the DynamicDSGenerator. In my case, there could be 0 datasources, or there could be hundreds. It depends on the user, the user's project files, etc. Does the DynamicDSGenerator address this use case as well?

Thanks!
-Ryan

[ryan.hollom]

Ok, I now see that the issue was that I did not first register my dynamic DS generator in a startup servlet, so I can now see my dynamic ds generator getting called. I no longer see the JspException, however now if I call

DataSource.get("myPrefix_suffix")

it returns null.

In addition, there is still the issue where not all datasources can be specified in html beforehand, and in fact the dynamic data source creation requires an additional parameter (a project resource) in order to create it. Is there a way to do this dynamic creation lazily, and pass it the additional parameter? I had hoped that I could suffix the data source id with the necessary information, since the dynamic DS generator works on a prefix, but having to specify the datasource in the bootstrap html file does not allow for this.

[ryan.hollom]

Ok, another update. I had hard-coded a different data source ID in my dynamic DS generator, so when I changed that to "myPrefix_suffix", then I got the proper DataSource from DataSource.get("myPrefix_suffix") and everything is behaving as expected.

However, the concerns from my previous post still apply. Is there a way to lazily create these data sources, potentially 100s of them (with different IDs), and pass additional parameters to the dynamic DS generator during the creation (presumably in the DSRequest)?

The only other requirement that I have is to allow a modification (or recreation, if necessary) of the data source after the initial one is created. In my case, the user can create a new column in the table, which of course would change the data source. Is there a way to recreate/reload the data source as well?

If these issues can be addressed, then certainly it looks like the Pro+ edition will meet our requirements.

Thanks again,
Ryan

[Isomorphic]

You can call DataSource.load() at any time (this just uses the DataSourceLoader servlet).

If you change the definition of the DataSource, it's best to just change it's ID (that is, effectively create a new DataSource with a different ID, and destroy() the old one). All existing DataBoundComponents that were using the old DataSource will need to be bound to the new one via setDataSource().

If the DataSource definition continues to be stored under the same id wherever you're storing DataSource metadata, you can use some naming convention such as baseDataSourceId_version_1 in the ID you provide to SmartGWT.

[ryan.hollom]

Indeed, DataSource.load was the final piece for which I was looking. Using that, I am now able to dynamically (and lazily) create a new DataSource and assign it to a ListGrid. Everything is behaving as expected.

Thanks so much for your help!

-Ryan