Adding fields to datasource at runtime

[jay.l.fisher]

I have a data source definition in XML to which I would like to add a set of additional fields at runtime. I use DataSource ds = DataSource.getDataSource("myDS") followed by ds.addField() but I get an error saying "fields cannot be added to a datasource after the underlying component has been created". How do I load the base definition and then add fields to it. Or do I have to create the entire data source in code?

[Isomorphic]

Hi Jay,

It's not clear what either client-side or server-side components would be expected to do about a DataSource changing fields on the fly - it creates many very ambiguous cases. Generally, you create a new DataSource if you are working with a dynamic set of fields, or you set fields on the component only (no DataSource).

If you can explain the specifics of your use case, we can suggest the best approach.

[jay.l.fisher]

I have a legacy SQL table with a large number of fields, many of which may not be used, depending on the way the legacy app has been configured. There is a basic set of fields that is always used which I have defined in my datasource XML. When the SmartGWT app loads I want to interrogate the legacy app config (just another couple of SQL tables) and, based on what I find there, add any additional fields to the data source.

I have already coded a method that reads the config data and adds the fields which then got the runtime error about "can't add fields after component has been created". So then I tried creating a new data source in code followed by a call my method that adds the optional fields but I'm still getting the same error. If you could please provide a small example of how to create a data source in code and add fields to it I would appreciate it. Although I would like to use the datasource XML I've already created I would re-code it in java if necessary.

[Isomorphic]

Hi Jay,

If you want the new fields to actually imply access to further SQL columns, you have to generate the DataSource dynamically server-side, otherwise, you'd have a security violation (GWT-translated Java code that runs client side should not allow access to new DataSource fields).

To generate a DataSource dynamically server-side, create the XML on the fly on the server and pass it to DataSource.fromXML(). You must do this in the server-side processing chain *before* the DataSource would need to be accessed for the first time, that is, you'll need to create your own servlet that creates the RPCManager class, calls getRequests(), and iterates through them.

Since you have a core, static set of fields, you might consider having a partial DataSource saved to disk, loading it, and generating XML for additional fields on the fly. Use whatever you're most familiar with - XSLT, DOM manipulation, XStream, Velocity templates, or just straight XML String generation.

[jay.l.fisher]

Is there an example of such a thing being done somewhere? I'm finally getting my head around the client side stuff and haven't a clue how to do what you've suggested. I can take care of the XML generation if you could show me how to "create your own servlet that creates the RPCManager class, calls getRequests(), and iterates through them". I'm also not clear on what you mean by "do this in the server-side processing chain *before* the DataSource would need to be accessed for the first time".

[Isomorphic]

This overview should help de-mystify the server-side processing chain. The simplest way to do what you want is to subclass the IDACall built-in servlet and override handleDSRequest. In this method, generate the XML DataSource definition dynamically and call DataSource.fromXML(). You can then take the DSRequest and pass it to DataSource.execute().

[jay.l.fisher]

Please bear with me. I'm new to most of this, but I think I'm on the right path. To start with I thought I would simply return the unmodified datasource XML I already have. Once that is working I can figure out how to modify it before sending it back. Here's the code I've just written. Does it look right? If so, how do I use it in my client code?

package com.islandpacific.gui.server.customDataSource;

import java.io.FileReader;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.datasource.DataSource;
import com.isomorphic.rpc.RPCManager;
import com.isomorphic.servlet.IDACall;
import com.isomorphic.servlet.RequestContext;

public class ItemMasterDS extends IDACall {
	/**
	 * 
	 */
	private static final long serialVersionUID = -1483088157905503631L;

	public DSResponse handleDSRequest(DSRequest dsRequest,
            RPCManager rpc,
            RequestContext context)
     throws java.lang.Exception {
		DSResponse resp = new DSResponse();
		DataSource ds = DataSource.fromXML(new FileReader("IPItemMasterDS.ds.xml"));
		resp = ds.execute(dsRequest);
		return resp;
	}
}

[Isomorphic]

Right, now just go into web.xml and replace the IDACall servlet registration with this custom servlet (at the same path).

[jay.l.fisher]

So far so good, except that I get a NullPointerException when trying to ListGrid.setDataSource to the data source in question. I haven't changed my original client side code. It just does this.

  private final DataSource ipItemMasterDS = DataSource.getDataSource("IPItemMasterDS");

and later ...

  itemGrid.setDataSource(ipItemMasterDS);

It also seems to me that this would mean that ALL ds requests would get back this one data source. Don't I need to condition the logic so that it only applies to the one ds I want to override. Is this the right way to accomplish that?

public DSResponse handleDSRequest(DSRequest dsRequest,
        RPCManager rpc,
        RequestContext context)
 throws java.lang.Exception {
	if (dsRequest.getDataSourceName().equals("IPItemMasterDS")) {
		DSResponse resp = new DSResponse();
		DataSource ds = DataSource.fromXML(new FileReader("ds/IPItemMasterDS.ds.xml"));
		resp = ds.execute(dsRequest);
		return resp;
	}
	else {
		return super.handleDSRequest(dsRequest, rpc, context);
	}
}

[Isomorphic]

If your DataSource file remains at the same location and is in the same format, nothing changes about client-side behavior, but the client will only see the fields declared in the actual file.

To make the client see the dynamic fields, create a .jsp or servlet that produces the same thing the DataSourceLoader servlet produces (JavaScript DataSource definitions) by again calling DataSource.fromXML() with dynamic XML and now also using JSTranslater.toJS() to transform the dynamic DataSource to JS (which the JSP or servlet should stream back as its HTTP response). Then just point a <script src=> tag at the URL of this servlet/.jsp

[jay.l.fisher]

I've made more progress and am sure that I'm executing my new code on the server side, but it seems that the IDACall.handleDSRequest method is only called when requesting the actual data from the data source, not the data source definition. Is IDACall the right servlet to be subclassing and if so, am I overriding the correct method?

[jay.l.fisher]

If I do that, do I even need to subclass IDACall. It seems to me that once the DataSourceLoader loads the modified datasource definition the rest of the DSRequest calls should work without needing to be overridden.

[Isomorphic]

No; DataSource.fromXML() is not dynamically registering a DataSource or anything similar. Once the .jsp / servlet has completed the response to the client, the server still does not know anything about your dynamic DataSource.

[jay.l.fisher]

Success!! Thanks for your help.

[rathiandi]

Success!! Thanks for your help.

Hi Jay and Isomorphic,

I am also dealing with same problem but unlike Jay i have to generate data source based on few parameters coming from client every time.

Jay - Can you please post your client side integration code?

Thanks,