mixing builtin SmartGWT Server backed DataSources with subclass of RestDataSource

[abthere]

Hello,

We’re trying to design an approach that would allow us to seamlessly mix the standard SmartGWT DataSources with a custom subclass of RestDataSource within the same application. The idea is to be able to use SmartGWT server plus client-side data integration pattern in a way where the client is not aware which DataSource is a standard and which is a custom one. We were able to mimic the behavior of the DataSourceLoader by implementing a custom servlet that produces the DataSource config, but how should we call create() on the custom subclass instead of DataSource class? Would that RestDataSource subclass be accessible through the regular DataSource.get() call or we would have to implement a parallel API for REST DataSources?

[Isomorphic]

Could you elaborate on your purpose here? Because you seem to be trying to achieve something which is already the default behavior:

1. UI components are already unaware of whether they are using a .ds.xml-based DataSource, RestDataSource, clientOnly DataSource or other type

2. All .ds.xml DataSources are already available via the RestDataSource protocol, via the built-in RestHandler servlet (see the QuickStart Guide entry on this)

If you have something further in mind, could you be more specific about what it is, and especially what benefits you are hoping for with your approach?

[abthere]

We are forced to implement a client-side data integration with a RESTFul backend that is external to SmartGWT server. That’s why we can’t use RestDataSource directly, but implemented a subclass that is basically configures the operation bindings to support the RESTFul style with different HTTP commands. Everything works if we create the DataSource programmatically. The goal is to be able to feed the custom RESTFul DataSource definitions from the server to the client in the same way SmartGWT DataSourceLoader does. Is it possible to use a class, that is a subclass of RestDataSource, without creating it in GWT (java), but use SmartClient API like isc.DataSource.create() that we can incorporate into a custom DataSourceLoader implementation?

[Blama]

Hi abthere,

I'm not saying that this is the best solution (I assume what you want is already possible with the standard DataSourceLoader and .ds.xml modifications like serverConstructor) but you could load the normal DataSources on startup like you are used to via your bootstrap html file and in your onModuleLoad do this on startup:

1. DataSource.setLoaderURL("yourLoaderURL");
2. DataSource.load(...YourRestDS...)
3. perhaps DataSource.setLoaderURL("defaultURL");

If you are already doing your own implementation of DataSourceLoader, the text it returns to the client in F12 browser tools should just be the same format as for the normal loader. If you have this, you can just add a call to this in your bootstrap html file, as multiple calls to DataSourceLoader are already supported (in case you need to load more DataSources than the normal URL length allows).

But reading your post again, I'm not sure what you want to do.

• Do you have DataSources that go to one backend and other DataSources that go to another (different IP etc)?
• Are all RestDataSources on one server?
• Do you also use normal DataSources that are handled by the SmartGWT serverside RPCManager?

Best regards,
Blama

[abthere]

Blama,

thank you for your help. I appreciate the questions that I should have answered in the first post!

We have 2 types of DataSources:
1. regular RPC DataSource
2. client only RestDataSource that we subclassed to configure operation bindings.

Both SmartGWT server and REST backend are parts of the same application (war file). RPC DataSource definitions are loaded by the client through SmartGWT DataSourceLoader and we also want to load RestDataSource definitions in the same fashion, so a custom DataSourceLoader was introduced. The problem we're facing is how to instruct the client to create RestDataSource using our subclass and the supplied config? The builtin DataSourceLoader simply calls isc.DataSource.create(), but we need to do something like our.custom.RestDataSource.create() instead in a way where the custom configured RestDataSources can be used on the client by calling the same DataSource.get() API. If it's even possible, of course.

[Isomorphic]

So just to make sure we understand: you are using SmartGWT server DataSources (with which connector?) but you have a requirement to control how data is sent from the client to the server specifically?

You also mentioned regular RPC DataSources? What do you mean by this? GWT-RPC? SmartGWT RPC DMI?

[Isomorphic]

Also, if indeed you have a requirement to control client-server communications despite software on both sides being the same, can you specify what's in this requirement? Because the RestDataSource already has settings that would allow you to use different HTTP verbs for example - there would be no need to make up a new protocol just for that requirement.

[abthere]

We have an application that uses the traditional SmartGWT RPC DMI approach. Now, there’s a large suite of applications in the enterprise that we would like to be part of. The enterprise architecture has several constrains that affect us:

1. The application must be able to access the core services using RESTFul protocol (even if it means going through SmartGWT server proxy or adding custom proxies). It can’t be SmartGWT style REST that RestDataSource/RestHandler Servlet support out of the box, but the fully implemented RESTFul protocol between SmartGWT client and the server. It would eliminate the built in support for queuing operations, but it’s a constrain.
2. The DataSource definition can be supplied at run-time by another system e.g. rules engine.

As far as I understand, we need to build a custom DataSourceLoader that can provide DataSource definition to the client at run-time and also being able to instruct SmartGWT server to use a certain serverConstructor to handle the requests. We subclassed RestDataSource on the client to get the operation URLs and data format configured and implemented a custom DataSourceLoader servlet that produces the DataSource config JSON, but how can we create our custom RestDataSource based on the provided config? And, is there a way to provide DataSource definition to the server without having ds.xml file for the DataSources that are using the SmartGWT RPC DMI approach?

[Isomorphic]

The DataSourceLoader protocol is undocumented and subject to change without notice. So you cannot build your own DataSourceLoader, but that's OK, because that approach doesn't make sense anyway - you'd get a client-side DataSource definition, but no server-side functionality for it.

Instead, if you need to create DataSources from external metadata, you could simplify generate the XML files on disk before deploying the app, or if it truly needs to be on the fly, use a DynamicDSGenerator. Docs for this start in the QuickStart Guide, beginning of the Server Framework chapter.

On your REST protocol - you didn't specify how exactly your REST protocol is required to differ from SmartGWT's RestDataSource protocol. If the difference is *solely* HTTP verbs, it makes sense to create a subclass of RestDataSource with different verbs specified via operationBindings. If you need to adjust the protocol in more fundamental ways, don't subclass RestDataSource, start from DataSource (the docs for RestDataSource mention this explicitly).

Keep in mind, one of the reasons strict REST doesn't make sense *aside from* queueing is that the amount of data that needs to be sent in fetch requests can easily exceed the HTTP protocol's URL limit for GET requests (this happens with AdvancedCriteria, for example). This is covered in the RestDataSource docs as well. You might want to take another crack at convincing the "powers that be" that leaving the system as-is is the right approach, since what you're doing right now is performing extra, unnecessary work that is making the system worse, as well as setting yourself up to have to do much, much more unnecessary work in future (when you run into the need to save a multi-record transaction and can't use queuing, or when you hit URL length limits).

You might find success in saying that you're going to use strict REST everywhere you can, but for use cases that strict REST simply doesn't handle, there's no point making up your own solution, which still won't be strict REST, when there's a ready-made, already-working solution in SmartGWT.

[abthere]

Thank you for the excellent points! We added a custom RESTFul backend that talks SmartGWT protocol, but uses GET, PUT, POST and DELETE verbs to satisfy the architectural constraints.
DynamicDSGenerator solves the server side support, but how can we specify that this particular DataSource should be handled by our custom RestDataSource on the client?

[Isomorphic]

Use the "constructor" attribute to specify your client-side class, and ensure that you have read the Reflection overview to make your class reflectable.

[abthere]

Adding the "constructor" attribute to the DataSource definition didn't change the client side logic unfortunately. DataSource operations are still handled by the standard DataSource, not by our subclass. DataSource definition:

 <DataSource
      ID="animals"
      serverType="sql"
      tableName="animals"
      testFileName="animals.data.xml"
      constructor="foo.DefaultDataSource"
  >
      <fields>
          <field name="commonName"      title="Animal"             type="text"/>
          <field name="scientificName"  title="Scientific Name"    type="text"  primaryKey="true"  required="true"/>
      </fields>
  </DataSource>

custom DataSource:

 package [U]foo[/U];
   
  import com.google.gwt.core.client.JavaScriptObject;
  import com.smartgwt.client.data.DSRequest;
  import com.smartgwt.client.data.DSResponse;
  import com.smartgwt.client.data.OperationBinding;
  import com.smartgwt.client.data.RestDataSource;
  import com.smartgwt.client.types.DSDataFormat;
  import com.smartgwt.client.types.DSOperationType;
  import com.smartgwt.client.types.DSProtocol;
  import com.smartgwt.client.util.SC;
  import com.smartgwt.client.bean.BeanFactory;
   
  @BeanFactory.Generate
  public class DefaultDataSource extends RestDataSource {
      
      public interface MyMetaFactory extends BeanFactory.MetaFactory {
          BeanFactory<DefaultDataSource> getDefaultDataSourceFactory();
      }
      
      public DefaultDataSource() {
          SC.logWarn("creating DefaultDataSource()");
          configureDataSource(this);
      }
      
      public DefaultDataSource(JavaScriptObject jsObj){
          super(jsObj);
          
          SC.logWarn("Loading custom DataSource(jsObj)");
      }
      
      public static RestDataSource getOrCreateRef(JavaScriptObject jsObj) {
          SC.logWarn("called getOrCreateRef ");
          RestDataSource restDataSource = RestDataSource.getOrCreateRef(jsObj);
          DefaultDataSource.configureDataSource(restDataSource);
          
          return restDataSource;
      }
      
      protected static void configureDataSource(RestDataSource dataSource) {
          dataSource.setDataFormat(DSDataFormat.JSON);
          dataSource.setDataProtocol(DSProtocol.POSTMESSAGE);
          dataSource.setDataURL("foo/dispatch");
          dataSource.setJsonPrefix("");
          dataSource.setJsonSuffix("");
          
          dataSource.setDisableQueuing(false);
          
          OperationBinding fetch = new OperationBinding();
          fetch.setOperationType(DSOperationType.FETCH);
          fetch.setDataProtocol(DSProtocol.POSTMESSAGE);
          OperationBinding add = new OperationBinding();
          add.setOperationType(DSOperationType.ADD);
          add.setDataProtocol(DSProtocol.POSTMESSAGE);
          OperationBinding update = new OperationBinding();
          update.setOperationType(DSOperationType.UPDATE);
          update.setDataProtocol(DSProtocol.POSTMESSAGE);
          OperationBinding remove = new OperationBinding();
          remove.setOperationType(DSOperationType.REMOVE);
          remove.setDataProtocol(DSProtocol.POSTMESSAGE);
          dataSource.setOperationBindings(fetch, add, update, remove);
      }
  }

registering for reflection:

 public void onModuleLoad() { 
          GWT.create(DefaultDataSource.MyMetaFactory.class);
          [U]BeanFactory[/U] factory = BeanFactory.getFactory(DefaultDataSource.class);
          SC.logWarn("Got factory for " + factory.getClass());

I'm getting the factory back, but DefaultDataSource methods are never called. Is there something else that needs to happen?

[Isomorphic]

We see the problem you describe. The BeanFactory is actually being created, but there's an ordering issue between that and how we load the DataSource. We're investigating the best solution.

[abthere]

thank you for looking into this.

[Isomorphic]

You have more than enough above to enable the BeanFactory for your class. The reason it isn't working is that the BeanFactories are enabled during SGWT's onModuleLoad(), whereas presumably you're loading "animals" using the default mechanism from the sample - the following line in BuiltInDS.html:

 <script src="builtinds/sc/DataSourceLoader?dataSource=supplyItem,animals,employees"></script>

The problem is that that triggers the DataSourceLoader too soon, before onModuleLoad() has run. Instead, what you need to do is remove "animals" from that line and load it from Java, in onModuleLoad(). So you'll need to modify the sample to call DataSource.load(), triggering the original onModuleLoad() as that method's callback:

public class BuiltInDS implements EntryPoint, Function {
        
    public void onModuleLoad() {
        DataSource.load("animals", this);
    }
  
    // we've renamed onModuleLoad() as execute(), preserving the content
    public void execute() {
        KeyIdentifier debugKey = new KeyIdentifier();
        debugKey.setCtrlKey(true);
        debugKey.setKeyName("D");

        Page.registerKey(debugKey, new PageKeyHandler() {
            public void execute(String keyName) {
                SC.showConsole();
            }
        });
               :

If you make that change to the sample, your DefaultDataSource.configureDataSource() method should fire. For more information about DataSource.load(), see our SGWT EE javadocs.

-

One more note - you really just need the @BeanFactory.Generate pragma - the following sections aren't needed:

public interface MyMetaFactory extends BeanFactory.MetaFactory {
    BeanFactory<DefaultDataSource> getDefaultDataSourceFactory();
}

and

GWT.create(DefaultDataSource.MyMetaFactory.class);
BeanFactory factory = BeanFactory.getFactory(DefaultDataSource.class);