Loading DataSource using client-side ds.xml for demo without a server

[philfriesen]

I'm trying to load a DataSource using supplyItemClientSide.ds.xml which points to test data /ds/test_data/supplyItem.data.xml. I want to run a demo by launching the html file directly from the file system without using a server. My problem is that the callback in XMLTools.loadXMLSchema(schemaFileName, callback) never fires.

To isolate the problem I wrote a test class with buttons to represent 3 options:
1) Use the DataSource loaded by the server processing the html file specifying a ds.xml file on the server.
2) Use a custom subclass of DataSource that points to test data.xml on the client.
3) Load the DataSource on the client using the ds.xml file on the client.

Option 1 works when running within a server. It doesn't work when launching from the file system. This is as expected.

Option 2 works when running within a server or from the client file system. This is as expected.

Option 3 doesn't work for either environment. The XMLTools.loadXMLSchema never fires.

Am I using a wrong approach? Do I have a configuration problem?

I'm using Jetty for the server from within Eclipse. I set server.properties to use a webRoot value that should work for both environments:

webRoot: C:\eclipse_workspace\FriSc\war

My Java class to test these scenarios is:

import com.smartgwt.client.data.DataSource;
import com.smartgwt.client.data.SchemaSet;
import com.smartgwt.client.data.XMLTools;
import com.smartgwt.client.data.XSDLoadCallback;
import com.smartgwt.client.types.Alignment;
import com.smartgwt.client.util.BooleanCallback;
import com.smartgwt.client.util.SC;
import com.smartgwt.client.widgets.IButton;
import com.smartgwt.client.widgets.Label;
import com.smartgwt.client.widgets.events.ClickEvent;
import com.smartgwt.client.widgets.events.ClickHandler;
import com.smartgwt.client.widgets.grid.ListGrid;
import com.smartgwt.client.widgets.layout.VLayout;
import com.smartgwt.client.widgets.layout.VStack;

/**
 * Why won't DataSource load using XMLTools.loadXMLSchema when launching html directly
 * from local file system?
 * 
 * @author PhilFriesen
 * @creationDate Dec 3, 2009
 */
public class DataSourceLoadingProblem {
    protected String     myDataSourceName = "supplyItem";
    protected DataSource myDataSource     = null;
    protected ListGrid   myListGrid       = null;
    protected VLayout    myContainer      = null;

    public void run() {
        Label label = newLabel("Load DataSource from:");

        // Build button to load DataSource from server for auto-load DataSource specified
        // in html
        Runnable serverHtmlLoader = new Runnable() {
            public void run() {
                myDataSource = DataSource.get(myDataSourceName);
                buildListGrid();
            }
        };
        IButton serverHtmlButton = newButtonWithTitle_dsLoader("Server Autoloads Html",
            serverHtmlLoader);

        // Build button to load DataSource from custom Java class
        Runnable customClassLoader = new Runnable() {
            public void run() {
                myDataSource = SupplyItemXmlDS.getInstance();
                buildListGrid();
            }
        };
        IButton customClassButton = newButtonWithTitle_dsLoader(
            "Custom Java Class Using Client-Side Data", customClassLoader);

        // Build button to load DataSource from ds.xml on client-side using client-side
        // data.
        final String schemaFile = "/ds/" + myDataSourceName + "ClientSide.ds.xml";
        final XSDLoadCallback callback = new XSDLoadCallback() {
            @Override
            public void execute(SchemaSet schemaSet) {
                myDataSource = schemaSet.getSchema(myDataSourceName);
                buildListGrid();
            }
        };
        Runnable clientXmlLoader = new Runnable() {
            public void run() {
                XMLTools.loadXMLSchema(schemaFile, callback);
            }
        };
        IButton clientXmlButton = newButtonWithTitle_dsLoader("Client-Side Data And ds.xml",
            clientXmlLoader);

        // Build button bar.
        VStack buttonBar = new VStack(10);
        buttonBar.addMember(label);
        buttonBar.addMember(serverHtmlButton);
        buttonBar.addMember(customClassButton);
        buttonBar.addMember(clientXmlButton);

        // Build container.
        myContainer = new VLayout(10);
        myContainer.setWidth100();
        myContainer.setHeight100();
        myContainer.addMember(buttonBar);
        myContainer.draw();
    }

    protected IButton newButtonWithTitle_dsLoader(final String title,
        final Runnable dataSourceLoader) {
        IButton result = newButton(title);
        final BooleanCallback callback = new BooleanCallback() {
            public void execute(Boolean value) {
                myDataSource = null;
                dataSourceLoader.run();
            }
        };
        result.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent event) {
                if(myListGrid != null){
                    myContainer.removeMember(myListGrid);
                }

                SC.say("Load DataSource [" + title + "].", callback);
            }
        });
        return result;
    }

    protected void buildListGrid() {
        if(myDataSource != null){
            myListGrid = new ListGrid();
            myListGrid.setDataSource(myDataSource);
            myListGrid.setCanEdit(Boolean.TRUE);
            myListGrid.setHeight100();
            myContainer.addMember(myListGrid);
            myListGrid.fetchData();
        }
        String status = "DataSource was" + (myDataSource == null ? " not" : "")
            + " loaded.";
        SC.say(status);
    }

    protected IButton newButton(String title) {
        IButton result = new IButton(title);
        result.setWidth(calcWidthOf(title));
        return result;
    }

    protected Label newLabel(String title) {
        Label result = new Label(title);
        result.setWidth(calcWidthOf(title));
        result.setHeight(15);
        result.setAlign(Alignment.LEFT);
        return result;
    }

    protected int calcWidthOf(String text) {
        return 15 + (8 * text.length());
    }

}

I copied the SupplyItemXmlDS Java class from your showcase:

import com.smartgwt.client.data.DataSource;
import com.smartgwt.client.data.fields.DataSourceBooleanField;
import com.smartgwt.client.data.fields.DataSourceDateField;
import com.smartgwt.client.data.fields.DataSourceEnumField;
import com.smartgwt.client.data.fields.DataSourceFloatField;
import com.smartgwt.client.data.fields.DataSourceIntegerField;
import com.smartgwt.client.data.fields.DataSourceTextField;
import com.smartgwt.client.widgets.form.validator.FloatPrecisionValidator;
import com.smartgwt.client.widgets.form.validator.FloatRangeValidator;

public class SupplyItemXmlDS extends DataSource {
    private static SupplyItemXmlDS instance = null;  
    
    public static SupplyItemXmlDS getInstance() {  
        if (instance == null) {  
            instance = new SupplyItemXmlDS("supplyItemDS");  
        }  
        return instance;  
    }  
  
    public SupplyItemXmlDS(String id) {  
  
        setID(id);  
        setRecordXPath("/List/supplyItem");  
        DataSourceIntegerField pkField = new DataSourceIntegerField("itemID");  
        pkField.setHidden(true);  
        pkField.setPrimaryKey(true);  
  
        DataSourceTextField itemNameField = new DataSourceTextField("itemName", "Item Name", 128, true);  
        DataSourceTextField skuField = new DataSourceTextField("SKU", "SKU", 10, true);  
  
        DataSourceTextField descriptionField = new DataSourceTextField("description", "Description", 2000);  
        DataSourceTextField categoryField = new DataSourceTextField("category", "Category", 128, true);  
        categoryField.setForeignKey("supplyCategoryDS.categoryName");  
  
        DataSourceEnumField unitsField = new DataSourceEnumField("units", "Units", 5);  
        unitsField.setValueMap("Roll", "Ea", "Pkt", "Set", "Tube", "Pad", "Ream", "Tin", "Bag", "Ctn", "Box");  
  
        DataSourceFloatField unitCostField = new DataSourceFloatField("unitCost", "Unit Cost", 5);  
        FloatRangeValidator rangeValidator = new FloatRangeValidator();  
        rangeValidator.setMin(0);  
        rangeValidator.setErrorMessage("Please enter a valid (positive) cost");  
  
        FloatPrecisionValidator precisionValidator = new FloatPrecisionValidator();  
        precisionValidator.setPrecision(2);  
        precisionValidator.setErrorMessage("The maximum allowed precision is 2");  
  
        unitCostField.setValidators(rangeValidator, precisionValidator);  
  
        DataSourceBooleanField inStockField = new DataSourceBooleanField("inStock", "In Stock");  
  
        DataSourceDateField nextShipmentField = new DataSourceDateField("nextShipment", "Next Shipment");  
  
        setFields(pkField, itemNameField, skuField, descriptionField, categoryField, unitsField,  
                  unitCostField, inStockField, nextShipmentField);  
  
        setDataURL("ds/test_data/supplyItem.data.xml");  
        setClientOnly(true);          
    }  
}

The server version of the DataSource is supplyItem.ds.xml:

<DataSource
    ID="supplyItem"
    serverType="sql"
    tableName="supplyItem"
    titleField="itemName"
    clientOnly="false"
    dataFormat="xml"
    testFileName="/ds/test_data/supplyItem.data.xml"
    dbImportFileName="/examples/shared/ds/test_data/supplyItemLarge.data.xml"
>
    <fields>
        <field name="itemID"      type="sequence" hidden="true"       primaryKey="true"/>
        <field name="itemName"    type="text"     title="Item"        length="128"       required="true"/>
        <field name="SKU"         type="text"     title="SKU"         length="10"        required="true"/>
        <field name="description" type="text"     title="Description" length="2000"/>
        <field name="category"    type="text"     title="Category"    length="128"       required="true"
               foreignKey="supplyCategory.categoryName"/>
        <field name="units"       type="enum"     title="Units"       length="5">
            <valueMap>
                <value>Roll</value>
                <value>Ea</value>
                <value>Pkt</value>
                <value>Set</value>
                <value>Tube</value>
                <value>Pad</value>
                <value>Ream</value>
                <value>Tin</value>
                <value>Bag</value>
                <value>Ctn</value>
                <value>Box</value>
            </valueMap>
        </field>
        <field name="unitCost"    type="float"    title="Unit Cost"   required="true">
            <validators>
                <validator type="floatRange" min="0" errorMessage="Please enter a valid (positive) cost"/>
                <validator type="floatPrecision" precision="2" errorMessage="The maximum allowed precision is 2"/>
            </validators>
        </field>
        <field name="inStock"   type="boolean"  title="In Stock"/>
        <field name="nextShipment"  type="date" title="Next Shipment"/>
    </fields>
</DataSource>

The client version of the DataSource is supplyItemClientSide.ds.xml:

<DataSource
    ID="supplyItemClientSide"
    serverType="sql"
    tableName="supplyItem"
    titleField="itemName"
    clientOnly="true"
    dataFormat="xml"
    dataURL="/ds/test_data/supplyItem.data.xml"
    testFileName="/ds/test_data/supplyItem.data.xml"
    dbImportFileName="/examples/shared/ds/test_data/supplyItemLarge.data.xml"
>
    <fields>
        <field name="itemID"      type="sequence" hidden="true"       primaryKey="true"/>
        <field name="itemName"    type="text"     title="Item"        length="128"       required="true"/>
        <field name="SKU"         type="text"     title="SKU"         length="10"        required="true"/>
        <field name="description" type="text"     title="Description" length="2000"/>
        <field name="category"    type="text"     title="Category"    length="128"       required="true"
               foreignKey="supplyCategory.categoryName"/>
        <field name="units"       type="enum"     title="Units"       length="5">
            <valueMap>
                <value>Roll</value>
                <value>Ea</value>
                <value>Pkt</value>
                <value>Set</value>
                <value>Tube</value>
                <value>Pad</value>
                <value>Ream</value>
                <value>Tin</value>
                <value>Bag</value>
                <value>Ctn</value>
                <value>Box</value>
            </valueMap>
        </field>
        <field name="unitCost"    type="float"    title="Unit Cost"   required="true">
            <validators>
                <validator type="floatRange" min="0" errorMessage="Please enter a valid (positive) cost"/>
                <validator type="floatPrecision" precision="2" errorMessage="The maximum allowed precision is 2"/>
            </validators>
        </field>
        <field name="inStock"   type="boolean"  title="In Stock"/>
        <field name="nextShipment"  type="date" title="Next Shipment"/>
    </fields>
</DataSource>

The only differences in these 2 xml files are the ID, clientOnly, and dataURL attributes.

From the isc console, I see that the DataSource schema on the client is parsed, even though the callback is never fired.

16:49:40.770:TMR8:INFO:xmlComm:loading XML from: /ds/supplyItemClientSide.ds.xml
16:49:40.770:TMR8:INFO:RPCManager:sendQueue[5]: 1 RPCRequest(s); transport: xmlHttpRequest; target: /ds/supplyItemClientSide.ds.xml
16:49:40.785:XRP0:INFO:RPCManager:transaction 5 arrived after 15ms
16:49:40.785:XRP0:INFO:RPCManager:rpcResponse(unstructured) results -->"<DataSource
    ID="supplyItemClientSide"
    serverType="sql"
    tableName="supplyItem"
    titleField="itemName"
    clientOnly="true"
    dataFormat="xml"
    dataURL="/ds/test_data/supplyItem.data.xml"
    testFileName="/ds/test_data/supplyItem.data.xml"
    dbImportFileName="/examples/shared/ds/test_data/supplyItemLarge.data.xml"
>
    <fields>
        <field name="itemID"      type="sequence" hidden="true"       primaryKey="true"/>
        <field name="itemName"    type="text"     title="Item"        length="128"       required="true"/>
        <field name="SKU"         type="text"     title="SKU"         length="10"        required="true"/>
        <field name="description" type="text"     title="Description" length="2000"/>
        <field name="category"    type="text"     title="Category"    length="128"       required="true"
               foreignKey="supplyCategory.categoryName"/>
        <field name="units"       type="enum"     title="Units"       length="5">
            <valueMap>
                <value>Roll</value>
                <value>Ea</value>
                <value>Pkt</value>
                <value>Set</value>
                <value>Tube</value>
                <value>Pad</value>
                <value>Ream</value>
                <value>Tin</value>
                <value>Bag</value>
                <value>Ctn</value>
                <value>Box</value>
            </valueMap>
        </field>
        <field name="unitCost"    type="float"    title="Unit Cost"   required="true">
            <validators>
                <validator type="floatRange" min="0" errorMessage="Please enter a valid (positive) cost"/>
                <validator type="floatPrecision" precision="2" errorMessage="The maximum allowed precision is 2"/>
            </validators>
        </field>
        <field name="inStock"   type="boolean"  title="In Stock"/>
        <field name="nextShipment"  type="date" title="Next Shipment"/>
    </fields>
</DataSource>
"<--
16:49:40.801:XRP0:INFO:xmlComm:XML reply with text: "<DataSource\n    ID="supplyItemClientSide..."[1983]
16:49:40.801:XRP0:INFO:xmlComm:transforming schema: [XMLDoc <DataSource>] with translator [XMLDoc <xsl:stylesheet>]
16:49:44.004:TMR4:INFO:IButton:isc_IButton_9:call to deprecated setClassName() property - use setStyleName() instead
16:49:44.754:BLR8:INFO:IButton:isc_OID_3:call to deprecated setClassName() property - use setStyleName() instead

What am I missing? Thanks in advance.

[Isomorphic]

loadXMLSchema() is actually for loading XML Schema, as in, the schema definition language files found within WSDL files. It's not expecting SmartClient DataSources.

Since you're already looking at maintaining a separate client-side .ds.xml, the quickest approach is probably just to capture the JavaScript output the server produces from your server-side .ds.xml, save that as a JavaScript file, and load that instead for the serverless demo case.

[philfriesen]

I don't understand exactly what you mean in your reply.

How do I instantiate a Java DataSource object that gets all its state by reading the /war/ds/supplyItemClientSide.ds.xml file when I am in server-less mode?

If I subclass DataSource on the client-side and set its dataURL property to "ds/test_data/supplyItem.data.xml", your client code reads that xml file. Do you expose that functionality on the client side so I can take advantage of it? That way I don't have to create Java classes for every possible DataSource I want to demo in order to define their fields.

[Isomorphic]

Hi Phil,

Yes, a clientOnly DataSource, if given a dataURL, does a one-time fetch against that URL and reads the same .xml format that is used in .data.xml files.

As far as the best way to do a serverless demo, not sure if you got our recommendation or not - basically load the version of your app that *does* use a server, and capture the JavaScript DataSource definitions being emitted by the server side. Basically this is the HTTP response of the DataSourceLoader servlet, which you can capture via Firebug.

Then take those responses and save them as .js files that you can load via a simple <script src=> tag. Tweak them as necessary to point to static sources of data that work without a server (such as a .data.xml, as mentioned above). This is much simpler/better than the approach you seemed to be headed down, of trying to find a way to use a .ds.xml file without the server.

[philfriesen]

Thanks for the reply. I saw your earlier response about copying and pasting to create .js files. I don't know the steps to do that, but even if I took that approach, it's still not the approach I want to take.

I'd like to create a generic DataSource subclass that takes a URL for the ds.xml file (say /ds/supplyItem.ds.xml) in its constructor. In the constructor I want it to find and parse the ds.xml file so I can set all the DataSource attributes and DataSourceField attributes dynamically to the values just read from the ds.xml file.

I know you have the ability to read and parse xml files in server-less mode because that's what you do to load client-side data. I just want to apply the same technique to loading the DataSource before I ask it to load client-side data.

This seems simpler to me and is Java-centric instead of being JavaScript-centric.

Does this make sense?

[Isomorphic]

Hi Phil,

You could certainly do it that way - for example define a DataSource that itself expresses the structure of a .ds.xml file, use the XML databinding approaches you see in samples to take that data and create a DataSource via new DataSource(), new DataSourceField(), etc. This will take you perhaps a half day. You will end up with a DataSource class that asynchronously constructs itself from data retrieved over the network, so you will need to be careful about when it's actually done loading and ready for use.

Or you could cut and paste the generated JavaScript and be done in a few minutes and not have this drawback..

Not really sure why you would go with the former approach :)