Please help - One XML, nested Data Sources

[jaredm]

Hi,

I've been racking my head over this all weekend so I need some help from Isomorphic. I've looked at your samples and discussions and I can't find a single example of actually using an XML datasource that contains 1 record and contains a nested structure (built from XSDs) without having any primary key/foreign key relations.

There's lots in the documentation about how SmartClient builds the DataSources but I've struggled to find any documentation on how to use it.

- SmartClient version - 8.0 nightly from last week.
- No server-side problems
- No client-side errors

First my use-case: I have one XML document that is described by an XSD and it contains a list of ComplexTypes. The XML document is locked to the user and the user can and will only be able to ever look at that one document in the UI. I write the XML document out into a page variable and want the user to be able to manipulate it and then I want to be able to post back the updated XML to the server.

I have produced a superficial test case: My root document is of the type Author and nested type is Book:

XSD:

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://localhost:19086/schema/" targetNamespace="http://localhost:19086/schema/" elementFormDefault="qualified" attributeFormDefault="unqualified">
  <xs:complexType name="Author">
    <xs:sequence>
      <xs:element name="name" type="xs:string" minOccurs="0" maxOccurs="1" nillable="true" />
      <xs:element name="telephoneNumber" type="xs:string" minOccurs="0" maxOccurs="1" nillable="true" />
      <xs:element name="dateOfBirth" type="xs:dateTime" minOccurs="0" maxOccurs="1" nillable="true" />
      <xs:element name="lastRoyalty" type="xs:double" minOccurs="0" maxOccurs="1" nillable="true" />
      <xs:element name="books" type="tns:ArrayOfBook" minOccurs="0" maxOccurs="1" nillable="true" />
    </xs:sequence>
  </xs:complexType>
  <xs:complexType name="Book">
    <xs:sequence>
      <xs:element name="name" type="xs:string" minOccurs="0" maxOccurs="1" nillable="true" />
      <xs:element name="isbn" type="xs:string" minOccurs="0" maxOccurs="1" nillable="true" />
    </xs:sequence>
  </xs:complexType>
  <xs:complexType name="ArrayOfBook">
    <xs:sequence>
      <xs:element name="Book" type="tns:Book" minOccurs="0" maxOccurs="unbounded" nillable="true" />
    </xs:sequence>
  </xs:complexType>
  <xs:element name="Author" type="tns:Author" />
</xs:schema>

Some valid example XML:

<Author xmlns="http://localhost:19086/schema/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<name>A Jones</name>
<telephoneNumber>0208 123 4321</telephoneNumber>
<dateOfBirth>2011-01-31T13:56:27+00:00</dateOfBirth>
<lastRoyalty>2531.12</lastRoyalty>
<books>
    <Book>
        <name>Another Reality</name>
        <isbn>010332X</isbn>
    </Book>
    <Book>
        <name>First Sight</name>
        <isbn>928421Z</isbn>
    </Book>
</books>
</Author>

My JS code:

isc.DynamicForm.create({
      ID: "dfAuthor"
    , useAllDataSourceFields: true
    , position: "relative"
    , showComplexFields: false
    , autoDraw: false
});

isc.ListGrid.create({
      ID: "lgBook"
    , useAllDataSourceFields: true
    , position: "relative"
    , autoDraw: false
});

var localData = {};
localData.author = "<Author xmlns=\"http://localhost:19086/schema/\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"><name>A Jones</name><telephoneNumber>0208 123 4321</telephoneNumber><dateOfBirth>2011-01-31T11:21:45+00:00</dateOfBirth><lastRoyalty>2531.12</lastRoyalty><books><Book><name>Another Reality</name><isbn>010332X</isbn></Book></books></Author>";

isc.XMLTools.loadXMLSchema('/webapi/ViewSchema.jsp?type=Author&version=2064.c53ee320-f8c2-4217-bace-7aa84478aea8T', schemaLoaded, {}, true);


function schemaLoaded(schemaSet) {
        var xmlDoc = isc.xml.parseXML(localData.author);
        var set = isc.SchemaSet.get('http://localhost:19086/schema/');
        var ds = isc.DataSource.get('Author');
        ds.setCacheAllData(true);
        var records = ds.recordsFromXML(new Array(xmlDoc));
        ds.setCacheData(records);
        dfAuthor.setDataSource(ds);
        dfAuthor.fetchData();
        dfAuthor.draw();

        // Help me here
        var dsBook = isc.DataSource.get('Book');
        lgBook.setDataSource(dsBook);
        lgBook.setData(records[0].books);
        lgBook.draw();
}

In the above code the DynamicForm is populated just fine and the ListGrid shows the correct columns but no data.

When I have one Xml document (which seems to map to one record in a Data Source), how do I set things up so I can display a nested structure in a ListGrid?

Note: The code in schemaLoaded is the result of trying lots of permutations to get this working. I know it shouldn't work (since '.books' contains the type 'ArrayOfBook') but I don't know how to make it work.

[Isomorphic]

What do you mean by "display a nested structure in a ListGrid"? What visual appearance do you want?

You should find the nested records stored under the "books" property of the Author record, as an Array of Object (like any set of Records). Form here you could implement a formatCellValue that uses this data, implement record expansion with an override of getExpansionComponent, etc.

[jaredm]

Hi Isomorphic, I realise now reading that back that I chose very ambiguous words.

The ListGrid will be a 'regular' list grid, it's just that I'm not sure how to get it to display any data.

From my simple example I'd like to have a List Grid that has 'isbn' and 'name' columns and some data. What I'm finding though is that while the column headings display, no data is shown.

[Isomorphic]

Inspect what's in your "records" variable (result of recordsFromXML) using echo()/echoAll(). After the recordsFromXML call, it's an ordinary JavaScript object and will work identically to any other JavaScript object you might pass to setData(), no XML schema involved. So your problem is likely to be a mundane mismatch between column names and property names in the data, or an extra level of nesting in the data object, something like that.

[jaredm]

Hi,

echoAll on the records object shows:

[
{xmlns: "http://localhost:19086/schema/",
name: "A Jones",
telephoneNumber: "0208 123 4321",
dateOfBirth: Date(01/31/2011),
lastRoyalty: 2531.12,
books: Obj}
]

the JSON encoder is more useful:

[
    {
        "xmlns":"http://localhost:19086/schema/", 
        "name":"A Jones", 
        "telephoneNumber":"0208 123 4321", 
        "dateOfBirth":"2011-01-31T19:30:03", 
        "lastRoyalty":2531.12, 
        "books":{
            "Book":[
                {
                    "name":"Another Reality", 
                    "isbn":"010332X"
                }, 
                {
                    "name":"First Sight", 
                    "isbn":"928421Z"
                }
            ]
        }
    }
]

My real question is what is the best way to get the Book ListGrid to show data?

(I want it so that if the list grid were made editable, the root Author object would have the updated values)

[Isomorphic]

If you're trying to display the books, pass records[0].books.Book as the data (because that's an Array of Objects and records[0].books is a single Object with one property "Book").

Note, not sure if you have some control over the schema, but look at the docs for DataSourceField.multiple - if this property were set, it eliminate that extra level of "Book" in the path to the data. Note in the docs for this property, it mentions the SOAP Array idiom that is usually used in this case to convey that a tag like <books> is a kind of "wrapper tag" for multiple <Book> elements.

[jaredm]

Thank you, especially for the tip - I was able to influence the schema so that it didn't wrap the Book element.

I said earlier that my goal is to get an updated XML document back to the server (via form post, not SOAP call). So for example if someone edits the ISBN in a ListGrid record I want it to update in the Author object. Can you explain if my approach is the most suitable:

First, I set all of my visual widgets to the DataSource of the type they will display (like I did for Book and ListGrid in my example).

Second, when the schemas have loaded I take the DataSource for the root object (e.g. Author) and call setCacheData() with my record and then setCacheAllData(true).

Finally, I go through all my visual widgets and use setData to set their data (e.g. the Books ListGrid gets set to Author.cacheData[0].author.books)

Will this approach achieve what I want? I know its referred to as the disconnected approach but I'm not sure how to make it more connected.

[Isomorphic]

At what granularity are you saving back to the server - every change, or only once a batch of updates are made?

Your strategy (using cacheAllData to create a write-through cache) is the best you can do if the pre-existing server behavior is to provide a nested XML structure as a blob but expect partial updates via form POST.

Note there's a new subsystem which allows components to be bound to a nested structure by a path (eg /Author/Books[0]) which will slightly simplify this in the future.

[jaredm]

Typically we'll save back to the server whenever the user clicks save or when there's an auto-save (every few mins).

You understood my approach well except for one part - I can't do partial updates (as in not a portion of the XML), I can only send back the whole of my root structure. Does that change your advice?

Given that at the moment my client is prototyping and evaluating SmartClient is this new sub-system something that's available in the nightly builds to experiment with?

[Isomorphic]

This new subsystem is partially visible in nightly builds - see the dataPath property on DataBoundComponent. There are a number of known bugs so not necessarily something to get into given that it may not eliminate that much code for you.

Partial vs complete updates doesn't really change the advice - basically you will need to intercept the attempted save in transformRequest, and either prevent it going to the server at all (see eg dataProtocol:"clientCustom") so you can form the request manually (via RPCManager.sendRequest()) or modify the save request so it includes the complete, replacement document (probably generated via DataSource.xmlSerialize()).

[jaredm]

Hi Isomorphic,

I just wanted to let you know I've finished the prototype this weekend very much thanks to your help in this thread. I'll be presenting it this week to my customer and am really hoping they like what they see.

The data path API is fantastic, you're right it has a few quirks but it's been amazingly stable for me so far and has saved writing lots of code.

I've also used the AutoChild pattern and its delightful, I'm surprised it doesn't get more of a mention in your marketing because it makes creating widgets very easy and flexible.

Some nice-to-haves from my experience so far:
- If I specify a ListGrid with a data-path then automatic field definitions aren't retrieved. If I add the name of the DataSource I get a mismatched datasource warning (in the isc Console) but the automatic field definitions are picked up. From experimentation it seems to be safe to ignore the warning in the console.
- Ability to specify a datapath on a SectionStack or SectionStackItem and have that item repeat for as many records as there are at that data-path (repeating sections).
- If summaryTitle could be used for field names when using the hilite editor and formula builder (In my list grid I have some header-spans and some of the field titles are the same). It seems summaryTitle is intended for this case but by default its applied only to the context menu.

I think I know how to work around the points above and they'll only really be needed if the customer decides to go forward from this prototype in which case I hope they'll be purchasing a commercial license.

Thanks again! I continue to be amazed by what this thing offers.