6.1p: Severe problem with ValuesManager when a field has the same name as an optionDataSource field of a SelectItem

[Blama]

Hi Isomorphic,

please see this testcase (v11.1p_2018-01-11) where the field name already starts with the wrong data (data in RPC Tab is correct, NAME:"Reseller 1") and also changes after selecting an entry of the SelectItem.
As I needed equal names, I did not use animals or supplyItem/employees, but own tables instead. DDL is included.

IMHO the issue is that the name column of the company table (used for the SelectItem) somehow interferes with the name column of the reseller table (used for the DynamicForm).



BuiltInDS.java:

package com.smartgwt.sample.client;

import com.google.gwt.core.client.EntryPoint;
import com.smartgwt.client.Version;
import com.smartgwt.client.core.KeyIdentifier;
import com.smartgwt.client.data.Criteria;
import com.smartgwt.client.data.DataSource;
import com.smartgwt.client.util.Page;
import com.smartgwt.client.util.PageKeyHandler;
import com.smartgwt.client.util.SC;
import com.smartgwt.client.widgets.IButton;
import com.smartgwt.client.widgets.Window;
import com.smartgwt.client.widgets.events.ClickEvent;
import com.smartgwt.client.widgets.events.ClickHandler;
import com.smartgwt.client.widgets.form.DynamicForm;
import com.smartgwt.client.widgets.form.ValuesManager;
import com.smartgwt.client.widgets.form.fields.SelectItem;
import com.smartgwt.client.widgets.form.fields.TextItem;
import com.smartgwt.client.widgets.layout.VLayout;

public class BuiltInDS implements EntryPoint {
    private VLayout mainLayout;
    private IButton recreateBtn;

    public void onModuleLoad() {
        KeyIdentifier debugKey = new KeyIdentifier();
        debugKey.setCtrlKey(true);
        debugKey.setKeyName("D");

        Page.registerKey(debugKey, new PageKeyHandler() {
            public void execute(String keyName) {
                SC.showConsole();
            }
        });

        mainLayout = new VLayout(20);
        mainLayout.setWidth100();
        mainLayout.setHeight100();

        recreateBtn = new IButton("Recreate");
        recreateBtn.addClickHandler(new ClickHandler() {
            @Override
            public void onClick(ClickEvent event) {
                recreate();
            }
        });
        mainLayout.addMember(recreateBtn);
        recreate();
        mainLayout.draw();
    }

    private void recreate() {
        Window w = new Window();
        w.setWidth("95%");
        w.setHeight("95%");
        w.setMembersMargin(0);
        w.setModalMaskOpacity(70);
        w.setTitle(" (" + Version.getVersion() + "/" + Version.getSCVersionNumber() + ")");
        w.setTitle("ValuesManager causes SelectItem selection to change TextItem as well." + w.getTitle());
        w.setShowMinimizeButton(false);
        w.setIsModal(true);
        w.setShowModalMask(true);
        w.centerInPage();

        final ValuesManager valuesManager = new ValuesManager();
        valuesManager.setDataSource(DataSource.get("RESELLER"));

        {
            final DynamicForm df = new DynamicForm();
            df.setIsGroup(true);
            df.setWidth(500);
            df.setTitleWidth(200);
            df.setAutoFetchData(false);

            TextItem nameTI = new TextItem("NAME");
            TextItem city = new TextItem("CITY");
            TextItem phone = new TextItem("TELEPHONE");
            SelectItem mcbiFlow = new SelectItem("COMPANY_ID");
            mcbiFlow.setDisplayField("NAME");
            mcbiFlow.setValueField("ID");
            df.setDataSource(valuesManager.getDataSource());
            df.setFields(nameTI, city, phone, mcbiFlow);
            valuesManager.addMember(df);
            w.addItem(df);
        }
        {
            final DynamicForm df2 = new DynamicForm();
            df2.setIsGroup(true);
            df2.setWidth(500);
            df2.setTitleWidth(200);
            df2.setAutoFetchData(false);
            TextItem cat1 = new TextItem("CAT1");
            TextItem cat2 = new TextItem("CAT2");
            df2.setDataSource(valuesManager.getDataSource());
            df2.setFields(cat1, cat2);
            valuesManager.addMember(df2);
            w.addItem(df2);
        }
        {
            IButton btn = new IButton("Save data");
            btn.addClickHandler(new ClickHandler() {
                @Override
                public void onClick(ClickEvent event) {
                    valuesManager.saveData();
                };
            });
            w.addItem(btn);
        }

        valuesManager.fetchData(new Criteria("ID", "1"));
        w.show();
    }
}

RESELLER.ds.xml (add in BuiltInDS.html):

<DataSource ID="RESELLER" serverType="sql" tableName="RESELLER">
    <fields>
        <field name="ID" type="integer" primaryKey="true" required="true" />
        <field foreignKey="COMPANY.ID" displayField="COMPANY_NAME" name="COMPANY_ID" type="integer" />
        <field name="COMPANY_NAME" includeFrom="COMPANY.NAME" />
        <field name="NAME" type="text" />
        <field name="CITY" type="text" />
        <field name="TELEPHONE" type="text" />
        <field name="CAT1" type="text" />
        <field name="CAT2" type="text" />
    </fields>
</DataSource>

COMPANY.ds.xml (add in BuiltInDS.html):

<DataSource ID="COMPANY" serverType="sql" tableName="COMPANY">
    <fields>
        <field name="ID" type="integer" primaryKey="true" required="true" />
        <field name="NAME" type="text" />
        <field name="CITY" type="text" />
        <field name="TELEPHONE" type="text" />
    </fields>
</DataSource>

DDL (Oracle, but any should do):

DROP TABLE reseller;
DROP TABLE company;

CREATE TABLE company (
    id          INTEGER NOT NULL,
    name        VARCHAR2(20),
    city        VARCHAR2(20),
    telephone   VARCHAR2(20),
    CONSTRAINT company_pk PRIMARY KEY ( id ) ENABLE
);

CREATE TABLE reseller (
    id           INTEGER NOT NULL,
    company_id   INTEGER,
    name         VARCHAR2(20),
    city         VARCHAR2(20),
    telephone    VARCHAR2(20),
    cat1         VARCHAR2(20),
    cat2         VARCHAR2(20),
    CONSTRAINT reseller_pk PRIMARY KEY ( id ),
    CONSTRAINT reseller_fk1 FOREIGN KEY ( company_id )
        REFERENCES company ( id )
    ENABLE
);

INSERT INTO company (ID,NAME,city,telephone) VALUES (1,'Company 1','NY','012345');
INSERT INTO company (ID,NAME,city,telephone) VALUES (2,'Company 2','NJ','65431');
INSERT INTO reseller (ID,company_id,NAME,city,telephone,cat1,cat2) VALUES (1,1,'Reseller 1','SF','987654','ABC','DEF');
COMMIT;

This is happening in my application as well.

Best regards
Blama

[Blama]

Hi Isomorphic,

do you have an update on this one?

Best regards
Blama

[Blama]

Hi Isomorphic,

it turns out this is reproduced way more easy (without ValuesManager) and therefore most likely way more severe (using v11.1p_2018-01-18).
Please see the video, where the on the 2nd open the name (displayField?) changes unexpectedly and on selecting an entry the data in the other ListGridField changes unexpectedly.

Again, the issue is dependent on the fields having the same name in different DataSources.

Sidequestion: I also tried to build the testcase with clientOnly DataSources, but there is no includeFrom in the DataSourceField java class. I assume therefore this testcase would not be that way, correct?



BuiltInDS.java:

package com.smartgwt.sample.client;

import com.google.gwt.core.client.EntryPoint;
import com.smartgwt.client.Version;
import com.smartgwt.client.core.KeyIdentifier;
import com.smartgwt.client.data.DataSource;
import com.smartgwt.client.util.Page;
import com.smartgwt.client.util.PageKeyHandler;
import com.smartgwt.client.util.SC;
import com.smartgwt.client.widgets.IButton;
import com.smartgwt.client.widgets.Window;
import com.smartgwt.client.widgets.events.ClickEvent;
import com.smartgwt.client.widgets.events.ClickHandler;
import com.smartgwt.client.widgets.form.fields.ComboBoxItem;
import com.smartgwt.client.widgets.form.fields.SelectItem;
import com.smartgwt.client.widgets.grid.ListGrid;
import com.smartgwt.client.widgets.grid.ListGridField;
import com.smartgwt.client.widgets.layout.VLayout;

public class BuiltInDS implements EntryPoint {
    public void onModuleLoad() {
        KeyIdentifier debugKey = new KeyIdentifier();
        debugKey.setCtrlKey(true);
        debugKey.setKeyName("D");

        Page.registerKey(debugKey, new PageKeyHandler() {
            public void execute(String keyName) {
                SC.showConsole();
            }
        });

        VLayout mainLayout = new VLayout(20);
        mainLayout.setWidth100();
        mainLayout.setHeight100();

        IButton recreateBtn = new IButton("Recreate SelectItem", new ClickHandler() {
            @Override
            public void onClick(ClickEvent event) {
                recreate(true);
            }
        });
        recreateBtn.setWidth(200);

        IButton recreateBtn2 = new IButton("Recreate ComboBoxItem", new ClickHandler() {
            @Override
            public void onClick(ClickEvent event) {
                recreate(false);
            }
        });
        recreateBtn2.setWidth(200);

        mainLayout.addMembers(recreateBtn, recreateBtn2);

        recreate(false);
        mainLayout.draw();

    }

    private void recreate(boolean asSelectItem) {
        Window w = new Window();
        w.setWidth("20%");
        w.setHeight("20%");
        w.setMembersMargin(0);
        w.setModalMaskOpacity(70);
        w.setTitle(" (" + Version.getVersion() + "/" + Version.getSCVersionNumber() + ")");
        w.setTitle("DynamicForm problems with same fieldnames in different DataSources" + w.getTitle());
        w.setShowMinimizeButton(false);
        w.setIsModal(true);
        w.setShowModalMask(true);
        w.centerInPage();

        final ListGrid lg = new ListGrid();
        lg.setDataSource(DataSource.get("RESELLER"));
        lg.setCanEdit(true);

        ListGridField companyIdLGF = new ListGridField("COMPANY_ID", asSelectItem ? "As SelectItem" : "As ComboBoxItem");

        if (asSelectItem) {
            SelectItem si = new SelectItem("ID");
            si.setCachePickListResults(false);
            si.setOptionDataSource(DataSource.get("COMPANY"));
            si.setValueField("ID");
            si.setDisplayField("NAME");
            companyIdLGF.setEditorProperties(si);
        } else {
            ComboBoxItem cbi = new ComboBoxItem("ID");
            cbi.setCachePickListResults(false);
            cbi.setOptionDataSource(DataSource.get("COMPANY"));
            cbi.setValueField("ID");
            cbi.setDisplayField("NAME");
            companyIdLGF.setEditorProperties(cbi);
        }

        ListGridField nameLGF = new ListGridField("NAME");

        lg.setFields(companyIdLGF, nameLGF);
        lg.fetchData();
        w.addItem(lg);
        w.show();
    }

}

RESELLER.ds.xml (add in BuiltInDS.html):

<DataSource ID="RESELLER" serverType="sql" tableName="RESELLER">
    <fields>
        <field name="ID" type="integer" primaryKey="true" required="true" />
        <field foreignKey="COMPANY.ID" displayField="COMPANY_NAME" name="COMPANY_ID" type="integer" />
        <field name="COMPANY_NAME" includeFrom="COMPANY.NAME" />
        <field name="NAME" type="text" />
    </fields>
</DataSource>

COMPANY.ds.xml (add in BuiltInDS.html):

<DataSource ID="COMPANY" serverType="sql" tableName="COMPANY">
    <fields>
        <field name="ID" type="integer" primaryKey="true" required="true" />
        <field name="NAME" type="text" />
        <field name="NAMETWICE" customSelectExpression="COMPANY.NAME||COMPANY.NAME" />
    </fields>
</DataSource>

Oracle DDL:

DROP TABLE reseller;
DROP TABLE company;

CREATE TABLE company (
    id          INTEGER NOT NULL,
    name        VARCHAR2(20),
    CONSTRAINT company_pk PRIMARY KEY ( id ) ENABLE
);

CREATE TABLE reseller (
    id           INTEGER NOT NULL,
    company_id   INTEGER,
    name         VARCHAR2(20),
    CONSTRAINT reseller_pk PRIMARY KEY ( id ),
    CONSTRAINT reseller_fk1 FOREIGN KEY ( company_id )
        REFERENCES company ( id )
    ENABLE
);

INSERT INTO company (ID,NAME) VALUES (1,'Company 1');
INSERT INTO company (ID,NAME) VALUES (2,'Company 2');
INSERT INTO reseller (ID,company_id,NAME) VALUES (1,1,'Reseller 1');
INSERT INTO reseller (ID,company_id,NAME) VALUES (2,1,'Reseller 2');
INSERT INTO reseller (ID,company_id,NAME) VALUES (3,2,'Reseller 3');
COMMIT;

There is also another issue that came up creating this testcase. I'll post another report for it.

Best regards
Blama

[Isomorphic]

Hi Blama,

As you've observed, the ambiguity in this case comes from the fact that there are "NAME" fields in both dataSources with different meanings, coupled with the fact that "displayField" can refer to a field within the local record, or a field within the optionDataSource.

The fix here is to use the "foreignDisplayField" attribute to disambiguate between the "displayField" on the local record (for static record display) and the displayField used by the SelectItem to pick up display values from the optionDataSource.


In concrete terms - your dataSources are set up correctly, following the pattern described in the discussion about editing found in the JavaDoc entry for DataSourceField.includeFrom.

RESELLER dataSource has the editable field "COMPANY_ID" which has a foreignKey relationship set up with the COMPANY dataSource.

The RESELLER.COMPANY_NAME field is using the DataSourceField.includeFrom feature to pick it up from the related record in COMPANY (using the COMPANY_ID value to find the appropriate related record).

RESELLER.COMPANY_ID has displayField set to COMPANY_NAME. This means that when a record is delivered to the client, if the COMPANY_ID field is visible, instead of displaying its raw data value (a numeric ID), it will display the COMPANY_NAME value from the record, which was picked up from the COMPANY dataSource.
All good so far.

In your application code, you have an editor for this field set up as a SelectItem. The optionDataSource for this is set to COMPANY, and valueField is set to ID, and displayField to NAME. This causes the selectItem to use the COMPANY dataSource as a server side valueMap, allowing the user to pick a new company ID, while displaying the associated NAME.

The problem is that this now overrides the previously defined DataSourceField.displayField of COMPANY_NAME, and there's a built in behavior when editing that if you change a field value which has a specified displayField, the record will have the displayField value updated along with the actual field being edited.

If you instead set the foreignDisplayField value on your SelectItem instace to "NAME", the displayField value will (correctly) continue to be set to "COMPANY_NAME" and that field will get updated in the record with the newly picked display value instead (which is appropriate behavior).

We'll be looking over our default behavior and the JavaDocs for this subsystem and see if we can make this clearer to avoid other developers hitting similar problems in the future

Regards
Isomorphic Software

[Blama]

Hi Isomorphic,

thanks for the detailed explanation.
Just using FormItem.setForeignDisplayField() instead of FormItem.setDisplayField() does indeed solve the immediate problem in the testcase. I must admit I don't know why I started with setDisplayField() years back, but as you say the docs are not very clear on this matter, especially the docs for setDisplayField().

The questions/issues I have w.r.t. to the docs are the following:

The item will display the displayField value from the record currently being edited if useLocalDisplayFieldValue is true, (or if unset and the conditions outlined in the documentation for that property are met).
Note that DataSourceField.useLocalDisplayFieldValue will default to true if not explicitly set in some cases, as described in the documentation for that property.

The docs there speak about when the getter will return true if the the property is unset. They never speak about when this will default to true.

Otherwise this item will perform a fetch against the optionDataSource to find a record where the value field matches this item's value, and use the displayField, or foreignDisplayField value from that record.
Note that the specified displayField must be explicitly defined in the optionDataSource to be used - see getDisplayFieldName() for more on this behavior.

That fetch is not happening for ListGridField-editors (it is for DynamicForm fields) , see this related thread (also happening with setForeignDisplayField()).
What I still don't get is why to use setDisplayField() on a FormItem(!), and not always setForeignDisplayField(). I do set the displayField for the record in my .ds.xml.
(I do have an assumption, though: Perhaps when not using the server framework with SQL joins, instead of the join, you can fire many fetchMissingValueReply requests to solve this?)
If that's true it would be great if the docs started "If using .ds.xml based DataSources alsways use setForeignDisplayField() instead of setDisplayField()".



Best regards
Blama

[Blama]

Hi Isomorphic,

can you elaborate on the docs question and especially on "If using .ds.xml based DataSources alsways use setForeignDisplayField() instead of setDisplayField()"?

Thank you & Best regards
Blama

[Isomorphic]

We've actually already updated the docs with more interlinking and a clearer explanation - please take a look and see if you think there is still more to cover.

[Blama]

Hi Isomorphic,

thanks - all the docs mention all relevant APIs so that nobody will miss an API like setForeignDisplayField() like I did.

Best regards
Blama