Issue with saveLocally configured ListGrids

[markok]

Hi,

We have some custom editors based on saveLocally configured grids for editing. We also have a server backend behind our data sources which return "null" values as missing elements in the response. What this results to is that the records returned by the server don't have the attributes set at all when the values are null. The saveLocally configured grids are used to edit the data in these data sources.

We noticed some odd behaviour with these grids and tracked it down to the issue, that if the saveLocally grid is updated with a updateData(record) call, the attributes which have no values have no effect at all. A simple test case to demonstrate is as follows:

    public void doOnModuleLoad() {
        viewport = new VLayout();
        viewport.setWidth100();
        viewport.setHeight100();
        viewport.setOverflow(Overflow.HIDDEN);

        // simulate a data source

        final DataSource ds = new DataSource();
        ds.setID("myDS");
        DataSourceIntegerField f1 = new DataSourceIntegerField("f1");
        f1.setPrimaryKey(true);
        f1.setHidden(true);
        DataSourceTextField f2 = new DataSourceTextField("f2");
        DataSourceTextField f3 = new DataSourceTextField("f3");
        ds.setFields(f1, f2, f3);
        ds.setClientOnly(true);

        final ListGridRecord r1 = new ListGridRecord();
        r1.setAttribute("f1", "1");
        r1.setAttribute("f2", "first value");
        r1.setAttribute("f3", "second value");
        final ListGridRecord r2 = new ListGridRecord();
        r2.setAttribute("f1", "2");
        r2.setAttribute("f2", "third value");
        r2.setAttribute("f3", "fourth value");

        ds.setTestData(r1, r2);

        // our editable grid in saveLocally mode

        final ListGrid g = new ListGrid();
        g.setDataSource(ds);
        g.setSaveLocally(true);
        g.setCanEdit(true);
        g.setEditByCell(false);
        g.setAutoSaveEdits(false);
        ListGridField gf1 = new ListGridField("f1");
        ListGridField gf2 = new ListGridField("f2");
        ListGridField gf3 = new ListGridField("f3");
        g.setFields(gf1, gf2, gf3);

        // fetch data to griud

        ds.fetchData(new Criteria(), new DSCallback() {
            @Override
            public void execute(DSResponse dsResponse, Object data, DSRequest dsRequest) {
                g.setData(dsResponse.getData());
            }
        });

        Button b = new Button("save edits");
        b.addClickHandler(new ClickHandler() {
            @Override
            public void onClick(ClickEvent event) {
                int[] editRows = g.getAllEditRows();
                for(int i=0; i<editRows.length; i++) {
                    int row = editRows[i];
                    final Record er = g.getEditedRecord(row);

                    //
                    // here we call our server backend and commit edits there, we call update data on server
                    //

                    ds.updateData(er, new DSCallback() {
                        @Override
                        public void execute(DSResponse dsResponse, Object data, DSRequest dsRequest) {
                            Record serverRecord = dsResponse.getData()[0];

                            //
                            // our backend returns null values as missing elements in the response
                            //  which results in the attributes being not present in the record
                            //

                            // we simulate the above by removing all null attributes grom the record
                            String[] attributes = serverRecord.getAttributes();
                            for(String a : attributes) {
                                if(serverRecord.getAttribute(a) == null) {
                                    JSOHelper.deleteAttribute(serverRecord.getJsObj(), a);
                                }
                            }

                            SC.say(
                                    "server edited record: " + serverRecord.getAttribute("f1")
                                            + "/" + serverRecord.getAttribute("f2")
                                            + "/" + serverRecord.getAttribute("f3")
                            );

                            // update grid with data source returned record
                            g.updateData(serverRecord);
                        }
                    });

                }
            }
        });

        Button b2 = new Button("do we have edits ?");
        b2.addClickHandler(new ClickHandler() {
            @Override
            public void onClick(ClickEvent event) {
                SC.say("Does grid have changes ?: " + g.hasChanges());
            }
        });

        Button b3 = new Button("discard all edits");
        b3.addClickHandler(new ClickHandler() {
            @Override
            public void onClick(ClickEvent event) {
                g.discardAllEdits();
            }
        });

        viewport.addMember(b);
        viewport.addMember(b2);
        viewport.addMember(b3);
        viewport.addMember(g);
        viewport.draw();
    }

To demonstrate the issue, run the sample above and do as follows:

1. Click "do we have edits ?" and note that the grid reports no edits present
2. Set a blank value to cell f2 on some row (remove the text completely)
3. Click "do we have edits ?" and note that the grid reports edits present as expected
4. Click "save edits", the record saved is printed out
5. Click "do we have edits ?" and note that the grid still reports edits present, incorrectly
6. Click "discard all edits" and note that the text reappears to the edited f2 cell

We debugged this and noticed that the local updateData call only updates the records if the matching attributes are present. If the record used for the update does not have the attributes, they are completely ignored. Is this intentional ? Wouldn't it be more logical to interpret missing attributes for the record as null values ?

Using SmartGWT LGPL 6.0p (NIGHTLY-2017-03-11).

[Isomorphic]

You seem to have already identified what to do - if you want values in the client-side Records wiped out due to server data where they'll been set null, then deliver null values in the record.

[markok]

Yes, well it's impossible for us to change the backend but we probably go around this in the client code relating to the editors mentioned.

It's quite typical for JSON-based backends to drop the null values from the responses, as you probably know. We were just wondering is this intentional as we are surprised our application works at all if this behavior is consistent and same everywhere in the framework.

[Isomorphic]

Actually, looking at this again, the test case and reported issue don't make much sense. You've set saveLocally so there is no server record that could come back with changes from other users, and your seeming attempt to simulate this doesn't make sense either - you're wiping attributes off a record in the cache, so you're just breaking delta detection by doing something invalid.

If there's a bad behavior here that actually has to do with a server returning JSON records in response to an update, try demonstrating that. You shouldn't need an actual server, just a hardcoded response file for the update will do.

If the problem is indeed related to saveLocally, then you'll need a way of demonstrating it that doesn't just mangle the cached records in the middle of the update, which is invalid.

[markok]

Yes, well the same thing happens even if you clone the "serverRecord" record to a new JavaScriptObject and update the grid with that.

If you check DataBoundComponent. _uploadLocalData you'll see that the update operation is handled by a call to "isc.addProperties" with records to be combined. This addProperties function - as defined - ignores null values in records.

[Isomorphic]

Yes, and that addProperties() call correctly applies changed properties to the record in the grid's dataset, including the ability to set a field to null which previously had a value.

So again we have an invalid test case and a strange complaint about difficulty with JSON records from a server, in a use case where no server is involved. Let us know if you can put together a valid bug report.