Efficient way to edit 1000 rows of data in a grid

[Ben07]

Currently I'm using smartclient 12.1p,
Is there any proper ways to edit 1000 rows of records with the most efficient ways?
Currently my program will use a loop to assign data into 1000 rows,
I am trying to make this process faster, is it possible to disable redraw and enable it back after I finished edit those 1000 rows?

[Blama]

Hi Ben07

Update the DataSource and not the ListGrid, in a Queue if the updates are all structurally different or as DSRequest to a special operationBinding with allowMultiUpdate=true.
For sync in the latter case either set invalidateCache on the clientside DSRequest or directly send the updated data with the server response using DSResponse.addRelatedUpdates().

Best regards
Blama

[Ben07]

Hi Blama,
Currently I am using CacheAllData DataSource, and due to we need to use validate row to validate before save, seteditvalue to grid is needed in order to validate row. So when I am looping 1000 rows and edit it programmatically, updateData and seteditvalue is needed to edit a record. And I realised that this kind of process uses 30+seconds to perform edit process. Does it have any other way to solve this?

[Blama]

As I said, update the DataSource in bulk, not the ListGrid.

[Isomorphic]

The BatchUploader component is designed for this exact use case.

[Ben07]

I have tried to update Datasource without update grid, when comes to validate row, I have found that validate row malfunction, so means that i should use BatchUploader?

[Isomorphic]

If you're going to claim a "malfunction", you will need a standalone, ready-to-run test case that has been tested against the latest patched version of SmartClient, and demonstrates a flaw in the SmartClient framework.

You should be careful to check that your own application code has not simply crashed and that the SmartClient log is not full of warnings (which are expensive to log, and can slow things down) telling you that your code is doing something wrong.

Note that there's nothing wrong with your use case and the framework supports it. BatchUploader is simply a pre-built component that uses the framework as it is - no internal usage - just doing things the right way. You can replicate it on your own, but, it looks like you're struggling, so just using the pre-built BatchUploader would be the easier path for you.

If you expect more help, either from Isomorphic or from the community, you should:

1) read the forum rules, and the Debugging overview in the Reference, to understand how to make it possible to help you

2) buy a support package if you expect help when you haven't done the basics. We do offer support on an hourly basis, for situations where you haven't figured out if the problem is your code or the framework, and you don't have time to do that, so you'd prefer we figure that out for you

[Ben07]

Hi there,
I think the test case below can provide a clear explanation of my current situation.
Apply button is to update only datasource.
ApplyWithSetEditValue button is to update datasource and list grid.
ValidateRow is to validate all rows of the grid.

Scenario1
Step1: Click on Apply button
Step2: Click on Validate button

Scenario2
Step1: Click on ApplyWithSetEditValue button
Step2: Click on Validate button

Scenario 1, If i use Apply button on to apply data, ValidateRow button will not validate all rows on this as no editValues is set.
However in Scenario 2, if i use ApplyWithSetEditValue to apply data, ValidateRow button is works to validate.
Btw i have realised that the blue font style didn't set to the ListGrid for changes field in both scenario, am I missing some setting?

isc.RestDataSource.create({
        ID: "TestDataSource",
        dataFormat:"json",
        fields: [
            { name: "No", type: "text" },
            { name: "Field1", type: "text", primaryKey:true },
            { name: "Field2", type: "text" },
        ],
        cacheAllData: true,
        transformRequest: function (dsRequest) {
            dsRequest["dataProtocol"] = "clientCustom";

            var response = this.getClientOnlyResponse(dsRequest, this.getCacheData());

            this.processResponse(dsRequest.requestId, response);

            return dsRequest.data;

        },
        transformResponse: function (dsResponse, dsRequest, data) {

        }

    });


    isc.defineClass("TestGridWithControl", isc.ListGrid);

    isc.TestGridWithControl.addProperties({
        enablePaging: false,
        dataFetchMode: "basic",
        initWidget() {
            this.Super("initWidget", arguments);
            this.clearData(true);
        },
        clearData(clearCache = true) {
            const ds = this.getDataSource();
            const emptyData = isc.ResultSet.create({
                dataSource: ds,
                fetchMode: "basic",
                initialData: [],
                initialLength: 0
            });
            this.setData(emptyData);

            this.discardAllEdits();

            if (clearCache) {
                ds.setCacheAllData(true);
                ds.setCacheData([]);
            }
        }
    });

    isc.TestGridWithControl.create({
        ID: "TestListGrid",
        fields: [
            { name: "No", type: "text" },
            { name: "Field1", type: "text" },
            {
                name: "Field2",
                type: "integer",
                validators: [{
                    type: "integerRange", min: 1, max: 2,
                    errorMessage: "Value Must be a 1-2"
                }]
            },
        ],
        dataSource: TestDataSource,
    });

    isc.Button.create({
        ID: "TestApplyButton",
        title: "ApplyButton",
        width: "100%",
        click: function () {
            var singleRecord = {};
            var editedRecord = {};
            console.log("start apply on " + new Date());
            for (a = 0; a < TestListGrid.getTotalRows(); a++) {
                singleRecord = TestListGrid.getRecord(a);
                editedRecord = Object.assign(TestListGrid.getEditedRecord(a), { Field2: 4 });
                TestListGrid.updateData(editedRecord, null, { oldValues: singleRecord });
            }
            console.log("end apply on " + new Date());
        }
    });
    isc.Button.create({
        ID: "TestApplywithSetEditValuesButton",
        title: "ApplyWithSetEditValueButton",
        width: "100%",
        click: function () {
            var singleRecord = {};
            console.log("start applyWithSetEditvalues on " + new Date());
            for (a = 0; a < TestListGrid.getTotalRows(); a++) {
                singleRecord = TestListGrid.getRecord(a);
                TestListGrid.setEditValue(a, "Field2", 4);
                TestListGrid.updateData(TestListGrid.getEditedRecord(a), null, { oldValues: singleRecord, _operationType: "update" });
            }
            console.log("end applyWithSetEditvalues on " + new Date());
        }
    });
    isc.Button.create({
        ID: "ValidateRowButton",
        title: "ValidateRowButton",
        width: "100%",
        click: function () {
            for (a = 0; a < TestListGrid.getTotalRows(); a++) {
                TestListGrid.validateRow(a);
            }
        }
    });

    isc.VLayout.create({
        ID: "GUITest",
        baseID: "GUITest",
        overflow: "scroll",
        height: "100%",
        width: "100%",
        members: [TestApplyButton, TestApplywithSetEditValuesButton, ValidateRowButton , TestListGrid],
    });
    function run() {
        var testData = [];
        for (var i = 0; i < 1000; i++)
        {
            testData.push({ No: i, Field1: "Data" + i, Field2: 1 });
        }

        TestDataSource.setCacheData(testData);


        TestListGrid.fetchData(null, null, { operationId: "FetchLocal" });
    }
    run();

[Blama]

Hi Ben07,

are you using SmartClient also in the back end?

My suggestion was to send one update to the DataSource (targeting multiple rows), not one update per row.
You control what happens on the server and can send back error messages that your code then can show.
You can decide if your server side changes are in a transaction and rolled back for even 1 error or if you apply the good changes.

This is all independent of you using SmartClient also in the back end, but would pretty likely be easier that way.

Also, validation must also happen on the server. Client side validation is good, as it saves unnecessary round trips, but the server also must have validation.
I'm saying this as you have your validator on the ListGrid, not on the DataSource, in your test case.

Best regards
Blama

[Ben07]

Hi @Blama,

Currently, I'm not using smartclient as a backend.

Currently, my system structure designed to maintain one pending request list that record all the changes made from grid(updateData) and will use this pending request list send to server when perform save operation.

You are correct, I have maintained validation in both serverside and clientside, for client side is to save the round trips from server, serverside serve as an extra protection.

Currently, I'm using updateData to perform one update per row, does it have any reference from the doc mentioned to update multiple rows?

[Blama]

Hi Ben07,

sorry, this is only possible from the server side. Client side DSRequest does not allow to target more than one row. I remembered that wrongly.

So if your scenario is to set all Field2: 4 like in the example, I'd send one request after a button click to your server that then does this server side.
In your response you specify invalidateCache: true and then the ListGrid will automatically reload it's data.

Best regards
Blama