6.1p How To: MultiSelectItem in BatchUpload

[Blama]

Hi Isomorphic,

how is the following use case improvement best done (6.1p)?
I have a DataSource with a child DataSource I upload data to, leads and products, 1:n. Of course, I do not send productIds with the uploaded data, but product names.
I now have a text-field in my DS where I can upload data in the form "product1, product2, product3" from CSV (it gets returned the same way to the uploader-ListGrid) and the system recognizes this, splits at "," and validates product name existence. This is done by my custom validator.

The editor in the BatchUploader is a TextAreaItem and I can fix issues here in the data just fine. I now want to improve this by having a MultiSelectItem or MultiComboBoxItem to edit these values.
Does this somewhere involve the setting of multiple=true in the .ds.xml-field? I tried many of these settings, but nothing worked.

Prod1,Prod2,Prod3
"Prod1,Prod2,Prod3"
"Prod1","Prod2","Prod3"
["Prod1","Prod2","Prod3"]

By "nothing worked" I mean that I can see the values in the ListGrid, but the MultiSelectItem is unchecked. I somehow must tell the framework that this string (ID conversion only happens at upload) needs to result in those three items checked in the MultiSelectItem.
In the RPC tab, I see always stuff like this returned:

MYFIELD: "my content with quotes escaped"
I never see
MYFIELD: [whatever in braces]

I assume that this way the MultiSelectItem can't parse the data, so what am I expected to do?

1st step would be to get this working, 2nd step (if necessary) would be to really be able to work with "Prod1,Prod2,Prod3" and "Prod1, Prod2, Prod3" (with spaces) as data in Excel, because one can't expect from an end user to stick to some arbitrary syntax.

Thank you & Best regards
Blama

[Isomorphic]

The expected data format for a multiple:true field is an array of values. If you are trying to produce this from server-side code, use a Java List.

[Blama]

Hi Isomorphic,

thanks a lot. This worked, but I found a bug in this area (using v11.1p_2017-12-16). Try the sample with the upload data and have a look at the RPC Tab - everything fine array-wise.
The BatchUploader ListGrid also does look fine, but when you double click a Friend field here with more than one friend in the list, you'll notice that only the 1st entry is selected in the picklist.

Please see this testcase

BuiltInDS.java:

package com.smartgwt.sample.client;

import com.google.gwt.core.client.EntryPoint;
import com.smartgwt.client.core.KeyIdentifier;
import com.smartgwt.client.data.DataSource;
import com.smartgwt.client.util.Page;
import com.smartgwt.client.util.PageKeyHandler;
import com.smartgwt.client.util.SC;
import com.smartgwt.client.widgets.BatchUploader;
import com.smartgwt.client.widgets.IButton;
import com.smartgwt.client.widgets.Window;
import com.smartgwt.client.widgets.events.ClickEvent;
import com.smartgwt.client.widgets.events.ClickHandler;
import com.smartgwt.client.widgets.form.fields.SelectItem;
import com.smartgwt.client.widgets.form.fields.SpinnerItem;
import com.smartgwt.client.widgets.grid.ListGridField;
import com.smartgwt.client.widgets.layout.VLayout;

public class BuiltInDS implements EntryPoint {
    private VLayout mainLayout;
    private IButton recreateBtn;

    public void onModuleLoad() {
        KeyIdentifier debugKey = new KeyIdentifier();
        debugKey.setCtrlKey(true);
        debugKey.setKeyName("D");

        Page.registerKey(debugKey, new PageKeyHandler() {
            public void execute(String keyName) {
                SC.showConsole();
            }
        });

        mainLayout = new VLayout(20);
        mainLayout.setWidth100();
        mainLayout.setHeight100();

        recreateBtn = new IButton("Recreate");
        recreateBtn.addClickHandler(new ClickHandler() {
            @Override
            public void onClick(ClickEvent event) {
                recreate();
            }
        });
        mainLayout.addMember(recreateBtn);
        mainLayout.draw();
    }

    private void recreate() {
        Window w = new Window();
        w.setWidth("95%");
        w.setHeight("95%");
        w.setMembersMargin(0);
        w.setModalMaskOpacity(70);
        w.setTitle("Import data");
        w.setShowMinimizeButton(false);
        w.setIsModal(true);
        w.setShowModalMask(true);
        w.centerInPage();

        BatchUploader batchUploader = new BatchUploader();
        batchUploader.setWidth100();
        batchUploader.setUploadDataSource(DataSource.get("employeesUpload"));

        ListGridField nameLGF = new ListGridField("Name");

        ListGridField managerLGF = new ListGridField("ReportsTo");
        managerLGF.setOptionDataSource(DataSource.get("employees"));
        managerLGF.setValueField("EmployeeId");
        managerLGF.setDisplayField("Name");

        ListGridField mentorLGF = new ListGridField("Mentor");
        mentorLGF.setOptionDataSource(DataSource.get("employees"));
        mentorLGF.setValueField("EmployeeId");
        mentorLGF.setDisplayField("Name");

        ListGridField jobLGF = new ListGridField("Job");

        ListGridField genderLGF = new ListGridField("Gender");

        ListGridField salaryLGF = new ListGridField("Salary");
        salaryLGF.setEditorProperties(new SpinnerItem());

        ListGridField friendsLGF = new ListGridField("Friends");
        SelectItem friendsSI = new SelectItem();
        friendsSI.setOptionDataSource(DataSource.get("employees"));
        friendsSI.setDisplayField("Name");
        friendsSI.setValueField("Name");
        friendsSI.setSortField("Name");
        friendsSI.setMultiple(true);
        friendsLGF.setEditorProperties(friendsSI);

        batchUploader.setGridFields(nameLGF, genderLGF, jobLGF, managerLGF, mentorLGF, salaryLGF, friendsLGF);

        w.addItem(batchUploader);
        w.show();
    }
}

BuiltInDS.html:

addition of "batchUpload,employeesUpload" in DataSources

batchUpload.ds.xml:

<!-- DataSource to support the BatchUploader component
     NOTE: This is not the DataSource resposible for performing the batch update of verified
           records, it is an internal resource used to handle the initial upload of the client-
           side file. The only time you would ever want to change this DataSource is if you wanted 
           a hook into the upload process (for security reasons, for example).  See the 
           BatchUploader documentation in the SmartClient reference.
-->

<DataSource ID="batchUpload">
    <operationBindings>
        <operationBinding operationType="add" operationId="upload" serverMethod="batchUpload" />
        <operationBinding operationType="custom" operationId="wipeData" serverMethod="wipeData" />
        <operationBinding operationType="loadSchema" />
    </operationBindings>
    <serverObject ID="batchUpload" className="com.smartgwt.sample.server.listener.BatchUploadDMI" dropExtraFields="false">
        <visibleMethods>
            <method name="batchUpload" />
            <method name="wipeData" />
        </visibleMethods>
    </serverObject>
</DataSource>

employeesUpload.ds.xml:

<DataSource ID="employeesUpload" serverType="sql" tableName="employeeTable" recordName="employee" useAnsiJoins="true">
    <fields>
        <field name="EmployeeId" primaryKey="true" hidden="true" type="sequence" />

        <field name="Name" uploadFieldName="Name" title="Name" type="text" length="128">
            <validators>
                <validator type="isUnique" caseSensitive="true" />
            </validators>
        </field>

        <field name="ReportsTo" uploadFieldName="ReportsTo" displayField="ReportsToName" title="Manager" required="true" importStrategy="display"
            foreignKey="employees.EmployeeId" relatedTableAlias="relatedReportsTo">
            <validators>
                <validator type="hasRelatedRecord" errorMessage="Unknown ReportsTo" />
            </validators>
        </field>
        <field name="ReportsToName" includeFrom="employees.Name" includeVia="ReportsTo" />

        <field name="Mentor" uploadFieldName="Mentor" displayField="MentorName" title="Mentor" importStrategy="display" foreignKey="employees.EmployeeId"
            relatedTableAlias="relatedMentor">
            <validators>
                <validator type="hasRelatedRecord" errorMessage="Unknown Mentor" />
            </validators>
        </field>
        <field name="MentorName" includeFrom="employees.Name" includeVia="Mentor" />

        <field name="Job" uploadFieldName="Job" title="Title" type="text" length="128">
            <valueMap>
                <value>Developer</value>
                <value>IT-Infrastructure</value>
                <value>Other</value>
            </valueMap>
        </field>

        <field name="Gender" uploadFieldName="Gender" title="Gender" type="text" length="7">
            <valueMap>
                <value>male</value>
                <value>female</value>
            </valueMap>
        </field>

        <field name="Salary" uploadFieldName="Salary" title="Salary" type="integer">
        </field>
    </fields>

    <field name="Friends" uploadFieldName="Friends" title="Friends" type="text" length="100" multiple="true" />

</DataSource>

BatchUploadDMI.java:

package com.smartgwt.sample.server.listener;

import java.util.Arrays;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.log.Logger;
import com.isomorphic.tools.BatchUpload;

public class BatchUploadDMI {
    final Logger logger = new Logger(BatchUploadDMI.class);

    @SuppressWarnings({ "unchecked", "rawtypes" })
    public DSResponse batchUpload(DSRequest dsRequest) throws Exception {
        String dsName = dsRequest.getValues().get("dsName").toString();
        BatchUpload batchUpload = new BatchUpload();

        // parse data and get the result Map
        DSResponse response = batchUpload.parseUploadData(dsRequest);
        Map respData = response.getDataMap();

        // do not proceed to validation if parsing failed
        if (respData.containsKey("errorMessage"))
            return response;

        // get upload data
        List<Map<String, Object>> uploadData = (List<Map<String, Object>>) respData.get("gridRows");

        // Do ,-separated to Java List conversions for all kind of uploads
        doListConversions(uploadData, dsName);

        // Validate data and return
        return batchUpload.validateUploadData(response);
    }

    private void doListConversions(List<Map<String, Object>> uploadData, String dsName) {
        // perform data manipulations
        for (Map<String, Object> row : uploadData) {
            Object friendsString = row.get("Friends");
            if (friendsString != null) {
                String[] splittedFriedString = friendsString.toString().split(",");
                for (String friendName : splittedFriedString) {
                    friendName = friendName.trim();
                }
                List<String> friendsList = new LinkedList<String>(Arrays.asList(splittedFriedString));
                row.put("Friends", friendsList);
            }
        }
    }
}

TestdataImportEmployees.csv:

"Name","ReportsTo","Mentor","Job","Gender","Salary","Friends"
"Anna Sun","Charles Madigen",,"Developer","female",1000,"Charles Madigen, Ralph Brogan"
"Mike Sun","Charles Madigen","Gene Porter","Developer","male",1000,"Bhushan Sambhus"
"Foo Bar","Charles Madigen","TYPO","Developer","male",3000,"Betsy Rosenbaum, Francine Bergeron, Amanda Jones"
"Ray Sun","Charles Madigen","Carol Finley","IT-Infrastructure",,2000,
"Maddie Smith","TYPO","Gene Porter","IT-Infrastructure","female",4000,

Best regards
Blama

[Isomorphic]

Looking at the code, it looks like your call to split() would not remove leading spaces, which would cause a mismatch with the valueMap for the second and later entries.

[Blama]

There is also a "trim" in the code.
can you have look again - I'm not at a computer right now but I think the code is OK.

[Blama]

Hi Isomorphic,

sorry, you were right. I retested now myself. The way I tried to modify the array variable was not correct.
Using this code it seems to work as expected:

    private void doListConversions(List<Map<String, Object>> uploadData, String dsName) {
        // perform data manipulations
        for (Map<String, Object> row : uploadData) {
            Object friendsString = row.get("Friends");
            if (friendsString != null) {
                List<String> friendsList = new LinkedList<String>();
                String[] splittedFriendString = friendsString.toString().split(",");
                for (String friendName : splittedFriendString) {
                    friendsList.add(friendName.trim());
                }
                row.put("Friends", friendsList);
            }
        }
    }

Thank you & Best regards
Blama

[Blama]

Hi Isomorphic,

as written before, using the changed code the values for the SelectItem are present as expected when opening the pickList.
But for some reason the "Friends" field does look edited (font color blue) after editing e.g. the Salary field without ever opening or editing the Friends field itself.

Please see the testcase from #1 and the correction from #6 and just edit any other field to see that also the text of Friends gets blue. Tested with v11.1p_2018-01-07.

Best regards
Blama

[Isomorphic]

We've made a change to address this issue. Please try the next nightly build, dated January 11.

Regards
Isomorphic Software

[Blama]

Hi Isomorphic,

this is solved for me using v11.1p_2018-01-23, thanks a lot. There is one issue though remaining, but this could also be related to the setForeignDisplayField() vs setDisplayField() problem, so I'll change code here before and report back, if that's still current afterwards.

Best regards
Blama