6.1p DataImport importToDataSource issue with boolean values

[Blama]

Hi Isomorphic,

while the issue in this thread is still valid, the real issue I had there was a different one, related to DataImport (v11.1p_2017-12-05). In absence of the feature mentioned here

Generally speaking this is working, but IMHO the workaround with CSV should not be necessary. DataImport-methods should have an overload that take Map<String, Object> and do the conversion displayName->ID

I need to convert my values to CSV, and then run DataImport. DataImport does only convert "true" to true, "false" is converted to null.

Please see this example:
supplyItem.ds.xml change:

        <field name="inStock"   type="boolean"  length="1" title="In Stock" escapeHTML="true" sqlStorageStrategy="singleCharYN" />
        <field name="nextShipment"  type="date" title="Next Shipment"/>
    </fields>
        <serverObject lookupStyle="new" className="com.smartgwt.sample.server.listener.SupplyItem" />
        <operationBindings>
            <operationBinding operationType="custom" operationId="addApi" serverMethod="addApi" />
        </operationBindings>

SupplyItem.java:

package com.smartgwt.sample.server.listener;

import java.io.Reader;
import java.io.StringReader;
import java.io.StringWriter;
import java.util.List;
import java.util.Map;

import javax.servlet.http.HttpServletRequest;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.datasource.DataSource;
import com.isomorphic.tools.DataImport;
import com.isomorphic.tools.DataImport.ImportFormat;

public class SupplyItem {
    public DSResponse addApi(DSRequest dsRequest, HttpServletRequest servletRequest) throws Exception {
        @SuppressWarnings("unchecked")
        Map<String, Object> map = dsRequest.getValues();

        StringWriter csvHeader = new StringWriter();
        StringWriter csvData = new StringWriter();
        for (Map.Entry<String, Object> entry : map.entrySet()) {
            csvHeader.append(entry.getKey() + ";");
            if (entry.getValue() != null)
                csvData.append("\"" + entry.getValue().toString().replace("\"", "\"\"") + "\"" + ";");
            else
                csvData.append(";");
        }

        Reader jsonReader = new StringReader(csvHeader.toString() + "\r\n" + csvData.toString());

        DataImport dataImporter = new DataImport(ImportFormat.CSV, ";");
        dataImporter.setPopulateDisplayFields(true);

        @SuppressWarnings("unchecked")
        List<Map<String, Object>> dataImporterResult = dataImporter.importDataSourceRecords(jsonReader, dsRequest.getDataSourceName());

        DSRequest addRequest = new DSRequest(dsRequest.getDataSourceName(), DataSource.OP_ADD, dsRequest.getRPCManager());
        addRequest.setValues(dataImporterResult);

        DSResponse addResponse = addRequest.execute();
        return addResponse;
    }
}

ARC request (good):

<request>
    <dataSource>supplyItem</dataSource>
    <operationType>custom</operationType>
    <operationId>addApi</operationId>
    <data>
        <itemName>Agent pen</itemName>
        <SKU>007</SKU>
        <category>Office Paper Products</category>
        <unitCost>0.07</unitCost>
        <units>Ea</units>
[B]<inStock>true</inStock>[/B]
    </data>
</request>

Data in Eclipse when setting a breakpoint:

csvData: "Agent pen";"007";"Office Paper Products";"0.07";"Ea";[B]"true"[/B];
dataImporterResult: [{itemName=Agent pen, unitCost=0.07, [B]inStock=true[/B], units=Ea, category=Office Paper Products, SKU=007}]

ARC request (bad):

<request>
    <dataSource>supplyItem</dataSource>
    <operationType>custom</operationType>
    <operationId>addApi</operationId>
    <data>
        <itemName>Agent pen</itemName>
        <SKU>007</SKU>
        <category>Office Paper Products</category>
        <unitCost>0.07</unitCost>
        <units>Ea</units>
[B]<inStock>false</inStock>[/B]
    </data>
</request>

Data in Eclipse when setting a breakpoint:

csvData: "Agent pen";"007";"Office Paper Products";"0.07";"Ea";[B]"false"[/B];
dataImporterResult: {itemName=Agent pen, unitCost=0.07, [B]inStock=null[/B], units=Ea, category=Office Paper Products, SKU=007}

As you can see, the value is null, and not false, as expected.

This one is a important one for me, as basically all RESTHandler issues (other boolean issue, queue status and enhancement for this usecase (more easy FK-resolution via RESTHandler) and DataImport function and docs improvement, RESTHandler with relatedUpdate (not as important as the others for me right now)).
I also made these enhancement suggestion: more powerful Expanded Raw REST mode URI Syntax (minor for me right now).

Best regards
Blama

[Blama]

Hi Isomorphic,

this is perhaps related to this change of yours, which addresses exactly this issue for GUI-based Batch CSV Upload.

Best regards
Blama

[Isomorphic]

We've made a fix to address the boolean-parsing issue affecting your sample code. It's been applied to SGWT 6.1p/SC 11.1p and newer releases, and will be in the nightly builds dated 2017-12-12 and beyond.

[Blama]

Hi Isomorphic,

I can see that null,false and true work as expected (v11.1p_2017-12-27).
Additionally, all other values including empty string are converted to true as well. This is minor IMHO, but I think this should result in some kind of error.

Best regards
Blama