Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

  • Blama
    replied
    Hi Isomorphic,

    thank you, this seems to be working again with current 6.1p.

    Best regards
    Blama

    Leave a comment:


  • Isomorphic
    replied
    Check out new BatchUploader.autoInterpretBooleans flag, which is true by default, but setting it to false would cause BatchUpload.parseUploadData() to skip interpreting values for boolean fields, so you are able to control them manually.

    This will be available for download in nightly builds since June 6 (today).

    Leave a comment:


  • Blama
    replied
    Hi Isomorphic,

    the reason I noticed this is that you interpret "Nein" (German "No") as true. Before I could control this.
    How would I access the dsRequest before? There is no documented API that gives me access to the CSV Upload data in the incoming DSRequest. I can only see it *after* parseUploadData(), which will use some internal APIs I assume.

    Best regards
    Blama

    Leave a comment:


  • Isomorphic
    replied
    We added and documented some heuristics that will "do the right thing" for common boolean formats. We don't really think there are cases where you'd need to override this, but if you did, you could access the dsRequest before calling parseUploadData.

    Leave a comment:


  • Blama
    replied
    Hi Isomorphic,

    is this approach still correct (current 6.1p)? After some time (long ago), this changed and DSResponse response = batchUpload.parseUploadData(dsRequest) does directly convert to Boolean.
    I have the feeling that the only way for me to get the original values is, us to use the incoming dsRequest. But this is against the recommendation and I think I don't even have documented get*-APIs here, so that this is not possible at all. From the changes to my BatchUploadDMI.java-DMI class, the change must have happened before 2018-02-21. Not sure ho much before, though.

    This is my conversion code, which never reaches the String->Boolean switch:
    Code:
        private Boolean toBoolean(Object o) {
            if (o == null)
                return null;
            else if (o instanceof Boolean)
                return (Boolean) o;
    [B]// Seems not to be hit anymore, as values are already transmitted as Boolean[/B]
            else
                switch (o.toString().toLowerCase()) {
                case "x":
                case "true":
                case "ja":
                case "yes":
                case "y":
                    return true;
                default:
                    return false;
                }
        }
    This is the recommendation:
    Originally posted by Isomorphic View Post
    We've made some changes to address your use case. There is a possibility now to manipulate upload data before it is validated.

    Configure the builtin batchUpload.ds.xml DataSource to use custom DMI, which should parse uploaded data using new BatchUpload.parseUploadData(dsRequest) API, perform data manipulations and then validate data using another new BatchUpload.validateUploadData(dsResponse) API. Your DMI may look like this:
    Code:
    public DSResponse batchUpload(DSRequest dsRequest) throws Exception {
    BatchUpload batchUpload = new BatchUpload();
    
    // parse data and get the result Map
    [B]DSResponse response = batchUpload.parseUploadData(dsRequest); // Boolean fields contain boolean values after this line - not possible to access the old, CSV-supplied value[/B]
    Map respData = response.getDataMap();
    
    // do not proceed to validation if parsing failed
    if (respData.containsKey("errorMessage")) return response;
    
    // get upload data
    List<Map> uploadData = (List<Map>) respData.get("gridRows");
    
    // perform data manipulations
    for (Map row: uploadData) {
    if (row.containsKey("inStock")) {
    row.put("inStock", toBoolean(row.get("inStock")));
    }
    }
    
    // validate data and return
    return batchUpload.validateUploadData(response);
    }
    Best regards
    Blama

    Leave a comment:


  • Blama
    replied
    Hi Isomorphic,

    thanks for the hint on this correct way to get the dsName. I adjusted my code accordingly.

    Also again thanks for implementing this feature - as it turns out this is also useful to add default values that are not present in the CSV file to the data returned to the client.
    Like you already write:
    There are scenarios when mentioned earlier pre- and post-processing approach is not enough. For example if uploaded data needs to be transformed in a certain way before initial validation.
    For anyone finding this thread via search:
    • My usecase is that I have required fields that are not necessarily included in the CSV data.
    • Normally, an user would then have to edit every row and enter/select a value. Very time consuming for CSVs with many rows.
    • Using myBatchUploader.setUploadFormFields() it is possible to have a FormItem on top, select a value there, which then is sent to the server as well. On the server one can add this as default value to the data that later is returned to the client. This way an requiredField-error that would have to be removed manually in every row can be mitigated by including the selected value as default value.
    • (If you are using this lookup feature you'll most likely also use a SelectItem or ComboBoxItem as editor in the upload grid - in this case you need to include another HiddenItem in setUploadFormFields() for the displayValue in the grid data. Store the displayValue in the HiddenItem with an onChange Handler in the SelectItem and set the data for both fields during server processing.)
    Hope this helps.

    Best regards
    Blama

    Leave a comment:


  • Isomorphic
    replied
    The correct way to get dataSource name is shown below:
    Code:
     dsRequest.getValues().get("dsName");
    We added note on this to BatchUpload javadoc also.

    Leave a comment:


  • Blama
    replied
    Hi Isomorphic,

    regarding the feature itself: It seems to be working as expected for me, thanks a lot. I think it is a very valuable addition, as other use cases might benefit from this as well.

    Further remark, perhaps also as addition for your example code here:
    I get the "real" DataSource (dataSource:"batchUpload" is used for every upload, regardless of the business object uploaded) via:
    Code:
    dsRequest.getValues().toString().contains("myDSName")
    This might be important in more transformation-orientated use cases than mine is, where you want to handle different "target" DataSources differently.

    Thank you & Best regards
    Blama
    Last edited by Blama; 28 Sep 2016, 04:53.

    Leave a comment:


  • Blama
    replied
    Hi Isomorphic,

    first feedback is that in order to have a "default BatchUploader with a nop-DMI", you have to use this DMI:
    Code:
    <serverObject [I]ID="batchUpload" lookupStyle="new"[/I] className="com.lmscompany.lms.server.worker.BatchUploadDMI" [B]dropExtraFields="false"[/B]>
    where the italic parts are optional as I assume (sure for lookupStyle). dropExtraFields must be present, as otherwise the GUI does not show any entries (also no data in the Developer Console's RPC Tab, even though the server log shows the rows with data, validation errors etc).

    I'll test the new feature itself next.

    Best regards
    Blama

    Leave a comment:


  • Blama
    replied
    Hi Isomorphic,

    OK, that's embarrassing. I'll fix this, retry and let you know.

    PS: The docs for visibleMethods don't make you think that this would be something you need / could use in a "normal" SmartGWT application.
    .ds.xml's operation bindings define what serverMethod can be used / is used and which is not.

    Best regards
    Blama

    Leave a comment:


  • Isomorphic
    replied
    The problem is in batchUpload.ds.xml - you can't have multiple <serverObject> elements at the same level. Remove (comment out) original serverObject declaration if you are replacing it with your own. Let us know how it worked for you.

    Leave a comment:


  • Blama
    replied
    Hi Isomorphic,

    this seems to do exactly what I need.
    I tested using v10.1p_2016-09-27 and hit the following error:
    Code:
    === 2016-09-27 17:58:23,864 [c-11] INFO  RequestContext - URL: '/lms/lms/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.113 Safari/537.36': Safari with Accept-Encoding header
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: host:localhost:8080
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: connection:keep-alive
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: content-length:1443
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: origin:http://localhost:8080
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: user-agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.113 Safari/537.36
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: content-type:application/x-www-form-urlencoded; charset=UTF-8
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: accept:*/*
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: dnt:1
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: referer:http://localhost:8080/lms/
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: accept-encoding:gzip, deflate
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: accept-language:de-DE,de;q=0.8,en;q=0.6,en-US;q=0.4
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: cookie:JSESSIONID=88A199BAEE291935AF978D87973937B0; isc_cState=ready; GLog=%7B%0A%20%20%20%20trackRPC%3Atrue%2C%20%0A%20%20%20%20isc_pageURL%3A%22http%3A//localhost%3A8080/lms/%22%2C%20%0A%20%20%20%20isc_pageGUID%3A%2215EE2470-7FD4-4573-922A-CFAC63494D33%22%2C%20%0A%20%20%20%20priorityDefaults%3A%7B%0A%20%20%20%20%20%20%20%20sgwtInternal%3A1%0A%20%20%20%20%7D%2C%20%0A%20%20%20%20defaultPriority%3A3%2C%20%0A%20%20%20%20left%3A76%2C%20%0A%20%20%20%20top%3A39%2C%20%0A%20%20%20%20width%3A1844%2C%20%0A%20%20%20%20height%3A905%0A%7D
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - session exists: 88A199BAEE291935AF978D87973937B0
    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - remote user: Administrator
    === 2016-09-27 17:58:23,865 [ec-7] INFO  RequestContext - URL: '/lms/lms/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.113 Safari/537.36': Safari with Accept-Encoding header
    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: host:localhost:8080
    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: connection:keep-alive
    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: content-length:2285
    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: cache-control:max-age=0
    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: origin:http://localhost:8080
    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: upgrade-insecure-requests:1
    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: user-agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.113 Safari/537.36
    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: content-type:multipart/form-data; boundary=----WebKitFormBoundaryzTLEFZ92tchjlkl7
    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: dnt:1
    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: referer:http://localhost:8080/lms/
    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: accept-encoding:gzip, deflate
    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: accept-language:de-DE,de;q=0.8,en;q=0.6,en-US;q=0.4
    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: cookie:JSESSIONID=88A199BAEE291935AF978D87973937B0; isc_cState=ready; GLog=%7B%0A%20%20%20%20trackRPC%3Atrue%2C%20%0A%20%20%20%20isc_pageURL%3A%22http%3A//localhost%3A8080/lms/%22%2C%20%0A%20%20%20%20isc_pageGUID%3A%2215EE2470-7FD4-4573-922A-CFAC63494D33%22%2C%20%0A%20%20%20%20priorityDefaults%3A%7B%0A%20%20%20%20%20%20%20%20sgwtInternal%3A1%0A%20%20%20%20%7D%2C%20%0A%20%20%20%20defaultPriority%3A3%2C%20%0A%20%20%20%20left%3A76%2C%20%0A%20%20%20%20top%3A39%2C%20%0A%20%20%20%20width%3A1844%2C%20%0A%20%20%20%20height%3A905%0A%7D
    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - session exists: 88A199BAEE291935AF978D87973937B0
    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - remote user: Administrator
    === 2016-09-27 17:58:23,869 [c-11] DEBUG RPCManager - Processing 1 requests.
    === 2016-09-27 17:58:23,869 [c-11] DEBUG RPCManager - Request #1 (RPCRequest) data: {
        appID:"isc_builtin",
        className:"com.isomorphic.tools.BuiltinRPC",
        methodName:"setAttributes",
        arguments:[
            "session",
            [
                {
                    name:"dsName",
                    type:"hidden",
                    value:"V_RESELLERUPLOAD__32__2016_09_09_15_09_25"
                },
                {
                    name:"delimiter",
                    type:"hidden",
                    value:";"
                },
                {
                    name:"quoteString",
                    type:"hidden",
                    value:"\""
                },
                {
                    name:"dataFormat",
                    type:"hidden",
                    value:"csv"
                }
            ],
            null
        ],
        is_ISC_RPC_DMI:true
    }
    === 2016-09-27 17:58:23,869 [c-11] INFO  LMSIDACall - Performing 1 operation(s)
    === 2016-09-27 17:58:23,870 [ec-7] DEBUG RPCManager - Processing 1 requests.
    === 2016-09-27 17:58:23,871 [ec-7] DEBUG RPCManager - Request #1 (DSRequest) payload: {
        values:{
            file:"C:\\fakepath\\reseller.csv",
            dsName:"V_RESELLERUPLOAD__32__2016_09_09_15_09_25",
            delimiter:";",
            quoteString:"\"",
            dataFormat:"csv",
            _transaction:null
        },
        operationConfig:{
            dataSource:"batchUpload",
            repo:null,
            operationType:"add",
            textMatchStyle:"exact"
        },
        componentId:"isc_DynamicForm_5",
        appID:"builtinApplication",
        operation:"upload",
        oldValues:{
        },
        criteria:{
        }
    }
    === 2016-09-27 17:58:23,871 [ec-7] INFO  LMSIDACall - Performing 1 operation(s)
    === 2016-09-27 17:58:23,871 [c-11] DEBUG RPCDMI - appConfig: isc.Application.create({
        rpcBindings:[
            {
                ID:"builtin",
                className:"com.isomorphic.rpc.BuiltinRPC",
                visibleMethods:[
                    {
                        name:"downloadWSDL"
                    },
                    {
                        name:"downloadClientContent"
                    },
                    {
                        name:"downloadClientExport"
                    },
                    {
                        name:"xmlToJS"
                    },
                    {
                        name:"uploadProgressCheck"
                    },
                    {
                        name:"saveFile"
                    },
                    {
                        name:"appendToFile"
                    },
                    {
                        name:"loadFile"
                    },
                    {
                        name:"deleteFile"
                    },
                    {
                        name:"loadSharedXML"
                    },
                    {
                        name:"saveSharedXML"
                    },
                    {
                        name:"getAvailableScriptEngines"
                    },
                    {
                        name:"devConsoleEvalServerScript"
                    },
                    {
                        name:"evalJava"
                    },
                    {
                        name:"getLogNames"
                    },
                    {
                        name:"getLogEntries"
                    },
                    {
                        name:"clearLogEntries"
                    },
                    {
                        name:"getLogThresholds"
                    },
                    {
                        name:"setLogThreshold"
                    },
                    {
                        name:"setTemporaryLogThreshold"
                    },
                    {
                        name:"revertTemporaryLogThresholds"
                    },
                    {
                        name:"getPdfObject"
                    },
                    {
                        name:"exportImage"
                    },
                    {
                        name:"areServerTimingsTracked"
                    },
                    {
                        name:"trackServerTimings"
                    },
                    {
                        name:"messagingSend"
                    },
                    {
                        name:"downloadZip"
                    }
                ]
            },
            {
                ID:"builtin_tools",
                className:"com.isomorphic.tools.BuiltinRPC",
                visibleMethods:[
                    {
                        name:"getDataSourceFromTable"
                    },
                    {
                        name:"getDataSourceJSONFromTable"
                    },
                    {
                        name:"getDataSourceFromHibernateMapping"
                    },
                    {
                        name:"getDataSourceJSONFromHibernateMapping"
                    },
                    {
                        name:"getTables"
                    },
                    {
                        name:"getFieldsFromTable"
                    },
                    {
                        name:"getBeanFields"
                    },
                    {
                        name:"getHibernateBeans"
                    },
                    {
                        name:"getDatabaseProductNameAndVersion"
                    },
                    {
                        name:"getDatabaseTableTypes"
                    },
                    {
                        name:"setAttributes"
                    },
                    {
                        name:"clearAttributes"
                    },
                    {
                        name:"getAttributes"
                    },
                    {
                        name:"getAttribute"
                    },
                    {
                        name:"getDataSourceConfigFromJavaClass"
                    },
                    {
                        args:"cName",
                        language:"groovy",
                        name:"getJavaSource",
                        script:"\n                    if (!com.isomorphic.auth.DevModeAuthFilter.devModeAuthorized(request)) throw new Exception(\"Not Authorized\");                    \n                    //import org.apache.bcel.Repository;\n\n                    try {\n                        return org.apache.bcel.Repository.lookupClass(cName).toString();\n                    } catch (Throwable e) {\n                        return \"Unable to reverse engineer class \"+cName+\": \"+e.getMessage();\n                    }\n                "
                    },
                    {
                        name:"loadDataSource"
                    },
                    {
                        name:"dsFromXML"
                    },
                    {
                        name:"dsConfigFromXML"
                    },
                    {
                        name:"getDefinedDataSources"
                    },
                    {
                        name:"importData"
                    },
                    {
                        name:"checkForTestData"
                    }
                ]
            },
            {
                ID:"builtin_adminconsole",
                className:"com.isomorphic.tools.AdminConsole",
                visibleMethods:[
                    {
                        name:"getDefinedDatabases"
                    },
                    {
                        name:"testDB"
                    },
                    {
                        name:"saveDBConfig"
                    },
                    {
                        name:"setDefaultDB"
                    },
                    {
                        name:"importDataSources"
                    },
                    {
                        name:"discoverJNDIDatabases"
                    }
                ]
            }
        ]
    })
    
    === 2016-09-27 17:58:23,872 [ec-7] WARN  RequestContext - dsRequest.execute() failed: 
    java.lang.ClassCastException: java.util.ArrayList cannot be cast to java.util.Map
        at com.isomorphic.datasource.DataSource.getServerObjectConfig(DataSource.java:1225)
        at com.isomorphic.datasource.DataSourceDMI.validateOperationBinding(DataSourceDMI.java:499)
        at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:150)
        at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:64)
        at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2702)
        at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:230)
        at com.lmscompany.lms.server.LMSIDACall.handleDSRequest(LMSIDACall.java:180)
        at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:187)
        at com.lmscompany.lms.server.LMSIDACall.processRequest(LMSIDACall.java:89)
        at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:119)
        at com.isomorphic.servlet.IDACall.doPost(IDACall.java:79)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)
        at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:162)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
        at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:88)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Unknown Source)
    === 2016-09-27 17:58:23,873 [ec-7] DEBUG RPCManager - Content type for RPC transaction: text/html; charset=UTF-8
    === 2016-09-27 17:58:23,874 [ec-7] DEBUG RPCManager - non-DMI response, dropExtraFields: false
    === 2016-09-27 17:58:23,874 [c-11] DEBUG RPCDMI - rpc returned data
    === 2016-09-27 17:58:23,874 [ec-7] INFO  Compression - /lms/lms/sc/IDACall: 696 -> 432 bytes
    === 2016-09-27 17:58:23,874 [c-11] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
    === 2016-09-27 17:58:23,875 [c-11] INFO  Compression - /lms/lms/sc/IDACall: 67 -> 73 bytes
    This is my BatchUploadDMI.java (closely following your example):
    Code:
    package com.lmscompany.lms.server.worker;
    
    import java.util.List;
    import java.util.Map;
    
    import com.isomorphic.datasource.DSRequest;
    import com.isomorphic.datasource.DSResponse;
    import com.isomorphic.tools.BatchUpload;
    
    public class BatchUploadDMI {
        @SuppressWarnings({ "unchecked", "rawtypes" })
        public DSResponse batchUpload(DSRequest dsRequest) throws Exception {
            BatchUpload batchUpload = new BatchUpload();
    
            // parse data and get the result Map
            DSResponse response = batchUpload.parseUploadData(dsRequest);
            Map respData = response.getDataMap();
    
            // do not proceed to validation if parsing failed
            if (respData.containsKey("errorMessage"))
                return response;
    
            // get upload data
            List<Map> uploadData = (List<Map>) respData.get("gridRows");
    
            // perform data manipulations
            for (Map row : uploadData) {
                if (row.containsKey("ADMINUSER")) {
                    row.put("ADMINUSER", toBoolean(row.get("ADMINUSER")));
                }
            }
    
            // validate data and return
            return batchUpload.validateUploadData(response);
        }
    
        private Boolean toBoolean(Object o) {
            if (o == null)
                return null;
            else
                switch (o.toString().toLowerCase()) {
                case "x":
                case "true":
                case "ja":
                case "yes":
                    return true;
                }
            return false;
        }
    };
    This is my batchUpload.ds.xml:
    Code:
    <!-- DataSource to support the BatchUploader component
         NOTE: This is not the DataSource resposible for performing the batch update of verified [B]//"resposible" has a typo[/B]
               records, it is an internal resource used to handle the initial upload of the client-
               side file. The only time you would ever want to change this DataSource is if you wanted 
               a hook into the upload process (for security reasons, for example).  See the 
               BatchUploader documentation in the SmartClient reference.
    -->
    <DataSource ID="batchUpload">
        <serverObject lookupStyle="new" className="com.lmscompany.lms.server.worker.BatchUploadDMI" />
        <operationBindings>
            <operationBinding operationType="add" operationId="upload" serverMethod="batchUpload" />
            <operationBinding operationType="custom" operationId="wipeData" serverMethod="wipeData" />
            <operationBinding operationType="loadSchema" />
        </operationBindings>
        <serverObject ID="batchUpload" className="com.isomorphic.tools.BatchUpload" dropExtraFields="false">
            <visibleMethods>
                <method name="batchUpload" />
                <method name="wipeData" />
            </visibleMethods>
        </serverObject>
    </DataSource>
    Just commenting out the DMI in .ds.xml removes the error.
    If I put a breakpoint in the very top of BatchUploadDMI.java, it is not hit!

    I'm using v10.1p_2016-09-27.

    Best regards
    Blama

    Leave a comment:


  • Isomorphic
    replied
    Hi Blama,

    We've made some changes to address your use case. There is a possibility now to manipulate upload data before it is validated.

    Configure the builtin batchUpload.ds.xml DataSource to use custom DMI, which should parse uploaded data using new BatchUpload.parseUploadData(dsRequest) API, perform data manipulations and then validate data using another new BatchUpload.validateUploadData(dsResponse) API. Your DMI may look like this:
    Code:
        public DSResponse batchUpload(DSRequest dsRequest) throws Exception {
            BatchUpload batchUpload = new BatchUpload();
            
            // parse data and get the result Map
            DSResponse response = batchUpload.parseUploadData(dsRequest);
            Map respData = response.getDataMap();
            
            // do not proceed to validation if parsing failed
            if (respData.containsKey("errorMessage")) return response;
            
            // get upload data
            List<Map> uploadData = (List<Map>) respData.get("gridRows"); 
            
            // perform data manipulations
            for (Map row: uploadData) {
                if (row.containsKey("inStock")) {
                    row.put("inStock", toBoolean(row.get("inStock")));
                }
            }
            
            // validate data and return
            return batchUpload.validateUploadData(response);
        }
    Note how data manipulations and validation are skipped if parsing failed. Please see BatchUpload javadocs for more details.

    Regards,
    Isomorphic

    Leave a comment:


  • Blama
    replied
    Hi Isomorphic,

    how about the following:

    Incoming data for boolean fields can have many formats (T, X, x, F, no, yes, ....), as we already wrote.

    You check a method
    Code:
    Boolean textToBoolean(String value) {
    return null;
    }
    in the DMI class for the DataSource used for a BatchUpload. If it is there, you call it during upload (not during add!) and return true, false or a validation error (for return of null) to the client BatchUploader ListGrid.
    This way, on the client one can use a BooleanItem as editor - something that is not possible for fields defined like:
    Code:
            <field name="BRANCH" uploadFieldName="BRANCH" [B]type="text"[/B] length="5" escapeHTML="true">
                <valueMap>
                    <value>TRUE</value>
                    <value>True</value>
                    <value>true</value>
                    <value>YES</value>
                    <value>Yes</value>
                    <value>yes</value>
                    <value>JA</value>
                    <value>Ja</value>
                    <value>ja</value>
                    <value>X</value>
                    <value>x</value>
                    <value>FALSE</value>
                    <value>False</value>
                    <value>false</value>
                    <value>NO</value>
                    <value>No</value>
                    <value>no</value>
                    <value>NEIN</value>
                    <value>Nein</value>
                    <value>nein</value>
                </valueMap>
            </field>
    For these fields, the UI looks like this when using a BooleanItem as editor:
    Click image for larger version

Name:	Unbenannt.PNG
Views:	198
Size:	6.6 KB
ID:	240140
    and you can't have a nice clean UI with Checkboxes both in display and edit.
    I as programmer and not you as framework provider would still be responsible for clean Boolean-value detection, but I could do so using a clean GUI.

    All approaches of converting the value later at the ADD stage and not at the UPLOAD stage suffer from a poor UI like shown.

    Best regards
    Blama

    Leave a comment:


  • Blama
    replied
    Hi Isomorphic,

    I tried this and this is pretty easy if you do it (valid strings -> boolean-conversion) in the upload-DataSources'-Add-DMI.
    If you want the uploadGrid field to be a BooleanItem, this does not work, as then the .ds.xml-field itself is type="text".

    I think one way would be to enter the process earlier (when validation (and conversion?) of the csv-values) is done.
    Per the docs one should:
    A couple of server-side techniques are interesting in conjunction with the BatchUploader. One is to set the DataSource.serverConstructor property to point at your own class that inherits fromcom.isomorphic.datasource.BasicDataSource. The most interesting reason for doing this is to override the validate method and provide complete custom validation - for example, checking relations to other tables.
    Is this also where you do the conversion of String->ID in case of importStrategy="display", meaning not only validation but also alteration of uploaded data? I think this would be the right place for me.

    Best regards
    Blama

    Leave a comment:

Working...
X