Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

    5.1 BatchUploader: How to upload "false"-value to boolean field?

    Hi Isomorphic,

    I'm having trouble to upload "false" to a type="boolean" sqlStorageStrategy="singleCharYN"-ds.xml field (v10.1p_2016-03-27).
    The framework seems to only recognize empty cells as "false", and not one of "N", "false", "False", "No".

    supplyItem.ds.xml inStock-addition to match my configuration:
    Code:
    sqlStorageStrategy="singleCharYN"
    BuiltInDS.java:
    Code:
    package com.smartgwt.sample.client;
    
    import com.google.gwt.core.client.EntryPoint;
    import com.smartgwt.client.core.KeyIdentifier;
    import com.smartgwt.client.data.DataSource;
    import com.smartgwt.client.util.Page;
    import com.smartgwt.client.util.PageKeyHandler;
    import com.smartgwt.client.util.SC;
    import com.smartgwt.client.widgets.BatchUploader;
    import com.smartgwt.client.widgets.IButton;
    import com.smartgwt.client.widgets.Window;
    import com.smartgwt.client.widgets.events.ClickEvent;
    import com.smartgwt.client.widgets.events.ClickHandler;
    import com.smartgwt.client.widgets.grid.ListGridField;
    import com.smartgwt.client.widgets.layout.VLayout;
    
    public class BuiltInDS implements EntryPoint {
        public void onModuleLoad() {
            KeyIdentifier debugKey = new KeyIdentifier();
            debugKey.setCtrlKey(true);
            debugKey.setKeyName("D");
    
            Page.registerKey(debugKey, new PageKeyHandler() {
                public void execute(String keyName) {
                    SC.showConsole();
                }
            });
    
            VLayout mainLayout = new VLayout(20);
            mainLayout.setWidth100();
            mainLayout.setHeight100();
    
            IButton recreateBtn = new IButton("Recreate");
            recreateBtn.addClickHandler(new ClickHandler() {
                @Override
                public void onClick(ClickEvent event) {
                    recreate();
                }
            });
            mainLayout.addMember(recreateBtn);
            recreate();
            mainLayout.draw();
        }
    
        private void recreate() {
            Window w = new Window();
            w.setWidth("95%");
            w.setHeight("95%");
            w.setMembersMargin(0);
            w.setModalMaskOpacity(70);
            w.setTitle("Import data");
            w.setShowMinimizeButton(false);
            w.setIsModal(true);
            w.setShowModalMask(true);
            w.centerInPage();
    
            BatchUploader batchUploader = new BatchUploader();
            batchUploader.setWidth100();
            batchUploader.setUploadDataSource(DataSource.get("supplyItem"));
            batchUploader.setDefaultDelimiter(",");
            batchUploader.setDefaultQuoteString("\"");
    
            ListGridField itemName = new ListGridField("itemName");
    
            ListGridField sku = new ListGridField("SKU");
    
            ListGridField category = new ListGridField("category");
    
            ListGridField unitCost = new ListGridField("unitCost");
    
            ListGridField inStock = new ListGridField("inStock");
    
            batchUploader.setGridFields(itemName, sku, category, unitCost, inStock);
    
            w.addItem(batchUploader);
            w.show();
        }
    }
    See the attached CSV-file for a example.

    I'd say that the correct solution would be to have uploadFieldTrueValue and uploadFieldFalseValue-attributes (or similar named) in the .ds.xml that have (like date-fields) locale-specific defaults.

    Best regards
    Blama
    Attached Files

    #2
    This is expected - the SQLStorageStrategy affects storage only and has no impact outside of the SQL layer - anyone using the DataSource API (client or server) is unaware of the translation and just sees a normal Boolean field.

    If you wanted to accept additional values for a Boolean field on import, you could do that via custom code in your uploadDataSource.

    Comment


      #3
      Hi Isomorphic,

      I'm talking about something else - Boolean conversion for CSV-BatchImports. How am I supposed to transmit an false value?
      Everything is treated as true if you run the testcase.

      As I wrote I added the .ds.xml just to match my configuration and suggested a *similar* attribute for CSV-imports.

      Best regards
      Blama

      Comment


        #4
        As you mentioned, empty cells work for false. For those other values, you would add custom logic in an uploadDataSource.

        Comment


          #5
          Hi Isomorphic,

          what do you mean by "uploadDataSource"? The DMI reference in the .ds.xml? I could do it there, but this does not seem right.
          IMHO it should be possible to have explicit false values in the csv-data as well.

          As the specific string is not going to be language-agnostic, but more locale-specific like a date, it should have a per-locale default, which can be changed.
          Why should by default "foobar" be translated to true? Why should it not make validation fail by default?
          Why should by default "no" be translated to true?
          I don't have the problem, but how would you tell apart NULL and false in an import with the way things are?

          As there is a workaround with DMI, which I'm going to use now, this perhaps is more of an enhancement and not a bug, but it is definitely is not good default behavior, more like PHP :)

          Best regards
          Blama

          Comment


            #6
            As you seem to have realized while talking it through, there is no good way for the framework to automatically deal with all the messy things that might appear in uploaded data. It's a job for application-specific custom code, as we previously suggested.

            Comment


              #7
              Hi Isomorphic,

              I tried this and this is pretty easy if you do it (valid strings -> boolean-conversion) in the upload-DataSources'-Add-DMI.
              If you want the uploadGrid field to be a BooleanItem, this does not work, as then the .ds.xml-field itself is type="text".

              I think one way would be to enter the process earlier (when validation (and conversion?) of the csv-values) is done.
              Per the docs one should:
              A couple of server-side techniques are interesting in conjunction with the BatchUploader. One is to set the DataSource.serverConstructor property to point at your own class that inherits fromcom.isomorphic.datasource.BasicDataSource. The most interesting reason for doing this is to override the validate method and provide complete custom validation - for example, checking relations to other tables.
              Is this also where you do the conversion of String->ID in case of importStrategy="display", meaning not only validation but also alteration of uploaded data? I think this would be the right place for me.

              Best regards
              Blama

              Comment


                #8
                Hi Isomorphic,

                how about the following:

                Incoming data for boolean fields can have many formats (T, X, x, F, no, yes, ....), as we already wrote.

                You check a method
                Code:
                Boolean textToBoolean(String value) {
                return null;
                }
                in the DMI class for the DataSource used for a BatchUpload. If it is there, you call it during upload (not during add!) and return true, false or a validation error (for return of null) to the client BatchUploader ListGrid.
                This way, on the client one can use a BooleanItem as editor - something that is not possible for fields defined like:
                Code:
                        <field name="BRANCH" uploadFieldName="BRANCH" [B]type="text"[/B] length="5" escapeHTML="true">
                            <valueMap>
                                <value>TRUE</value>
                                <value>True</value>
                                <value>true</value>
                                <value>YES</value>
                                <value>Yes</value>
                                <value>yes</value>
                                <value>JA</value>
                                <value>Ja</value>
                                <value>ja</value>
                                <value>X</value>
                                <value>x</value>
                                <value>FALSE</value>
                                <value>False</value>
                                <value>false</value>
                                <value>NO</value>
                                <value>No</value>
                                <value>no</value>
                                <value>NEIN</value>
                                <value>Nein</value>
                                <value>nein</value>
                            </valueMap>
                        </field>
                For these fields, the UI looks like this when using a BooleanItem as editor:
                Click image for larger version

Name:	Unbenannt.PNG
Views:	124
Size:	6.6 KB
ID:	240140
                and you can't have a nice clean UI with Checkboxes both in display and edit.
                I as programmer and not you as framework provider would still be responsible for clean Boolean-value detection, but I could do so using a clean GUI.

                All approaches of converting the value later at the ADD stage and not at the UPLOAD stage suffer from a poor UI like shown.

                Best regards
                Blama

                Comment


                  #9
                  Hi Blama,

                  We've made some changes to address your use case. There is a possibility now to manipulate upload data before it is validated.

                  Configure the builtin batchUpload.ds.xml DataSource to use custom DMI, which should parse uploaded data using new BatchUpload.parseUploadData(dsRequest) API, perform data manipulations and then validate data using another new BatchUpload.validateUploadData(dsResponse) API. Your DMI may look like this:
                  Code:
                      public DSResponse batchUpload(DSRequest dsRequest) throws Exception {
                          BatchUpload batchUpload = new BatchUpload();
                          
                          // parse data and get the result Map
                          DSResponse response = batchUpload.parseUploadData(dsRequest);
                          Map respData = response.getDataMap();
                          
                          // do not proceed to validation if parsing failed
                          if (respData.containsKey("errorMessage")) return response;
                          
                          // get upload data
                          List<Map> uploadData = (List<Map>) respData.get("gridRows"); 
                          
                          // perform data manipulations
                          for (Map row: uploadData) {
                              if (row.containsKey("inStock")) {
                                  row.put("inStock", toBoolean(row.get("inStock")));
                              }
                          }
                          
                          // validate data and return
                          return batchUpload.validateUploadData(response);
                      }
                  Note how data manipulations and validation are skipped if parsing failed. Please see BatchUpload javadocs for more details.

                  Regards,
                  Isomorphic

                  Comment


                    #10
                    Hi Isomorphic,

                    this seems to do exactly what I need.
                    I tested using v10.1p_2016-09-27 and hit the following error:
                    Code:
                    === 2016-09-27 17:58:23,864 [c-11] INFO  RequestContext - URL: '/lms/lms/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.113 Safari/537.36': Safari with Accept-Encoding header
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: host:localhost:8080
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: connection:keep-alive
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: content-length:1443
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: origin:http://localhost:8080
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: user-agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.113 Safari/537.36
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: content-type:application/x-www-form-urlencoded; charset=UTF-8
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: accept:*/*
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: dnt:1
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: referer:http://localhost:8080/lms/
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: accept-encoding:gzip, deflate
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: accept-language:de-DE,de;q=0.8,en;q=0.6,en-US;q=0.4
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - Header Name:Value pair: cookie:JSESSIONID=88A199BAEE291935AF978D87973937B0; isc_cState=ready; GLog=%7B%0A%20%20%20%20trackRPC%3Atrue%2C%20%0A%20%20%20%20isc_pageURL%3A%22http%3A//localhost%3A8080/lms/%22%2C%20%0A%20%20%20%20isc_pageGUID%3A%2215EE2470-7FD4-4573-922A-CFAC63494D33%22%2C%20%0A%20%20%20%20priorityDefaults%3A%7B%0A%20%20%20%20%20%20%20%20sgwtInternal%3A1%0A%20%20%20%20%7D%2C%20%0A%20%20%20%20defaultPriority%3A3%2C%20%0A%20%20%20%20left%3A76%2C%20%0A%20%20%20%20top%3A39%2C%20%0A%20%20%20%20width%3A1844%2C%20%0A%20%20%20%20height%3A905%0A%7D
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - session exists: 88A199BAEE291935AF978D87973937B0
                    === 2016-09-27 17:58:23,864 [c-11] DEBUG LMSIDACall - remote user: Administrator
                    === 2016-09-27 17:58:23,865 [ec-7] INFO  RequestContext - URL: '/lms/lms/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.113 Safari/537.36': Safari with Accept-Encoding header
                    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: host:localhost:8080
                    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: connection:keep-alive
                    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: content-length:2285
                    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: cache-control:max-age=0
                    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: origin:http://localhost:8080
                    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: upgrade-insecure-requests:1
                    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: user-agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.113 Safari/537.36
                    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: content-type:multipart/form-data; boundary=----WebKitFormBoundaryzTLEFZ92tchjlkl7
                    === 2016-09-27 17:58:23,865 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
                    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: dnt:1
                    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: referer:http://localhost:8080/lms/
                    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: accept-encoding:gzip, deflate
                    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: accept-language:de-DE,de;q=0.8,en;q=0.6,en-US;q=0.4
                    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - Header Name:Value pair: cookie:JSESSIONID=88A199BAEE291935AF978D87973937B0; isc_cState=ready; GLog=%7B%0A%20%20%20%20trackRPC%3Atrue%2C%20%0A%20%20%20%20isc_pageURL%3A%22http%3A//localhost%3A8080/lms/%22%2C%20%0A%20%20%20%20isc_pageGUID%3A%2215EE2470-7FD4-4573-922A-CFAC63494D33%22%2C%20%0A%20%20%20%20priorityDefaults%3A%7B%0A%20%20%20%20%20%20%20%20sgwtInternal%3A1%0A%20%20%20%20%7D%2C%20%0A%20%20%20%20defaultPriority%3A3%2C%20%0A%20%20%20%20left%3A76%2C%20%0A%20%20%20%20top%3A39%2C%20%0A%20%20%20%20width%3A1844%2C%20%0A%20%20%20%20height%3A905%0A%7D
                    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - session exists: 88A199BAEE291935AF978D87973937B0
                    === 2016-09-27 17:58:23,866 [ec-7] DEBUG LMSIDACall - remote user: Administrator
                    === 2016-09-27 17:58:23,869 [c-11] DEBUG RPCManager - Processing 1 requests.
                    === 2016-09-27 17:58:23,869 [c-11] DEBUG RPCManager - Request #1 (RPCRequest) data: {
                        appID:"isc_builtin",
                        className:"com.isomorphic.tools.BuiltinRPC",
                        methodName:"setAttributes",
                        arguments:[
                            "session",
                            [
                                {
                                    name:"dsName",
                                    type:"hidden",
                                    value:"V_RESELLERUPLOAD__32__2016_09_09_15_09_25"
                                },
                                {
                                    name:"delimiter",
                                    type:"hidden",
                                    value:";"
                                },
                                {
                                    name:"quoteString",
                                    type:"hidden",
                                    value:"\""
                                },
                                {
                                    name:"dataFormat",
                                    type:"hidden",
                                    value:"csv"
                                }
                            ],
                            null
                        ],
                        is_ISC_RPC_DMI:true
                    }
                    === 2016-09-27 17:58:23,869 [c-11] INFO  LMSIDACall - Performing 1 operation(s)
                    === 2016-09-27 17:58:23,870 [ec-7] DEBUG RPCManager - Processing 1 requests.
                    === 2016-09-27 17:58:23,871 [ec-7] DEBUG RPCManager - Request #1 (DSRequest) payload: {
                        values:{
                            file:"C:\\fakepath\\reseller.csv",
                            dsName:"V_RESELLERUPLOAD__32__2016_09_09_15_09_25",
                            delimiter:";",
                            quoteString:"\"",
                            dataFormat:"csv",
                            _transaction:null
                        },
                        operationConfig:{
                            dataSource:"batchUpload",
                            repo:null,
                            operationType:"add",
                            textMatchStyle:"exact"
                        },
                        componentId:"isc_DynamicForm_5",
                        appID:"builtinApplication",
                        operation:"upload",
                        oldValues:{
                        },
                        criteria:{
                        }
                    }
                    === 2016-09-27 17:58:23,871 [ec-7] INFO  LMSIDACall - Performing 1 operation(s)
                    === 2016-09-27 17:58:23,871 [c-11] DEBUG RPCDMI - appConfig: isc.Application.create({
                        rpcBindings:[
                            {
                                ID:"builtin",
                                className:"com.isomorphic.rpc.BuiltinRPC",
                                visibleMethods:[
                                    {
                                        name:"downloadWSDL"
                                    },
                                    {
                                        name:"downloadClientContent"
                                    },
                                    {
                                        name:"downloadClientExport"
                                    },
                                    {
                                        name:"xmlToJS"
                                    },
                                    {
                                        name:"uploadProgressCheck"
                                    },
                                    {
                                        name:"saveFile"
                                    },
                                    {
                                        name:"appendToFile"
                                    },
                                    {
                                        name:"loadFile"
                                    },
                                    {
                                        name:"deleteFile"
                                    },
                                    {
                                        name:"loadSharedXML"
                                    },
                                    {
                                        name:"saveSharedXML"
                                    },
                                    {
                                        name:"getAvailableScriptEngines"
                                    },
                                    {
                                        name:"devConsoleEvalServerScript"
                                    },
                                    {
                                        name:"evalJava"
                                    },
                                    {
                                        name:"getLogNames"
                                    },
                                    {
                                        name:"getLogEntries"
                                    },
                                    {
                                        name:"clearLogEntries"
                                    },
                                    {
                                        name:"getLogThresholds"
                                    },
                                    {
                                        name:"setLogThreshold"
                                    },
                                    {
                                        name:"setTemporaryLogThreshold"
                                    },
                                    {
                                        name:"revertTemporaryLogThresholds"
                                    },
                                    {
                                        name:"getPdfObject"
                                    },
                                    {
                                        name:"exportImage"
                                    },
                                    {
                                        name:"areServerTimingsTracked"
                                    },
                                    {
                                        name:"trackServerTimings"
                                    },
                                    {
                                        name:"messagingSend"
                                    },
                                    {
                                        name:"downloadZip"
                                    }
                                ]
                            },
                            {
                                ID:"builtin_tools",
                                className:"com.isomorphic.tools.BuiltinRPC",
                                visibleMethods:[
                                    {
                                        name:"getDataSourceFromTable"
                                    },
                                    {
                                        name:"getDataSourceJSONFromTable"
                                    },
                                    {
                                        name:"getDataSourceFromHibernateMapping"
                                    },
                                    {
                                        name:"getDataSourceJSONFromHibernateMapping"
                                    },
                                    {
                                        name:"getTables"
                                    },
                                    {
                                        name:"getFieldsFromTable"
                                    },
                                    {
                                        name:"getBeanFields"
                                    },
                                    {
                                        name:"getHibernateBeans"
                                    },
                                    {
                                        name:"getDatabaseProductNameAndVersion"
                                    },
                                    {
                                        name:"getDatabaseTableTypes"
                                    },
                                    {
                                        name:"setAttributes"
                                    },
                                    {
                                        name:"clearAttributes"
                                    },
                                    {
                                        name:"getAttributes"
                                    },
                                    {
                                        name:"getAttribute"
                                    },
                                    {
                                        name:"getDataSourceConfigFromJavaClass"
                                    },
                                    {
                                        args:"cName",
                                        language:"groovy",
                                        name:"getJavaSource",
                                        script:"\n                    if (!com.isomorphic.auth.DevModeAuthFilter.devModeAuthorized(request)) throw new Exception(\"Not Authorized\");                    \n                    //import org.apache.bcel.Repository;\n\n                    try {\n                        return org.apache.bcel.Repository.lookupClass(cName).toString();\n                    } catch (Throwable e) {\n                        return \"Unable to reverse engineer class \"+cName+\": \"+e.getMessage();\n                    }\n                "
                                    },
                                    {
                                        name:"loadDataSource"
                                    },
                                    {
                                        name:"dsFromXML"
                                    },
                                    {
                                        name:"dsConfigFromXML"
                                    },
                                    {
                                        name:"getDefinedDataSources"
                                    },
                                    {
                                        name:"importData"
                                    },
                                    {
                                        name:"checkForTestData"
                                    }
                                ]
                            },
                            {
                                ID:"builtin_adminconsole",
                                className:"com.isomorphic.tools.AdminConsole",
                                visibleMethods:[
                                    {
                                        name:"getDefinedDatabases"
                                    },
                                    {
                                        name:"testDB"
                                    },
                                    {
                                        name:"saveDBConfig"
                                    },
                                    {
                                        name:"setDefaultDB"
                                    },
                                    {
                                        name:"importDataSources"
                                    },
                                    {
                                        name:"discoverJNDIDatabases"
                                    }
                                ]
                            }
                        ]
                    })
                    
                    === 2016-09-27 17:58:23,872 [ec-7] WARN  RequestContext - dsRequest.execute() failed: 
                    java.lang.ClassCastException: java.util.ArrayList cannot be cast to java.util.Map
                        at com.isomorphic.datasource.DataSource.getServerObjectConfig(DataSource.java:1225)
                        at com.isomorphic.datasource.DataSourceDMI.validateOperationBinding(DataSourceDMI.java:499)
                        at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:150)
                        at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:64)
                        at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2702)
                        at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:230)
                        at com.lmscompany.lms.server.LMSIDACall.handleDSRequest(LMSIDACall.java:180)
                        at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:187)
                        at com.lmscompany.lms.server.LMSIDACall.processRequest(LMSIDACall.java:89)
                        at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:119)
                        at com.isomorphic.servlet.IDACall.doPost(IDACall.java:79)
                        at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)
                        at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:162)
                        at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
                        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
                        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
                        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
                        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
                        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
                        at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
                        at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:88)
                        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
                        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
                        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
                        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
                        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
                        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
                        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
                        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
                        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
                        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
                        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
                        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
                        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
                        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
                        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
                        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
                        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
                        at java.lang.Thread.run(Unknown Source)
                    === 2016-09-27 17:58:23,873 [ec-7] DEBUG RPCManager - Content type for RPC transaction: text/html; charset=UTF-8
                    === 2016-09-27 17:58:23,874 [ec-7] DEBUG RPCManager - non-DMI response, dropExtraFields: false
                    === 2016-09-27 17:58:23,874 [c-11] DEBUG RPCDMI - rpc returned data
                    === 2016-09-27 17:58:23,874 [ec-7] INFO  Compression - /lms/lms/sc/IDACall: 696 -> 432 bytes
                    === 2016-09-27 17:58:23,874 [c-11] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
                    === 2016-09-27 17:58:23,875 [c-11] INFO  Compression - /lms/lms/sc/IDACall: 67 -> 73 bytes
                    This is my BatchUploadDMI.java (closely following your example):
                    Code:
                    package com.lmscompany.lms.server.worker;
                    
                    import java.util.List;
                    import java.util.Map;
                    
                    import com.isomorphic.datasource.DSRequest;
                    import com.isomorphic.datasource.DSResponse;
                    import com.isomorphic.tools.BatchUpload;
                    
                    public class BatchUploadDMI {
                        @SuppressWarnings({ "unchecked", "rawtypes" })
                        public DSResponse batchUpload(DSRequest dsRequest) throws Exception {
                            BatchUpload batchUpload = new BatchUpload();
                    
                            // parse data and get the result Map
                            DSResponse response = batchUpload.parseUploadData(dsRequest);
                            Map respData = response.getDataMap();
                    
                            // do not proceed to validation if parsing failed
                            if (respData.containsKey("errorMessage"))
                                return response;
                    
                            // get upload data
                            List<Map> uploadData = (List<Map>) respData.get("gridRows");
                    
                            // perform data manipulations
                            for (Map row : uploadData) {
                                if (row.containsKey("ADMINUSER")) {
                                    row.put("ADMINUSER", toBoolean(row.get("ADMINUSER")));
                                }
                            }
                    
                            // validate data and return
                            return batchUpload.validateUploadData(response);
                        }
                    
                        private Boolean toBoolean(Object o) {
                            if (o == null)
                                return null;
                            else
                                switch (o.toString().toLowerCase()) {
                                case "x":
                                case "true":
                                case "ja":
                                case "yes":
                                    return true;
                                }
                            return false;
                        }
                    };
                    This is my batchUpload.ds.xml:
                    Code:
                    <!-- DataSource to support the BatchUploader component
                         NOTE: This is not the DataSource resposible for performing the batch update of verified [B]//"resposible" has a typo[/B]
                               records, it is an internal resource used to handle the initial upload of the client-
                               side file. The only time you would ever want to change this DataSource is if you wanted 
                               a hook into the upload process (for security reasons, for example).  See the 
                               BatchUploader documentation in the SmartClient reference.
                    -->
                    <DataSource ID="batchUpload">
                        <serverObject lookupStyle="new" className="com.lmscompany.lms.server.worker.BatchUploadDMI" />
                        <operationBindings>
                            <operationBinding operationType="add" operationId="upload" serverMethod="batchUpload" />
                            <operationBinding operationType="custom" operationId="wipeData" serverMethod="wipeData" />
                            <operationBinding operationType="loadSchema" />
                        </operationBindings>
                        <serverObject ID="batchUpload" className="com.isomorphic.tools.BatchUpload" dropExtraFields="false">
                            <visibleMethods>
                                <method name="batchUpload" />
                                <method name="wipeData" />
                            </visibleMethods>
                        </serverObject>
                    </DataSource>
                    Just commenting out the DMI in .ds.xml removes the error.
                    If I put a breakpoint in the very top of BatchUploadDMI.java, it is not hit!

                    I'm using v10.1p_2016-09-27.

                    Best regards
                    Blama

                    Comment


                      #11
                      The problem is in batchUpload.ds.xml - you can't have multiple <serverObject> elements at the same level. Remove (comment out) original serverObject declaration if you are replacing it with your own. Let us know how it worked for you.

                      Comment


                        #12
                        Hi Isomorphic,

                        OK, that's embarrassing. I'll fix this, retry and let you know.

                        PS: The docs for visibleMethods don't make you think that this would be something you need / could use in a "normal" SmartGWT application.
                        .ds.xml's operation bindings define what serverMethod can be used / is used and which is not.

                        Best regards
                        Blama

                        Comment


                          #13
                          Hi Isomorphic,

                          first feedback is that in order to have a "default BatchUploader with a nop-DMI", you have to use this DMI:
                          Code:
                          <serverObject [I]ID="batchUpload" lookupStyle="new"[/I] className="com.lmscompany.lms.server.worker.BatchUploadDMI" [B]dropExtraFields="false"[/B]>
                          where the italic parts are optional as I assume (sure for lookupStyle). dropExtraFields must be present, as otherwise the GUI does not show any entries (also no data in the Developer Console's RPC Tab, even though the server log shows the rows with data, validation errors etc).

                          I'll test the new feature itself next.

                          Best regards
                          Blama

                          Comment


                            #14
                            Hi Isomorphic,

                            regarding the feature itself: It seems to be working as expected for me, thanks a lot. I think it is a very valuable addition, as other use cases might benefit from this as well.

                            Further remark, perhaps also as addition for your example code here:
                            I get the "real" DataSource (dataSource:"batchUpload" is used for every upload, regardless of the business object uploaded) via:
                            Code:
                            dsRequest.getValues().toString().contains("myDSName")
                            This might be important in more transformation-orientated use cases than mine is, where you want to handle different "target" DataSources differently.

                            Thank you & Best regards
                            Blama
                            Last edited by Blama; 28 Sep 2016, 04:53.

                            Comment


                              #15
                              The correct way to get dataSource name is shown below:
                              Code:
                               dsRequest.getValues().get("dsName");
                              We added note on this to BatchUpload javadoc also.

                              Comment

                              Working...
                              X