Announcement

Collapse
No announcement yet.
X
Collapse
  • Page of 1
  • Filter
  • Time
Clear All
new posts
Previous template Next
    #1

    5.1 BatchUploader quoting issues

    Hi Isomorphic,

    I found some quoting issues with v10.1p_2016-03-27 BatchUploader that are not shown to the user as they don't make it to the client.

    Please use the 1st two items from the csv in this sample and this testcase:

    BuiltInDS.java:
    Code:
    package com.smartgwt.sample.client;

import com.google.gwt.core.client.EntryPoint;
import com.smartgwt.client.core.KeyIdentifier;
import com.smartgwt.client.data.DataSource;
import com.smartgwt.client.util.Page;
import com.smartgwt.client.util.PageKeyHandler;
import com.smartgwt.client.util.SC;
import com.smartgwt.client.widgets.BatchUploader;
import com.smartgwt.client.widgets.IButton;
import com.smartgwt.client.widgets.Window;
import com.smartgwt.client.widgets.events.ClickEvent;
import com.smartgwt.client.widgets.events.ClickHandler;
import com.smartgwt.client.widgets.grid.ListGridField;
import com.smartgwt.client.widgets.layout.VLayout;

public class BuiltInDS implements EntryPoint {
    public void onModuleLoad() {
        KeyIdentifier debugKey = new KeyIdentifier();
        debugKey.setCtrlKey(true);
        debugKey.setKeyName("D");

        Page.registerKey(debugKey, new PageKeyHandler() {
            public void execute(String keyName) {
                SC.showConsole();
            }
        });

        VLayout mainLayout = new VLayout(20);
        mainLayout.setWidth100();
        mainLayout.setHeight100();

        IButton recreateBtn = new IButton("Recreate");
        recreateBtn.addClickHandler(new ClickHandler() {
            @Override
            public void onClick(ClickEvent event) {
                recreate();
            }
        });
        mainLayout.addMember(recreateBtn);
        recreate();
        mainLayout.draw();
    }

    private void recreate() {
        Window w = new Window();
        w.setWidth("95%");
        w.setHeight("95%");
        w.setMembersMargin(0);
        w.setModalMaskOpacity(70);
        w.setTitle("Import data");
        w.setShowMinimizeButton(false);
        w.setIsModal(true);
        w.setShowModalMask(true);
        w.centerInPage();

        BatchUploader batchUploader = new BatchUploader();
        batchUploader.setWidth100();
        batchUploader.setUploadDataSource(DataSource.get("supplyItem"));
        batchUploader.setDefaultDelimiter(";");
        batchUploader.setDefaultQuoteString("\"");

        ListGridField itemName = new ListGridField("itemName");

        ListGridField sku = new ListGridField("SKU");

        ListGridField category = new ListGridField("category");

        ListGridField unitCost = new ListGridField("unitCost");

        batchUploader.setGridFields(itemName, sku, category, unitCost);

        w.addItem(batchUploader);
        w.show();
    }
}
    Issue 1:
    As you can see, the BatchUploader uses ";" as delimiter, while the data uses ",".
    This of course can't work, but is not reported to the user. The server throws an exception and does not handle it:
    Code:
    === 2016-03-31 20:26:03,730 [1-35] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
    },
    values:{
        file:"supplyItemTest.csv",
        dsName:"supplyItem",
        delimiter:";",
        quoteString:"\"",
        dataFormat:"csv"
    },
    operationConfig:{
        dataSource:"batchUpload",
        repo:null,
        operationType:"update",
        textMatchStyle:"exact"
    },
    componentId:"isc_DynamicForm_4",
    appID:"builtinApplication",
    operation:"upload",
    oldValues:{
        dsName:"supplyItem",
        delimiter:";",
        quoteString:"\"",
        dataFormat:"csv",
        file:null
    }
}
=== 2016-03-31 20:26:03,730 [1-35] INFO  IDACall - Performing 1 operation(s)
=== 2016-03-31 20:26:03,730 [1-35] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2016-03-31 20:26:03,730 [1-35] DEBUG DeclarativeSecurity - DataSource batchUpload is not in the pre-checked list, processing...
=== 2016-03-31 20:26:03,730 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'supplyItem'
=== 2016-03-31 20:26:03,731 [1-35] DEBUG SQLDataSource - DataSource 159 acquired SQLDriver instance 72106219 during initialization
=== 2016-03-31 20:26:03,731 [1-35] DEBUG PoolableDataSourceFactory - Created DataSource 159 of type 'supplyItem' and assigned it to thread qtp1386020581-35
=== 2016-03-31 20:26:03,731 [1-35] DEBUG PoolableDataSourceFactory - Created DataSource 159 of type 'supplyItem' in the pooling flow
=== 2016-03-31 20:26:03,731 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 159 of type 'supplyItem'
=== 2016-03-31 20:26:03,731 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'supplyItem'
=== 2016-03-31 20:26:03,731 [1-35] DEBUG SQLDataSource - DataSource 160 acquired SQLDriver instance 683215103 during initialization
=== 2016-03-31 20:26:03,731 [1-35] DEBUG PoolableDataSourceFactory - Created DataSource 160 of type 'supplyItem' and assigned it to thread qtp1386020581-35
=== 2016-03-31 20:26:03,731 [1-35] DEBUG PoolableDataSourceFactory - Created DataSource 160 of type 'supplyItem' in the pooling flow
=== 2016-03-31 20:26:03,731 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 160 of type 'supplyItem'
=== 2016-03-31 20:26:03,731 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:26:03,731 [1-35] DEBUG PoolableDataSourceFactory - Created DataSource 161 of type 'Object' and assigned it to thread qtp1386020581-35
=== 2016-03-31 20:26:03,731 [1-35] DEBUG PoolableDataSourceFactory - Created DataSource 161 of type 'Object' in the pooling flow
=== 2016-03-31 20:26:03,731 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] ERROR DataImport - Column units had unknown column type enum
=== 2016-03-31 20:26:03,732 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] ERROR DataImport - Column inStock had unknown column type boolean
=== 2016-03-31 20:26:03,732 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 161 of type 'Object'
=== 2016-03-31 20:26:03,732 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 161 of type 'Object'
java.text.ParseException: Delimiter or end of line expected after quoted value=== 2016-03-31 20:26:03,733 [1-35] DEBUG BatchUpload - Rows: [
]
=== 2016-03-31 20:26:03,733 [1-35] DEBUG BatchUpload - Errors: [
]
=== 2016-03-31 20:26:03,733 [1-35] DEBUG BatchUpload - Errors after conversion: [
    "Delimiter or end of line expected after quoted value\n\nSee the log for more details about this error."
]
=== 2016-03-31 20:26:03,733 [1-35] INFO  DSResponse - DSResponse: Map with 3 keys
=== 2016-03-31 20:26:03,733 [1-35] DEBUG DSRequest - About to free up resources for request of type update on DataSource batchUpload
=== 2016-03-31 20:26:03,733 [1-35] DEBUG DSRequest - Ignoring freeResources call because this is not a primary request!
=== 2016-03-31 20:26:03,733 [1-35] DEBUG RPCManager - Content type for RPC transaction: text/html; charset=UTF-8

    at com.isomorphic.tools.DataImport.splitRow(DataImport.java:1439)
    at com.isomorphic.tools.DataImport.importToRows(DataImport.java:1058)
    at com.isomorphic.tools.DataImport.importAndValidateDSRows(DataImport.java:444)
    at com.isomorphic.tools.DataImport.importAndValidateDSRows(DataImport.java:384)
    at com.isomorphic.tools.BatchUpload.internalBatchUpload(BatchUpload.java:148)
    at com.isomorphic.tools.BatchUpload.batchUpload(BatchUpload.java:88)
    at sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.isomorphic.base.Reflection.adaptArgsAndInvoke(Reflection.java:964)
    at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:414)
    at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:64)
    at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2690)
    at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:228)
    at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:187)
    at com.isomorphic.servlet.IDACall.processRequest(IDACall.java:152)
    at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:119)
    at com.isomorphic.servlet.IDACall.doPost(IDACall.java:79)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
    at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:162)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494)
    at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
    at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:88)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:68)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:370)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
    at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Unknown Source)
=== 2016-03-31 20:26:03,733 [1-35] DEBUG RPCManager - DMI response, dropExtraFields: false
=== 2016-03-31 20:26:03,733 [1-35] DEBUG DSRequest - Ignoring freeQueueResources call because this is not a primary request!
=== 2016-03-31 20:26:03,734 [1-35] INFO  Compression - /builtinds/sc/IDACall: 811 -> 502 bytes
    Issue 2:
    Please change the BatchUploader separator to "," (like the csv file) and remove the " at the end of "Small grey widget" (now: "Small grey widget).
    This outputs:
    Code:
    === 2016-03-31 20:22:36,324 [1-35] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
    },
    values:{
        dsName:"supplyItem",
        delimiter:",",
        quoteString:"\"",
        dataFormat:"csv",
        file:"supplyItemTest.csv"
    },
    operationConfig:{
        dataSource:"batchUpload",
        repo:null,
        operationType:"update",
        textMatchStyle:"exact"
    },
    componentId:"isc_DynamicForm_3",
    appID:"builtinApplication",
    operation:"upload",
    oldValues:{
        dsName:"supplyItem",
        delimiter:",",
        quoteString:"\"",
        dataFormat:"csv",
        file:null
    }
}
=== 2016-03-31 20:22:36,324 [1-35] INFO  IDACall - Performing 1 operation(s)
=== 2016-03-31 20:22:36,324 [1-35] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2016-03-31 20:22:36,324 [1-35] DEBUG DeclarativeSecurity - DataSource batchUpload is not in the pre-checked list, processing...
=== 2016-03-31 20:22:36,325 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'supplyItem'
=== 2016-03-31 20:22:36,325 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 144 of type 'supplyItem'
=== 2016-03-31 20:22:36,325 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'supplyItem'
=== 2016-03-31 20:22:36,326 [1-35] DEBUG SQLDataSource - DataSource 149 acquired SQLDriver instance 493549059 during initialization
=== 2016-03-31 20:22:36,326 [1-35] DEBUG PoolableDataSourceFactory - Created DataSource 149 of type 'supplyItem' and assigned it to thread qtp1386020581-35
=== 2016-03-31 20:22:36,326 [1-35] DEBUG PoolableDataSourceFactory - Created DataSource 149 of type 'supplyItem' in the pooling flow
=== 2016-03-31 20:22:36,326 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 149 of type 'supplyItem'
=== 2016-03-31 20:22:36,326 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:22:36,326 [1-35] DEBUG PoolableDataSourceFactory - Created DataSource 150 of type 'Object' and assigned it to thread qtp1386020581-35
=== 2016-03-31 20:22:36,326 [1-35] DEBUG PoolableDataSourceFactory - Created DataSource 150 of type 'Object' in the pooling flow
=== 2016-03-31 20:22:36,326 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,326 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,326 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:22:36,326 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,326 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,326 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] ERROR DataImport - Column units had unknown column type enum
=== 2016-03-31 20:22:36,327 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] ERROR DataImport - Column inStock had unknown column type boolean
=== 2016-03-31 20:22:36,327 [1-35] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Activated DataSource 150 of type 'Object'
=== 2016-03-31 20:22:36,327 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 150 of type 'Object'
[B]=== 2016-03-31 20:22:36,327 [1-35] ERROR DataImport - Malformed line 2: java.text.ParseException: Delimiter or end of line expected after quoted value[/B]
=== 2016-03-31 20:22:36,329 [1-35] DEBUG ValidationContext - Adding validation errors at path '/supplyItem/units/units': {errorMessage=Must be one of [Roll, Ea, Pkt, Set, Tube, Pad, Ream, Tin, Bag, Ctn, Box].}
=== 2016-03-31 20:22:36,329 [1-35] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 149
=== 2016-03-31 20:22:36,329 [1-35] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 149
=== 2016-03-31 20:22:36,329 [1-35] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 149 of type 'supplyItem'
=== 2016-03-31 20:22:36,329 [1-35] DEBUG BatchUpload - Rows: [
    {
        itemName:"Large grey widget",
        unitCost:0.41,
        units:"EACH",
        category:"Audio",
        SKU:"45300",
        nextShipment:new Date(1232492400000)
    }
]
=== 2016-03-31 20:22:36,330 [1-35] DEBUG BatchUpload - Errors: [
    {
        recordPath:"/supplyItem",
        units:{
            errorMessage:"Must be one of [Roll, Ea, Pkt, Set, Tube, Pad, Ream, Tin, Bag, Ctn, Box]."
        },
        rowNum:0
    }
]
=== 2016-03-31 20:22:36,330 [1-35] DEBUG BatchUpload - Errors after conversion: [
    {
        errors:{
            units:[
                "Must be one of [Roll, Ea, Pkt, Set, Tube, Pad, Ream, Tin, Bag, Ctn, Box]."
            ]
        },
        rowNum:0
    }
]
=== 2016-03-31 20:22:36,330 [1-35] INFO  DSResponse - DSResponse: Map with 3 keys
=== 2016-03-31 20:22:36,330 [1-35] DEBUG DSRequest - About to free up resources for request of type update on DataSource batchUpload
=== 2016-03-31 20:22:36,330 [1-35] DEBUG DSRequest - Ignoring freeResources call because this is not a primary request!
=== 2016-03-31 20:22:36,330 [1-35] DEBUG RPCManager - Content type for RPC transaction: text/html; charset=UTF-8
=== 2016-03-31 20:22:36,330 [1-35] DEBUG RPCManager - DMI response, dropExtraFields: false
=== 2016-03-31 20:22:36,330 [1-35] DEBUG DSRequest - Ignoring freeQueueResources call because this is not a primary request!
=== 2016-03-31 20:22:36,330 [1-35] INFO  Compression - /builtinds/sc/IDACall: 933 -> 596 bytes
    See the "Malformed line 2" in the log, which is catched, but not displayed to the user.
    A larger file with an unknown row-count could easily be thought of as imported successfully, as the removal of the erroneous line in the data returned to the client is not reported.

    Best regards
    Blama
    Tags: None
    #2
    Hi Isomorphic,

    can you reproduce this one?

    Best regards
    Blama

    Comment

      #3
      In order to improve the error messages that are shown to the user during the uploading process, various i18n messages haven been added.

      You will see specific errors when the first row has problems. For errors in other rows, you will see a warning message with information to help you to identify where the errors are.

      Please try the next nightly build, dated April 12.

      Regards
      Isomorphic Software

      Comment

        #4
        Hi Isomorphic,

        I just retested with the new version. The new reporting is working great.
        I was able to find some (minor) edge cases though I'll report soon.

        Thanks a lot,
        Blama

        Comment

          #5
          Hi Isomorphic,

          here the testcases. Please try the attached csv files with the BuiltInDS.java from #1:
          1. 1st row emtpy: "-1" reported
          2. (Minor) Quoting issue reporting
            Reads: Some rows could not be parsed; the grid below shows 9 of 11. Row number 3 was the first row to fail to be parsed.
            Better: Some rows could not be parsed; the grid below shows 9 of 11 data rows. Row number 3 was the first row to fail to be parsed. (Is also not localized)
          3. Row is there even though being reported as errorneous
          4. More data fields than expected (the one you found here)
          Best regards
          Blama
          Attached Files

          Comment

            #6
            These issues have been solved. Please try the next nightly build, dated April 27.

            Regards
            Isomorphic Software

            Comment

              #7
              Hi Isomorphic,

              I can confirm the the issues from #5 are fixed as well using v10.1p_2016-04-27.

              Thank you & Best regards
              Blama
              Last edited by Blama; 27 Apr 2016, 08:25. Reason: c&p error with version string

              Comment

                #8
                Hi Isomorphic,

                there is another CSV parsing bug:
                Linebreaks in CSV files are allowed and have to be inside the used quote-character (Chapter 2.6 here), like in this testfile for the current showcase example (now on v11.0p_2017-01-30, but also happening for me using 5.1p).
                Code:
                "nextShipment","category","itemName","itemID","unitCost","description","SKU","inStock","units"
[B]"2009-01-21","Audio","Small grey widget line1
SGW line2",,0.41,"","545315","","Ea"[/B]
"2009-01-21","Audio","Large grey widget",,0.41,"","45300","","EACH"
"2008-11-17","Pencils","Small blue gizmo",,-0.47,"","90600","","Ea"
"","Waste Bins","Large green gizmo",,0.61,"","135900","","Packet"
"","String","Small red widget",,0.58,"","190000","","Ea"
"2014-11-17","String","Small piece of string",,0.22,"","90600","","Ea"
"2012-03-11","String","More string",,0.58,"","1900111","","Roll"
                This results in:
                Some rows could not be parsed; the grid below shows 7 of 8. Row number 2 was the first row to fail to be parsed.
                This is an important one for me, as a potential new and currently testing customer hit this and reported it back to me.

                Best regards
                Blama

                Comment

                  #9
                  Hi Isomorphic,

                  also, the import has problems when the separator-char is in the quoted data, like here in row 2:
                  Code:
                  "nextShipment","category","itemName","itemID","unitCost","description","SKU","inStock","units"
"2009-01-21","Audio","Small grey[B],[/B] widget",,0.41,"","545315","","Ea"
"2009-01-21","Audio","Large grey widget",,0.41,"","45300","","EACH"
"2008-11-17","Pencils","Small blue gizmo",,-0.47,"","90600","","Ea"
"","Waste Bins","Large green gizmo",,0.61,"","135900","","Packet"
"","String","Small red widget",,0.58,"","190000","","Ea"
"2014-11-17","String","Small piece of string",,0.22,"","90600","","Ea"
"2012-03-11","String","More string",,0.58,"","1900111","","Roll"
                  See the comma after "Small grey" in row 2.

                  It results in:
                  Some rows could not be parsed; the grid below shows 6 of 7. Row number 2 was the first row to fail to be parsed.
                  Is is also a very important one for me, as it was reported by the customer as well.

                  Best regards
                  Blama

                  Comment

                    #10
                    Hi Isomorphic,

                    any update on this one?

                    Best regards
                    Blama

                    Comment

                      #11
                      Bugs reported in posts #8 and #9 are fixed and will be available in nightly builds since Feb 4 (tomorrow). Thank you for your patience.

                      Comment

                        #12
                        Great! Thank you. I'll try it ASAP.

                        Best regards
                        Blama

                        Comment

                          #13
                          Hi Isomorphic,

                          I'm using v10.1p_2017-02-04 now and the bug seems to be gone, both in the showcase and in my application.

                          Thanks a lot
                          Blama

                          Comment

                          Previous template Next
                          Working...
                          X