Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

    6.1p: Bug with double linefeeds in CSV data and Batch data upload

    Hi Isomorphic,

    The normal clientside Batch uploader (v11.1p_2018-11-15) has a similar problem like the one reported here.

    Please see the 6.1p Batch Uploader sample and the supplyItemTestGood.csv.txt and supplyItemTestBad.csv.txt files.
    Linefeeds are uploaded correctly from the good file (like defined here), but empty rows (double-linefeeds) from the bad file are recognized as new data row, which is a bug.

    This is a problem here, as a customer wants to upload data exported from a CRM where the comment field has double-linefeeds.

    Code:
    isc.BatchUploader.create({
        ID:"uploader",
        height: 400,
        width: "100%",
        uploadDataSource: supplyItemCustom,
        // BatchUploader supports extra fields, which are uploaded along with the file 
        // data and can be accessed from the HttpSession on the server side; here we
        // define some fields to be used in this way
        uploadFormFields: [
            { name: "stringValue", title: "String Value", type: "text", wrapTitle: false },
            { name: "numericValue", title: "Numeric Value", type: "number", wrapTitle: false }
        ],
        uploadFormProperties: {
            titleWidth: 110 // make room for long titles defined above without overflowing
        },
        // We have limited vertical screen space when running in the Feature Explorer, 
        // so we force the embedded grid to be quite short in an attempt to avoid 
        // the need for page scrolling
        gridProperties: {
            height: 200
        },
        gridFields :[
            {name:"itemID"},
            {name:"itemName", editorType: "TextAreaItem"}, [B]// Adjusted, only relevant change[/B]
            {name:"SKU"},
            {name:"description"},
            {name:"category"},
            {name:"units"},
            {name:"unitCost"},
            {name:"inStock"},
            {name:"nextShipment"}
        ]
    });
    
    isc.DynamicForm.create({
        ID: "partialCommitsForm",
        items: [
            {
                name:"partialCommits", 
                wrapTitle: false,
                title:"Partial Commit Mode", 
                type:"select", 
                defaultValue: "prompt",
                valueMap:{
                    allow: "Allow", 
                    prevent: "Prevent", 
                    prompt: "Prompt",
                    retain: "Retain"
                },
                changed : function(form, item, value) {
                    uploader.partialCommit = value;
                }
            }
        ]
    });
    
    isc.VStack.create({
        width:"100%",
        layoutMargin: 10,
        membersMargin:10,
        members:[
            partialCommitsForm,
            uploader
        ]
    });
    Click image for larger version

Name:	Double LineFeed issue.PNG
Views:	116
Size:	33.7 KB
ID:	255854

    Best regards
    Blama
    Attached Files

    #2
    This seems to be the same issue previously discussed with you - embedded linefeeds must be escaped or this is not a valid CSV.

    Comment


      #3
      Hi Isomorphic,

      yes, started to discuss it here, but not to the end, as you introduced the ImportDataSourceRecord(Map,...) and ImportToDataSource(Map,...) APIs shortly afterwards.
      Also and more important I was responsible for creating the CSV data there and could have easily done what you suggested (=escaping linefeeds).

      Now we are talking about a client side Batch Upload of a Libre Office (or Excel) generated CSV file. Here those programs will generate the file and I can't interfere.
      To generate Linefeeds press Alt-Enter or Shift-Enter inside a cell. Then safe the file as CSV and open it in Notepad++ or similar to see the result.
      With my "Good" file you can see that the unescaped linefeeds do indeed work already - just double click the row and see the TextAreaItem field. There is only a problem with double linefeeds.
      Also, it seems that Libre Office / Excel are correct in the way they generate the CSV. Please see the RFC here and in the document the definition of escaped on page 2.

      Best regards
      Blama

      Comment


        #4
        This is fixed and available for download in latest nightly builds.

        Comment


          #5
          Hi Isomorphic,

          I can see this is changed in v11.1p_2018-11-29 with the testcase from #1 and the supplyItemTestBad.csv.txt file.

          A minor issue is though that it seems you are merging many successive line feeds into one. Please see the attached demo file supplyItemTestNoEmptyRow.csv.txt.

          Best regards
          Blama
          Attached Files

          Comment


            #6
            Indeed we did and that was a bug, which is now fixed and available for download in latest nightly builds.

            Comment


              #7
              Hi Isomorphic,

              I can see this is fixed now using the file from #5 and v11.1p_2018-12-04 in the online showcase.

              Thank you
              Blama

              Comment

              Working...
              X