Form's saveData not passing uploaded file to the server after an error

[subashk]

SmartClient Version: v9.0p_2013-11-03/PowerEdition
Browsers: Chrome 33, safari 5.1.7

We are seeing this issue on chorme and safari browsers. IE works fine.

We have a dynamic form with multiple text fields and one field to upload csv files as follows.

{
name: "file",
required: true,
width: 250,
type: "binary",
colSpan: 3
}

We throw parsing Exception in the server, if the uploaded file is not a csv file and set status as 100 and return a message.

In the saveData call back we look for the status code and display the error message as shown below. Once we close the message window, we clear the values and the empty form is displayed.

this.form.saveData(
function(dsResponse, data, dsRequest) {
if(dsResponse.status==100||dsResponse.status==-4){
isc.say(data); windowRef.uploadDynamicForm.clearValues();
}else { //we display a grid.}
}
, {
params: {

}
});


Now if we try to upload the correct csv file and submit, all the values except the file data is available in the server.

As you see, the way we are handling the error in the form is by catching an exception on the server

catch (ParseException e) {
log.error("Issue with uploading file", e);
dsResponse.setData(ERROR_MSG);
dsResponse.setStatus(100);
}
By doing this are we missing anything (clearing out any values) which is causing the file not to be uploaded to the server in subsequent submissions.

Issues noted:

1) If there is any exception thrown, in the subsequent submissions the file Data is not being sent to the server.

[Isomorphic]

If you call clearValues() on a form, the user is going to have to re-choose the CSV file using the native "Browse..." button.

If you intended to say that the user is already doing that, we aren't reproducing a problem with calling clearValues() and then picking a new file, so we'll need a way to reproduce the issue.

[subashk]

Some more notes to the issue. The following is what we see when we inspect the web traffic. The first is when the excel file is uploaded and we throw a parse exception and the 2nd is when we upload the correct csv file.


------WebKitFormBoundaryTuwBKzaBzJvG0cJq
Content-Disposition: form-data; name="file"; filename="ErrorFile.xlsx"
Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet


------WebKitFormBoundaryPhFkD8Hck8rlOmVJ
Content-Disposition: form-data; name="file"; filename=""
Content-Type: application/octet-stream

[subashk]

Yes the user is choosing a new file and entering values in the other fields and upon submit we are seeing the file data not being sent to the user.

Before calling the saveData method , we are validating to see if the user has entered values in all the fields by calling the form.getValue(fieldName) method and all the validations are passing fine.

[Isomorphic]

OK, as we mentioned, causing a validation error and using clearValues() is not enough to reproduce this issue (easily verified here), so we need a way we can reproduce the issue before we can help.

[subashk]

We were able to resolve the issue by setting the type of file to upload instead of binary as shown below.

{
name: "file",
required: true,
width: 250,
type: "upload",
colSpan: 3
}

[Isomorphic]

Hmm.

For the benefit of other readers, no one should do this as it is incorrect and will cause UploadItem to be used instead of FileItem, whereas FileItem is what you want.

Best guess is that this setting helps this particular user by compensating for a mistake made elsewhere.

[subashk]

We definitely do not want to use the wrong properties. But coming back to original problem this was the form that we had.

isc.DynamicForm.create({
autoDraw: false,
dataSource: "uploadDS",
wrapItemTitles: false,
width: 375,
numCols: 3,
colWidths: [200, 250, "*"],
padding:5,
fields: [
{
name: "file",
required: true,
width: 250,
type: "binary",
colSpan: 3
},
{
name:"param1",
title:"Name",
required: true,
width: "250",
},
{
name:"param2",
visible: false,
colSpan: 3
},
{
name: "param3",
title: "Overwrite Existing?",
labelAsTitle: true,
required: true,
type: "checkbox",
colSpan: 3,
visible: false
},
{
name:"param4",
required: true,
title: "field",
editorType:"comboBox",
width:250,
colSpan: 3,
optionDataSource: "dataSource",
completeOnTab: true,
pickListWidth:'*',
displayField:"fullName",
valueField:"userName",
autoFetchData: false,
addUnknownValues: false
}
]
});

In this case when we uploaded an excel file we validated it on the server side and had the server return back an error message and a statusCode which we retrieve in the saveData callBack method to display a message as show below.

this.form.saveData(
function(dsResponse, data, dsRequest) {
if(dsResponse.status==100||dsResponse.status==-4){
isc.say(data);
windowRef.uploadDynamicForm.clearValues();
}else {
//do some thing else.
}
}
, {
params: {

}
});

Now at this point when I inspected the dsRequest.data object this is what I get.

Object {file: "C:\fakepath\ErrorFile.xlsx", param1: "val1", param2: "val2", param3: "val3", param4: "val4"}

Once I dismiss the message box, I am presented with the empty form and I selected the correct csv file and entered the rest of the fields and hit submit, the file is not sent to the server.

Again in the callBack when I inspect the dsRequest.data object this is what I see

Object {0: "E", 1: "r", 2: null, 3: null, 4: null, 5: null, 6: null, 7: null, 8: null, 9: null, 10: null, 11: null, 12: null, 13: null, 14: null, 15: null, 16: null, 17: null, 18: null, 19: null, 20: null, 21: null, 22: null, 23: null, 24: null, 25: null, 26: null, 27: null, 28: null, 29: null, 30: null, 31: null, 32: null, 33: null, 34: null, 35: null, 36: null, 37: null, 38: null, 39: null, 40: null, 41: null, 42: null, 43: null, 44: null, 45: null, 46: null, 47: null, 48: null, 49: null, 50: null, 51: null, 52: null, 53: null, 54: null, 55: null, 56: null, 57: null, 58: null, 59: null, 60: null, 61: null, 62: null, 63: null, 64: null, 65: null, 66: null, file: null, param1: "val1", param2: "val2", param3: "val3", param4: "val4"…}

As you can see a null value for "file" and a bunch of other data which was not there in the initial request.

I tried to set the dsRequest.data to an empty object in the call back after the first call, but that did not help.
==============================================
The following is the code on the server side. The csvStreamProcessor is throwing a parseException in the initial upload (xlsx file) which we catch and return back the status code 100 with the message.

In all the subsequent tries dsRequest.getUploadedFile("file") is empty and this is the issue that we are having.

try {

ISCHttpServletRequest iscHttpServletRequest = dsRequest.context.request;
Map valuesMap = dsRequest.getValues();

String param1 = (String) valuesMap.get("param1");
String param2 = (String) valuesMap.get("param2");
String param3 = (String) valuesMap.get("param3");
String param4 = (String) valuesMap.get("param4");

CSVStreamProcessor csvStreamProcessor = new CSVStreamProcessor();

csvStreamProcessor.processStream(new InputStreamReader(
dsRequest.getUploadedFile("file")
.getInputStream()), fieldMapProcessor);
//bunch of other code

} catch (ParseException e) {
log.error("Issue with uploading ", e);
dsResponse.setData(ERROR_MSG);
dsResponse.setStatus(100);
} catch (IOException e) {
log.error("Issue with uploading ", e);
dsResponse.setData(ERROR_MSG);
dsResponse.setStatus(100);
} catch (ProcessingException e) {
dsResponse.setData(ERROR_MSG);
log.error("Issue with uploading ", e);
dsResponse.setStatus(100);
} catch (Exception e) {
log.error("Issue with uploading ", e);
dsResponse.setData("Issue with uploading " + e.getMessage());
dsResponse.setStatus(100);
}

return dsResponse;
}

As I mentioned earlier this issue was happening only with webKit browsers like chrome and Safari. IE works fine.

Once we changed the type to "upload" from "binary" the issue is resolved, rest of the code being the same.

Basing on the information I provided do you see if we are doing any thing wrong either on the client side or the server side ?

[Isomorphic]

There's the real problem - you are sending back an error message while signalling success (positive number with setStatus()).

The client-side then thinks the String you sent must be the record-as-saved (the standard response for an "add" or "update") and treats it like a big array of field values (the weird 0:"E", 1:"r" etc value you later see).

[subashk]

That change worked. Instead of 100 we are passing -100 when we catch the exception and changed the type of "file" back to "binary".

But one question remains though, with the initial setup,
type : "binary" and status code 100 , we did not see any issue in IE but only in web kit browsers. Any reason why that would have happened ?

[Isomorphic]

No idea, we don't really plan to try to chase that down as it resulted from incorrect usage.