No announcement yet.
  • Filter
  • Time
Clear All
new posts

    Using MultiFileItem with RESTDataSource

    We are using:
    SmartClient Version: v10.0p_2015-09-01/LGPL Development Only (built 2015-09-01)
    Browser: Chrome Version 44.0.2403.157 m (but we have had the same result in IE11 and Firefox)

    Basic data:
    We have a DynamicForm with a RestDataSource attached as the master. In the dynamicForm we have added a MultiFileItem field pointing to the detail datasource that contains only an id as primary key, a binary field and a foreign key pointing to the master datasource that is attached to the DynamicForm.

    Add a new register in the table. Pick up some files to be uploaded and submit the form.

    What happens:
    We receive the call in the REST web service and completes successful as can be seen in the RPC tab of the developer console. Later on we receive a call to the transformRequest of the detail datasource and when leaving this method happens two different things:
    1) We receive the following message in the developer console:

    11:56:44.651:XRP4:WARN:Log:TypeError: Cannot read property 'transactionNum' of undefined
    Stack from error.stack:
    MultiFilePicker.saveData(<no args: exited>) on [MultiFilePicker ID:isc_MultiFileItem_1_picker] @ ISC_Forms.js:2258:210
    MultiFileItem.formSaved(<no args: exited>) on [MultiFileItem ID:isc_MultiFileItem_1 name:attachment] @ ISC_Forms.js:2243:1834
    DynamicForm.formSavedComplete(<no args: exited>) on [DynamicForm ID:isc_DynamicForm_0] @ ISC_DataBinding.js:2260:317
    DynamicForm.$49z(<no args: exited>) on [DynamicForm ID:isc_DynamicForm_0] @ ISC_DataBinding.js:2260:55
    RestDataSource.eval(<no args: exited>) on [RestDataSource ID:RESTDailyLogBookDS] @ [no file]:3:19
    [c]Class.fireCallback(_1=>"isc.Comm.performXmlTransactionReply(1, x..."[54], _2=>"xmlHttpRequest", _3=>Array[1], _4=>[object Window], _5=>true) @ ISC_Core.js:268:49
    [a]RestDataSource.fireCallback(<no args: exited>) on [RestDataSource ID:RESTDailyLogBookDS] @ ISC_Core.js:336:302
    RestDataSource.fireResponseCallbacks(<no args: exited>) on [RestDataSource ID:RESTDailyLogBookDS] @ ISC_DataBinding.js:705:13
    RestDataSource.completeResponseProcessing(<no args: exited>) on [RestDataSource ID:RESTDailyLogBookDS] @ ISC_DataBinding.js:702:6
    RestDataSource.handleJSONReply(<no args: exited>) on [RestDataSource ID:RESTDailyLogBookDS] @ ISC_DataBinding.js:593:6

    2) We receive the request to the servlet that handles the file upload (we receive one call per file) but this call has an encoding text/xml despite the fact that the dynamicForm was configured with an encondign Multipart, so we cannot recover the file being uploaded and the rest of the process cannot complete.

    Any ideas where the problem can be located.
    Thank you very much.
    Attached Files

    From a look at the code, this suggests a possible invalid transformRequest() implementation that fails to return the dsRequest data as specified in the docs.

    If that doesn't appear to be the problem, we'll probably need a test case to look further. You should be able to build a test case fairly quickly by starting from SDK samples, which include DataSources that have binary fields, since all server-side DataSources (declared via .ds.xml files) support the RestDataSource protocol via the RestHandler servlet.


      I'll try to give you an small test case but just to confirm. RestHandler is from the EE side? We are using the LGPL version so I think we cannot use it.


        After checking what you sugested maybe I'm wrong but I think the problem is not caused by my implementation of the dsRequest.

        Here it goes a test case built using Eclipse. I have removed all the library but are easily found. The packages needed a part from GWT and SmartGWT, are:

        The FileUploadServlet class is not completelly implemented is just to check that when we fill the form adding one or two files and click the submit button, the REST add operation on the Master DS is completed and after that the call to the detailDS servlet to upload the file is done but not as multipart but as text/xml

        Thank you very much for your help.

        Attached Files


          Sorry, but an Eclipse project with a dozen different Java files is not acceptable as a test case. The problem could be anywhere. We only accept minimal test cases.

          The RestHandler servlet is indeed part of Pro and above, however, you can use the free Evaluation version to build a minimal test case, which will then be a valid bug report we can look at. Or, the process of doing so may reveal a problem in your application code, as often happens.


            I believe what you said. The most probable is that I'm responsible of the problem and is not a bug in the SmartGWT library so I'm gonna take a look again and try to build the same example using the PRO version but if you can take a look at least to the data echanged in the two requests that would be lovely and can point me to the right direction. Find the data attached as two txt files.

            One stupid question just in case. I have seen that the Javadoc of the dynamicForm.setEncoding method says: <cite>encoding for the form, use MULTIPART_ENCODING for file upload forms</cite> but as far as the method accepts an Encoding object we cannot use the value said. I think it won't be the cause at all but just to poiint this out because one never knows where the problem is.

            A part from the txt attachements I've attached again the test case after removing some unused classes (sorry about that) and refactoring some others to have a cleaner test. On top of that I clarify the contents of the different packages later on this post. I'll try the move to the PRO version but don't you think that it could introduce so many changes and can execute a differnt part of the library or load data in a different way leading may be to a different behaviour that is expressed in the LGPL version and not in the PRO?

            Explanation of the test case packages:
            1. com.testcases.smartgwt.multifileitem.client.ds- This package contains the master/detail DS definitions and a bean that will hold the information definined by the Master DS.
            2. com.testcases.smartgwt.multifileitem.client.jsonHelpClasses - This package contains only java classes to help Jackson to generate the JSON that we are gonna exchange between smartgwt client side and the backend. And is not needed to be checked
            3. com.testcases.smartgwt.multifileitem.fileUploadServlet - This contains the servlet class used to receive the files uploaded. The only thing to be checked in the class is that when the request arrives is not marked as a multipart
            4. com.testcases.smartgwt.multifileitem.restProxy - This package contains the REST web service that implements the fetch, add, update, remove operations. This web service will be called by the masterDS

            Thank you very much again
            Attached Files


              The test case relies on lots of third party libraries which have not been included and as such is not runnable. However taking the code you've provided and modifying it slightly (removing all the CFX rest code) and replacing it with static responses we can see that the browser request is indeed sent as a multipart request. If this is not what happens for you then we suggest that you narrow down the test case even more and tell us what you expect to happen and what you believe is wrong.

              In regards to the documentation of encoding, thanks for pointing that out, we'll take a look at that.

              Request headers

              Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
              Accept-Encoding: gzip, deflate
              Accept-Language: en-GB,en;q=0.8
              Cache-Control: no-cache
              Connection: keep-alive
              Content-Length: 144
              Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryh69c5LlUIFsxkuxZ
              Host: localhost:8080
              Origin: http://localhost:8080
              Pragma: no-cache
              Referer: http://localhost:8080/
              Upgrade-Insecure-Requests: 1
              User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.13 Safari/537.36


                Thank you very much for the effort you took to check this test case.

                I gues the request you've pasted into the response is related to the second request, the one that is in charge of the upload. And definetly is using multipart.

                Can you post the static data you've used, because I guess that in this point is where my problem is located. I have double checked my attachment [COLOR=#d95b46] [/COLOR]002_fileupload_request.txt[COLOR=#d95b46] [/COLOR](1.6 KB, 1 view) of the previous post to confirm that I was not wrong and the content-type wasn't multipart.

                Thank you very much.


                  Yes, that's right, the second request is an HTTP multi part request to the fileuploadservlet. However what you should be doing is removing the dynamicForm.setEncoding() call. Don't set this to MULTIPART as you don't want the entire form to be sent as a multi part request only the file upload to the detail DS which is done automatically for you. In the testing we did, it does not matter what kind of data we uploaded, it works either way. Tested, PNG, PDF, TXT and CSV, all which worked fine. However you will need to correctly implement your file upload servlet to handle the data and return an appropriate response.

                  As a side note to this issue, you should also avoid using GWT standard widgets with SmartGWT widgets, so remove the following line of code.

                  You can safely remove this as SmartGWT will automatically draw itself when you call the draw() method on your top component.


                    I have removed as you suggested, both the setEncoding and the RootPanel.get().add(...) but there is no change, the first call to store the register in the main DS completes ok but the second call to store the attachment is received in the servlet but is not multipart.

                    I cannot understand why the component is not sending the request as multipart as you said the component must do it automatically for me.

                    Can you please provide the static data you used to replace my server side? Hopefully I could find the explanation of the problem there.

                    Thank you very much


                      I have removed my server side of the first request to avoid problems in my code. Attached to this message you could find the JSONs used to complete the fetch, add and update requests from the main datasource.

                      The request for the detail datasource is still being done without using the multipart. encoding type. Just for remember I have removed the setEncoding from the dynamic form and GWT standard widgets from the code.

                      What's different from my static files from yours that makes the component build the request to upload the details in such a different way.

                      Thank you very much again.
                      Attached Files


                        I have been digging a little bit more in the differences trying to understand how you were able to make my test case to run just replacing the backend by static files. But I'm not able to understand how this can be achieved by the time.

                        Correct me if I'm wrong but I'm expecting that the client side components will handle the transactions automatically. I mean the MultiFilePicker will wait for the master register to be stored and then will build automatically a transaction to send all the files attached.

                        This is not happening and I have checked the difference between the requests sent by the client side components in the EE sample and in the components using the JSON approach and they are quite diffrents because the EE sample it's adding the transaction elements. Must I add manually this elements in the JSON approach?

                        Request from EE component to add the main register:
                        <transaction xmlns:xsi=""
                        <transactionNum xsi:type="xsd:long">1</transactionNum>
                        <operations xsi:type="xsd:List">
                        <elem xsi:type="xsd:Object">
                        <values xsi:type="xsd:Object">
                        <operationConfig xsi:type="xsd:Object">
                        <repo xsi:nil="true" />
                        <oldValues xsi:type="xsd:Object"></oldValues>

                        Request from the JSON object to add the master register:

                        Thank you very much


                          After some tests that I'm gonna explain I have found something that may be the cause of the behaviour I have reported.

                          First of all I tried to determine what was wrong in my static files in the backend as you said I must receive a multipart request if I put the right content into this files. But despite reading the docs I haven't been able to achieve it. The test always shows the same error: Uncaught TypeError: Cannot read property 'transactionNum' of undefined. I cannot understand which data must be put into the static files to achieve that the components works properly.

                          After that I decided to use the FileItem instead of the MultiFileItem just in case it was easier to use. But I have not been able to make this component launch a multipart request neither. In this component as the dinamic form contains the FileItem I used the Encoding.MULTIPLE instead of the Encoding.NORMAL.

                          After checking the SmartGWT SDK source code I have found in DynamicForm.js the property MULTIPART_ENCODING that is mapped to "multipart/form-data" and the property MULTIPART mapped to "multipart". And I remembered that I told you that the Javadoc in the DynamicForm.setEnconding method was speaking about a value (MULTIPART_ENCODING) that was not in the DSEncoding enum. I thought that was a problem in the Javadoc but now I think is a problem in the code related with this.

                          To do a small check isolating pieces, I have written an HTML with two forms. Each form contains only an input type file but one form has the enctype to multipart and the other to multipart/form-data. The results after submitting each form are:
                          1. The form with the multipart enctype never launches a Multipart request.
                          2. The form with the multipart/form-data launches a Multipart request.
                          Can you help me to solve this. Seems that the LGPL version of the library is doing something strange and is not configuring properly the property enctype of the form element to upload the files.

                          The HTML used to test the behaviour of the different enctypes:
                          <!DOCTYPE html>
                          <html lang="en">
                          <form method="POST" action="fileUploadServlet" enctype="multipart">
                          <div>NOT working version (This version doesn't produces a multipart
                          <input type="file" name="file" id="file" />
                          <input type="submit" value="Upload" />
                          <br />
                          <form method="POST" action="fileUploadServlet"
                          <div>Working version (This version produces a multipart request):</div>
                          <input type="file" name="file" id="file" /> <input
                          type="submit" value="Upload" />