Problem with FileUpload in SGWTPro 12.1p

[stonebranch2]

Hi Isomorphic ,

We have an serious issue relating to the upgraded libraries.

All file uploads stopped functioning after applying the new libraries.

The raw response for the request shows [Request in Progress].

Thanks

[Isomorphic]

Hi!
We're not seeing a general problem with file uploads in the latest SmartGWT 12.1p EE build. The File Upload samples in the showcase appear to be working as expected, and our internal automated testing didn't catch any problems with this build. The changes we made to form item values handling specifically for this thread also seem unlikely to cause file upload to fail.

Obviously this is a stable production branch and we take regressions like this seriously and want to get them resolved as quickly as possible

Can you show us a way to reproduce the problem either starting with one of our built in samples or by showing us the relevant snippets of your code?

Also worth knowing:
- which browser / OS are you seeing this on
- what is the full SmartGWT version string for the build that has the problem
- what is the full SmartGWT version string you were using before (where this works)

Thanks
Isomorphic Software

[stonebranch2]

Hi Isomorphic,

Thanks for the quick response.

We are pretty sure there is something here, as we started getting notices from internal users once we merged the new libraries.

Here is the build we were using prior.

SmartClient Version: v12.1p_2022-01-29/Pro Deployment (built 2022-01-29)

We tried the showcase and see it is working. In our case, we are uploading a zip file.

We see the problem on Chrome and Firefox, windows.

Regards

[Isomorphic]

Hi Stonebranch
We believe you that something is going on, but we'll need some more information since this doesn't seem like a general breakage of file upload. (It seems that you've confirmed this for your specific build since file upload is working in the showcase).

Ultimately we may need to see a full standalone test case that demonstrates the problem, but to start with can you share whatever information may be relevant about the uploads that are failing. This will give us a starting point for investigation.
- How are the uploads kicked off (can you share the code that builds the relevant UI?)
- What is your storage mechanism? SQL backed server side dataSource or something else? Can you share the field definition for the datasource in question.
- You mention that the upload is failing with zip files. Are other uploads working?
- Are you seeing any warnings or errors in the browser console, the SmartGwt developer console, or the server side logs that may be relevant?

Thanks

[stonebranch2]

Hi Isomorphic,

Still collecting information, will follow up soon as we can.

Thanks

[stonebranch2]

Some additional details.

User clicks an import button, form with FileItem presented, they add the file, and submit.

Application hangs with spinning pointer.

We don't believe the upload is even happening, as we do not see any indication the entry point on server is hit.

 
            DSRequest requestProperties = new DSRequest();
            requestProperties.setWillHandleError(Boolean.TRUE);

            form.saveData((dsResponse, data, dsRequest) -> {
                if (dsResponse.getStatus() == RPCResponse.STATUS_SUCCESS) {
                   // success
                } else {
                  // error
                }
            }, requestProperties);

This is just an extension of BasicDataSource on the server side.

<DataSource ID="xxxxFile" serverConstructor="xxxxFileDS">
    <fields>
        <field name="sysId" type="text" primaryKey="true"/>
        <field name="file" type="binary" title="Template Zip File" required="true"/>
    </fields>
</DataSource>

@Override
    public DSResponse executeAdd(DSRequest dsRequest) throws Exception {
        ISCFileItem fileItem = dsRequest.getUploadedFile(FILE);
        ...
   }

Regards

[stonebranch2]

Also, should note, the showcase is 13.x, so we haven't been able to try this in 12.1p.

[Isomorphic]

We have tried this out in 12.1p [using the showcase available as part of the SDKPackage download].
There *is* an initial problem with the #upload_sql sample in the showcase in this build, but it's just a missing dataSource (mediaLibrary) and once that's added the sample works fine, so this wouldn't explain the problems you're seeing.

The sample uses this dataSource:

<DataSource isSampleDS="true" ID="mediaLibrary" serverType="sql" tableName="mediaLibrary">
  <fields>
    <field name="pk" type="sequence" hidden="true" primaryKey="true"/>
    <field name="title"/>
    <field name="image" type="imageFile" maxFileSize="5242880" required="true"/>
  </fields>
</DataSource>

And the source for the actual upload is essentially this:

  DataSource dataSource = DataSource.get("mediaLibrary");

  final DynamicForm uploadForm = new DynamicForm();
  uploadForm.setDataSource(dataSource);
  uploadForm.setWidth(300);

  TextItem uploadTitleItem = new TextItem("title");
  uploadTitleItem.setRequired(true);

  FileItem imageItem = new FileItem("image");
  imageItem.setHint("Maximum file-size is 5 MiB");
  imageItem.setRequired(true);
  imageItem.setMultiple(false);

  ButtonItem saveItem = new ButtonItem("save", "Save");
  saveItem.addClickHandler(new com.smartgwt.client.widgets.form.fields.events.ClickHandler() {
    public void onClick(com.smartgwt.client.widgets.form.fields.events.ClickEvent event) {
      uploadForm.saveData(new com.smartgwt.client.data.DSCallback() {
        public void execute(DSResponse response, Object data, DSRequest request) {
          if (response.getStatus() >= 0 ) uploadForm.editNewRecord();
        }
      });
    }
  });

  uploadForm.setFields(uploadTitleItem, imageItem, saveItem);

So I think we've confirmed that basic file upload works in the build, using a form.saveData() call as in your code snippet above.

We definitely need more information to proceed.

We'd recommend you share the suggested logs if there are any -- anything reported in the browser console, the developer console or the server side logs.

Being stuck with a spinner implies that the framework has started a queue which has never returned. This can be due to a javascript error during the submission flow, a native browser problem with the request failing to send, and a problem on the server where it fails to respond, or returns an unexpected response that the client doesn't understand.
In many of these scenarios we'll get a good hint from the logs.

You can also use the browser tools' "Network" tab to track whether a request is actually sent and what the response looks like this. This could certainly shed some light on what's going on.

Other than that, assuming you can reproduce the problem in house with your full app, we'd recommend trying to trim this down to a minimal test case. A good starting point might be to create a simple EntryPoint class that loads the relevant dataSource binds a form to it, and provides a save button, similar to the UI within your application. You can also try swapping out the custom server dataSource (subclass of BasicDataSource) for a simple SQL backed DS with the same set of fields.
Essentially try to identify where exactly things are going wrong which will either give you enough information to understand the problem, or a standalone test case we can run on our end for debugging purproses.

[stonebranch2]

There appears to be no activity logged in the Network tab when clicking upload.

[Isomorphic]

Hi
That tells us that the problem is on the client side, but I think we still need some more information to debug this.
So far we know that you're seeing this on Chrome and Firefox, on Windows. You've seen the problem with zip files, and we've seen the code to save the form (which is a simple form.saveData() call).
[We are able to upload a .zip file using 12.1p pro in the showcase sample, which also does a simple "saveData()" on the form to perform the save]

Some other things that might reveal more information:
- Any logs you can share, or any other information revealed in the browser console or SmartGWT developer console? -- JS errors, warnings, validation errors, etc
- Anything special about the files being uploaded or the user interaction? For example -- Does the problem occur for some files, not others? Anything special about file size? Are you modifying the files in the filesystem after selecting them / before upload? Anything special about the user interaction -- is there a successful upload followed by the failed upload? etc
- Any luck reproducing this in a standalone test case? Does it make a difference if you swap out your custom dataSource for a simple SQL dataSource, etc?
- Failing that - what do the relevant fields in the DS.xml file look like? What does the code to build the form with the FileItem look like?

Thanks

[stonebranch2]

Hi Isomorphic,

This is a major release week for us, so we are busy with that. We plan to do further analysis and follow up as soon as we can.

Thanks

[stonebranch3]

Hi Isomorphic ,

I was able to make some progress on this.

We create a Window with a DynamicForm for the FileItem. The ClickHandler for the import Button on the Window saves the form and then calls the markForDestroy method on the Window instance in order to close it.

Here is a simplified version of the code:

public static void createImportWindow() {
        Window window = new Window();
        VLayout formLayout = new VLayout();
        DynamicForm form = new DynamicForm();
        HLayout buttonStrip = new HLayout();

        FileItem fileItem = new FileItem("file");
        form.setFields(fileItem);
        formLayout.addMember(form);

        Button importButton = new Button("Import");
        importButton.addClickHandler(event -> {
            if (!form.validate()) return;

            DSRequest requestProperties = new DSRequest();
            requestProperties.setWillHandleError(Boolean.TRUE);

            form.saveData((dsResponse, data, dsRequest) -> {
                if (dsResponse.getStatus() == RPCResponse.STATUS_SUCCESS) {
                    // Success
                } else {
                    // Failure
                }
            }, requestProperties);

            window.markForDestroy(); // Causes save operation to hang
        });
        buttonStrip.addMember(importButton);
        formLayout.addMember(buttonStrip);

        window.addItem(formLayout);
        window.show();
    }

The problem with the save operation hanging appears to be related to marking the Window to be destroyed after calling saveData for the form. It is resolved by using window.close() or window.hide() instead of window.markForDestroy().

Do we still need to explicitly destroy the Window once we are done with it in this case?

Thanks

[Isomorphic]

Wow, OK, the only remaining mystery is how this worked at all!

What is basically happening is that the form is in the middle of uploading the file when it suddenly gets destroyed. You can fix this by just moving your markForDestroy() call into the callback.

Note you probably only want to do the destroy() in the success case, since if there is a failure (file too large, or whatever) you want the Window to stick around so the user can see the validation error and correct it.

We will add a note to destroy() / markForDestroy() to try to avoid others running into this issue in the future.

[stonebranch3]

OK, that makes sense.

We'll move the destroy into the callback as you suggested.

Thanks again!