file content blob not available immediately after an upload.

[dbscott525]

I am using SmartGWT 3.1-P20130714.

I am using FileItem to upload files.

Here is my code that sets up FileItem

final FileItem upload = new FileItem(Schema.DSF_UPLOADEDFILE);
upload.setTitle(Constants.BROWSE_TAXONOMY_OR_CONTACTLIST);
upload.setEndRow(true);
RegExpValidator regExpValidator = new RegExpValidator();
regExpValidator.setExpression(Constants.FILE_NAME_VALIDATION);
regExpValidator.setErrorMessage(ErrorMessages.NOT_XLSX);
upload.setValidators(regExpValidator);
upload.setValidateOnChange(true);

Here is the code I used to do the upload:

uploadForm.saveData(new DSCallback() {

	@Override
	public void execute(DSResponse response, Object rawData, DSRequest request) {
		uploadForm.clearValues();
		uploadList.getUploadList().refresh();
	}
});

Here is the data source XML file used in the upload:

<DataSource 
	dbName="Mysql"
	tableName="uploads"
	ID="uploads"
	dataSourceVersion="1"
	generatedBy="v8.3p_2013-02-26/EVAL Deployment 2013-02-26"
	serverType="sql"
>
	<fields>
		<field name="ID"                        title="ID"            type="sequence"          hidden="true" primaryKey="true"       />
		<field name="AccountID"                 title="Account ID"    type="integer"           hidden="true" foreignKey="accounts.ID"/>
		<field name="Name"                      title="Name"          type="text" length="45"  hidden="false"                        />
		<field name="Type"                      title="Type"          type="text" length="20"  hidden="true"                         />
		<field name="UploadedFile"              title="Uploaded File" type="binary"            hidden="true"                         />
		<field name="UploadedFile_filename"     title="File Name"     type="text" length="255" hidden="false"                        />
		<field name="UploadedFile_filesize"     title="Size"          type="integer"           hidden="false"                        />
		<field name="UploadedFile_date_created" title="Date Created"  type="datetime"          hidden="false"                        />
	</fields>
</DataSource>

This code successfully creates a row in the database that can be read both on the client and server side.

On the server side, using Hibernate, I can read the row. However, I cannot read the blob in the row. Hibernate returns a null value for the blob field.

After restarting the server the file content blob can be read with Hibernate on the server.

An additional symptom is that after doing the upload MySQL will shut down access to our server after 10 hours. This is because the MySQL I’m using has a ten hour time limit on open sessions.

Of course, I don’t know what may be the issue causing this problem. However, my theory is that the Isomorphic code is committing the creation of the row that contains the upload meta data (e.g. UploadedFile_filename, UploadedFile_filesize) but it is not committing the writing of the blob that contains the content of the file. Additionally, I believe the session used to write the blob is left open leaving a permanent connection to the MySQL database.

The server restart causes this commit to happen and releases the database connection causing the blob to become available.

I’m also wondering if there is something I need to do after the upload completes to force the commit of the file content blob on the server side.

This is a side note: I have been using this upload code successfully for several months and never noticed the missing file content blob problem. This was because I frequently restarted the server as part of development and testing. However, now that we have deployed this to our production server this problem has been exposed. It's stopping our production server from functioning properly and causes us to need to restart the server after every upload.

[Isomorphic]

Please post the server logs (you should always do this for a server problem). Also, can you see the binary data if you query the database from some ordinary SQL client, like the MySQL Query Browser?

Incidentally, it isn't possible for some columns of a row to be committed and others not, unless there is some bizarre non-transactional behavior of MySQL that we're not aware of.

[dbscott525]

second uploaded file not visible on the server side

I have discovered a few new things.

First, although I thought I was using 3.1P-20130727 I was using an earlier version. Now that I am actually using 3.1P-20130727 the behavior of upload has changed.

After I upload a single file I can access it properly. If I upload a second file it is not visible on the server side through Hibernate. I was wrong about only the blob being inaccessible. The entire row is inaccessible.

Also, if I upload a file, delete it using the ListGrid, and then upload another file; the second upload file is not accessible.

Is it possible that autoJoinTransactions is on? I don't really understand autoJoinTransactions but when I read about autoJoinTransactions it seems that its possible that updates the upload table are being batched and not committed.

I've attached the log in two files. The first one shows the first upload and the second one shows the second upload.

[Isomorphic]

Both of these logs show SmartGWT Server borrowing a database connection from the pool, using it, and then correctly returning it - there is nothing wrong in the regular server-side insert flow. The second log looks like it has been truncated - it doesn't show anything beyond the arrival on the server of the fetch request on dataSource "uploads", so we can't comment on any difference between the two.

However, from looking at the first log, it seems possible that it is your interaction with Hibernate that is causing the problem. As mentioned above, in both logs it is clear that the connection being used by SmartGWT Server for the insert is being closed, so if something is hanging onto a connection for ten hours, as you say, then Hibernate seems the most likely culprit. And if you are working with an uncommitted Hibernate session, that may explain why it works once and then not again until the server is rebooted. As we originally suggested, you should query the database with a regular client tool like MySQL Query Browser.

Incidentally, autoJoinTransactions (which is on for updates by default) causes the creation of a transaction that lasts for the life of the server request only - we do not create batches of updates that get run some time later, or anything of that nature. Inferring from the first log you sent, autoJoinTransactions cannot be having any effect, but it is not possible to state this categorically because the second log is incomplete. If you wish to prove to yourself that autoJoinTransactions is not affecting anything, you can switch it off for the dataSource, database or for the entire framework, as described here

[dbscott525]

Hibernate Usage Issue

Thank you for your help! This turned out not to be an Isomorphic problem. The problem is in the way I'm using Hibernate.