Compression for one DB field

[edulid]

I have a binary field in my db which I want to compress. It contains documents uploaded by the users using smartGWT components.

Is it possible to compress a field using smartGWT? If not, which would be the best way to implement this? Using a servlet? Or what would you recommend?

Using smartgwt 5.0p power 27.01

[Isomorphic]

Probably the simplest thing is to see if your database supports automatic and transparent compression for a binary field, and enable that.

If you have to do it yourself, you would implement this via the techniques shown in the Custom Binary field sample and other samples in the same folder: for files being uploaded, use a DMI to replace the InputStream in the inbound DSRequest with a compressed input stream. For files being downloaded, modify DSResponse.data to replace the compressed InputStream returned by the DB with a non-compressed one.

[edulid]

Should I change the size in the f_datei_filesize field used by smartGWT be the compressed size ?

[edulid]

For the decompression I am trying to use the following:

<field name="f_datei" type="binary" customSelectExpression="dbo.fn_decompress(f_datei)" />

It uses the T-SQL function fn_decompress(binary).

But what to use for the insert?
For this, I have a similar function: dbo.fn_compress(binary).
But how to integrate this ?

[Isomorphic]

If there's a corresponding compression function, then using customInsertExpression() may work, but this is getting into details of SQL Server rather than SmartGWT.

It's unclear whether/why you would be considering changing the size of the field. As far as SmartGWT validation based on maxFileSize, we would be looking at uploaded file size, not the compressed file size.

[edulid]

Yes, I was thinking of:

<field name="f_datei" type="binary" customInsertExpression="dbo.fn_compress(f_datei)" />

But wouldn't this result in :

insert into table set dbo.fn_compress(f_datei) = ...."

This would be incorrect if I understand "customInsertExpression".

Or something like this ?

<values fieldName="f_datei" value="dbo.fn_compress($rawValue.values.f_datei)" />

[Isomorphic]

customInsertExpression affects the expression assigned to the SQL column. It does not replace the column name as you guess.

Your <values> usage is not correct as SQL is not expected there.

[edulid]

It's unclear whether/why you would be considering changing the size of the field. As far as SmartGWT validation based on maxFileSize, we would be looking at uploaded file size, not the compressed file size.

Ah ok, so I think I should leave the original size.
Is this field *only* used for validation based on maxFileSize?

[Isomorphic]

Is this field *only* used for validation based on maxFileSize?

Try rephrasing this. Obviously, the field is used for a variety of things, including telling the framework the DB column name and enabling binary storage. So you must have some other question in mind.

[edulid]

I mean only the field field_filesize. Not the field_filename. But I think I have to leave the _filesize field as with the uncompressed file.

I tried the following:

<operationBinding operationType="add">
			<values fieldName="f_user_created" value="$session.userId" />
			<values fieldName="f_datei" value="dbo.fn_compress($rawValue.values.f_datei)" />
		</operationBinding>

(The first field (f_user_created) has always been here. I want to add modify the field f_datei with the compressed version).

But I am now getting:

{
    dataSource:"dokumente", 
    operationType:"add", 
    componentId:"isc_DynamicForm_9", 
    data:{
        f_datei:"C:\\fakepath\\Microsoft Word-Dokument (neu).docx", 
        f_beschreibung:"", 
        f_vertrag:null, 
        f_date_created:null, 
        name_created:null, 
        f_user_created:null, 
        f_schueler_id:7544
    }, 
    textMatchStyle:"exact", 
    callback:{
        target:[DynamicForm ID:isc_DynamicForm_9], 
        methodName:"saveEditorReply"
    }, 
    showPrompt:true, 
    prompt:"Speichere Formulardaten...", 
    oldValues:{
    }, 
    requestId:"dokumente$62734", 
    clientContext:{
        saveDataTarget:[Error in echoLeaf: TypeError: undefined is not a function]
    }, 
    internalClientContext:{
    }, 
    fallbackToEval:false, 
    afterFlowCallback:"isc_DynamicForm_9.$49z(dsRequest, dsResponse, data)", 
    directSubmit:true, 
    submitForm:[DynamicForm ID:isc_DynamicForm_9], 
    editor:[DynamicForm ID:isc_DynamicForm_9], 
    lastClientEventThreadCode:"MUP2", 
    bypassCache:true
}

And the exception:

=== 2015-03-24 17:02:25,307 [ec-3] DEBUG RPCManager - Processing 1 requests.
=== 2015-03-24 17:02:25,307 [ec-3] DEBUG ISCKeyedObjectPool - Borrowing object for 'dokumente'
=== 2015-03-24 17:02:25,307 [ec-3] DEBUG PoolableDataSourceFactory - Activated DataSource 103 of type 'dokumente'
=== 2015-03-24 17:02:25,307 [ec-3] DEBUG DSRequest - Caching instance 103 of DS 'dokumente' from DSRequest.getDataSource()
=== 2015-03-24 17:02:25,307 [ec-3] DEBUG DSRequest - Caching instance 103 of DS dokumente
=== 2015-03-24 17:02:25,308 [ec-3] INFO  DSRequest - We parsed 1 uploaded file(s): "C:/fakepath/Microsoft Word-Dokument (neu).docx" (0 bytes)
=== 2015-03-24 17:02:25,308 [ec-3] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 103 of type 'dokumente'
=== 2015-03-24 17:02:25,308 [ec-3] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    values:{
        f_datei:"C:\\fakepath\\Microsoft Word-Dokument (neu).docx",
        f_beschreibung:"",
        f_vertrag:null,
        f_date_created:null,
        name_created:null,
        f_user_created:null,
        f_schueler_id:7544,
        f_datei_date_created:new Date(1427212945308),
        f_datei_filename:"Microsoft Word-Dokument (neu).docx",
        f_datei_filesize:0
    },
    operationConfig:{
        dataSource:"dokumente",
        repo:null,
        operationType:"add",
        textMatchStyle:"exact"
    },
    componentId:"isc_DynamicForm_9",
    appID:"builtinApplication",
    operation:"dokumente_add",
    oldValues:{
    },
    criteria:{
    }
}
=== 2015-03-24 17:02:25,308 [ec-3] INFO  IDACall - Key[type=com.isomorphic.servlet.IDACall, annotation=[none]] - Performing 1 operation(s)
=== 2015-03-24 17:02:25,309 [ec-3] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2015-03-24 17:02:25,309 [ec-3] DEBUG DeclarativeSecurity - DataSource dokumente is not in the pre-checked list, processing...
=== 2015-03-24 17:02:25,312 [ec-3] DEBUG AppBase - [builtinApplication.dokumente_add] No userTypes defined, allowing anyone access to all operations for this application
=== 2015-03-24 17:02:25,312 [ec-3] DEBUG AppBase - [builtinApplication.dokumente_add] No public zero-argument method named '_dokumente_add' found, performing generic datasource operation
=== 2015-03-24 17:02:25,314 [ec-3] INFO  SQLDataSource - [builtinApplication.dokumente_add] Performing add operation with
	criteria: {f_beschreibung:"",f_vertrag:null,f_date_created:new Date(1427212945309),name_created:null,f_user_created:1,f_schueler_id:7544,f_datei_date_created:new Date(1427212945308),f_datei_filename:"Microsoft Word-Dokument (neu).docx",f_datei_filesize:0}	values: {f_beschreibung:"",f_vertrag:null,f_date_created:new Date(1427212945309),name_created:null,f_user_created:1,f_schueler_id:7544,f_datei_date_created:new Date(1427212945308),f_datei_filename:"Microsoft Word-Dokument (neu).docx",f_datei_filesize:0,f_mandant:1}
=== 2015-03-24 17:02:25,318 [ec-3] DEBUG SQLValuesClause - [builtinApplication.dokumente_add] Sequences: {f_id=__default}
=== 2015-03-24 17:02:25,325 [ec-3] DEBUG SQLConnectionManager - [builtinApplication.dokumente_add] Borrowed connection '1091369584'
=== 2015-03-24 17:02:25,326 [ec-3] DEBUG SQLTransaction - [builtinApplication.dokumente_add] Started new SQLSERVER transaction "1091369584"
=== 2015-03-24 17:02:25,326 [ec-3] DEBUG SQLDriver - [builtinApplication.dokumente_add] About to execute SQL update in 'SQLSERVER' using connection'1091369584'
=== 2015-03-24 17:02:25,326 [ec-3] INFO  SQLDriver - [builtinApplication.dokumente_add] Executing SQL update on 'SQLSERVER': INSERT INTO t_dokumente (f_beschreibung, f_date_created, f_datei_date_created, f_datei_filename, f_datei_filesize, f_mandant, f_schueler_id, f_user_created, f_vertrag) VALUES ('', '2015-03-24T17:02:25', '2015-03-24T17:02:25', N'Microsoft Word-Dokument (neu).docx', 0, 1, 7544, 1, NULL)
=== 2015-03-24 17:02:25,334 [ec-3] DEBUG SQLDriver - [builtinApplication.dokumente_add] Found 1 sequence(s)/autoGenerated PK field(s): [f_id]
=== 2015-03-24 17:02:25,336 [ec-3] WARN  SQLDriver - [builtinApplication.dokumente_add] Exception thrown during saveGeneratedKeys()
com.microsoft.sqlserver.jdbc.SQLServerException: Die Anweisung muss ausgeführt werden, bevor Ergebnisse abgerufen werden können.
	at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:190)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.getGeneratedKeys(SQLServerStatement.java:2040)
	at org.apache.commons.dbcp.DelegatingStatement.getGeneratedKeys(DelegatingStatement.java:315)
	at com.isomorphic.sql.SQLDriver.saveGeneratedKeys(SQLDriver.java:1327)
	at com.isomorphic.sql.SQLDriver.doUpdate(SQLDriver.java:911)
	at com.isomorphic.sql.SQLDriver.update(SQLDriver.java:804)
	at com.isomorphic.sql.SQLDriver.executeUpdate(SQLDriver.java:989)
	at com.isomorphic.sql.SQLDataSource.executeNativeUpdate(SQLDataSource.java:570)
	at com.isomorphic.sql.SQLDataSource.SQLExecute(SQLDataSource.java:1992)
	at com.isomorphic.sql.SQLDataSource.processRequest(SQLDataSource.java:443)
	at com.isomorphic.sql.SQLDataSource.executeAdd(SQLDataSource.java:396)
	at de.mks_infofabrik.kids.server.KidsDataSource.executeAdd(KidsDataSource.java:55)
	at com.isomorphic.datasource.DataSource.execute(DataSource.java:1950)
	at com.isomorphic.application.AppBase.executeDefaultDSOperation(AppBase.java:726)
	at com.isomorphic.application.AppBase.executeAppOperation(AppBase.java:658)
	at com.isomorphic.application.AppBase.execute(AppBase.java:491)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2548)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:218)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:183)
	at com.isomorphic.servlet.IDACall.processRequest(IDACall.java:147)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:76)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:156)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
	at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
	at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
	at de.mks_infofabrik.kids.server.filter.GWTCacheControlFilter.doFilter(GWTCacheControlFilter.java:50)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
	at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:83)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
	at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:136)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:526)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:655)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Unknown Source)
=== 2015-03-24 17:02:25,343 [ec-3] DEBUG SQLDriver - [builtinApplication.dokumente_add] FAILED to execute SQL update in 'SQLSERVER' using connection'1091369584'
=== 2015-03-24 17:02:25,343 [ec-3] DEBUG DSRequest - freeOnExecute is false for request of type add on DataSource dokumente - not freeing resources!
=== 2015-03-24 17:02:25,343 [ec-3] WARN  RequestContext - dsRequest.execute() failed: 
com.microsoft.sqlserver.jdbc.SQLServerException: Der Index "1" liegt außerhalb des gültigen Bereichs.
	at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:190)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.setterGetParam(SQLServerPreparedStatement.java:714)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.setStream(SQLServerPreparedStatement.java:733)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.setBinaryStream(SQLServerPreparedStatement.java:820)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.setBinaryStream(DelegatingPreparedStatement.java:157)
	at com.isomorphic.sql.SQLDriver.doUpdate(SQLDriver.java:882)
	at com.isomorphic.sql.SQLDriver.update(SQLDriver.java:804)
	at com.isomorphic.sql.SQLDriver.executeUpdate(SQLDriver.java:989)
	at com.isomorphic.sql.SQLDataSource.executeNativeUpdate(SQLDataSource.java:570)
	at com.isomorphic.sql.SQLDataSource.SQLExecute(SQLDataSource.java:1992)
	at com.isomorphic.sql.SQLDataSource.processRequest(SQLDataSource.java:443)
	at com.isomorphic.sql.SQLDataSource.executeAdd(SQLDataSource.java:396)
	at de.mks_infofabrik.kids.server.KidsDataSource.executeAdd(KidsDataSource.java:55)
	at com.isomorphic.datasource.DataSource.execute(DataSource.java:1950)
	at com.isomorphic.application.AppBase.executeDefaultDSOperation(AppBase.java:726)
	at com.isomorphic.application.AppBase.executeAppOperation(AppBase.java:658)
	at com.isomorphic.application.AppBase.execute(AppBase.java:491)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2548)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:218)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:183)
	at com.isomorphic.servlet.IDACall.processRequest(IDACall.java:147)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:76)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:156)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
	at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
	at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
	at de.mks_infofabrik.kids.server.filter.GWTCacheControlFilter.doFilter(GWTCacheControlFilter.java:50)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
	at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:83)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
	at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:136)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:526)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:655)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Unknown Source)
=== 2015-03-24 17:02:25,345 [ec-3] DEBUG RPCManager - Content type for RPC transaction: text/html; charset=UTF-8
=== 2015-03-24 17:02:25,346 [ec-3] DEBUG SQLTransaction - Rolling back SQLSERVER transaction "1091369584"
=== 2015-03-24 17:02:25,346 [ec-3] DEBUG RPCManager - non-DMI response, dropExtraFields: false
=== 2015-03-24 17:02:25,347 [ec-3] DEBUG SQLTransaction - getConnection() found transactional connection for SQLSERVER with hashcode "1091369584"
=== 2015-03-24 17:02:25,347 [ec-3] DEBUG SQLTransaction - Ending SQLSERVER transaction "1091369584"
=== 2015-03-24 17:02:25,347 [ec-3] DEBUG SQLConnectionManager - About to close  with hashcode "1091369584"

Now I will try the customInsertExpression variant.

[edulid]

Same for the other variant:

<field name="f_datei" type="binary" customSelectExpression="dbo.fn_decompress(f_datei)" customInsertExpression="dbo.fn_compress($rawValue.values.f_datei)" />

=== 2015-03-24 17:18:11,892 [ec-3] DEBUG PoolableDataSourceFactory - Activated DataSource 609 of type 'Object'
=== 2015-03-24 17:18:11,892 [ec-3] DEBUG ISCKeyedObjectPool - Borrowing object for 'List'
=== 2015-03-24 17:18:11,893 [ec-3] DEBUG PoolableDataSourceFactory - Created DataSource 610 of type 'List' and assigned it to thread http-nio-8081-exec-3
=== 2015-03-24 17:18:11,893 [ec-3] DEBUG PoolableDataSourceFactory - Created DataSource 610 of type 'List' in the pooling flow
=== 2015-03-24 17:18:11,893 [ec-3] DEBUG PoolableDataSourceFactory - Activated DataSource 610 of type 'List'
=== 2015-03-24 17:18:11,893 [ec-3] DEBUG ISCKeyedObjectPool - Borrowing object for 'elem'
=== 2015-03-24 17:18:11,894 [ec-3] DEBUG PoolableDataSourceFactory - Created DataSource null of type 'elem' in the pooling flow
=== 2015-03-24 17:18:11,894 [ec-3] DEBUG RPCManager - Processing 1 requests.
=== 2015-03-24 17:18:11,894 [ec-3] DEBUG ISCKeyedObjectPool - Borrowing object for 'dokumente'
=== 2015-03-24 17:18:11,895 [ec-3] DEBUG PoolableDataSourceFactory - Activated DataSource 103 of type 'dokumente'
=== 2015-03-24 17:18:11,895 [ec-3] DEBUG DSRequest - Caching instance 103 of DS 'dokumente' from DSRequest.getDataSource()
=== 2015-03-24 17:18:11,895 [ec-3] DEBUG DSRequest - Caching instance 103 of DS dokumente
=== 2015-03-24 17:18:11,895 [ec-3] INFO  DSRequest - We parsed 1 uploaded file(s): "C:/fakepath/buffer.txt" (3839 bytes)
=== 2015-03-24 17:18:11,896 [ec-3] DEBUG PoolableDataSourceFactory - Cleared and passivated DataSource 103 of type 'dokumente'
=== 2015-03-24 17:18:11,896 [ec-3] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    values:{
        f_datei:"C:\\fakepath\\buffer.txt",
        f_beschreibung:"",
        f_vertrag:null,
        f_date_created:null,
        name_created:null,
        f_user_created:null,
        f_schueler_id:6600,
        f_datei_date_created:new Date(1427213891895),
        f_datei_filename:"buffer.txt",
        f_datei_filesize:3839
    },
    operationConfig:{
        dataSource:"dokumente",
        repo:null,
        operationType:"add",
        textMatchStyle:"exact"
    },
    componentId:"isc_DynamicForm_9",
    appID:"builtinApplication",
    operation:"dokumente_add",
    oldValues:{
    },
    criteria:{
    }
}
=== 2015-03-24 17:18:11,896 [ec-3] INFO  IDACall - Key[type=com.isomorphic.servlet.IDACall, annotation=[none]] - Performing 1 operation(s)
=== 2015-03-24 17:18:11,896 [ec-3] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2015-03-24 17:18:11,896 [ec-3] DEBUG DeclarativeSecurity - DataSource dokumente is not in the pre-checked list, processing...
=== 2015-03-24 17:18:11,897 [ec-3] DEBUG AppBase - [builtinApplication.dokumente_add] No userTypes defined, allowing anyone access to all operations for this application
=== 2015-03-24 17:18:11,897 [ec-3] DEBUG AppBase - [builtinApplication.dokumente_add] No public zero-argument method named '_dokumente_add' found, performing generic datasource operation
=== 2015-03-24 17:18:11,899 [ec-3] INFO  SQLDataSource - [builtinApplication.dokumente_add] Performing add operation with
	criteria: {f_beschreibung:"",f_vertrag:null,f_date_created:new Date(1427213891897),name_created:null,f_user_created:1,f_schueler_id:6600,f_datei_date_created:new Date(1427213891895),f_datei_filename:"buffer.txt",f_datei_filesize:3839}	values: {f_beschreibung:"",f_vertrag:null,f_date_created:new Date(1427213891897),name_created:null,f_user_created:1,f_schueler_id:6600,f_datei_date_created:new Date(1427213891895),f_datei_filename:"buffer.txt",f_datei_filesize:3839,f_mandant:1}
=== 2015-03-24 17:18:11,906 [ec-3] DEBUG SQLValuesClause - [builtinApplication.dokumente_add] Sequences: {f_id=__default}
=== 2015-03-24 17:18:11,913 [ec-3] DEBUG SQLConnectionManager - [builtinApplication.dokumente_add] Borrowed connection '1323759591'
=== 2015-03-24 17:18:11,913 [ec-3] DEBUG SQLTransaction - [builtinApplication.dokumente_add] Started new SQLSERVER transaction "1323759591"
=== 2015-03-24 17:18:11,914 [ec-3] DEBUG SQLDriver - [builtinApplication.dokumente_add] About to execute SQL update in 'SQLSERVER' using connection'1323759591'
=== 2015-03-24 17:18:11,914 [ec-3] INFO  SQLDriver - [builtinApplication.dokumente_add] Executing SQL update on 'SQLSERVER': INSERT INTO t_dokumente (f_beschreibung, f_date_created, f_datei, f_datei_date_created, f_datei_filename, f_datei_filesize, f_mandant, f_schueler_id, f_user_created, f_vertrag) VALUES ('', '2015-03-24T17:18:11', dbo.fn_compress($rawValue.values.f_datei), '2015-03-24T17:18:11', N'buffer.txt', 3839, 1, 6600, 1, NULL)
=== 2015-03-24 17:18:11,920 [ec-3] DEBUG SQLDriver - [builtinApplication.dokumente_add] Found 1 sequence(s)/autoGenerated PK field(s): [f_id]
=== 2015-03-24 17:18:11,920 [ec-3] WARN  SQLDriver - [builtinApplication.dokumente_add] Exception thrown during saveGeneratedKeys()
com.microsoft.sqlserver.jdbc.SQLServerException: Die Anweisung muss ausgeführt werden, bevor Ergebnisse abgerufen werden können.
	at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:190)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.getGeneratedKeys(SQLServerStatement.java:2040)
	at org.apache.commons.dbcp.DelegatingStatement.getGeneratedKeys(DelegatingStatement.java:315)
	at com.isomorphic.sql.SQLDriver.saveGeneratedKeys(SQLDriver.java:1327)
	at com.isomorphic.sql.SQLDriver.doUpdate(SQLDriver.java:911)
	at com.isomorphic.sql.SQLDriver.update(SQLDriver.java:804)
	at com.isomorphic.sql.SQLDriver.executeUpdate(SQLDriver.java:989)
	at com.isomorphic.sql.SQLDataSource.executeNativeUpdate(SQLDataSource.java:570)
	at com.isomorphic.sql.SQLDataSource.SQLExecute(SQLDataSource.java:1992)
	at com.isomorphic.sql.SQLDataSource.processRequest(SQLDataSource.java:443)
	at com.isomorphic.sql.SQLDataSource.executeAdd(SQLDataSource.java:396)
	at de.mks_infofabrik.kids.server.KidsDataSource.executeAdd(KidsDataSource.java:55)
	at com.isomorphic.datasource.DataSource.execute(DataSource.java:1950)
	at com.isomorphic.application.AppBase.executeDefaultDSOperation(AppBase.java:726)
	at com.isomorphic.application.AppBase.executeAppOperation(AppBase.java:658)
	at com.isomorphic.application.AppBase.execute(AppBase.java:491)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2548)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:218)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:183)
	at com.isomorphic.servlet.IDACall.processRequest(IDACall.java:147)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:76)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:156)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
	at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
	at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
	at de.mks_infofabrik.kids.server.filter.GWTCacheControlFilter.doFilter(GWTCacheControlFilter.java:50)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
	at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:83)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
	at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:136)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:526)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:655)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Unknown Source)
=== 2015-03-24 17:18:11,921 [ec-3] DEBUG SQLDriver - [builtinApplication.dokumente_add] FAILED to execute SQL update in 'SQLSERVER' using connection'1323759591'
=== 2015-03-24 17:18:11,921 [ec-3] DEBUG DSRequest - freeOnExecute is false for request of type add on DataSource dokumente - not freeing resources!
=== 2015-03-24 17:18:11,921 [ec-3] WARN  RequestContext - dsRequest.execute() failed: 
com.microsoft.sqlserver.jdbc.SQLServerException: Der Index "1" liegt außerhalb des gültigen Bereichs.
	at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:190)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.setterGetParam(SQLServerPreparedStatement.java:714)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.setStream(SQLServerPreparedStatement.java:733)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.setBinaryStream(SQLServerPreparedStatement.java:820)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.setBinaryStream(DelegatingPreparedStatement.java:157)
	at com.isomorphic.sql.SQLDriver.doUpdate(SQLDriver.java:882)
	at com.isomorphic.sql.SQLDriver.update(SQLDriver.java:804)
	at com.isomorphic.sql.SQLDriver.executeUpdate(SQLDriver.java:989)
	at com.isomorphic.sql.SQLDataSource.executeNativeUpdate(SQLDataSource.java:570)
	at com.isomorphic.sql.SQLDataSource.SQLExecute(SQLDataSource.java:1992)
	at com.isomorphic.sql.SQLDataSource.processRequest(SQLDataSource.java:443)
	at com.isomorphic.sql.SQLDataSource.executeAdd(SQLDataSource.java:396)
	at de.mks_infofabrik.kids.server.KidsDataSource.executeAdd(KidsDataSource.java:55)
	at com.isomorphic.datasource.DataSource.execute(DataSource.java:1950)
	at com.isomorphic.application.AppBase.executeDefaultDSOperation(AppBase.java:726)
	at com.isomorphic.application.AppBase.executeAppOperation(AppBase.java:658)
	at com.isomorphic.application.AppBase.execute(AppBase.java:491)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2548)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:218)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:183)
	at com.isomorphic.servlet.IDACall.processRequest(IDACall.java:147)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:76)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:156)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
	at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
	at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
	at de.mks_infofabrik.kids.server.filter.GWTCacheControlFilter.doFilter(GWTCacheControlFilter.java:50)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
	at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:83)
	at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
	at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
	at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:136)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:526)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:655)
	at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Unknown Source)
=== 2015-03-24 17:18:11,922 [ec-3] DEBUG RPCManager - Content type for RPC transaction: text/html; charset=UTF-8
=== 2015-03-24 17:18:11,923 [ec-3] DEBUG SQLTransaction - Rolling back SQLSERVER transaction "1323759591"
=== 2015-03-24 17:18:11,923 [ec-3] DEBUG RPCManager - non-DMI response, dropExtraFields: false
=== 2015-03-24 17:18:11,924 [ec-3] DEBUG SQLTransaction - getConnection() found transactional connection for SQLSERVER with hashcode "1323759591"
=== 2015-03-24 17:18:11,924 [ec-3] DEBUG SQLTransaction - Ending SQLSERVER transaction "1323759591"
=== 2015-03-24 17:18:11,924 [ec-3] DEBUG SQLConnectionManager - About to close  with hashcode "1323759591"
=== 2015-03-24 17:18:11,925 [ec-3] INFO  Compression - /kids-dev/V010306SNAPSHOT/sc/IDACall: 701 -> 448 bytes

[edulid]

I think $rawValue.values.f_datei is not the correct way to set the binary data. Or is this correct? If yes, why am I getting these exceptions ?

[Isomorphic]

For binary fields, since the actual file contents cannot be inlined in the SQL statement, the default SQL includes a placeholder char (typically ? or $1) and we subsequently provide the input stream via JDBC (this is all standard JDBC stuff).

It's possible your approach will work if, instead of $rawValue.fieldName, you use the same placeholder character we use (just execute an uncustomized SQL query and check the logs to see it).

Whether this works or not is up to the quality of SQLServer and its JDBC driver. If it doesn't, you'll need to go back to our recommendations.

[edulid]

Replacing with ? actually works. Is this standard or may it be changed when I upgrade the nightly?
and what if I had 2 files? what would be the placeholder for the second file ?

[Isomorphic]

We can't really guarantee this will continue to work - partly it depends on whether SQLServer's behavior changes, or if they introduce a bug where we can no longer use our current approach for providing binary values. But no framework change or SQLServer change over the past several years would have invalidated this approach.

Still, for a definitely safe approach, follow one of our original recommendations.

You would use a second ? for another file. It's done positionally - the standard JDBC approach.