Bug with BatchUploader and setOptionDataSource ListGridField

[Blama]

...for example, replacing "Event" with 1 in every LEADTYPE_ID attribute. Anything you change in that data parameter will be used by downstream processing.

But with 5.0, I would not be able to get the value "Event" from the .csv to the client-ListGrid as you wrote, would I?
At least not, if the field is defined as integer.

So shouldn't I let SmartGWT deal with the strings as type="text"-DSFields and validate the strings in validate() and then transform in PL/SQL?

I'm happy with either solution, but I'd like to go the correct/best way directly.

Best regards,
Blama

[Isomorphic]

You can do this in the validate() method - it is passed a "data" parameter which is a Map. There is nothing to stop you manipulating the contents of that Map - for example, replacing "Event" with 1 in every LEADTYPE_ID attribute. Anything you change in that data parameter will be used by downstream processing.

[Blama]

Hi Isomorphic,

The BatchUploader docs in question are here, and the salient section directly addresses your issue:
The most interesting reason for doing this is to override the validate method and provide complete custom validation - for example, checking relations to other tables

I understood that in the way that I can do that for errors in the data, e.g. "Event1" instead of "Event" and assumed that in case of mappable data the conversion is done automatically. Perhaps a misreading/misunderstanding on my part.

Automatically translating a display value to its underlying key value using a declared foreignKey property is not something that the import process currently supports. It could be added, of course, and it would qualify for feature sponsorship if you are interested.

I'll contact you on on this.

Aside of a feature sponsorship:
The other way would be to define the field like the name field of employees-DS, have an SelectItem with optionDataSource in place with idField="name" and displayField="name" and do the checking in the overridden validate() and the real conversion and saving in PL/SQL, correct?

Best regards,
Blama

[Isomorphic]

Yes, the cell formatting does appear to be a genuine problem in 4.1 - we are looking into it.

Automatically translating a display value to its underlying key value using a declared foreignKey property is not something that the import process currently supports. It could be added, of course, and it would qualify for feature sponsorship if you are interested.

The other thread you link to answers your question by saying this:

Because you have the ability to override the validate() method as covered in the BatchUploader docs, you can catch any error case you like and report errors.

The BatchUploader docs in question are here, and the salient section directly addresses your issue:

The most interesting reason for doing this is to override the validate method and provide complete custom validation - for example, checking relations to other tables

[Blama]

Hi Isomorphic,

I think I understood. Let me echo in my own words:

1. My issue with cell-formatting is really an issue
2. My issue with the string "Event" being used as value instead of being translated is an real issue as well. But the solution is not to apply the value (as done in 5.0) instead of translating it (as suggested by me)

Correct?
If so, I'd really need a best practice on my use-case. I already asked for exactly this in this thread three month ago. From your reply I thought that the BatchUploader can handle this.
If switching to 5.0 (or 5.1, if REALLY no other way possible) is necessary, I think I can do that as well.

Obviously it is somehow possible to solve this problem (e.g. after applying to valueField check if an optionDataSource is set and try to translate and apply translated value). But of course I don't know if this is a clean solution in our inner workings (most likely not).
I did not fully understand yet if my use-case should be/is already supported as of now. From your last post I'd say no, from your answer in the post I linked I'd say yes.

Of course I could solve it with varchar2(bigValue) fields and some hand-written PL/SQL-translation/NULL-setting, but that would mean WAY more code.

Thank you & Best regards,
Blama

[Isomorphic]

Sorry, the last post should have been clearer - we will continue to investigate the claim of an issue with cell-formatting. The errors on the field with an optionDataSource are correct. Although it appears that the error system is mistakenly saying that "Event" is not an option, what it is actually saying is that an underlying value of "Event" has been applied to the field. That is, the valueField has a value of "Event", rather than 1 or 5 or whatever. I think you already realise that this is happening because your import data contains "Event" rather than the underlying value that means "Event".

If there is a bug to fix here, it is that the import process should prevent string values from being imported into integer fields; in fact, version 5.0 and greater do have that validation in place, which is why we were initally unable to reproduce your problem.

Note, your suggestion that we could validate the ListGrid before drawing it would not help; the problem is that the underlying data value is incorrect.

[Isomorphic]

OK, we see this now - we're investigating

[Blama]

Hi Isomorphic,

I reproduced in BuiltInDS:

Employees.csv

"EmployeeId","ReportsTo","Salary"
1000001,"Ralph Brogan",-123
1000002,"Bhushan Sambhus",456

BuiltInDS.java:

package com.smartgwt.sample.client;

import com.google.gwt.core.client.EntryPoint;
import com.google.gwt.i18n.client.NumberFormat;
import com.smartgwt.client.core.KeyIdentifier;
import com.smartgwt.client.data.DataSource;
import com.smartgwt.client.util.PageKeyHandler;
import com.smartgwt.client.util.Page;
import com.smartgwt.client.util.SC;
import com.smartgwt.client.widgets.BatchUploader;
import com.smartgwt.client.widgets.grid.CellFormatter;
import com.smartgwt.client.widgets.grid.ListGridField;
import com.smartgwt.client.widgets.grid.ListGridRecord;

public class BuiltInDS implements EntryPoint {
	public void onModuleLoad() {
		KeyIdentifier debugKey = new KeyIdentifier();
		debugKey.setCtrlKey(true);
		debugKey.setKeyName("D");

		Page.registerKey(debugKey, new PageKeyHandler() {
			public void execute(String keyName) {
				SC.showConsole();
			}
		});

		BatchUploader batchUploader = new BatchUploader();
		batchUploader.setWidth(600);
		batchUploader.setUploadDataSource(DataSource.get("employees"));

		ListGridField employeeIdLGF = new ListGridField("EmployeeId");
		
		ListGridField reportsToLGF = new ListGridField("ReportsTo");
		reportsToLGF.setOptionDataSource(DataSource.get("employees"));
		reportsToLGF.setValueField("EmployeeId");
		reportsToLGF.setDisplayField("Name");
		
		ListGridFieldAmount salaryLGFA = new ListGridFieldAmount("Salary");

		batchUploader.setGridFields(employeeIdLGF, reportsToLGF, salaryLGFA);
		batchUploader.draw();
	}

	public final class ListGridFieldAmount extends ListGridField {

		public ListGridFieldAmount(String name) {
			super(name);

			setCellFormatter(new CellFormatter() {
				public String format(Object value, ListGridRecord record, int rowNum, int colNum) {
					if (value == null)
						return null;
					else
						return formatNumberValue(value);
				}
			});
		}

		private String formatNumberValue(Object value) {
			String val = null;
			try {
				NumberFormat nf = NumberFormat.getFormat("###,###,###,###");
				val = nf.format(((Number) value).longValue());
			} catch (Exception e) {
				return value.toString();
			}
			return val + " EUR";
		}
	}
}

Changes to ds.xml

<field name="ReportsTo" title="Manager" type="integer" required="true" foreignKey="employees.EmployeeId" rootValue="1" />

war/ds changes:

Copy batchUpload.ds.xml there

The resulting log is:

=== 2015-02-24 13:29:43,897 [2-34] INFO  RequestContext - URL: '/builtinds/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0': Moz (Gecko) with Accept-Encoding header
=== 2015-02-24 13:29:43,897 [2-34] DEBUG IDACall - Header Name:Value pair: Cookie:isc_cState=ready; JSESSIONID=1igviebo1e92i2mj8qybleabu
=== 2015-02-24 13:29:43,898 [2-34] DEBUG IDACall - Header Name:Value pair: DNT:1
=== 2015-02-24 13:29:43,898 [2-34] DEBUG IDACall - Header Name:Value pair: Host:127.0.0.1:8888
=== 2015-02-24 13:29:43,898 [2-34] DEBUG IDACall - Header Name:Value pair: Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
=== 2015-02-24 13:29:43,898 [2-34] DEBUG IDACall - Header Name:Value pair: Content-Length:1262
=== 2015-02-24 13:29:43,898 [2-34] DEBUG IDACall - Header Name:Value pair: Content-Type:application/x-www-form-urlencoded; charset=UTF-8
=== 2015-02-24 13:29:43,898 [2-34] DEBUG IDACall - Header Name:Value pair: Accept-Language:de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
=== 2015-02-24 13:29:43,898 [2-34] DEBUG IDACall - Header Name:Value pair: Pragma:no-cache
=== 2015-02-24 13:29:43,898 [2-34] DEBUG IDACall - Header Name:Value pair: Connection:keep-alive
=== 2015-02-24 13:29:43,898 [2-34] DEBUG IDACall - Header Name:Value pair: Referer:http://127.0.0.1:8888/BuiltInDS.html?gwt.codesvr=127.0.0.1:9997
=== 2015-02-24 13:29:43,899 [2-34] DEBUG IDACall - Header Name:Value pair: User-Agent:Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0
=== 2015-02-24 13:29:43,899 [2-34] DEBUG IDACall - Header Name:Value pair: Cache-Control:no-cache
=== 2015-02-24 13:29:43,899 [2-34] DEBUG IDACall - Header Name:Value pair: Accept-Encoding:gzip, deflate
=== 2015-02-24 13:29:43,899 [2-34] DEBUG IDACall - session exists: 1igviebo1e92i2mj8qybleabu
=== 2015-02-24 13:29:43,899 [2-34] DEBUG IDACall - remote user: null
=== 2015-02-24 13:29:43,903 [2-33] INFO  RequestContext - URL: '/builtinds/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0': Moz (Gecko) with Accept-Encoding header
=== 2015-02-24 13:29:43,903 [2-33] DEBUG IDACall - Header Name:Value pair: Cookie:isc_cState=ready; JSESSIONID=1igviebo1e92i2mj8qybleabu
=== 2015-02-24 13:29:43,904 [2-33] DEBUG IDACall - Header Name:Value pair: DNT:1
=== 2015-02-24 13:29:43,904 [2-33] DEBUG IDACall - Header Name:Value pair: Host:127.0.0.1:8888
=== 2015-02-24 13:29:43,904 [2-33] DEBUG IDACall - Header Name:Value pair: Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
=== 2015-02-24 13:29:43,904 [2-33] DEBUG IDACall - Header Name:Value pair: Content-Length:1272
=== 2015-02-24 13:29:43,904 [2-33] DEBUG IDACall - Header Name:Value pair: Content-Type:multipart/form-data; boundary=---------------------------21173203358543
=== 2015-02-24 13:29:43,905 [2-33] DEBUG IDACall - Header Name:Value pair: Accept-Language:de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
=== 2015-02-24 13:29:43,905 [2-33] DEBUG IDACall - Header Name:Value pair: Connection:keep-alive
=== 2015-02-24 13:29:43,905 [2-33] DEBUG IDACall - Header Name:Value pair: Referer:http://127.0.0.1:8888/BuiltInDS.html?gwt.codesvr=127.0.0.1:9997
=== 2015-02-24 13:29:43,905 [2-33] DEBUG IDACall - Header Name:Value pair: User-Agent:Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0
=== 2015-02-24 13:29:43,905 [2-33] DEBUG IDACall - Header Name:Value pair: Accept-Encoding:gzip, deflate
=== 2015-02-24 13:29:43,905 [2-33] DEBUG IDACall - session exists: 1igviebo1e92i2mj8qybleabu
=== 2015-02-24 13:29:43,905 [2-33] DEBUG IDACall - remote user: null
=== 2015-02-24 13:29:43,914 [2-34] DEBUG XML - Parsed XML from (in memory stream): 7ms
=== 2015-02-24 13:29:43,915 [2-34] DEBUG ISCKeyedObjectPool - Borrowing object for 'transaction'
=== 2015-02-24 13:29:43,915 [2-34] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2015-02-24 13:29:43,916 [2-34] DEBUG ISCKeyedObjectPool - Borrowing object for 'List'
=== 2015-02-24 13:29:43,919 [2-34] DEBUG ISCKeyedObjectPool - Borrowing object for 'elem'
=== 2015-02-24 13:29:43,921 [2-34] DEBUG RPCManager - Processing 1 requests.
=== 2015-02-24 13:29:43,923 [2-34] DEBUG RPCManager - Request #1 (RPCRequest) data: {
    appID:"isc_builtin",
    className:"com.isomorphic.tools.BuiltinRPC",
    methodName:"setAttributes",
    arguments:[
        "session",
        [
            {
                name:"dsName",
                type:"hidden",
                value:"employees"
            },
            {
                name:"delimiter",
                type:"hidden",
                value:","
            },
            {
                name:"quoteString",
                type:"hidden",
                value:"\""
            }
        ],
        null
    ],
    is_ISC_RPC_DMI:true
}
=== 2015-02-24 13:29:43,924 [2-34] INFO  IDACall - Performing 1 operation(s)
=== 2015-02-24 13:29:43,947 [2-34] DEBUG XML - Parsed XML from jar:file:/C:/Users/myuser/workspace/lib/smartgwtee-4.1p/lib/isomorphic_core_rpc.jar!/isc_builtin.app.xml: 10ms
=== 2015-02-24 13:29:43,947 [2-34] DEBUG ISCKeyedObjectPool - Borrowing object for 'Application'
=== 2015-02-24 13:29:43,956 [2-33] DEBUG XML - Parsed XML from (in memory stream): 10ms
=== 2015-02-24 13:29:43,957 [2-34] DEBUG XML - Parsed XML from C:\Users\myuser\workspace\lib\smartgwtee-4.1p\samples\built-in-ds\war\builtinds\sc\system\schema\Application.ds.xml: 9ms
=== 2015-02-24 13:29:43,957 [2-34] DEBUG ISCKeyedObjectPool - Borrowing object for 'DataSource'
=== 2015-02-24 13:29:43,963 [2-34] DEBUG ISCKeyedObjectPool - Borrowing object for 'field'
=== 2015-02-24 13:29:43,964 [2-34] DEBUG ISCKeyedObjectPool - Borrowing object for 'DataSourceField'
=== 2015-02-24 13:29:43,971 [2-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'transaction'
=== 2015-02-24 13:29:43,972 [2-34] DEBUG ISCKeyedObjectPool - Borrowing object for 'ServerObject'
=== 2015-02-24 13:29:43,974 [2-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2015-02-24 13:29:43,976 [2-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'List'
=== 2015-02-24 13:29:43,977 [2-34] DEBUG ISCKeyedObjectPool - Borrowing object for 'Method'
=== 2015-02-24 13:29:43,979 [2-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'elem'
=== 2015-02-24 13:29:43,982 [2-33] DEBUG RPCManager - Processing 1 requests.
=== 2015-02-24 13:29:43,982 [2-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'batchUpload'
=== 2015-02-24 13:29:43,983 [2-33] DEBUG DSRequest - Caching instance 29 of DS batchUpload from DSRequest.getDataSource()
=== 2015-02-24 13:29:43,985 [2-33] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
    },
    values:{
        file:"Employees.csv",
        dsName:"employees",
        delimiter:",",
        quoteString:"\""
    },
    operationConfig:{
        dataSource:"batchUpload",
        repo:null,
        operationType:"update",
        textMatchStyle:"exact"
    },
    componentId:"isc_DynamicForm_0",
    appID:"builtinApplication",
    operation:"upload",
    oldValues:{
        dsName:"employees",
        delimiter:",",
        quoteString:"\""
    }
}
=== 2015-02-24 13:29:43,985 [2-33] INFO  IDACall - Performing 1 operation(s)
=== 2015-02-24 13:29:43,986 [2-33] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2015-02-24 13:29:43,986 [2-33] DEBUG DeclarativeSecurity - DataSource batchUpload is not in the pre-checked list, processing...
[WARN] Server class 'com.isomorphic.tools.BatchUpload' could not be found in the web app, but was found on the system classpath
   [WARN] Adding classpath entry 'file:/C:/Users/myuser/workspace/lib/smartgwtee-4.1p/lib/isomorphic_tools.jar' to the web app classpath for this session
   For additional info see: file:/C:/eclipse/plugins/com.google.gwt.eclipse.sdkbundle_2.6.1/gwt-2.6.1/doc/helpInfo/webAppClassPath.html
=== 2015-02-24 13:29:43,998 [2-34] DEBUG RPCDMI - appConfig: isc.Application.create({
    rpcBindings:[
        {
            ID:"builtin",
            className:"com.isomorphic.rpc.BuiltinRPC",
            visibleMethods:[
                {
                    name:"downloadWSDL"
                },
                {
                    name:"downloadClientContent"
                },
                {
                    name:"downloadClientExport"
                },
                {
                    name:"xmlToJS"
                },
                {
                    name:"uploadProgressCheck"
                },
                {
                    name:"saveFile"
                },
                {
                    name:"appendToFile"
                },
                {
                    name:"loadFile"
                },
                {
                    name:"deleteFile"
                },
                {
                    name:"loadSharedXML"
                },
                {
                    name:"saveSharedXML"
                },
                {
                    name:"getAvailableScriptEngines"
                },
                {
                    name:"devConsoleEvalServerScript"
                },
                {
                    name:"evalJava"
                },
                {
                    name:"getLogNames"
                },
                {
                    name:"getLogEntries"
                },
                {
                    name:"clearLogEntries"
                },
                {
                    name:"getLogThresholds"
                },
                {
                    name:"setLogThreshold"
                },
                {
                    name:"setTemporaryLogThreshold"
                },
                {
                    name:"revertTemporaryLogThresholds"
                },
                {
                    name:"getPdfObject"
                },
                {
                    name:"exportImage"
                },
                {
                    name:"areServerTimingsTracked"
                },
                {
                    name:"trackServerTimings"
                }
            ]
        },
        {
            ID:"builtin_tools",
            className:"com.isomorphic.tools.BuiltinRPC",
            visibleMethods:[
                {
                    name:"getDataSourceFromTable"
                },
                {
                    name:"getDataSourceJSONFromTable"
                },
                {
                    name:"getDataSourceFromHibernateMapping"
                },
                {
                    name:"getDataSourceJSONFromHibernateMapping"
                },
                {
                    name:"getTables"
                },
                {
                    name:"getFieldsFromTable"
                },
                {
                    name:"getBeanFields"
                },
                {
                    name:"getHibernateBeans"
                },
                {
                    name:"getDatabaseProductNameAndVersion"
                },
                {
                    name:"getDatabaseTableTypes"
                },
                {
                    name:"setAttributes"
                },
                {
                    name:"clearAttributes"
                },
                {
                    name:"getAttributes"
                },
                {
                    name:"getAttribute"
                },
                {
                    name:"getDataSourceConfigFromJavaClass"
                },
                {
                    args:"cName",
                    language:"groovy",
                    name:"getJavaSource",
                    script:"\n                    if (!com.isomorphic.auth.DevModeAuthFilter.devModeAuthorized(request)) throw new Exception(\"Not Authorized\");                    \n                    //import org.apache.bcel.Repository;\n\n                    try {\n                        return org.apache.bcel.Repository.lookupClass(cName).toString();\n                    } catch (Throwable e) {\n                        return \"Unable to reverse engineer class \"+cName+\": \"+e.getMessage();\n                    }\n                "
                },
                {
                    name:"loadDataSource"
                },
                {
                    name:"dsFromXML"
                },
                {
                    name:"dsConfigFromXML"
                },
                {
                    name:"getDefinedDataSources"
                }
            ]
        },
        {
            ID:"builtin_adminconsole",
            className:"com.isomorphic.tools.AdminConsole",
            visibleMethods:[
                {
                    name:"getDefinedDatabases"
                },
                {
                    name:"testDB"
                },
                {
                    name:"saveDBConfig"
                },
                {
                    name:"setDefaultDB"
                },
                {
                    name:"importDataSources"
                },
                {
                    name:"discoverJNDIDatabases"
                }
            ]
        }
    ]
})

=== 2015-02-24 13:29:44,011 [2-34] DEBUG RPCDMI - rpc returned data
=== 2015-02-24 13:29:44,012 [2-34] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
=== 2015-02-24 13:29:44,013 [2-34] INFO  Compression - /builtinds/sc/IDACall: 67 -> 73 bytes
[WARN] Server class 'org.apache.commons.cli.CommandLineParser' could not be found in the web app, but was found on the system classpath
   [WARN] Adding classpath entry 'file:/C:/Users/myuser/workspace/lib/smartgwtee-4.1p/lib/commons-cli-1.2.jar' to the web app classpath for this session
   For additional info see: file:/C:/eclipse/plugins/com.google.gwt.eclipse.sdkbundle_2.6.1/gwt-2.6.1/doc/helpInfo/webAppClassPath.html
=== 2015-02-24 13:29:44,029 [2-33] DEBUG BatchUpload - Rows: [
    {
        Salary:"-123",
        ReportsTo:"Ralph Brogan",
        EmployeeId:"1000001"
    },
    {
        Salary:"456",
        ReportsTo:"Bhushan Sambhus",
        EmployeeId:"1000002"
    }
]
[WARN] Server class 'org.apache.commons.lang.StringUtils' could not be found in the web app, but was found on the system classpath
   [WARN] Adding classpath entry 'file:/C:/Users/myuser/workspace/lib/smartgwtee-4.1p/lib/commons-lang-2.4.jar' to the web app classpath for this session
   For additional info see: file:/C:/eclipse/plugins/com.google.gwt.eclipse.sdkbundle_2.6.1/gwt-2.6.1/doc/helpInfo/webAppClassPath.html
=== 2015-02-24 13:29:44,221 [2-33] DEBUG ValidationContext - Adding validation errors at path '/employees/ReportsTo/ReportsTo': {errorMessage=Must be a whole number.}
=== 2015-02-24 13:29:44,222 [2-33] DEBUG ValidationContext - Adding validation errors at path '/employees/ReportsTo/ReportsTo': {errorMessage=Must be a whole number.}
=== 2015-02-24 13:29:44,223 [2-33] INFO  DSResponse - DSResponse: Map with 2 keys
=== 2015-02-24 13:29:44,223 [2-33] DEBUG RPCManager - Content type for RPC transaction: text/html; charset=UTF-8
=== 2015-02-24 13:29:44,224 [2-33] DEBUG RPCManager - DMI response, dropExtraFields: false
=== 2015-02-24 13:29:44,230 [2-33] INFO  Compression - /builtinds/sc/IDACall: 926 -> 536 bytes
=== 2015-02-24 13:29:45,219 [2-33] INFO  RequestContext - URL: '/builtinds/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0': Moz (Gecko) with Accept-Encoding header
=== 2015-02-24 13:29:45,220 [2-33] DEBUG IDACall - Header Name:Value pair: Cookie:isc_cState=ready; JSESSIONID=1igviebo1e92i2mj8qybleabu
=== 2015-02-24 13:29:45,220 [2-33] DEBUG IDACall - Header Name:Value pair: DNT:1
=== 2015-02-24 13:29:45,220 [2-33] DEBUG IDACall - Header Name:Value pair: Host:127.0.0.1:8888
=== 2015-02-24 13:29:45,220 [2-33] DEBUG IDACall - Header Name:Value pair: Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
=== 2015-02-24 13:29:45,220 [2-33] DEBUG IDACall - Header Name:Value pair: Content-Length:838
=== 2015-02-24 13:29:45,220 [2-33] DEBUG IDACall - Header Name:Value pair: Content-Type:application/x-www-form-urlencoded; charset=UTF-8
=== 2015-02-24 13:29:45,220 [2-33] DEBUG IDACall - Header Name:Value pair: Accept-Language:de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
=== 2015-02-24 13:29:45,220 [2-33] DEBUG IDACall - Header Name:Value pair: Pragma:no-cache
=== 2015-02-24 13:29:45,221 [2-33] DEBUG IDACall - Header Name:Value pair: Connection:keep-alive
=== 2015-02-24 13:29:45,221 [2-33] DEBUG IDACall - Header Name:Value pair: Referer:http://127.0.0.1:8888/BuiltInDS.html?gwt.codesvr=127.0.0.1:9997
=== 2015-02-24 13:29:45,221 [2-33] DEBUG IDACall - Header Name:Value pair: User-Agent:Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0
=== 2015-02-24 13:29:45,221 [2-33] DEBUG IDACall - Header Name:Value pair: Cache-Control:no-cache
=== 2015-02-24 13:29:45,221 [2-33] DEBUG IDACall - Header Name:Value pair: Accept-Encoding:gzip, deflate
=== 2015-02-24 13:29:45,221 [2-33] DEBUG IDACall - session exists: 1igviebo1e92i2mj8qybleabu
=== 2015-02-24 13:29:45,221 [2-33] DEBUG IDACall - remote user: null
=== 2015-02-24 13:29:45,233 [2-33] DEBUG XML - Parsed XML from (in memory stream): 6ms
=== 2015-02-24 13:29:45,233 [2-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'transaction'
=== 2015-02-24 13:29:45,234 [2-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
=== 2015-02-24 13:29:45,234 [2-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'List'
=== 2015-02-24 13:29:45,236 [2-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'elem'
=== 2015-02-24 13:29:45,237 [2-33] DEBUG RPCManager - Processing 1 requests.
=== 2015-02-24 13:29:45,237 [2-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'employees'
=== 2015-02-24 13:29:45,237 [2-33] DEBUG DSRequest - Caching instance 18 of DS employees from DSRequest.getDataSource()
=== 2015-02-24 13:29:45,238 [2-33] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
    },
    operationConfig:{
        dataSource:"employees",
        repo:null,
        operationType:"fetch"
    },
    componentId:"isc_BatchUploader_0_grid",
    appID:"builtinApplication",
    operation:"employees_fetch",
    oldValues:null
}
=== 2015-02-24 13:29:45,239 [2-33] INFO  IDACall - Performing 1 operation(s)
=== 2015-02-24 13:29:45,240 [2-33] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2015-02-24 13:29:45,240 [2-33] DEBUG DeclarativeSecurity - DataSource employees is not in the pre-checked list, processing...
=== 2015-02-24 13:29:45,240 [2-33] DEBUG AppBase - [builtinApplication.employees_fetch] No userTypes defined, allowing anyone access to all operations for this application
=== 2015-02-24 13:29:45,241 [2-33] DEBUG AppBase - [builtinApplication.employees_fetch] No public zero-argument method named '_employees_fetch' found, performing generic datasource operation
=== 2015-02-24 13:29:45,242 [2-33] INFO  SQLDataSource - [builtinApplication.employees_fetch] Performing fetch operation with
	criteria: {}	values: {}
=== 2015-02-24 13:29:45,267 [2-33] INFO  SQLWhereClause - [builtinApplication.employees_fetch] empty condition
=== 2015-02-24 13:29:45,269 [2-33] INFO  SQLDataSource - [builtinApplication.employees_fetch] derived query: SELECT $defaultSelectClause FROM $defaultTableClause WHERE $defaultWhereClause
=== 2015-02-24 13:29:45,281 [2-33] INFO  SQLDataSource - [builtinApplication.employees_fetch] 18: Executing SQL query on 'HSQLDB': SELECT employeeTable.userOrder, employeeTable.Name, employeeTable.EmployeeId, employeeTable.ReportsTo, employeeTable.Job, employeeTable.Email, employeeTable.EmployeeType, employeeTable.EmployeeStatus, employeeTable.Salary, employeeTable.OrgUnit, employeeTable.Gender, employeeTable.MaritalStatus FROM employeeTable WHERE ('1'='1')
[WARN] Server class 'com.isomorphic.hibernate.HibernateDataSource' could not be found in the web app, but was found on the system classpath
   [WARN] Adding classpath entry 'file:/C:/Users/myuser/workspace/lib/smartgwtee-4.1p/lib/isomorphic_hibernate.jar' to the web app classpath for this session
   For additional info see: file:/C:/eclipse/plugins/com.google.gwt.eclipse.sdkbundle_2.6.1/gwt-2.6.1/doc/helpInfo/webAppClassPath.html
=== 2015-02-24 13:29:45,312 [2-29] INFO  Download - Returning 304: Not modified on conditional get of: C:\Users\myuser\workspace\lib\smartgwtee-4.1p\samples\built-in-ds\war\builtinds\sc\skins\Enterprise\images\actions\exclamation.png
=== 2015-02-24 13:29:45,379 [2-33] INFO  PoolManager - [builtinApplication.employees_fetch] SmartClient pooling disabled for 'HSQLDB' objects
=== 2015-02-24 13:29:45,381 [2-33] DEBUG PoolableSQLConnectionFactory - [builtinApplication.employees_fetch] Initializing SQL config for 'HSQLDB' from system config - using DriverManager:  org.hsqldb.jdbcDriver
=== 2015-02-24 13:29:45,381 [2-33] DEBUG PoolableSQLConnectionFactory - [builtinApplication.employees_fetch] org.hsqldb.jdbcDriver lookup successful
=== 2015-02-24 13:29:45,382 [2-33] DEBUG PoolableSQLConnectionFactory - [builtinApplication.employees_fetch] DriverManager fetching connection for HSQLDB via jdbc url jdbc:hsqldb:hsql://localhost/isomorphic
=== 2015-02-24 13:29:45,382 [2-33] DEBUG PoolableSQLConnectionFactory - [builtinApplication.employees_fetch] Passing JDBC URL only to getConnection
=== 2015-02-24 13:29:45,503 [2-33] DEBUG PoolableSQLConnectionFactory - [builtinApplication.employees_fetch] makeObject() created an unpooled Connection '73942687'
=== 2015-02-24 13:29:45,503 [2-33] DEBUG SQLConnectionManager - [builtinApplication.employees_fetch] Borrowed connection '73942687'
=== 2015-02-24 13:29:45,504 [2-33] DEBUG SQLDriver - [builtinApplication.employees_fetch] About to execute SQL query in 'HSQLDB' using connection '73942687'
=== 2015-02-24 13:29:45,504 [2-33] INFO  SQLDriver - [builtinApplication.employees_fetch] Executing SQL query on 'HSQLDB': SELECT employeeTable.userOrder, employeeTable.Name, employeeTable.EmployeeId, employeeTable.ReportsTo, employeeTable.Job, employeeTable.Email, employeeTable.EmployeeType, employeeTable.EmployeeStatus, employeeTable.Salary, employeeTable.OrgUnit, employeeTable.Gender, employeeTable.MaritalStatus FROM employeeTable WHERE ('1'='1')
=== 2015-02-24 13:29:45,575 [2-33] INFO  DSResponse - [builtinApplication.employees_fetch] DSResponse: List with 115 items
=== 2015-02-24 13:29:45,576 [2-33] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
=== 2015-02-24 13:29:45,576 [2-33] DEBUG RPCManager - non-DMI response, dropExtraFields: false
=== 2015-02-24 13:29:45,624 [2-33] DEBUG SQLDriver - Freeing SQLDriver dbConnection 73942687
=== 2015-02-24 13:29:45,625 [2-33] DEBUG SQLConnectionManager - About to close JDBCConnection with hashcode "73942687"
=== 2015-02-24 13:29:45,627 [2-33] INFO  Compression - /builtinds/sc/IDACall: 27015 -> 4132 bytes

As you can see, the log includes the

=== 2015-02-24 13:16:35,727 [7-36] DEBUG ValidationContext - Adding validation errors at path '/employees/ReportsTo/ReportsTo': {errorMessage=Must be a whole number.}
=== 2015-02-24 13:16:35,728 [7-36] DEBUG ValidationContext - Adding validation errors at path '/employees/ReportsTo/ReportsTo': {errorMessage=Must be a whole number.}

but it is working as in my case regarding the rest (see screenshot). I'll double check if I get the integer-validation-error as well.

Please note that the change to the .ds.xml is really necessary, as the reportsTo-ListGridField is not displayed otherwise.

Regarding your comment "the importer knows nothing about it": In this case, the field is even defined as FK (not true in my 1st example). I agree that this might be difficult to deal with, but a BatchUpload with IDs instead of human-readable unique identifiers would be of way less use. Perhaps you could validate the ListGrid after the upload, before you draw?

Best regards,
Blama

[Isomorphic]

Are you sure that the DataSource and .csv settings you post above are correct? When trying to reproduce this, we just see it fail at the initial import stage, because "LEADTYPE_ID" is declared as an integer but the .csv file provides strings ("Event"). Sure, the point of the ODS is to do that integer-to-string mapping, but the importer knows nothing about it.

[Isomorphic]

Hi Blama
This is just a quick note to let you know that although this has sat for a few days it is not forgotten. We are taking a look and will let you know when we have some information for you.

Thanks
Isomorphic Software

[Blama]

Just to show that "Event" really exists and has no trailing spaces or whatsoever, see screenshot.

[Blama]

Server log: (please note that the Oracle error is expected. Problem is with the other ADD request that fails validation (search LEADTYPE_ID:"Event"))

=== 2015-02-19 13:11:58,322 [c-26] INFO  Download - Returning 304: Not modified on conditional get of: C:\Program Files\Apache Software Foundation\Tomcat 7.0\webapps\lms\lms\sc\skins\Simplicity\images\Dialog\warn.png
=== 2015-02-19 13:12:00,083 [c-26] INFO  RequestContext - URL: '/lms/lms/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0': Moz (Gecko) with Accept-Encoding header
=== 2015-02-19 13:12:00,091 [c-26] DEBUG RPCManager - Processing 2 requests.
=== 2015-02-19 13:12:00,093 [c-26] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    values:{
        POTREV_RESELLER:123,
        LEADTYPE_ID:22,
        _selection_17:false,
        ID:null
    },
    operationConfig:{
        dataSource:"V_LEADUPLOAD",
        repo:null,
        operationType:"add",
        textMatchStyle:"exact"
    },
    appID:"builtinApplication",
    operation:"V_LEADUPLOAD_add",
    oldValues:{
        POTREV_RESELLER:123,
        LEADTYPE_ID:22,
        _selection_17:false,
        ID:null
    },
    criteria:{
    }
}
=== 2015-02-19 13:12:00,094 [c-26] DEBUG RPCManager - Request #2 (DSRequest) payload: {
    values:{
        POTREV_RESELLER:4456,
        [B]LEADTYPE_ID:"Event",[/B]
        _selection_17:true,
        ID:null
    },
    operationConfig:{
        dataSource:"V_LEADUPLOAD",
        repo:null,
        operationType:"add",
        textMatchStyle:"exact"
    },
    appID:"builtinApplication",
    operation:"V_LEADUPLOAD_add",
    oldValues:{
        POTREV_RESELLER:4456,
        LEADTYPE_ID:"Event",
        _selection_17:true,
        ID:null
    },
    criteria:{
    }
}
=== 2015-02-19 13:12:00,094 [c-26] INFO  LMSIDACall - Performing 2 operation(s)
=== 2015-02-19 13:12:00,096 [c-26] DEBUG AppBase - [builtinApplication.V_LEADUPLOAD_add] No userTypes defined, allowing anyone access to all operations for this application
=== 2015-02-19 13:12:00,096 [c-26] DEBUG AppBase - [builtinApplication.V_LEADUPLOAD_add] No public zero-argument method named '_V_LEADUPLOAD_add' found, performing generic datasource operation
=== 2015-02-19 13:12:00,097 [c-26] INFO  SQLDataSource - [builtinApplication.V_LEADUPLOAD_add] Performing add operation with
	criteria: {POTREV_RESELLER:123,LEADTYPE_ID:22,_selection_17:false,ID:null,CREATED_BY:"1085",CREATED_AT:new Date(1424347920095)}	values: {POTREV_RESELLER:123,LEADTYPE_ID:22,_selection_17:false,ID:null,CREATED_BY:"1085",CREATED_AT:new Date(1424347920095),TENANT_ID:1}
=== 2015-02-19 13:12:00,097 [c-26] INFO  SQLValuesClause - [builtinApplication.V_LEADUPLOAD_add] Ignored data for non-existent or included columns: [_selection_17]
=== 2015-02-19 13:12:00,098 [c-26] DEBUG SQLValuesClause - [builtinApplication.V_LEADUPLOAD_add] Sequences: {ID=__default}
=== 2015-02-19 13:12:00,101 [c-26] DEBUG SQLConnectionManager - [builtinApplication.V_LEADUPLOAD_add] Borrowed connection '736744514'
=== 2015-02-19 13:12:00,101 [c-26] DEBUG SQLTransaction - [builtinApplication.V_LEADUPLOAD_add] Started new Oracle transaction "736744514"
=== 2015-02-19 13:12:00,102 [c-26] DEBUG SQLDriver - [builtinApplication.V_LEADUPLOAD_add] About to execute SQL update in 'Oracle' using connection'736744514'
=== 2015-02-19 13:12:00,102 [c-26] INFO  SQLDriver - [builtinApplication.V_LEADUPLOAD_add] Executing SQL update on 'Oracle': INSERT INTO T_LEAD (CREATED_AT, CREATED_BY, LEADTYPE_ID, POTREV_RESELLER, TENANT_ID, ID) VALUES (TO_DATE('2015-02-19 13:12:00','YYYY-MM-DD HH24:MI:SS'), '1085', 22, 123, 1, T_LEAD_ID.NextVal)
=== 2015-02-19 13:12:00,105 [c-26] DEBUG SQLDriver - [builtinApplication.V_LEADUPLOAD_add] SequenceMode is not JDBC_DRIVER, skipping search for generated values
=== 2015-02-19 13:12:00,106 [c-26] DEBUG SQLDriver - [builtinApplication.V_LEADUPLOAD_add] FAILED to execute SQL update in 'Oracle' using connection'736744514'
=== 2015-02-19 13:12:00,106 [c-26] DEBUG DataSourceDMI - Invocation threw exception
java.sql.SQLIntegrityConstraintViolationException: ORA-01400: cannot insert NULL into (???)

	at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
	at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
	at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
	at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
	at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
	at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
	at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
	at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1046)
	at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1336)
	at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
	at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3694)
	at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1354)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:102)
	at com.isomorphic.sql.SQLDriver.doUpdate(SQLDriver.java:876)
	at com.isomorphic.sql.SQLDriver.update(SQLDriver.java:780)
	at com.isomorphic.sql.SQLDriver.executeUpdate(SQLDriver.java:961)
	at com.isomorphic.sql.SQLDataSource.executeNativeUpdate(SQLDataSource.java:575)
	at com.isomorphic.sql.SQLDataSource.SQLExecute(SQLDataSource.java:1989)
	at com.isomorphic.sql.SQLDataSource.processRequest(SQLDataSource.java:448)
	at com.isomorphic.sql.SQLDataSource.executeAdd(SQLDataSource.java:401)
	at com.lmscompany.lms.server.LMSSQLDataSource.executeAdd(LMSSQLDataSource.java:53)
	at com.isomorphic.datasource.DataSource.execute(DataSource.java:1609)
	at com.isomorphic.application.AppBase.executeDefaultDSOperation(AppBase.java:723)
	at com.isomorphic.application.AppBase.executeAppOperation(AppBase.java:658)
	at com.isomorphic.application.AppBase.execute(AppBase.java:491)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2535)
	at com.lmscompany.lms.server.worker.V_LEADUPLOAD.add(V_LEADUPLOAD.java:11)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.isomorphic.base.Reflection.adaptArgsAndInvoke(Reflection.java:975)
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:416)
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:64)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2531)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:215)
	at com.lmscompany.lms.server.LMSIDACall.handleDSRequest(LMSIDACall.java:57)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:172)
	at com.lmscompany.lms.server.LMSIDACall.processRequest(LMSIDACall.java:39)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:73)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:152)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.isomorphic.servlet.CompressionFilter.doFilter(CompressionFilter.java:260)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:611)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:409)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1044)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:315)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
=== 2015-02-19 13:12:00,108 [c-26] WARN  RequestContext - dsRequest.execute() failed: 
java.sql.SQLIntegrityConstraintViolationException: ORA-01400: cannot insert NULL into (???)

	at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
	at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
	at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
	at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
	at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
	at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
	at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
	at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1046)
	at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1336)
	at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
	at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3694)
	at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1354)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:102)
	at com.isomorphic.sql.SQLDriver.doUpdate(SQLDriver.java:876)
	at com.isomorphic.sql.SQLDriver.update(SQLDriver.java:780)
	at com.isomorphic.sql.SQLDriver.executeUpdate(SQLDriver.java:961)
	at com.isomorphic.sql.SQLDataSource.executeNativeUpdate(SQLDataSource.java:575)
	at com.isomorphic.sql.SQLDataSource.SQLExecute(SQLDataSource.java:1989)
	at com.isomorphic.sql.SQLDataSource.processRequest(SQLDataSource.java:448)
	at com.isomorphic.sql.SQLDataSource.executeAdd(SQLDataSource.java:401)
	at com.lmscompany.lms.server.LMSSQLDataSource.executeAdd(LMSSQLDataSource.java:53)
	at com.isomorphic.datasource.DataSource.execute(DataSource.java:1609)
	at com.isomorphic.application.AppBase.executeDefaultDSOperation(AppBase.java:723)
	at com.isomorphic.application.AppBase.executeAppOperation(AppBase.java:658)
	at com.isomorphic.application.AppBase.execute(AppBase.java:491)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2535)
	at com.lmscompany.lms.server.worker.V_LEADUPLOAD.add(V_LEADUPLOAD.java:11)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.isomorphic.base.Reflection.adaptArgsAndInvoke(Reflection.java:975)
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:416)
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:64)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2531)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:215)
	at com.lmscompany.lms.server.LMSIDACall.handleDSRequest(LMSIDACall.java:57)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:172)
	at com.lmscompany.lms.server.LMSIDACall.processRequest(LMSIDACall.java:39)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:73)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:152)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.isomorphic.servlet.CompressionFilter.doFilter(CompressionFilter.java:260)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:611)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:409)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1044)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:315)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
=== 2015-02-19 13:12:00,112 [c-26] DEBUG ValidationContext - Adding validation errors at path '/V_LEADUPLOAD/LEADTYPE_ID/LEADTYPE_ID': {errorMessage=Must be a whole number.}
=== 2015-02-19 13:12:00,113 [c-26] INFO  Validation - Validation error: [
    {
        LEADTYPE_ID:{
            errorMessage:"Must be a whole number."
        }
    }
]
=== 2015-02-19 13:12:00,113 [c-26] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
=== 2015-02-19 13:12:00,114 [c-26] DEBUG SQLTransaction - Rolling back Oracle transaction "736744514"
=== 2015-02-19 13:12:00,114 [c-26] DEBUG RPCManager - non-DMI response, dropExtraFields: false
=== 2015-02-19 13:12:00,115 [c-26] DEBUG RPCManager - non-DMI response, dropExtraFields: false
=== 2015-02-19 13:12:00,115 [c-26] DEBUG SQLTransaction - getConnection() found transactional connection for Oracle with hashcode "736744514"
=== 2015-02-19 13:12:00,115 [c-26] DEBUG SQLTransaction - Ending Oracle transaction "736744514"
=== 2015-02-19 13:12:00,116 [c-26] DEBUG SQLConnectionManager - About to close ISCPoolableConnection with hashcode "736744514"
=== 2015-02-19 13:12:00,117 [c-26] INFO  Compression - /lms/lms/sc/IDACall: 354 -> 243 bytes
=== 2015-02-19 13:12:00,980 [c-26] INFO  Download - Returning 304: Not modified on conditional get of: C:\Program Files\Apache Software Foundation\Tomcat 7.0\webapps\lms\lms\sc\skins\Simplicity\images\Dialog\say.png
=== 2015-02-19 13:12:00,981 [ec-4] INFO  RequestContext - URL: '/lms/lms/sc/system/reference/skin/images/opener_opened.png', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0': Moz (Gecko) with Accept-Encoding header
=== 2015-02-19 13:12:00,982 [c-21] INFO  RequestContext - URL: '/lms/lms/sc/system/reference/skin/images/server_network_closed.png', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0': Moz (Gecko) with Accept-Encoding header
=== 2015-02-19 13:12:00,982 [c-28] INFO  RequestContext - URL: '/lms/lms/sc/skins/Enterprise/images/DynamicForm/unchecked_Disabled.png', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0': Moz (Gecko) with Accept-Encoding header
=== 2015-02-19 13:12:00,982 [c-31] INFO  RequestContext - URL: '/lms/lms/sc/system/reference/skin/images/server_client_exchange.png', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0': Moz (Gecko) with Accept-Encoding header
=== 2015-02-19 13:12:00,985 [c-21] INFO  Download - File C:/Program Files/Apache Software Foundation/Tomcat 7.0/webapps/lms/lms/sc/system/reference/skin/images/server_network_closed.png not found, sending 404
=== 2015-02-19 13:12:00,985 [ec-4] INFO  Download - File C:/Program Files/Apache Software Foundation/Tomcat 7.0/webapps/lms/lms/sc/system/reference/skin/images/opener_opened.png not found, sending 404
=== 2015-02-19 13:12:00,988 [c-28] INFO  Download - done streaming: C:/Program Files/Apache Software Foundation/Tomcat 7.0/webapps/lms/lms/sc/skins/Enterprise/images/DynamicForm/unchecked_Disabled.png
=== 2015-02-19 13:12:00,988 [c-31] INFO  Download - done streaming: C:/Program Files/Apache Software Foundation/Tomcat 7.0/webapps/lms/lms/sc/system/reference/skin/images/server_client_exchange.png

Client-request screenshots are included.

Summarized my bug reports are:

1. FK resolution (or more specific displayField-to-valueField resolution) does only happen after selecting the very same value from the SelectItem
2. Formatters are only applied after editing the text, not already after uploading the data

Best regards,
Blama

[Blama]

Hi Isomorphic,

I have a problem with current 4.1p (v9.1p_2015-02-17) and BatchUploader. Basically my use-case is like the BatchUploader sample, but with the "Unit"-field being a optionDataSource instead of a valueMap like in the sample.

This is my setup:
Client Code:

batchUploader = new BatchUploader();
batchUploader.setWidth(600);
batchUploader.setUploadDataSource(leadUploadDS);

ListGridField leadtypeLGFL = new ListGridField("LEADTYPE_ID");
leadtypeLGFL.setOptionDataSource(DataSource.get(DatasourceEnum.T_LEADTYPE.getValue()));
leadtypeLGFL.setValueField("ID");
leadtypeLGFL.setDisplayField("NAME");
ListGridFieldAmount potrevResellerLGFA = new ListGridFieldAmount("POTREV_RESELLER");

batchUploader.setGridFields(leadtypeLGFL, potrevResellerLGFA);

ListGridFieldAmount.java (for not-applied formatting):

public final class ListGridFieldAmount extends ListGridField {

	public ListGridFieldAmount(String name) {
		super(name);

		setCellFormatter(new CellFormatter() {
			public String format(Object value, ListGridRecord record, int rowNum, int colNum) {
				if (value == null)
					return null;
				else
					return formatNumberValue(value);
			}
		});
	}

	private static String formatNumberValue(Object value) {
		String val = null;
		try {
			NumberFormat nf = NumberFormat.getFormat("###,###,###,###");
			val = nf.format(((Number) value).longValue());
		} catch (Exception e) {
			return value.toString();
		}
		return val + " €";
	}
}

.ds.xml fields:

<fields>
	<field primaryKey="true" hidden="true" name="ID" type="sequence"/>
	<field hidden="true" name="TENANT_ID" type="integer" canEdit="false" />
	<field foreignKey="V_USER_CREATED_BY.ID" relatedTableAlias="USER_CREATED_BY" name="CREATED_BY" title="Erstellt von" type="creator" />
	<field name="CREATED_AT" title="Erstellt am" type="creatorTimestamp" />

	<field name="LEADTYPE_ID" title="Leadtyp1" type="integer" />
	<field name="POTREV_RESELLER" title="Gesamtprojekt netto (geschätzt)" type="integer">
		<validators>
			<validator type="integerRange" min="1" errorMessage="Bitte geben Sie einen Betrag größer Null ein." />
		</validators>
	</field>
</fields>

.csv content

"LEADTYPE_ID","POTREV_RESELLER"
"Event",-123
"Event",456

I'll show my problems in the 1st picture.
As you can see the number is not with € sign and the FK is unresolved.

In the 2nd picture you can see that the supplied "Event" is valid.

3rd picture shows that the ListGrid thinks the value changed after selecting from the SelectItem (text is blue)

4th picture shows amount-values after setting the amounts to 123 and 456 (now with Euro-sign).

Now I'm ready to commit (one error left in place in order to show what data is actually sent).

I'll show this in the next post.

Best regards,
Blama