[Isomorphic]

Specifying a valueMap on a field isn't sufficient to enforce that submitted values fall within the valueMap - we allow fields to use valueMaps as essentially a suggested value, but also allow additional values to be applied (for example via a ComboBoxItem with addUnknownValues set to true).

From the docs for DataSourceField.valueMap:

...To enforce that a field should be constrained to only the values in the valueMap, either declare field.type as "enum", or use a ValidatorType of "isOneOf" with explicitly listed values. Otherwise, although a normal SelectItem control will only allow values from the valueMap to be entered, other controls such as a ComboBox will allow other values to be entered.

No comment on the second warning. If you can get it to reproduce we'd recommend looking at the RPC Tab in the dev console to see what the request / response looks like, and the server logs for the transaction. Hopefully this'll reveal more information about what's going on...

[Blama]

Hi Isomorphic,

to #1: I just wanted to suggest that Validator.isOneOf could use the valueMap so that there is no need to repeat it, until I saw the docs:

Tests whether the value for this field matches any value from an arbitrary list of acceptable values. The set of acceptable values is specified via the list property on the validator, which should be set to an array of values. If validator.list is not supplied, the valueMap for the field will be used. If there is no valueMap, not providing validator.list is an error.

It's already there - once again very impressive!

Best regards
Blama

[Blama]

Hi Isomorphic,

to #2:
I got to the bottom of this. Funny thing is that the responses are exactly the same for good and bad request, see the attached files. So it is related to the internal state of the BatchUploader.
Steps to reproduce:

1. Make .csv file good ("NOTKNOWN" -> "Charles Madigan)", enter new IDs in 1st column (important)
2. Upload the file and commit
3. 2 records added success message is shown
4. Upload the very same file again and commit
5. You'll get an Unique Constraint error in a popup (not a validation error!)
6. Change csv file, enter new IDs in 1st column
7. Upload the changed file and commit
8. "null" warning is shown.

Best regards
Blama

[Blama]

Hi Isomorphic,

the main bug from this thread is NOT fixed. It works in the sample, but only because name and uploadFieldName is the same in the .ds.xml.
If you change uploadFieldName="ReportsTo" to uploadFieldName="ReportsTo2" and also change the column name in the .csv to ReportsTo2 you'll notice that the string->ID lookup does not take place and the validator uses a wrong (string) value.
If you then also change the name to ReportsTo2 it is working again.

The fix here is most likely easy, but identifying this as the reason took me half a day.

Best regards
Blama

[Blama]

Hi Isomorphic,

could you also make the string -> ID lookup query case sensitivity configurable (like you did here for Validator.ISUNIQUE) or change the default sensitivity to CS?

Best regards
Blama

[Isomorphic]

We see the problem with the uploadFieldName mapping - a fix will be going in today or possibly Monday, once some internal discussions about exact required behavior have been resolved.

On your second request - can you provide a use case where this would be useful/helpful? For data import, we attempt to be as accommodating as possible with field identifiers because the import source might well be a spreadsheet provided by an end user. So if you have a field definition like this:

<field name="Foo" title="Bar" />

the intent is that we would recognise values for this field from a case-insensitive check of both "Foo" and "Bar". It seems like this could only be a problem if this leads to a collision - for example, if your import data contains two separate fields called "FOO" and "Foo", or "foo" and "bar". Does your use case have such a collision?

[Blama]

Hi Isomorphic,

We see the problem with the uploadFieldName mapping - a fix will be going in today or possibly Monday, once some internal discussions about exact required behavior have been resolved.

Great, thank you!

On your second request - can you provide a use case where this would be useful/helpful? For data import, we attempt to be as accommodating as possible with field identifiers because the import source might well be a spreadsheet provided by an end user.

Sorry for being unclear here. What you described I'll solve with a user-configurable uploadFieldName in the .ds.xml - and there I don't care for case-sensitivity, but having this CI seems reasonable.

I was talking about the real ID lookup in the DB:

=== 2015-10-16 13:16:52,242 [ec-6] INFO  SQLDataSource - [builtinApplication.null] Performing fetch operation with
    criteria: {criteria:[[B]{value:"Springs offensive 2015",fieldName:"NAME",operator:"iEquals"}[/B],{value:1,fieldName:"TENANT_ID",operator:"equals"},{value:1,fieldName:"CREATED_BY_TENANT_ID",operator:"equals"},{value:1,fieldName:"MODIFIED_BY_TENANT_ID",operator:"equals"}],operator:"and",_constructor:"AdvancedCriteria"}    values: {criteria:[{value:"Springs offensive 2015",fieldName:"NAME",operator:"iEquals"},{value:1,fieldName:"TENANT_ID",operator:"equals"},{value:1,fieldName:"CREATED_BY_TENANT_ID",operator:"equals"},{value:1,fieldName:"MODIFIED_BY_TENANT_ID",operator:"equals"}],operator:"and",_constructor:"AdvancedCriteria"}
=== 2015-10-16 13:16:52,243 [ec-6] INFO  SQLDataSource - [builtinApplication.null] derived query: SELECT $defaultSelectClause FROM $defaultTableClause$defaultAnsiJoinClause WHERE $defaultWhereClause
=== 2015-10-16 13:16:52,244 [ec-6] INFO  SQLDataSource - [builtinApplication.null] 1787: Executing SQL query on 'Oracle': SELECT T_CAMPAIGN.ID, T_CAMPAIGN.TENANT_ID, T_CAMPAIGN.CREATED_BY, T_CAMPAIGN.CREATED_AT, T_CAMPAIGN.MODIFIED_BY, T_CAMPAIGN.MODIFIED_AT, ....
[B]WHERE ((LOWER(T_CAMPAIGN.NAME) = LOWER('springs offensive 2015')[/B] AND T_CAMPAIGN.NAME IS NOT NULL) ...

(Other criteria added by my SQLDataSource subclass)




Do you also see the issue from #18?


Best regards
Blama

[Isomorphic]

Yes we do. It's a bit of an edge case - we only send records that are known to pass validation, so we don't really expect an error response. The only reason we get one here is that the PK of the record is keyed in by the user, which is pretty unusual. Be that as it may, it was a bug; the fix will be present in nightly builds as of tomorrow, 17 October

[Blama]

Hi Isomorphic,

do you already have the solution you mentioned in #21?

Best regards
Blama

[Blama]

Hi Isomorphic,

the issue from #18 is fixed for me using today's nightly.

Thank you,
Blama

[Blama]

Hi Isomorphic,

did you already implement the solution you mentioned in #21?

Best regards
Blama

[Blama]

Hi Isomorphic,

do you have an ETA for the solution you mentioned in #21? On 16th Oct it sounded like it was almost finished.

Best regards
Blama

[Isomorphic]

Sorry, this problem got put to one side pending feedback, and then forgotten about. We will pick this up tomorrow (the developer assigned to it is in a European timezone)

[Isomorphic]

A fix for this was committed today, you should see the issue go away with tomorrow's nightly (13 November)

[Blama]

The bug from #18 (uploadFieldName) is now fixed for me for some time now.

Thank you
Blama