Serverside bug with DynamicDSGenerator and wrongly included validator for includeFrom field

[Blama]

Hi Isomorphic,

I have a strange serverside issue (latest 6.1p).
I already narrowed it down so that I think I know whats happening here.

Setting: I have a DataSource I use to upload data. It can be used either in default mode or with configured settings for uploadFieldName (not important here) AND includeFrom (important) via DataSourceGenerator
Reason for the includeFrom: I don't know if the user has country-data in the form "DE", "DEU", "Deutschland", "Germany". They can edit it and it generally works.
I recently changed the the fields-default includeFrom (if nothing is overwritten) from NAME_DE to ISO-2 in order to be more international.
This is where the problems started.

It seems that the length-validator loaded for an field is derived from the default-includeFrom without DataSourceGenerator. Because of this I had a possible length of 40 before, that worked in all cases, even when configured to use ISO-2. Now that ISO-2 is the default, the validator always requests a max length of 2, which fails if configured to use NAME_DE like "Deutschland".

It seems that this problem only occurs in standalone usage (my mailimport).

Does this information ("length validator imported without waiting for DataSourceGenerator-includeFrom changes in standalone mode") already help you to identify the issue?

Best regards
Blama

[Isomorphic]

We're not following your description of the problem. It's not clear what is being included that doesn't seem up to date or correct (data, part of the field definition..?) or how a DataSourceGenerator is involved: it sounds as if there may be two versions of the same-named DataSource, one static, one generated? And it's also not clear what was changed and when - was that charset change a data change or a field definition change? Was it done or without restarting the app? And do you actually have a length validator that has a length of 2 somewhere, just in a different version of the DataSource?

This description of the problem had a lot of ambiguities, so, rather than answering the questions above one by one, please try again to explain everything starting from nothing, and just make sure that the above ambiguities are fully addressed.

[Blama]

Hi Isomorphic,

I decided to create a testcase and while doing so I found the problem to be related to my code.

Problem was the following (for me to recap and for you, if interested): The mailimport also needs to do string-to-ID conversion, client side BatchUpload does (Feature sponsorship from 2015). It does so via DataImport.importDataSourceRecords(). Here I'm using the correct DataSource that goes though DynamicDSGenerator (name V_LEADUPLOAD__someid__changedatesomeid). This fills my two fields as expected:

        <field name="CUST_ADDRESS_COUNTRY_ID" importStrategy="display" uploadFieldName="ADDRESS_COUNTRY_ID" displayField="CUST_ADDRESS_COUNTRY_NAME"
            foreignKey="T_COUNTRY.ID">
        </field>
        <!-- This includeFrom will be replaced in RegisterDS for imports with a profile. Be very careful when editing. Suffix COUNTRY_NAME is important -->
        <field name="CUST_ADDRESS_COUNTRY_NAME" includeFrom="[B]T_COUNTRY.ISO_3166_1_ALPHA_2[/B]" />

The someid tells the DynamicDSGenerator how to replace the includeFrom-value.
DataImport fills field CUST_ADDRESS_COUNTRY_ID with the correct T_COUNTRY.ID and field CUST_ADDRESS_COUNTRY_NAME with the name it did the lookup for (e.g. "Germany").

Now when I do the ADD-operation with the correct data, I only used V_LEADUPLOAD. This has the problem that the includeFrom="T_COUNTRY.ISO_3166_1_ALPHA_2" is NOT replaced. But my name, it did the lookup for before (e.g. "Germany") is as data in the CUST_ADDRESS_COUNTRY_NAME field. And the field get an imported implicit length validator from T_COUNTRY.ISO_3166_1_ALPHA_2, making the ADD fail, because "Germany" is longer than 2 chars.

Another minor difference I noticed:
If the default includeFrom (here T_COUNTRY.ISO_3166_1_ALPHA_2) had canSave="false", no validator would be generated for the field with includeFrom and the ADD worked. I don't know if this is on purpose.
Gnerally I'd say that ALL includeFrom fields should be canSave="false" themselves (which I believe is already the case) and therefore do NOT need to import any validator from the field they link to.

Best regards
Blama