More foreignKey and includeFrom problems in 3.1p

[esa3]

Hi Isomorphic,
I am currently running into a few problems when using foreignKey relations in our DMI datasources. I am running latest patch 3.1p of SmartGwt Pro.
The related datasource "ndoktype" has a composite primary key consisting of two text fields "doktype" and "art". A third field "ejfid" is to be included in another datasource "dokmal".
In this other "dokmal" datasource, the two linked foreignKey fields have names "ndoktype" and "jpart".

On returning my response (a list of maps) from "dokmal", with foreignKey fields "ndoktype" and "jpart" set, a fetchRelated... is triggered automatically towards the "ndoktype" datasource.
This request contains an AdvancedCriteria expressing the required result ( (pk1=val1 and pk2=val2) or (pk1=val3 and pk2=val4) or .... ).

My problems are these:
1. The criteria primary fieldnames are not "doktype" and "art", but rather the foreignKey field names "ndoktype" and "jpart".

2. Another annoyance is that the criteria for the foreignKey request holds duplicate criterias when more than one "dokmal" response has the same foreign keys "ndoktype" and "jpart".

I am supplying an extract of the server log for the relevant RPCRequest.

...
=== 2013-04-12 15:58:45,180 [l0-7] INFO  DSResponse - DSResponse: List with 1 items
=== 2013-04-12 15:58:45,180 [l0-7] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
=== 2013-04-12 15:58:45,183 [l0-7] DEBUG RPCManager - DMI response, dropExtraFields: true
=== 2013-04-12 15:58:45,442 [l0-7] DEBUG RPCManager - Processing 1 requests.
=== 2013-04-12 15:58:45,443 [l0-7] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
        filterContext:"DOKMAL_ALL"
    },
    operationConfig:{
        dataSource:"dokmal",
        operationType:"fetch"
    },
    componentId:"NEWJOURNPOST1_Tab1_LocatableDynamicForm_Dokmal",
    useStrictJSON:true,
    appID:"builtinApplication",
    operation:"dokmal_fetch",
    oldValues:{
        filterContext:"DOKMAL_ALL"
    }
}
=== 2013-04-12 15:58:45,444 [l0-7] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2013-04-12 15:58:45,456 [l0-7] DEBUG DeclarativeSecurity - DataSource dokmal is not in the pre-checked list, processing...
=== 2013-04-12 15:58:45,456 [l0-7] DEBUG DeclarativeSecurity - Processing security checks for DataSource ndoktype, field ejfid
=== 2013-04-12 15:58:45,456 [l0-7] DEBUG DeclarativeSecurity - DataSource ndoktype is not in the pre-checked list, processing...
=== 2013-04-12 15:58:45,457 [l0-7] INFO  ServerObject - DMI on Spring bean: hjelperegisterDMI
...
=== 2013-04-12 15:58:45,789 [l0-7] INFO  DSResponse - DSResponse: List with 29 items
...
=== 2013-04-12 15:58:46,624 [l0-7] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2013-04-12 15:58:46,624 [l0-7] DEBUG DeclarativeSecurity - DataSource ndoktype is not in the pre-checked list, processing...
=== 2013-04-12 15:58:46,624 [l0-7] INFO  ServerObject - DMI on Spring bean: hjelperegisterDMI
=== 2013-04-12 15:58:47,265 [l0-7] INFO  DSResponse - DSResponse: List with 5 items
=== 2013-04-12 15:58:47,265 [l0-7] WARN  DSRequest - In response from related DataSource 'ndoktype' we encountered a record where primary key field values could not be matched with foreign key field values (in other words, there is a missing record in the response)
=== 2013-04-12 15:58:47,265 [l0-7] WARN  DSRequest - In response from related DataSource 'ndoktype' we encountered a record where primary key field values could not be matched with foreign key field values (in other words, there is a missing record in the response)
=== 2013-04-12 15:58:47,266 [l0-7] WARN  DSRequest - In response from related DataSource 'ndoktype' we encountered a record where primary key field values could not be matched with foreign key field values (in other words, there is a missing record in the response)
=== 2013-04-12 15:58:47,266 [l0-7] WARN  DSRequest - In response from related DataSource 'ndoktype' we encountered a record where primary key field values could not be matched with foreign key field values (in other words, there is a missing record in the response)
=== 2013-04-12 15:58:47,267 [l0-7] WARN  DSRequest - In response from related DataSource 'ndoktype' we encountered a record where primary key field values could not be matched with foreign key field values (in other words, there is a missing record in the response)
=== 2013-04-12 15:58:47,268 [l0-7] WARN  DSRequest - In manual filter, we found filter values for which there is no corresponding field in the first record of the resultset.  This may be expected, but it may be an indicator that you are not fetching all required fields - manual (ie, non-SQL) sort and filter can only work if you actually fetch the criteria and sortBy fields from the database. Affected fields: filterContext
=== 2013-04-12 15:58:47,268 [l0-7] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
=== 2013-04-12 15:58:47,271 [l0-7] DEBUG RPCManager - DMI response, dropExtraFields: true
...

Here follows the datasource definitions:

<DataSource ID="dokmal" serverType="generic" dataSourceVersion="1" beanClassName="com.edb.esa.dto.dokument.DokmalDto">
    <fields>
        <field name="betegn" type="text"/>
        <field name="state" type="enum">
            <valueMap>
                    <value id="NEW">New</value>
                    <value id="CHANGED">Changed</value>
                    <value id="UNCHANGED">Unchanged</value>
                    <value id="DELETED">Deleted</value>
            </valueMap>
        </field>
        <field name="id" type="integer" hidden="true" primaryKey="true"/>
        <field name="malart" type="text"/>
        <field name="kode" type="text"/>
        <field name="ndoktype" foreignKey="ndoktype.doktype"/>
        <field name="maltype" type="text"/>
        <field name="jpart" foreignKey="ndoktype.art"/>
        <field name="kategori" type="text"/>
        <field name="admid" type="integer"/>
        <field name="ejfid" includeFrom="ndoktype.ejfid"/>
    </fields>
<!--     <serverObject lookupStyle="new" className="com.edb.esa.server.ds.HjelperegisterDMI" />     -->
    <serverObject lookupStyle="spring" bean="hjelperegisterDMI" />    
    <xmlFromConfig>true</xmlFromConfig>
    <generatedBy>SC_SNAPSHOT-2012-03-19_v8.2p/Pro Deployment 2012-03-19</generatedBy>
</DataSource>

<DataSource ID="ndoktype" serverType="generic" dataSourceVersion="1" beanClassName="com.edb.esa.dto.dokument.NdoktypeDto">
    <fields>
        <field name="doktype" type="text" primaryKey="true"/>
        <field name="betegn" type="text"/>
        <field name="fullBetegn" type="text"/>
        <field name="ekstprod" type="text"/>
        <field name="ekstmot" type="text"/>
        <field name="art" type="text" primaryKey="true"/>
        <field name="ejfid" type="integer"/>
    </fields>
<!--     <serverObject lookupStyle="new" className="com.edb.esa.server.ds.HjelperegisterDMI" />     -->
    <serverObject lookupStyle="spring" bean="hjelperegisterDMI" />    
    <xmlFromConfig>true</xmlFromConfig>
</DataSource>

And finally, upon inspecting the originating dsRequest in the debugger, all properties look to be correct:

dsRequest
	.includeFrom
		.includedField {typeExplicitlyDeclared=true, name=ejfid, type=integer}

		.relation.fromFields  [{name=ndoktype, type=text, foreignKey=ndoktype.doktype}, {name=jpart, type=text, foreignKey=ndoktype.art}]


		.relation.toFields [{typeExplicitlyDeclared=true, primaryKey=true, name=doktype, type=text}, {typeExplicitlyDeclared=true, primaryKey=true, name=art, type=text}]

		.relation.thisField  {typeExplicitlyDeclared=true, includeFrom=ndoktype.ejfid, name=ejfid, type=integer, canEdit=false}

Here a criteria for a single dokmal

{criteria=[{value=I, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and, _constructor=AdvancedCriteria}

And here one for a list of dokmal records. As you can see, mostly duplicates

{criteria=[{criteria=[{value=N, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=, fieldName=ndoktype, operator=equals}, {value=, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=PLN, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=, fieldName=ndoktype, operator=equals}, {value=, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=, fieldName=ndoktype, operator=equals}, {value=, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=, fieldName=ndoktype, operator=equals}, {value=, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=, fieldName=ndoktype, operator=equals}, {value=, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=S, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=PLN, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=I, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=I, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=PLN, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}, {criteria=[{value=U, fieldName=ndoktype, operator=equals}, {value=DOK, fieldName=jpart, operator=equals}], operator=and}], operator=or, _constructor=AdvancedCriteria}

Regards Hans Petter Simonsen - EVRY Norway

[esa3]

Would Isomorphic care to comment? While these issues are not showstoppers, I would rather have that I did not have to code workarounds for these issues.

[sabaka]

Both issues are fixed, you may download next nightly build and try it out.