Potential bug: hasRelatedRecord-validator run for missing field

[Blama]

Hi Isomorphic,

I think I found an issue with hasRelatedRecord-validator. This is also run when there is no value transmitted for a required="false" field. IMHO the validator should not run then.

If you agree that this should not happen I'll create a BuiltInDS-based testcase for it.

Best regards
Blama

[Isomorphic]

Yes, in the absence of a value we would not expect this validator to run, and it doesn't seem to in a quick test. So please do let us know if you can create a test case showing misbehavior.

[Blama]

Hi Isomorphic,

please see this modifed employees.ds.xml in an otherwise unchanged BuiltInDS (v11.1p_2018-02-16):

<DataSource
    ID="employees"
    serverType="sql"
    tableName="employeeTable"
    recordName="employee"
    testFileName="/examples/shared/ds/test_data/employees.data.xml"
    titleField="Name"
>
    <fields>
        <field name="userOrder"       title="userOrder"       type="integer"  canEdit="false"    hidden="true"/>
        <field name="Name"            title="Name"            type="text"     length="128"/>
        <field name="EmployeeId"      title="Employee ID"     type="integer"  primaryKey="true"  required="true"/>
[B]       <field name="ReportsTo"       title="Manager"         type="integer"  foreignKey="employees.EmployeeId">
                    <validators>
                        <validator type="hasRelatedRecord" />
                    </validators>
        </field>[/B]
        <field name="Job"             title="Title"           type="text"     length="128"/>
        <field name="Email"           title="Email"           type="text"     length="128"/>
        <field name="EmployeeType"    title="Employee Type"   type="text"     length="40"/>
        <field name="EmployeeStatus"  title="Status"          type="text"     length="40"/>
        <field name="Salary"          title="Salary"          type="float"/>
        <field name="OrgUnit"         title="Org Unit"        type="text"     length="128"/>
        <field name="Gender"          title="Gender"          type="text"     length="7">
            <valueMap>
                <value>male</value>
                <value>female</value>
            </valueMap>
        </field>
        <field name="MaritalStatus"   title="Marital Status"  type="text"     length="10">
            <valueMap>
                <value>married</value>
                <value>single</value>
            </valueMap>
        </field>
    </fields>
</DataSource>

ARC (Advanced REST Client Addon in Chrome) request is working as expected (leaving out the log as it is working):

<request>
    <dataSource>employees</dataSource>
    <operationType>add</operationType>
    <data>
        <EmployeeId>1000005</EmployeeId>
    </data>
</request>

ADD request from unchanged BuiltInDS.java is not working, as it sends an explicit null as value.

1. Open Sample
2. Select "employees"
3. Click "New"
4. Enter unused employeeId, e.g. 1000006
5. Click "Save"

This results in a validation error and this server log:

=== 2018-02-17 19:35:10,955 [0-49] INFO  RequestContext - URL: '/builtinds/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0': Moz (Gecko) with Accept-Encoding header
=== 2018-02-17 19:35:10,957 [0-49] DEBUG XML - Parsed XML from (in memory stream): 1ms
=== 2018-02-17 19:35:10,959 [0-49] DEBUG RPCManager - Processing 1 requests.
=== 2018-02-17 19:35:10,959 [0-49] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    values:{
        Name:null,
        EmployeeId:1000006,
[B]        ReportsTo:null,[/B]
        Job:null,
        Email:null,
        EmployeeType:null,
        EmployeeStatus:null,
        Salary:null,
        OrgUnit:null,
        Gender:null,
        MaritalStatus:null
    },
    operationConfig:{
        dataSource:"employees",
        repo:null,
        operationType:"add",
        textMatchStyle:"exact"
    },
    componentId:"isc_DynamicForm_0",
    appID:"builtinApplication",
    operation:"employees_add",
    oldValues:{
    },
    criteria:{
    }
}
=== 2018-02-17 19:35:10,959 [0-49] INFO  IDACall - Performing 1 operation(s)
=== 2018-02-17 19:35:10,959 [0-49] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2018-02-17 19:35:10,959 [0-49] DEBUG DeclarativeSecurity - DataSource employees is not in the pre-checked list, processing...
=== 2018-02-17 19:35:10,960 [0-49] DEBUG AppBase - [builtinApplication.employees_add] No userTypes defined, allowing anyone access to all operations for this application
=== 2018-02-17 19:35:10,960 [0-49] DEBUG AppBase - [builtinApplication.employees_add] No public zero-argument method named '_employees_add' found, performing generic datasource operation
=== 2018-02-17 19:35:10,962 [0-49] DEBUG DeclarativeSecurity - [builtinApplication.employees_add] Processing security checks for DataSource null, field null
=== 2018-02-17 19:35:10,962 [0-49] DEBUG DeclarativeSecurity - [builtinApplication.employees_add] Request is not a client request, ignoring security checks.
=== 2018-02-17 19:35:10,962 [0-49] DEBUG AppBase - [builtinApplication.employees_add, builtinApplication.null] No userTypes defined, allowing anyone access to all operations for this application
=== 2018-02-17 19:35:10,962 [0-49] DEBUG AppBase - [builtinApplication.employees_add, builtinApplication.null] No public zero-argument method named '_null' found, performing generic datasource operation
=== 2018-02-17 19:35:10,962 [0-49] INFO  SQLDataSource - [builtinApplication.employees_add, builtinApplication.null] Performing fetch operation with
    criteria: {EmployeeId:null}    values: {EmployeeId:null}
=== 2018-02-17 19:35:10,963 [0-49] DEBUG SQLDataSource - [builtinApplication.employees_add, builtinApplication.null] DataSource 588 acquired SQLDriver instance 1683716488 during initialization
=== 2018-02-17 19:35:10,963 [0-49] INFO  SQLDataSource - [builtinApplication.employees_add, builtinApplication.null] derived query: SELECT $defaultSelectClause FROM $defaultTableClause WHERE $defaultWhereClause
=== 2018-02-17 19:35:10,964 [0-49] INFO  SQLDataSource - [builtinApplication.employees_add, builtinApplication.null] 588: Executing SQL query on 'HSQLDB': SELECT employeeTable.userOrder, employeeTable.Name, employeeTable.EmployeeId, employeeTable.ReportsTo, employeeTable.Job, employeeTable.Email, employeeTable.EmployeeType, employeeTable.EmployeeStatus, employeeTable.Salary, employeeTable.OrgUnit, employeeTable.Gender, employeeTable.MaritalStatus FROM employeeTable WHERE (employeeTable.EmployeeId IS NULL)
=== 2018-02-17 19:35:10,964 [0-49] DEBUG SQLConnectionManager - [builtinApplication.employees_add, builtinApplication.null] Borrowed connection '1025429278'
[B]=== 2018-02-17 19:35:10,964 [0-49] INFO  SQLDriver - [builtinApplication.employees_add, builtinApplication.null] Executing SQL query on 'HSQLDB' using connection '1025429278': SELECT employeeTable.userOrder, employeeTable.Name, employeeTable.EmployeeId, employeeTable.ReportsTo, employeeTable.Job, employeeTable.Email, employeeTable.EmployeeType, employeeTable.EmployeeStatus, employeeTable.Salary, employeeTable.OrgUnit, employeeTable.Gender, employeeTable.MaritalStatus FROM employeeTable WHERE (employeeTable.EmployeeId IS NULL)[/B]
=== 2018-02-17 19:35:10,965 [0-49] INFO  DSResponse - [builtinApplication.employees_add] DSResponse: List with 0 items
=== 2018-02-17 19:35:10,966 [0-49] DEBUG ValidationContext - [builtinApplication.employees_add] Adding validation errors at path '/employees/ReportsTo/ReportsTo': {errorMessage=Related record does not exist}
[B]=== 2018-02-17 19:35:10,966 [0-49] INFO  Validation - [builtinApplication.employees_add] Validation error: [
    {
        ReportsTo:{
            errorMessage:"Related record does not exist"
        }
    }
][/B]
=== 2018-02-17 19:35:10,966 [0-49] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
=== 2018-02-17 19:35:10,966 [0-49] DEBUG RPCManager - non-DMI response, dropExtraFields: false
=== 2018-02-17 19:35:10,967 [0-49] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 578
=== 2018-02-17 19:35:10,967 [0-49] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 578
=== 2018-02-17 19:35:10,967 [0-49] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 578
=== 2018-02-17 19:35:10,967 [0-49] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 588
=== 2018-02-17 19:35:10,967 [0-49] DEBUG SQLDriver - Freeing SQLDriver dbConnection 1025429278 for SQLDriver instance 1683716488
=== 2018-02-17 19:35:10,967 [0-49] DEBUG SQLConnectionManager - About to close connection with hashcode "1025429278"
=== 2018-02-17 19:35:10,967 [0-49] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 588
=== 2018-02-17 19:35:10,967 [0-49] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 588
=== 2018-02-17 19:35:10,967 [0-49] INFO  Compression - /builtinds/sc/IDACall: 226 -> 192 bytes

Best regards
Blama

[Isomorphic]

We're looking into the behavior you mention.

[Isomorphic]

We've applied a fix to SGWT 5.1p/SC 10.1p and newer releases to avoid the nulls being sent for unmodified fields by DynamicForm.saveData() in situations like that found in the BuiltInDS sample. It will be in the nightly builds dated 2018-02-23 and beyond.