Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

    Repeated validation

    Browser: Firefox 3.6.6
    SmartClient version: Isomorphic SmartClient Framework (SC_SNAPSHOT-2011-01-29/PowerEdition Deployment 2011-01-29)

    We're using the DMI validation capability to do field level validation. It seems the DMI code is being called an excessive number of times for each of the fields being validated.

    The datasource I'm referring to is defined in XML and has a custom server constructor that we created. The custom server overrides the execute() method.

    Four fields are marked for validation using the <validators> tag. Two fields are setup with DMI validators while two others are setup with other types of validators.

    Looking at the SmartClient console, I'm seeing that a validation RPC call is made for each of the fields in which validation is requested. That's fine. However, what I'm finding is that for each of these four calls, both DMI objects are being executed. The result is that each of the DMI validator objects is being called four times rather than one. This is a real problem for us since both objects make expensive calls to another server.

    The initial call is being done via the saveData() method of a ValuesManager.

    Here are the field definitions that include validators (from the datasource definition):

    Code:
    <field name="ISBN" title="ISBN" type="text" length="10" tableName="IPITHDR" canEdit="true" detail="true" menuItem="Description>ISBN">
           <validators>  
                  <validator type="serverCustom">  
                      <serverObject lookupStyle="new" className="com.islandpacific.gui.server.customDataSource.ISBNValidatorDMI"/>  
                      <errorMessage>$ErrorMessage</errorMessage>  
                  </validator>  
              </validators>  
    </field>
    Code:
    <field name="IGTIN" title="GTIN" type="text" length="14" tableName="IPITHDR" canEdit="true" detail="true" 
    		defaultValue="" menuItem="Description>GTIN">
              <validators>  
                  <validator type="serverCustom">  
                      <serverObject lookupStyle="new" className="com.islandpacific.gui.server.customDataSource.GTINValidatorDMI"/>  
                      <errorMessage>$ErrorMessage</errorMessage>  
                  </validator>  
              </validators>  
    </field>
    Code:
    <field name="IWHB" title="History from" hint="MMDD" mask="####" type="text" length="4" tableName="IPITHDR" canEdit="true" detail="true" menuItem="Features>Keep History from MMDD">
    	<validators>
    		<!-- enforces 4 digit MMDD requirement -->
    		<validator type="regexp"
    			expression="([0][1-9]|[1][012])([0][1-9]|[12][0-9]|[3][01])"
    			errorMessage="Invalid MMDD entry." /> 
    	</validators>		 
    </field>
    Code:
    <field name="IWHE" title="History to" hint="MMDD" mask="####" type="text" length="4" tableName="IPITHDR" canEdit="true" detail="true" menuItem="Features>Keep History to MMDD">
    	<validators>
    		<!-- enforces 4 digit MMDD requirement -->
    		<validator type="regexp"
    			expression="(0[1-9]|1[0-2])(0[1-9]|[1-2][0-9]|3[0-1])"
    			errorMessage="Invalid MMDD entry." /> 
    	</validators>		
    </field>
    Here's the header from the DataSource definition:

    Code:
    <DataSource ID="PoItem" dataFormat="iscServer" serverType="sql" webServiceOperations="all"
    	dbName="as400" sparseUpdates="true" tableName="IPPOITM" inheritsFrom="IPPOITM"
    	serverConstructor="com.islandpacific.gui.server.customDataSource.PoItemDS">

    Any thoughts?

    Thank you,
    Larry

    #2
    Can you provide the RPC request/responses of the validation calls for saveData()?

    Comment


      #3
      I'm listing 3 request/responses here.

      Note that while (under the ClientContext tag) each of these requests has a different field name, they all respond with the same error message, "Value does not follow GTIN standards".

      That error message should only come back for the last call ( for the field name IGTIN). This shows (and debug confirms) that the DMI code for this field is being called during all validation calls. What I find even more interesting is that the field IVST is not defined with a DMI validator. It's defined with another, more generic, validator.

      Request 1:
      Code:
      {
          "dataSource":"PoItem", 
          "operationType":"validate", 
          "data":{
              "ItemNumber":"0001-00281-0100-000-0001", 
              "IQTY":5, 
              "IDES":"AN ITEM", 
              "ISDS":"AN ITEM", 
              "IVST":"5654", 
              "IOVC":"COLOR", 
              "IPKTYPE":"", 
              "ITPINV":false, 
              "ISDI":"2011-02-09", 
              "IADI":"2011-02-09", 
              "IVNC":7.32, 
              "ILNC":6, 
              "INVC":6, 
              "ILVC":6, 
              "IRET":12, 
              "IORT":12, 
              "ISUG":12, 
              "IPLU":12, 
              "ITMP":0, 
              "IALT":0, 
              "ISPR":0, 
              "ISPQ":0, 
              "ITKT":"T", 
              "ITYN":false, 
              "ISKU":8383, 
              "ISBN":"", 
              "IGTIN":"122", 
              "IGTINTP":"1", 
              "IATT01":1, 
              "IATT02":999
          }, 
          "willHandleError":true, 
          "showPrompt":false, 
          "prompt":"Validating...", 
          "oldValues":{
              "ItemNumber":"0001-00281-0100-000-0001", 
              "IQTY":5, 
              "IDES":"AN ITEM", 
              "ISDS":"AN ITEM", 
              "IVST":"5654", 
              "IOVC":"COLOR", 
              "IPKTYPE":"", 
              "ITPINV":false, 
              "ISDI":"2011-02-09", 
              "IADI":"2011-02-09", 
              "IVNC":7.32, 
              "ILNC":6, 
              "INVC":6, 
              "ILVC":6, 
              "IRET":12, 
              "IORT":12, 
              "ISUG":12, 
              "IPLU":12, 
              "ITMP":0, 
              "IALT":0, 
              "ISPR":0, 
              "ISPQ":0, 
              "ITKT":"T", 
              "ITYN":false, 
              "ISKU":8383, 
              "ISBN":"", 
              "IGTIN":"122", 
              "IGTINTP":"1", 
              "IATT01":1, 
              "IATT02":999
          }, 
          "clientContext":{
              "component":[DynamicForm ID:isc_DynamicForm_57], 
              "fieldName":"IVST", 
              "rowNum":null, 
              "pendingFields":[
                  "IVST"
              ]
          }, 
          "requestId":"PoItem$627200"
      }
      Response 1:
      Code:
      [
          {
              queueStatus:-1, 
              errors:[
                  {
                      recordPath:"/PoItem", 
                      IGTIN:{
                          errorMessage:"Value does not follow GTIN standards"
                      }
                  }
              ], 
              isDSResponse:true, 
              invalidateCache:false, 
              status:-4, 
              data:null
          }
      ]
      Request 2:
      Code:
      {
          "dataSource":"PoItem", 
          "operationType":"validate", 
          "data":{
              "ItemNumber":"0001-00281-0100-000-0001", 
              "IQTY":5, 
              "IDES":"AN ITEM", 
              "ISDS":"AN ITEM", 
              "IVST":"5654", 
              "IOVC":"COLOR", 
              "IPKTYPE":"", 
              "ITPINV":false, 
              "ISDI":"2011-02-09", 
              "IADI":"2011-02-09", 
              "IVNC":7.32, 
              "ILNC":6, 
              "INVC":6, 
              "ILVC":6, 
              "IRET":12, 
              "IORT":12, 
              "ISUG":12, 
              "IPLU":12, 
              "ITMP":0, 
              "IALT":0, 
              "ISPR":0, 
              "ISPQ":0, 
              "ITKT":"T", 
              "ITYN":false, 
              "ISKU":8383, 
              "ISBN":"", 
              "IGTIN":"122", 
              "IGTINTP":"1", 
              "IATT01":1, 
              "IATT02":999
          }, 
          "willHandleError":true, 
          "showPrompt":false, 
          "prompt":"Validating...", 
          "oldValues":{
              "ItemNumber":"0001-00281-0100-000-0001", 
              "IQTY":5, 
              "IDES":"AN ITEM", 
              "ISDS":"AN ITEM", 
              "IVST":"5654", 
              "IOVC":"COLOR", 
              "IPKTYPE":"", 
              "ITPINV":false, 
              "ISDI":"2011-02-09", 
              "IADI":"2011-02-09", 
              "IVNC":7.32, 
              "ILNC":6, 
              "INVC":6, 
              "ILVC":6, 
              "IRET":12, 
              "IORT":12, 
              "ISUG":12, 
              "IPLU":12, 
              "ITMP":0, 
              "IALT":0, 
              "ISPR":0, 
              "ISPQ":0, 
              "ITKT":"T", 
              "ITYN":false, 
              "ISKU":8383, 
              "ISBN":"", 
              "IGTIN":"122", 
              "IGTINTP":"1", 
              "IATT01":1, 
              "IATT02":999
          }, 
          "clientContext":{
              "component":[DynamicForm ID:isc_DynamicForm_57], 
              "fieldName":"ISBN", 
              "rowNum":null, 
              "pendingFields":[
                  "ISBN"
              ]
          }, 
          "requestId":"PoItem$627201"
      }
      Response 2:
      Code:
      [
          {
              queueStatus:-1, 
              errors:[
                  {
                      recordPath:"/PoItem", 
                      IGTIN:{
                          errorMessage:"Value does not follow GTIN standards"
                      }
                  }
              ], 
              isDSResponse:true, 
              invalidateCache:false, 
              status:-4, 
              data:null
          }
      ]
      Request 3:
      Code:
      {
          "dataSource":"PoItem", 
          "operationType":"validate", 
          "data":{
              "ItemNumber":"0001-00281-0100-000-0001", 
              "IQTY":5, 
              "IDES":"AN ITEM", 
              "ISDS":"AN ITEM", 
              "IVST":"5654", 
              "IOVC":"COLOR", 
              "IPKTYPE":"", 
              "ITPINV":false, 
              "ISDI":"2011-02-09", 
              "IADI":"2011-02-09", 
              "IVNC":7.32, 
              "ILNC":6, 
              "INVC":6, 
              "ILVC":6, 
              "IRET":12, 
              "IORT":12, 
              "ISUG":12, 
              "IPLU":12, 
              "ITMP":0, 
              "IALT":0, 
              "ISPR":0, 
              "ISPQ":0, 
              "ITKT":"T", 
              "ITYN":false, 
              "ISKU":8383, 
              "ISBN":"", 
              "IGTIN":"122", 
              "IGTINTP":"1", 
              "IATT01":1, 
              "IATT02":999
          }, 
          "willHandleError":true, 
          "showPrompt":false, 
          "prompt":"Validating...", 
          "oldValues":{
              "ItemNumber":"0001-00281-0100-000-0001", 
              "IQTY":5, 
              "IDES":"AN ITEM", 
              "ISDS":"AN ITEM", 
              "IVST":"5654", 
              "IOVC":"COLOR", 
              "IPKTYPE":"", 
              "ITPINV":false, 
              "ISDI":"2011-02-09", 
              "IADI":"2011-02-09", 
              "IVNC":7.32, 
              "ILNC":6, 
              "INVC":6, 
              "ILVC":6, 
              "IRET":12, 
              "IORT":12, 
              "ISUG":12, 
              "IPLU":12, 
              "ITMP":0, 
              "IALT":0, 
              "ISPR":0, 
              "ISPQ":0, 
              "ITKT":"T", 
              "ITYN":false, 
              "ISKU":8383, 
              "ISBN":"", 
              "IGTIN":"122", 
              "IGTINTP":"1", 
              "IATT01":1, 
              "IATT02":999
          }, 
          "clientContext":{
              "component":[DynamicForm ID:isc_DynamicForm_57], 
              "fieldName":"IGTIN", 
              "rowNum":null, 
              "pendingFields":[
                  "IGTIN"
              ]
          }, 
          "requestId":"PoItem$627202"
      }
      Response 3:
      Code:
      [
          {
              queueStatus:-1, 
              errors:[
                  {
                      recordPath:"/PoItem", 
                      IGTIN:{
                          errorMessage:"Value does not follow GTIN standards"
                      }
                  }
              ], 
              isDSResponse:true, 
              invalidateCache:false, 
              status:-4, 
              data:null
          }
      ]
      Thanks.

      Comment


        #4
        Ah OK - the problem, a DMI validator is free to inspect values from other fields, hence, we don't currently have a way of knowing that it's safe to run each of your validator in isolation without validating other values first.

        Two solutions for this:

        1. we continue to run validators each time, but you cache the results of the expensive operations you perform. We will enable these DMI validators to be submitted in a queue to make this easier (eg you could cache on the servletRequest object, and for only the request lifetime).

        2. we add some properties to validators that would allow you to declare that they can run in isolation - but this may be tricky to define in terms of exactly which validators are safe in isolation vs require other fields, especially when the user is allowed to enter data in any order.

        Comment


          #5
          Sorry to barge in on this thread, but lgorlin and I are colleagues so I'm sure he won't mind.

          I'm not sure I understand why validators aren't always run in isolation. The validator gets a copy of the "record" being validated so it can refer to other field values if it needs them. But that doesn't mean that every other field must be *validated* before the each custom validator sees it.

          Can I suggest an alternative to option 2, and ask that only when Validator.setDependentFields() is used should the dependent fields (and only those) be validated. I'm not sure what other purpose the dependentFields API serves for custom validators. In our case there are no dependent fields so each validator would run in isolation.

          I can imagine some scenarios when you might want to write custom validation logic that depends on other referenced fields having already been validated. But I would think that is the exception rather than the rule.

          Comment


            #6
            To clarify, if other fields have not been validated, they could be of unexpected type or have some other unexpected value that causes a custom validator to misbehave. So at least for general type validators and other built-in validators, we want to keep the default that other validators have run first, so you don't have to worry about this.

            dependentFields could plausibly be used server-side as a means of explicitly preventing other values from being validated, right now, it doesn't do this.

            However we have added queuing, so in future builds, you should see that these 3 custom validators trigger only one network request, which gives you an opportunity to cache any expensive results for the lifetime of the servletRequest.

            Comment

            Working...
            X