Announcement

**mandrachek** · 14 Feb 2016, 05:18

I have also tried with the 02/14 build, sad to say the results are the same.

**Isomorphic** · 15 Feb 2016, 07:30

There's nothing obviously wrong with the field description here. We're attempting to reproduce this issue on our end and we'll let you know what we find.

Regards
Isomorphic Software

**Isomorphic** · 15 Feb 2016, 09:59

We've made some attempts to reproduce this issue against a working DataSource, with no luck.
We've also done some sanity checking for recent changes in the server-side validation flow which could cause this problem, and we're not seeing anything obvious.

For us to pursue this further we'll need some more information. Could you show us
- The dsRequest that leads to this failure. This can be captured from the "RPC" tab of the developer console. Enable "track RPCs" and execute the save operation from the app. You should see a single operation in the RPC tag which reflects the failed save attempt. If you can post that we can be sure we understand exactly what flow is executing (obviously you may want to sanitize any sensitive information in the request before sharing)
- The complete dataSource definition, or failing that, at least clarify the serverType, and any custom operation bindings, etc which might apply to this.
- Any custom server code which is actually being executed on the save attempt. We're thinking of custom logic executed via operationBindings [DataSource DMI], or overrides to the DataSource execute... methods. You can also determine whether custom server logic is responsible yourself by temporarily disabling it and attempting to reproduce the problem (you already mentioned that getting rid of the serverCustom type validator doesn't resolve the issue, implying that the problem is unrelated to that particular piece of custom server logic).

Ultimately this may prove to be a configuration issue rather than a bug in either application or framework code. Another worthwhile step would be to create a duplicate of your dataSource - something that hits the same database table (I'm assuming you're working with SQL here), but with no custom logic in place and see if you hit the same issue. If you do, we'd recommend carefully checking over your database configuration using the admin console and verifying the upgrade to the more recent SmartClient build didn't inadvertently clobber some necessary configuration you previously had in place.

Regards
Isomorphis Software

**mandrachek** · 17 Feb 2016, 09:40

We are using SQL Server. Specifically we're using SQL 2012, with the microsoft jdbc driver. We have setup a JNDI datasource in server level context.xml, using the apache db pool, as the pool will be shared amongst several applications. We're using maven, and have a server.properties in our src/main/resources, so if anything, our settings should clobber anything provided by the framework. Relevant portions:

Code:

 sql.defaultDatabase: SQLServer
  sql.SQLServer.driver.name: jdbc/ourdb
  sql.SQLServer.interface.type: jndi
  sql.SQLServer.database.type: sqlserver
  sql.SQLServer.pool.enabled: false
  sql.SQLServer.autoJoinTransactions: true
  sql.defaultPaging: sqlLimit

sql.pool.enabled:false

Here's the DSRequest (sanitized, with some fields removed:

Code:

{
    dataSource:"UserManagement", 
    operationType:"validate", 
    data:{
        USER_ID:5112, 
        LAST_NAME:"XXXX", 
        EMAIL:"xxxx@host.com", 
        FIRST_NAME:"XXXX", 
        rowID:1
    }, 
    textMatchStyle:"exact", 
    willHandleError:true, 
    showPrompt:true, 
    prompt:"Validating...", 
    oldValues:{
        USER_ID:5112, 
        LAST_NAME:"XXXX", 
        EMAIL:"xxxx@host.com", 
        FIRST_NAME:"XXXX", 
        rowID:1
    }, 
    requestId:"UserManagement_request37", 
    internalClientContext:{
        component:{
            ns:Obj, 
            creator:[UserDetailsLayout ID:isc_UserDetailsLayout_0], 
            ID:"isc_UserDetailsLayout_0_userDetailsForm", 
            name:"UserDetailsForm", 
            width:688, 
            height:435, 
            titleAlign:"right", 
            requiredTitlePrefix:"* ", 
            titleSuffix:":", 
            autoFetchData:false, 
            useAllDataSourceFields:false, 
            dataSource:[DataSource ID:UserManagement], 
            validateOnExit:true, 
            longTextEditorThreshold:400, 
            parentUserDetails:[UserDetailsLayout ID:isc_UserDetailsLayout_0], 
            parentUserManagement:Obj, 
            position:"absolute", 
            className:"normal", 
            values:Obj, 
            items:Array[23], 
            errors:Obj, 
            originalFields:Array[49], 
            parentElement:[VLayout ID:isc_UserDetailsLayout_0_userDetailsEditor], 
            topElement:[Window ID:WindowUserDetailsEditor], 
            tabIndex:2909, 
            children:Array[3], 
            ruleScope:"WindowUserDetailsEditor", 
            cacheOffsetCoords:true, 
            zIndex:203330
        }, 
        fieldName:"EMAIL", 
        rowNum:{
        }
    }, 
    fallbackToEval:false, 
    validationMode:"full", 
    lastClientEventThreadCode:"MUP7", 
    bypassCache:true, 
    dataProtocol:"getParams"
}

The raw response is:

Code:

{
    affectedRows:0, 
    invalidateCache:false, 
    isDSResponse:true, 
    queueStatus:-1, 
    status:-1, 
    data:null
}

I can't share the full datasource (some of the fields are sensitive), but here's what's left that matches the dsRequest:

Code:

<DataSource serverType="sql" dbName="SQLServer" schema="dbo" tableName="USERS_TABLE" ID="UserManagement">      <field name="USER_ID" type="sequence" primaryKey="true" showIf="false" canEdit="false"/>
      <field name="LAST_NAME" title="Last Name" type="text" length="60" required="true" width="128"/>
      <field name="FIRST_NAME" title="First Name" type="text" length="60" required="true" width="128"/>
      <field name="MIDDLE_INITIAL" title="MI" type="text" length="1" width="64"/>
 
    <field name="EMAIL" title="Email" type="text" length="320" required="false" width="192">
        <validators>
          <validator type="serverCustom">
            <serverObject lookupStyle="new" className="com.myapp.validators.EmailValidator" />
            <errorMessage>Please enter a valid email address.</errorMessage>
          </validator>
          <validator type="isUnique" errorMessage="This email address is already in our system.  Please use a different email address"/>
        </validators>
      </field>
   <operationBindings>
        <!-- the only thing we're modifying here, is the requiresRole, and the tableClause for some fields that are sensitive/omitted -->
        <operationBinding operationType="fetch" requiresRole="Admin"/>    
       <!-- we have a few other operationBindings, mostly add/update/delete, with different roles.  a few fetches with different operationId's and roles. -->      
   </operationBindings>
 </DataSource>

This is a validation request from within a DynamicForm.

The only custom code at all is our EmailValidator. I can't share the complete code for that, but it's pretty simple - just two regexes it switches two when deployed in development mode vs production mode:

Code:

 
public class EmailValidator {      private static final Logger log = new Logger(EmailValidator.class.getName());
      private static final Pattern PATTERN_NORMAL = Pattern.compile(
              "(?:(^([A-Z0-9\\._%+-]+@[A-Z0-9\\.-]+\\.[A-Z]{2,4})))*(?:((.*\\.internaldomain|.*\\.customerhost.com)))",
              Pattern.CASE_INSENSITIVE);
      private static final Pattern PATTERN_DEVELOPMENT = Pattern.compile(
              "(?:(^([A-Z0-9\\._%+-]+@[A-Z0-9\\.-]+\\.[A-Z]{2,4})))*(?:((.*\\.internaldomain|.*\\.customerhost.com|.*\\@host.com)))",
              Pattern.CASE_INSENSITIVE);
  @SuppressWarnings("rawtypes")
      public boolean condition(Object value, Validator validator, String fieldName, Map record, DataSource ds) throws Exception {
          log.warn("Validating email!");
          Config config = Config.getInstance();
          Matcher matcher = PATTERN_NORMAL.matcher((String) value);
   
          // check to see if we're in development, if so, use the
          // developmentPattern.
          try {
              if (config.get("development") != null) {
                  int development = Integer.parseInt((String) config.get("development"));
                  if (development == 1) {
                      matcher = PATTERN_DEVELOPMENT.matcher((String) value);
                  }
              }
          }
          catch (Exception e) {
              log.error("Error getting development flag.", e);
          }
   
          return matcher.matches();
      }
  }

**mandrachek** · 17 Feb 2016, 14:18

Aha! The key to reproducing appears to be having fields defined with SimpleTypes.
Omitted from the DSRequest above, we have several simple types used for commonly included fields. For example:

Code:

<field name="USER_GROUP_ID" type="USER_GROUP_ID_TYPE"/>
<field name="USER_GROUP" type="DISPLAY_FIELD" customSelectExpression="USER_GROUP.GROUP_TITLE" canSave="false"/>

<!-- in the fetch -->
<operationBinding type="fetch">
   <tableClause>
       USERS_TABLE
       LEFT OUTER JOIN USER_GROUP ON USER_GROUP.USER_GROUP_ID = USERS_TABLE.USER_GROUP_ID
   </tableClause>
</operationBinding>

SimpleTypes defined like this:

Code:

 <?xml version="1.0" encoding="UTF-8"?>
  <SimpleType  name="BASE_TYPE" inheritsFrom="sequence" editorType="UserGroupIdItem" filterEditorType="UseGroupIdFilter">
  <validOperators>equals</validOperators>
  <validOperators>notEqual</validOperators>
  </SimpleType>
  
<?xml version="1.0" encoding="UTF-8"?>
  <SimpleType name="USER_GROUP_ID_TYPE" inheritsFrom="BASE_TYPE" editorType="UserGroupIdItem" filterEditorType="UseGroupIdFilter">
  <fieldProperties>
          <foreignKey>USER_GROUP.USER_GROUP_ID</foreignKey>
          <foreignDisplayField>GROUP_TITLE</foreignDisplayField>
          <displayField>USER_GROUP</displayField>
          <title>GRP</title>
          <prompt>User Group</prompt>
  </fieldProperties>
  </SimpleType>

editorType and filterEditorType are extended ComboBox's.

It seems if I eliminate all the fields that use these custom types, the error when attempting to validate goes away. And moving the attributes directly from the SimpleType to the field works just fine. We currently have about ~150 SimpleType definitions. Most of them are used 2-5 times, a few of them significantly more, across about ~175 datasources, and we've had to modify all of them to even be able to run the most recent builds.

This isn't the first time we've had problems with SimpleTypes, and I'm really wondering if we wouldn't be better off just getting rid of them all together - but it becomes harder to ensure that common fields are defined and configured in the same manner across datasources.

**Isomorphic** · 17 Feb 2016, 16:39

Thanks for the clarification and follow up. We're taking a look at why this is causing you problems and we'll follow up as soon as we have anything to share with you!

Regards
Isomorphic Software

**Isomorphic** · 18 Feb 2016, 05:18

This is fixed now, you may download next nightly build (Feb 19) and try it out. Please let us know how it worked for you.

Announcement

NPE when validating

Comment

Comment

Comment

Comment

Comment

Comment

Comment