Criteria matching

[hawkettc]

Hi,

I've got some issues with listgrid criteria failing to match incoming records. I suspect SC is behaving as intended, but just wondering what the best approach to the problem is.

Data Record:

<response>
  <status>0</status>
  <data>
    <rec>
      <multiVal>1</multiVal>
      <multiVal>2</multiVal>
    </rec>
  </data>
</response>

Criteria

{multiVal:1}

A. On a GET, if the server returns the above record, then the ListGrid renders the record.

B. On a PUT or a POST, if the server returns the above record, the the ListGrid does not render the record.

I assume in situation A that SC doesn't attempt to apply the filter because it expects the server to return a correct set of records.

In situation B, I assume SC does apply the filter, because the update may return a record that no longer matches the filter.

So I guess I've got two questions -

1. Is it expected behaviour? i.e. do I need to match *all* values for a multivalued property?
2. Given that the logic I actually want is - 'Match all records that contain this value in this field' - how do I go about structuring that?

Thanks,

Colin

[hawkettc]

It might be worth adding that this would be a pretty generic use case for tagging - when you click a tag, it retrieves a set of records that may each have different tags applied - the only common thing about all the records is that they each have the tag that the user clicked.

In this situation, which is definitely not an edge case, we want to set up a criteria that does not match *all* values - it should only match one or a some. The following data illustrates this scenario more specifically -

Data Record:

<response>
  <status>0</status>
  <data>
    <rec>
      <tags>Tag A</tags>
      <tags>Tag B</tags>
      <tags>Tag C</tags>
    </rec>
  </data>
</response>

Some possible Criteria that would match this record:

{tags:"Tag A"}

{tags:["Tag A", "Tag B"]}

I'm feeling like this situation would be so common that I must have missed something about the way criteria work. Any suggestions for this use case would be welcome. Thanks,

Colin

[Isomorphic]

This isn't a built-in search operator, but the AdvancedCriteria system allows you define your own operators - see the docs.

[hawkettc]

Ok, useful to know this use-case is not available by default. Using the docs to resolve this one proved somewhat tricky, but not impossible. Here's what I did if anyone else needs to know how to create their own custom operator for an advanced criteria -

// The new operator
// Ensures that all supplied values for the field exist in the record
// supplied values may be a single value, or an array of values
// record field may be a single value or an array of values (mutli-valued)
var containsValuesOperator = {
	ID: "containsValues",
	
        // I couldn't get this to fire, but I think this logic is right - apologies for any errors or typos.
	compareCriteria: function(newCriterion, oldCriterion) {
		var newValue = newCriterion.value;
		var newValues;
		if (newValue.constructor == Array || isc.isAn.Array(newValue) || (newValue.Class && newValue.Class == "Array")) newValues = newValue;
		else newValues = [newValue];
		
		var oldValue = oldCriterion.value;
		var oldValues;
		if (oldValue.constructor == Array || isc.isAn.Array(oldValue) || (oldValue.Class && oldValue.Class == "Array")) oldValues = oldValue;
		else oldValues = [oldValue];
		
                // This loop can be made more efficient by removing items from oldValues when they are matched.
		var matchCount = 0;
		for(var i = 0; i < newValues.length; i++) {
			for(var j = 0; j < oldValues.length; j++) {
				if(newValues[i] == oldValues[j]) {
                                      matchCount++;
                                      break;
                                }
			}
		}
		
                // The criterion are identical (careful here if you improve efficiency above)
		if(oldValues.length == newValues.length && matchCount == oldValues.length) return 0;

                // All the new values were in the old criterion
		if(matchCount == newValues.length) return 1;

                // There are values in the new criterion that were not in the old criterion
		return -1;
	},
	
	// Every value must be in record[fieldName]
	condition: function(value, record, fieldName, criterion, operator) {
		var values;
		if (value.constructor == Array || isc.isAn.Array(value) || (value.Class && value.Class == "Array")) values = value;
		else values = [value];
		
		var recValue = record[fieldName];
		var recValues;
		if (recValue.constructor == Array || isc.isAn.Array(recValue) || (recValue.Class && recValue.Class == "Array")) recValues = recValue;
		else recValues = [recValue];
		
		for(var i = 0; i < values.length; i++) {
			var valMatch = false;
			for(var j = 0; j < recValues.length; j++) {
				if(values[i] == recValues[j]) {
					valMatch = true;
					break;
				}
			}
			if(!valMatch) return false;  // One of the values was not in the record
		}

		return true;
	},
	
        // My use-case was just text strings, but should work for others
	fieldTypes: ["text"],
	
        // We are providing a collection
	valueType: "valueSet"
};

// Make this operator available to all datasources
isc.DataSource.addSearchOperator(containsValuesOperator);

// Constructs an AdvancedCriteria from a list of tags and a standard criteria
function getHasTagsAdvCriteria(tags, standardCriteria) {
	var andCriteria = [{ fieldName:"tags", operator:"containsValues", value:tags }];
	
	for(fieldName in standardCriteria) andCriteria.push({ fieldName: fieldName, operator: "equals", value: standardCriteria[fieldName]});
	
	return {
	  _constructor:"AdvancedCriteria",
	  operator:"and",
	  criteria: andCriteria
        };
}

and you would use it like this -

myListGrid.filterData(getHasTagsAdvCriteria(["Tag A", "Tag B"], {someName: someValue});

And any record that has both 'Tag A' and 'Tag B' among its tags will be matched (assuming the other criteria match as well).

Also, if you need the parameters to be received on the server like normal, you need to modify the transformRequest method of your datasource - something like this -

transformRequest: function(dsRequest) {
    	var dsResponse = this.Super("transformRequest", arguments);
        var params = {};
      
        if(dsRequest.data.criteria) {
    		for(var i = 0; i <  dsRequest.data.criteria.length; i++) {
    			var criteria = dsRequest.data.criteria[i];
    			// We know criteria have 'fieldName', 'operator', 'value'
    			params[criteria.fieldName] = criteria.value;
    		}
    	}

        params = isc.addProperties(params, dsRequest.data, dsRequest.params);

        return params;
}

Would appreciate feedback from Isomorphic to confirm this looks about right, any issues, recommendations etc. Thanks,

Colin

[Isomorphic]

Looks good, thanks for posting that sample.

[hawkettc]

I'm experiencing some issues with the 'fieldTypes' attribute of the Operator.

My datasources are built from xsd files, and as such the types of the fields in the generated datasources match the declared complex types in the xsd.

I am getting the following warning

22:16:14.884:XRP9:WARN:RestfulDataSource:_k_tm_as_S1_DS:Operator containsValues is not valid for field contextKeys. Continuing anyway.

getting a print out of the type of that field -

22:16:13.974:MUP5:INFO:Log:contextKeys field type:key-type

I changed the operator listed above to have the following -

...
fieldTypes: ["text", "key-type"],
...

But the warning is still being shown. Is there anything else required to make the custom operator acceptable for the type generated by xsd? It doesn't appear to be affecting behaviour, but I am concerned my implementation is not correct, and may have problems down the track. Thanks,

Colin

[Isomorphic]

Is "key-type" a custom XSD simpleType? If so, are you loading that type definition? Otherwise SmartClient doesn't know about it.

simpleType declarations in XSD normally generate corresponding isc.SimpleType definitions in SmartClient.

[hawkettc]

'key-type' is an xsd simpleType -

<xs:simpleType name="key-type">
  <xs:restriction base="xs:string">
   <xs:pattern value="(None)|(\\S{12,})"/>
  </xs:restriction>
</xs:simpleType>

In terms of 'loading' the type definition - the datasource is built from an xsd containing that simpleType decaration (using isc.XMLTools.loadXMLSchema) - I don't do anything else to explicitly load the type though - is something else required? Thanks,

Colin

[hawkettc]

Due to this issue, and to the above, I needed to find an alternative to AdvancedCriteria, and it turns out there was a much simpler option for my use case.

override the fieldMatchesFilter() method of your datasource.

fieldMatchesFilter : function(fieldValue, filterValue, requestProperties) {
    	// When matching an array in the filter, as long as all the values in fieldValue are present in the array, they match
    	if (filterValue.constructor == Array || isc.isAn.Array(filterValue) || (filterValue.Class && filterValue.Class == "Array")) {
    		isc.Log.logInfo("Doing array logic...");
    		var fieldValues;
    		if (fieldValue.constructor == Array || isc.isAn.Array(fieldValue) || (fieldValue.Class && fieldValue.Class == "Array")) fieldValues = fieldValue;
    		else fieldValues = [fieldValue];
    		
    		for(var i = 0; i < filterValue.length; i++) {
    			var valMatch = false;
    			for(var j = 0; j < fieldValues.length; j++) {
    				
    				if(filterValue[i] == fieldValues[j]) {
    					valMatch = true;
    					break;
    				}
    			}
    			if(!valMatch) return false;  // One of the filter values was not in the record
    		}

    		return true;
    	}
    	
        return this.Super("fieldMatchesFilter", arguments);
}

The above achieves the same result as adding an operator and using AdvancedCriteria above, and you call as you would normally with simple criteria, as long as you always supply an array, even for a single value.

myListGrid.filterData({tags:["Tag A", "Tag B"], someName: someValue});
...
myListGrid.filterData({tags:["Tag A"], someName: someValue});

This approach only works when what you want to change is the matching behaviour, and you still want the basic 'AND' logic for each criteria.