Filtering on the client using AdvancedCriteria is normally at least 5x slower than Cr

[knguyen]

SmartGWT 2.2
Firefox 3.6.12
Ubuntu Linux 10.04

One of many things I like about SmartGWT/SmartClient is that it can
handle large dataset. It works reasonably well when applying
criteria on a large data set. However, the performance degrades
significantly when using AdvancedCriteria.

I looked at DataSource.js, particularly at evaluateCriterion method,
and noticed that it has some significant overhead that could be
avoided or reduced:

 1. var op = this.getSearchOperator(criterion.operator);

    Finds the operator from isc.DataSource._operators array using
    linear search.  If the operators are stored using dictionary with
    operator ID as the key, it would be faster.

 2. if (criterion.fieldName) {
      var validOps = this.getFieldOperators(criterion.fieldName);
      if (!validOps.contains(op.ID)) {
        this.logWarn("Operator " + op.ID + " is not valid for field " + criterion.fieldName +
                     ". Continuing anyway.");
      }
    }

    Ensures the operator is valid on the fieldName and does this
    (numberOfCriteria * numberOfRecordsInDataSet) times for
    getFieldOperators() and contains().  Perhaps, it could be done
    once before applying the criteria?

I set up simple test cases to test performance of filtering a DataSource
of 30,000 records: using Criteria, using equivalent AdvancedCriteria, and using
equivalent AdvancedCriteria without overhead from (1) and (2) above.
Here is the result I get on Firefox:
Using Criteria took: 824 ms
Using AdvancedCriteria took: 4305 ms
Using AdvancedCriteria without overhead took: 356 ms.

The number are small because the records and criteria are simple, but
relatively significant. In real use-case, we got slow script warning
when using AdvancedCriteria. Currently we have to use native interface
to override the default implement for (1) and (2) above, but it would
be nice if SmartGWT/SmartClient addresses it.

I attached source code of the test.

-Kevin

[Isomorphic]

Thanks. We've extended these optimizations slightly (eg, to make sure that the "operator is valid for field" checks are run once) and rolled them into the product

[knguyen]

Glad to hear. Would it be part of next SmartGWT release? What about using dictionary to store operators?

Since these checks only signal warnings which are ignored in production, would it be possible to have a flag to disable these checks?

Thanks,
Kevin