12.1p+ Sample for searching with new "regexp" and "iregexp" operators in ListGrid and other questions to RegExp filtering

[Blama]

Hi Isomorphic,

I saw in the 12.1p Changelogs support for the new "regexp" and "iregexp" operators.
Also I saw that when you replace the search operator for countryName with iregexp in this sample, the GUI shows "/re" as operator icon, which is great.
I was wondering if you could add a sample using this new function to the 12.1p+ showcases for SmartClient and SmartGWT.
It is not clear to me how I could show the RegExp operator in the rightclick menu, and no entry is selected in that menu, when I start the sample with "iregexp".

Also, I think I found an issue with filtering and case insensitivity here.
It does not matter if you use iregexp or regexp, in order to find countries that contain a "T" and end with an "e" - you will always have to use RegExp T.*e$
t.*e$ does not work, also not with iregexp. In fact, the results don't match for T.*e$ and t.*e$ when using iregexp, which is unexpected.

This could of course be the fault of the underlying DB engine, so I was wondering what SQL functions you are using per DBMS.
HSQLDB's REGEXP_MATCHES for example (used to power the showcase I think) seems to support Java flags like e.g. (?i).
PostgreSQL has ~ and ~* for case insensitive.

Best regards
Blama

[Isomorphic]

The reported problem was specific to HSQLDB, which has been fixed. You will see the issue fixed as of as tomorrow's builds.

Regarding the use of operators in the rightclick menu, you can achieve this by using the dataSourceField.validOperators attribute in the .ds.xml file, for example:

       <field .......... >
            <validOperators>
                <value>lessThan</value>
                <value>startsWith</value>
                <value>iregexp</value>
            </validOperators>
        </field>

or in a clientOnly DataSource:

isc.DataSource.create({
    ID: "countryDS",
    fields:[
        {name:"countryName", validOperators:["lessThan", "startsWith", "endsWith", "iregexp"] },
        ............

On the other hand, we don't plan a Showcase sample of this as it's usually not used by end users.

Regards
Isomorphic Software

[Blama]

Hi Isomorphic,

I can see the filtering issue is fixed using v12.1p_2023-01-05.

Regarding specifying the operators:
The DataSourceField in the sample has no validOperators set, which I assume results in all operators being allowed.
But the /re operators don't show up in the ListGridField filterField context menu.
So somewhere in ListGrid there must be a setting validOperators as well, which, when used causes the ListGrid to only show the union of DataSourceField.validOperators and ListGridField.validOperators? But I don't think this is doc'd, I can't find anything like it on ListGridField or ListGrid.
How could I make this available in the GUI?

On the other hand here I can see this working as you describe it with the following code, so it's even less clear to me how this happens.

countryDS:

isc.DataSource.create({
    ID: "countryDS",
    fields:[
        {name:"countryCode", title:"Code"},
        {name:"countryName", title:"Country", validOperators:["lessThan", "startsWith", "iStartsWith", "endsWith", "iEndsWith", "iregexp", "regexp"]},
        {name:"capital", title:"Capital"}
    ],
    clientOnly: true,
    testData: countryData
})

databound.js:

isc.ListGrid.create({
    ID: "countryList",
    width:500, height:224, alternateRecordStyles:true, showAllRecords:true,
    dataSource: countryDS,
    autoFetchData: true,
    showFilterEditor: true
})

Best regards
Blama

[Isomorphic]

hi Blama ,

Simply put, by default, grid operator-menus show operators applicable to the field-type, unless they are marked hidden: true (we do this to show the simpler/more popular ops) - both regex operators are applicable to "text" type fields, so they'll be returned by getTypeOperators() - but both are marked hidden: true, so they wouldn't be returned by getFieldOperatorMap(), for example, by default.

However, any time you apply a validOperators array, the grid ignores the operator.hidden flag and shows the operators you specified, assuming they are valid for the field-type.

We don't allow global changing of operator.hidden for internal reasons, so using validOperators is the only current way to add the regex ops. Unfortunately, validOperators arrays are currently always replacement lists, they don't compound, so you'd have to reiterate all the operators you want to make available, as well as the regex ones.

For now, the neatest way to do this is probably to create a custom SimpleType that inherits from "text", apply your operator list there, and then use that text-type pervasively in your dataSources. If you want to do this, please wait for tomorrow's build - there was a bug where simpleType.validOperators was not being treated in quite the same way as validOperators set in other places.

For the future, we plan to make an enhancement similar to this below that lets you add to or remove from the standard list of available ops without listing them all out - no ETA as yet:

    ["*", "iregexp", "regexp"] // all normal operators plus these
    ["*-", "iContains"] // all normal operators [I]except[/I] these

thanks

Isomorphic Support

[Blama]

Hi Isomorphic,

thanks. That makes sense to me. The current way is annoying if one wants to use it (I don't, it was more an interest question), so I do think that your suggested enhancement makes sense.

Would this enhancement be on DataSourceField.validOperators and/or would it be a new ListGridField.validOperators?

Searching for validOperators I also saw FormItem.validOperators, so perhaps ListGridField.validOperators would not be needed because one could do this with ListGridField.filterEditorProperties? But then the question is would be new API be on DataSourceField.validOperators and/or FormItem.validOperators?

Best regards
Blama

[Isomorphic]

We haven't finalized a spec yet, but the idea would be to add support for a special syntax to validOperators anywhere you can use them, client- and server-side.

Technically, a FormItem will grab validOperators from its own validOperators array, or from an array applied to an associated LGField or DSField (DataSourceField.validOperators effectively ends up applied to ListGridField.validOperators), or from SimpleType.validOperators - so, right now in SmartClient, you could assign validOperators directly to a LGF and it would work anyway, despite not being documented.

We'll consider these details, and whether additional APIs or docs are needed, in enhancing the feature.