DataSource.addSearchOperator and DataSourceField.setValidOperators interoperability

[stonebranch2]

Hi Isomorphic,

If you add a custom search operator via DataSource.addSearchOperator(Operator, FieldType[]), how would you specify it as one of multiple valid operators for say DataSourceField.setValidOperators(OperatorId...)?

Thanks

[Isomorphic]

In SmartClient, this is just natural (enum values are really just Strings) but in Java, it looks like we will need some additional structure.

We'll add it, but you can do it today via either JSNI or via .ds.xml declarations - just provide the String ID of your custom operator.

[stonebranch2]

Thank you.

Building on that, something like this appears to work, where I created a custom UcOperatorId enum, which extends ValueEnum and use the setAttribute API directly.

dsField.setAttribute("validOperators", new UcOperatorId[] { UcOperatorId.ICONTAINS, UcOperatorId.ISTARTS_WITH, UcOperatorId.IENDS_WITH,
                UcOperatorId.IEQUALS, UcOperatorId.INOT_CONTAINS, UcOperatorId.INOT_STARTS_WITH, UcOperatorId.INOT_ENDS_WITH, UcOperatorId.INOT_EQUAL,
                UcOperatorId.IS_NULL, UcOperatorId.NOT_NULL, UcOperatorId.IN_SET, UcOperatorId.NOT_IN_SET, UcOperatorId.LESS_THAN,
                UcOperatorId.LESS_OR_EQUAL, UcOperatorId.GREATER_THAN, UcOperatorId.GREATER_OR_EQUAL, UcOperatorId.AND, UcOperatorId.OR,
                UcOperatorId.NOT, UcOperatorId.CUSTOM_OP});

However, the moment I set a FieldOperatorCustomizer in FilterBuilder, for example.

FieldOperatorCustomizer() {...}.getFieldOperators(String, OperatorId[], FilterBuilder)

The defaultOperators array appears to contain a null entry, presumably for the custom operator.

For an override point, such as FilterBuilder.getValueFieldProperties(FieldType, String, OperatorId, ValueItemType, String), would the OperatorId be null here too? I suspect there is no way to represent those custom operators as an OperatorId here?

Thanks

[Isomorphic]

That's a really good attempt. We'll look at whether we can make your code "just work", but the JSNI approach definitely works. We will be looking at various ways of creating an "expandable enum" in Java (which is what this situation is) and try to pick the one that has the least hassle to it, considering existing APIs.

[stonebranch2]

Hi Isomorphic,

I think I need a little guidance on the JSNI approach.

The DataSource is created on the fly, and is completely dynamic.

Could you give me some hint as to how I would use JSNI to apply those operators to a DataSourceTextField, being created below? Is there another forum post that demonstrates this?

DataSourceTextField dsField = new DataSourceTextField(name, label);

Also, even I were able to apply the valid operators through the JSNI approach, I cannot see how it could ever be passed into a FilterBuilder subclass implementation of the below method as an OperatorId.

@Override
public FormItem getValueFieldProperties(FieldType type, String fieldName, OperatorId operatorId, ValueItemType itemType, String fieldType) {
// operatorId can never represent my custom operation. I believe for this custom operator, operatorId would always be null no matter what solution I used?
}

I will step back and give a high level overview what we are attempting to do here.

a) We want to introduce a new custom operation for text type, only applicable to the specific client-only DataSource (built dynamically).

When the custom operation is selected in the filter builder, we need to display a CanvasItem with multiple fields.

Some basic experimenting with this showed that a CanvasItem value was converted into it's own AdvancedCriteria containing all the values of the fields within the CanvasItem, however, I am not 100% sure if the proof of concept was correctly set up.

Ideally, we'd like to define some custom toValue / fromValue behavior for this operator's CanvasItem, so that in the end, we end up with something like.

{
    "_constructor":"AdvancedCriteria",
    "operator":"and",
    "criteria":[
        {
            "fieldName":"name",
            "operator":"customOp",
            "value":"someValueDerivedFromCanvasItem"
        }
    ]
}

b) This filter isn't used for any client-side filtering, it is 100% processed server-side.

We decode this filter on the server using com.isomorphic.criteria.AdvancedCriteria.decodeClientCriteria(String).

We handle how the filter is applied for this specific custom operation and its value.


If you could provide some guidance that would allow me to proceed further with my proof of concept, I would greatly appreciate it.

Thanks

[stonebranch2]

One quick clarification on:

@Override
public FormItem getValueFieldProperties(FieldType type, String fieldName, OperatorId operatorId, ValueItemType itemType, String fieldType) {
// operatorId can never represent my custom operation. I believe for this custom operator, operatorId would always be null no matter what solution I used?
}

My point here is that I cannot really write any conditional logic here based on a custom operation id.

[stonebranch2]

For the toValue / fromValue aspect, I suspect I need to leverage CanvasItem.setCriterionGetter(FormItemCriterionGetter) and CanvasItem.setCriterionGetter(FormItemCriterionGetter).

[stonebranch2]

Hi Isomorphic,

Providing some additional progress I made, as I feel I am getting closer to where I need to be with my proof of concept.

Most of the places I feel I need to override, explicitly use the Java enum OperationId, so I decided to hijack a predefined operation that we don't use, specifically, the matchPattern operation.

We use override the label to make it our own.

 @Override
    @DefaultMessage("custom operation")
    @Key("operators_matchesPatternTitle")
    String operators_matchesPatternTitle();

Then, I created a CustomItem class (extends CanvasItem), and use setCriterionGetter and setCriterionSetter.

setCriterionGetter(new FormItemCriterionGetter() {

            @Override
            public Criterion getCriterion(DynamicForm form, FormItem item, TextMatchStyle textMatchStyle) {
                return new Criterion(CustomItem.this.getName(), OperatorId.MATCHES_PATTERN,
                    CustomItem.this.dynamicForm.getField("Option1").getValue() + ":" + CustomItem.this.dynamicForm.getField("Option2").getValue());
            }

            @Override
            public Criterion getCriterion(DynamicForm form, FormItem item) {
                return new Criterion(CustomItem.this.getName(), OperatorId.MATCHES_PATTERN,
                    CustomItem.this.dynamicForm.getField("Option1").getValue() + ":" + CustomItem.this.dynamicForm.getField("Option2").getValue());
            }
        });

setCriterionSetter(new FormItemCriterionSetter() {

            @Override
            public void setCriterion(DynamicForm form, FormItem item, Criterion criterion) {
                String value = criterion.getValueAsString();
                if (value != null) {
                    String[] valueParts = value.split(":");
                    if (valueParts.length == 2) {
                        CustomItem.this.dynamicForm.getField("Option1").setValue(valueParts[0]);
                        CustomItem.this.dynamicForm.getField("Option2").setValue(valueParts[1]);
                    }
                }
            }
        });

To get the FilterBuilder to use the CanvasItem, I override getValueFieldProperties.

 @Override
    public FormItem getValueFieldProperties(FieldType type, String fieldName, OperatorId operatorId, ValueItemType itemType, String fieldType) {
        if (OperatorId.MATCHES_PATTERN.equals(operatorId)) return new CustomItem(fieldName);
        return super.getValueFieldProperties(type, fieldName, operatorId, itemType, fieldType);
    }

When creating the filter, I see the desired result on the form.



When saving, I get the desired filter as JSON.

{
"_constructor":"AdvancedCriteria",
"operator":"and",
"criteria":[
{
"fieldName":"name",
"operator":"matchesPattern",
"value":"1:2"
}
]
}

When I reload the record, the custom operation displays as a single value.



I am at the point where I am trying to figure out how to get this to display my CustomItem here.

I thought the FormItemCriterionSetter would be called here, but doesn't look like it.

Thanks

[Isomorphic]

Clever approach, hijacking and existing OperatorId. We still owe you a response on how to officially extend the enum, but this works for now.

By "reload the record", do you mean using FilterBuilder.setCriteria()? If so, what's going on in getValueFieldProperties()? It looks like your CustomItem isn't being used at all.

[stonebranch2]

Yes, essentially coming back to read the criteria, by setting up a new FilterBuilder and calling setCriteria.

The getValueFieldProperties doesn't appear to be called in this case, only when adding new criteria or modifying existing criteria operators.

We expected it would here.

[stonebranch2]

Hi Isomorphic,

I am still trying to create a standalone sample, however, I am not yet able to reproduce the CustomItem properties not being applied when setting the filter criteria from existing criteria.

It would appear to display correctly in my sandbox, so I still have some investigation to go on my end for that.

However, while I am seeing the CustomItem properties being picked up in my standalone sandbox, I am still not seeing the Option 1/ Option 2 being populated by my FormItemCriterionSetter.

setCriterionSetter(new FormItemCriterionSetter() {

            @Override
            public void setCriterion(DynamicForm form, FormItem item, Criterion criterion) {
                String value = criterion.getValueAsString();
                if (value != null) {
                    String[] valueParts = value.split(":");
                    if (valueParts.length == 2) {
                        CustomItem.this.dynamicForm.getField("Option1").setValue(valueParts[0]);
                        CustomItem.this.dynamicForm.getField("Option2").setValue(valueParts[1]);
                    }
                }
            }
        });

I am just trying to confirm that, when setCriteria is called on my FilterBuilder, and the Option 1/ Option 2 are displayed for the OperatorId.MATCHES_PATTERN operator, it should call into setCriterion on my FormItemCriterionSetter, and parse the 1:2 and set the Option1 and Option2 values?

Thanks

[stonebranch2]

Hi Isomorphic,

I was able to determine what we are doing that results in the getValueFieldProperties not being called when calling setCriteria with existing criteria.

We have an override on setCriteria for desired behavior, which I extracted out into my sample code below.

FilterBuilder filterBuilder = new FilterBuilder() {

            @Override
            public FormItem getValueFieldProperties(FieldType type, String fieldName, OperatorId operatorId, ValueItemType itemType,
                String fieldType) {
                if (OperatorId.MATCHES_PATTERN.equals(operatorId)) return new CustomItem(fieldName);
                return super.getValueFieldProperties(type, fieldName, operatorId, itemType, fieldType);
            }

            @Override
            public FilterBuilder setCriteria(AdvancedCriteria criteria) {
                if (criteria != null && !DataSource.isFlatCriteria(criteria) && DataSource.canFlattenCriteria(criteria)) {
                    criteria = DataSource.flattenCriteria(criteria);
                }
                setModeForCriteria(criteria);
                return (criteria != null) ? super.setCriteria(criteria) : this;
            }

            public void setModeForCriteria(AdvancedCriteria criteria) {
                // if no criteria default to Simple (RADIO) mode
                if (criteria == null) {
                    if (getTopOperatorAppearance() != TopOperatorAppearance.RADIO) {
                        setTopOperatorAppearance(TopOperatorAppearance.RADIO);
                    }
                    return;
                }

                // use Simple (RADIO) mode if possible
                if (DataSource.canFlattenCriteria(criteria)) {
                    if (getTopOperatorAppearance() != TopOperatorAppearance.RADIO) {
                        setTopOperatorAppearance(TopOperatorAppearance.RADIO);
                    }
                } else {
                    if (getTopOperatorAppearance() != TopOperatorAppearance.BRACKET) {
                        setTopOperatorAppearance(TopOperatorAppearance.BRACKET);
                    }
                }
            }
};

So, to summarize, putting OperatorId extensibility aside, two key topics remain.

• the question posed in post #11
• the sample code above resulting in getValueFieldProperties not being called for setCriteria, and hence, displaying a simple TextItem field with 1:2 instead of the Option 1/ Option 2.

Thanks

[Isomorphic]

Thanks for the test-cases and code - we're taking a look and will update here shortly.

[stonebranch2]

Hi Isomorphic,

Are there any updates that you could share?

Thanks

[Isomorphic]

Hi stonebranch2 ,

Apologies for the delay in getting back to you - there are a couple of problems here

1) the operator you've hijacked is marked internally as hidden: true and FilterBuilder won't include hidden operators - you could flag it as available by calling field.setValidOperators() in your DataSource.

[Edit]
However, if you take a new build dated April 18 or later, you won't need to. We've enhanced FilterBuilder so that setCriteria() can be passed AdvancedCriteria that uses hidden operators, and they'll be made fully available as if they were not hidden. The capability is controlled by new attribute FilterBuilder.allowHiddenOperators, which is turned on by default.

[Edit]
2) your custom FormItem not receiving initial-values from setCriteria() - we're looking into this, and it may also need a FilterBuilder change, but if you still see this problem with the latest build, you should be able to work around it for now by having your CriterionSetter convert an incoming null value into item.defaultValue, which is where FilterBuilder has put the incoming criterion's value.

As we noted, manipulating operators is easier in SmartClient, and we're looking at ways to make it easier in SGWT - if you see more issues in meantime, it would save time if you can provide a full test we can run against.

thanks

Isomorphic Support