TextMatchStyle substring not working anymore?

[Sytematic]

Using: SmartClient/SmartGWT Framework (v8.3p_2014-08-19/PowerEdition Deployment 2014-08-19)

I changed to the nightly above recently. Now there is a different behaviour that screws up my filter editors in certain SelectItems.
When I type something in the filter editor of a SelectItem that has multiple columns (ie. a dropdown listgrid), it sends the following request to the server,

Note the textMatchStyle "substring".

{
    dataSource:"Person", 
    operationType:"fetch", 
    componentId:"isc_PickListMenu_3", 
    data:{
        firstname:"E"
    }, 
    startRow:0, 
    endRow:75, 
    sortBy:[
        "firstname"
    ], 
    textMatchStyle:"substring", 
    resultSet:[ResultSet ID:isc_ResultSet_7 (created by: isc_PickListMenu_3)], 
    callback:{
        caller:[ResultSet ID:isc_ResultSet_7 (created by: isc_PickListMenu_3)], 
        methodName:"fetchRemoteDataReply"
    }, 
    willHandleError:true, 
    showPrompt:false, 
    prompt:"Rijen die voldoen aan de zoektermen worden opgehaald", 
    oldValues:{
        firstname:"E"
    }, 
    requestId:"Person$62715", 
    internalClientContext:{
        requestIndex:{
            Class:"Number", 
            formatter:"toString", 
            localeStringFormatter:"toString", 
            localeProperties:Obj
        }
    }, 
    fallbackToEval:false, 
    bypassCache:true
}

In my IDACall subclass handling my requests, I have the line

...
if (o instanceof DSRequest) {
	DSRequest r = (DSRequest) o;
	AdvancedCriteria c = r.getAdvancedCriteria();
	...
}
...

It turns out that this AdvancedCriteria resembles the following:
[and:[{firstname equals E}]]

This 'equals' used to be 'iContains' or something similar, because in a
previous nightly I tried (i think somewhere around january 2013 mark) it
certainly worked.

Is this a regression? How can I get the old behaviour back (without resorting
to the older nightly please)

[Sytematic]

Any ideas? Are you missing information that I need to put here in order for you guys to investigate it?

[Isomorphic]

If this operator was failing in general we should see 100s or 1000s of failing automated tests. So we'll need a test case to look into this further.

[Sytematic]

Alright, a full-on testcase is a bit hard for me to do now, but I do have this.

considering this RPCRequest

=== 2014-09-03 08:45:14,717 [l0-2] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
        surname:"E"
    },
    operationConfig:{
        dataSource:"Person",
        operationType:"fetch",
        textMatchStyle:"substring"
    },
    startRow:0,
    endRow:75,
    sortBy:[
        "firstname"
    ],
    componentId:"isc_PickListMenu_7",
    appID:"builtinApplication",
    operation:"Person_fetch",
    oldValues:{
        surname:"12"
    }
}

With this SecureIDACall servlet that I've built years back:

public class SecureIDACall extends IDACall {

	protected final Log logger = LogFactory.getLog(getClass());

	public void processRequest(HttpServletRequest request,
			HttpServletResponse response) throws ServletException, IOException {

		//retrieve from SpringSecurity framework
		Object principal = SecurityContextHolder.getContext()
				.getAuthentication().getPrincipal();

		SytematicSaltedUser user = null;
		Collection<GrantedAuthority> roles = new ArrayList<GrantedAuthority>();
		Boolean isAuthed;

		if (principal instanceof String && "anonymousUser".equals(principal)) {
			isAuthed = false;
		} else {
			user = (SytematicSaltedUser) principal;
			roles = user.getAuthorities();
			isAuthed = (user != null && user.isEnabled()
					&& user.isAccountNonExpired() && user.isAccountNonLocked());
		}

		if (roles != null && !roles.isEmpty()) {
			try {
				RequestContext context = RequestContext.instance(this, request,
						response);
				RPCManager rpc = new RPCManager(request, response);

				rpc.setAuthenticated(isAuthed);

				String stringRoles = this.getCommaSeparatedRoles(roles);

				rpc.setUserRoles(stringRoles);

				// call processRPCTransaction() to iterate through all RPCRequests and
				// DSRequests and execute them
				List reqs = rpc.getRequests();

				Iterator iter = reqs.iterator();
				while (iter.hasNext()) {
					Object o = iter.next();
					if (o instanceof RPCRequest) {
						RPCResponse resp = handleRPCRequest((RPCRequest) o,
								rpc, context);
						rpc.send(resp);
					}

					if (o instanceof DSRequest) {
						DSRequest r = (DSRequest) o;

						if (!r.getDataSourceName().equals("saasCustomers")

						) {
							if (r.getOperationType().equals("fetch")) {
								AdvancedCriteria c = r.getAdvancedCriteria();
								c = addSaasCustomerIdToCriteria(c,
										user.getSaasCustomer_id());
								r.setAdvancedCriteria(c);
							} else {
								r.setFieldValue("SaasCustomer_id",
										user.getSaasCustomer_id());
							}
						}
						// ... etc.
				}

			} catch (Throwable e) {
				handleError(response, e);
			}
		} else {
			super.processRequest(request, response);
		}

	}

With this older nightly it works:

Isomorphic SmartClient/SmartGWT Framework (v8.3p_2013-01-09/PowerEdition Deployment 2013-01-09) - Initialization Complete

//the line AdvancedCriteria c = r.getAdvancedCriteria(); yields  [and:[{firstname contains E}]]

//CORRECT

With this (newer) nightly however:

Isomorphic SmartClient/SmartGWT Framework (v8.3p_2014-08-19/PowerEdition Deployment 2014-08-19) - Initialization Complete
//the line AdvancedCriteria c = r.getAdvancedCriteria(); yields  [and:[{firstname equals E}]]

//INCORRECT

I basically use the class above (which worked for like 2 years now), and it now behaves differently at this line.

Is this enough information, eventhough it's not a testcase?

[Isomorphic]

Sorry, but no, not really.

This area of the system is absolutely blanketed in automated tests. It's very unlikely we'll be able to replicate the behavior you're seeing through trial and error, even given this snippet of code which might be the source of the problem (or might not!).

[Sytematic]

Hello, I created a server-side, minimal unit test:

The following server side unit test (there are no dependencies, apart from what's necessary, and succeeds on the older SmartGWT 3.1 Pro , but yields the error below on the latest nightly)

@Test
/**
 * You might need to change the webRoot in server.properties to something like System.getProperty("user.dir")+"/war"
 * Change the first two String variables into a datasource that you have.
 */
public void testAdvCriteriaWithSubstringMatching() throws Exception {
	String aDataSourceName = "Project";
	String anExistingField = "projectNumber";
	String substringToMatch = "abb";
	
	DSRequest r = new DSRequest();
	r.setTextMatchStyle("substring");
	r.setOperationType("fetch");
	r.setDataSourceName(aDataSourceName);
	Map<String,Object> crit = new HashMap<String,Object>();
	crit.put(anExistingField, substringToMatch);
	r.setCriteria(crit);
	
	AdvancedCriteria c = r.getAdvancedCriteria();
	System.out.println(c);
	assertEquals("AdvancedCriteria:[and:[{"+anExistingField+" contains "+substringToMatch+"}]]", c.toString());
}

On the failing nightly it gives this error:

org.junit.ComparisonFailure: expected:<...and:[{projectNumber [contain]s abb}]]> but was:<...and:[{projectNumber [equal]s abb}]]>
	at org.junit.Assert.assertEquals(Assert.java:123)
	at org.junit.Assert.assertEquals(Assert.java:145)
	...

Hopefully you can look into this issue, because I don't know how to minimize the testcase even more :-)

Also, in the case that it DOES pass in your environment, then please let me know, as it might be a problem with another lib or something causing a weird side-effect in my situation.

[Isomorphic]

It looks like this issue affects 8.3 only and not any subsequent versions, and only applies to the server-side getAdvancedCriteria() API (not to execution of criteria passed from the client) and only when criteria are specified as simple criteria with a non-default search operator and then converted to AdvancedCriteria. So that explains why it wasn't hit by tests (and there are actually tests for 9.0 and beyond that cover this usage).

Regardless this is fixed for tomorrow's 8.3 builds.

[Sytematic]

Awesome, thanks for working this out. Will try ASAP.