[mgoertz]

Wonderful, that seems to have fixed it! Thanks for all your help!

[Isomorphic]

The issue here is that one of your fields is of type "time" - this results in a filterEditor item of type "TimeItem", which assumes a default of allowing expressions (FormItem.allowExpressions).

This should not be the case in a filterEditor and we've fixed it in the framework - you could then fix your issue by setting ListGrid.allowFilterExpressions(false).

Alternatively, you can switch allowExpressions off for a specific field via its filterEditorProperties, or for TimeItem's globally like this:

TimeItem ti = new TimeItem();
ti.setAllowExpressions(false);
TimeItem.setDefaultProperties(ti);

[Isomorphic]

We're looking into it and will update here when we have more information

[mgoertz]

Hi Blama, thanks again. No other settings have been applied to the ListGrids. I can run with a very simple case of just instantiating a grid and fetching data, and it won't work. I've done a clean build and I've emptied my caches and these actions also don't help. Any filtering that requires a trip to the server to fetch data creates AdvancedCriteria, regardless of the field being filtered.

Isomorphic, can you weigh in at all here??? If I need to, I'll just build a filter from scratch, but it kind of seems a shame to have to do so.

[Blama]

Hi mgoertz,

sorry, no further input from my side besides the usual:

• Are the shown ListGrid settings really all settings applied (also with respect to subclassing and setDefaultProperties()?
• Is the application freshly built (clean, build, deploy))
• Is the client side cache emptied?

Best regards,
Blama

[mgoertz]

Blama - thanks for your help.

Below I've listed the response from the DataSourceLoader servlet for the data source I've been using to test with (with autoDeriveSchema set to true):

isc.DataSource.create({
    allowAdvancedCriteria:false,
    tableCode:"eed43bb901b0465cc139bb5a8ee7f1b2",
    ID:"telemetryrecords",
    autoDeriveSchema:true,
    inheritsFrom:isc.DataSource.create({
        allowAdvancedCriteria:false,
        serverType:"sql",
        generatedBy:"v9.0p_2013-10-23/Pro Deployment 2013-10-23",
        tableCode:"eed43bb901b0465cc139bb5a8ee7f1b2",
        ID:"telemetryrecords_inheritsFrom",
        dataSourceVersion:"1",
        fields:[
            {
                columnCode:"70385caadc704c12cdda7f41b5d69431",
                name:"TelemetryRecordsID",
                primaryKey:true,
                type:"sequence"
            },
            {
                columnCode:"c6a84842b14d0bde746e2bf1d33e12ac",
                name:"DeviceID",
                type:"integer"
            },
            {
                columnCode:"ba569b80f7bb7762f073f1be57cc36aa",
                name:"longitude",
                type:"float"
            },
            {
                columnCode:"28c1e37e317b935a387dbe232bc9f803",
                name:"latitude",
                type:"float"
            },
            {
                columnCode:"ce4ef3ec98443ed10da505114b58f153",
                name:"speed",
                type:"float"
            },
            {
                columnCode:"a0fdbaf1721d616e3113035ed539284d",
                name:"brakeValue",
                type:"float"
            },
            {
                columnCode:"f9ac6b05beccb0fc5837b6a7fef4c1d3",
                name:"abs",
                type:"float"
            },
            {
                columnCode:"8d9ff6e49f0dff253a444e518582afea",
                name:"turnSignal",
                length:15,
                type:"text"
            },
            {
                columnCode:"df729cc51281acf7113b320db2f075b1",
                name:"engineRPM",
                type:"float"
            },
            {
                columnCode:"b4f4f5e3c80cffa0f9cb7debe0bed63f",
                name:"mpg",
                type:"float"
            },
            {
                columnCode:"c813d2411ca2829573c912b22c264341",
                name:"mileage",
                type:"float"
            },
            {
                columnCode:"6ef7ebd6f88581fc5aa50d59c2d2a84c",
                name:"seatBeltStatus",
                length:10,
                type:"text"
            },
            {
                columnCode:"6038d1f6ff84df2a62bcf30d950edbf7",
                name:"steeringWheelAngle",
                type:"float"
            },
            {
                columnCode:"6d4ef5781d09ace89b2ed99411879941",
                name:"Time_of_Reading",
                type:"datetime"
            },
            {
                columnCode:"5fc732311905cb27e82d67f4f6511f7f",
                name:"Date",
                type:"date"
            },
            {
                columnCode:"07cc694b9b3fc636710fa08b6922c42b",
                name:"Time",
                type:"time"
            },
            {
                columnCode:"19194a96742903fcff88fe1366bff56a",
                name:"accl_x",
                type:"float"
            },
            {
                columnCode:"5f64be2173b5a60bcf01e1ae65c895dd",
                name:"accl_y",
                type:"float"
            },
            {
                columnCode:"b2e1f48cedbfc251711fb94b111e3aed",
                name:"accl_z",
                type:"float"
            },
            {
                columnCode:"ef72c37be9d1b9e6e5bbd6ef09448abe",
                name:"direction",
                length:15,
                type:"text"
            },
            {
                columnCode:"4ad8ca24f4da5c9be3470f970bcf49c4",
                name:"engineHour",
                type:"float"
            }
        ]
    })
,
    fields:[
        {
            hidden:true,
            columnCode:"70385caadc704c12cdda7f41b5d69431",
            primaryKey:true,
            name:"TelemetryRecordsID",
            type:"sequence"
        },
        {
            hidden:true,
            columnCode:"c6a84842b14d0bde746e2bf1d33e12ac",
            name:"DeviceID",
            foreignKey:"drivesafe3.DriveSafe3ID"
        }
    ],
    serverType:"sql"
})

I'd appreciate any further help here - as suspected, removing the autoDeriveSchema property didn't help permanently - which obviously has me even more confused.

[Blama]

How do I capture the entire output of the DataSourceLoaderServlet?

Use Firebug, look at the result of the GET request to project/sc/DataSourceLoader and search for your DataSource-ID (not table name).

Best regards,
Blama

[mgoertz]

To be honest, I assumed it was user error or some weird configuration. How do I capture the entire output of the DataSourceLoaderServlet? My logs only show me the loading of cached instances.

[Isomorphic]

Surely not intentional, but you are very good at dropping hints that there just might be an obscure framework bug here, but not quite providing the data needed to diagnose it..

All autoDeriveSchema does is generate field definitions from your database. The grid is entirely unaware of the setting. So if we had the fields we'd be able to reproduce the problem if it's really a bug.

An easy way to provide them would be to capture the output of the DataSourceLoaderServlet when you load the DataSource.

[mgoertz]

BUT, ding ding ding! We have a winner. I removed the autoDeriveSchema from my datasource descriptor and everything worked. Why in the world would that have caused such problems? Did I miss something in the documentation about autoDeriveSchema? It seems odd that it should break all filtering.

In any case, thanks for leading me to the solution and keeping me from ripping my hair out!

[mgoertz]

It's never shown a DateRangeItem, only a DateItem. And here is the log snippet for filtering against a float-value column using the exact code I pasted above (so as you can see, my problem turns out to be not limited to just dates):

=== 2014-01-16 14:41:00,791 [l0-0] DEBUG RPCManager - Request #1 (DSRequest) payload: {
    criteria:{
        operator:"and",
        _constructor:"AdvancedCriteria",
        criteria:[
            {
                fieldName:"abs",
                operator:"equals",
                value:"3",
                _constructor:"AdvancedCriteria"
            }
        ]
    },
    operationConfig:{
        dataSource:"telemetryrecords",
        operationType:"fetch",
        textMatchStyle:"substring"
    },
    startRow:0,
    endRow:75,
    componentId:"isc_ListGrid_0",
    appID:"builtinApplication",
    operation:"telemetryrecords_fetch",
    oldValues:{
        operator:"and",
        _constructor:"AdvancedCriteria",
        criteria:[
            {
                fieldName:"abs",
                operator:"equals",
                value:"3",
                _constructor:"AdvancedCriteria"
            }
        ]
    }
}
=== 2014-01-16 14:41:00,791 [l0-0] INFO  IDACall - Performing 1 operation(s)
=== 2014-01-16 14:41:00,793 [l0-0] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2014-01-16 14:41:00,793 [l0-0] DEBUG DeclarativeSecurity - DataSource telemetryrecords is not in the pre-checked list, processing...
=== 2014-01-16 14:41:00,794 [l0-0] WARN  RequestContext - dsRequest.execute() failed: 
java.lang.Exception: DSRequest for DataSource 'telemetryrecords' contained AdvancedCriteria, which the DataSource does not support
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:142)
	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:64)
	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2427)
	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:215)
	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:172)
	at com.isomorphic.servlet.IDACall.processRequest(IDACall.java:137)
	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:73)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:152)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1097)
	at com.isomorphic.servlet.CompressionFilter.doFilter(CompressionFilter.java:260)
	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1088)
	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:729)
	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.handler.RequestLogHandler.handle(RequestLogHandler.java:49)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.Server.handle(Server.java:324)
	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505)
	at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:843)
	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:647)
	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
	at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488)

[Isomorphic]

Well, you've got autoDeriveSchema="true", so your grid would be showing whatever fields are in the SQL table, so if one was a date, it would be showing a DateRangeItem, which would produce AdvancedCriteria.

But your new code doesn't line up with your previously posted logs. There is no criteria applied to this grid at all according to your new code, but the old logs show an AdvancedCriteria. Grids do not just spontaneously decide to filter against "new Date(1389679200000)" so this can't be all the code involved.

[mgoertz]

My apologies. I guess I didn't attach it before because my use case was so simple. In fact, I still get the same problem with any of my data sources just by using this code:

ListGrid eventsGrid = new ListGrid();
eventsGrid.setDataSource(DataSource.get("telemetryrecords"));
eventsGrid.setShowFilterEditor(true);
eventsGrid.setAutoFetchData(true);

This is literally all that's running in my project right now besides some layout code; I've stripped everything else out to figure out this issue.

Here is my database configuration in server.properties:

sql.test.driver.user: ******
sql.test.database.type: mysql
sql.test.driver.serverName: localhost
sql.test.driver.useUnicode: true
sql.test.database.ansiMode: false
sql.test.driver.driverName: mysql
sql.test.pool.enabled: true
sql.test.interface.credentialsInURL: true
sql.test.driver.networkProtocol: tcp
sql.test.driver.password: ******
sql.test.driver: com.mysql.jdbc.Driver
sql.test.driver.databaseName: test
sql.test.autoJoinTransactions: true
sql.test.driver.context: 
sql.test.driver.portNumber: 3306
sql.test.interface.type: driverManager
sql.test.useUTCDateTimes: true

And, as I said, I get this error with any data source that has enough records that they don't get cached client-side. Here is one of the simplest definitions:

<DataSource ID="telemetryrecords" serverType="sql"
	tableName="telemetryrecords" autoDeriveSchema="true">
	<fields>
		<field primaryKey="true" name="TelemetryRecordsID" hidden="true"
			type="sequence" />
		<field name="DeviceID" foreignKey="drivesafe3.DriveSafe3ID"
			hidden="true" />
	</fields>
</DataSource>

As you can imagine, I'm pretty much stuck. I figure at this point it's got to be some kind of overall setting but I can't find it. I would greatly appreciate any insight.

[Isomorphic]

See the docs for setFilterEditorType(). You should pass a Class, not a String, or you'll be using the Reflection mechanism those docs link to, which is not necessary here.

Several settings, such as allowFilterExpressions:true, would cause AdvancedCriteria to always be generated. You haven't made it possible for anyone to point out which setting it is (no code for your grid).

[mgoertz]

Actually, upon closer inspection, any filtering I do creates AdvancedCriteria, not just on date columns. Is this related to some kind of incorrect setting? I'm just totally at a loss here.