Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

    Mysql ListGrid Date Filtering

    Using: v9.0p_2013-10-23/Pro Deployment

    I've been banging my head against a wall for a while here, and I know I'm just missing something simple. I've got a datasource hooked up to a table in my mysql backend, and a listgrid bound to this datasource.

    All I need to do is to ensure proper filtering of my data. Unfortunately, every way I've tried filtering a date column has not been successful. I understand that only the Power editions and above allow for server-side advanced criteria, but surely there has to be some basic date filtering for the Pro users as well? I just don't understand why the filter editor defaults to sending advanced criteria to the server when it's just sending a simple "equals" request.

    So in short, what's the correct setup for date filtering in a listgrid with a mysql backend?

    My current ds.xml for this datasource:

    Code:
    <DataSource ID="event_media" serverType="sql"
                tableName="event_media" autoDeriveSchema="true" useAnsiJoins="true">
                <fields>
                	<field primaryKey="true"
                		name="EventID"		hidden="true"		type="sequence" />
                	<field name="globalEventId" hidden="true" />
                	<field name="Event_DeviceID" foreignKey="drivesafe3.DriveSafe3ID"
                		hidden="true"/>
                	<field includeFrom="vehicle.VehicleName" title="Vehicle"
                		readOnlyEditorType="staticText" canEdit="false" />
                	<field name="DriverID" foreignKey="employee.EmployeeID" hidden="true"/>
                	<field includeFrom="employee.EmployeeName" title="Driver Name" 
                		canEdit="false" readOnlyEditorType="staticText"/>
                	<field name="Event_TelemetryRecordsID" hidden="true" />
                	<field includeFrom="drivesafe3.Serial" hidden="true" />
                	<field name="State" title="Status">
                		<valueMap>
                			<value>New</value>
                			<value>Reviewed</value>
                			<value>Coached</value>
                			<value>User-Triggered</value>
                		</valueMap>
                	</field>
                	<field name="Date" type="date" />
                	<field name="image" hidden="true" />
                	<field name="video" hidden="true" />
                	<field name="videoFileId" hidden="true" />
                </fields>
            <operationBindings>
            	<operationBinding operationType="fetch" operationId="fetchReviewed">
            		<criteria fieldName="State" value="Reviewed" />
            	</operationBinding>
            </operationBindings>
    </DataSource>

    What gets sent to the server:
    Code:
        criteria:{
            operator:"and",
            _constructor:"AdvancedCriteria",
            criteria:[
                {
                    fieldName:"Date",
                    operator:"equals",
                    value:new Date(1389679200000),
                    _constructor:"AdvancedCriteria"
                }
            ]
        },
        operationConfig:{
            dataSource:"event_media",
            operationType:"fetch",
            textMatchStyle:"substring"
        },
        startRow:0,
        endRow:75,
        sortBy:[
            "-timeOfEvent"
        ],
        componentId:"isc_ListGrid_1",
        appID:"builtinApplication",
        operation:"event_media_fetch",
        oldValues:{
            operator:"and",
            _constructor:"AdvancedCriteria",
            criteria:[
                {
                    fieldName:"Date",
                    operator:"equals",
                    value:new Date(1389679200000),
                    _constructor:"AdvancedCriteria"
                }
            ]
        }
    }

    And my error:
    Code:
    DSRequest for DataSource 'event_media' contained AdvancedCriteria, which the DataSource does not support

    #2
    The default DateRangeItem used in the FilterEditor is always going to produce AdvancedCriteria, because there's no other to way to express a date range.

    To keep simple criteria, just set filterEditorType to "DateItem" so the user just enters a single date.

    Comment


      #3
      The filter editor actually already populated that column with a single date chooser, but I explicitly set it to be a DateItem to see if that would help. It did not. I get the same error as before, with the same AdvancedCriteria being sent to the server.

      Comment


        #4
        Sorry, that behavior is not reproducible. DateItems are designed to produce simple criteria by default, and they do in our tests.

        Comment


          #5
          Is there no event handler that I can intercept and force the request to send with simple criteria?

          On a related note, when I tried using the field.setFilterEditorType("DateItem") method, I got the error - "No BeanFactory has been registered for: DateItem". Am I using this call incorrectly?

          Comment


            #6
            Actually, upon closer inspection, any filtering I do creates AdvancedCriteria, not just on date columns. Is this related to some kind of incorrect setting? I'm just totally at a loss here.

            Comment


              #7
              See the docs for setFilterEditorType(). You should pass a Class, not a String, or you'll be using the Reflection mechanism those docs link to, which is not necessary here.

              Several settings, such as allowFilterExpressions:true, would cause AdvancedCriteria to always be generated. You haven't made it possible for anyone to point out which setting it is (no code for your grid).

              Comment


                #8
                My apologies. I guess I didn't attach it before because my use case was so simple. In fact, I still get the same problem with any of my data sources just by using this code:

                Code:
                ListGrid eventsGrid = new ListGrid();
                eventsGrid.setDataSource(DataSource.get("telemetryrecords"));
                eventsGrid.setShowFilterEditor(true);
                eventsGrid.setAutoFetchData(true);
                This is literally all that's running in my project right now besides some layout code; I've stripped everything else out to figure out this issue.

                Here is my database configuration in server.properties:

                Code:
                sql.test.driver.user: ******
                sql.test.database.type: mysql
                sql.test.driver.serverName: localhost
                sql.test.driver.useUnicode: true
                sql.test.database.ansiMode: false
                sql.test.driver.driverName: mysql
                sql.test.pool.enabled: true
                sql.test.interface.credentialsInURL: true
                sql.test.driver.networkProtocol: tcp
                sql.test.driver.password: ******
                sql.test.driver: com.mysql.jdbc.Driver
                sql.test.driver.databaseName: test
                sql.test.autoJoinTransactions: true
                sql.test.driver.context: 
                sql.test.driver.portNumber: 3306
                sql.test.interface.type: driverManager
                sql.test.useUTCDateTimes: true
                And, as I said, I get this error with any data source that has enough records that they don't get cached client-side. Here is one of the simplest definitions:

                Code:
                <DataSource ID="telemetryrecords" serverType="sql"
                	tableName="telemetryrecords" autoDeriveSchema="true">
                	<fields>
                		<field primaryKey="true" name="TelemetryRecordsID" hidden="true"
                			type="sequence" />
                		<field name="DeviceID" foreignKey="drivesafe3.DriveSafe3ID"
                			hidden="true" />
                	</fields>
                </DataSource>
                As you can imagine, I'm pretty much stuck. I figure at this point it's got to be some kind of overall setting but I can't find it. I would greatly appreciate any insight.

                Comment


                  #9
                  Well, you've got autoDeriveSchema="true", so your grid would be showing whatever fields are in the SQL table, so if one was a date, it would be showing a DateRangeItem, which would produce AdvancedCriteria.

                  But your new code doesn't line up with your previously posted logs. There is no criteria applied to this grid at all according to your new code, but the old logs show an AdvancedCriteria. Grids do not just spontaneously decide to filter against "new Date(1389679200000)" so this can't be all the code involved.

                  Comment


                    #10
                    It's never shown a DateRangeItem, only a DateItem. And here is the log snippet for filtering against a float-value column using the exact code I pasted above (so as you can see, my problem turns out to be not limited to just dates):

                    Code:
                    === 2014-01-16 14:41:00,791 [l0-0] DEBUG RPCManager - Request #1 (DSRequest) payload: {
                        criteria:{
                            operator:"and",
                            _constructor:"AdvancedCriteria",
                            criteria:[
                                {
                                    fieldName:"abs",
                                    operator:"equals",
                                    value:"3",
                                    _constructor:"AdvancedCriteria"
                                }
                            ]
                        },
                        operationConfig:{
                            dataSource:"telemetryrecords",
                            operationType:"fetch",
                            textMatchStyle:"substring"
                        },
                        startRow:0,
                        endRow:75,
                        componentId:"isc_ListGrid_0",
                        appID:"builtinApplication",
                        operation:"telemetryrecords_fetch",
                        oldValues:{
                            operator:"and",
                            _constructor:"AdvancedCriteria",
                            criteria:[
                                {
                                    fieldName:"abs",
                                    operator:"equals",
                                    value:"3",
                                    _constructor:"AdvancedCriteria"
                                }
                            ]
                        }
                    }
                    === 2014-01-16 14:41:00,791 [l0-0] INFO  IDACall - Performing 1 operation(s)
                    === 2014-01-16 14:41:00,793 [l0-0] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
                    === 2014-01-16 14:41:00,793 [l0-0] DEBUG DeclarativeSecurity - DataSource telemetryrecords is not in the pre-checked list, processing...
                    === 2014-01-16 14:41:00,794 [l0-0] WARN  RequestContext - dsRequest.execute() failed: 
                    java.lang.Exception: DSRequest for DataSource 'telemetryrecords' contained AdvancedCriteria, which the DataSource does not support
                    	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:142)
                    	at com.isomorphic.datasource.DataSourceDMI.execute(DataSourceDMI.java:64)
                    	at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2427)
                    	at com.isomorphic.servlet.IDACall.handleDSRequest(IDACall.java:215)
                    	at com.isomorphic.servlet.IDACall.processRPCTransaction(IDACall.java:172)
                    	at com.isomorphic.servlet.IDACall.processRequest(IDACall.java:137)
                    	at com.isomorphic.servlet.IDACall.doPost(IDACall.java:73)
                    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
                    	at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:152)
                    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
                    	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
                    	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1097)
                    	at com.isomorphic.servlet.CompressionFilter.doFilter(CompressionFilter.java:260)
                    	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1088)
                    	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
                    	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
                    	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
                    	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:729)
                    	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
                    	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
                    	at org.mortbay.jetty.handler.RequestLogHandler.handle(RequestLogHandler.java:49)
                    	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
                    	at org.mortbay.jetty.Server.handle(Server.java:324)
                    	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505)
                    	at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:843)
                    	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:647)
                    	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
                    	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
                    	at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
                    	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488)

                    Comment


                      #11
                      BUT, ding ding ding! We have a winner. I removed the autoDeriveSchema from my datasource descriptor and everything worked. Why in the world would that have caused such problems? Did I miss something in the documentation about autoDeriveSchema? It seems odd that it should break all filtering.

                      In any case, thanks for leading me to the solution and keeping me from ripping my hair out!

                      Comment


                        #12
                        Surely not intentional, but you are very good at dropping hints that there just might be an obscure framework bug here, but not quite providing the data needed to diagnose it..

                        All autoDeriveSchema does is generate field definitions from your database. The grid is entirely unaware of the setting. So if we had the fields we'd be able to reproduce the problem if it's really a bug.

                        An easy way to provide them would be to capture the output of the DataSourceLoaderServlet when you load the DataSource.

                        Comment


                          #13
                          To be honest, I assumed it was user error or some weird configuration. How do I capture the entire output of the DataSourceLoaderServlet? My logs only show me the loading of cached instances.

                          Comment


                            #14
                            Originally posted by mgoertz View Post
                            How do I capture the entire output of the DataSourceLoaderServlet?
                            Use Firebug, look at the result of the GET request to project/sc/DataSourceLoader and search for your DataSource-ID (not table name).

                            Best regards,
                            Blama

                            Comment


                              #15
                              Blama - thanks for your help.

                              Below I've listed the response from the DataSourceLoader servlet for the data source I've been using to test with (with autoDeriveSchema set to true):

                              Code:
                              isc.DataSource.create({
                                  allowAdvancedCriteria:false,
                                  tableCode:"eed43bb901b0465cc139bb5a8ee7f1b2",
                                  ID:"telemetryrecords",
                                  autoDeriveSchema:true,
                                  inheritsFrom:isc.DataSource.create({
                                      allowAdvancedCriteria:false,
                                      serverType:"sql",
                                      generatedBy:"v9.0p_2013-10-23/Pro Deployment 2013-10-23",
                                      tableCode:"eed43bb901b0465cc139bb5a8ee7f1b2",
                                      ID:"telemetryrecords_inheritsFrom",
                                      dataSourceVersion:"1",
                                      fields:[
                                          {
                                              columnCode:"70385caadc704c12cdda7f41b5d69431",
                                              name:"TelemetryRecordsID",
                                              primaryKey:true,
                                              type:"sequence"
                                          },
                                          {
                                              columnCode:"c6a84842b14d0bde746e2bf1d33e12ac",
                                              name:"DeviceID",
                                              type:"integer"
                                          },
                                          {
                                              columnCode:"ba569b80f7bb7762f073f1be57cc36aa",
                                              name:"longitude",
                                              type:"float"
                                          },
                                          {
                                              columnCode:"28c1e37e317b935a387dbe232bc9f803",
                                              name:"latitude",
                                              type:"float"
                                          },
                                          {
                                              columnCode:"ce4ef3ec98443ed10da505114b58f153",
                                              name:"speed",
                                              type:"float"
                                          },
                                          {
                                              columnCode:"a0fdbaf1721d616e3113035ed539284d",
                                              name:"brakeValue",
                                              type:"float"
                                          },
                                          {
                                              columnCode:"f9ac6b05beccb0fc5837b6a7fef4c1d3",
                                              name:"abs",
                                              type:"float"
                                          },
                                          {
                                              columnCode:"8d9ff6e49f0dff253a444e518582afea",
                                              name:"turnSignal",
                                              length:15,
                                              type:"text"
                                          },
                                          {
                                              columnCode:"df729cc51281acf7113b320db2f075b1",
                                              name:"engineRPM",
                                              type:"float"
                                          },
                                          {
                                              columnCode:"b4f4f5e3c80cffa0f9cb7debe0bed63f",
                                              name:"mpg",
                                              type:"float"
                                          },
                                          {
                                              columnCode:"c813d2411ca2829573c912b22c264341",
                                              name:"mileage",
                                              type:"float"
                                          },
                                          {
                                              columnCode:"6ef7ebd6f88581fc5aa50d59c2d2a84c",
                                              name:"seatBeltStatus",
                                              length:10,
                                              type:"text"
                                          },
                                          {
                                              columnCode:"6038d1f6ff84df2a62bcf30d950edbf7",
                                              name:"steeringWheelAngle",
                                              type:"float"
                                          },
                                          {
                                              columnCode:"6d4ef5781d09ace89b2ed99411879941",
                                              name:"Time_of_Reading",
                                              type:"datetime"
                                          },
                                          {
                                              columnCode:"5fc732311905cb27e82d67f4f6511f7f",
                                              name:"Date",
                                              type:"date"
                                          },
                                          {
                                              columnCode:"07cc694b9b3fc636710fa08b6922c42b",
                                              name:"Time",
                                              type:"time"
                                          },
                                          {
                                              columnCode:"19194a96742903fcff88fe1366bff56a",
                                              name:"accl_x",
                                              type:"float"
                                          },
                                          {
                                              columnCode:"5f64be2173b5a60bcf01e1ae65c895dd",
                                              name:"accl_y",
                                              type:"float"
                                          },
                                          {
                                              columnCode:"b2e1f48cedbfc251711fb94b111e3aed",
                                              name:"accl_z",
                                              type:"float"
                                          },
                                          {
                                              columnCode:"ef72c37be9d1b9e6e5bbd6ef09448abe",
                                              name:"direction",
                                              length:15,
                                              type:"text"
                                          },
                                          {
                                              columnCode:"4ad8ca24f4da5c9be3470f970bcf49c4",
                                              name:"engineHour",
                                              type:"float"
                                          }
                                      ]
                                  })
                              ,
                                  fields:[
                                      {
                                          hidden:true,
                                          columnCode:"70385caadc704c12cdda7f41b5d69431",
                                          primaryKey:true,
                                          name:"TelemetryRecordsID",
                                          type:"sequence"
                                      },
                                      {
                                          hidden:true,
                                          columnCode:"c6a84842b14d0bde746e2bf1d33e12ac",
                                          name:"DeviceID",
                                          foreignKey:"drivesafe3.DriveSafe3ID"
                                      }
                                  ],
                                  serverType:"sql"
                              })
                              I'd appreciate any further help here - as suspected, removing the autoDeriveSchema property didn't help permanently - which obviously has me even more confused.

                              Comment

                              Working...
                              X