Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

    Entering range of DATEITEM messes up the queries

    smartgwtpower5.0p.

    We have a DatasourceField defined as
    <field name="severity" type="text" title="severity" hidden="false" width="5%" initSortColumn="true" multiple="true" >

    In the client code, we have
    Criteria criteria = new Criteria();
    String[] values = new String[4];
    values[0] = "crit";
    values[1] = "maj";
    values[2] = "min";
    values[3] = "warn";
    criteria.setAttribute("severity", values);

    When the user selects two values from the filter and runs the query, the where clause looks like this
    WHERE ...
    AND ((system_event_table.severity LIKE '%crit%' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%maj%' AND system_event_table.severity IS NOT NULL)) ...

    However, when the user specifies start and end time as the filter criteria in another column, it seems like AdvancedCriteria is sent to the server and we see the where clause like this.
    ((system_event_table.severity = 'crit' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE 'crit, %' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%, crit, %' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%, crit' AND system_event_table.severity IS NOT NULL)) OR ((system_event_table.severity = 'maj' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE 'maj, %' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%, maj, %' AND system_event_table.severity IS NOT NULL)

    Granted that the values are using the "OR" clause and since
    the first value in the OR clause satisfies the query works fine. But this seems inefficient to have the where clause contain redundant clause and the wildcard pattern (i.e., LIKE '%, crit') when we know for sure that the column only holds enumerated values as listed in the client code.

    Why is there two different behaviors (and two different queries) when time range is specified?

    Looking at the data sent, the client logic seems to be sending the values in this manner.

    criteria:[
    {
    value:"warn",
    fieldName:"severity",
    operator:"iEquals"
    },
    {
    value:"warn, ",
    fieldName:"severity",
    operator:"iStartsWith"
    },
    {
    value:", warn, ",
    fieldName:"severity",
    operator:"iContains"
    },
    {
    value:", warn",
    fieldName:"severity",
    operator:"iEndsWith"
    }
    ],
    operator:"or
    Last edited by leeyuiwah; 20 Feb 2015, 08:40.

    #2
    Please remember to always post your full version (not just 5.0p).

    Please test this against the patched 5.0p version, as this looks like an issue that was fixed a long time ago.

    If you're still experiencing an issue, finish posting other required details: in this case you've omitted the complete server log for the request, and the request as shown in the RPC tab of the Developer Console.

    Comment


      #3
      Originally posted by Isomorphic View Post
      Please remember to always post your full version (not just 5.0p).

      Please test this against the patched 5.0p version, as this looks like an issue that was fixed a long time ago.

      If you're still experiencing an issue, finish posting other required details: in this case you've omitted the complete server log for the request, and the request as shown in the RPC tab of the Developer Console.
      Downloaded smartgwtpower this morning (02/25/2015). Please see the version info below. Running in CodeServer mode.

      09:30:27.612:INFO:Log:initialized
      09:30:27.616:INFO:Page:app dir is http://127.0.0.1:9000/
      09:30:27.782:INFO:Page:SmartClient Core (v10.0p_2015-02-25/PowerEdition Deployment 2015-02-25) initialized: 228ms

      RPC Request:
      {
      dataSource:"sysEventsDS",
      operationType:"fetch",
      componentId:"isc_ListGrid_0",
      data:{
      operator:"and",
      criteria:[
      {
      fieldName:"severity",
      operator:"inSet",
      value:[
      "crit",
      "maj",
      "min",
      "warn"
      ]
      },
      {
      fieldName:"createtime",
      operator:"greaterOrEqual",
      value:"2015-02-23T05:00:00.000"
      },
      {
      fieldName:"createtime",
      operator:"lessOrEqual",
      value:"2015-02-26T04:59:00.000"
      },
      {
      fieldName:"status",
      operator:"inSet",
      value:[
      "active",
      "acknowledged"
      ]
      },
      {
      fieldName:"detector_str",
      operator:"inSet",
      value:[
      "1"
      ]
      }
      ]
      },
      startRow:0,
      endRow:75,
      sortBy:[
      "severity",
      "-createtime"
      ],
      textMatchStyle:"substring",
      resultSet:[ResultSet ID:isc_ResultSet_0 (dataSource: sysEventsDS, created by: isc_ListGrid_0)],
      callback:{
      caller:[ResultSet ID:isc_ResultSet_0 (dataSource: sysEventsDS, created by: isc_ListGrid_0)],
      methodName:"fetchRemoteDataReply"
      },
      willHandleError:true,
      showPrompt:true,
      prompt:"Finding Records that match your criteria...",
      oldValues:{
      operator:"and",
      criteria:[
      {
      fieldName:"severity",
      operator:"inSet",
      value:[
      "crit",
      "maj",
      "min",
      "warn"
      ]
      },
      {
      fieldName:"createtime",
      operator:"greaterOrEqual",
      value:"2015-02-23T05:00:00.000"
      },
      {
      fieldName:"createtime",
      operator:"lessOrEqual",
      value:"2015-02-26T04:59:00.000"
      },
      {
      fieldName:"status",
      operator:"inSet",
      value:[
      "active",
      "acknowledged"
      ]
      },
      {
      fieldName:"detector_str",
      operator:"inSet",
      value:[
      "1"
      ]
      }
      ]
      },
      requestId:"sysEventsDS$6272",
      internalClientContext:{
      requestIndex:3
      },
      fallbackToEval:false,
      lastClientEventThreadCode:"TMR5",
      bypassCache:true
      }

      ONLY THE WHERE CLAUSE ENCLOSEE BELOW:

      WHERE ((((system_event_table.severity = 'crit' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE 'crit, %' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%, crit, %' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%, crit' AND system_event_table.severity IS NOT NULL)) OR ((system_event_table.severity = 'maj' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE 'maj, %' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%, maj, %' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%, maj' AND system_event_table.severity IS NOT NULL)) OR ((system_event_table.severity = 'min' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE 'min, %' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%, min, %' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%, min' AND system_event_table.severity IS NOT NULL)) OR ((system_event_table.severity = 'warn' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE 'warn, %' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%, warn, %' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%, warn' AND system_event_table.severity IS NOT NULL))) AND (createtime >= UNIX_TIMESTAMP('2015-02-23 00:00:00')) AND (createtime <= UNIX_TIMESTAMP('2015-02-25 23:59:00')) AND (((system_event_table.status = 'active' AND system_event_table.status IS NOT NULL) OR (system_event_table.status LIKE 'active, %' AND system_event_table.status IS NOT NULL) OR (system_event_table.status LIKE '%, active, %' AND system_event_table.status IS NOT NULL) OR (system_event_table.status LIKE '%, active' AND system_event_table.status IS NOT NULL)) OR ((system_event_table.status = 'acknowledged' AND system_event_table.status IS NOT NULL) OR (system_event_table.status LIKE 'acknowledged, %' AND system_event_table.status IS NOT NULL) OR (system_event_table.status LIKE '%, acknowledged, %' AND system_event_table.status IS NOT NULL) OR (system_event_table.status LIKE '%, acknowledged' AND system_event_table.status IS NOT NULL))) AND (('0'='1' OR '0'='1' OR '0'='1' OR '0'='1')))

      The DatasourceField for name="severity" is not generating the where clause correctly. Following is the field definition.

      <field name="severity" type="text" title="severity" hidden="false" width="5%" initSortColumn="true" multiple="true" >
      <valueMap>
      <value ID="crit">Critical</value>
      <value ID="maj">Major</value>
      <value ID="min">Minor</value>
      <value ID="warn">Warn</value>
      <value ID="info">Info</value>
      <value ID="clear">Clear</value>
      <value ID="other">Other</value>
      </valueMap>
      </field>

      The datasource field "status" is not used in the where clause correctly. Following is the field definition.
      <field name="status" type="text" title="Status" hidden="false" width="8%" multiple="true" >
      <valueMap>
      <value>active</value>
      <value>reset_cleared</value>
      <value>manual_cleared</value>
      <value>auto_cleared</value>
      <value>acknowledged</value>
      </valueMap>
      </field>

      [java] 09:48:02.244 [qtp1971798564-33] DEBUG org.eclipse.jetty.server.Server - REQUEST /events/sc/IDACall on AsyncHttpConnection@50712646,g=HttpGenerator{s=0,h=-1,b=-1,c=-1},p=HttpParser{s=2,l=8,c=3948},r=325
      [java] 09:48:02.244 [qtp1971798564-33] DEBUG o.e.j.server.handler.ContextHandler - scope null||/events/sc/IDACall @ c.g.g.d.s.j.WebAppContextWithReload{/,file:.../Dashboard/target/},C:...\target
      [java] 09:48:02.244 [qtp1971798564-33] DEBUG o.e.j.server.handler.ContextHandler - context=||/events/sc/IDACall @ c.g.g.d.s.j.WebAppContextWithReload{/,file:.../target/},C:...\target
      [java] 09:48:02.244 [qtp1971798564-33] DEBUG org.eclipse.jetty.server.session - Got Session ID 12arh47ze825e1rahynbnssu1u from cookie
      [java] 09:48:02.244 [qtp1971798564-33] DEBUG org.eclipse.jetty.server.session - sessionManager=org.eclipse.jetty.server.session.HashSessionManager@237219c
      [java] 09:48:02.244 [qtp1971798564-33] DEBUG org.eclipse.jetty.server.session - session=org.eclipse.jetty.server.session.HashedSession:12arh47ze825e1rahynbnssu1u@104200936
      [java] 09:48:02.244 [qtp1971798564-33] DEBUG o.e.jetty.servlet.ServletHandler - servlet |/events/sc/IDACall|null -> sysIDACall
      [java] 09:48:02.244 [qtp1971798564-33] DEBUG o.e.jetty.servlet.ServletHandler - chain=null
      [java] === 2015-02-25 09:48:02,244 [4-33] INFO RequestContext - URL: '/events/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0': Moz (Gecko) with Accept-Encoding header
      [java] 09:48:02.246 [qtp1971798564-33] DEBUG o.e.jetty.webapp.WebAppClassLoader - getResource(META-INF/services/javax.xml.parsers.DocumentBuilderFactory)=jar:file:/C:/.../dev/gwt-dev.jar!/META-INF/services/javax.xml.parsers.DocumentBuilderFactory
      [java] 09:48:02.247 [qtp1971798564-33] DEBUG o.e.jetty.webapp.WebAppClassLoader - loaded class org.apache.xerces.jaxp.DocumentBuilderFactoryImpl from sun.misc.Launcher$AppClassLoader@1bab2a6
      [java] 09:48:02.248 [qtp1971798564-33] DEBUG o.e.jetty.webapp.WebAppClassLoader - loaded class org.apache.xerces.parsers.XIncludeAwareParserConfiguration from sun.misc.Launcher$AppClassLoader@1bab2a6
      [java] 09:48:02.248 [qtp1971798564-33] DEBUG o.e.jetty.webapp.WebAppClassLoader - loaded class org.apache.xerces.impl.dv.dtd.DTDDVFactoryImpl from sun.misc.Launcher$AppClassLoader@1bab2a6
      [java] === 2015-02-25 09:48:02,250 [4-33] DEBUG XML - Parsed XML from (in memory stream): 4ms
      [java] === 2015-02-25 09:48:02,250 [4-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'transaction'
      [java] === 2015-02-25 09:48:02,250 [4-33] DEBUG PoolableDataSourceFactory - Created DataSource null of type 'transaction' in the pooling flow
      [java] === 2015-02-25 09:48:02,250 [4-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'Object'
      [java] === 2015-02-25 09:48:02,250 [4-33] DEBUG PoolableDataSourceFactory - Created DataSource 22 of type 'Object' and assigned it to thread qtp1971798564-33
      [java] === 2015-02-25 09:48:02,250 [4-33] DEBUG PoolableDataSourceFactory - Created DataSource 22 of type 'Object' in the pooling flow
      [java] === 2015-02-25 09:48:02,250 [4-33] DEBUG PoolableDataSourceFactory - Activated DataSource 22 of type 'Object'
      [java] === 2015-02-25 09:48:02,250 [4-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'List'
      [java] === 2015-02-25 09:48:02,251 [4-33] DEBUG PoolableDataSourceFactory - Created DataSource 23 of type 'List' and assigned it to thread qtp1971798564-33
      [java] === 2015-02-25 09:48:02,251 [4-33] DEBUG PoolableDataSourceFactory - Created DataSource 23 of type 'List' in the pooling flow
      [java] === 2015-02-25 09:48:02,251 [4-33] DEBUG PoolableDataSourceFactory - Activated DataSource 23 of type 'List'
      [java] === 2015-02-25 09:48:02,251 [4-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'elem'
      [java] === 2015-02-25 09:48:02,251 [4-33] DEBUG PoolableDataSourceFactory - Created DataSource null of type 'elem' in the pooling flow
      [java] === 2015-02-25 09:48:02,254 [4-33] DEBUG RPCManager - Processing 1 requests.
      [java] === 2015-02-25 09:48:02,254 [4-33] DEBUG ISCKeyedObjectPool - Borrowing object for 'sysEventsDS'
      [java] === 2015-02-25 09:48:02,254 [4-33] DEBUG PoolableDataSourceFactory - Activated DataSource 10 of type 'sysEventsDS'
      [java] === 2015-02-25 09:48:02,255 [4-33] DEBUG DSRequest - Caching instance 10 of DS 'sysEventsDS' from DSRequest.getDataSource()
      [java] === 2015-02-25 09:48:02,255 [4-33] DEBUG DSRequest - Caching instance 10 of DS sysEventsDS


      [java] === 2015-02-25 09:48:02,580 [4-33] DEBUG PoolableSQLConnectionFactory - [builtinApplication.sysEventsDS_fetch] Executing pingTest 'select 1 from dual' on connection 311337778
      [java] === 2015-02-25 09:48:02,582 [4-33] DEBUG SQLConnectionManager - [builtinApplication.sysEventsDS_fetch] Borrowed connection '311337778'
      [java] === 2015-02-25 09:48:02,583 [4-33] DEBUG SQLTransaction - [builtinApplication.sysEventsDS_fetch] Started new central_aware_db transaction "311337778"
      [java] === 2015-02-25 09:48:02,583 [4-33] DEBUG SQLDriver - [builtinApplication.sysEventsDS_fetch] About to execute SQL query in 'central_aware_db' using connection '311337778'
      Last edited by leeyuiwah; 25 Feb 2015, 08:04.

      Comment


        #4
        We're aware this issue has sat for a while. Just to reassure you, this is currently under investigation. We will follow up when we have more information.

        Regards
        Isomorphic Software

        Comment


          #5
          Hello leeyuiwah,

          We've now had a look at this and what you believe is wrong is actually intended behaviour.

          The key to all this lies in the definition of the severity field.

          Code:
          <field name="severity" type="text" title="severity" hidden="false" width="5%" initSortColumn="true" multiple="true" >
          The multiple="true" attribute is what makes the query a lot bigger than you think it needs to be. The reason for this is that even though you're providing an "enumerated" list of possible values for "severity" this attribute means that the field can hold multiple of these values. For instance the severity field could be crit, maj, min and this is why the query that is generated becomes a bit cumbersome with lots of OR and LIKE matches. For more information on how multiple true fields work please see http://www.smartclient.com/smartgwte...#getMultiple()

          Why is there two different behaviors (and two different queries) when time range is specified?
          Date/time is a red herring here, it's irrelevant what type of fields these are. What happens in these two cases is that in the first case you're supplying a simple criteria which basically matches a field against some values based on textMatchStyle as pointed out in the documentation for multiple true fields. Most likely set as "substring" in your case. When you later use the filter editor in the grid it constructs an advanced criteria, hence making the behaviour different.

          Bottom line is that all this is intended behaviour but what you're eluding to is that you might not need to have "severity" as a "multiple true" field and in which case if you get rid of that attribute your SQL queries will become shorter but you will only be able to store ONE severity in the field ie. crit, maj, min or warn.

          Regards,
          Isomorphic Software

          Comment


            #6
            Thank you very much. The suggestion you made has fixed the issue. However, I do have a follow up question.
            When we don't specify the start/end time (resulting in Criteria object), we see the query as
            WHERE (((system_event_table.severity LIKE '%crit%' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%maj%' AND system_event_table.severity IS NOT NULL) OR (system_event_table.severity LIKE '%min%' AND system_event_table.severity IS NOT NULL) ...

            However when we do specify start/end time (this results in AdvancedCriteria object) and the query is better as follows:
            ((system_event_table.severity IN ('crit', 'maj', 'min', 'warn')) AND system_event_table.severity IS NOT NULL) AND (createtime >= UNIX_TIMESTAMP('2015-03-12 00:00:00'))

            In theory, the semantics of two queries are different for same action by the user. i.e.,
            system_event_table.severity LIKE '%maj%'
            is not the same as
            system_event_table.severity in ('maj')

            For example, rows with severity value = 'submajor' would be picked up by 1st where clause but not 2nd where clause...

            This may have nothing to do with DATETIME column, rather the use of AdvancedCriteria vs. Criteria objects.

            Is there any way to force it to use AdvancedCriteria constructor from the client side at all?

            Thanks in advance.
            Last edited by leeyuiwah; 13 Mar 2015, 09:19.

            Comment


              #7
              This may have nothing to do with DATETIME column, rather the use of AdvancedCriteria vs. Criteria objects.
              That's right. A simple crtieria does not know exactly what to do in order to match the values so it defaults to contains/substring in your case (this can be controlled through defaultTextMatchStyle), hence '%value%' in the query but this can be controlled through textMatchStyle.

              In the client code, we have
              Criteria criteria = new Criteria();
              String[] values = new String[4];
              values[0] = "crit";
              values[1] = "maj";
              values[2] = "min";
              values[3] = "warn";
              criteria.setAttribute("severity", values);
              The assumption made here is that you're constructing this criteria your self. If so just change it and construct an AdvancedCriteria instead with the correct operators you need.

              If not, we need to know what component and set up for it you are using in order to generate the criteria and also how you generate the criteria.

              Regards,
              Isomorphic Software
              Last edited by Isomorphic; 16 Mar 2015, 01:44.

              Comment

              Working...
              X