Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

    RESTHandler / RestRequestParser has problem with date-criteria

    Hi Isomorphic,

    please see this request where the client sent lessOrEqual-criteria is transformed to "field IS NULL" (using v10.1p_2016-11-25):
    Client request:
    Code:
    {
        dataSource:"T_CAMPAIGN", 
        operationType:"fetch", 
        operationId:"fetchREST", 
        componentId:"isc_PickListMenu_3", 
        data:{
            fieldName:"STARTDATE", 
            operator:"lessOrEqual", 
            value:"2016-12-01T21:40:59.881", 
            isc_metaDataPrefix:"_", 
            isc_dataFormat:"xml"
        }, 
        startRow:0, 
        endRow:75, 
        sortBy:[
            "NAME"
        ], 
        textMatchStyle:"startsWith", 
        resultSet:[ResultSet ID:isc_ResultSet_8 (dataSource: T_CAMPAIGN, created by: isc_PickListMenu_3)], 
        callback:{
            caller:[ResultSet ID:isc_ResultSet_8 (dataSource: T_CAMPAIGN, created by: isc_PickListMenu_3)], 
            methodName:"fetchRemoteDataReply"
        }, 
        willHandleError:true, 
        showPrompt:false, 
        prompt:"Finding Records that match your criteria...", 
        requestId:"T_CAMPAIGN$62710", 
        internalClientContext:{
            requestIndex:1
        }, 
        fallbackToEval:false, 
        componentContext:"isc_DynamicForm_0.CAMPAIGN_ID", 
        lastClientEventThreadCode:"MUP2", 
        bypassCache:true, 
        dataProtocol:"getParams", 
        isRestRequest:false, 
        dataFormat:"xml"
    }
    Server log (important parts in bold):
    Code:
    === 2016-12-01 22:41:19,725 [ec-4] INFO  Compression - /connector/connector/sc/HttpProxy: 588 -> 242 bytes
    === 2016-12-01 22:41:19,817 [ec-1] INFO  RequestContext - URL: '/connector/connector/sc/HttpProxy', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0': Moz (Gecko) with Accept-Encoding header
    === 2016-12-01 22:41:19,820 [ec-1] DEBUG HttpProxyServlet - HttpProxy - ProxyData is: {
        xsi:"http://www.w3.org/2000/10/XMLSchema-instance",
        url:"http://lms.localhost:8080/lms/lms/sc/RESTHandler",
        httpMethod:"GET",
        params:{
            fieldName:"STARTDATE",
            operator:"lessOrEqual",
    [B]        value:new Date(1480546800000),[/B]
            _constructor:"AdvancedCriteria",
            _operationType:"fetch",
            _operationId:"fetchREST",
            _startRow:0,
            _endRow:75,
            _sortBy:[
                "NAME"
            ],
            _textMatchStyle:"startsWith",
            _componentId:"isc_PickListMenu_3",
            _dataSource:"T_CAMPAIGN",
            isc_metaDataPrefix:"_",
            isc_dataFormat:"xml"
        },
        contentType:null,
        requestBody:null,
        username:null,
        password:null,
        httpHeaders:null,
        uploadFileName:null,
        callbackParam:null
    }
    === 2016-12-01 22:41:19,823 [ec-7] INFO  RequestContext - URL: '/lms/lms/sc/RESTHandler', User-Agent: 'Apache-HttpClient/4.5.1 (Java/1.8.0_111)': Unsupported with Accept-Encoding header
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'fieldName'. Value:'STARTDATE'.
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'operator'. Value:'lessOrEqual'.
    [B]=== 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'value'. Value:'Thu Dec 01 00:00:00 CET 2016'.[/B]
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'_constructor'. Value:'AdvancedCriteria'.
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'_operationType'. Value:'fetch'.
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'_operationId'. Value:'fetchREST'.
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'_startRow'. Value:'0'.
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'_endRow'. Value:'75'.
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'_sortBy'. Value:'NAME'.
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'_textMatchStyle'. Value:'startsWith'.
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'_componentId'. Value:'isc_PickListMenu_3'.
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'_dataSource'. Value:'T_CAMPAIGN'.
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'isc_metaDataPrefix'. Value:'_'.
    === 2016-12-01 22:41:19,825 [ec-7] DEBUG RestRequestParser - Parameter:'isc_dataFormat'. Value:'xml'.
    === 2016-12-01 22:41:19,826 [ec-7] DEBUG SQLDataSource - DataSource 272 acquired SQLDriver instance 1507925835 during initialization
    === 2016-12-01 22:41:19,826 [ec-7] INFO  LMSRESTHandler - Performing 1 operation(s)
    === 2016-12-01 22:41:19,826 [ec-7] DEBUG ServerObject - Couldn't find a public method named: fetch on class: com.lmscompany.lms.server.worker.T_CAMPAIGN
    === 2016-12-01 22:41:19,826 [ec-7] DEBUG DataSourceDMI - DataSourceDMI: no public method name: fetch available on class: com.lmscompany.lms.server.worker.T_CAMPAIGN - defaulting to builtin operations.
    === 2016-12-01 22:41:19,826 [ec-7] DEBUG AppBase - [builtinApplication.fetchREST] No userTypes defined, allowing anyone access to all operations for this application
    === 2016-12-01 22:41:19,826 [ec-7] DEBUG AppBase - [builtinApplication.fetchREST] No public zero-argument method named '_fetchREST' found, performing generic datasource operation
    [B]=== 2016-12-01 22:41:19,827 [ec-7] INFO  SQLDataSource - [builtinApplication.fetchREST] Performing fetch operation with
        criteria: {_constructor:"AdvancedCriteria",criteria:[{fieldName:"TENANT_ID",value:1,operator:"equals"},{fieldName:"STARTDATE",value:"Thu Dec 01 00:00:00 CET 2016",operator:"lessOrEqual"}],operator:"and"}    values: {_constructor:"AdvancedCriteria",criteria:[{fieldName:"TENANT_ID",value:1,operator:"equals"},{fieldName:"STARTDATE",value:"Thu Dec 01 00:00:00 CET 2016",operator:"lessOrEqual"}],operator:"and"}[/B]
    === 2016-12-01 22:41:19,827 [ec-7] INFO  SQLDataSource - [builtinApplication.fetchREST] derived query: SELECT $defaultSelectClause FROM $defaultTableClause WHERE $defaultWhereClause ORDER BY $defaultOrderClause
    === 2016-12-01 22:41:19,828 [ec-7] DEBUG SQLDataSource - [builtinApplication.fetchREST] Executing row count query: SELECT COUNT(*) FROM $defaultTableClause WHERE $defaultWhereClause
    [B]=== 2016-12-01 22:41:19,829 [ec-7] DEBUG SQLDataSource - [builtinApplication.fetchREST] Eval'd row count query: SELECT COUNT(*) FROM T_CAMPAIGN WHERE ((T_CAMPAIGN.TENANT_ID = 1 AND T_CAMPAIGN.TENANT_ID IS NOT NULL) AND (T_CAMPAIGN.STARTDATE IS NULL))[/B]
    === 2016-12-01 22:41:19,829 [ec-7] DEBUG SQLTransaction - [builtinApplication.fetchREST] Started new Oracle transaction "664111256"
    === 2016-12-01 22:41:19,829 [ec-7] DEBUG SQLDataSource - [builtinApplication.fetchREST] Setting DSRequest as being part of a transaction
    [B]=== 2016-12-01 22:41:19,829 [ec-7] INFO  SQLDriver - [builtinApplication.fetchREST] Executing SQL query on 'Oracle' using connection '664111256': SELECT COUNT(*) FROM T_CAMPAIGN WHERE ((T_CAMPAIGN.TENANT_ID = 1 AND T_CAMPAIGN.TENANT_ID IS NOT NULL) AND (T_CAMPAIGN.STARTDATE IS NULL))[/B]
    === 2016-12-01 22:41:19,830 [ec-7] INFO  DSResponse - DSResponse: List with 0 items
    === 2016-12-01 22:41:19,830 [ec-7] WARN  RequestContext - Content type has already been set to: application/json;charset=UTF-8 - setting to: text/html; charset=UTF-8
    === 2016-12-01 22:41:19,830 [ec-7] DEBUG RPCManager - Content type for RPC transaction: text/html; charset=UTF-8
    === 2016-12-01 22:41:19,830 [ec-7] DEBUG SQLTransaction - Committing Oracle transaction "664111256"
    === 2016-12-01 22:41:19,830 [ec-7] DEBUG RPCManager - non-DMI response, dropExtraFields: false
    === 2016-12-01 22:41:19,830 [ec-7] WARN  RequestContext - Content type has already been set to: text/html; charset=UTF-8 - setting to: text/xml
    === 2016-12-01 22:41:19,830 [ec-7] DEBUG SQLTransaction - getConnection() looked for transactional connection for Oracle:  hashcode "664111256"
    === 2016-12-01 22:41:19,830 [ec-7] DEBUG SQLTransaction - Ending Oracle transaction "664111256"
    === 2016-12-01 22:41:19,831 [ec-7] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 272
    === 2016-12-01 22:41:19,831 [ec-7] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 272
    === 2016-12-01 22:41:19,831 [ec-7] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 272
    === 2016-12-01 22:41:19,831 [ec-7] INFO  Compression - /lms/lms/sc/RESTHandler: 202 -> 121 bytes
    === 2016-12-01 22:41:19,831 [ec-7] DEBUG ServletTools - setting cookie 'isc_cState' to: 'ready'
    === 2016-12-01 22:41:19,832 [ec-1] INFO  HttpProxyServlet - HttpProxy - Method succeeded: HTTP/1.1 200 OK
    === 2016-12-01 22:41:19,832 [ec-1] INFO  HttpProxyServlet - HttpProxy - Response:
    <?xml version="1.0"?>
    <response>
    <status>0</status>
    <queueStatus>0</queueStatus><startRow>0</startRow>
    <endRow>0</endRow>
    <totalRows>0</totalRows>
    <queueStatus>0</queueStatus><data>
    </data>
    </response>
    My gut feeling says there is something wrong with the INFO SQLDataSource log. I don't remember seeing a date formatted like this here before. The first occurrence of this String is a bit above with DEBUG RestRequestParser, though.

    Best regards
    Blama

    #2
    Can you show the RestDataSource involved and the inbound request (presumably it's XML), as well the .ds.xml being targeted.

    Note that, one thing that would cause this problem is if your field is not declared as a "date" or "datetime" since it looks like your Date parameter is ending up as a String.

    Also, does this happen only through the HttpProxy, or does it also happen if you go direct?

    If it doesn't happen if you direct, we would want to see the logs for what the HttpProxy is sending, to see how that differs.

    Comment


      #3
      Hi Isomorphic,

      this only happens via REST, not inside my application.

      This is the RestDataSource:
      Code:
      package com.lmscompany.connector.client.datasources;
      
      import com.lmscompany.connector.shared.type.RestDatasourceEnum;
      import com.lmscompany.connector.shared.type.RestDatasourceFieldEnum;
      import com.smartgwt.client.data.DSRequest;
      import com.smartgwt.client.data.OperationBinding;
      import com.smartgwt.client.data.fields.DataSourceDateField;
      import com.smartgwt.client.data.fields.DataSourceIntegerField;
      import com.smartgwt.client.data.fields.DataSourceTextField;
      import com.smartgwt.client.types.DSOperationType;
      
      public class CampaignDS extends LMSRestDataSource {
          private static CampaignDS instance = new CampaignDS();
      
          public static CampaignDS getInstance() {
              return instance;
          }
      
          private CampaignDS() {
              super();
              setID(RestDatasourceEnum.T_CAMPAIGN.getValue());
      
              DataSourceIntegerField id = new DataSourceIntegerField(RestDatasourceFieldEnum.T_CAMPAIGN__ID.getValue());
              id.setPrimaryKey(true);
              id.setCanEdit(false);
      
              DataSourceTextField shortname = new DataSourceTextField(RestDatasourceFieldEnum.T_CAMPAIGN__SHORTNAME.getValue());
              DataSourceTextField name = new DataSourceTextField(RestDatasourceFieldEnum.T_CAMPAIGN__NAME.getValue());
              DataSourceTextField description = new DataSourceTextField(RestDatasourceFieldEnum.T_CAMPAIGN__DESCRIPTION.getValue());
              DataSourceDateField startdate = new DataSourceDateField(RestDatasourceFieldEnum.T_CAMPAIGN__STARTDATE.getValue());
              DataSourceDateField enddate = new DataSourceDateField(RestDatasourceFieldEnum.T_CAMPAIGN__ENDDATE.getValue());
              DataSourceIntegerField cost = new DataSourceIntegerField(RestDatasourceFieldEnum.T_CAMPAIGN__COST.getValue());
      
              setFields(id, shortname, name, description, startdate, enddate, cost);
      
              OperationBinding fetchREST = new OperationBinding();
              fetchREST.setOperationType(DSOperationType.FETCH);
              fetchREST.setOperationId("fetchREST");
              DSRequest dsr = new DSRequest();
              dsr.setOperationId("fetchREST");
              fetchREST.setRequestProperties(dsr);
              setOperationBindings(fetchREST);
          }
      }
      This is the remote DataSource:
      Code:
      <DataSource xmlns="lmscompany/ds" xmlns:fmt="lmscompany/fmt" dbName="Oracle" tableName="T_CAMPAIGN" ID="T_CAMPAIGN" serverType="sql"
          serverConstructor="com.lmscompany.lms.server.LMSSQLDataSource">
          <fmt:bundle basename="com.lmscompany.lms.server.i18n.DSXMLResources-utf8" encoding="utf-8" />
          <fields>
              <field primaryKey="true" hidden="true" name="ID" type="sequence" />
              <field hidden="true" name="TENANT_ID" type="integer" canEdit="false" />
              <field foreignKey="V_USER_CREATED_BY.ID" relatedTableAlias="USER_CREATED_BY" name="CREATED_BY" type="creator">
                  <title><fmt:message key="createdBy" /></title>
              </field>
              <field name="CREATED_AT" type="creatorTimestamp">
                  <title><fmt:message key="createdAt" /></title>
              </field>
              <field foreignKey="V_USER_MODIFIED_BY.ID" relatedTableAlias="USER_MODIFIED_BY" name="MODIFIED_BY" type="modifier">
                  <title><fmt:message key="modifiedBy" /></title>
              </field>
              <field name="MODIFIED_AT" type="modifierTimestamp">
                  <title><fmt:message key="modifiedAt" /></title>
              </field>
              <field name="CREATED_BY_COMPANY_ID" includeFrom="V_USER_CREATED_BY.BELONGSTO_COMPANY_ID" hidden="true" />
              <field name="CREATED_BY_DISTRIBUTOR_ID" includeFrom="V_USER_CREATED_BY.BELONGSTO_DISTRIBUTOR_ID" hidden="true" />
              <field name="CREATED_BY_RESELLER_ID" includeFrom="V_USER_CREATED_BY.BELONGSTO_RESELLER_ID" hidden="true" />
              <field name="SHORTNAME" length="50" type="text" escapeHTML="true" required="true">
                  <title><fmt:message key="shortname" /></title>
                  <validators>
                      <validator type="isUnique" serverOnly="true">
                          <errorMessage><fmt:message key="validatorShortNameInUse" /></errorMessage>
                      </validator>
                  </validators>
              </field>
              <field name="NAME" length="60" type="text" escapeHTML="true" required="true">
                  <title><fmt:message key="name" /></title>
                  <validators>
                      <validator type="isUnique" serverOnly="true">
                          <errorMessage><fmt:message key="validatorNameInUse" /></errorMessage>
                      </validator>
                  </validators>
              </field>
              <field name="DESCRIPTION" length="1000" type="text" escapeHTML="true">
                  <title><fmt:message key="description" /></title>
              </field>
              <field name="CAMPAIGN_COMMENT" length="1000" type="text" escapeHTML="true">
                  <title><fmt:message key="defaultLeadComment" /></title>
              </field>
              <field name="STARTDATE" type="date" required="true">
                  <title><fmt:message key="startDate" /></title>
                  <validators>
                      <validator type="serverCustom" serverOnly="true">
                          <serverCondition><![CDATA[
                            #if(         "$!record.ENDDATE" != "" )
                              #set( $gb = $!record.ENDDATE.getTime() )
                            ##elseif(     "$!dataSource.fetchById($record.ID).ENDDATE" != "" )
                              ##set( $gb = $!dataSource.fetchById($record.ID).ENDDATE.getTime() )
                            #else
                              #set( $gb = -1 )
                            #end
                            
                            $value.getTime() <= $gb || $gb == -1]]></serverCondition>
                          <errorMessage><fmt:message key="validatorStartDateBeforeEndDate2" /></errorMessage>
                      </validator>
                  </validators>
              </field>
              <field name="ENDDATE" type="date">
                  <title><fmt:message key="endDate" /></title>
                  <validators>
                      <!-- Erklärung unter http://forums.smartclient.com/showthread.php?p=108718#post108718 -->
                      <validator type="serverCustom" serverOnly="true">
                          <serverCondition><![CDATA[
                            #if(         "$!value" != "" )
                              #set( $gb = $!value.getTime() )
                            #else
                              #set( $gb = -1 )
                            #end
      
                            #if( "$!record.STARTDATE" != "" )
                              #set( $gv = $!record.STARTDATE.getTime() )
                            #else
                              #set( $gv = $!dataSource.fetchById($record.ID).STARTDATE.getTime() )
                            #end
                            $gv <= $gb || $gb == -1]]></serverCondition>
                          <errorMessage><fmt:message key="validatorStartDateBeforeEndDate2" /></errorMessage>
                      </validator>
                  </validators>
              </field>
              <field name="COST" type="integer">
                  <title><fmt:message key="campaignCost" /></title>
              </field>
              
              <!-- TENANT_IDs for WHERE-clause generation -->
              <field name="CREATED_BY_TENANT_ID" includeFrom="V_USER_CREATED_BY.TENANT_ID" hidden="true" />
              <field name="MODIFIED_BY_TENANT_ID" includeFrom="V_USER_MODIFIED_BY.TENANT_ID" hidden="true" />
          </fields>
          <serverObject lookupStyle="new" className="com.lmscompany.lms.server.worker.T_CAMPAIGN" />
          <operationBindings>
              <!-- Should be changed to editCampaign -->
              <operationBinding operationType="add" requiresRole="editSetting" />
              <operationBinding operationType="update" requiresRole="editSetting" />
              <operationBinding operationType="validate" requiresRole="editSetting" />
              <operationBinding operationType="remove" requiresRole="editSetting" />
              <operationBinding operationType="fetch" />
              <operationBinding operationType="fetch" operationId="fetchREST" requiresAuthentication="false" outputs="ID, NAME, STARTDATE" />
          </operationBindings>
      </DataSource>
      I do think that I included the HttpProxyServlet at the top of the server log - if you need more please let me know what logging categories to activate.

      Also FYI: I'm running my "local" connector application and my existing, now REST-enabled "remote" existing application on the same tomcat, so it might be difficult to tell apart which is what in the server log - but in this case this should be clear by the targeted URLs:
      Code:
      /connector/*: new connector app
      /lms/*: existing app
      Best regards
      Blama

      Comment


        #4
        This didn't answer our questions:

        Can you show the RestDataSource involved and the inbound request (presumably it's XML),
        We still don't have the inbound XML request.

        Also, does this happen only through the HttpProxy, or does it also happen if you go direct?
        We are asking whether contacting the server with a RestDataSource without going through the HttpProxy is working. You seem to have instead answered about what happens if you contact your server but do not use RestDataSource at all.

        Comment

        Working...
        X