Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

    SQL templating with MySQL's ifnull() function?

    Hi,

    I need to use SQL templating with customSelectExpression with a MySQL function 'ifnull()' to do something special when the DB column 'app_name' is null

    Code:
    ifnull( da.app_name, 'need decoding' ) as appDecoded
    (This is a simplified version of the final select expression that I will use, but even this one was not working.)

    Upon a 'fetch' operation, there were no MySQL syntax error messages, but the field values were all null (blank) in the databound visual component. Other fields in the same table could be correctly fetched.

    So has anyone tried using MySQL's ifnull() function with SmartGWT?

    My software and browser versions:

    SmartClient Version: v8.3p_2013-03-22
    Chrome Version 26.0.1410.43m

    Log file content:
    Code:
    === 2013-04-01 19:11:18,815 [0-28] INFO  RequestContext - URL: '/leecyDynDs2/sc/IDACall', User-Agent: 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43 Safari/537.31': Safari with Accept-Encoding header
    === 2013-04-01 19:11:18,817 [0-28] DEBUG XML - Parsed XML from (in memory stream): 2ms
    === 2013-04-01 19:11:18,818 [0-28] DEBUG RPCManager - Processing 1 requests.
    === 2013-04-01 19:11:18,818 [0-28] DEBUG RPCManager - Request #1 (DSRequest) payload: {
        criteria:{
        },
        operationConfig:{
            dataSource:"topAppsRealmsDaily",
            operationType:"fetch",
            textMatchStyle:"exact"
        },
        startRow:76,
        endRow:151,
        componentId:"isc_ListGrid_1",
        appID:"builtinApplication",
        operation:"topAppsRealmsDaily_fetch",
        oldValues:{
        }
    }
    === 2013-04-01 19:11:18,820 [0-28] DEBUG XML - Parsed XML from (in memory stream): 2ms
    === 2013-04-01 19:11:18,822 [0-28] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
    === 2013-04-01 19:11:18,822 [0-28] DEBUG DeclarativeSecurity - DataSource topAppsRealmsDaily is not in the pre-checked list, processing...
    === 2013-04-01 19:11:18,823 [0-28] DEBUG AppBase - [builtinApplication.topAppsRealmsDaily_fetch] No userTypes defined, allowing anyone access to all operations for this application
    === 2013-04-01 19:11:18,823 [0-28] DEBUG AppBase - [builtinApplication.topAppsRealmsDaily_fetch] No public zero-argument method named '_topAppsRealmsDaily_fetch' found, performing generic datasource operation
    === 2013-04-01 19:11:18,823 [0-28] INFO  SQLDataSource - [builtinApplication.topAppsRealmsDaily_fetch] Performing fetch operation with
    	criteria: {}	values: {}
    === 2013-04-01 19:11:18,824 [0-28] INFO  SQLWhereClause - [builtinApplication.topAppsRealmsDaily_fetch] empty condition
    === 2013-04-01 19:11:18,824 [0-28] INFO  SQLDataSource - [builtinApplication.topAppsRealmsDaily_fetch] derived query: SELECT 
                    t.id, t.date, t.application, da.app_name, t.realm_name, t.num_subscribers, t.upBytes
                    , ifnull( da.app_name, 'need decoding'
                            ) as appDecoded
                
                 FROM 
                    leecy_top_apps as t left outer join app_table as da on t.application = da.app_id
                 WHERE 
                    true or t.application >= (2<<24) 
                
                
    === 2013-04-01 19:11:18,824 [0-28] DEBUG SQLDataSource - [builtinApplication.topAppsRealmsDaily_fetch] Executing row count query: SELECT COUNT(*) FROM 
                    leecy_top_apps as t left outer join app_table as da on t.application = da.app_id
                 WHERE 
                    true or t.application >= (2<<24) 
                
                
    === 2013-04-01 19:11:18,824 [0-28] DEBUG SQLDataSource - [builtinApplication.topAppsRealmsDaily_fetch] Eval'd row count query: SELECT COUNT(*) FROM 
                    leecy_top_apps as t left outer join app_table as da on t.application = da.app_id
                 WHERE 
                    true or t.application >= (2<<24) 
                
                
    === 2013-04-01 19:11:18,824 [0-28] DEBUG PoolableSQLConnectionFactory - [builtinApplication.topAppsRealmsDaily_fetch] Executing pingTest 'select 1 from dual' on connection 2144383467
    === 2013-04-01 19:11:18,825 [0-28] DEBUG SQLConnectionManager - [builtinApplication.topAppsRealmsDaily_fetch] Returning borrowed connection '2144383467'
    === 2013-04-01 19:11:18,826 [0-28] DEBUG SQLTransaction - [builtinApplication.topAppsRealmsDaily_fetch] Started new central_aware_db_4100 transaction "2144383467"
    === 2013-04-01 19:11:18,826 [0-28] DEBUG SQLDriver - [builtinApplication.topAppsRealmsDaily_fetch] About to execute SQL query in 'central_aware_db_4100' using connection '2144383467'
    === 2013-04-01 19:11:18,826 [0-28] INFO  SQLDriver - [builtinApplication.topAppsRealmsDaily_fetch] Executing SQL query on 'central_aware_db_4100': SELECT COUNT(*) FROM 
                    leecy_top_apps as t left outer join app_table as da on t.application = da.app_id
                 WHERE 
                    true or t.application >= (2<<24) 
                
                
    === 2013-04-01 19:11:19,187 [0-28] INFO  SQLDataSource - [builtinApplication.topAppsRealmsDaily_fetch] Row count query found 223662 rows, which exceeds the DataSource's progressiveLoadingThreshold of 200000.  Switching to progressive loading mode.
    === 2013-04-01 19:11:19,187 [0-28] DEBUG SQLDataSource - [builtinApplication.topAppsRealmsDaily_fetch] Using SQL Limit query
    === 2013-04-01 19:11:19,187 [0-28] DEBUG SQLDataSource - [builtinApplication.topAppsRealmsDaily_fetch] SQL windowed select rows 76->151, result size 76. Query: SELECT 
                    t.id, t.date, t.application, da.app_name, t.realm_name, t.num_subscribers, t.upBytes
                    , ifnull( da.app_name, 'need decoding'
                            ) as appDecoded
                
                 FROM 
                    leecy_top_apps as t left outer join app_table as da on t.application = da.app_id
                 WHERE 
                    true or t.application >= (2<<24) 
                
                 LIMIT 76, 76
    === 2013-04-01 19:11:19,218 [0-28] INFO  DSResponse - [builtinApplication.topAppsRealmsDaily_fetch] DSResponse: List with 76 items
    === 2013-04-01 19:11:19,218 [0-28] DEBUG RPCManager - Content type for RPC transaction: text/plain; charset=UTF-8
    === 2013-04-01 19:11:19,218 [0-28] DEBUG SQLTransaction - Committing central_aware_db_4100 transaction "2144383467"
    === 2013-04-01 19:11:19,221 [0-28] DEBUG SQLTransaction - Returning transactional connection for central_aware_db_4100 with hashcode "2144383467"
    === 2013-04-01 19:11:19,221 [0-28] DEBUG SQLTransaction - Ending central_aware_db_4100 transaction "2144383467"
    === 2013-04-01 19:11:19,222 [0-28] DEBUG SQLConnectionManager - About to close PoolableConnection with hashcode "2144383467"
    === 2013-04-01 19:11:19,222 [0-28] DEBUG PoolableSQLConnectionFactory - Executing pingTest 'select 1 from dual' on connection 2144383467
    === 2013-04-01 19:11:19,224 [0-28] INFO  Compression - /leecyDynDs2/sc/IDACall: 12335 -> 1470 bytes

    #2
    MySQL's concat() also did not appear to work in customSelectExpression

    Hi,

    I also just tried MySQL's concat() function and it did not appear to work either. (The values in the visual component were blank, which I think mean null value).

    Code:
    concat(t.realm_name, ' (REALM)') realmNameExpanded
    Again, there were no complains about syntax errors, and other fields could be fetched. So the SQL statement seemed to have executed fine.

    Comment


      #3
      You can see the resulting SQL query, with your modifications caused by SQL templating, right in the log.

      It executes the same way it would if you ran it in MySQL, so if it's not doing what you hoped, the next step is to analyze the actual SQL - SmartGWT is no longer part of the processing.

      Comment


        #4
        Could run the query printed in the log, but in visual component the ifnull() failed

        Dear Isomorphic,

        Thanks for the quick reply.

        When you say the log file, is it showing the content that I printed above (in #1)? (This log content I obtained from the "Console" tab of my eclipse)

        In the log, I could see the MySQL SQL statement expanded correctly by SmartGWT, and if I ran it with a MySQL GUI (Workbench), the statement generated correct result for all fields -- including the field with the ifnull() function. Here let me the query:

        Code:
        SELECT 
                        t.id, t.date, t.application, da.app_name, t.realm_name, t.num_subscribers, t.upBytes
                        , ifnull( da.app_name, 'need decoding'
                                ) as appDecoded
                    
                     FROM 
                        leecy_top_apps as t left outer join app_table as da on t.application = da.app_id
                     WHERE 
                        true or t.application >= (2<<24) 
                    
                     LIMIT 76, 76
        However, in SmartGWT, the field with the ifnull() function become null. A similar test with the MySQL concat() function gave the same result (null value).

        Comment


          #5
          You've showed the customSelectExpression, but not the <field> definition and surrounding DataSource. If the field isn't named "appDecoded" for example, that would be a problem.

          Comment


            #6
            my ds.xml

            Here is my ds.xml, the three columns that involve MySQL ifnull() and concat() functions all display null values for all rows.

            Code:
            <DataSource
                ID="topAppsRealmsDaily"
            	serverType="sql"
            	tableName="leecy_top_apps"
            >
                <fields>
                    <field name="id"                  title="ID"
                             type="integer" primaryKey="true" required="true" customSelectExpression="t.id"/>
                    <field name="date"                title="Date"
                             type="integer" customSelectExpression="t.date" />
                    <field name="application"         title="App Id (Raw)"
                             type="integer" customSelectExpression="t.application" />
                    <field name="app_name"            title="App Name "
                             type="text" customSelectExpression="da.app_name" />
                    <field name="realm_name"          title="Realm"
                             type="text" customSelectExpression="t.realm_name"/>
                    <field name="app_name_decoded1"   title="App Name (1)"
                             type="text" customSelectExpression="appDecoded" />
                    <field name="app_name_decoded2"   title="App Name (2)"
                             type="text" customSelectExpression="ifnull( da.app_name, 'need decoding')" />
                    <field name="realm_name_expanded" title="Realm (E)"
                             type="text" customSelectExpression="realmNameExpanded"/>
                </fields>
            
                <operationBindings>
                    <operationBinding
                      operationType="fetch"
                    >
                        <selectClause><![CDATA[
                            t.id
                            , t.date
                            , t.application, da.app_name, t.realm_name
                            , concat(t.realm_name, ' (REALM)') as realmNameExpanded
                            , ifnull( da.app_name, 'need decoding') as appDecoded
                        ]]>
                        </selectClause>
                        <tableClause>
                            leecy_top_apps as t left outer join app_table as da on t.application = da.app_id
                        </tableClause>
                    </operationBinding>
                </operationBindings>
            </DataSource>
            One more note: I am not using a static ds.xml file as such. Instead, the above XML content is loaded dynamically using DataSource.fromXML()

            Comment


              #7
              This doesn't match the logs.

              What you need to do is ensure that if you use as an "as" expression, it matches the field name.

              Comment


                #8
                in XML field's name must match what is after AS

                Thanks for your tip ! I got it working now.

                I rewrote my field definition by making sure the the "name" matches what is after AS in the selectClause. And now I could see the column with MySQL function concat() (or ifnull() for that matter)

                In my <selectClause>
                Code:
                concat(`t`.`realm_name`, ' (REALM)') as realmNameExpanded
                In my field definition
                Code:
                <field name="realmNameExpanded" title="Realm (E)"
                                 type="text" />
                I think I got confused thinking that SmartGWT was trying to match 'customSelectExpression' in the field definition with what is after AS. Now I realize it is trying to match 'name' with what is after AS. So in the following, the 'customSelectExpression' is actually not used.

                Code:
                <field name="realm_name"          title="Realm"
                                 type="text" customSelectExpression="t.realm_name"/>
                Thanks for your help!

                Comment

                Working...
                X