6.1p: Bug with addToTemplateContext() and isSnippet where the snippet is not evaluated twice

Isomorphic replied

6 Apr 2018, 18:44
The primary purpose of the "snippets" feature is for SQL templates, therefore snippets must be fully evaluated before SQL operations are performed. Allowing snippets that *depend on* the results of the SQL to be executed would require a new API and separate tracking of the registered snippets. We don't plan to attempt to do this at this time; another approach for your use case is to simply do the SQL request as a separate DSRequest, then place the results in the template context for use in mail message formation.
Leave a comment:

Blama replied

3 Apr 2018, 06:13

Hi Isomorphic,

it almost works now, but it seems that in certain (common for me) cases you let Velocity do it's work before it has all the needed data.
Please see this testcase which works just fine in my current normal setup (using <templateFile>), but does not when called with DB content(=with isSnippet and <messageTemplate>$test</messageTemplate>).
Here Helper.longName() is fed a null value, because the velocity-if is analysed before the FETCH is done. You'll notice that the exception is thrown before a SELECT statement is created.

Attached the full changes to BuiltInDS (v11.1p_2018-03-17):

animals.ds.xml:

Code:

<DataSource ID="animals" serverType="sql" tableName="animals" testFileName="animals.data.xml" serverConstructor="com.smartgwt.sample.server.listener.MySQLDataSource">
    <fields>
        <field name="commonName" title="Animal" type="text" />
        <field name="scientificName" title="Scientific Name" type="text" primaryKey="true" required="true" />
        <field name="lifeSpan" title="Life Span" type="integer" />
        <field name="status" title="Endangered Status" type="text">
            <valueMap>
                <value>Threatened</value>
                <value>Endangered</value>
                <value>Not Endangered</value>
                <value>Not currently listed</value>
                <value>May become threatened</value>
                <value>Protected</value>
            </valueMap>
        </field>
        <field name="diet" title="Diet" type="text" />
        <field name="information" title="Interesting Facts" type="text" length="1000" />
        <field name="picture" title="Picture" type="image" detail="true" imageURLPrefix="/isomorphic/system/reference/inlineExamples/tiles/images/" />
    </fields>
    <operationBindings>
        <operationBinding operationType="fetch">
            <mail contentType="text/html">
                <from>[B]your from address[/B]</from>
                <to>[B]your mail account[/B]</to>
                <subject>TESTMAIL</subject>
[B]<messageTemplate>$test</messageTemplate>[/B]
            </mail>
            <criteria fieldName="commonName" operator="equals" value="Anteater" />
        </operationBinding>
    </operationBindings>
</DataSource>

MySQLDataSource.java:

Code:

package com.smartgwt.sample.server.listener;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.sql.SQLDataSource;

public class MySQLDataSource extends SQLDataSource {
    private static final long serialVersionUID = -2103887355803373948L;

    @Override
    public DSResponse executeFetch(DSRequest request) throws Exception {
[B]        request.addToTemplateContext("Helper", new Helper());
        request.addToTemplateContext("test", MailContent.getInstance().getContent(), true);[/B]
        return super.executeFetch(request);
    }
}

MailContent.java:

Code:

package com.smartgwt.sample.server.listener;

public class MailContent {
    private static MailContent instance = new MailContent();

    public static MailContent getInstance() {
        return instance;
    }

    public String getContent() {
        StringBuilder sb = new StringBuilder();
        sb.append("<html><head><meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" /></head><body>");
        sb.append("${commonName} - ${status} <br/>");
        sb.append("$Helper.interpretName(${scientificName}) <br/>");
[B]        // If you comment out the next three lines, it is working
        sb.append("#if($Helper.longName($scientificName))");
        sb.append("LONGNAME");
        sb.append("#end");[/B]
        sb.append("</body></html>");
        return sb.toString();
    }
}

Helper.java:

Code:

package com.smartgwt.sample.server.listener;

public class Helper {
    public String interpretName(String name) {
        if (name != null && name.length() > 20)
            return "Complicated name";
        else if (name != null && name.length() <= 20)
            return "Simple name";
        else
            return "Name is null";
    }

    public boolean longName(String name) {
        if (name == null)
            throw new NullPointerException("String name is null in Helper.longName()");
        return (name != null && name.length() >= 10);
    }
}

Best regards
Blama

Leave a comment:

Isomorphic replied

21 Mar 2018, 15:15
You need to make the following changes in your code in order to avoid that exception:

in MySQLDataSource:

Code:

request.addToTemplateContext("test", MailContent.getInstance().getContent(), true);

and in animals.ds.xml :

Code:

<messageTemplate>$test</messageTemplate>

So, while objects are allowed in the templateContext, they are not allowed as snippets.

Regards
Isomorphic Software
Leave a comment:

Blama replied

21 Mar 2018, 07:32

Hi Isomorphic,

I retested using v11.1p_2018-03-17. This is working in the case from #1, but not in the more complicated case I have in my application where I use a singleton-class method as source for the "$commonName - $status".
In this case I get a class cast exception.

Please see this sample:
animals.ds.xml:

Code:

<DataSource ID="animals" serverType="sql" tableName="animals" testFileName="animals.data.xml" serverConstructor="com.smartgwt.sample.server.listener.MySQLDataSource">
    <fields>
        <field name="commonName" title="Animal" type="text" />
        <field name="scientificName" title="Scientific Name" type="text" primaryKey="true" required="true" />
        <field name="lifeSpan" title="Life Span" type="integer" />
        <field name="status" title="Endangered Status" type="text">
            <valueMap>
                <value>Threatened</value>
                <value>Endangered</value>
                <value>Not Endangered</value>
                <value>Not currently listed</value>
                <value>May become threatened</value>
                <value>Protected</value>
            </valueMap>
        </field>
        <field name="diet" title="Diet" type="text" />
        <field name="information" title="Interesting Facts" type="text" length="1000" />
        <field name="picture" title="Picture" type="image" detail="true" imageURLPrefix="/isomorphic/system/reference/inlineExamples/tiles/images/" />
    </fields>
    <operationBindings>
        <operationBinding operationType="fetch">
            <mail contentType="text/html">
                <from>[B]your from address[/B]</from>
                <to>[B]yourmail[/B]</to>
                <subject>TESTMAIL</subject>
                <messageTemplate>[B]$test.getContent()[/B]</messageTemplate>
            </mail>
            <criteria fieldName="commonName" operator="equals" value="Anteater" />
        </operationBinding>
    </operationBindings>
</DataSource>

MailContent.java:

Code:

package com.smartgwt.sample.server.listener;

public class MailContent {
    private static MailContent instance = new MailContent();

    public static MailContent getInstance() {
        return instance;
    }

    public static String getContent() {
[B]return "$commonName - $status";[/B]
    }
}

MySQLDataSource.java:

Code:

package com.smartgwt.sample.server.listener;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.sql.SQLDataSource;

public class MySQLDataSource extends SQLDataSource {
    private static final long serialVersionUID = -2103887355803373948L;

    @Override
    public DSResponse executeFetch(DSRequest request) throws Exception {
        request.addToTemplateContext("test", MailContent.getInstance(), true);
        return super.executeFetch(request);
    }
}

The error returned for the same call as in #1 is this in the server.log:

Code:

=== 2018-03-21 16:30:10,304 [0-33] INFO  RequestContext - URL: '/builtinds/sc/RESTHandler', User-Agent: 'null': Unsupported WITHOUT Accept-Encoding header
=== 2018-03-21 16:30:10,306 [0-33] DEBUG RESTHandler - Defaulting response data format to xml
=== 2018-03-21 16:30:10,308 [0-33] DEBUG RestRequestParser - Parsing xml object: '<request>
    <dataSource>animals</dataSource>
    <operationType>fetch</operationType>
</request>'
=== 2018-03-21 16:30:10,310 [0-33] DEBUG XML - Parsed XML from (in memory stream): 2ms
=== 2018-03-21 16:30:10,312 [0-33] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 30
=== 2018-03-21 16:30:10,313 [0-33] DEBUG ProcessedFileCache - STALE object for file 'C:\Users\ST\workspace\lib\smartgwtpower-6.1p\samples\built-in-ds\war\ds\animals.ds.xml', reloading (file timestamp 1521645993669, cache timestamp 1521645699047)
=== 2018-03-21 16:30:10,315 [0-33] DEBUG XML - Parsed XML from C:\Users\ST\workspace\lib\smartgwtpower-6.1p\samples\built-in-ds\war\ds\animals.ds.xml: 2ms
=== 2018-03-21 16:30:10,326 [0-33] INFO  RESTHandler - Performing 1 operation(s)
=== 2018-03-21 16:30:10,326 [0-33] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2018-03-21 16:30:10,326 [0-33] DEBUG DeclarativeSecurity - DataSource animals is not in the pre-checked list, processing...
=== 2018-03-21 16:30:10,326 [0-33] DEBUG DSTransaction - About to add to criteria: commonName equals Anteater
=== 2018-03-21 16:30:10,327 [0-33] DEBUG AppBase - [builtinApplication.animals_fetch] No userTypes defined, allowing anyone access to all operations for this application
=== 2018-03-21 16:30:10,327 [0-33] DEBUG AppBase - [builtinApplication.animals_fetch] No public zero-argument method named '_animals_fetch' found, performing generic datasource operation
=== 2018-03-21 16:30:10,327 [0-33] INFO  SQLDataSource - [builtinApplication.animals_fetch] Performing fetch operation with
    criteria: {commonName:"Anteater"}    values: {commonName:"Anteater"}
=== 2018-03-21 16:30:10,328 [0-33] DEBUG SQLDataSource - [builtinApplication.animals_fetch] DataSource 39 acquired SQLDriver instance 1624147538 during initialization
=== 2018-03-21 16:30:10,328 [0-33] INFO  SQLDataSource - [builtinApplication.animals_fetch] derived query: SELECT $defaultSelectClause FROM $defaultTableClause WHERE $defaultWhereClause
=== 2018-03-21 16:30:10,328 [0-33] WARN  RequestContext - dsRequest.execute() failed: 
[B]java.lang.ClassCastException: com.smartgwt.sample.server.listener.MailContent cannot be cast to java.lang.String
    at com.isomorphic.velocity.Velocity.evaluateWithSnippets(Velocity.java:491)[/B]
    at com.isomorphic.sql.SQLDataSource.generateSQLStatement(SQLDataSource.java:1491)
    at com.isomorphic.sql.SQLDataSource.SQLExecute(SQLDataSource.java:1728)
    at com.isomorphic.sql.SQLDataSource.processRequest(SQLDataSource.java:439)
    at com.isomorphic.sql.SQLDataSource.executeFetch(SQLDataSource.java:384)
    at com.smartgwt.sample.server.listener.MySQLDataSource.executeFetch(MySQLDataSource.java:13)
    at com.isomorphic.datasource.DataSource.execute(DataSource.java:2274)
    at com.isomorphic.application.AppBase.executeDefaultDSOperation(AppBase.java:628)
    at com.isomorphic.application.AppBase.executeAppOperation(AppBase.java:548)
    at com.isomorphic.application.AppBase.execute(AppBase.java:491)
    at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2815)
    at com.isomorphic.servlet.RESTHandler.handleDSRequest(RESTHandler.java:678)
    at com.isomorphic.servlet.RESTHandler.processRestTransaction(RESTHandler.java:561)
    at com.isomorphic.servlet.RESTHandler.processRequest(RESTHandler.java:524)
    at com.isomorphic.servlet.RESTHandler.doPost(RESTHandler.java:421)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
    at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:176)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494)
    at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:247)
    at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:93)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:68)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:370)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
    at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Unknown Source)
=== 2018-03-21 16:30:10,329 [0-33] DEBUG RPCManager - Content type for RPC transaction: text/html; charset=UTF-8
=== 2018-03-21 16:30:10,329 [0-33] DEBUG RPCManager - non-DMI response, dropExtraFields: false
=== 2018-03-21 16:30:10,329 [0-33] WARN  RequestContext - Content type has already been set to: text/html; charset=UTF-8 - setting to: text/xml
=== 2018-03-21 16:30:10,330 [0-33] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 39
=== 2018-03-21 16:30:10,330 [0-33] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 39
=== 2018-03-21 16:30:10,330 [0-33] DEBUG SQLDataSource - About to clear SQLDriver state for DS instance 39

Best regards
Blama

6.1p: Bug with addToTemplateContext() and isSnippet where the snippet is not evaluated twice

Hi Isomorphic,

please see this testcase where I use addToTemplateContext() with isSnippet:true, as you suggest here.
I'd expect to see in my mail body "Anteater - Not Endangered", like I do with commented out

Code:

<!-- <messageTemplate>$commonName - $status</messageTemplate> -->

in animals.ds.xml. Instead I get "$commonName - $status". I believe this is a bug (using v11.1p_2018-02-27).

animals.ds.xml:

Code:

<DataSource
  ID="animals"
    serverType="sql"
    tableName="animals"
  testFileName="animals.data.xml" serverConstructor="com.smartgwt.sample.server.listener.MySQLDataSource">
    <fields>
        <field name="commonName"      title="Animal"             type="text"/>
        <field name="scientificName"  title="Scientific Name"    type="text"  primaryKey="true"  required="true"/>
        <field name="lifeSpan"        title="Life Span"          type="integer"/>
        <field name="status"          title="Endangered Status"  type="text">
            <valueMap>
                <value>Threatened</value>
                <value>Endangered</value>
                <value>Not Endangered</value>
                <value>Not currently listed</value>
                <value>May become threatened</value>
                <value>Protected</value>
            </valueMap>
        </field>
        <field name="diet"            title="Diet"               type="text"/>
        <field name="information"     title="Interesting Facts"  type="text"  length="1000"/>
        <field name="picture"         title="Picture"            type="image" detail="true"
               imageURLPrefix="/isomorphic/system/reference/inlineExamples/tiles/images/"/>
    </fields>
    <operationBindings>
        <!-- Server operationBindings  -->
        <operationBinding operationType="fetch">
            <mail contentType="text/html">
                <from>[B]your server mail address[/B]</from>
                <to>[B]your user mail account[/B]</to>
                <subject>TESTMAIL</subject>
                <messageTemplate>$test</messageTemplate>
                <!-- <messageTemplate>$commonName - $status</messageTemplate> -->
            </mail>
          <criteria fieldName="commonName" operator="equals" value="Anteater" />
        </operationBinding>
    </operationBindings>
</DataSource>

MySQLDataSource.java:

Code:

package com.smartgwt.sample.server.listener;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.sql.SQLDataSource;

public class MySQLDataSource extends SQLDataSource {
    private static final long serialVersionUID = -2103887355803373948L;

    @Override
    public DSResponse executeFetch(DSRequest request) throws Exception {
[B]request.addToTemplateContext("test", "$commonName - $status", true);[/B]
        return super.executeFetch(request);
    }
}

Call the sample with ARC (Advanced REST Client Addon in Chrome) and this data:
URL: http://127.0.0.1:8888/builtinds/sc/RESTHandler
Body:

Code:

<request>
    <dataSource>animals</dataSource>
    <operationType>fetch</operationType>
</request>

My use case is that I want to pass DB content here with addToTemplateContext() instead of using <templateFile>.
The DB content will be the file content at start, but then every tenant can modify this.

This is an important one for me.

Best regards
Blama

As enhancement:
It would be even more easy for me if <messageTemplate> supported something like <messageTemplate isSnippet=true> - Then I would not even need to change my SQLDataSource-subclass, as I can easily get the DB content with my existing custom-Velocity variables.

Last edited by Blama; 8 Mar 2018, 02:42.

Tags: None

Previous template Next

Announcement

Leave a comment:

Leave a comment:

Leave a comment:

Leave a comment:

Leave a comment:

6.1p: Bug with addToTemplateContext() and isSnippet where the snippet is not evaluated twice