6.1p: viewRequires + velocity problem with empty template variable and therefore broken viewRequires

[Blama]

Hi Isomorphic,

I'm trying to use viewRequires as you describe it in the docs:

Indicates that the specified VelocityExpression must evaluate to true if values for the field are to be fetched. If the specified expression does not evaluate to true, the field will be dropped as described for viewRequiresAuthentication.

In addition to the normal context variables available to Velocity expressions in Smart GWT, expressions you write for field-level requires clauses - viewRequires, editRequires, initRequires and updateRequires - can reference two additional variables: $fieldName and $dsName. These are the names of the dataSource and field currently undergoing requires checks. They are helpful because they allow you to write a generic checker function that can be used to handle requires checks for multiple fields and dataSources.

I do have a problem with a template variable I need in velocity to evaluate the value, though. To me it seems that this template variable is never set - the breakpoint for it is never hit for a fetch nor for DataSourceLoader (which might be correct, but then I'd need to know how to add those here). Please see this BuiltInDS-based sample (v11.1p_2018-03-17) and select "Animals":

animals.ds.xml:

<DataSource
  ID="animals"
    serverType="sql"
    tableName="animals"
    testFileName="animals.data.xml" serverConstructor="com.smartgwt.sample.server.listener.MySQLDataSource">
    <fields>
        <field name="commonName"      title="Animal"             type="text"/>
        <field name="scientificName"  title="Scientific Name"    type="text"  primaryKey="true"  required="true"/>
        <field name="lifeSpan"        title="Life Span"          type="integer"/>
        <field name="status"          title="Endangered Status"  type="text">
            <valueMap>
                <value>Threatened</value>
                <value>Endangered</value>
                <value>Not Endangered</value>
                <value>Not currently listed</value>
                <value>May become threatened</value>
                <value>Protected</value>
            </valueMap>
        </field>
[B]       <field name="diet"            title="Diet"               type="text"                viewRequires="$t_config.getVisible(&quot;diet&quot;, $servletRequest)" />
        <field name="information"     title="Interesting Facts"  type="text"  length="1000" viewRequires="$t_config.getVisible(&quot;diet&quot;)" /> [/B]
          <!--
                <field name="diet"            title="Diet"               type="text"  />
        <field name="information"     title="Interesting Facts"  type="text"  length="1000" />
        -->
        <field name="picture"         title="Picture"            type="image" detail="true"
               imageURLPrefix="/isomorphic/system/reference/inlineExamples/tiles/images/"/>
    </fields>
</DataSource>

MySQLDataSource.java:

package com.smartgwt.sample.server.listener;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.sql.SQLDataSource;

public class MySQLDataSource extends SQLDataSource {

    private static final long serialVersionUID = 1L;

[B]    public static DSRequest addDefaultTemplateContext(DSRequest request) throws Exception {
        request.addToTemplateContext("t_config", T_CONFIG.getInstance());
        return request;
    }[/B]

    @Override
    public DSResponse executeFetch(DSRequest request) throws Exception {
        // Not hit during DataSourceLoader - most likely OK
[B]        addDefaultTemplateContext(request);[/B]
        DSResponse fetchResponse = super.executeFetch(request);
        return fetchResponse;
    }
}

T_CONFIG.java - I add all template classes like this as singleton with a timed data reload (no data here):

package com.smartgwt.sample.server.listener;

import javax.servlet.http.HttpServletRequest;

public class T_CONFIG {
    private static T_CONFIG instance = new T_CONFIG();

    public static T_CONFIG getInstance() {
        return instance;
    }

    public boolean getVisible(String fieldName, HttpServletRequest servletRequest) {
        return true;
    }

    public boolean getVisible(String fieldName) {
        return true;
    }

    public String getVisible2(String fieldName, HttpServletRequest servletRequest) {
        return "true";
    }

    public String getVisible2(String fieldName) {
        return "true";
    }
}

If you look at the DataSourceLoader-result, you'll see a canView:false for both edited fields in the .ds.xml. I assume that when I know where to inject the template variable here, everything will be good.
It is OK that the function then will rely on user grants and on database data to return either true or false, correct?
So differing results for different users are OK, like it is the case with viewRequires, correct? (This basically means, viewRequires is an boilerplate-free version of canView?)

This is an important one for me, as a customer of mine wants to see extra fields, but I want to show those fields only there and not for other customers.

Best regards
Blama

[Isomorphic]

Your placement is correct for a fetch, but your need a subclass of DataSourceLoader that injects the variable there as well.

[Blama]

Hi Isomorphic,

can you tell me which method to override and how? The docs show only one method in DataSourceLoader.

Best regards
Blama

[Isomorphic]

It's a servlet, so any place in the normal servlet lifecycle is fine, eg, doGet(), doPost().

[Blama]

Hi Isomorphic,

sorry, still unclear. What to add here where? The addToTemplateContext() I use in my SQLDataSource-subclass' overridden executeFetch() is defined on DSRequest. I do not have a DSRequest in DataSourceLoader.

public class MyDataSourceLoader extends DataSourceLoader {
    private static final long serialVersionUID = 3994104605640773331L;

    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        super.doGet(request, response);
    }
}

Thank you & Best regards
Blama

[Isomorphic]

Yes, when contacting the DataSourceLoader, there is no dsRequest. However, other standard Velocity variables are available, including servletRequest, so you can make your object available via setAttribute().

[Blama]

Hi Isomorphic,

OK, one step closer now, but I still have a problem with executeFetch(), which does not seem to use the Velocity expression and silently defaults to false. Attached the full changes to BuiltInDS (v11.1p_2018-03-17):

animals.ds.xml:

<DataSource
  ID="animals"
    serverType="sql"
    tableName="animals"
        testFileName="animals.data.xml" [B]serverConstructor="com.smartgwt.sample.server.listener.MySQLDataSource"[/B]>
    <fields>
        <field name="commonName"      title="Animal"             type="text"/>
        <field name="scientificName"  title="Scientific Name"    type="text"  primaryKey="true"  required="true"/>
        <field name="lifeSpan"        title="Life Span"          type="integer" viewRequires="true" />
        <field name="status"          title="Endangered Status"  type="text">
            <valueMap>
                <value>Threatened</value>
                <value>Endangered</value>
                <value>Not Endangered</value>
                <value>Not currently listed</value>
                <value>May become threatened</value>
                <value>Protected</value>
            </valueMap>
        </field>
[B]       <!-- visible -->
        <field name="diet"            title="Diet"               type="text"  viewRequires="$servletRequest.getAttribute(&quot;t_config&quot;).getVisible(&quot;diet&quot;)" />
        <!-- not visible -->
        <field name="information"     title="Interesting Facts"  type="text"  viewRequires="$servletRequest.getAttribute(&quot;t_config&quot;).getVisible(&quot;information&quot;)" length="1000" />[/B]
        <field name="picture"         title="Picture"            type="image" detail="true"
               imageURLPrefix="/isomorphic/system/reference/inlineExamples/tiles/images/"/>
    </fields>
</DataSource>

T_CONFIG.java:

package com.smartgwt.sample.server.listener;

public class T_CONFIG {
    private static T_CONFIG instance = new T_CONFIG();

    public static T_CONFIG getInstance() {
        return instance;
    }

    public boolean getVisible(String fieldName) {
[B]return "diet".equals(fieldName);[/B]
    }
}

MyDataSourceLoader.java:

package com.smartgwt.sample.server.listener;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.isomorphic.servlet.DataSourceLoader;

public class MyDataSourceLoader extends DataSourceLoader {
    private static final long serialVersionUID = 616663945156810788L;

    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        request.setAttribute("t_config", T_CONFIG.getInstance()); // breakpoint hit, if set
        super.doGet(request, response);
    }
}

MySQLDataSource.java:

package com.smartgwt.sample.server.listener;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.sql.SQLDataSource;

public class MySQLDataSource extends SQLDataSource {
    private static final long serialVersionUID = -2103887355803373948L;

    @Override
    public DSResponse executeFetch(DSRequest request) throws Exception {
        request.addToTemplateContext("t_config", T_CONFIG.getInstance()); // breakpoint hit, if set
        return super.executeFetch(request);
    }
}

web.xml change:

    <servlet>
        <servlet-name>DataSourceLoader</servlet-name>
        <!-- <servlet-class>com.isomorphic.servlet.DataSourceLoader</servlet-class> -->
        <servlet-class>com.smartgwt.sample.server.listener.MyDataSourceLoader</servlet-class>
    </servlet>

This is an excerpt of what I see for http://127.0.0.1:8888/builtinds/sc/D...Source=animals (expected):

{name:"diet",columnCode:"8cc164d47468f433251653c8ab7971f1",title:"Diet",type:"text",validators:[]},
{[B]canView:false[/B],[B]name:"information"[/B],columnCode:"bb3ccd5881d651448ded1dac904054ac"}

This is the SQL if you fetch animals-DS in the GUI (unexpected):

SELECT LIMIT 0 75  animals.commonName, animals.scientificName, animals.lifeSpan, animals.status, animals.picture FROM animals WHERE ('1'='1')

As you can see both columns (diet and information) are missing in the SQL. I'd expect diet to be included. As no data is fetched here, that column is also empty in the GUI.

As a breakpoint in T_CONFIG.getVisible() is not hit for a fetch (only on startup for DataSourceLoader twice, as expected), I assume there is some issue here.
A breakpoint in MySQLDataSource.executeFetch() is hit, so the setup is correct in general.

Best regards
Blama

[Isomorphic]

executeFetch() is too late. Use addToTemplateContext() in an override of execute().

[Blama]

Hi Isomorphic,

I changed the code of MySQLDataSource.java:

package com.smartgwt.sample.server.listener;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.sql.SQLDataSource;

public class MySQLDataSource extends SQLDataSource {
    private static final long serialVersionUID = -2103887355803373948L;

    @Override
    public DSResponse execute(DSRequest request) throws Exception {
        request.addToTemplateContext("t_config", T_CONFIG.getInstance()); // breakpoint hit, if set
        return super.execute(request);
    }
}

The code is used (breakpoint hit), but still the code in T_CONFIG itself is not called when doing a fetch.

Can you have a look and also perhaps document what to do in this case somewhere, my suggestion being in viewRequires? IMHO this attribute does not make sense without own velocity variables, so it should be clear how to inject them.
In this docs, a link to SQLDataSource would also help. Additionally, in SQLDataSource itself there are not many methods documented (like executeFetch(), which is mentioned in the forums very often or any executeX()).


Thank you & Best regards
Blama

[Isomorphic]

execute(), executeFetch() etc are all documented in the superclass (as they should be - they apply across all DataSource types, not just SQLDataSource).

We'll see if we can reproduce the issue you're claiming.

[Blama]

execute(), executeFetch() etc are all documented in the superclass (as they should be - they apply across all DataSource types, not just SQLDataSource).

Sorry, I did not see this.
Then it's only explaning what most likely will be the necessary steps when one wants to use viewRequires.

We'll see if we can reproduce the issue you're claiming.

Great, thank you. It's easily reproduced with the testcase from #7 plus the change from #9.

Best regards,
Blama

[Isomorphic]

Declarative Security checks are actually run before DMI, well before DataSource.execute(). Sorry, this was obvious in retrospect. This means you'll need to inject custom context at the servlet level, with an override of IDACall.

[Blama]

Hi Isomorphic,

this is working like this. I get this SQL (expected, with diet and without information) and on FETCH my breakpoint in T_CONFIG.getVisible() is hit twice (expected). I do have a question though (see below).

SELECT LIMIT 0 75  animals.commonName, animals.scientificName, animals.lifeSpan, animals.status, animals.diet, animals.picture FROM animals WHERE ('1'='1')

package com.smartgwt.sample.server.listener;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.isomorphic.servlet.IDACall;

public class MyIDACall extends IDACall {
    private static final long serialVersionUID = -2103887355803373948L;

    /*
    @Override
    public DSResponse handleDSRequest(DSRequest arg0, RPCManager arg1, RequestContext arg2) throws Exception {
        // arg0.addToTemplateContext("t_config", T_CONFIG.getInstance()); // breakpoint hit, if set
        return super.handleDSRequest(arg0, arg1, arg2);
    }

    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        // request.setAttribute("t_config", T_CONFIG.getInstance()); // breakpoint hit, if set
        super.doGet(request, response);
    }
    */

    @Override
    public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        request.setAttribute("t_config", T_CONFIG.getInstance()); // breakpoint hit, if set
        super.doPost(request, response);
    }
}

See comments, only doPost() overwritten. Is this what you meant? Not an override of handleDSRequest()? Will this also work for queues?

Can you add this to the documentation of viewRequires? I'm pretty sure noone could figure this out on his or her own currently.

Best regards
Blama

[Isomorphic]

There are obscure cases where doGet() would also need to be overridden. However, there's no need to know this, as handleDSRequest() would also work.

[Blama]

Hi Isomorphic,

that's why I was asking. I expected it should be handleDSRequest(), but that does not work - breakpoint in T_CONFIG.getVisible() not hit on fetch, diet- and information-fields missing in SQL.

not working MyIDACall.java:

package com.smartgwt.sample.server.listener;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.rpc.RPCManager;
import com.isomorphic.servlet.IDACall;
import com.isomorphic.servlet.RequestContext;

public class MyIDACall extends IDACall {
    private static final long serialVersionUID = -2103887355803373948L;

    @Override
    public DSResponse handleDSRequest(DSRequest arg0, RPCManager arg1, RequestContext arg2) throws Exception {
        arg0.addToTemplateContext("t_config", T_CONFIG.getInstance()); // breakpoint hit, if set
        return super.handleDSRequest(arg0, arg1, arg2);
    }

    /*
    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        // request.setAttribute("t_config", T_CONFIG.getInstance()); // breakpoint hit, if set
        super.doGet(request, response);
    }

    @Override
    public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        request.setAttribute("t_config", T_CONFIG.getInstance()); // breakpoint hit, if set
        super.doPost(request, response);
    }
    */
}

Best regards
Blama