Problem with *servlet-started* DSRequest that is sent through declarative security

[Blama]

Hi Isomorphic,

please see this sample, based on the sample here (using recent 5.1p).

- Add lines from the other thread to web.xml to load the servlet
- Should run against HSQLDB as well
- Add this to supplyItem.ds.xml:

    <operationBindings>
        <operationBinding operationType="fetch" requiresRole="shouldNotMatter" />
    </operationBindings>

- Use this java servlet file:

package com.smartgwt.sample.server.listener;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.datasource.DataSource;
import com.isomorphic.log.Logger;

public class AddToSupplyItemServlet extends HttpServlet {
    private static final long serialVersionUID = -536025913450167992L;

    @Override
    public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        try {
            addData(request, response);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Override
    public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        doGet(request, response);
    }

    private void addData(HttpServletRequest request, HttpServletResponse response) throws Exception {
        Logger l = new Logger(com.isomorphic.sql.SQLDataSource.class);
        DSRequest dsRequest = new DSRequest("supplyItem", DataSource.OP_FETCH);
        // Comment the "setUserId"-line in order to get the sample running.
        // In my application I'm using ADD instead of FETCH and I need this for type="creator" fields.
        dsRequest.setUserId("1");
        dsRequest.setUserRoles("blubb");
        dsRequest.setCriteria("itemName", "Adding Machine Roll 57x76mm Standard");
        DSResponse dsResponse = dsRequest.execute();

        if (dsResponse.statusIsSuccess()) {
            l.warning("IS SUCCESS");
            l.warning("Total rows: " + dsResponse.getTotalRows());
        } else
            l.warning("IS ERROR");
    }
};

As written in the comment in the source code, I'm using ADD instead of FETCH in my application and I need the setUserId() for type="creator"-fields.
If I call the servlet under http://127.0.0.1:8888/AddToSupplyItemServlet, I get the following exception (for the sample, in my application I get a log-message that the user does not fulfill the requiredRoles):

Server log:

=== 2016-01-19 17:06:27,165 [7-31] INFO  RequestContext - URL: '/AddToSupplyItemServlet', User-Agent: 'Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36': Safari with Accept-Encoding header
=== 2016-01-19 17:06:27,166 [7-31] DEBUG ISCKeyedObjectPool - Borrowing object for 'supplyItem'
=== 2016-01-19 17:06:27,172 [7-31] DEBUG SQLDataSource - DataSource 15 acquired SQLDriver instance 1337635883 during initialization
=== 2016-01-19 17:06:27,174 [7-31] DEBUG PoolableDataSourceFactory - Created DataSource 15 of type 'supplyItem' and assigned it to thread qtp722333067-31
=== 2016-01-19 17:06:27,174 [7-31] DEBUG PoolableDataSourceFactory - Created DataSource 15 of type 'supplyItem' in the pooling flow
=== 2016-01-19 17:06:27,174 [7-31] DEBUG PoolableDataSourceFactory - Activated DataSource 15 of type 'supplyItem'
=== 2016-01-19 17:06:27,175 [7-31] DEBUG DSRequest - Caching instance 15 of DS 'supplyItem' from DSRequest.getDataSource()
=== 2016-01-19 17:06:27,175 [7-31] DEBUG DSRequest - Caching instance 15 of DS supplyItem
=== 2016-01-19 17:06:27,175 [7-31] DEBUG DeclarativeSecurity - Processing security checks for DataSource null, field null
=== 2016-01-19 17:06:27,177 [7-31] DEBUG DeclarativeSecurity - DataSource supplyItem is not in the pre-checked list, processing...
=== 2016-01-19 17:06:27,177 [7-31] DEBUG DSRequest - About to free up resources for request of type fetch on DataSource supplyItem
=== 2016-01-19 17:06:27,177 [7-31] DEBUG DSRequest - Ignoring freeResources call because this is not a primary request!
=== 2016-01-19 17:06:27,177 [7-31] DEBUG DSRequest - Ignoring freeQueueResources call because this is not a primary request!
java.lang.NullPointerException
    at com.isomorphic.datasource.DeclarativeSecurity.dsRequestPassesSecurityChecks(DeclarativeSecurity.java:277)
    at com.isomorphic.datasource.DSRequest.passesSecurityChecks(DSRequest.java:4406)
    at com.isomorphic.datasource.DSRequest.passesSecurityChecks(DSRequest.java:4402)
    at com.isomorphic.datasource.DSRequest.execute(DSRequest.java:2474)
    at com.smartgwt.sample.server.listener.AddToSupplyItemServlet.addData(AddToSupplyItemServlet.java:40)
    at com.smartgwt.sample.server.listener.AddToSupplyItemServlet.doGet(AddToSupplyItemServlet.java:21)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494)
    at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:260)
    at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:88)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:68)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:370)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:949)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1011)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Unknown Source)

From my understanding of the Quick Start Guide and some forum posts declarative security is something executed by IDACall only for client-requests, only.
Server initiated requests never go though declarative security, so the failing methods should not be executed.
From the QSG:

It often makes sense to create DataSources purely for server-side use—a quick idiom to make a DataSource inaccessible to browser requests is to add requires="false" to the <DataSource> tag—why this works is explained in the upcoming Declarative Security section.

Best regards
Blama


PS:
I think that the difference between server-initiated and client-initiated requests could be better documented, perhaps best in docs/StandaloneDataSourceUsage.
In the docs there should be a sentence: "Server initiated requests don't go though declarative security" IMHO, like it is here in the forum. I knew it were somewhere and found it in the QSG, only.
Also information on when <criteria>-tags are used for operationBindings is missing, like here in the forum. This becomes more important now that you explicitly document standalone DataSource usage.
If there are more differences between client- and server-initiated requests, it would be good to know about them.

[Isomorphic]

Your usage is strange here: you've got a server-initiated request, for which Declarative Security does not apply. But you've added Declarative Security settings to it. We agree these should be ignored, but if so, what's the purpose of settings them?

As far as your suggestions for more places of discussing nuances of server-initiated requests, we don't think it makes sense to add more information to the Standalone DataSource Usage doc, since both this test case and the possible confusions you talk about have nothing to do with Standalone DataSource Usage. Is there somewhere else in the docs you think it would make sense to discuss these details, aside from the existing coverage in the QuickStart Guide?

[Blama]

Hi Isomorphic,

in my application I have the ADD server-only, but FETCH and UPDATE also available from the client. My use case is:
ADD creates log entries (status=Processing) from a servlet
FETCH shows log entries from the client
UPDATE sets status=Processing entries to status=finished from a servlet (FETCHes before)
UPDATE sets status=Processing entries to status=failed from a UI.

As you can see, FETCH and UPDATE are used from client and from the server, therefore I use a operationBinding with requiresRole from a server initiated request, too.
Even if this weren't the case and I'd need the UPDATE server-only, I'd need to set requiresRole="false" in order to protect it. So declarative security on a server-only operationBinding should be very common.

Or is there a better way to solve this?

Regarding the docs: Perhaps you could put it at docs/DeclarativeSecurity and explain there which .ds.xml-settings are not used for server-initiated requests? Before someone will use such requests, he or she will most likely have used declarative security before and therefore have read the docs.

Best regards
Blama

[Isomorphic]

To restate, if you simply do not call setUserId() / setUserRoles() on your server-side DSRequest at all, it will never be subject to any Declarative Security checks. So you can declare whatever rules you want on your operationBindings to control which requests from the browser can invoke them, and your server-created DSRequests will bypass those rules, as expected.

So just remove those calls and you're done.

[Isomorphic]

Note, although you don't need these calls, we do agree there's a bug here. Your call to setUserId() marks the request as a clientRequest, so it is then subject to Declarative Security checks. The NPE in the Declarative Security code happened because it expected there to be a servletRequest available, but it should not need one - so we'll fix that.

But again, for your use case, the answer is: you don't need those calls at all.

[Blama]

Hi Isomorphic,

I do need the calls as I have database-NOT NULL fields of type="creator" and type="modifier" in my .ds.xml.

How is this best solved?
Also providing the roles would be possible with another FETCH before, but this seems unnecessary complex for a server request.

How about an API like setIsClientRequest() or setUseDeclarativeSecurity()?

Best regards
Blama

[Isomorphic]

If what you want is for the new request to be performed in the same security context as the current user, just calling setRPCManager() does that, is simpler, would include the operation in the same transaction, and would definitely not hit this obscure issue in Declarative Security. Is there any reason this approach cannot be used?

Note there is already a setClientRequest() API, but this is redundant in this case, as setUserId() already makes the request a client request, as we pointed out in #5 above.

[Blama]

Hi Isomorphic,

I can't do that, as I don't have a RPCManager. Like the title says, this happens in a servlet (hit by wget).

Would this order of calls be OK?

myRequest.setUserId("1");
myRequest.setClientRequest(false);

BaseRequest.setClientRequest() speaks about setting it to true, I want to set it to false.


Best regards
Blama

[Isomorphic]

That should work, but so should your original code (per docs), you've just hit a corner case. If you need a workaround before we fix the problem of using Declarative Security outside of the normal RPCManager flow, it's worth trying.

[Blama]

Hi Isomorphic,

like written in #1, I get the NPE only in the testcase, in my application I get the not-passed-Declarative-Security error message (so I'm not affected by the bug you mention in #5).
I'll try with setClientRequest(false) and let you know how this works for me.

Thank you & Best regards
Blama

[Blama]

Hi Isomorphic,

setting setClientRequest(false) seems to work for me, thank you.

Best regards
Blama