Should a custom DS operation really log warnings for missing fields?

[mathias]

I have a custom operation defined on a DS to fetch a JWT related to that DS. I need it to fetch images off another server.

All works fine, although i saw today that DataSource still logs warnings for "missing" datasource fields in the response, like this:

WARN com.isomorphic.datasource.DataSource - Couldn't get value at valueXPath: location/id for datasource: image - ignoring. Actual error: org.apache.commons.jxpath.JXPathNotFoundException: No pointer for xpath: //location

Naturally, all fields are missing since for that operation i'm only returning a String.

I would argue for a custom operation there should be no validation, nor any warnings logged. Or am i missing something?

[Isomorphic]

It sounds like the inputs and outputs your custom DataSource operation are so unrelated to the defining DataSource's fields that, ultimately, it ends up being inconvenient to provide values for those fields in your response.

This may indicate that the custom operation might belong in its own DataSource. If you had a separate DataSource for this custom operation, you could use that to define the valid inputs and expected outputs of that operation (including validation rules in inputs, types of outputs, etc). This operation-specific DataSource could help you define whatever form provides inputs to this custom operation, and handle errors from it.

But if this works poorly for you - like maybe there are things you need to share from the original DataSource, even if most of the fields don't apply - then give us some more information about the scenario (from an end user's perspective) and we could suggest an approach.

[mathias]

I have an database table called "images" with various metadata related to images. They are displayed in a listgrid to the end user, where the user can add remove etc. The actual image, however, it loaded from an external S3 storage, and for this i have a custom operation to fetch the JWT to put in a cookie so that it is used when calling s3 to display the images.

This is an operation that is related to images. I have crud operations and other things related to images in my image datasource. This is a custom operation totally related to images, a coder would expect to find the operation in this datasource, it belongs here logically.

---

This is your documentation regarding custom operations:

custom operations can return whatever they like, including nothing. Custom operations are like RPC calls in this respect - the exchanged data is unstructured, so it is up to you to make sure the client and server agree

I have used custom operations for a long time, and haven't noticed this logging before. It's possible it has always been there and i missed it, but in any case it doesn't make sense to log it like that given the documentation.

[Isomorphic]

Yes, custom operations are meant for exactly this (as we document).

However, this log indicates that you are asking the DataSource to extra values from an Object or XML structure, and that structure doesn't actually have the elements you declared. If you just avoid doing that, you shouldn't see this log.

It may be that there is some implicit way in which this is happening, and you're not explicitly doing it, but we would need to see actual code - ideally a runnable test case - to look into this.

[mathias]

Not sure i get what you mean - shouldn't smartgwt be totally oblivious to what i return if it is declared as a custom operation?

Here's what it looks like:

1. I have a DataSource.xml with a bunch of fields, including some defined with xpath. Specifically, you can see i have defined a "custom" operation that i am calling

<?xml version="1.0" encoding="UTF-8"?>
<DataSource ID="image" xmlns:fmt="http://www.apache.org/internal/xmlbeans/wsdlsubst">
    <fmt:bundle basename="com.nuba.i18n.ds"/>
    <fields>
        <field name="id" type="long" hidden="true" primaryKey="true"/>
        <field name="locationId" type="integer" foreignKey="location.id" valueXPath="location/id" required="true" combo="false" hidden="true"/>
        ...
    </fields>

    <serverObject lookupStyle="spring" bean="imageDS"/>
    <operationBinding operationType="custom" operationId="info">
        <serverMethod>info</serverMethod>
    </operationBinding>
</DataSource>

2. This is my spring bean "info" method which is called. I return a pojo with some custom data, not related to the fields defined in the datasource. (i also set the JWT token as a cookie so that it's sent along with image requests in the client)

@Secured(ADMIN)
public ApiResponse info(HttpServletResponse httpResp) {
    UserDetails details = getLoggedInDetails();
    setTokenCookie(details, httpResp);
    return new ApiResponse(ErrorCode.OK, imageService.getImageServiceUrl());
}[B][/B]

3. This is returned as expected and i use it in the client, everything works. smartGWT call log:

//isc_RPCResponseStart-->[{affectedRows:0,data:{code:"0",message:"http://127.0.0.1:8787/api/images/"},invalidateCache:false,isDSResponse:true,operationType:"custom",status:0}]//isc_RPCResponseEnd

4. However, the call generates smartgwt warning logs that don't understand why they pop up:

2022-08-24 14:46:04.550 WARN com.isomorphic.datasource.DataSource - Couldn't get value at valueXPath: location/id for datasource: image - ignoring. Actual error: org.apache.commons.jxpath.JXPathNotFoundException: No pointer for xpath: //location
2022-08-24 14:46:04.550 WARN com.isomorphic.datasource.DataSource - Couldn't get value at valueXPath: location/name for datasource: image - ignoring. Actual error: org.apache.commons.jxpath.JXPathNotFoundException: No pointer for xpath: //location

So, this is the issue. Not sure if it helps - but a theory might be that it's an issue with that the fields it complains about are all xpath-fields. In any case, as i said from the beginning seems like some sort of bug to me. Happy to be proven wrong :)

[Isomorphic]

As a baseline, if you explicitly called DataSource.getProperties(bean) and your DataSource had valueXPaths for properties that did not exist in the bean, you would expect these logs. It wouldn't matter if you were in a custom operation or running a command line batch job - these would be correct warnings.

Here, you are returning a bean from a custom operation that is totally unrelated to your field declarations. That's allowed, however, the default expectation is that you want your fields applied.

This is exactly what you'd want and expect, and probably rely upon elsewhere in your app, perhaps without realizing.

And it also works fine even in the case that you do return some data that is totally unrelated to your fields - it works here, there are just some harmless warnings.

But if you want to get rid of the warnings, you can either:

1. turn off field processing for this DSResponse via bypassDataFilter

https://smartclient.com/smartgwtee-l....lang.Boolean-

OR

2. set dsResponse.outputs() to omit fields that have a valueXPath that doesn't apply to the returned object

[mathias]

I know how it works, yes, and i return beans all over the place. I realize that that's how the smartgwt-formatted response is generated, of course. That's what all mvc frameworks do. I didn't expect the same thing to happen for a custom operation after reading the documentation - again, "the exchanged data is unstructured, so it is up to you to make sure the client and server agree"

Hmmm, both suggested workarounds involves doing things with the dsresponse. I try to share my web-mvc classes between smartgwt and other api's we have so i usually try to avoid using those directly, but i guess i'll have to make it work somehow. There is nothing that can be configured in the datasource xml?

Anyway, thanks for explaining.

[Isomorphic]

Yes, the documentation is correct exactly as you have quoted it: "the exchanged data is unstructured, so it is up to you to make sure the client and server agree". And that is working for you and is exactly what your are experiencing.

As far as the warning, which again is a correct warning, and has no effect on functionality, we have just provided two ways to avoid it, both of which are possible in .ds.xml (the first via Server Scripting, the second via setting outputs on the operationBinding).

[mathias]

ok. thanks for claryfing further, ill do it in the xml then. I still don't think those warnings should be output on a "custom" operation, but i hope we still can get along :)

[mathias]

Hi, i changed my operationbinding config in the xml to:

<operationBinding operationType="custom" operationId="info" outputs="code, message">
        <!-- called when page loads, fetches image related info -->
        <serverMethod>info</serverMethod>
    </operationBinding>

But it still logs warnings about the missing fields as above. Am i doing it wrong?

[Isomorphic]

Apologies for not replying earlier. We've made a bunch of changes in this area, so the reported warning should be long gone by now.