[claudiobosticco]

We'll check on the condition - it certainly should not lead to an NPE.

Thanks in advance. Regardless of the clarification on using customSQL="true", I believe that small fix would also resolve the issue in my current implementation.

However, what do you mean those fields are "ignored"? Are you hoping to have that metadata stored in SQL by the framework, even as the binary data itself is stored elsewhere?

Using the workaround with customSelectExpression="''", when the upload comes from the browser, those metadata fields are now actually being stored.
Isn't this expected behaviour?

[Isomorphic]

So, setting customSelectExpression means you are going to handle "fetch" specially, but the system is expecting to use SQL operations for "add", "update" and "remove".

You'd got an odd hybrid here where, as we understand it, you do want the SQL storage of the metadata fields, but not the SQL storage of the binary itself.

So for that, it's correct to not use customSQL="true" (as that would remove metadata handling) but then you will need to avoid binary storage occurring by removing the stream from the values before SQLDataSource performs the "add" or "update".

[claudiobosticco]

Actually, I am only specifying customSelectExpression, and I am not using customUpdateExpression nor customInsertExpression, so the field is ignored in UPDATE/INSERT operations.

This combination seemed convenient to achieve exactly that behavior - storing only the metadata while keeping the binary field completely external.

[Isomorphic]

Hi

Before asking if you have any best practice to recommend for this scenario, I’d like to insist on one point regarding getUploadedFileStream().

it wasn't exactly "reversed," per se, but there was a logical error in that method. Thanks for letting us know. We've committed a fix for it to SmartClient 12.1 and newer releases, so if you pick up the latest nightly build, you'll have it.

[claudiobosticco]

SmartClient Version: v13.1p_2025-11-21/Enterprise Deployment (built 2025-11-21)

Thanks a lot, with the fix applied, my Custom Server DataSource implementation now works even in the case where an Add operation with a binary field is created inside a DMI.

The only remaining question I have is: are there any reasons why the following field declaration would not be recommended?

<field name="DOCUMENTO" type="binary" customSelectExpression="''">

note that there are two single quotes inside the double quotes

[Isomorphic]

There's not really anything that we can add - we've described above what that setting will do. It is neither recommended or not recommended, it just does what it says it will do.

[claudiobosticco]

SmartClient Version: v13.1p_2026-02-13/AllModules Development Only (built 2026-02-13)

Hi, I’ve now subclassed IDACall as you suggested, and the transactional aspect is working correctly - thank you again for the tip.

Hi again, thanks for the previous suggestion about subclassing IDACall, which works well when I have an RPCManager available in the servlet lifecycle.

However, I now have a follow-up question regarding cases where a transaction is created entirely server-side using DSTransaction.

Is there an officially supported pattern in SmartClient that I may use in this case? Are there APIs/hooks designed for this use case?

[Isomorphic]

There's nothing special to do here because you're in control of the execution of requests. In particular, you don't need to call DSTransaction.processQueue(), you can just dsRequest.setTransaction() and then execute() - see the second instance of using DSTransaction here:

https://smartclient.com/smartclient-...ataSourceUsage

Since you're calling execute(), and you're calling dsTransaction.complete(), you can put custom code before all requests, in between any two requests, after all requested but before committing the transaction... anywhere you want.

[claudiobosticco]

Hi, since I’m looking for a way to invoke the same cleanup logic that I currently run in an override of IDACall.processRPCTransaction(), I was wondering whether it is safe to subclass DSTransaction and override freeQueueResources() / freeAllResources() in order to trigger the appropriate cleanup at the end of the transaction lifecycle, something like:

    @Override
    public void freeQueueResources() {
        performCleanup();
        super.freeQueueResources();
    }

    @Override
    public void freeAllResources() {
        performCleanup();
        super.freeAllResources();
    }

Then, in my custom server DataSource, I would check that an instance of my subclass is used whenever there is no RPCManager.

Is subclassing DSTransaction in this way considered safe and supported?

[Isomorphic]

Those methods are doc'd as something you call, not something you override (and actually, they basically say you shouldn't need to call them).

We're not sure what's up with the structure of your code that you have you asking this, but as explained above, you already have the ability to put code at any point in the process. There is nothing unique about the timing of when these methods are called by the framework.

So, better not to override a method that is not designed as an override point, when it doesn't get you anything that you don't already have.

[claudiobosticco]

Hi, just to clarify the reasoning behind my question: I need to perform external cleanup actions (e.g., Azure Blob delete) when the transaction represented by a DSTransaction has ended, with different resources to delete depending on whether the queue ended with success or failures.

With browser-initiated queues, subclassing IDACall and using RPCManager allows me to detect the final queue status at the end of the cycle. In that scenario, my custom server DataSource stores attributes in the HttpServletRequest which will be used in the cleanup process triggered in the IDACall subclass.

However, when the queue is built and executed server-side using a DSTransaction (no HttpServletRequest, no RPCManager), I do not have a single hook where I can trigger the cleanup logic after the transaction outcome is known.

That's why I was considering overriding freeQueueResources/freeAllResources since they are always invoked: either via complete() or explicitly by my code if I use commit()/rollback(). In the DSTransaction subclass I could also store transaction-scoped attributes, similarly to how I currently use HttpServletRequest attributes in the RPCManager scenario (I noticed that DSTransaction exposes attribute-related methods, although they are not documented).
As depicted in post #24, in my performCleanup() method I would check dsTransaction.queueHasFailures() to decide which resource to delete. This would keep the application-level code symmetrical with the RPCManager scenario and would also work when using processQueue().

If the recommended approach is instead to invoke the cleanup explicitly in application code around complete() or freeQueueResources()/freeAllResources(), I can adapt to that, possibly via a wrapper abstraction.

In that case, I still have one question regarding timing:

- When using commit() / rollback(), I assume I should call queueHasFailures() immediately before invoking freeQueueResources() / freeAllResources().
- When using complete() (which internally invokes freeQueueResources() / freeAllResources()), should I call queueHasFailures() after complete() has returned?

[Isomorphic]

You've kind of lost us when you talk about doing this in "application code" vs somewhere else. If you're not talking about code from our framework, then it's all application code.

So here's the snippet from the docs that we referred to earlier:

     DSTransaction dst = new DSTransaction();
     try {
        DSRequest req1 = new DSRequest("myDataSource", "fetch");
        req1.setDsTransaction(dst);
        DSRequest req2 = new DSRequest("myDataSource", "update");
        req2.setDsTransaction(dst);

        req1.execute();
        // Use the response from req1 to modify req2 here
        req2.execute();
    } catch (Exception e) {
        throw new RuntimeException(e);
    } finally {
        // We put this in a "finally" block to ensure it always runs even on exceptions.
        dst.complete();
    }

You've got every conceivably desirable point of control here. You know when the transaction succeeded vs failed - you can call the further APIs you mentioned if you like.

It seems as if you are looking at the earlier snippet in the docs, which just shows a call to processQueue(), and wondering where to put your logic. But you don't need to call processQueue() - that's what the second snippet is showing you.

So you have total control.

[claudiobosticco]

Hi, sorry for the confusion. What I meant was the difference between the implementation of the custom server DataSource and the application code where that custom DataSource is used.

Let me try to clarify with some code snippets.

Currently I have a custom server DataSource. For example, for add operations:

public class MyCustomDataSource extends SQLDataSource {
....
    @Override
    public DSResponse executeAdd(DSRequest dsRequest) throws Exception {
        for (String fieldName : binaryFieldNames) {
            if (dsRequest.getFieldValue(fieldName) != null) {
                ....
                handleAddFile(dsRequest, fieldName);
            }
        }
        return super.executeAdd(dsRequest);
    }

    private void handleAddFile(DSRequest dsRequest, String fieldName) {
        // handle upload
        // store file reference in ServletRequest (to delete file on queue failure)
    }
....
}

Then I have an override of IDACall:

public class MyCustomIDACall extends IDACall {

    @Override
    public void processRPCTransaction(RPCManager rpc, RequestContext context) throws Exception {
        super.processRPCTransaction(rpc, context);
        cleanupStorageBasedOnStatus(rpc, context);
    }

    /**
     * Cleans up Azure storage by deleting files based on their status and
     * associated attributes stored in the HTTP request.
     */
    private void cleanupStorageBasedOnStatus(RPCManager rpc, RequestContext context) throws Exception {
        ....
    }
}

With this approach I get correct transactional behavior, both when the queue is built on the client side and when DSRequests use the RPCManager directly:

DSRequest dsRequest = new DSRequest(uploadDS, DataSource.OP_ADD, rpcManager);
dsRequest.setFieldValue(binaryFieldName, file);
dsRequest.setFieldValue(binaryFieldName + "_filename", filename);
dsRequest.setFieldValue(binaryFieldName + "_filesize", filesize);
dsRequest.setFieldValue(binaryFieldName + "_date_created", dateCreated);
dsRequest.execute();

Thanks to the extensibility of these mechanisms, using the custom server DataSource (which stores files outside the DB) is effectively identical to using a standard DataSource that stores blobs directly in the database. Which is very cool, BTW.

What I wanted to achieve was being able to use the same custom server DataSource when working with DSTransaction:

DSRequest dsRequest = new DSRequest(uploadDS, DataSource.OP_ADD, rpcManager);
dsRequest.setDSTransaction(dsTransaction);
dsRequest.setFieldValue(binaryFieldName, file);
dsRequest.setFieldValue(binaryFieldName + "_filename", filename);
dsRequest.setFieldValue(binaryFieldName + "_filesize", filesize);
dsRequest.setFieldValue(binaryFieldName + "_date_created", dateCreated);
dsRequest.execute();

Ideally, I would like to use the same patterns as with SQLDataSource:

• use commit/rollback when needed,
• freely use processQueue() or execute() for each DSRequest,
• and not have to remember to manually trigger cleanup at the correct point (although in some cases it may be desirable).

If overriding as in post #24 were supported:

@Override
public void freeQueueResources() {
    try {
        performCleanup();
    } finally {
        super.freeQueueResources();
    }
}

@Override
public void freeAllResources() {
    try {
        performCleanup();
    } finally {
        super.freeAllResources();
    }
}

I could write application code like this:

DSTransaction dsTransaction = new MyDSTransaction();
try {
    DSRequest dsRequest_1 = new DSRequest(uploadDS_1, DataSource.OP_ADD, rpcManager);
    dsRequest_1.setDSTransaction(dsTransaction);
    dsRequest_1.setFieldValue(binaryFieldName, file_1);
    ....
    dsRequest_1.execute();

    DSRequest dsRequest_2 = new DSRequest(uploadDS_2, DataSource.OP_ADD, rpcManager);
    dsRequest_2.setDSTransaction(dsTransaction);
    dsRequest_2.setFieldValue(binaryFieldName, file_2);
    ....
    dsRequest_2.execute();
} finally {
    dsTransaction.complete();
}

And in MyDSTransaction I could also store the attributes needed for cleanup.

If extending DSTransaction is not recommended, I am trying to implement this in the cleanest possible way.

So I still have at least two doubts:

1. Where could I store the attributes needed for cleanup?
   Is it acceptable to use dsTransaction.setAttribute()/getAttribute() even though they are not documented?
2. Regarding queueHasFailures(), I believe the correct pattern would be:

....
} finally {
    boolean queueHasFailures = dsTransaction.queueHasFailures();
    try {
        dsTransaction.complete();
    } finally {
        performCleanup(queueHasFailures, attributes);
    }
}

because I am not sure it is safe to call queueHasFailures() after complete().
Could you please confirm whether this is correct?

[Isomorphic]

OK, that clears things up. Answers:

1. that's correct timing for calling queueHasFailures

2. definitely do not use get/setAttribute on DSTransaction - those aren't even doc'd

3. instead of overrides on DSTransaction, you could just create a trivial wrapper class to store your additional state - something like the following, which would be easy to generalize across IDACall vs Standalone usage.

  // Wrapper to hold transaction + cleanup state
  public class TransactionContext {
      private final DSTransaction dsTransaction;
      private final List<FileReference> filesToCleanup = new ArrayList<>();

      public TransactionContext() {
          this.dsTransaction = new DSTransaction();
      }

      public DSTransaction getDSTransaction() { return dsTransaction; }
      public void addFileForCleanup(FileReference ref) { filesToCleanup.add(ref); }
      public List<FileReference> getFilesToCleanup() { return filesToCleanup; }
  }

  Then usage becomes:
  TransactionContext ctx = new TransactionContext();
  DSTransaction dsTransaction = ctx.getDSTransaction();
  try {
      // ... execute requests, call ctx.addFileForCleanup() from DataSource
      dsRequest.execute();
  } finally {
      boolean failed = dsTransaction.queueHasFailures();
      try {
          dsTransaction.complete();
      } finally {
          performCleanup(failed, ctx.getFilesToCleanup());
      }
  }