Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
Clear All
new posts

    12.1p FileUpload serverside exception question

    Hi Isomorphic,

    I'm in the process of upgrading my 12.1p build from v12.1p_2022-08-10 to v12.1p_2025-07-29.
    All my major functions seem to work fine as before.
    But testing I found a reproducible error in my app, that is present with both framework versions. No client complained so far, so this is not a deal breaker, but I'd like to understand what is happening and if this is a bug.

    I'm using FileUpload like here, but with a few more details, which should not matter (I included a screenshot nevertheless, if you are interested). Anyway, for every upload I'm seeing 2 VALIDATE requests to my DataSource like this:
    Code:
    {
        dataSource:"T_ATTACHMENT",
        operationType:"validate",
        data:{
            LEAD_ID:211,
            ATTACHMENTTYPE_ID:3,
            NAME_TRANSLATED:"Angebote",
            DATA:"C:\\fakepath\\whatTartan.pdf",
            TITLE:"qwe"
        },
        textMatchStyle:"exact",
    ...
    ...
    Reproducible for the file used in the upload now one of these two things happens:
    1. Both requests fail, but then no error message is shown and the upload is started nevertheless and is successful. Because no error message is shown in the GUI I did not notice this so far. The error that I can see in the Developer Response Raw Response tab for the two VALIDATE requests is:
      Code:
      {
      	   affectedRows:0,
      	   errors:[
      	       {
      	           DATA_FILENAME:{
      	               severity:"ERROR",
      	               errorMessage:"Field is required"
      	           },
      	           DATA_FILESIZE:{
      	               severity:"ERROR",
      	               errorMessage:"Field is required"
      	           }
      	       }
      	   ],
      	   invalidateCache:false,
      	   isDSResponse:true,
      	   operationType:"validate",
      	   queueStatus:-1,
      	   status:-4,
      	   data:null
      	}
    2. Both requests cause a serverside exception and in the GUI an error message about broken internet connection is shown and the upload is started nevertheless (and is successful). The error that I can see in the Developer Response Raw Response tab for the two VALIDATE requests is:
      Code:
      [
      	   {
      	       response:{
      	           data:"Transport error - HTTP code: 500 for URL: https://myurl.com/lms/sc/IDACall?locale=de",
      	           status:-90
      	       }
      	   }
      	]
    So I have two problems now:
    1. Why are these validate requests sent in the first place? Possibly you need a testcase for this, or do you already have an idea?
    2. The serverside issue that is causing the exception and in turn the clientside error message. Could you look up if you can see what might be happening there? Perhaps in it is in base64Decode(DataTools.java:5436). This is the stack trace (using v12.1p_2025-07-29):
    Code:
    2025-08-04 15:55:52,581 ERROR com.lmsc.lms.serv.LMSIDACall [https-openssl-nio-443-exec-2] com.lmscompany.lms.server.LMSIDACall top-level exception
    java.lang.IllegalArgumentException: Last encoded character (before the paddings if any) is a valid base 64 alphabet but not a possible value
        at org.apache.commons.codec.binary.Base64.validateCharacter(Base64.java:798) ~[commons-codec-1.13.jar:1.13]
        at org.apache.commons.codec.binary.Base64.decode(Base64.java:472) ~[commons-codec-1.13.jar:1.13]
        at org.apache.commons.codec.binary.BaseNCodec.decode(BaseNCodec.java:412) ~[commons-codec-1.13.jar:1.13]
        at org.apache.commons.codec.binary.Base64.decodeBase64(Base64.java:708) ~[commons-codec-1.13.jar:1.13]
        at com.isomorphic.util.DataTools.base64Decode(DataTools.java:5436) ~[isomorphic_core_rpc.jar:?]
        at com.isomorphic.datasource.DSRequest.decodeUploadedStrings(DSRequest.java:969) ~[isomorphic_core_rpc.jar:?]
        at com.isomorphic.datasource.DSRequest.<init>(DSRequest.java:840) ~[isomorphic_core_rpc.jar:?]
        at com.isomorphic.rpc.RPCManager.parseRequest(RPCManager.java:2712) ~[isomorphic_core_rpc.jar:?]
        at com.isomorphic.rpc.RPCManager.<init>(RPCManager.java:484) ~[isomorphic_core_rpc.jar:?]
        at com.lmscompany.lms.server.LMSIDACall.processRequest(LMSIDACall.java:37) [classes/:?]
        at com.isomorphic.servlet.IDACall._processRequest(IDACall.java:119) [isomorphic_core_rpc.jar:?]
        at com.isomorphic.servlet.IDACall.doPost(IDACall.java:79) [isomorphic_core_rpc.jar:?]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:652) [servlet-api.jar:4.0.FR]
        at com.isomorphic.servlet.BaseServlet.service(BaseServlet.java:178) [isomorphic_core_rpc.jar:?]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) [servlet-api.jar:4.0.FR]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.37]
        at com.lmscompany.lms.server.session.SecurityFilter.doFilter(SecurityFilter.java:74) [classes/:?]
        at javax.servlet.http.HttpFilter.doFilter(HttpFilter.java:52) [servlet-api.jar:4.0.FR]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.37]
        at com.lmscompany.lms.server.datasource.LmsPersistenceUnitSelectionFilter.doFilter(LmsPersistenceUnitSelectionFilter.java:42) [classes/:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.37]
        at com.lmscompany.lms.server.ExceptionLoggingFilter.doFilter(ExceptionLoggingFilter.java:28) [classes/:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.37]
        at com.lmscompany.lms.server.RequestLoggingFilter.doFilter(RequestLoggingFilter.java:41) [classes/:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.37]
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) [tomcat-websocket.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.37]
        at com.lmscompany.lms.server.session.DeviceFilter.doFilter(DeviceFilter.java:40) [classes/:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.37]
        at org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:109) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.37]
        at com.lmscompany.lms.server.XSSProtectionFilter.doFilter(XSSProtectionFilter.java:27) [classes/:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.37]
        at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.37]
        at com.isomorphic.servlet.CompressionFilter._doFilter(CompressionFilter.java:263) [isomorphic_core_rpc.jar:?]
        at com.isomorphic.servlet.BaseFilter.doFilter(BaseFilter.java:91) [isomorphic_core_rpc.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.37]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.37]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) [catalina.jar:9.0.37]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [catalina.jar:9.0.37]
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) [catalina.jar:9.0.37]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [catalina.jar:9.0.37]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [catalina.jar:9.0.37]
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690) [catalina.jar:9.0.37]
        at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:289) [catalina.jar:9.0.37]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [catalina.jar:9.0.37]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [catalina.jar:9.0.37]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373) [tomcat-coyote.jar:9.0.37]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-coyote.jar:9.0.37]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) [tomcat-coyote.jar:9.0.37]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589) [tomcat-coyote.jar:9.0.37]
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-coyote.jar:9.0.37]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_202]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_202]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:9.0.37]
        at java.lang.Thread.run(Unknown Source) [?:1.8.0_202]
    Thank you and Best regards
    Blama
    Attached Files

    #2
    When you declare a binary field in the DataSource, two other fields are implicitly created to hold the fileSize and fileName. You can also declare them yourself.

    These look like validation errors on those fields, as if they were declared required=true, which would be a declaration that probably doesn't make sense, since the values of these fields are dynamically determined during the upload process.

    Comment


      #3
      Hi Isomorphic,

      thanks for the fast answer. You are right about the required=true. Removing it makes the two validate requests succeed, but they were still there.
      I then checked for other validators and I do have a serverCustom-validator on one of my extra fields. I removed that validator for a test and the two requests disappeared.
      So now I have two issues:
      • Why are there two requests and not one? (Minor to me, but will create a testcase)
      • What is it with the serverside exception? As the validator is there for a reason I will always have a validate request (or two). Could you have a look if you see anything suspicious about the log from #1?
      Thank you & Best regards
      Blama

      Comment


        #4
        The double validate request is likely a subtle missing performance optimization, so yes, a test case would be great.

        Then as far as the exception, if it's gone away, it's likely because we weren't expecting a validation request with a binary file (because those related fileSize / fileName fields should not be marked required), and we tried to treat it as another class of request, where binary is submitted as base64. This is probably more of a curiosity than something needing a fix, but if we had a test case for that, too, we could take a look.

        Comment


          #5
          Hi Isomorphic,

          OK, will create a testcase. The field and validator look like this. Perhaps this is already enough to give you an idea.
          Code:
          <field name="ATTACHMENTTYPE_ID" type="integer" foreignKey="T_ATTACHMENTTYPE.ID" displayField="ATTACHMENTTYPE_NAME_TRANSLATED" joinType="outer">
                      <title><fmt:message key="type" /></title>
                      <validators>
                          <validator type="serverCustom" serverOnly="true">
                              <serverObject lookupStyle="new" className="com.lmscompany.lms.server.worker.validator.ValidatorAttachmentType" />
                              <errorMessage>$errorMessage</errorMessage>
                              <applyWhen operator="and">
                                  <criteria>
                                      <criterion fieldName="LEAD_ID" operator="notNull" />
                                  </criteria>
                              </applyWhen>
                              <dependentFields>
                                  <dependentField>DATA_FILENAME</dependentField>
                              </dependentFields>
                          </validator>
                      </validators>
                  </field>
          Regarding the serverside exception this is still an issue for me, as one validate request is expected because of my validator, just not two. So if you could look into the possibly wrong base64 path, that would be great.

          I'm off for vacation on Friday, so I wont be able to create a testcase here just now, but perhaps this is already enough information?

          Best regards
          Blama

          Comment


            #6
            Hi Isomorphic,

            I was able to reproduce locally in downloaded https://smartclient.com/builds/Smart...val/2025-07-29
            I change the title field in mediaLibrary.ds.xml in the upload sample (v12.1p_2025-07-29) to this:

            Code:
            <field name="title" title="titel2">
                <validators>
                    <validator type="isUnique" serverOnly="true">
                    </validator>
                </validators>
            </field>
            Then if you try to upload this test PDF file (from here: https://www.sbr.gov.au/sites/default...20document.pdf), you'll see the following problems:
            1. operationType:"validate" request has a Transport error and the exception from #1 in the server logs
            2. Whether the validate requests ends with a Transport error, a Validation error or a success - there is always an add-Request afterwards. It's only expected in the last case.

            Best regards
            Blama
            Attached Files
            Last edited by Blama; 26 Aug 2025, 02:30.

            Comment


              #7
              Hi Isomorphic,

              for whatever reason the filename also seems to be important. It does only happen when the file is named "Test PDF document.pdf". As "Test PDF document(1).pdf" it seems not to create the the same problem, even though the content is identical (I just tried again with the file downloaded from #6 to make sure the issue will happen for you as well and Windows renamed the downloaded file).

              Best regards
              Blama

              Comment


                #8
                Originally posted by Isomorphic View Post
                The double validate request is likely a subtle missing performance optimization, so yes, a test case would be great.
                Hi Isomorphic,

                the 2nd request is most likely because the ClickHandler code for my save button looks like this for whatever reason:
                Code:
                saveBI.addClickHandler(new ClickHandler() {
                                public void onClick(ClickEvent event) {
                                    if (UploadForm.this.validate()) {
                                        UploadForm.this.saveData(new DSCallback() {
                So the only problem left here is the server side exception from #6.

                Best regards
                Blama

                Comment


                  #9
                  Hi Isomorphic,

                  could you reproduce the issue from #6?

                  Best regards
                  Blama

                  Comment

                  Working...
                  X