12.0p: Content-Type Request header of exportClientData missing charset-parameter

[Blama]

Hi Isomorphic,

if you compare normal IDACall requests like the one here with the exportClientData one here, you'll notice these headers in the F12 browser tools (using v12.0p_2020-06-13):

• normal:
  Content-Type: application/x-www-form-urlencoded; charset=UTF-8
• exportClientData:
  Content-Type: application/x-www-form-urlencoded

We had a web.xml change that also involved some filters. Apparently Tomcat only reads parameters once and the encoding has to be matching then. It does not help to set it afterwards (see #6 in these Tomcat docs). This change broke exportClientData for us.
According to the servlet docs having the charset=UTF-8 explicitly in the request would have solved the issue for us as well. We fixed it now like this in the web.xml with an 1st filter:

    <filter>
        <description>Tomcat Set Character Encoding filter</description>
        <filter-name>SetCharacterEncodingFilter</filter-name>
        <filter-class>org.apache.catalina.filters.SetCharacterEncodingFilter</filter-class>
        <init-param>
            <param-name>encoding</param-name>
            <param-value>UTF-8</param-value>
        </init-param>
    </filter>
    <filter-mapping>
        <filter-name>SetCharacterEncodingFilter</filter-name>
        <url-pattern>/app/sc/IDACall/*</url-pattern>
    </filter-mapping>

I assume that the header is just missing here as other requests do have it. If so, can you add it here as well?

Thank you & Best regards
Blama

[Blama]

Hi Isomorphic,

I just realized this also applies to the exportData() requests in this sample, where this leads to scrambled Umlauts in criteria.

Best regards
Blama

[Blama]

Hi Isomorphic,

can you have a look at this one?

Best regards
Blama

[Isomorphic]

Apologies for the delay, we're looking into this and will follow up soon.

[Isomorphic]

Thank you for your patience, this is fixed now and will be available for download in nightly builds since Jan 20 (today). Let us know please how it worked for you .

The form we submit to initiate the download request is part of a hidden iFrame. There's no way to set charset to the form directly, it uses the charset of the HTML page it is part of. Although on the server, if you do not access request parameters before the IDACall servlet is executed, consider using its init params to set the encoding (note that we default to UTF-8, unless you explicitly set encoding to "none"). This charset will be used in the Content-Type of the export response and this is what the fix is about, everything else is not new.

[Blama]

Hi Isomorphic,

OK. We do have our own filters mangling the requests, for example a SecurityFilter that has to be in front of IDACall.

Although on the server, if you do not access request parameters before the IDACall servlet is executed, consider using its init params to set the encoding

Some of these filters also access request parameters. I'm not sure how accessing those parameters can stop you from doing whatever you want in IDACall, can you explain?

In general, this is an old thread and we a have a working workaround, so this is fine for us now, but if your fix helps others, that's great.

There's no way to set charset to the form directly, it uses the charset of the HTML page it is part of.

I don't understand. Our main jsp file already sets

<meta http-equiv="content-type" content="text/html; charset=UTF-8">

but still the normal IDACall request is different from the download IDACall request.
The same is true for the showcase, this also has:

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>

Best regards
Blama

[Isomorphic]

Some of these filters also access request parameters. I'm not sure how accessing those parameters can stop you from doing whatever you want in IDACall, can you explain?

This means that if you rely on request params within IDACall execution scope, they will be read correctly, cause IDACall would use UTF-8 even if original request does not have it set. But this is not the case as you have some logic in front.

You're right, request issued by submitting a form does not have charset explicitly set (cause it can't), but it uses the page charset (UTF-8 in your case) to encode params into the URL. So, in your case it is safe to assume the encoding to be UTF-8 and "force" it on the server. This is what we do in IDACall and this is what you've done with the filter.

[Blama]

Hi Isomorphic,

thanks for the explanations, both make sense.

Best regards
Blama