downloadClientExport with euro sign fails

[levi]

Hi,

I'm doing a grid.exportClientData but it fails since my client-data grid contains the euro sign (tried with CSV and XML export format).

The error doesn't seem to be boiling up to the client. Instead, part of the export file that did succeed is sent. Making someone think that the export was OK. Can we get the error "Not an ISO 8859-1 character" exception somehow to be passed to the client?

And how can I choose another character encoding to use?

RequestContext - rpcRequest.execute() failed: 
java.io.CharConversionException: Not an ISO 8859-1 character: €
	at javax.servlet.ServletOutputStream.print(ServletOutputStream.java:89)
	at com.isomorphic.rpc.BuiltinRPC.downloadClientExport(BuiltinRPC.java:732)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.isomorphic.base.Reflection.adaptArgsAndInvoke(Reflection.java:911)
	at com.isomorphic.base.Reflection.adaptArgsAndInvoke(Reflection.java:764)
	at com.isomorphic.rpc.RPCDMI.execute(RPCDMI.java:305)
	at com.isomorphic.rpc.RPCDMI.execute(RPCDMI.java:66)
	at com.isomorphic.rpc.RPCRequest.execute(RPCRequest.java:229)
	at com.isomorphic.servlet.IDACall.handleRPCRequest(IDACall.java:219)

thanks
SmartGWT EE SC_SNAPSHOT-2011-03-17

[Isomorphic]

You need to set the JVM charset to UTF-8. It's a command-line parameter.

[levi]

Thanks, but can you specify how and which property to set?

I tried setting -Dfile.encoding=UTF-8 or -Dfile.encoding=UTF8 (file.encoding is actually not recommended by Java Support to use) and gave me this error for the euro sign in the String:

java.io.CharConversionException: Not an ISO 8859-1 character: €
	at javax.servlet.ServletOutputStream.print(ServletOutputStream.java:89)
	at com.isomorphic.rpc.BuiltinRPC.downloadClientExport(BuiltinRPC.java:732)

is there maybe a mixup between the ServletOutputStream encoding and the JVM's?
Or is it a problem here that a ServletOutputStream iso a PrintWriter is used for textual content?

[Isomorphic]

We can't actually tell you, because it depends on your server environment. It's likely that somewhere in a configuration file that is specific to your server, encoding is being forced to that Charset instead of UTF8.

[hin3x]

May I be so bold to suggest a code improvement in your product that will resolve this issue without the need to set a property that influences the whole JVM on which the web application is running (often not allowed in hosting situations).

If the 'downloadClientExport' method in the 'BuiltInRPC' class would be using a 'PrintWriter' instead of the more low level 'ServletOutputStream', encoding would be applied automatically using the character-set as specified in the content type property (more specifically its charset sub property - which is set to UTF-8 in this case) of the servlet response.

It would be great if you would be able to include this enhancement in the final product, this can be viewed as a bug since the data is not being sent back as specified in the content type setting of the HTTP response.

Thanks for your thoughts on this.

[Isomorphic]

Not all download formats are compatible with PrintWriter, and if you are using international charsets, this is the right setting for your JVM anyway.

Regardless, because various servlet engines have really awful behavior here (eg, forcing the ISO charset even when the JVM charset is UTF-8) we've added code that forces UTF-8 unless you tell us not to (RPCManager.defaultCharset). This went in several weeks ago.

[hin3x]

Indeed, I forgot about the fact that there are other output types than the CSV I was testing with.

We updated to v3.0 SC_SNAPSHOT-2011-08-13/Pro Deployment (built 2011-08-13), is the change you refer to included there?

Thanks!

[Isomorphic]

Yes it should be.