SGWTEE Realtime Messaging Unicode Problem

[paatal]

hi all ,
Unicode doesn't work in real-time messaging .
Is this Bug or did i miss something ???
Here is screenshot from SGWTEE showcase -> Simple Chat.


____________________________
Regards,
Paata Lominadze.

[paatal]

also I tried to write my own servlet.
but the result is the same.
I have Listgrid filled from server which contains unicode data.
I took example from SGWTEE Showcase -> Simulated Stock Quotes.
here is my program screenshot :

Env :
1. SGWTEE 2.5 (Evaluation)
2. GWT 2.3.0
3. Firefox 5.
4. Kubuntu
5. jdk1.6.0_26
6. Oracle DB 11g

_______________________
Regards,
Paata Lominadze

[Isomorphic]

What's your exact version? 2.5 is not exact, please always post the exact build date. Someone else reported this problem with an older nightly (we didn't reproduce it) but then reported it fixed with a more recent nightly.

[paatal]

What's your exact version? 2.5 is not exact, please always post the exact build date. Someone else reported this problem with an older nightly (we didn't reproduce it) but then reported it fixed with a more recent nightly.

Build Date : 2011-07-19


_______________________
Regards,
Paata Lominadze

[paatal]

Hi again,
Also I have updated version to : SGWTEE 2.5 build Date - 2011-08-02

But the result is the same.

Unicode doesn't work.

Also I tried to set content type and Character Encoding into my servlet :

public void processRequest(HttpServletRequest request,
			HttpServletResponse response) throws ServletException, IOException {
		response.setCharacterEncoding("UTF-8");
		request.setCharacterEncoding("UTF-8");
		//response.setContentType("text/html;charset=UTF-8");
                response.setContentType("text/plain;charset=UTF-8");
		ClientMessageSender.startListening();
	}

But no success.



_______________________
Regards.
Paata Lominadze.

[Isomorphic]

Use Firebug to check whether you've successfully set the character encoding (look at the HTTP headers received by the browser). Probably, you have some filter servlet or something else involved which is overriding your attempts to set the encoding.

[paatal]

Use Firebug to check whether you've successfully set the character encoding (look at the HTTP headers received by the browser). Probably, you have some filter servlet or something else involved which is overriding your attempts to set the encoding.

Hi Isomorphic.
Here is my screenshot from firebug :
Also this is my web.xml :

<?xml version="1.0"?>
<web-app version="2.4" xmlns="http://java.sun.com/xml/ns/j2ee"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">
	<!-- //>FileLoader Dynamic Compression -->
	<filter>
		<filter-name>CompressionFilter</filter-name>
		<filter-class>com.isomorphic.servlet.CompressionFilter</filter-class>
	</filter>
	<!-- CompressionFilter for dynamic compression -->
	<filter-mapping>
		<filter-name>CompressionFilter</filter-name>
		<url-pattern>/*</url-pattern>
	</filter-mapping>
	<!-- //<FileLoader Servlets -->
	<servlet>
		<servlet-name>IDACall</servlet-name>
		<servlet-class>com.isomorphic.servlet.IDACall</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>DataSourceLoader</servlet-name>
		<servlet-class>com.isomorphic.servlet.DataSourceLoader</servlet-class>
	</servlet>
	<!-- The FileDownload servlet downloads static files, like a webserver -->
	<servlet>
		<servlet-name>FileDownload</servlet-name>
		<servlet-class>com.isomorphic.servlet.FileDownload</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>MessagingServlet</servlet-name>
		<servlet-class>com.isomorphic.messaging.MessagingServlet</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>commonServiceServlet</servlet-name>
		<servlet-class>com.info08.billing.callcenter.server.impl.CommonServiceImpl</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>callCenterServlet</servlet-name>
		<servlet-class>com.info08.billing.callcenter.server.listener.CallCenterServlet</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>discoveryServlet</servlet-name>
		<servlet-class>com.info08.billing.callcenter.server.listener.DiscoveryServlet</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>Init</servlet-name>
		<servlet-class>com.isomorphic.base.Init</servlet-class>
		<load-on-startup>1</load-on-startup>
	</servlet>
	<servlet>
		<servlet-name>HttpProxy</servlet-name>
		<servlet-class>com.isomorphic.servlet.HttpProxyServlet</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>PreCache</servlet-name>
		<servlet-class>com.isomorphic.servlet.PreCache</servlet-class>
		<load-on-startup>2</load-on-startup>
	</servlet>
	<!-- RPCManager uses this URL by default for Built-in DataSource operations -->
	<servlet-mapping>
		<servlet-name>IDACall</servlet-name>
		<url-pattern>/callcenter/sc/IDACall/*</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>IDACall</servlet-name>
		<url-pattern>/callcenter/sc/IDACall.aspx/*</url-pattern>
	</servlet-mapping>
	<!-- DataSourceLoader requests -->
	<servlet-mapping>
		<servlet-name>DataSourceLoader</servlet-name>
		<url-pattern>/callcenter/sc/DataSourceLoader</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>HttpProxy</servlet-name>
		<url-pattern>/callcenter/sc/HttpProxy/*</url-pattern>
	</servlet-mapping>
	<!-- Use FileDownload servlet to download all static content that's part 
		of the skin, such as image files, so we can set Expires headers and other 
		cache control directives. In a production deployment, you'd want to use a 
		webserver such as Apache to do this... -->
	<servlet-mapping>
		<servlet-name>FileDownload</servlet-name>
		<url-pattern>/callcenter/sc/skins/*</url-pattern>
	</servlet-mapping>
	<!-- serve ISC modules compressed, with expires headers -->
	<servlet-mapping>
		<servlet-name>FileDownload</servlet-name>
		<url-pattern>/callcenter/sc/system/modules/*</url-pattern>
	</servlet-mapping>
	<!-- serve ISC development modules compressed, with expires headers -->
	<servlet-mapping>
		<servlet-name>FileDownload</servlet-name>
		<url-pattern>/callcenter/sc/system/development/*</url-pattern>
	</servlet-mapping>
	<!-- server skin assets with expires headers -->
	<servlet-mapping>
		<servlet-name>FileDownload</servlet-name>
		<url-pattern>/callcenter/sc/system/reference/skin/*</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>MessagingServlet</servlet-name>
		<url-pattern>/callcenter/sc/messaging/*</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>commonServiceServlet</servlet-name>
		<url-pattern>/callcenter/commonService</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>callCenterServlet</servlet-name>
		<url-pattern>/callcenter/callCenterServlet/*</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>discoveryServlet</servlet-name>
		<url-pattern>/callcenter/discoveryServlet/*</url-pattern>
	</servlet-mapping>
	<!-- General config -->
	<session-config>
		<session-timeout>30</session-timeout>
	</session-config>
	<mime-mapping>
		<extension>manifest</extension>
		<mime-type>text/cache-manifest</mime-type>
	</mime-mapping>
	<!-- Default page to serve -->
	<welcome-file-list>
		<welcome-file>CallCenter.html</welcome-file>
	</welcome-file-list>
	<jsp-config>
		<!-- Isomorphic JSP tags -->
		<taglib>
			<taglib-uri>isomorphic</taglib-uri>
			<taglib-location>/WEB-INF/iscTaglib.xml</taglib-location>
		</taglib>
	</jsp-config>
</web-app>

I don't know how to override this "text/html; charset=iso-8859-1" encoding.

Also I have only one servlet filter "CompressionFilter" into my application and without this servlet I got the same problem ? I see only "?" symbols instead of unicode .





Regards,
Paata Lominadze.

[Isomorphic]

Your attempts seem correct (see this Tomcat FAQ but again you may be making the call too late or have a setting elsewhere thats overriding.

[paatal]

Your attempts seem correct (see this Tomcat FAQ but again you may be making the call too late or have a setting elsewhere thats overriding.

Hello Isomorphic,
I've no success with real-time messaging and unicode yet.

I read this doc :
Tomcat FAQ

and tried to do everything what is described there :

1 . Set URIEncoding="UTF-8" on your <Connector> in server.xml. References: HTTP Connector, AJP Connector.

2. Use a character encoding filter with the default encoding set to UTF-8

3. Change all your JSPs to include charset name in their contentType.
For example, use <%@page contentType="text/html; charset=UTF-8" %> for the usual JSP pages and <jsp:directive.page contentType="text/html; charset=UTF-8" /> for the pages in XML syntax (aka JSP Documents).

4 . Change all your servlets to set the content type for responses and to include charset name in the content type to be UTF-8.
Use response.setContentType("text/html; charset=UTF-8") or response.setCharacterEncoding("UTF-8").


Change any content-generation libraries you use (Velocity, Freemarker, etc.) to use UTF-8 and to specify UTF-8 in the content type of the responses that they generate.

Disable any valves or filters that may read request parameters before your character encoding filter or jsp page has a chance to set the encoding to UTF-8. For more information see http://www.mail-archive.com/users@tomcat.apache.org/msg21117.html.

1. My Tomcat (6.0.32) server.xml looks like :

<Connector URIEncoding="UTF-8" port="8080" protocol="HTTP/1.1"                    
connectionTimeout="20000" redirectPort="8443" />


<Connector URIEncoding="UTF-8" port="8009" protocol="AJP/1.3" redirectPort="8443" />

2. Also I've added character encoding filter into my application :
2.1 : web.xml :

<!--  Setting Character Encoding -->
	<filter>
		<filter-name>Set Character Encoding</filter-name>
		<filter-class>com.info08.billing.callcenter.server.servlets.SetCharacterEncodingFilter</filter-class>
		<init-param>
			<param-name>encoding</param-name>
			<param-value>UTF-8</param-value>
		</init-param>
		<init-param>
			<param-name>forceEncoding</param-name>
			<param-value>true</param-value>
		</init-param>
	</filter>
	<filter-mapping>
		<filter-name>Set Character Encoding</filter-name>
		<url-pattern>/*</url-pattern>
	</filter-mapping>

2.2 SetCharacterEncodingFilter class (I took it from tomcat)

public class SetCharacterEncodingFilter implements Filter {
	protected String encoding = null;
	protected FilterConfig filterConfig = null;
	protected boolean ignore = true;
	public void destroy() {
		this.encoding = null;
		this.filterConfig = null;
	}
	public void doFilter(ServletRequest request, ServletResponse response,
			FilterChain chain) throws IOException, ServletException {
		// Conditionally select and set the character encoding to be used
		if (ignore || (request.getCharacterEncoding() == null)) {
			String encoding = selectEncoding(request);
			if (encoding != null)
				request.setCharacterEncoding(encoding);
		}
		// Pass control on to the next filter
		chain.doFilter(request, response);
	}
	public void init(FilterConfig filterConfig) throws ServletException {
		this.filterConfig = filterConfig;
		this.encoding = filterConfig.getInitParameter("encoding");
		String value = filterConfig.getInitParameter("ignore");
		if (value == null)
			this.ignore = true;
		else if (value.equalsIgnoreCase("true"))
			this.ignore = true;
		else if (value.equalsIgnoreCase("yes"))
			this.ignore = true;
		else
			this.ignore = false;
	}
	protected String selectEncoding(ServletRequest request) {
		return (this.encoding);
	}
}

3. and what about this section : I have no any jps into my application.

4. My Servlet for sending messages to the clients :

public class DiscoveryServlet extends HttpServlet {

	private static final long serialVersionUID = 6200724895227971697L;

	public void doPost(HttpServletRequest request, HttpServletResponse response)
			throws ServletException, IOException {
		processRequest(request, response);
	}

	// RPC transactions may be encoded as a GET request, so handle those as well
	public void doGet(HttpServletRequest request, HttpServletResponse response)
			throws ServletException, IOException {
		processRequest(request, response);
	}

	public void processRequest(HttpServletRequest request,
			HttpServletResponse response) throws ServletException, IOException {		
		response.setCharacterEncoding("UTF-8");
		request.setCharacterEncoding("UTF-8");
		response.setContentType("text/html; charset=UTF-8");
		ClientMessageSender.startListening();
	}
}

and client code which receives data :

public class TabDiscovery extends Tab {

	private VLayout mainLayout;

	private ToolStripButton addBtn;
	private ToolStripButton editBtn;
	private ToolStripButton disableBtn;
	private ToolStripButton activateBtn;

	// ListGrid
	private ListGrid listGrid;

	// DataSource
	protected DataSource datasource;

	public TabDiscovery() {
		try {
			setTitle(CallCenter.constants.menuMisc());
			setCanClose(true);

			datasource = DataSource.get("DiscoveryDS");

			mainLayout = new VLayout(5);
			mainLayout.setWidth100();
			mainLayout.setHeight100();
			mainLayout.setMargin(5);

			ToolStrip toolStrip = new ToolStrip();
			toolStrip.setWidth(780);
			toolStrip.setPadding(5);
			mainLayout.addMember(toolStrip);

			addBtn = new ToolStripButton(CallCenter.constants.add(),
					"addIcon.png");
			addBtn.setLayoutAlign(Alignment.LEFT);
			addBtn.setWidth(50);
			toolStrip.addButton(addBtn);

			editBtn = new ToolStripButton(CallCenter.constants.modify(),
					"editIcon.png");
			editBtn.setLayoutAlign(Alignment.LEFT);
			editBtn.setWidth(50);
			toolStrip.addButton(editBtn);

			disableBtn = new ToolStripButton(CallCenter.constants.disable(),
					"deleteIcon.png");
			disableBtn.setLayoutAlign(Alignment.LEFT);
			disableBtn.setWidth(50);
			toolStrip.addButton(disableBtn);

			activateBtn = new ToolStripButton(CallCenter.constants.enable(),
					"restoreIcon.gif");
			activateBtn.setLayoutAlign(Alignment.LEFT);
			activateBtn.setWidth(50);
			toolStrip.addButton(activateBtn);

			toolStrip.addSeparator();

			listGrid = new ListGrid() {
				protected String getCellCSSText(ListGridRecord record,
						int rowNum, int colNum) {
					ListGridRecord countryRecord = (ListGridRecord) record;
					if (countryRecord == null) {
						return super.getCellCSSText(record, rowNum, colNum);
					}
					Integer deleted = countryRecord
							.getAttributeAsInt("deleted");
					if (deleted != null && !deleted.equals(0)) {
						return "color:red;";
					} else {
						return super.getCellCSSText(record, rowNum, colNum);
					}
				};
			};

			DiscoveryClientDS discoveryClientDS = DiscoveryClientDS
					.getInstance();
			listGrid.setWidth(780);
			listGrid.setHeight(320);
			listGrid.setAlternateRecordStyles(true);
			listGrid.setDataSource(discoveryClientDS);
			listGrid.setAutoFetchData(true);
			listGrid.setShowFilterEditor(false);
			listGrid.setCanEdit(false);
			listGrid.setCanRemoveRecords(false);
			listGrid.setShowRowNumbers(true);
			listGrid.setCanHover(true);
			listGrid.setShowHover(true);
			listGrid.setShowHoverComponents(true);
			listGrid.setUseAllDataSourceFields(true);
			listGrid.setShowAllRecords(true);
			mainLayout.addMember(listGrid);

			TabSet tabSet = new TabSet();
			tabSet.setWidth(780);
			Tab tabDetViewer = new Tab(CallCenter.constants.view());
			final DetailViewer detailViewer = new DetailViewer();
			detailViewer.setDataSource(discoveryClientDS);
			detailViewer.setWidth(750);
			tabDetViewer.setPane(detailViewer);

			listGrid.addRecordClickHandler(new RecordClickHandler() {
				public void onRecordClick(RecordClickEvent event) {
					detailViewer.viewSelectedData(listGrid);
				}
			});

			listGrid.addRecordDoubleClickHandler(new RecordDoubleClickHandler() {
				@Override
				public void onRecordDoubleClick(RecordDoubleClickEvent event) {
					ListGridRecord listGridRecord = listGrid
							.getSelectedRecord();
					if (listGridRecord == null) {
						SC.say(CallCenter.constants.pleaseSelrecord());
						return;
					}
					DlgAddEditNonStandartInfo dlgAddEditNonStandartInfo = new DlgAddEditNonStandartInfo(
							listGrid, listGridRecord);
					dlgAddEditNonStandartInfo.show();
				}
			});

			tabSet.setTabs(tabDetViewer);
			mainLayout.addMember(tabSet);
			setPane(mainLayout);
			start();
		} catch (Exception e) {
			e.printStackTrace();
			SC.say(e.getMessage().toString());
		}
	}

	public void start() {
		try {
			boolean flag = true;
			if (!flag) {
				return;
			}
			Messaging.subscribe("discoveryServlet", new MessagingCallback() {
				@Override
				public void execute(Object data) {
					updateCallsLog(data);
				}
			});
			RPCRequest request = new RPCRequest();
			request.setContentType("text/html; charset=UTF-8");
			request.setActionURL("callcenter/discoveryServlet");
			RPCManager.sendRequest(request);
		} catch (Exception e) {
			e.printStackTrace();
			SC.say(e.toString());
		}
	}

	@SuppressWarnings("unchecked")
	private void updateCallsLog(Object data) {
		try {
			List<LinkedHashMap<?, ?>> callsData = (List<LinkedHashMap<?, ?>>) JSOHelper
					.convertToJava((JavaScriptObject) data);
			List<Record> newData = new ArrayList<Record>();
			for (LinkedHashMap<?, ?> recordData : callsData) {
				Integer discover_id = (Integer) recordData.get("discover_id");
				String call_id = (String) recordData.get("call_id");
				Integer ccr = (Integer) recordData.get("ccr");
				Integer deleted = (Integer) recordData.get("deleted");
				Integer discover_type_id = (Integer) recordData
						.get("discover_type_id");
				Integer execution_status = (Integer) recordData
						.get("execution_status");
				Integer iscorrect = (Integer) recordData.get("iscorrect");
				Integer response_type_id = (Integer) recordData
						.get("response_type_id");
				Date upd_date = (Date) recordData.get("upd_date");
				String upd_user = (String) recordData.get("upd_user");
				String discover_type = (String) recordData.get("discover_type");
				String phone = (String) recordData.get("phone");
				String contact_phone = (String) recordData.get("contact_phone");
				String contact_person = (String) recordData
						.get("contact_person");
				String discover_txt = (String) recordData.get("discover_txt");
				String rec_user = (String) recordData.get("rec_user");
				Date rec_date = (Date) recordData.get("rec_date");
				String status_descr = (String) recordData.get("status_descr");
				Record record = listGrid.getDataAsRecordList().find(
						"discover_id", discover_id);
				if (record == null) {
					record = new Record();
				}
				record.setAttribute("discover_id", discover_id);
				record.setAttribute("call_id", call_id);
				record.setAttribute("ccr", ccr);
				record.setAttribute("deleted", deleted);
				record.setAttribute("discover_type_id", discover_type_id);
				record.setAttribute("execution_status", execution_status);
				record.setAttribute("iscorrect", iscorrect);
				record.setAttribute("response_type_id", response_type_id);
				record.setAttribute("upd_date", upd_date);
				record.setAttribute("discover_type", discover_type);
				record.setAttribute("phone", phone);
				record.setAttribute("contact_phone", contact_phone);
				record.setAttribute("contact_person", contact_person);
				record.setAttribute("discover_txt", discover_txt);
				record.setAttribute("rec_user", rec_user);
				record.setAttribute("rec_date", rec_date);
				record.setAttribute("status_descr", status_descr);
				record.setAttribute("upd_user", upd_user);
				newData.add(record);
			}
			DSResponse dsResponse = new DSResponse();
			dsResponse.setData((Record[]) newData.toArray(new Record[newData
					.size()]));

			DSRequest dsRequest = new DSRequest();
			dsRequest.setContentType("text/html; charset=UTF-8");
			dsRequest.setOperationType(DSOperationType.UPDATE);
			DiscoveryClientDS.getInstance().updateCaches(dsResponse, dsRequest);
		} catch (Exception e) {
			e.printStackTrace();
			SC.say(e.toString());
		}
	}
}

And At least My full web.xml :

<?xml version="1.0"?>
<web-app version="2.4" xmlns="http://java.sun.com/xml/ns/j2ee"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">

	<!--  Setting Character Encoding -->
	<filter>
		<filter-name>Set Character Encoding</filter-name>
		<filter-class>com.info08.billing.callcenter.server.servlets.SetCharacterEncodingFilter</filter-class>
		<init-param>
			<param-name>encoding</param-name>
			<param-value>UTF-8</param-value>
		</init-param>
		<init-param>
			<param-name>forceEncoding</param-name>
			<param-value>true</param-value>
		</init-param>
	</filter>
	<filter-mapping>
		<filter-name>Set Character Encoding</filter-name>
		<url-pattern>/*</url-pattern>
	</filter-mapping>

	<!-- //>FileLoader Dynamic Compression -->
	<filter>
		<filter-name>CompressionFilter</filter-name>
		<filter-class>com.isomorphic.servlet.CompressionFilter</filter-class>
	</filter>
	<!-- CompressionFilter for dynamic compression -->
	<filter-mapping>
		<filter-name>CompressionFilter</filter-name>
		<url-pattern>/*</url-pattern>
	</filter-mapping>
	<!-- //<FileLoader Servlets -->
	<servlet>
		<servlet-name>IDACall</servlet-name>
		<servlet-class>com.isomorphic.servlet.IDACall</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>DataSourceLoader</servlet-name>
		<servlet-class>com.isomorphic.servlet.DataSourceLoader</servlet-class>
	</servlet>
	<!-- The FileDownload servlet downloads static files, like a webserver -->
	<servlet>
		<servlet-name>FileDownload</servlet-name>
		<servlet-class>com.isomorphic.servlet.FileDownload</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>MessagingServlet</servlet-name>
		<servlet-class>com.isomorphic.messaging.MessagingServlet</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>commonServiceServlet</servlet-name>
		<servlet-class>com.info08.billing.callcenter.server.impl.CommonServiceImpl</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>callCenterServlet</servlet-name>
		<servlet-class>com.info08.billing.callcenter.server.listener.CallCenterServlet</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>discoveryServlet</servlet-name>
		<servlet-class>com.info08.billing.callcenter.server.listener.DiscoveryServlet</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>Init</servlet-name>
		<servlet-class>com.isomorphic.base.Init</servlet-class>
		<load-on-startup>1</load-on-startup>
	</servlet>
	<servlet>
		<servlet-name>HttpProxy</servlet-name>
		<servlet-class>com.isomorphic.servlet.HttpProxyServlet</servlet-class>
	</servlet>
	<servlet>
		<servlet-name>PreCache</servlet-name>
		<servlet-class>com.isomorphic.servlet.PreCache</servlet-class>
		<load-on-startup>2</load-on-startup>
	</servlet>
	<!-- RPCManager uses this URL by default for Built-in DataSource operations -->
	<servlet-mapping>
		<servlet-name>IDACall</servlet-name>
		<url-pattern>/callcenter/sc/IDACall/*</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>IDACall</servlet-name>
		<url-pattern>/callcenter/sc/IDACall.aspx/*</url-pattern>
	</servlet-mapping>
	<!-- DataSourceLoader requests -->
	<servlet-mapping>
		<servlet-name>DataSourceLoader</servlet-name>
		<url-pattern>/callcenter/sc/DataSourceLoader</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>HttpProxy</servlet-name>
		<url-pattern>/callcenter/sc/HttpProxy/*</url-pattern>
	</servlet-mapping>
	<!-- Use FileDownload servlet to download all static content that's part 
		of the skin, such as image files, so we can set Expires headers and other 
		cache control directives. In a production deployment, you'd want to use a 
		webserver such as Apache to do this... -->
	<servlet-mapping>
		<servlet-name>FileDownload</servlet-name>
		<url-pattern>/callcenter/sc/skins/*</url-pattern>
	</servlet-mapping>
	<!-- serve ISC modules compressed, with expires headers -->
	<servlet-mapping>
		<servlet-name>FileDownload</servlet-name>
		<url-pattern>/callcenter/sc/system/modules/*</url-pattern>
	</servlet-mapping>
	<!-- serve ISC development modules compressed, with expires headers -->
	<servlet-mapping>
		<servlet-name>FileDownload</servlet-name>
		<url-pattern>/callcenter/sc/system/development/*</url-pattern>
	</servlet-mapping>
	<!-- server skin assets with expires headers -->
	<servlet-mapping>
		<servlet-name>FileDownload</servlet-name>
		<url-pattern>/callcenter/sc/system/reference/skin/*</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>MessagingServlet</servlet-name>
		<url-pattern>/callcenter/sc/messaging/*</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>commonServiceServlet</servlet-name>
		<url-pattern>/callcenter/commonService</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>callCenterServlet</servlet-name>
		<url-pattern>/callcenter/callCenterServlet/*</url-pattern>
	</servlet-mapping>
	<servlet-mapping>
		<servlet-name>discoveryServlet</servlet-name>
		<url-pattern>/callcenter/discoveryServlet/*</url-pattern>
	</servlet-mapping>
	<!-- General config -->
	<session-config>
		<session-timeout>30</session-timeout>
	</session-config>
	<mime-mapping>
		<extension>manifest</extension>
		<mime-type>text/cache-manifest</mime-type>
	</mime-mapping>
	<!-- Default page to serve -->
	<welcome-file-list>
		<welcome-file>CallCenter.html</welcome-file>
	</welcome-file-list>
	<jsp-config>
		<!-- Isomorphic JSP tags -->
		<taglib>
			<taglib-uri>isomorphic</taglib-uri>
			<taglib-location>/WEB-INF/iscTaglib.xml</taglib-location>
		</taglib>
	</jsp-config>
</web-app>

But till now I have not any success, I see '?' chars insteadof unicode character.


Any Idea ?
What I did incorrect or did i miss something ?


Here is screenshot also from firebug.

What can I do to resolve this problem ??


Env :
1. SGWTEE 2.5 (Evaluation) - build Date - 2011-08-02
2. GWT 2.3.0
3. Firefox 5.
4. OpenSuse 11.4
5. jdk1.6.0_26
6. Oracle DB 11g
7. Eclipse Indigo.

_____________________________
Regards,
Paata Lominadze

[paatal]

Any idea will be appreciated.

[paatal]

Hello Team.
Problem with unicode and real-time messaging resolved.
Here is my example which works correctly.
Maybe this example will be helpful for someone :)


Env :
1. SGWTEE 2.5 (Evaluation) - build Date - 2011-08-02
2. GWT 2.3.0
3. Firefox 5.
4. OpenSuse 11.4
5. jdk1.6.0_26
6. Oracle DB 11g
7. Eclipse Indigo (With GWT Plugin.)


1. Tomcat (Version 6.0.32) configuration . $TOMCAT_HOME/conf/server.xml

<Connector URIEncoding="UTF-8" port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" />

<Connector URIEncoding="UTF-8" port="8009" protocol="AJP/1.3" redirectPort="8443" />

2. Application web.xml configuration (Filter Servlet).

<!--  Setting Character Encoding -->
	<filter>
		<filter-name>CharsetFilter</filter-name>
		<filter-class>com.info08.billing.callcenter.server.servlets.SetCharacterEncodingFilter</filter-class>
		<init-param>
			<param-name>encoding</param-name>
			<param-value>UTF-8</param-value>
		</init-param>
	</filter>
	<filter-mapping>
		<filter-name>CharsetFilter</filter-name>
		<url-pattern>/*</url-pattern>
	</filter-mapping>

3. Character Encoder Filter Class

public class SetCharacterEncodingFilter implements Filter {
	private String encoding;
	public void init(FilterConfig config) throws ServletException {
		encoding = config.getInitParameter("requestEncoding");
		if (encoding == null) {
			encoding = "UTF-8";
		}
	}
	public void doFilter(ServletRequest request, ServletResponse response,
			FilterChain next) throws IOException, ServletException {
		if (null == request.getCharacterEncoding()) {
			request.setCharacterEncoding(encoding);
		}
		response.setContentType("text/html; charset=UTF-8");
		response.setCharacterEncoding("UTF-8");
		
		next.doFilter(request, response);
	}
	public void destroy() {
	}
}

4. My Servlet which sends Data to the client :

public class DiscoveryServlet extends HttpServlet {

	private static final long serialVersionUID = 6200724895227971697L;

	public void doPost(HttpServletRequest request, HttpServletResponse response)
			throws ServletException, IOException {
		processRequest(request, response);
	}

	// RPC transactions may be encoded as a GET request, so handle those as well
	public void doGet(HttpServletRequest request, HttpServletResponse response)
			throws ServletException, IOException {
		processRequest(request, response);
	}

	public void processRequest(HttpServletRequest request,
			HttpServletResponse response) throws ServletException, IOException {
		
		
		response.setContentType("text/html; charset=UTF-8");
		response.setHeader("Cache-Control", "private");
		response.setHeader("Pragma", "no-cache");
		
		request.setCharacterEncoding("UTF-8");
		response.setCharacterEncoding("UTF-8");
		
		new Thread() {
			@SuppressWarnings("rawtypes")
			public void run() {

				EntityManager oracleManager = null;
				try {
					ISCMessageDispatcher dispatcher = ISCMessageDispatcher
							.instance();
					while (true) {
						try {
							oracleManager = EMF.getEntityManager();
							List list = oracleManager.createNativeQuery(
									QueryConstants.Q_GET_DISCOVERY_LIST)
									.getResultList();
							ArrayList<Discover> listData = new ArrayList<Discover>();
							if (list != null && !list.isEmpty()) {
								for (Object object : list) {
									Object columns[] = (Object[]) object;
									Discover discover = new Discover();
									// fill discover class ....
									// discover.setSomefielf()
									listData.add(discover);
								}
								ISCMessage iscMessage = new ISCMessage(
										"discoveryServlet", listData);
								dispatcher.send(iscMessage);
							}
							Thread.sleep(10000);
						} catch (Exception e) {
							e.printStackTrace();
						} finally {
							if (oracleManager != null) {
								try {
									EMF.returnEntityManager(oracleManager);
								} catch (Exception e2) {
									e2.printStackTrace();
								}
							}
						}
					}

				} catch (Exception e) {
					e.printStackTrace();
				}
			};
		}.start();
	}
}

5. And the client code which displays these data :

public class TabDiscovery extends Tab {

	private VLayout mainLayout;

	private ToolStripButton addBtn;
	private ToolStripButton editBtn;
	private ToolStripButton disableBtn;
	private ToolStripButton activateBtn;

	// ListGrid
	private ListGrid listGrid;

	// DataSource
	protected DataSource datasource;

	public TabDiscovery() {
		try {
			setTitle(CallCenter.constants.menuMisc());
			setCanClose(true);

			datasource = DataSource.get("DiscoveryDS");

			mainLayout = new VLayout(5);
			mainLayout.setWidth100();
			mainLayout.setHeight100();
			mainLayout.setMargin(5);

			ToolStrip toolStrip = new ToolStrip();
			toolStrip.setWidth(780);
			toolStrip.setPadding(5);
			mainLayout.addMember(toolStrip);

			addBtn = new ToolStripButton(CallCenter.constants.add(),
					"addIcon.png");
			addBtn.setLayoutAlign(Alignment.LEFT);
			addBtn.setWidth(50);
			toolStrip.addButton(addBtn);

			editBtn = new ToolStripButton(CallCenter.constants.modify(),
					"editIcon.png");
			editBtn.setLayoutAlign(Alignment.LEFT);
			editBtn.setWidth(50);
			toolStrip.addButton(editBtn);

			disableBtn = new ToolStripButton(CallCenter.constants.disable(),
					"deleteIcon.png");
			disableBtn.setLayoutAlign(Alignment.LEFT);
			disableBtn.setWidth(50);
			toolStrip.addButton(disableBtn);

			activateBtn = new ToolStripButton(CallCenter.constants.enable(),
					"restoreIcon.gif");
			activateBtn.setLayoutAlign(Alignment.LEFT);
			activateBtn.setWidth(50);
			toolStrip.addButton(activateBtn);

			toolStrip.addSeparator();

			listGrid = new ListGrid() {
				protected String getCellCSSText(ListGridRecord record,
						int rowNum, int colNum) {
					ListGridRecord countryRecord = (ListGridRecord) record;
					if (countryRecord == null) {
						return super.getCellCSSText(record, rowNum, colNum);
					}
					Integer deleted = countryRecord
							.getAttributeAsInt("deleted");
					if (deleted != null && !deleted.equals(0)) {
						return "color:red;";
					} else {
						return super.getCellCSSText(record, rowNum, colNum);
					}
				};
			};

			DiscoveryClientDS discoveryClientDS = DiscoveryClientDS
					.getInstance();
			listGrid.setWidth(780);
			listGrid.setHeight(320);
			listGrid.setAlternateRecordStyles(true);
			listGrid.setDataSource(discoveryClientDS);
			listGrid.setAutoFetchData(true);
			listGrid.setShowFilterEditor(false);
			listGrid.setCanEdit(false);
			listGrid.setCanRemoveRecords(false);
			listGrid.setShowRowNumbers(true);
			listGrid.setCanHover(true);
			listGrid.setShowHover(true);
			listGrid.setShowHoverComponents(true);
			listGrid.setUseAllDataSourceFields(true);
			listGrid.setShowAllRecords(true);
			mainLayout.addMember(listGrid);

			TabSet tabSet = new TabSet();
			tabSet.setWidth(780);
			Tab tabDetViewer = new Tab(CallCenter.constants.view());
			final DetailViewer detailViewer = new DetailViewer();
			detailViewer.setDataSource(discoveryClientDS);
			detailViewer.setWidth(750);
			tabDetViewer.setPane(detailViewer);

			listGrid.addRecordClickHandler(new RecordClickHandler() {
				public void onRecordClick(RecordClickEvent event) {
					detailViewer.viewSelectedData(listGrid);
				}
			});

			listGrid.addRecordDoubleClickHandler(new RecordDoubleClickHandler() {
				@Override
				public void onRecordDoubleClick(RecordDoubleClickEvent event) {
					ListGridRecord listGridRecord = listGrid
							.getSelectedRecord();
					if (listGridRecord == null) {
						SC.say(CallCenter.constants.pleaseSelrecord());
						return;
					}
					DlgAddEditNonStandartInfo dlgAddEditNonStandartInfo = new DlgAddEditNonStandartInfo(
							listGrid, listGridRecord);
					dlgAddEditNonStandartInfo.show();
				}
			});
			
			tabSet.setTabs(tabDetViewer);
			mainLayout.addMember(tabSet);
			setPane(mainLayout);
			start();
		} catch (Exception e) {
			e.printStackTrace();
			SC.say(e.getMessage().toString());
		}
	}

	public void start() {
		try {
			boolean flag = true;
			if (!flag) {
				return;
			}
			Messaging.subscribe("discoveryServlet", new MessagingCallback() {
				@Override
				public void execute(Object data) {
					updateCallsLog(data);
				}
			});
			RPCRequest request = new RPCRequest();
			request.setContentType("text/html; charset=UTF-8");
			request.setActionURL("callcenter/discoveryServlet");
			RPCManager.sendRequest(request);
		} catch (Exception e) {
			e.printStackTrace();
			SC.say(e.toString());
		}
	}

	@SuppressWarnings("unchecked")
	private void updateCallsLog(Object data) {
		try {
			List<LinkedHashMap<?, ?>> callsData = (List<LinkedHashMap<?, ?>>) JSOHelper
					.convertToJava((JavaScriptObject) data);
			List<Record> newData = new ArrayList<Record>();
			for (LinkedHashMap<?, ?> recordData : callsData) {
				Integer discover_id = (Integer) recordData.get("discover_id");
				String call_id = (String) recordData.get("call_id");
				Integer ccr = (Integer) recordData.get("ccr");
				Integer deleted = (Integer) recordData.get("deleted");
				Integer discover_type_id = (Integer) recordData
						.get("discover_type_id");
				Integer execution_status = (Integer) recordData
						.get("execution_status");
				Integer iscorrect = (Integer) recordData.get("iscorrect");
				Integer response_type_id = (Integer) recordData
						.get("response_type_id");
				Date upd_date = (Date) recordData.get("upd_date");
				String upd_user = (String) recordData.get("upd_user");
				String discover_type = (String) recordData.get("discover_type");
				String phone = (String) recordData.get("phone");
				String contact_phone = (String) recordData.get("contact_phone");
				String contact_person = (String) recordData
						.get("contact_person");
				String discover_txt = (String) recordData.get("discover_txt");
				String rec_user = (String) recordData.get("rec_user");
				Date rec_date = (Date) recordData.get("rec_date");
				String status_descr = (String) recordData.get("status_descr");
				Record record = listGrid.getDataAsRecordList().find(
						"discover_id", discover_id);
				if (record == null) {
					record = new Record();
				}
				record.setAttribute("discover_id", discover_id);
				record.setAttribute("call_id", call_id);
				record.setAttribute("ccr", ccr);
				record.setAttribute("deleted", deleted);
				record.setAttribute("discover_type_id", discover_type_id);
				record.setAttribute("execution_status", execution_status);
				record.setAttribute("iscorrect", iscorrect);
				record.setAttribute("response_type_id", response_type_id);
				record.setAttribute("upd_date", upd_date);
				record.setAttribute("discover_type", discover_type);
				record.setAttribute("phone", phone);
				record.setAttribute("contact_phone", contact_phone);
				record.setAttribute("contact_person", contact_person);
				record.setAttribute("discover_txt", discover_txt);
				record.setAttribute("rec_user", rec_user);
				record.setAttribute("rec_date", rec_date);
				record.setAttribute("status_descr", status_descr);
				record.setAttribute("upd_user", upd_user);
				newData.add(record);
			}
			DSResponse dsResponse = new DSResponse();
			dsResponse.setData((Record[]) newData.toArray(new Record[newData
					.size()]));

			DSRequest dsRequest = new DSRequest();
			dsRequest.setContentType("text/html; charset=UTF-8");
			dsRequest.setOperationType(DSOperationType.UPDATE);
			DiscoveryClientDS.getInstance().updateCaches(dsResponse, dsRequest);
		} catch (Exception e) {
			e.printStackTrace();
			SC.say(e.toString());
		}
	}
}

5. HTML code looks like :

 <?xml version="1.0" encoding="UTF-8"?>
<html>
<head>
	<meta http-equiv="content-type" content="text/html; charset=UTF-8">
	
	<meta name="gwt:property" content="locale=en">
	<link type="text/css" rel="stylesheet" href="CallCenter.css">
	<link type="image/x-icon" href="images/favicon.ico" rel="shortcut icon" />
	<title>Info Billing 08</title>
	 <script>
    	var isomorphicDir="callcenter/sc/"
    </script>
	<script type="text/javascript" language="javascript" src="callcenter/callcenter.nocache.js"></script>
	<script type="text/javascript" language="javascript" src="callcenter/sc/DataSourceLoader?dataSource=DiscoveryDS</script>
</head>
<body>
</body>
</html>

Also it works fine with glassfish 3.1 and JBoss AS 7.0.0.Final


The problem was that servlet filter was incorrect (I took it from tomcat example).
after a change a bit it works.



________________________
Regards,
Paata Lominadze.

[Isomorphic]

Nice job and thanks for posting the solution!

[Isomorphic]

Also note: we have now corrected this at the servlet level by making MessagingServlet behave like IDACall - ie, we force UTF-8 on all responses unless an init-param directs otherwise. This fix will show up in nightly builds as of tomorrow (Friday August 12)