Announcement

**ibrandt** · 6 Aug 2013, 15:29

The JSP page in question has 3 loadDS tags in succession:

Code:

<isc:loadDS ID="MyPageDS"/>
<isc:loadDS ID="MyPageDetailsDS"/>
<isc:loadDS ID="MyPageNotesDS"/>

The matching datasource descriptors:

Code:

<DataSource ID="MyPageDS" serverConstructor="spring:myPageDataSource"
	schemaBean="com.mycompany.MyPage" validateRelatedRecords="true">
	<fields>
		<field primaryKey="true" name="id" title="ID" type="sequence" />
		<field name="additionalCriteria1Id" type="integer" />
		<field name="additionalCriteria2Id" type="integer" />
		<field name="additionalCriteria3Id" type="integer" />
		<field name="additionalCriteria4Id" type="integer" />
		<field name="aTextField" type="text" />
		<field name="aDateField" type="datetime" />
		<field name="anotherTextField" type="text" />
		<field name="aStatusField" type="enum">
			<valueMap>
				<value>Unresolved</value>
				<value>Resolved</value>
			</valueMap>
		</field>
		<field name="mostRecentNote" type="MyPageNotesDS" />
		<field name="comments" type="any" />
	</fields>

	<operationBindings>
		<operationBinding operationType="fetch" operationId="myPageExport">
			<serverObject bean="myPageDSExportDMI" lookupStyle="spring" />
			<serverMethod>executeExport</serverMethod>
		</operationBinding>
	</operationBindings>

</DataSource>

Code:

<DataSource ID="MyPageDetailsDS" serverConstructor="spring:myPageDetailsDataSource"
	schemaBean="com.mycompany.MyPageDetail">
	<fields>
		<field primaryKey="true" name="id" title="ID" type="sequence" />
		<field name="pageDetails" title="Page Details" />
	</fields>
</DataSource>

Code:

<DataSource ID="MyPageNotesDS" serverConstructor="spring:myPageNotesDataSource"
	schemaBean="com.mycompany.MyPageNote">
	<fields>
		<field primaryKey="true" name="id" title="ID" type="sequence" />
		<field name="additionalCriteria1Id" title="Survey ID" type="integer" />
		<field name="timestamp" title="Date" type="datetime" />
		<field name="userId" title="User ID" type="integer" />
		<field name="userName" title="User Name" type="text" />
		<field name="note" title="Note" type="text" />
	</fields>
</DataSource>

**Isomorphic** · 6 Aug 2013, 17:10

This makes no sense at all, no NullPointerExceptions could possibly happen in this code block given the preceding logs.

Also, like your other thread from today, something about your environment seems to be making us unable to get a stack trace from thrown exceptions to report in the log.

You haven't mentioned anything about your environment. Can you try to normalize it:

1. if you're not using the stock JVM, switch back

2. if you have JIT enabled, disable it

3. if you're on an unusual app server, and are able to change to a common one (eg Tomcat), do that

You'll know you've succeeded when the exception report includes a stack trace with a few more frames.

**ibrandt** · 10 Oct 2013, 09:31

Originally posted by Isomorphic View Post

This makes no sense at all, no NullPointerExceptions could possibly happen in this code block given the preceding logs.

Also, like your other thread from today, something about your environment seems to be making us unable to get a stack trace from thrown exceptions to report in the log.

You haven't mentioned anything about your environment. Can you try to normalize it:

1. if you're not using the stock JVM, switch back

Code:

$ java -version
java version "1.6.0_24"
Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode)
$ cat /proc/version 
Linux version 2.6.18-308.24.1.el5 (mockbuild@builder17.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-52)) #1 SMP Tue Dec 4 17:43:34 EST 2012

Originally posted by Isomorphic View Post

2. if you have JIT enabled, disable it

I can't do that in production, and haven't been able to reproduce this in test. I would think it more likely that this is a concurrency related bug than a JIT bug.

Originally posted by Isomorphic View Post

3. if you're on an unusual app server, and are able to change to a common one (eg Tomcat), do that

Tomcat 7.0.27.

Originally posted by Isomorphic View Post

You'll know you've succeeded when the exception report includes a stack trace with a few more frames.

The full stack trace from today's occurrence is here: https://gist.github.com/ianbrandt/8c0a8078d0d340d45785 (was too big to include in post or attach). If what you're looking for isn't there could you be a bit more specific about what's missing? I have logging configured per this post: http://forums.smartclient.com/showpo...4&postcount=12. To the best of my knowledge everything logged to Log4J is redirected to Logback and output. If expected log entries are being filtered I should be able to track down why if I know what to look for.

**ibrandt** · 10 Oct 2013, 10:46

I'm also getting intermittent ConcurrentModificationExceptions out of LoadDSTag: https://gist.github.com/ianbrandt/d3de50d003b6c34bdc41. Is this related, or should I start another thread? I ask because not knowing anything about the code, with intermittent NullPointerExceptions, ClassCastExceptions, and ConcurrentModificationExceptions all coming from LoadDSTag--which as you say shouldn't be happening--does this not start to smell of a thread safety issue?

**Isomorphic** · 10 Oct 2013, 13:02

This is a completely different NPE, but is again one that looks impossible from code inspection. If you can't reproduce it at all, that's unfortunate, there's not much we can do about it if it's only been seen anecdotally and the stack trace looks like nonsense. We already have automated tests that fire hundreds of concurrent requests at the system.

There was a recent fixed threading issue that was reported for 9.0, could arise *only* if DataSource pooling were explicitly turned off, and only for server-initiated requests.. doesn't seem relevant here, but gives you an idea of how hard you *should* have to work to find an actual concurrency issue. And possibly implies you should upgrade to see if it somehow fixes the issue - you'll eventually need to for other reasons anyway.

**ibrandt** · 10 Oct 2013, 16:13

Originally posted by Isomorphic View Post

This is a completely different NPE, but is again one that looks impossible from code inspection. If you can't reproduce it at all, that's unfortunate, there's not much we can do about it if it's only been seen anecdotally and the stack trace looks like nonsense. We already have automated tests that fire hundreds of concurrent requests at the system.

There was a recent fixed threading issue that was reported for 9.0, could arise *only* if DataSource pooling were explicitly turned off, and only for server-initiated requests.. doesn't seem relevant here, but gives you an idea of how hard you *should* have to work to find an actual concurrency issue. And possibly implies you should upgrade to see if it somehow fixes the issue - you'll eventually need to for other reasons anyway.

We'll upgrade to 9.0 and see if that fixes this.

How about the ConcurrentModificationException, should I start another thread for that? Safe to say that is a thread safety issue, no? Perhaps investigation of it will shed light on these others.

**Isomorphic** · 10 Oct 2013, 16:20

No need for a separate forum thread on that - we suspect that all 3 of these issues are related, since they seem to be unique to you. Just let us know whether the 9.0 upgrade seems to fix them; if it doesn't, let us know the symptoms you get with 9.0.

Announcement

Intermittent NullPointerException in LoadDSTag

Comment

Comment

Comment

Comment

Comment

Comment

Comment