Velocity mail problem: some values not avail in template, and sometimes mail sent 2x

[Sytematic]

Hello,

Using Power, (06-07-2011 nightly)

I think there is are two problems with the velocity mail template system , in some occasions.

Consider this form to be submitted, to a Student.ds.xml (SQL) datasource:
client side:

Map vals = form.getValues();
vals.put("hostname", "someHostname");
vals.put("courseEditions", courseWid.getSelectedIds()); //List<Integer> is passed here
		
form.setValues(vals);
form.saveData();

As you can see,
A student fills out his personal details (email, name, etc.) and presses a button that saves the form. He also marked a few courses he wants to enroll for. This is passed on in the map 'courseEditions', which is a List<Integer>

Student.ds.xml

<DataSource ...>
 ...
 <operationBinding operationType="add" requiresAuthentication="false" cacheSyncOperation="publicStudentRegistrationFetch" requiresRole="" operationId="publicStudentRegistration" > 
      <serverObject className="nl.sytematic.projects.MyProject.server.custom.dmi.CustomStudentDMI"/>
      <mail> 
        <from>noreply@mydomain.nl</from>  
        <to>$email</to>  
        <templateFile>emailtemplates/student_subscribe.txt</templateFile>  
        <subject>My subject</subject> 
      </mail> 
    </operationBinding>  
    <operationBinding operationType="fetch" requiresAuthentication="true" requiresRole="CAN_READ_STUDENT">

First problem: the $courseEditions and $hostname (which are the extra values put into the map) are not available in the template. Putting them in the template as $hostname just puts '$hostname' in the email. Also, in my dmi method, dsRequest.getFieldValue("hostname") does actually work in Java, it returns the correct value.

Second problem: somehow, because courseEditions is a list<Integer> the mail is sent twice. If I remove this part, the mail is sent once. I am not totally convinced yet what happens, but I am still investigating this.

As far as the first problem is concerned: is this on purpose, or is this just a missing feature? Would love to be able to add some extra stuff to the form. The values ARE available in the dmi serverObject that is called, i've checked this.

Any thoughts on any of the problems?

[Isomorphic]

The immediate variables like $email access dsResponse.data, so it's expected and by design that this contains only post-validation, stored values. However $dsRequest is available to you if you want to access additional submitted data.

[Sytematic]

Thanks that works great.

Now I 'm still stuck with the message being sent twice. If I outcomment the

vals.put("courseEditions", courseWid.getSelectedIds()); //List<Integer> is passed in the map

It is not sent twice.

How can that line cause it to send it twice? Because just 1 student gets added, and a student enrolls in multiple course ID's.

Any thoughts?

[Isomorphic]

That really sounds like it's got to be an application-level bug, but if you can isolate it to a test case we'll take a look.

[Sytematic]

I'll try that, but I am not doing anything funny. It is caused by my custom dmi, which just does

1) another query to fetch some value from another table,
2) then does the main Student insert query, (which triggers the mail)
3) and then subscribes the student to the courses he selected.

If I remove the <serverObject> definition, and drop back to the default (ie. bypassing the dmi), the twice sending is not happening.

Dmi code as follows:

public DSResponse add(DSRequest dsRequest) throws Exception {
	//resolve SaasCustomer id based on hostname
        //  1)
	String hostname = (String) dsRequest.getFieldValue("hostname");
	
	DSRequest saas = new DSRequest("createSaasCustomer", "fetch");
	saas.setOperationId("fetchFromPortalPanelToGetLatestInsertID");
	saas.setCriteria("hostname", hostname);
	saas.setRPCManager(dsRequest.getRPCManager());
	
	DSResponse tempResp = saas.execute();		
	List data = tempResp.getDataList();
	int saasId = 0;
	for(Object o : data){
		//Create a new row for each LectureTemplate in the Lecture ds, making sure the 'repeat' works correctly.
		Map record = (Map) o;
		saasId =  (Integer) record.get("SaasCustomer_id");
	}
	
	if(saasId == 0) {
		throw new Exception("Warning: SaasCustomer_id not found!");
	}


	//SaasCustomer_id is found, and valid, add it to the current dsRequest.
	dsRequest.setFieldValue("SaasCustomer_id", saasId);
		
	List<Long> courseEditions = (List<Long>) dsRequest.getFieldValue("courseEditions");
		
         // 2)
	DSResponse res = dsRequest.execute();
		
	data = res.getDataList();

	int studentId = 0;
	List<Map<String,Object>> enrollmentData = new ArrayList<Map<String, Object>>();
		
	for(Object o: data){
		Map record = (Map) o;
		studentId = (Integer) record.get("Student_id");
	}

	for(Long ced : courseEditions){
		Map<String, Object> rec = new HashMap<String,Object>();
		rec.put("Student_id", studentId);
		rec.put("CourseEdition_id", ced);
		enrollmentData.add(rec);
	}
		
         // 3) enroll students
         //table that stores enrollment of a student in a Course
         DSRequest enroll = new DSRequest("CourseEdition_students_Student", "add");		
	enroll.setOperationId("insertFromPublicSubscription");
	enroll.setRPCManager(dsRequest.getRPCManager());
	enroll.setValues(enrollmentData);
	enroll.execute();
	return res;
}

I tried removing the enrollment query, and removing the first query (marked 1) and 3)), which didn't work, mail was sent twice still.

Only way I was able to get the mail to be sent once, was by
1) removing vals.put("courseEditions", courseWid.getSelectedIds()); //List<Integer> is passed in the map
2) removing the dmi altogether (but then the enrollment doesn't work).

Any thoughts?
Is it a problem if I send a List<Integer> to the server, which I did?

[Sytematic]

I've got a bit more information, which causes the mail to be sent twice, and I think it is a bug.

In my DMI, when step-debugging through it I can see the mail being sent on the lines marked ####

This is when calling DSResponse res =dsRequest.execute() AND when returnin the eventual res

public DSResponse add(DSRequest dsRequest) throws Exception {
        //cut out some non-dependent code / business rules checks
//<snip>

	ArrayList<String> courseEditions = (ArrayList<String>) dsRequest.getFieldValue("courseEditions");
	
	// ##### after this line, an email is sent.
	DSResponse res = dsRequest.execute();
		
	data = res.getDataList();

	int studentId = 0;
	List<Map<String,Object>> enrollmentData = new ArrayList<Map<String, Object>>();
		
	for(Object o: data){
		Map record = (Map) o;
		studentId = (Integer) record.get("Student_id");
		break;
	}

	for(String ced : courseEditions){
		if(ced != null) {
			Map<String, Object> rec = new HashMap<String,Object>();
			rec.put("Student_id", studentId);
			rec.put("CourseEdition_id", Integer.parseInt(ced));
			enrollmentData.add(rec);
		}
	}
	
	DSRequest enroll = new DSRequest("CourseEdition_students_Student", "add");		
	enroll.setOperationId("insertFromPublicSubscription");
	enroll.setRPCManager(dsRequest.getRPCManager());
	enroll.setValues(enrollmentData);
	enroll.execute();

	// ##### after this line, an email is sent.
       return res;
}

Is this by design, or is it happening non-intentionally (read, is it a bug?).

How should I rewrite the code?


*** edit: solved by returning a new DSResponse(); with status set to Success

[Isomorphic]

This was a bug that we have now corrected.