Relative Dates not converting to absolute dates

[saltwater]

We recently upgraded to SmartGWT 4.0 from 3.4. Most things seem to work, but the relative date criteria don't seem to be converting to absolute dates in the datasource.

I've tried explicitly calling convertRelativeDates and setAutoConvertRelativeDates(doc says default is true) but the xhr shows that something like $today is being passed to the server. Here's some example code to demonstrate. I also ran this code under our old jar and it converted to an absolute date properly.

HLayout hlay = new HLayout();

final FilterBuilder filterBuilder = new FilterBuilder();
filterBuilder.setDataSource(new DataSource(){{
	addField(new DataSourceDateTimeField("name", "label"){{
		setValidOperators(OperatorId.GREATER_THAN, OperatorId.LESS_THAN, OperatorId.IS_NULL, OperatorId.NOT_NULL);
	}});
}});		
hlay.addMember(filterBuilder);

hlay.addMember(new Button("send"){{
	addClickHandler(new ClickHandler() {
		@Override
		public void onClick(ClickEvent event) {
			DataSource datasource = new DataSource("index.jsp"){{
				setDataFormat(DSDataFormat.JSON);
				setAutoConvertRelativeDates(true);
			}};
			AdvancedCriteria criteria = filterBuilder.getCriteria(true);
			datasource.fetchData(criteria, null);
		}
	});
}});

hlay.show();

[saltwater]

Isomorphic,

Do you have any updates on this? Were you able to replicate it?

[saltwater]

This appears to be a bug.

Using the example code above in 3.4, it sent to the server an absolute date when using a relative date in the filter builder date function. With the 4.0 and 4.1 nightly builds it returns a relative date, even when explicitly converted to an absolute date.

Is this working differently for a reason? Is there an alternate way to get a relative date from an absolute date than Datasource's setAutoConvertRelativeDate and convertRelativeDate?

[Isomorphic]

For conversion to an absolute date to occur, you must actually declare the field as a date (or datetime).

[saltwater]

This is where I declare the filterbuilder:

filterBuilder.setDataSource(new DataSource(){{
	addField(new DataSourceDateTimeField("name", "label"){{
		setValidOperators(OperatorId.GREATER_THAN, OperatorId.LESS_THAN, OperatorId.IS_NULL, OperatorId.NOT_NULL);
	}});
}});

I assume by adding this as a field:
DataSourceDateTimeField
that I would be getting the full benefits of date. Is there an additional way this should work.

Also, like I mentioned earlier, I've run this test code in 3.4 and 4.0/4.1 and got different results, with 3.4 returning absolute dates and 4.0/4.1 returning relative dates.

[Isomorphic]

That bit of code is fine, however in your ClickHandler you declare yet a new DataSource which you then call fetch on, it is that DataSource which is required to have a date or datetime field.

You could alter your code slightly to make this work.

HLayout hlay = new HLayout();

final FilterBuilder filterBuilder = new FilterBuilder();
filterBuilder.setDataSource(new DataSource(){{
	addField(new DataSourceDateTimeField("name", "label"){{
		setValidOperators(OperatorId.GREATER_THAN, OperatorId.LESS_THAN, OperatorId.IS_NULL, OperatorId.NOT_NULL);
	}});
}});		
hlay.addMember(filterBuilder);

hlay.addMember(new Button("send"){{
	addClickHandler(new ClickHandler() {
		@Override
		public void onClick(ClickEvent event) {
			AdvancedCriteria criteria = filterBuilder.getCriteria(true);
			filterBuilder.getDataSource().fetchData(criteria, null);
		}
	});
}});

hlay.show();

As you can see the ClickHandler now grabs the datasource from the filterBuilder and performs a fetchData call.

[saltwater]

That does solve the example code I sent you. Essentially, the datasource needs to know what fields to convert. I want to point out that this isn't how SmartGWT worked before. The example I gave sends an absolute date in 3.4 and not in 4.0.

In our code, the filterbuilder with the date time datasourcefields is separate from the datasource that sends the info to the server. In fact, sometimes we save the filter builder's criteria to run it without the filterbuilder. The filterbuilder datasource isn't created. We would need a whole new dynamic ajax call and processing to just know what any of the fields are, let alone what are dates.

We've relied on the datasource converting any relative date to absolute date. Which isn't how it appears to work now. Is there some way we can force it to convert all relative dates like it did before?

[Isomorphic]

There is no such thing as SmartGWT 3.4, but yes, this is a behavior change relative to SmartGWT 3.1, albeit a behavior that isn't specifically documented.

We made this change because scanning the entire outbound data structure creates a performance problem for things like saving lots of edited rows.

Conversion to relative date is *one* thing that would go wrong if you don't have fields declared on the DataSource, other things are not encoding date and datetime values in the appropriate way. Getting the field information to the client would be the preferred solution (presumably there is a way to bundle this with other calls via queuing, and not have a separate call).

Another approach would be to call DateUtil.getAbsoluteDate() on each RelativeDate value in order to manually convert the Criteria over to using absolute dates.

[saltwater]

Thank you for your reply. That does give us options.

A related problem is that we've found a reason to have the option of skipping the client code altogether for this part of the the code(for scheduled reports).

Is there any java server code available that you know of for parsing your relative dates into absolute dates? Or at very least, do you have the regex hinted at here?:

    public RelativeDate(String value) {
        //TODO add regex based assert to validate base syntax, along supported with built in types
        this.value = value;
    }

[Isomorphic]

Sorry, no, we don't know of any kind of server-side library that could do this.

And no one has tried to implement such a regex for validation.

[saltwater]

For other people who come across the same problem, I prettified the core code and translated it into java. I only kept functionality for criteria builder date fields, since that's what I needed. If you need the full power, you would need to add the other time fields back in and deal with the qualifier in a more complex way.

package com.testing;

import java.util.Calendar;
import java.util.GregorianCalendar;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class DateConverter {

	private static String regex = 
			"([+-])(\\d+)(d|D|w|W|m|M)(\\[([+-]\\d+(d|D|w|W|m|M))\\])?";
	
	private static String mapRelativeDateShortcut(String shortcut){
		switch (shortcut) {
		case "$today":
			return "+0D";
		case "$yesterday":
			return "-1d[+0D]";
		case "$tomorrow":
			return "+1D";
		case "$weekFromNow":
			return "+1w[+0D]";
		case "$weekAgo":
			return "-1w[+0D]";
		case "$monthFromNow":
			return "+1m[+0D]";
		case "$monthAgo":
			return "-1m[+0D]";
		}			
		return null;			
	}	
	
	public static Calendar getAbsoluteDate(String relativeDate){
	    Pattern pattern = Pattern.compile(regex);
	    
	    if (relativeDate.startsWith("$")){
	    	relativeDate = mapRelativeDateShortcut(relativeDate);
	    }
	    
	    Matcher matcher = pattern.matcher(relativeDate);
	    if (!matcher.matches()){
	    	return null;
	    } 
	    
	  	int direction = matcher.group(1).equals("+") ? 1 : -1;
		int count = Integer.parseInt(matcher.group(2));
	    String period = matcher.group(3);
	    String qualifier = matcher.group(5);
	    
	    Calendar date = new GregorianCalendar();
		
		if (qualifier != null){
		    matcher = pattern.matcher(qualifier);
		    if(!matcher.matches()){
		    	return null;
		    } 

		    int qualifierDirection = matcher.group(1).equals("+") ? 1 : -1;
		    int qualifierCount = Integer.parseInt(matcher.group(2));
		    String qualifierPeriod = matcher.group(3);
		    
		    date = dateAdd(date, qualifierDirection, qualifierCount, qualifierPeriod);
		}
		
    	date = dateAdd(date, direction, count, period);
    	return date;
	}

	private static Calendar dateAdd(Calendar date, int direction, int count, String qualifierPeriod){
		switch (qualifierPeriod) {
		case "D":
			if (direction > 0) {
				date.set(Calendar.HOUR_OF_DAY, 0);
				date.set(Calendar.MINUTE, 0);
				date.set(Calendar.SECOND, 0);
				date.set(Calendar.MILLISECOND, 0);
			} else {
				date.set(Calendar.HOUR_OF_DAY, date.getMaximum(Calendar.HOUR_OF_DAY));
				date.set(Calendar.MINUTE, date.getMaximum(Calendar.MINUTE));
				date.set(Calendar.SECOND, date.getMaximum(Calendar.SECOND));
				date.set(Calendar.MILLISECOND, date.getMaximum(Calendar.MILLISECOND));
			}
		case "d":
			date.set(Calendar.DAY_OF_YEAR, date.get(Calendar.DAY_OF_YEAR) + count * direction);
			break;
		case "w":
			date.set(Calendar.WEEK_OF_YEAR, date.get(Calendar.WEEK_OF_YEAR) + count * direction);
			break;
		case "m":
			date.set(Calendar.MONTH, date.get(Calendar.MONTH) + count * direction);
			break;
		}
		return date;
	}

	public static void main(String[] args){
		 System.out.println(getAbsoluteDate("$today").getTime());		 
		 System.out.println(getAbsoluteDate("$yesterday").getTime());		 
		 System.out.println(getAbsoluteDate("$tomorrow").getTime());	
		 
		 System.out.println(getAbsoluteDate("$weekFromNow").getTime());		 
		 System.out.println(getAbsoluteDate("$weekAgo").getTime());		 
		 System.out.println(getAbsoluteDate("$monthFromNow").getTime());	
		 System.out.println(getAbsoluteDate("$monthAgo").getTime());	

		 
		 System.out.println(getAbsoluteDate("+365D").getTime());		 
		 System.out.println(getAbsoluteDate("+35w[-2D]").getTime());		 
		 System.out.println(getAbsoluteDate("-14m").getTime());		 
	}
}