encoding date to XML Schema

[janusz]

Hi,

When encoding Date object to XML Schema Date format using JSONEncoder.encodeDate the result is date does not have any information about timezone and it is assumed to be in UTC.

Wouldn't it be better to append "Z" to date to specify that it is really UTC. Some backends (like mine) assumes that date without timezone are in local timezone as in ISO 8601 and XML Schema.

I believe that some problems mentioned by others on this forum was also caused by that.

Regards,
Janusz

[janusz]

And here is a small fix:

isc.addMethods(Date.prototype, {

    toSchemaDate : function (logicalType) {
      // logical date values have no meaningful time
      // Note that they also have "no meaningful timezone" - we display native browser locale time
      // to the user and when we serialize to send to the server we serialize in that same
      // local timezone.
      if ((logicalType == "date") || this.logicalDate) {
	return isc.SB.concat(
	  this.getFullYear().stringify(4),
	  "-",
	  (this.getMonth() + 1).stringify(2), 	// getMonth() is zero-based
	  "-",	
	  this.getDate().stringify(2)
	);
      }

      // logical times are serialized as truncated schema strings (HH:MM:SS) by default
      if ((!isc.DataSource || !isc.DataSource.serializeTimeAsDatetime) && 
	(logicalType == "time" || this.logicalTime))
      {
	return isc.SB.concat(
	  this.getUTCHours().stringify(2),":", 
	  this.getUTCMinutes().stringify(2), ":", 
	  this.getUTCSeconds().stringify(2)
	);
      }

      // represent date time values in UTC
      return isc.SB.concat(
	this.getUTCFullYear().stringify(4),
	"-",
	(this.getUTCMonth() + 1).stringify(2), 	// getMonth() is zero-based
	"-",	
	this.getUTCDate().stringify(2),
	"T",
	this.getUTCHours().stringify(2),
	":",
	this.getUTCMinutes().stringify(2),
	":",
	this.getUTCSeconds().stringify(2),
	"Z"
      );
    }

[Isomorphic]

This patch should not be used.

By design, a field of type "date" is a logical date that does not have a timezone (for example, the date of a holiday). If you want a date that has a time, use datetime.

[janusz]

I think, that it does not have anything with DateItem, but serialization of Date object to JSON according to ISO standard.

Here is my scenario. If you can show me better solution, I will be grateful.
I am in +01:00 timezone.

DynamicForm.create({
  ID: "form",
  autoDraw: true,
    fields: [
      {name: "date", type: "date"},
      {name: "datetime", type: "datetime"}
    ]
});

I am setting both fields manually to 1.01.2011.
Then trying to send both values to server with JSON.

isc.JSON.encode(form.getValues());

I am getting: "{ "date":"2010-12-31T23:00:00", "datetime":"2010-12-31T00:00:00" }"
While I should be getting something like: "{ "date":"2011-01-01T00:00:00", "datetime":"2011-01-01T00:00:00" }"
or even better: "{ "date":"2010-12-31T23:00:00Z", "datetime":"2010-12-31T00:00:00Z" }"

My PHP backend is then thinking that both dates are encoded in local timezone so it treats both as one hour before midnight local time.
"date" field is then interpreted as date, so hours, minutes and seconds are discarded and we are getting 2010-12-31 - one day earlier that we picked on DateItem.


To finally convince you - even JS has problems with this situation:

new Date("2010-12-31T23:00:00Z") is 
Sat Jan 01 2011 00:00:00 GMT+0100 (CET)
new Date(new Date("2010-12-31T23:00:00Z").toSchemaDate())
Fri Dec 31 2010 23:00:00 GMT+0100 (CET)

[Isomorphic]

The XML Schema standard recommends that a date without a time be transmitted as just a date with no time specified. This is what happens for a DataSource field of type "date" when sending JSON (eg RestDataSource), but obviously, it does not happen if you just serialize to JSON with no DataSource involved at all.

XML Schema also recommends that a datetime without an explicit timezone is treated as UTC. Don't confuse wire format with what happens with new Date() - separate concepts.

[janusz]

The XML Schema standard recommends that a date without a time be transmitted as just a date with no time specified. This is what happens for a DataSource field of type "date" when sending JSON (eg RestDataSource), but obviously, it does not happen if you just serialize to JSON with no DataSource involved at all.

That is absolutely true and from what I have seen in code it happens when Date has logicalDate set to true.
Is it doable, that not only dates from DataSources, but also dates from DateItem from DynamicForm not linked to any DataSource would have logicalDate set to true, so they would be encoded as date only string?

XML Schema also recommends that a datetime without an explicit timezone is treated as UTC. Don't confuse wire format with what happens with new Date() - separate concepts.

Well not exactly. Dates in XML Schema are based on ISO 8601 and this is part of the XML specification:

"Local" or untimezoned times are presumed to be the time in the timezone of some unspecified locality as prescribed by the appropriate legal authority

So datetime without "Z" is my or yours or anybody's local time. And this is how my PHP interprets it according to ISO 8601.

Take a look at these posts, as I think, that this is the same problem as I have:
http://forums.smartclient.com/showth...light=timezone
http://forums.smartclient.com/showth...light=timezone

[Isomorphic]

Some Date instances in JavaScript represent logical dates and some represent full date times. The former have logicalDate:true and this is automatically added whenever you are working on the context of a DataSource where the field has been declared type "date" so we know a date is intended.

It makes no sense to add this to all Date instances, since again, some represent datetime values.

XML Schema dates are based on ISO8601 but XML Schema is not ISO8601 - it separately recommends treatment of full date times without an explicit timezone qualifier as UTC.

[janusz]

Some Date instances in JavaScript represent logical dates and some represent full date times. The former have logicalDate:true and this is automatically added whenever you are working on the context of a DataSource where the field has been declared type "date" so we know a date is intended.

It makes no sense to add this to all Date instances, since again, some represent datetime values.

I was writing about Date objects from DateItem - they are logically dates without time, but do not have logicalDate = true. This is another topic, so let's move this conversation to http://forums.smartclient.com/showthread.php?t=15045 and concentrate here on implementation of Date.toSchemaDate and it's conformity to XML Schema.

XML Schema dates are based on ISO8601 but XML Schema is not ISO8601 - it separately recommends treatment of full date times without an explicit timezone qualifier as UTC.

I am sorry, but you are wrong, I have cited XML specification - link from your reference.
http://www.w3.org/TR/xmlschema-2/#dateTime
Take a look at point 3.2.7, third paragraph starting with words "All timezoned times", third sentence.

This paragraph states that timezoned times are in UTC.
What is timezoned time is normalized in point 3.2.7.2 third subpoint - "for timezoned values, the timezone must be represented with 'Z' (All timezoned dateTime values are UTC.)."
Non canonical forms let us use times in other timezones by replacing "Z" with "+/-mm:ss"

XML encoding of dates are based so much on ISO 8601 that I cannot name any difference between those two specifications.


Let us take sample Date without logicalDate and logicalTime and encode it assuming that right now is 17.01.2011 21:00 here in Warsaw (UTC+01:00).

var d = new Date();
d.toSchemaDate();

In current version of SmartClient I am getting "2011-01-17T20:00:00", because Date.toSchemaDate uses getUTC... functions but omits "Z".

According to XML Schema specs. cited about expected result for this date is:
"2011-01-17T20:00:00Z" - canonical form that should be used
"2011-01-17T21:00:00+01:00" - another acceptable form

[Isomorphic]

What you've cited says that times without an explicit timezone are "presumed to be the time in the timezone of some unspecified locality as prescribed by the appropriate legal authority". In other words, no recommendation is given.

Here is the more complete and nuanced explanation of timezone treatment that the framework's behavior is based on. You will see that in many places, the recommendation is that UTC is used if no explicit timezone is supplied - particularly for date comparison, the most nearly analogous use case (consider that datetimes appear in criteria).

We have no plans to change the framework behavior here. Look for an option to have your PHP library apply UTC instead of local time, or change the date values to Strings of whatever format you'd like before they are serialized.

[Isomorphic]

Just to really make this abundantly clear - the XML Schema note discusses notions of datetime that do not have a timezone. For example, 9am August 1st exists as a datetime value that does not have a timezone - it's a different UTC time in each timezone.

SmartClient/SmartGWT does not offer separate field types for these different notions of datetime - there's just datetime - but if we were to add a timezone qualifier for all datetime values that would prevent timezoneless interpretation.

[janusz]

Ok. I understand idea of using timezoneless datetimes.

But tell me why then you use getUTC... functions in Date.toSchemaDate when encoding a Date object with logicalTime or without logicalDate or logicalTime? It effectively moves time to UTC shifting it few hours from what user intended.

Tell me if my way of thinking here is ok: I have a form where I ask users from around a world (in all timezones) what is your favourite time of day (it is not connected to date, timezone, DST... - just plain time)

var form = DynamicForm({
  fields: [{name: "favourite_time", type: "time"}]
});

// let's say user put 13:15 in the field and we want to encode this in JSON for sending to backend
var d = form.getValue("favourite_time");

// this is just a time, so let's mark it to ignore date part
d.logicalTime = true;

var json = isc.JSON.encode(d);

According to what you said, I should expect to have json == "\"13:15:00\"", but it is "\"12:15:00\"" on my computer and different in other parts of the world, so I can't really tell what is the user's favourite time without checking in which timezone it was encoded.

The same is Date objects without logicalDate and logicalTime.
You should encode it with getDate and then it would be perfect timezoneless datetime or encode it with getUTCDate and append "Z" to it as it would be UTC datetime.

[Isomorphic]

The timezone offset that is used for user input is configurable so that you can handle both use cases.

The only thing we still need to add is a per-field timezone offset so that it's easy to handle both timezoned and timezoneless inputs in the same form.

[janusz]

I am sorry, but I am confused.

Can you tell me if this behaviour in my simple example two posts ago is by design or is it a bug or is this a feature that is unfinished?

I can rewrite this example dropping any references to fields, DynamicForms, because I believe there is a problem in implementation of Data.toSchemaDate which is not related to Time.setDefaultDisplayTimezone and for me following code should and with result "13:15:00" and not "12:15:00":

var d = new Date("Tue Jan 18 2011 13:15:00 GMT+0100 (CET)");
d.logicalTime = true;
var result = d.toSchemaDate();

Value of result is the same no matter what is the value of Time.getDefaultDisplayTimezone().

[Isomorphic]

That's correct and as expected. The display timezone affects the display of dates to users. Serialization to JSON is not display, and neither is a call to toSchemaDate().