GAEJPADataSource and Key as Encoded String

[atomatom]

Hi Alius. I wanted to say thank you for the example. This POJO is getting less PO. :)

It very interesting to see an example of how to do such things with the getter/setters.

[alius]

Ok - now we know that we are not in realm of SmartClient but in GAE engine + your application design.

If I understand you correctly... here is one possibility:

<DataSource
    ID="myClass_DataSource"
    serverConstructor="com.isomorphic.jpa.GAEJPADataSource"
    beanClassName="com.smartgwt.sample.server.MyClass"
>
	<fields>
		<field name="encodedKey" type="text" hidden="true" primaryKey="true" />
		<field name="keyNameVisible" type="text" />
		<field name="someData" type="text" />
	</fields>
</DataSource>

package com.smartgwt.sample.server;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import org.datanucleus.jpa.annotations.Extension;

@Entity
public class MyClass {
	private String encodedKey;

	private String keyName;

        private String someData;

	@Id
	@Column(nullable = false)
	@GeneratedValue(strategy = GenerationType.IDENTITY)
	@Extension(vendorName = "datanucleus", key = "gae.encoded-pk", value = "true")
	public String getEncodedKey() {
		return encodedKey;
	}

	public void setEncodedKey(String encodedKey) {
		this.encodedKey = encodedKey;
	}

	@Column
	@Extension(vendorName = "datanucleus", key = "gae.pk-name", value = "true")
	public String getKeyName() {
		return keyName;
	}

	public void setKeyName(String keyName) {
		this.keyName = keyName;
	}

	@Column
	public String getKeyNameVisible() {
		return keyName;
	}

	public void setKeyNameVisible(String keyNameVisible) {
		this.keyName = keyNameVisible;
	}

        @Column
        public String getSomeData() {
                return someData;
        }

        public void setSomeData(String someData) {
                this.someData = someData;
        }

}

I've moved annotations to getters;
Now table has 3 fields: encodedKey, keyNameVisible, someData;
Class has 4 properties: encodedKey, keyName (calculated by provider), keyNameVisible, someData;
Notice that getters/setters for keyName and keyNameVisible use same variable making it identical.

Alius
This allows you to filter by your PK value.

[atomatom]

Hi Alius. Thanks for your patience. Sorry if I wasn't clear enough on the need. I want to be able to provide my own keys (say we call it externalKey). For example ID001, and so on. At the moment, I have this annoyingly convoluted method of having two fields, encodedKey and externalKey. Since I'm not using the gae.pk-name annotations, I need to do a fetch/lookup each time I want to transform the encodedKey into an externalKey.

The sample works if I remove the "gae.pk-name" annotation, but it does not let me specify the key.

If I try and specify the key myself it says

Invalid primary key for com.smartgwt.sample.server.MyClass. The primary key field is an encoded String but an unencoded value has been provided. If you want to set an unencoded value on this field you can either change its type to be an unencoded String (remove the "gae.encoded-pk" extension), change its type to be a com.google.appengine.api.datastore.Key and then set the Key's name field, or create a separate String field for the name component of your primary key and add the "gae.pk-name" extension.

With the gae.pk-name annotation in, the example is really close to working - I can force it to work by forcing a refresh on the ListGrid, but that seems a bit like a hack.

[alius]

Once again: remove field "keyName" (or remove @Extension annotation and make it simple field) or explain why do you need it...

Here is excerpts from: http://code.google.com/appengine/docs/java/datastore/jdo/creatinggettinganddeletingdata.html#Keys (Key as Encoded String)

A "gae.pk-name" field can be set to a key name prior to saving the object. When the object is saved, the encoded key field is populated with the complete key that includes the key name. Its type must be String.

When a new object with a generated key (a key field using valueStrategy = IdGeneratorStrategy.IDENTITY) is created, its key value starts out null.

"keyName" ("gae.pk-name") allows you to generate encoded key with your provided string part. Identical encoded key will be generated for identical "keyName" thus only single entity is saved into data store.

BTW: After I removed @Extension annotation from "keyName" field (making it simple field) your sample works well:

package com.smartgwt.sample.server;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import org.datanucleus.jpa.annotations.Extension;

@Entity
public class MyClass {
	@Id
	@Column(nullable = false)
	@GeneratedValue(strategy = GenerationType.IDENTITY)
	@Extension(vendorName = "datanucleus", key = "gae.encoded-pk", value = "true")
	private String encodedKey;

	@Column
//	Commented out this annotation !!!
//	@Extension(vendorName = "datanucleus", key = "gae.pk-name", value = "true")
//	Now this property represents simple string field.
	private String keyName;

	public void setEncodedKey(String encodedKey) {
		this.encodedKey = encodedKey;
	}

	public String getEncodedKey() {
		return encodedKey;
	}

	public void setKeyName(String keyName) {
		this.keyName = keyName;
	}

	public String getKeyName() {
		return keyName;
	}
}

Alius.

[atomatom]

Note, duplicates are shown when you add records with the same keyName, when you refresh the duplicates are gone.

[atomatom]

Ok, I'm back to where I was with the ListGrid showing duplicates.

Here's the full sample.

myClass_DataSource.ds.xml

<DataSource ID="myClass_DataSource" serverConstructor="com.isomorphic.jpa.GAEJPADataSource"
	beanClassName="com.smartgwt.sample.server.MyClass">
	<fields>
		<field name="encodedKey" type="text" hidden="false" primaryKey="true" />
		<field name="keyName" type="text" />
	</fields>
</DataSource>

MyClass.java

package com.smartgwt.sample.server;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;

import org.datanucleus.jpa.annotations.Extension;

@Entity
public class MyClass {
	@Id
	@Column(nullable = false)
	@GeneratedValue(strategy = GenerationType.IDENTITY)
	@Extension(vendorName = "datanucleus", key = "gae.encoded-pk", value = "true")
	private String encodedKey;

	@Column
	@Extension(vendorName = "datanucleus", key = "gae.pk-name", value = "true")
	private String keyName;

	public void setEncodedKey(String encodedKey) {
		this.encodedKey = encodedKey;
	}

	public String getEncodedKey() {
		return encodedKey;
	}

	public void setKeyName(String keyName) {
		this.keyName = keyName;
	}

	public String getKeyName() {
		return keyName;
	}
}

and the code that uses it

	DataSource ds = DataSource.get("myClass_DataSource");
		final ListGrid grid = new ListGrid();
		grid.setDataSource(ds);
		grid.setCanEdit(true);
		grid.setAutoSaveEdits(true);
		grid.setAutoFetchData(true);

		Button add = new Button("Add");
		add.addClickHandler(new ClickHandler() {
			@Override
			public void onClick(ClickEvent event) {
				grid.startEditingNew();	
			}
		});	
		
		VLayout vLayout = new VLayout();
		vLayout.setWidth100();
		vLayout.setHeight100();
		vLayout.addMember(add);
		vLayout.addMember(grid);
		vLayout.draw();

[alius]

You've mixed JDO and JPA annotations.

GAE example contains classes City and Country with correct PK definition. For example:

    @Id
    @Column (nullable = false)
    @GeneratedValue (strategy = GenerationType.IDENTITY)
    @Extension (vendorName = "datanucleus", key = "gae.encoded-pk", value = "true")
    private String cityId;

Remove field "keyName" - you are not going to use it (at least until you will know exactly what to do with it).

You haven't included DS mapping file and code snippet which gives error.

Alius.

[atomatom]

ok, one step closer to understanding.

I tried to recreate a small example using the above code. In this process I realized the different annotations.

If I try and use the code above, DataNucleus blows up. It's angry because there is no @Id annotation.

DataNucleus Enhancer (version 1.1.4) : Enhancement of classes
27-Jan-2011 7:05:23 AM org.datanucleus.metadata.AbstractClassMetaData determineObjectIdClass
SEVERE: Class com.sample.data.MyClass has application-identity and no objectid-class specified yet has 0 primary key fields. Unable to use SingleFieldIdentity.
Class com.sample.MyClass has application-identity and no objectid-class specified yet has 0 primary key fields. Unable to use SingleFieldIdentity.
27-Jan-2011 7:05:24 AM org.datanucleus.enhancer.DataNucleusEnhancer main
SEVERE: DataNucleus Enhancer completed with an error. Please review the enhancer log for full details. Some classes may have been enhanced but some caused errors
DataNucleus Enhancer completed with an error. Please review the enhancer log for full details. Some classes may have been enhanced but some caused errors
Class com.sample.data.MyClass has application-identity and no objectid-class specified yet has 0 primary key fields. Unable to use SingleFieldIdentity.
org.datanucleus.metadata.InvalidMetaDataException: Class com.sample.MyClass has application-identity and no objectid-class specified yet has 0 primary key fields. Unable to use SingleFieldIdentity.
	at org.datanucleus.metadata.AbstractClassMetaData.determineObjectIdClass(AbstractClassMetaData.java:1032)
	at org.datanucleus.metadata.ClassMetaData.populate(ClassMetaData.java:205)
	at org.datanucleus.metadata.MetaDataManager$1.run(MetaDataManager.java:2317)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.datanucleus.metadata.MetaDataManager.populateAbstractClassMetaData(MetaDataManager.java:2311)
DataNucleus Enhancer completed and no classes were enhanced. Consult the log for full details
	at org.datanucleus.metadata.MetaDataManager.populateFileMetaData(MetaDataManager.java:2148)
	at org.datanucleus.metadata.MetaDataManager.initialiseFileMetaDataForUse(MetaDataManager.java:864)
	at org.datanucleus.metadata.MetaDataManager.loadClasses(MetaDataManager.java:433)
	at org.datanucleus.enhancer.DataNucleusEnhancer.getFileMetadataForInput(DataNucleusEnhancer.java:743)
	at org.datanucleus.enhancer.DataNucleusEnhancer.enhance(DataNucleusEnhancer.java:545)
	at org.datanucleus.enhancer.DataNucleusEnhancer.main(DataNucleusEnhancer.java:1252)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at com.google.appengine.tools.enhancer.Enhancer.execute(Enhancer.java:57)
	at com.google.appengine.tools.enhancer.Enhance.<init>(Enhance.java:60)
	at com.google.appengine.tools.enhancer.Enhance.main(Enhance.java:41)

If I add the @Id annotation, then I get the previous error

Attempting to fetch field with "gae.pk-name" extension but the entity is identified by an id, not a name.

import javax.jdo.annotations.IdGeneratorStrategy;
import javax.jdo.annotations.Persistent;
import javax.jdo.annotations.PrimaryKey;
import javax.persistence.Entity;
import javax.persistence.Id;
 
import org.datanucleus.jpa.annotations.Extension;

@Entity
public class MyClass{
	@PrimaryKey 
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    @Extension(vendorName="datanucleus", key="gae.encoded-pk", value="true")
    private String encodedKey;

    @Persistent
    @Extension(vendorName="datanucleus", key="gae.pk-name", value="true")
    private String keyName;

	public void setEncodedKey(String encodedKey) {
		this.encodedKey = encodedKey;
	}

	public String getEncodedKey() {
		return encodedKey;
	}

	public void setKeyName(String keyName) {
		this.keyName = keyName;
	}

	public String getKeyName() {
		return keyName;
	}
}

Is this because of mixing JDO/JPA annotations? Does DataNucleus blow up becuase I'm adding this to persistence.xml:

	<persistence-unit name="ds" transaction-type="RESOURCE_LOCAL">
		<provider>org.datanucleus.store.appengine.jpa.DatastorePersistenceProvider</provider>
[b]		<class>com.sample.data.MyClass</class>[/b]

[alius]

Can you post your entity, DS mapping file and code snippet which gives this error?

[atomatom]

Hi Alius. Thanks for the help. I spent a fair bit of time before reading about how the keys are stored, before I gave up on it. (I would still really like to get this working though, my workarounds are not great).

When I tried out the DataSource using the encodedKey as the primaryKey I get this error:

Attempting to fetch field with "gae.pk-name" extension but the entity is identified by an id, not a name.

[alius]

Hi,

Actually encoded string PK is a special (string) form of com.google.appengine.api.datastore.Key:
http://code.google.com/appengine/docs/java/javadoc/com/google/appengine/api/datastore/Key.html

"keyName" property is just a part of actual PK (it is calculated from real PK by provider). That is why you have records with duplicate PK's.

Use "encodedKey" (it represents full PK) instead:

<DataSource ID="myClass_DataSource" serverConstructor="com.isomorphic.jpa.GAEJPADataSource"
	beanClassName="com.smartgwt.sample.server.MyClass">
	<fields>
		<field name="encodedKey" type="text" primaryKey="true" />
	</fields>
</DataSource>

BTW: references between GAE entities are stored in PK as well.

Alius.

[atomatom]

Yeah, that example code is quite broken. I learned quite a lot about JPA since that post, but I still never figured out how to configure a datasource to read/write to using encoded strings.

@Entity
public class MyClass{
    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    @Extension(vendorName="datanucleus", key="gae.encoded-pk", value="true")
    private String encodedKey;

    @Persistent
    @Extension(vendorName="datanucleus", key="gae.pk-name", value="true")
    private String keyName;
}

so, for example, my understanding was I could now create a datasource

<DataSource ID="myClass_DataSource" serverConstructor="com.isomorphic.jpa.GAEJPADataSource"
	beanClassName="com.smartgwt.sample.server.MyClass">
	<fields>
		<field name="keyName" type="text" primaryKey="true" />
	</fields>
</DataSource>

Now, the above code works. And indeed, it uses my unencoded key string as the key.

However, if I attach the above DataSource to a ListGrid, as is, and add two records with the same value for keyName it creates two records. If I refresh the page, then the duplicate is gone - the second add overwrites the first record (I checked this by adding an extra field to distinguish). I would expect it to (a) complain that the key is already being used, and (b) keep state consistent with the stored data.

Is there a step that I am missing?

[alius]

Hi,

Property marked with "datanucleus" extension "gae.pk-id" are populated by provider and can not be modified thus can not be primary key.

Check http://code.google.com/appengine/docs/java/datastore/jdo/creatinggettinganddeletingdata.html#Keys
section "Key as Encoded String".

Even though it is for JDO (not JPA) - concepts are the same.

Alius.

[atomatom]

Does anyone have an example of using a Key as Encoded String working with GAEJPADataSource. Here's what a Key as Encoded String is http://code.google.com/intl/en/appen...data.html#Keys

I want to do something like (and I've tried adding a gae.encoded-pk field in addition to the below, but I'm not sure how to connect the datasource then - do I reference the encodedKey or the longId?)

@Entity
public class MyClass{
	@Id
	@Column(nullable = false)
	@GeneratedValue(strategy = GenerationType.IDENTITY)
	@Extension(vendorName = "datanucleus", key = "gae.pk-id", value = "true")
	private Long longId;
}

The specific error I am getting is

Error in meta-data for com.example.MyClass.longId: A field with the "gae.pk-id" extension must not be the primary key.

I've checked out the writeup on http://www.smartclient.com/smartgwte...tegration.html and I think I've checked all the examples.

Did I miss an example of this working, or does it not work?

Thanks in advance for any help.