Client side caching of DataSourceLoader-call result?

[Blama]

Hi Isomorphic,

I saw this post and had a look at my application. I'm using caching for all your static resources like described here.
Now a call of my application with full cache looks like this:



As you can see, the most time is spent on the DataSourceLoader, which by now loads 100 .ds.xmls.

All the .ds.xml loaded at startup are static (no DynamicDSGenerator involved).
Until now I do not use field level declarative security, so the result should be the same for all users.

Question: Is there some reason I should not cache the result?


Even if I'd be using field level declarative security, I could include a cache-buster for

• the user (if many different users log in in the same browser)
• the last user-config change-date from the DB (for e.g. user-role changes)

Question 2: Is there some reason I should not cache the result if I do so? Am I missing something?

Thank you & Best regards
Blama

[Isomorphic]

Hi Blama,
There's nothing fundamentally wrong with cacheing static dataSource definitions client side, using the standard "set infinite expires headers in the server response, and avoid staleness by via an additional version parameter on the URL to bypass browser cache when necessary" approach.

There are a few things to consider:

Firstly - this is easiest to achieve by using explicit script src=... tags to hit the dataSource loader. If you are loading dataSources dynamically, using the client side DataSource.load(...) API won't give you a chance to add a version parameter to the URL. You could do something like use FileLoader to load the DataSource as a standard JS file and append the URL version param there if you do need dynamic loading.

Other than that your main concern will be anything in the dataSource definition which is generated dynamically on the server isn't stale.
Some cases to consider:

- You may be explicitly modifying your dataSource with dynamic content - for example, supplying localized errorMessages and so on as described here. If this is the case and the localized strings change, you'd have to change the version parameter to pick up a fresh version of the DS from the server.

- Declarative security /field visibility, etc (as you mentioned). In some cases, dataSource fields are suppressed entirely for particular users based on declarative security settings (viewRequires, editRequiresRole, etc). If you the dataSource is cached on the client, these checks won't run when the DataSource is created in client-side code, meaning a user will potentially see fields they shouldn't, or fail to see fields they should.
This is actually just a UI problem, not a security risk. When fetches are executed, etc, all the standard server-side logic will run, including re-evaluating these conditions, so the data returned won't contain values for the fields in question if not appropriate. In concrete terms, this means a user may see a field they shouldn't in a component, but it wouldn't contain any data.

- If you're using a SQL-backed dataSource, if the table structure changes in most cases you'd probably modify the .ds.xml file and as such require the version-param to change on the URL to avoid staleness.
Actually this wouldn't always be necessary - if you have explicit nativeNames on the dataSource field definitions, those could change (and be read by the server code only) and the client definition wouldn't need to change.
Though if you are relying on the autoDeriveSchema automated derivation of foreignKey relationships you would need to force a refresh if the table changed as the generated DataSource.tableCode / field.columnCode values would change.

So basically, yes, this kind of cacheing should be achievable for you.

Regards
Isomorphic Software

[Blama]

Hi Isomorphic

thank you for the detailed answer.
I implemented it as you suggested with a cache buster for the SmartClient version, the user name, the request language and the last-changed-date for my .ds.xml-files (I'm not using autoDeriveSchema).

I left the DataSourceLoader servlet protected like discussed here and manually set the header in my httpd vhosts file.

It is working fine and has an impact you can measure (but not feel). Load time went from 2,2sec -> 2sec.
But it is impressive to have a full enterprise application with current data with as little as 15kb of transferred data after switching to DS-caching (50kb otherwise, which is also impressive).

Best regards
Blama