Using many datasources with many fields

[u4ibm]

We're using SmartGWTEE and encountered errors with the datasources.
Our client browser is firefox 3 on windows.
My actual question are in bold

We were trying to scale our demo application to see if it can support the all the datasources we were planning to watch (around 300).
It appeared that as the server was going up, it's parsing the ds.xmls, but stopped after 100.
It didn't process some of smart's inner datasources, so the application wasn't functioning when trying to make server calls.

* Is there a way to make it support all the datasources
* Can the client load the datasources configuration only when it needs one, and not right in the beginning?

When we narrowed down the datasources to 90, problems continued:
Every refresh of the page caused the server to generate all the customDataSource instances of each datasource. It was very slow and almost caused timeout error.
* Why does it hold an instance per DS per session?
* Can I shared one instance of my CustomDataSource between all datasources that use this class?

Anyway, our datasources have around 50-100 fields each. some mostly empty, some very big.
it rendered the fetch results (about 25 record) very slowly, but once they were shown it was somewhat responsive.
When i tried to use autoFit on all fields, move a field, or sort the data - the CPU and memory consumption went VERY high (50% and 600MB)!! it took about a minute for each operation (25 records, I remind you..).
* Is this normal behaviour? Do you have experience with large datasources, and figured out best practices?


Thanks for the help,
Eyal.

[Isomorphic]

No, none of that is normal behavior and Isomorphic has built applications far larger than you describe. Next step is to isolate to a standalone test case.

Note, you can load DataSources on demand via the DataSource.load() API, however again, something is severely wrong with your basic setup and you need to isolate that first.

[u4ibm]

Hey,

We figured some things wrong on the server side and everything seems ok now (we called getConnection() in the Ctor of the CustomDataSource which extends SQLDataSource).

On the clientside however, it is still rendering very slowly.
To make sure the problem wasn't with our client code, I recreated the problem in BuiltinDS and it was slow there as well (firefox, explorer, chrome on Windows).
With about 100 fields (autofitted to their title) and 75 selected records, it was scrolling horizontally quite slowly. When trying to resize a field or hide it, it took about 30 seconds.
Again, this is on your builtinDS, with nothing from our code.
If you would like, we can generate a similiar DS though I can't generate the data. It is a mix of text, number and date fields.

Any offered steps? Changes to the configuration of the ListGrid?

Thanks!
eyal

[Isomorphic]

As documented, autofitting to titles has a performance impact. It should not be used for all fields in a ListGrid with lots of columns.