Valid parameters for DMI remove

[mathias]

Hi, i looked around in DMI/Springintegration but couldn't find the section about parameter matching, so:

For DMI/Spring integration, i would like to be able to define the remove operation like this:

public Long remove(Long id) { }

But i can only get it to work with a bean or DSRequest as parameter. Am i missing something? I basically want to specify the method to take the key only.

[Isomorphic]

There would be no way to directly call such a method, you would need to either create a different signature or use Server Scripting.

We have not tried to make this possible via DMI since, outside of simplified samples, any real persistence system is generally going to need something from the DSRequest.

[mathias]

I would disagree for delete operations. For rest API's for example, you usually just perform a http delete request with /url/{id}. But thanks, then i know!

[Isomorphic]

An inadvertently perfect example!

That’s how you design a REST architecture if you don’t realize that a real system needs to submit transactions with mixed operation types, such as a mixture of updates and deletes.

Separate URLs for each operation type prevents this. Not to mention the lack of oldValues for implementing “long transactions”. And no plan for multiple primary keys, or criteria-based delete.

Toy architecture! :)

[mathias]

Is intimidating customers really working for you? I know how you feel about transactions, it's not new.

I am not exactly the only one in the world using REST. Rest HAS separate URLs, since you use HTTP post for add, put for updates and delete for deletes. Also, put/post operations have beans, delete do not necessarily need them depending on the circumstances. For my purposes, rest is working great where i am using it. Your approach is great in many instances, when it is needed. When it is not needed, it's super verbose for no benefit.

It's trivial to have more complex key than in my example (or criteria), but i do not need it in this case.


I guess we have to agree to disagree on whether my system is "real" or a "toy architecture". It's truly mind-boggling to me that you would post a message like this to a customer.

[Isomorphic]

We're not intending to intimidate customers, rather share our (very very extensive) experience.

We are *extremely* familiar with the REST architecture you talk about. It's very popular. That can be true and yet it can still be true that, as far as enterprise business applications, it's the wrong architecture. This is an industry where whole groups of people get together and routinely all make the same mistake - look at all of our dead competitors, for example!

To give you some background, we have seen many projects start with the REST architecture you talk about. We will argue against it because it doesn't handle many common cases. We are then assured that those cases aren't common, and if they are, they can be handled specially.

As the project matures, then there is a slowly accruing list of "exceptions". Multiple-record updates and deletes. Cases of using AdvancedCriteria, multi-level sort.

Then it gets near the end of the project and profiling is going on. Now there are tons of cases where operations need to be combined together into a single HTTP request for efficiency, more exceptions.

Then, first deployment and oops, we have several cases that need to be transactions and can't be separate requests, or we will have database inconsistency or even deadlocks. More exceptions.

Finally, when the project is complete, there are more cases of "exceptions" than there uses of the "standard" REST architecture. And there's a colossal amount of code for the "exceptions" because they are mostly hand-coded for a particular case (eg a special update that returns data and can only do so for the particular entities involved!).

If you use our RESTDataSource you can avoid all of this. It handles every use case mentioned above and many more that are far more common than you might guess.

So yes, we are absolutely telling you that, when it comes to enterprise business applications, the most common REST architecture is a toy architecture, and it is the wrong architecture.

Our motivation is not to "intimidate" you but to save you from a mistake we have seen again and again and again.

And again.

Hope you avoid it!

[mathias]

Your product has many great features, no arguing there. I am also sure that you've seen tons of implementation mistakes by companies that benefits from your transaction and operation management.

Stating that an example of a rest delete is a "perfect example" of not realizing what a "real system" needs is something i do not agree with, nor that you can issue blanket statements about what every "enterprise application" needs to do. It's only a problem if a system has needs that cannot be fulfilled by singular CRUD operations, which i am pretty sure is not the case for every company.

Our system has been deployed in the wild since 2011 using SmartGWT (among other technologies) and we've so far never scratched our heads and felt that our crud approach is a mistake. I guess it's not too late for me to realize the error of my ways! Who knows what the future will hold.

[Isomorphic]

100% of the projects Isomorphic has been involved in where the simplified REST has been used have required a long list of "exceptions".

We've pointed out several cases above where simplified REST can't handle the requests typical of an enterprise UI, there are dozens more.

So for the edification of readers, perhaps you can explain how you got around *all* of those use cases without a single exception to simplified REST?

For example, simplified REST assumes fetch requests will be HTTP GET operations, but it's not safe to serialize AdvancedCriteria into the URL because of request length issues. So you basically can't use AdvancedCriteria. Our ListGrid with a FilterEditor will now produce AdvancedCriteria by default since users can pick filter operators; even with filter operators turned off, AdvancedCriteria comes up with date fields in grids and SearchForms, and you'd of course have to avoid using the FilterBuilder, multi-field filtering in ComboBoxes, and many other powerful features.

So did you just turn all of those features off? Or is your UI so simple it doesn't have a filterable grid at all?

Or perhaps you implemented what we have had to implement on multiple occasions to get around simplified REST: carefully design your UI so that AdvancedCriteria is produced for only limited fields at a time, write some client side code to encode those limited criteria into a URL, and some server-side code to process AdvancedCriteria on just those fields, and live with a slightly crippled UI.

That's what we mean by having to write "extensions" - writing code that is completely unnecessary if you use our RESTDataSource protocol, and also crippling your UI, just to fit into a simplified REST architecture.

We covered several other cases already.. again we have had to adapt SmartGWT UI to simplified REST multiple times, so we know all the horrible things that need to be done.

To sum up: SmartGWT components have powerful productivity features that *clearly require* a richer interface to the server than simplified REST. You either cripple the UI or you write special code to adapt to simplified REST.

Since those are the only two alternatives, we're not sure how you can be declaring success with simplified REST. Can you explain?

[mathias]

Sure, happy to give some details.

First of all, a general remark. Your generic statement "how i got around ALL those usecases" is a logcial fallacy. As i mention above, it's only a problem if it is a problem. I only need to get around the use cases i have a need for, not anything else. And this should hold true for anyone.

As for us, our decision to go with smartGWT from the beginning was to be quicker and more productive on the *client side* compared to having to learn javascript, pick the right library etc. Also, there's *nothing* like your widgets, especially the listgrid component that i've found anywhere else. We're a small company personnel-wise and looking back today, this was the right choice then, the javascript world is still a nightmare :). (The main drawback from our perspective is the look and feel of smartgwt, it feels dated IMHO).

I have no reason to disbelieve your statement about "100%" of customers, but perhaps it is that your product tends to attract those that actually have a great need for your transaction/client-server handling?

Now, regarding filters/advanced criteria:

I have a set of criteria i need in some display scenarios (grids or similar) like "userid", "locationId" , date period etc. I also occasionally have the need for other criteria/parameters, an example would be "include breaktime" or "only items with warning".

In my IOS/Android apps, I pass them as HTTP headers and Spring retrieves them and populates my web controller method parameters automatically. I could also have put them in the http body, and spring could automatically populate a "criteria bean" or something, but i have not had the need so far. (I'm pretty sure that the current rest spec does not prevent you from sending a http body.)

In smartGWT, i don't use the filter editors in my grids (we didn't think they were super user friendly back when we made them), i use dropdowns and calendar popup widgets in a "filter section" above the grids. I do have textfield filters, but i put those in the dropdowns themselves, so that the user can type to select items from the dropdown.
When the uses selects a combination to filter on, i put a criterias on the grid, which is then sent when fetching stuff from the server. On the server, I then have specialized smartGWT web controller methods that traverse the DSRequest to get the equivalent parameters out, then call the same middle layer service methods that my rest controllers call.

So, the most "advanced criteria" i need is basically a bunch or "AND|OR"'s. They are used in our middle layer to fetch the correct data and return it to the client.

---
I'm not saying we've made the perfect call in every detail, definitely not. I'm just saying that the client-server communication we're using is working for us, given how our system works in total. I have so far not felt constricted, we have definitely never had to "code around" just to be able to use rest.

However, the bulk of the coding of the grids in our web was done 5-7 years ago (but have of course been updated periodically), and smartGWT had less features then. We're currently investigating on a remake/update of our web application, and if we continue to go with smartgwt it will be a good time to re-evaluate our choices.

[Isomorphic]

Our product is for enterprise business applications, so we definitely attract people building enterprise business applications. In such applications, these use cases come up.

Just as they did in your application. The only use case we've looked at - AdvancedCriteria - came up.

And since you were trying to use simplified REST, you were forced to do exactly what we said you would have to do: you had to turn off features of the UI (limiting search operators) and extend the simplified REST protocol by smuggling criteria in HTTP headers.

We're glad you ended up making this work. But if you compare it to our recommended architecture - just using a grid with FilterEditor and the RestDataSource - you could have had a grid with greater search capability and a pre-built protocol, instantly. You then could have moved *immediately* to coding the server logic, with a well-documented protocol spec in hand, without any work on protocol design or on client-side code to specially serialize your requests (or, with our server product, you could have connected that grid, with full search, sort and paging behaviors, to a pre-existing SQL table or JPA/Hibernate bean with literally a one line DataSource file, via autoDeriveSchema - but that's a separate issue).

We've identified this much unnecessary effort and lost capability with just the first of ~20 typical use cases. By the time we ran through all the use cases we'd probably find several more instances of unnecessary effort and lost functionality.

So that's why we give the advice we do.

We'd prefer not to critique your approach further, but we need to drive home for others why we give this advice...

One of the main reasons given for using simplified REST is that it's easy to see what an application is doing from simple HTTP logs (the CRUD operations line up with HTTP verbs, etc). By placing part of the request in the URL string and part in the HTTP headers, you've partially broken this. Now you've got part of the request information in the URL, part of it in the headers, and in the headers, it's mixed in with random other things like cookies.

Surely no one would claim this is easier to read than the simple XML and JSON formats used by RestDataSource? That puts the information all in one place, and the format is uniform for all request types, and also uniform for situations like an update combined with a fetch for related data.

So by extending simplified REST via smuggling extra information in HTTP headers, but not going all the way to a uniform request body as we recommend, you've ended up with something that is less readable than either alternative.

Finally, please don't feel attacked, and please recognize what we're doing here: you're posting publicly that a strategy that has been a *disaster* for many projects, and which we strongly caution against, worked fine for you. Well, no it didn't - not relative to what could have been. And if you make people believe it works fine, you will harm many other customers, and ultimately Isomorphic.

This is our motivation for critiquing your approach, and there is no other.

[mathias]

Don't worry, I don't feel attacked, you're arguing your case in a very civilized manner. :) The arguments you're making, and the features you point out are not news to me. I too do not want to critique anything about your great product. We're a customer since 2010 and have seen the way you've diligently kept adding features over the years.

Let's just say that i am not in complete agreement with your characterizations. I would be very happy to discuss offline with you guys why the way we want our arcitechture to function does not make restdatasource and the autoderiveschema/jpa beans a great fit (*for us*), and other details. Cheers.

[Isomorphic]

There are some (rare) architectures in which autoDeriveSchema doesn't fit, because there is neither a suitable Java bean or a SQL table. However, we've found that roughly 3/4 of the time that people decide autoDeriveSchema won't work, they are simply unaware that they can add their own fields and override the defaults. So they think detecting an internal field is a dealbreaker (you just set hidden="true") or that a hierarchical subobject is an issue (you just use valueXPath).

As far as not using RestDataSource for projects that don't have our server framework, we're not aware of even a single instance where this has been a good idea (including your application, for the reasons explained above).

Every time, not using RestDataSource has meant, at best, significant extra work and a reduction in power and flexibility, or, at worst, completely killing the project due to repeatedly having to implement special server support for interactions that are handled easily with the RestDataSource protocol.

For anyone arriving at the end of this thread: by far the #1 reason people struggle with SmartGWT is that they simply didn't follow our best practices, and they run right into the problem that the best practices are meant to prevent. If for no other reason, please follow best practices because you can imagine how that feels for our team :)