True Content-Negotiation [SPR-13679]

[spring-projects-issues]

Enrique Ruiz (DiSiD) opened SPR-13679 and commented

Spring MVC provides three ways to support Content-Negotiation but IMHO it isn't a true CN because really only one method in the Controller cannot handle several formats requested by the user.

Due to the current CN depends on annotations in the handler methods a developer has to create 4 methods to support all the possible combinations:

• @RequestParam + @ResponseBody: received data as parameters and returned data as a JSON message
• @RequestParam + View: received data as parameters and returned data as HTML
• @RequestBody + @ResponseBody: received data as JSON message and return message as a JSON message
• @RequestBody + View: received data as JSON message and return message as HTML

Although you can delegate between these methods you have to maintain 4 methods that potentially will change along the time and could evolve to different implementations, so really you are duplicating control logic.

What is true CN? For me true CN is to choose the conversion and binding of the incoming parameters or the conversion and binding of the request body depending on the HTTP "Content-Type" header property of the request.

In the same way, the conversion and formatting of the model objects included in the response could be by using a View or a MessageConverter depending on the HTTP "Accept" header property of the request.

Of course, the logical view-name should be infered from the URL automatically if needed, for example by using a DefaultRequestToViewNameTranslator.

By doing that only one handler method is needed and we won't duplicate control logic. For example:

@Controller
class AccountController {

  @RequestMapping("/accounts")
  @ResponseStatus(HttpStatus.OK)
  public List<Account> list(Principal principal) {
    return accountManager.getAccounts(principal);
  }

}

---

Affects: 4.2.2

Issue Links:

• Controller method argument initialized either from the request body or from request parameters [SPR-13742] #18315 Controller method argument initialized either from the request body or from request parameters ("is superseded by")

5 votes, 5 watchers

[spring-projects-issues]

Rossen Stoyanchev commented

Content negotiation is about clients being able to request different representations of a resource (see wikipedia for example). It has nothing to do with the input which is fixed and there is nothing to be negotiated (it is what it is). So your question can be split into two -- one processing different kinds of input and two serving different representations of a resource.

@RequestParam vs @RequestBody ... can you provide more detail on what use cases you're trying to cover? A request parameter in the Servlet API could be from the query string or form data. Form data typically might be a human filling out an HTML form and submitting it with POST. The request body (with something like json content) on the other hand would be the first choice with a REST API. It's usually one or the other but not both. So please provide some more context here.

@ResponseBody vs view resolution ... same question about more context on what you're building. For an HTML-based application (used by humans) I would recommend using view resolution with the ContentNegotiatingViewResolver allowing you to also render as JSON, XML, PDF, etc.

[spring-projects-issues]

Enrique Ruiz (DiSiD) commented

Hi Rossen,

A common Use-Case is the multi-device applications. In these cases the user access to the application via web browser from a PC and uses HTML forms to manage the domain entities, what it is also know as CRUD operations.

Moreover the user has a native mobile application (Android, iOS, ...) to execute the same CRUD operations but via AJAX and using JSON messages. All remains the same, but we have to duplicate the control logic.

On other side, at Spring Roo project we're working in the version 2.0 that will include several improvements as the generation of Spring Boot applications or the full separation of the Control-Layer from the View-Layer. The goal of the full separation of the C-L form the V-L is to let the developers can generate a full web app Control-Layer > Service-Layer > Repository-Layer with the benefit of letting the developer can choose the View technology, as JSP, Thymeleaf, AngularJS, etc.

Technically the full separation of the C-L from the V-L implies that a handler method in the Controller must support any request and response. Note each view technology has its own way to call the server, so the support of several view technologies in Spring Roo means different handler methods should be generated in anticipation that the developer can chose the View technology, this could be avoided if Spring MVC would include the feature to chose the handler method, the MethodProcessor, the response generator, etc. based on the request headers and not to chose them based in method annotations.

[spring-projects-issues]

Rossen Stoyanchev commented

Thanks for the extra detail.

For a Web browser application serving HTML forms the controller logic is different. A model is prepared with multiple attributes -- the form-backing object (yes) but also other objects to help populate drop-down boxes, etc. After the form is submitted, you wouldn't want to use @RequestParam but rather @ModelAttribute with binding and validation. Then the controller returns to the same form view to display errors or in case of success redirect to another page (the POST-REDIRECT-GET pattern):

@RequestMapping(path="/accounts", method=POST)
public String addAccount(@Valid @ModelAttribute Account account, Errors errors) {
    if (errors.hasErrors) {
         return "editAccount";
    }
    // Save the account
    return "redirect:/accounts/{id}";
}

In a REST API there is no need to populate a model with extra attributes. No need to re-display the form with errors. No need to redirect either on success. Rather you return 201 or 400 with error details in the response body. You can add things like HATEOAS as well here that don't apply to HTML-based apps.

If you build an HTML-based Web application for humans, you can add the ContentNegotiatingViewResolver and that will allow you to get non-HTML representations for methods mapped to HTTP GET. However if also want to support a full-fledged REST API in addition I'd recommend a separate URL space, and separate controller logic.

In summary it's really a two-way (and not a 4-way) split between controller logic Web apps with HTML forms for use by humans vs REST API for use by programmatic clients.

[spring-projects-issues]

Enrique Ruiz (DiSiD) commented

Hi Rossen,

It is true that the developers could want to apply the post-redirect-get pattern, so in that case they must to add a new handler method to apply it.

But the web applications aren't composed by update/create forms only, the read and query actions are needed and in those cases the developers could have to duplicate the controller logic up to 4 times.

I will try to explain myself better, please take a look to the article Content Negotiation using Spring MVC : Combining Data and Presentation Formats, what I'm proposing is an improvement to avoid the duplicated controller logic, as shown in this article.

Let me simplify this example:

@Controller
class AccountController {

    @Autowired
    protected AccountService accountService;

    /**
     * METHOD A: RESTful method
     * TEST: 
     * curl -i -H 'Accept: application/json' http://localhost:8080/accounts?name=TheCustomerName
     */
    @RequestMapping(value="/accounts", produces="application/json")
    @ResponseStatus(HttpStatus.OK)
    public @ResponseBody List<Account> listWithMarshalling(Customer customer) {
        return accountService.findAllAccounts(customer.getName());
    }

    /**
     * METHOD B: View-based method
     * TEST: 
     * curl -i http://localhost:8080/accounts?name=TheCustomerName
     */
    @RequestMapping("/accounts")
    public String listWithView(Model model, Customer customer) {

        // Call RESTful method to avoid repeating account lookup logic
        model.addAttribute( listWithMarshalling(customer) );

        // Return the view to use for rendering the response
        return "accounts/list";
    }
}

These methods let you to handle the following requests:

• METHOD A

  • URL: http://localhost:8080/accounts?name=TheCustomerName
  • HTTP request header: "Accept: application/json"
  • INPUT: HTTP parameters
  • OUTPUT: JSON message

• METHOD B

  • URL: http://localhost:8080/accounts?name=CustomerName
  • HTTP request header: none
  • INPUT: HTTP parameters
  • OUTPUT: HTML

But the requeriments to create web applications are many and various, for example, moreover the above use cases, if you would need to handle the requests below:

• METHOD C
  • URL: http://localhost:8080/accounts
  • HTTP request header: "Content-Type: application/json" and "Accept: application/json"
  • INPUT: JSON message
  • OUTPUT: JSON message
• METHOD D
  • URL: http://localhost:8080/accounts
  • HTTP request header: "Content-Type: application/json"
  • INPUT: JSON message
  • OUTPUT: HTML

The handler methods should be something like this:

@Controller
class AccountController {
    ...

    /**
     * METHOD C: RESTful method
     * TEST: 
     * curl -i -H 'Content-Type: application/json' -H 'Accept: application/json' -d '{"name":"TheCustomerName"}' http://localhost:8080/accounts
     */
    @RequestMapping(value="/accounts", produces="application/json", consumes="application/json")
    @ResponseStatus(HttpStatus.OK)
    public @ResponseBody List<Account> listWithMarshalling(@Valid @RequestBody Customer customer, BindingResult errors) {
        return accountService.findAllAccounts(customer.getName());
    }

    /**
     * METHOD D: View-based method
     * TEST: 
     * curl -i -H 'Content-Type: application/json' -d '{"name":"TheCustomerName"}' http://localhost:8080/accounts
     */
    @RequestMapping(value="/accounts", consumes="application/json")
    public String listWithView(Model model, @Valid @RequestBody Customer customer, BindingResult errors) {

        // Call RESTful method to avoid repeating account lookup logic
        model.addAttribute( listWithMarshalling(customer) );

        // Return the view to use for rendering the response
        return "accounts/list";
    }
}

This is why I said you have to duplicate the controller logic up to 4 times.

Note that in all cases, all remains identical: the data binding, the object validation, ... what it only changes is how the data is in the HTTP request and the message of the response.

Rossen, note I think that Spring could manage all of them in only one controller method just with some info at the HTTP headers so I think it would be a great improvement that could be really useful for the developers.

The reality is that the HTML-based Web applications for humans is not the trend, the reality and the trend is that the Web applications must be ready for both html-based interaction as for the RESTful API interaction. In terms of application functionality, the clients require the same features via web, via mobile applications and in general via full RESTful API. This issue propose to improve code maintenance when it can be achieved, as for example on the control logic to read and to query data.

Finally, another use case is AngularJS which works via REST API and JSON messages, by having the proposed improvement a web application which View-Layer is based on JSP could be migrated to a V-L based on AngularJS easily, without haven't to modify the Controller-Layer.

[spring-projects-issues]

Rossen Stoyanchev commented

INPUT: HTTP parameters

What do you mean by "HTTP parameters"? The Servlet API has request parameters, I'm guessing that's what you mean. Those can come from query parameters or from form data in the body of the request. I think what you mean is the latter which in turn means method A and B should have Content-Type: application/x-www-form-urlencoded and input is data binding via "request parameters".

1. @RequestBody + @ResponseBody covers method A and C.
2. @ModelAttribute + view resolution covers B.

I question the need for method D since HTML-based applications send form data. Unlike a REST API which needs to support different kinds of client implementations who may send different kinds of input, in an HTML-based application it's the application serving the HTML and generally having one kind of input (form data) is what makes sense.

Rather than looking at all permutations of the programming model, from an HTTP perspective (nothing to do with Spring MVC) I see two main usage styles. One is HTML-centric and the other follows REST conventions. There are enough differences between the two that merit differences in the programming model. This is not just about the POST-REDIRECT-GET pattern but also about basic things like the status codes used, the way errors are handled, etc (I gave examples of these already). When it comes to POST-REDIRECT-GET it really should be the default way to do things in an HTML-centric approach, not just something you might want to do.

[spring-projects-issues]

Enrique Ruiz (DiSiD) commented

Yes, I meant HTTP Query parameters and HTTP Form data. What I'm not sure is why you are centered in form data only ... but anyway.

@RequestBody + @ResponseBody doesn't cover method A and C:

• The method A receives the data as HTTP Query parameters (as I illustrated in the Javadoc of that method). Note that due to the Customer argument is not annotated (neither @RequestParam nor @ModelAttribute) Spring MVC uses the ModelAttributeMethodProcessor for populating it (data binding) from String-based values extracted from the request parameters. Note for HTTP Servlet API, parameters are contained in the query string or posted form data (take a look at ServletRequest.html#getParameter())

• On the other hand in the method C, the Customer argument is annotated with @RequestBody, so Spring MVC uses the RequestResponseBodyMethodProcessor for populating it (data binding) from String-based messages, for example JSON messages, extracted from the request body but not extracted from the request parameters.

IMHO Spring MVC has plenty of similar features that makes the live of the developer easier, as for example the DefaultRequestToViewNameTranslator class. Moreover and beyond my explantion, I think the problem of having duplicated controller logic is perfectly documented at the Spring blog post Content Negotiation using Spring MVC : Combining Data and Presentation Formats

Deeply I think you and me have different view of the current applications, you see different programming models one HTML-centric and other REST. I see the same programming model, applications with an high percentage of controller logic duplicated when you combine both.

This issue is about improve Spring MVC to support Spring MVC applications that combines HTML with RESTful API in an easy way without having to duplicate the controller logic. But without loosing the possibility to customize each model by using the current way.

Take a look and test this Spring Boot application that illustrates the problem. Please debug and run the request tests.

All the best,

[spring-projects-issues]

Rossen Stoyanchev commented

I focus on form data because I'm thinking of the HTTP POST and PUT, those are the key areas where you have differences in controller logic and response status codes. Getting a controller method for GET methods isn't hard.

The example Spring Boot application does not make sense without further explanation. What is the "/accounts" endpoint? Is that a listing of accounts for a customer, in which case shouldn't there be something (like a path variable) that identifies the customer in the URL? It's not clear why identifying which customer has to come as JSON in the request body or require data binding?

[spring-projects-issues]

Enrique Ruiz (DiSiD) commented

I focus on form data because I'm thinking of the HTTP POST and PUT, those are the key areas where you have differences in controller logic and response status codes.

Yes, but as I said in one of my previous comments, this proposal doesn't prevent the developer to customize the handler methods, for example for managing specific cases.

The example Spring Boot application does not make sense without further explanation.

Well, it is just an example that illustrates the problem, I will try to explain further below.

What is the "/accounts" endpoint? Is that a listing of accounts for a customer, in which case shouldn't there be something (like a path variable) that identifies the customer in the URL?

The meaning of the "/accounts" follows the common RESTful pattern:

• GET /accounts - Retrieves a list of accounts
• GET /accounts/12 - Retrieves a specific account by account number
• GET /accounts?name=TheCustomerName - Searches for accounts with a certain Customer
• POST /accounts - Creates a new account
• PUT /accounts/12 - Updates account Work in Progress for SPR-8986 #12

Note, in the given Spring Boot application I only implemented the methods to manage the searches for accounts with a certain Customer.

It does not matter if the data comes as request parameter or comes as path variable, the problem is exactly the same. But yes, it is true that for the read method, for example "/accounts/{number}", you will have 2 duplicated controller methods only.

It's not clear why identifying which customer has to come as JSON in the request body or require data binding?

IMHO is the best way to avoid duplicated conversion logic. It does not matter from where the data comes: search form, AJAX search, ... there is no need to have methods like this:

@RequestMapping("/accounts")
public String listWithView(Model model, @RequestParam String customerName,
        @RequestParam @DateTimeFormat(pattern="MM/dd/yyyy") 
          Date customerBirthdate, 
        ...) {
}

Just set up the Customer entity with the right annotations and the DataBinder, ConversionService, etc. will do the rest. IMHO this pattern does the application more maintainable.

[spring-projects-issues]

Rossen Stoyanchev commented

Since I do not understand what you're trying to do in general, I will simply respond with this commit that reduces your example two controller methods, one accepting input from request parameters and the other from the request body (plus the ContentNegotiatingViewResolver config):

    @RequestMapping(value="/accounts")
    public String listWithRequestParams(Customer customer, Model model) {
        return listInternal(customer, model);
    }

    @RequestMapping(value="/accounts", consumes="application/json")
    public String listWithRequestBody(@RequestBody Customer customer, Model model) {
        return listInternal(customer, model);
    }

What's not clear is how each of these scenarios will be used from the client side? For example I can understand the reason for request parameter input with HTML output as well as JSON input and output. What's making use of request parameter input + JSON output? That sounds like a REST or Ajax call based on how it deals with the response in which case why not expect JSON input there which also sets up the right expectations for how to report errors (i.e. for consumption by programmatic client)? Same question for JSON input with HTML output. Based on the response it sounds like a response for rendering in a browser in which case there would be a search form presumably?

[spring-projects-issues]

Enrique Ruiz (DiSiD) commented

What's making use of request parameter input + JSON output? That sounds like a REST or Ajax call based on how it deals with the response in which case why not expect JSON input there which also sets up the right expectations for how to report errors (i.e. for consumption by programmatic client)?

Several advanced UI components use this way. Note that in these cases a developer cannot decide to use JSON input message or input request parameters, the API of the UI components is who set the data input format.

Just an example, one of the best jQuery plugins for managing tables is Datatables. Take a look to this Datatables example, enter text in the Search input and you will see how the filtering data is sent as HTTP query parameters and the response is given in a JSON message. Another datatables example, enter the same text in the Search input and you will see how the filtering data is sent as HTTP form data and the response is given in a JSON message too.

Same question for JSON input with HTML output. Based on the response it sounds like a response for rendering in a browser in which case there would be a search form presumably?

As an example take a look to the function jQuery .load(), it does an AJAX request and it expects an HTML message that it is rendered inside a container of the page.

This pattern is applied in UI components too, for example, you take a look to the gvNIX petclinic demo create Pet form, the owner field is a search field, just press the loupe and you will see how the HTML response of an AJAX request is rendered inside a dialog component.

[spring-projects-issues]

Rossen Stoyanchev commented

Okay that puts the discussion on much more concrete terms. So far what I see are examples of retrieving data for programmatic use by UI components in a browser. The 2 controller methods I showed above are the best (that I can think of) with out of the box behavior. For that note there is no duplication of controller logic.

Going to a single controller method where the Customer object is initialized either from request parameters or from the request body is not that hard at all from a technical perspective (more on that shortly). It's more an issue of semantics. For once a Customer argument without annotations already has a set meaning that we can't all of a sudden change. More importantly though I think a clear separation of initialization via data binding from request params or through the request body is a a good (and intuitive) foundation. Not having such clear semantics would certainly lead to many surprises or impossible cases (e.g. both request body and request parameters).

On top of this it's easy to create a custom annotation such as @AjaxAttribute with the combined semantics of @ModelAttribute and @RequestBody. The custom HandlerMethodArgumentResolver backing it might delegate to ModelAttributeMethodProcessor or RequestResponseBodyMethodProcessor respectively. Or it might wrap the request InputStream making the body look like form data based on query parameters (note that we already do that for POST requests although for different reasons, see ServletServerHttpRequest#getBodyFromServletRequestParameters).

We could even consider providing such an annotation out of the box, if it seems to be generally useful. Creating your own in the mean time should be reasonably straight forward.

[spring-projects-issues]

Enrique Ruiz (DiSiD) commented

For once a Customer argument without annotations already has a set meaning that we can't all of a sudden change.

Ok, I understand ... I suppose this is the reason because a solution based on a new annotation is needed.

More importantly though I think a clear separation of initialization via data binding from request params or through the request body is a a good (and intuitive) foundation. Not having such clear semantics would certainly lead to many surprises or impossible cases (e.g. both request body and request parameters).

Yes, for avoiding this impossible cases, you could take in account the request header attribute "Content-Type", for having a criteria to decide what are the data that must be used to populate an entity (data-binding) when the request has a body and request parameters together.

On top of this it's easy to create a custom annotation such as @AjaxAttribute with the combined semantics of @ModelAttribute and @RequestBody.

I agree with your proposal.

The custom HandlerMethodArgumentResolver backing it might delegate to ModelAttributeMethodProcessor or RequestResponseBodyMethodProcessor respectively. Or it might wrap the request InputStream making the body look like form data based on query parameters (note that we already do that for POST requests although for different reasons, see ServletServerHttpRequest#getBodyFromServletRequestParameters).

Totally agree. In the decission to delegate to MAMP or RRBMP you could take in account the request header attribute "Content-Type", for example.

We could even consider providing such an annotation out of the box, if it seems to be generally useful. Creating your own in the mean time should be reasonably straight forward.

Thanks Rossen for your support and patience :)

Rossen, IMHO the same improvement could be done for the response. For those methods that don't apply the POST-REDIRECT-GET pattern, a new annotation would be great with the combined semantics of @ResponseBody and the return of the View-Name. The custom HandlerMethodReturnValueHandler backing it might delegate to RequestResponseBodyMethodProcessor or ViewNameMethodReturnValueHandler respectively. The method structure would be equal to the methods annotated with @ResponseBody but in case it delegates to ViewNameMethodReturnValueHandler, the logical view name must be infered by a RequestToViewNameTranslator.

[spring-projects-issues]

Rossen Stoyanchev commented

No problem. I've created #18315 as a concrete actionable item.

As for alternating between @ResponseBody and a view name I don't think they can be combined like that. They're mutually exclusive by definition. The very meaning of @ResponseBody is that the return value will be processed with an HttpMessageConverter. In fact you can return HTML in a String from an @ResponseBody method. Furthermore as I demonstrated above, once #18315 is available, your use case should be down to a single controller method.