MockMvc doesn't use spring-boot's mvc exception handler

[xak2000]

I'm trying to test my Controller using MockMvc.
The service used by Controller throws RuntimeException if something is wrong.
The spring-boot's default exception resolver catches this exception and responds with status 500.
But when I use MockMvc in my test, the test method just throws the exception instead of catch status 500.
If I add explicit exception resolver using @ExceptionHandler, then test method starts to work.

The simple example. My Controller just throws new RuntimeException() for simplicity.

@RunWith(SpringRunner.class)
@WebMvcTest(MyController.class)
public class ControllerTests {

	@Autowired
	private MockMvc mvc;

	@Test
	public void testInternalServerError() throws Exception {
		mvc.perform(get("/api/data"))
				.andExpect(status().isInternalServerError());
	}
}

Expected result:
Test catches status 500 and passes.

Actual result:
The exception is thrown and test fails.

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.RuntimeException

Maybe I'm wrong assuming that spring-boot installs some default mvc exception handler. But anyway the real application returns status 500 with properly formatted JSON of error details. So I think the MockMvc should do the same.

I can add the reproducible example if someone confirms that this is really not intended behaviour.

[xak2000]

Also, even if I add explicit @ExceptionHandler like this:

    @ExceptionHandler(RuntimeException.class)
    public void hadnle(HttpServletResponse res) throws IOException {
        res.sendError(404, "bla bla");
    }

Then real response and MockMvc response is also different.

The real response contains JSON with error details, like this:

{
  "timestamp" : "2016-11-06T03:16:02.440+0000",
  "status" : 404,
  "error" : "Not Found",
  "exception" : "java.lang.RuntimeException",
  "message" : "bla bla",
  "path" : "/api/data"
}

But the MockMvc response body is empty:

MockHttpServletResponse:
           Status = 404
    Error message = bla bla
          Headers = {}
     Content type = null
             Body = 
    Forwarded URL = null
   Redirected URL = null
          Cookies = []

So we can't check, for example, the message field.

[bclozel]

This might be related to Spring Framework and it seems that MockMvc will follow redirects and forwards in the 5.0 version (see SPR-14342 and SPR-14755).

Could you share a sample repro project we could take a look at?
I'm not sure MockMvc is supposed to dispatch error requests back to the container, since we're not firing up a container for those tests.

[xak2000]

demo-mock-mvc-gh-7321.zip

• Try: gradle test and see the report.
• Try: gradle bootRun, then curl -i localhost:8080/unhandled and curl -i localhost:8080/handled and see the good JSON responses.

There are 2 tests here.

1. testUnhandledException().
   It tests the case when no explicit @ExceptionHandler provided by user code.

   This test doesn't work. It throws: org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.RuntimeException: Hello, unhandled Exception! while real request to the app (gradle bootRun, then curl -i localhost:8080/unhandled) shows good JSON response.

2. testHandledException().
   It test the case when explicit @ExceptionHandler (which just does res.sendError(400, "Explicit exception handler works!");) is provided by user code.

   This test also doesn't work. It not throws any exceptions. But checked response body is empty.

   java.lang.AssertionError: Response content
   Expected: not an empty string
    but: was ""
   

   But real request shows good JSON response:

   HTTP/1.1 400
   Content-Type: application/json;charset=UTF-8
   Transfer-Encoding: chunked
   Date: Thu, 17 Nov 2016 15:26:03 GMT
   Connection: close
   
   {
   "timestamp" : "2016-11-17T15:26:03.043+0000",
   "status" : 400,
   "error" : "Bad Request",
   "exception" : "java.lang.IllegalStateException",
   "message" : "Explicit exception handler works!",
   "path" : "/handled"
   }
   

I don't know the internals. Not sure how spring-boot provides this json response. But if you think it works as expected, then we can't rely on this behaviour in tests? It is inconsistent. :( How can we write tests for error conditions using default spring-boot JSON responses, then?

[philwebb]

/cc @rstoyanchev

[rstoyanchev]

Spring Boot's error handling is based on Servlet container error mappings that result in an ERROR dispatch to an ErrorController. MockMvc however is container-less testing so with no Servlet container the exception simply bubbles up with nothing to stop it.

So MockMvc tests simply aren't enough to test error responses generated through Spring Boot. I would argue that you shouldn't be testing Spring Boot's error handling. If you're customizing it in any way you can write Spring Boot integration tests (with an actual container) to verify error responses. And then for MockMvc tests focus on fully testing the web layer while expecting exceptions to bubble up.

This is a typical unit vs integration tests trade off. You do unit tests even if they don't test everything because they give you more control and run faster.

[wilkinsona]

Thanks, @rstoyanchev.

How can we write tests for error conditions using default spring-boot JSON responses, then?

@xak2000 Rossen's already covered this, but I wanted to give you a direct answer. If you really want to test the precise format of the error response then you can use an integration test using @SpringBootTest configured with a DEFINED_PORT or RANDOM_PORT web environment and TestRestTemplate.