MockMvc no longer handles UTF-8 characters with Spring Boot 2.2.0.RELEASE

Viewed 23778

After I upgraded to the newly released 2.2.0.RELEASE version of Spring Boot some of my tests failed. It appears that the MediaType.APPLICATION_JSON_UTF8 has been deprecated and is no longer returned as default content type from controller methods that do not specify the content type explicitly.

Test code like

String content = mockMvc.perform(get("/some-api")
            .contentType(MediaType.APPLICATION_JSON))
            .andExpect(status().isOk())
            .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8))
            .andReturn()
            .getResponse()
            .getContentAsString();

suddenly did not work anymore as the content type was mismatched as shown below

java.lang.AssertionError: Content type 
Expected :application/json;charset=UTF-8
Actual   :application/json

Changing the code to .andExpect(content().contentType(MediaType.APPLICATION_JSON)) resolved the issue for now.

But now when comparing content to the expected serialized object there is still a mismatch if there are any special characters in the object. It appears that the .getContentAsString() method does not make use of the UTF-8 character encoding by default (any more).

java.lang.AssertionError: Response content expected:<[{"description":"Er hörte leise Schritte hinter sich."}]> but was:<[{"description":"Er hörte leise Schritte hinter sich."}]>
Expected :[{"description":"Er hörte leise Schritte hinter sich."}]
Actual   :[{"description":"Er hörte leise Schritte hinter sich."}]

How can I get content in UTF-8 encoding?

8 Answers

Yes. This is problem from 2.2.0 spring-boot. They set deprecation for default charset encoding.

.getContentAsString(StandardCharsets.UTF_8) - good but in any response would be populated ISO 8859-1 by default.

In my project I updated current created converter:

@Configuration
public class SpringConfig implements WebMvcConfigurer {

    @Override
    public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
        converters.stream()
            .filter(converter -> converter instanceof MappingJackson2HttpMessageConverter)
            .findFirst()
            .ifPresent(converter -> ((MappingJackson2HttpMessageConverter) converter).setDefaultCharset(UTF_8));
    }
...

Using .getContentAsString(StandardCharsets.UTF_8) instead of .getContentAsString() resolves the problem.

The default encoding character is no longer UTF-8 since the 5.2.0 version of spring.

To continue using UTF-8, you must set it in the ServletResponse of the MockMvc result. To set the default character encoding to UTF-8, do something like this in your setup method:

@Before
public void setUp() {
   mockMvc = webAppContextSetup(wac).addFilter(((request, response, chain) -> {
                response.setCharacterEncoding("UTF-8");
                chain.doFilter(request, response);
            })).build();
}

Then you can use the mockMvc instance to perform your request.

Hope this help.

I am using Spring Boot 1.5.15.RELEASE and faced the same issue when writing tests.

The first solution that helped me was to add .characterEncoding("UTF-8")) like this:

String content = mockMvc.perform(get("/some-api")
            .contentType(MediaType.APPLICATION_JSON)
            .characterEncoding("UTF-8"))
            .andExpect(status().isOk())
            .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8))
            .andReturn()
            .getResponse()
            .getContentAsString();

I am using a StandaloneMockMvcBuilder in my test class, so the second solution that helped me was to create a filter e.g.:

private static class Utf8Filter extends OncePerRequestFilter {

    @Override
    protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
        throws ServletException, IOException {
        response.setCharacterEncoding(StandardCharsets.UTF_8.toString());
        filterChain.doFilter(request, response);
    }
}

and later add it to the standaloneSetup method in my test class like this:

@Before
public void setup() {
    MockitoAnnotations.initMocks(this);
    final SomeResource someResource = new SomeResource(someService);
    this.restLankMockMvc = MockMvcBuilders.standaloneSetup(someResource)
        .setCustomArgumentResolvers(pageableArgumentResolver)
        .setControllerAdvice(exceptionTranslator)
        .setConversionService(createFormattingConversionService())
        .setMessageConverters(jacksonMessageConverter)
        .addFilter(new Utf8Filter())
        .build();
}

To restore the original behavior (Content-Type=application/json;charset=UTF-8) and allow your tests to pass as is, you could do the following:

@Configuration
public class MyWebConfig implements WebMvcConfigurer {

    @Override
    public void configureContentNegotiation(ContentNegotiationConfigurer configurer) {
        configurer.defaultContentType(MediaType.APPLICATION_JSON_UTF8);
    }
...

However, since APPLICATION_JSON_UTF8 is deprecated and Spring's intention is not to include the charset, it might be better going forward to modify your tests. black4bird's solution worked for me.

Following on black4bird's answer, you can override character encoding for all your tests by placing a following MockMvcBuilderCustomizer implementation in your test Spring context:

@Component
class MockMvcCharacterEncodingCustomizer implements MockMvcBuilderCustomizer {
    @Override
    public void customize(ConfigurableMockMvcBuilder<?> builder) {
        builder.alwaysDo(result -> result.response.characterEncoding = "UTF-8");
    }
}

That might help if you don't want to explicitly setup MockMvc, and just use @AutoconfigureMockMvc.

According to this pull request from the spring developers the UTF-8 header is not required anymore and therefore it's deprecated. If you're using the UTF-8 header in your application you might consider removing it from your application instead of trying fix your test. Just make sure you're using the Content-Type: application/json header and you should be fine.

Additional setting to MockMvc, .accept(MediaType.APPLICATION_JSON_UTF8_VALUE):

    String content = mockMvc.perform(get("/some-api")
        .contentType(MediaType.APPLICATION_JSON)
        .accept(MediaType.APPLICATION_JSON_UTF8_VALUE))
        .andExpect(status().isOk())
        .andExpect(content().contentType(MediaType.APPLICATION_JSON))
        .andReturn()
        .getResponse()
        .getContentAsString();

This problem is not Spring Boot, but MockMvc specific one, I guess. So, a workaround must be applied to MockMvc only. (JSON must be encoded using UTF-8.)

related issue: Improper UTF-8 handling in MockMvc for JSON response · Issue #23622 · spring-projects/spring-framework

Related