Use spring-data-elasticsearch annotations with Jackson ObjectMapper?

user_3715185 0
Viewed 3326

Suppose there are some POJOs to be written to an Elasticsearch index. From my limited experience with ES, each field gets a type (e.g. text, keyword, ...). Is it possible to configure the Jackson ObjectMapper with the annotations from spring-data-elasticsearch to customize the type of the serialized Json? As an example:

@Document(indexName = "books", type = "book")
public class Book {
    @Id
    @Field(type = FieldType.Keyword)
    String isbn;
    @Field(type = FieldType.Text)
    String title;
}

Then the mapping should include something like

{
  "mappings":
    {
        "book": {
            "properties": {
                "id": {
                    "type": "keyword"
                },
                "title": {
                    "type": "text"
                }
            }
        }
    }
}

Please let me know if I think about this wrong!

spring-data-elasticsearch elasticsearch jackson java
2 Answers

With Elasticsearch you can either let it define mappings automatically using Dynamic Mapping (you persist your docs and ES builds mapping according to it input), or you can define your schema prior to indexing by sending ES with relevant mappings/templates (.json files).

Using dynamic mapping option is less recommended and might cause issues with mappings (wrong date format, int/float issues, etc..)

I usually set my mappings without the dynamic feature - setting it to 'strict', this will raise Exception automatically by ES (and will propagate with any library you use)

"dynamic": "strict",

strict: If new fields are detected, an exception is thrown and the document is rejected. New fields must be explicitly added to the mapping.

With Spring-Data-Elasticsearch, you can define your Object just the way you have posted - annotating your fields with wanted types - this will cause ElasticsearchTemplate.java to parse your Object (take a look at public <T> boolean putMapping(Class<T> clazz) and public ElasticsearchPersistentEntity getPersistentEntityFor(Class clazz)) and feed it to ES.

You can verify spring-es index/mapping creation according to your annotations with ElasticsearchTemplate: (Test taken from Spring-Data-ES project)

@Test
public void shouldReturnMappingForGivenEntityClass() {

    // given

    // when
    boolean created = elasticsearchTemplate.createIndex(SampleEntity.class);
    elasticsearchTemplate.putMapping(SampleEntity.class);
    Map<String, Object> mapping = elasticsearchTemplate.getMapping(SampleEntity.class);

    // then
    assertThat(created).isTrue();
    assertThat(mapping).isNotNull();
    assertThat(((Map<String, Object>) ((Map<String, Object>) mapping.get("properties")).get("message")).get("type"))
            .isEqualTo("text");
}

And the SampleEntity:

@Document(indexName = "test-index-sample-mapping", type = "mapping")
static class SampleMappingEntity {

    @Id private String id;

    @Field(type = Text, index = false, store = true, analyzer = "standard")
    private String message;
}

I would recommend separating index construction from the application and not to use the implicit POJO annotation option - this will be easier to control later on.

user_81624070

I don't know which version of Spring Data Elasticsearch you are using; the support of using the Jackson Object Mapper together with the @Field annotation is limited.

If you are using 3.2.0RC1, then change to use the ElasticsearchEntityMapper (see reference docs how to configure it). Although the Jackson Mapper still is the default one if no other is configured, current work is done on ElasticsearchEntityMapper. I think after release of 3.2 we probably will change the default mapper to ElasticsearchEntityMapper.

I know the reference documentation still lacks information, I'm just in the process of reworking it.

When creating an index with the Repository classes from 3.2, the annotations like @Field are taken into account and the correct mapping is written to the index, no need to write the mapping by yourself.

user_43935650
Related