Spring elastic : What is the correct way of using nested fields?

user_8792830 0
Viewed 451

I'm quite new to ElasticSearch. I'm working on a project where we need to search for an object (Offer) that contains a Set of two (OfferTranslation). The goal is to make research based on some of Offer fields, but also a lot of OfferTranslation fields. Here's a minified version of both classes :

Offer.class (note that I annotated the Set with @Field(type= FieldType.Nested) so i can make Nested queries, as mentionned in the official doc) :

@org.springframework.data.elasticsearch.annotations.Document(indexName = "offer")
@DynamicMapping(DynamicMappingValue.False)
public class Offer implements Serializable {

    private static final long serialVersionUID = 1L;

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    @Column(name = "id")
    @Field(type = FieldType.Long)
    private Long id;

    @OneToMany(mappedBy = "offer", targetEntity = OfferTranslation.class, cascade = CascadeType.ALL, fetch = FetchType.EAGER)
    @Cache(usage = CacheConcurrencyStrategy.READ_WRITE)
    @JsonIgnoreProperties(
        value = { "pictures", "videos", "owner", "contexts", "offer", "personOfInterests", "followers" },
        allowSetters = true
    )
    @Field(type = FieldType.Nested, store = true)
    private Set<OfferTranslation> offersTranslations = new HashSet<>();


}

OfferTranslation.class :

@DynamicMapping(DynamicMappingValue.False)
public class OfferTranslation implements Serializable {

    private static final long serialVersionUID = 1L;

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    @Column(name = "id")
    @Field(type = FieldType.Long)
    private Long id;

    @NotNull
    @Size(max = 100)
    @Column(name = "title", length = 100, nullable = false)
    @Field(type = FieldType.Text)
    private String title;

    @NotNull
    @Size(max = 2000)
    @Column(name = "summary", length = 2000, nullable = false)
    @Field(type = FieldType.Text)
    private String summary;

    @Size(max = 2000)
    @Column(name = "competitor_context", length = 2000)
    @Field(type = FieldType.Text)
    private String competitorContext;

    @NotNull
    @Size(max = 2000)
    @Column(name = "description", length = 2000, nullable = false)
    @Field(type = FieldType.Text)
    private String description;

    @NotNull
    @Enumerated(EnumType.STRING)
    @Column(name = "maturity", nullable = false)
    @Field(type = FieldType.Auto)
    private RefMaturity maturity;

    @ManyToOne
    @Field(type = FieldType.Object, store = true)
    private RefLanguage language;

    @NotNull
    @Column(name = "created_at", nullable = false)
    @Field(type = FieldType.Date)
    private Instant createdAt;
}

The expected behavior would be that i can make nestedQueries as so :

QueryBuilder qBuilder = nestedQuery("offersTranslations",boolQuery().must(termQuery("offersTranslations.language.code",language)), ScoreMode.None);

But what i get is an exception : failed to create query: [nested] nested object under path [offersTranslations] is not of nested type"

EDIT : I can access offersTranslations.language.code using normal queries (which doesn't really bother me at the moment). But I still don't really understand.

My mapping says the field offersTranslations is not of a nested type as you can see above, but since I used @Field(type = FieldType.Nested) i don't really understand that behavior. Could someone explain?

{
  "offer" : {
    "mappings" : {
      "properties" : {
        "_class" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "categories" : {
          "properties" : {
            "id" : {
              "type" : "long"
            }
          }
        },
        "criteria" : {
          "properties" : {
            "id" : {
              "type" : "long"
            }
          }
        },
        "id" : {
          "type" : "long"
        },
        "keywords" : {
          "properties" : {
            "id" : {
              "type" : "long"
            }
          }
        },
        "offersTranslations" : {
          "properties" : {
            "competitorContext" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "createdAt" : {
              "type" : "date"
            },
            "description" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "id" : {
              "type" : "long"
            },
            "language" : {
              "properties" : {
                "code" : {
                  "type" : "text",
                  "fields" : {
                    "keyword" : {
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                },
                "id" : {
                  "type" : "long"
                },
                "name" : {
                  "type" : "text",
                  "fields" : {
                    "keyword" : {
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                }
              }
            },
            "maturity" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "state" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "summary" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "title" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "updatedAt" : {
              "type" : "date"
            }
          }
        },
        "units" : {
          "properties" : {
            "id" : {
              "type" : "long"
            }
          }
        }
      }
    }
  }
}
spring-data-elasticsearch spring-data elasticsearch java
2 Answers

How was the index mapping created? It does not look like Spring Data Elasticsearch wrote this mapping. The nested type of offerTranslations is missing as you saw and the text fields have a .keyword subfield. This looks like data was inserted into the index without having a mapping defined and so Elasticsearch did an automatic mapping. In this case, the values of the @Field annotations are not used.

You need to have Spring Data Elasticsearch create the index with the mapping. This will happen automatically if the index does not exists and you are using Spring Data Elasticsearch repositories, or you need to use the IndexOperations.createWithMapping function in your application.

Another thing I noticed: it seems you are using the same entity class for different Spring Data stores, mixing heavily the annotations. You should use different entities for different stores.

user_43935650

STEPS TO SOLVE :

  • Use Kibana to make sure you DELETE <index_name>/_mapping
  • Look in your entity classes for objects you need that might be in a @JsonIgnoreProperties
  • Make sure you EAGERLY load your toMany relationship's attributes (otherwise elastic won't create a mapping for a data you never gave)
  • From this little experience, i'd say avoid using Nested fields, i couldn't see any advantage of using them. So check if that is the case for you too !
user_87928300
Related