Elasticsearch : nested VS flat indicies

Totally agree with you about pros and cons of ES nested type. Just want to elaborate the deepth of indexing cost. Keep in mind nested field opens query facilities also

If you use nested type and infrequent modification then it is awesome and creates broader scope for query but if you do frequent change then it will do huge cost.

nested type mapping has more impact in terms of indexing over flat type mappings. Since Lucene does not have any concept of nested object types, and everything is stored as flat objects. So there is an additional operation performed at the indexing time.

Imagine you have a large nested document that translates to 100k internal documents and compare this to a flat data model where we have indexed the 100k parts as independent documents. If we add a single nested document at the deepest, nested level, this would add a single flat document while the nested document would end up reindexing 100k+ 1 documents. If you on the other hand change something at the root, all documents need to be updated in both cases. So you can imagine a single doc change can cost you reindexing all nested field of that document.

Let's talk about pros and cons among flat fields, dynamic, object, flattened, nested.

flat fields

pros: Simple and clear, field type is specified in mappings.

cons: New field should do put mapping at first. Number of fields is limited by engine. Different fields have no relationship.

dynamic

For new fields or inner object which have no mappings represented.

set to false, will not index new fields, but new fields can be retrieved by _source.

set to strict, will report an error when unknown new fields in doc.

cons: strict is not elastic meanwhile dynamic will lead a mapping explosion problem, as Elasticsearch limit the max number of feild to 1000 by default, see also index.mapping.total_fields.limit: 1000

flattened

Better type than daynamic, comes with 7.3 +

pros: unlimited number of fields, all things flattened as its name described.

cons: only support few query: term or exist, no highlight.

object

Object is another default when you using {} in mappings.

pros: nature supported, also for inner object

cons: not support array of object. Elasticsearch actually has no concept of inner object.Therefore, it flattens object hierarchies into a simple list of field names and values, checkout https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html#nested-arrays-flattening-objects

nested

pros: Support complex structure of array of objects or object of arrays and also keep them separately, therefore the field relation inside single object is kept. Suppose we have a candidate indices and lots of candidates have multiple education backgrounds. with nested fields, we can now retrieve candidates graduated from CMU with a major in CS.

cons: Each nested object is indexed as a separate Lucene document, 1 doc with 100 nested object will create 101 Lucene documents.

Both fields and objects inside nested have limits, default to 50 and 10000.

see also https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html#_limits_on_nested_mappings_and_objects