Problem in ILM rollover alias in elastic stack

phoenixSid 0
Viewed 34

I am using logstash to ingest flat files into an elastic cluster and I want to make use of ILM. I have configured the lifecycle policy and created an index template in kibana. In my logstash file, I am using the policy name and the rollover alias. Here is my problem:

  • If I don't give the rollover alias in the index template, it throws error setting [index.lifecycle.rollover_alias] for index is empty or not defined when it tries to rollover according to the rollover conditions.
  • But if I do give it, then it complains that there is duplicate alias.
  • If I keep it in the index template and remove the alias from the logstash conf file, then logstash doesn't send the data to the correct index. It tries sending to some default index name.

Here is my output plugin from logstash conf file:

output {
    elasticsearch {
        hosts =>  XXXX
        ilm_enabled => true
        ilm_rollover_alias => "myaliasname"
        ilm_pattern => "000001"
        ilm_policy => "my_policy"
        user => XXXX
        password => XXXX
        manage_template => false    
    }
}

I have searched the elastic forums as well as on SO, but can't seem to narrow down the issue. Any help is greatly appreciated.

P.S. I am not using data streams currently.

kibana logstash elasticsearch
1 Answers

Tldr;

I think you are very close with your actual config.

Keep the logstash config as you have it. Logstash will handle creating the alias

Logstash will create a rollover alias for the indices to be written to,...

Do not put it in the template.

userxL1j0
Related