adding a custom plugin to google-fluentd

user_1725589 0
Viewed 388

I need to capture file descriptors for a given process. This is simular to what collectd's processes plugin does, but need to get this on the fluentd, google-fluentd specifically rails.

I've added my plugin under /etc/google-fluentd/plugin directory and no luck, it is not getting registered. I've even moved under /opt/google-fluentd/embedded/lib/ruby/gems/2.6.0/gems/fluentd-1.7.4/lib/fluent/plugin still no luck. Out of desperation I have also tried renaming in_tail.rb to in_tail2.rb and tail plugin is gone.

2020-08-14 18:28:16 -0700 [error]: fluent/log.rb:362:error: config error file="/etc/google-fluentd/google-fluentd.conf" error_class=Fluent::ConfigError error="Unknown input plugin 'tail'. Run 'gem search -rd fluent-plugin' to find plugins"

Which tells me that there is some other place where plugin must be mentioned. Is it too naive to think that I can just write a single file plugin under /etc/google-fluentd/plugin?

fluentd google-cloud-platform
1 Answers

After a few hours of going up and down the call stack in the fluentd trying to figure out the logic behind why and which plugins fluentd loads here is what I figured out.

@type has to match registration call and filename!

ie i had used

@type fc_count

my filename was /etc/google-fluentd/in_fd.rb with

 Fluent::Plugin.register_input('fd_count', self)

Although type and registration matched, fluent couldn't match file path to plugin/in_fd.rb as it loads configuration. Basically if you don't use a plugin it won't load it and the way it determines it is by going through config. This is the reason why when I renamed an existing input plugin it was no longer found.

user_17255890
Related