record transformer and ruby problem !

Open
#4 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
elasticsearch, ruby
Domain
data-engineering

Research direction

Start with the supplied configuration and fluentd-1.1.0/lib/fluent/plugin/filter_record_transformer.rb around line 310. Reproduce the Resolv.getname error using the record_transformer filter and inspect how the Ruby placeholder accesses ipv4_src_addr and ipv4_dst_addr. Done means the netflow configuration processes these fields without the reported undefined-variable error.

Written by the indexing model from the issue text.

Description

Hi,

I'm trying to migrate our netflow from es2.4.1 to es6.1.2 ! I use the same config for the new elastic and the latest fluentd and packages updates !

Do something changed and do i have to modify the fluentd config to index our netflow now ??

Thanks for your help.

2018-05-03 17:53:19 +0200 [warn]: #0 dump an error event: error_class=RuntimeError error="failed to expand Resolv.getname(ipv4_src_addr) : error = undefined local variable or method ipv4_src_addr' for #Fluent::Plugin::RecordTransformerFilter::RubyPlaceholderExpander::CleanroomExpander:0x007f1a9be38048" location="/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-1.1.0/lib/fluent/plugin/filter_record_transformer.rb:310:inrescue in expand'"

my fluentd.conf

<filter netflow.event.**>
@type record_transformer
enable_ruby true

ipname_src_addr ${Resolv.getname(ipv4_src_addr)}
ipname_dst_addr ${Resolv.getname(ipv4_dst_addr)}
.....

Dominant language
No language data
Stars
7
Forks
6
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from fluent/data-collection

All issues in fluent/data-collection

Similar issues

More Data Engineering issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.