A space after a comma in hosts breaks the next host: out_opensearch fails with URI::InvalidURIError and in_opensearch drops the host

Aperta Adatta ai principianti
#207 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
78/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
ruby
Ambito
backend

Direzione di ricerca

Inizia leggendo l’analisi degli host nelle posizioni indicate in lib/fluent/plugin/out_opensearch.rb e lib/fluent/plugin/in_opensearch.rb, oltre alla divisione in corrispondenza delle virgole in warn_host_placeholder_with_credentials nel plugin di output. Confronta questi punti con la divisione che tiene conto degli spazi e che viene usata per remove_keys e remove_keys_on_update. Il risultato è raggiunto quando entrambi i plugin accettano host separati da virgole con spazi, vengono usati entrambi gli host e gli errori URI segnalati non si verificano più.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug
Describe the bug

hosts is split on , only, so a space after a comma stays in the next host. With hosts localhost:9200, localhost:9201, the second host becomes " localhost".

  • out_opensearch raises URI::InvalidURIError in configure with the default verify_os_version_at_startup true, so Fluentd does not start. With verify_os_version_at_startup false, it starts, but every flush fails with RecoverableRequestFailure for the same reason.
  • in_opensearch with the default check_connection true cannot connect to that host, so it drops it and uses only the other hosts. With the URL format, it raises URI::InvalidURIError in configure.

https://github.com/fluent/fluent-plugin-opensearch/blob/76c177e/lib/fluent/plugin/out_opensearch.rb#L769
https://github.com/fluent/fluent-plugin-opensearch/blob/76c177e/lib/fluent/plugin/in_opensearch.rb#L189

Other list parameters of this plugin accept a space after a comma. remove_keys splits on /\s*,\s*/, and remove_keys_on_update does too since #206.

To Reproduce

Run Fluentd with the configuration below and an OpenSearch at localhost:19201. Both hosts point to the same OpenSearch. Fluentd exits at startup with the error log below. Without the space after the comma, it starts and writes the records.

I also checked each case with the test drivers of the two plugins and a real OpenSearch 2.19.1, on main (76c177e) and on 278e101:

hosts out_opensearch in_opensearch
localhost:19201,localhost:19201 Writes the record. Uses both hosts.
localhost:19201, localhost:19201 URI::InvalidURIError: bad URI (is not URI?): "http:// localhost:19201" in configure. With verify_os_version_at_startup false, the flush fails with RecoverableRequestFailure: could not push logs to OpenSearch cluster (... {host: " localhost", port: 19201, scheme: "http"}): bad URI .... Uses only the first host.
http://localhost:19201, http://localhost:19201 URI::InvalidURIError: bad URI (is not URI?): " http://localhost:19201" The same error in configure.
Expected behavior

Spaces around the commas are ignored, as in remove_keys, and both hosts are used.

Your Environment
- Fluentd version: 1.20.1
- TD Agent version: -
- fluent-plugin-opensearch version: main (76c177e). v1.1.8 has the same `split(',')`.
- OpenSearch version (optional): 2.19.1
- OpenSearch template(s) (optional): -
- Operating system: Ubuntu 24.04
- Kernel version: -
Your Configuration
<source>
  @type sample
  tag test
  sample {"message":"hello"}
</source>
<match test>
  @type opensearch
  hosts localhost:19201, localhost:19201
  index_name test
</match>
Your Error Log
[info]: starting fluentd-1.20.1 without supervision pid=2094060 ruby="3.4.12"
[error]: unexpected error error_class=URI::InvalidURIError error="bad URI (is not URI?): \"http:// localhost:19201\""
Additional context

Splitting on /\s*,\s*/ in both plugins would fix it. warn_host_placeholder_with_credentials in out_opensearch also splits hosts on ,:
https://github.com/fluent/fluent-plugin-opensearch/blob/76c177e/lib/fluent/plugin/out_opensearch.rb#L510

Lingua principale
Ruby
Stelle
64
Fork
25
Merge medio
1g 16h
PR unite (30g)
8

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di fluent/fluent-plugin-opensearch

Tutte le issue di fluent/fluent-plugin-opensearch

Issue simili

Altre issue su Ruby

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.