UB/crashes with libedit

readline_s_set_output() と clear_rl_outstream() から開始し、その後、rl_initialize() が libedit に rl_outstream を保存する方法を追跡します。libedit と musl libc でクラッシュを再現し、報告で言及されている関連する rl_instream の経路を調査します。readline の出力が閉じられた FILE* にアクセスしなくなり、報告されたクラッシュが防止されれば完了です。

the code mentions compatibility with libedit, however the following happens in practice (e.g. while using irb from ruby 2.5.0):

• libedit initializes internal copy of rl_outstream FILE* pointer during rl_initialize()
  e = el_init_internal(rl_readline_name, rl_instream, rl_outstream, ...). this saves the current rl_outstream in el->el_outfile.
  from that point on, libedit only operates on el_outfile in its EditLine* context.
• in this module, clear_rl_outstream() is called from readline_s_set_output() which fclose()s rl_outstream. then rl_outstream gets a fresh FILE* assigned - however this does not affect libedit which continues using its el_outfile struct member.
• next attempt to write anything to the output file via readline API causes a segfault - at least with musl libc, which is stricter in regards to UB.

probably this was never caught because the developers used glibc where the fclose() FILE structure happened to stay in RAM at the same address. or maybe the new FILE was allocated at the exact same address. it's 100% reproducible using libedit (latest release) and musl libc, and likely when using address sanitizer even on glibc.

it's likely that rl_instream handling has the same bug, but i only investigated the crash i actually got.

arguably it's a bug of libedit to supply rl_outstream but then use it only for initialization - but it appears nobody else was closing the FILE behind libedit's back so far, else it would be fixed in latest libedit version.

(background: i was about to update ruby from 2.5.0 to a newer version in my distro sabotage linux and noticed irb crashing, so i wanted to fix it before switching to a more modern version, so someone who needs to use the old version for whatever reason can go back in commit history and produce a 100% working version).