sh_show_command: Quotes are missing for an extra option including a space.

Bắt đầu tại lib/rake/file_utils.rb#sh_show_command, đặc biệt là dòng cmd.join " ", và tái hiện lệnh rake-compiler với --with-cflags='-Wundef -Werror'. Xác nhận rằng lệnh được hiển thị giữ nguyên việc đặt trong dấu ngoặc kép đối với các đối số chứa khoảng trắng và khớp với ranh giới đối số thực tế của lệnh.

Summary

In the following rake command to compile a native extension with rake-compiler gem, where the extra options whose value (--with-cflags='-Wundef -Werror') includes a space, in this case -Wundef -Werror is printed without quotes. And the command is different from the one actually executed and it's not reproducible.

$ bundle exec rake compile -- --with-cflags='-Wundef -Werror'
...
/home/jaruga/.local/ruby-cbeba2ed16/bin/ruby -I. -r.rake-compiler-siteconf.rb ../../../../ext/openssl/extconf.rb -- --with-cflags=-Wundef -Werror
...

I think quotes (double quotes or single quotes) are needed like the command below.

/home/jaruga/.local/ruby-cbeba2ed16/bin/ruby -I. -r.rake-compiler-siteconf.rb ../../../../ext/openssl/extconf.rb -- --with-cflags="-Wundef -Werror"

Note that @kou helped me to report this issue here at https://github.com/rake-compiler/rake-compiler/issues/215#issuecomment-1563493206.

My environment

I am using the following my environment on Fedora Linux 37. I think this issue doesn't depend on the specific version of the Ruby. I would share it with you just in case.

$ ruby -v
ruby 3.3.0dev (2023-05-25T12:45:21Z wip/mkmf-verbose-r.. cbeba2ed16) [x86_64-linux]

$ which gem
~/.local/ruby-cbeba2ed16/bin/gem
$ gem -v
3.5.0.dev

$ which bundle
~/.local/ruby-cbeba2ed16/bin/bundle

$ bundle -v
Bundler version 2.5.0.dev

Reproducing steps

Here are the reproducing steps with the ruby/openssl: https://github.com/ruby/openssl first.

$ git clone https://github.com/ruby/openssl.git

$ cd openssl

$ bundle install --standalone

$ bundle list | grep rake
  * rake (13.0.6)
  * rake-compiler (1.2.2)

Ran the rake with the rake-compiler gem. The following command is to compile the native extension setting the -Wundef -Werror as compiler flags. You see the command propagates the -Wundef -Werror to the gcc command lines. It's important to run with the rake-compiler latest stable version 1.2.2. Because it fixes https://github.com/rake-compiler/rake-compiler/issues/215 related to this issue. The environment variable MAKEFLAGS reserved in the GNU make is to print the compiler command lines.

$ MAKEFLAGS="V=1" bundle exec rake compile -- --with-cflags='-Wundef -Werror'
mkdir -p tmp/x86_64-linux/openssl/3.3.0
cd tmp/x86_64-linux/openssl/3.3.0
/home/jaruga/.local/ruby-cbeba2ed16/bin/ruby -I. -r.rake-compiler-siteconf.rb ../../../../ext/openssl/extconf.rb -- --with-cflags=-Wundef -Werror
checking for rb_io_maybe_wait(0, Qnil, Qnil, Qnil) in ruby/io.h... yes
...
gcc -I. -I/home/jaruga/.local/ruby-cbeba2ed16/include/ruby-3.3.0+0/x86_64-linux -I/home/jaruga/.local/ruby-cbeba2ed16/include/ruby-3.3.0+0/ruby/backward -I/home/jaruga/.local/ruby-cbeba2ed16/include/ruby-3.3.0+0 -I../../../../ext/openssl  -DRUBY_EXTCONF_H=\"extconf.h\"    -fPIC -Wundef -Werror   -o openssl_missing.o -c ../../../../ext/openssl/openssl_missing.c
...
cp tmp/x86_64-linux/openssl/3.3.0/openssl.so tmp/x86_64-linux/stage/lib/openssl.so

$ echo $?
0

The problem is the the line /home/jaruga/.local/ruby-cbeba2ed16/bin/ruby -I. -r.rake-compiler-siteconf.rb ../../../../ext/openssl/extconf.rb -- --with-cflags=-Wundef -Werror printed at sh in the lib/rake/extensiontask.rb. Because the --with-cflags=-Wundef -Werror is different from the one actually executed, and not reproducible. The argument --with-cflags's value is -Wundef -Werror. It should be printed as --with-cflags="-Wundef -Werror" with quotes.

Debug

I debugged with debug gem. Added the gem "debug" to the ruby/openssl's Gemfile.

$ pwd
/home/jaruga/git/ruby/openssl

$ vi Gemefile

$ bundle update

Next set the breakpoint before the sh method to print and execute the command.

$ cd /home/jaruga/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-compiler-1.2.2/lib/rake

$ pwd
/home/jaruga/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-compiler-1.2.2/lib/rake

$ diff -u extensiontask.rb.orig extensiontask.rb
--- extensiontask.rb.orig	2023-05-26 20:36:06.207487491 +0200
+++ extensiontask.rb	2023-05-26 21:14:35.794188802 +0200
@@ -215,6 +215,9 @@
         end
 
         chdir tmp_path do
+          require 'debug'
+          binding.break
+
           sh *cmd
         end
       end

Went back to the ruby/openssl, and cleaned the built files.

$ pwd
/home/jaruga/git/ruby/openssl

$ rm -rf tmp lib/openssl.so

And ran the command again.

$ MAKEFLAGS="V=1" bundle exec rake compile -- --with-cflags='-Wundef -Werror'
...
(rdbg) f    # frame command
=> 219|           binding.break
=>#0	block in define_compile_tasks (2 levels) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-compiler-1.2.2/lib/rake/extensiontask.rb:219
(rdbg) p cmd    # command
=> ["/home/jaruga/.local/ruby-cbeba2ed16/bin/ruby", "-I.", "-r.rake-compiler-siteconf.rb", "../../../../ext/openssl/extconf.rb", "--", "--with-cflags=-Wundef -Werror"]
...

The lib/rake/file_utils.rb#sh calling Rake.rake_output_message sh_show_command cmd. And the cmd.join " " in the sh_show_command causes this issue. Is it a bug?

(rdbg) n    # next command
[77, 86] in ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/file_utils.rb
    77|       env = cmd.first
    78|       env = env.map { |name, value| "#{name}=#{value}" }.join " "
    79|       cmd[0] = env
    80|     end
    81| 
=>  82|     cmd.join " "
    83|   end
    84|   private :sh_show_command
    85| 
    86|   def set_verbose_option(options) # :nodoc:

(rdbg) f    # frame command
=>  82|     cmd.join " "
=>#0	FileUtils#sh_show_command(cmd=["/home/jaruga/.local/ruby-cbeba2ed16/bin...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/file_utils.rb:82
(rdbg) p cmd    # command
=> ["/home/jaruga/.local/ruby-cbeba2ed16/bin/ruby", "-I.", "-r.rake-compiler-siteconf.rb", "../../../../ext/openssl/extconf.rb", "--", "--with-cflags=-Wundef -Werror"]
(rdbg) p cmd.join " "    # command
=> "/home/jaruga/.local/ruby-cbeba2ed16/bin/ruby -I. -r.rake-compiler-siteconf.rb ../../../../ext/openssl/extconf.rb -- --with-cflags=-Wundef -Werror"

I would show you the backtrace command just in case.

(rdbg) f    # frame command
=>  82|     cmd.join " "
=>#0	FileUtils#sh_show_command(cmd=["/home/jaruga/.local/ruby-cbeba2ed16/bin...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/file_utils.rb:82
(rdbg) bt    # backtrace command
=>#0	FileUtils#sh_show_command(cmd=["/home/jaruga/.local/ruby-cbeba2ed16/bin...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/file_utils.rb:82
  #1	FileUtils#sh(cmd=["/home/jaruga/.local/ruby-cbeba2ed16/bin..., block=nil) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/file_utils.rb:51
  #2	block in define_compile_tasks (2 levels) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-compiler-1.2.2/lib/rake/extensiontask.rb:221
  #3	[C] Dir.chdir at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/fileutils.rb:240
  #4	FileUtils#chdir(dir="tmp/x86_64-linux/openssl/3.3.0", verbose=#<Object:0x00007fd7eb4ad510>, block=#<Proc:0x00007fd7ea893108 /home/jaruga/va...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/fileutils.rb:240
  #5	Rake::FileUtilsExt#chdir(args=["tmp/x86_64-linux/openssl/3.3.0"], options={}, block=#<Proc:0x00007fd7ea8930b8 /home/jaruga/va...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/file_utils_ext.rb:35
  #6	block {|t=<Rake::FileTask tmp/x86_64-linux/openssl/...|} in define_compile_tasks at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-compiler-1.2.2/lib/rake/extensiontask.rb:217
  #7	block {|act=#<Proc:0x00007fd7efbd8b10 /home/jaruga/va...|} in execute at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:281
  #8	[C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:281
  #9	Rake::Task#execute(args=#<Rake::TaskArguments >) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:281
  #10	block in invoke_with_call_chain at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:219
  #11	[C] Monitor#synchronize at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
  #12	Rake::Task#invoke_with_call_chain(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::FileTask tmp/x86_64-linux/opens...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
  #13	block {|p=<Rake::FileTask tmp/x86_64-linux/openssl/...|} in invoke_prerequisites at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:243
  #14	[C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
  #15	Rake::Task#invoke_prerequisites(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::FileTask tmp/x86_64-linux/opens...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
  #16	block in invoke_with_call_chain at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:218
  #17	[C] Monitor#synchronize at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
  #18	Rake::Task#invoke_with_call_chain(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task copy:openssl:x86_64-linux:...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
  #19	block {|p=<Rake::FileTask tmp/x86_64-linux/openssl/...|} in invoke_prerequisites at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:243
  #20	[C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
  #21	Rake::Task#invoke_prerequisites(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task copy:openssl:x86_64-linux:...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
  #22	block in invoke_with_call_chain at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:218
  #23	[C] Monitor#synchronize at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
  #24	Rake::Task#invoke_with_call_chain(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task compile:openssl:x86_64-lin...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
  #25	block {|p=<Rake::Task copy:openssl:x86_64-linux:3.3...|} in invoke_prerequisites at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:243
  #26	[C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
  #27	Rake::Task#invoke_prerequisites(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task compile:openssl:x86_64-lin...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
  #28	block in invoke_with_call_chain at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:218
  #29	[C] Monitor#synchronize at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
  #30	Rake::Task#invoke_with_call_chain(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task compile:x86_64-linux => [c...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
  #31	block {|p=<Rake::Task compile:openssl:x86_64-linux ...|} in invoke_prerequisites at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:243
  #32	[C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
  #33	Rake::Task#invoke_prerequisites(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task compile:x86_64-linux => [c...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
  #34	block in invoke_with_call_chain at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:218
  #35	[C] Monitor#synchronize at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
  #36	Rake::Task#invoke_with_call_chain(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task compile => [compile:x86_64...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
  #37	block {|p=<Rake::Task compile:x86_64-linux => [comp...|} in invoke_prerequisites at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:243
  #38	[C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
  #39	Rake::Task#invoke_prerequisites(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task compile => [compile:x86_64...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
  #40	block in invoke_with_call_chain at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:218
  #41	[C] Monitor#synchronize at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
  #42	Rake::Task#invoke_with_call_chain(task_args=#<Rake::TaskArguments >, invocation_chain=LL()) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
  #43	Rake::Task#invoke(args=[]) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:188
  #44	Rake::Application#invoke_task(task_string="compile") at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:160
  #45	block {|task_name="compile"|} in top_level (2 levels) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:116
  #46	[C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:116
  #47	block in top_level at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:116
  #48	Rake::Application#run_with_threads at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:125
  #49	Rake::Application#top_level at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:110
  #50	block in run at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:83
  #51	Rake::Application#standard_exception_handling at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:186
  #52	Rake::Application#run(argv=["compile", "--", "--with-cflags=-Wundef ...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:80
  #53	<top (required)> at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/exe/rake:27
  #54	[C] Kernel#load at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/bin/rake:25
  #55	<top (required)> at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/bin/rake:25
  #56	[C] Kernel.load at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/cli/exec.rb:58
  #57	Bundler::CLI::Exec#kernel_load(file="/home/jaruga/var/git/ruby/openssl/bundle..., args=["compile", "--", "--with-cflags=-Wundef ...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/cli/exec.rb:58
  #58	Bundler::CLI::Exec#run at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/cli/exec.rb:23
  #59	Bundler::CLI#exec(args=["compile", "--", "--with-cflags=-Wundef ...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/cli.rb:492
  #60	Bundler::Thor::Command#run(instance=#<Bundler::CLI:0x00007fd7eb4ed160 @_initi..., args=["rake", "compile", "--", "--with-cflags=...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/vendor/thor/lib/thor/command.rb:27
  #61	Bundler::Thor::Invocation#invoke_command(command=#<struct Bundler::Thor::Command name="exe..., args=[["rake", "compile", "--", "--with-cflags...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/vendor/thor/lib/thor/invocation.rb:127
  #62	#<Class:Bundler::Thor>#dispatch(meth="exec", given_args=["rake", "compile", "--", "--with-cflags=..., given_opts=nil, config={:debug=>true, :shell=>#<Bundler::Thor::S...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/vendor/thor/lib/thor.rb:392
  #63	Bundler::CLI.dispatch at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/cli.rb:34
  #64	Bundler::Thor::Base::ClassMethods#start(given_args=["compile", "--", "--with-cflags=-Wundef ..., config={:debug=>true, :shell=>#<Bundler::Thor::S...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/vendor/thor/lib/thor/base.rb:485
  #65	Bundler::CLI.start at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/cli.rb:28
  #66	block in <top (required)> at ~/.local/ruby-cbeba2ed16/lib/ruby/gems/3.3.0+0/gems/bundler-2.5.0.dev/libexec/bundle:45
  #67	Bundler.with_friendly_errors at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/friendly_errors.rb:117
  #68	<top (required)> at ~/.local/ruby-cbeba2ed16/lib/ruby/gems/3.3.0+0/gems/bundler-2.5.0.dev/libexec/bundle:33
  #69	[C] Kernel#load at ~/.local/ruby-cbeba2ed16/bin/bundle:25
  #70	<main> at ~/.local/ruby-cbeba2ed16/bin/bundle:25