Cannot reflect proxy setting to run /install.sh --install-python3 --venv /opt/cycle/gridengine/venv in "cyclecloud-gridengine-pkg-2.0.12.tar.gz"

cyclecloud-gridengine-pkg-2.0.12.tar.gz 内の install.sh と gridengine.txt テンプレートから始め、次に scheduler のセットアップが ./install.sh --install-python3 --venv をどのように呼び出すかを追跡します。cloud-init からのプロキシ設定がそのプロセスにどのように渡るかを確認します。アーカイブ内の install.sh を手動で編集せずに、パッケージのインストールで設定済みのプロキシを使用できれば完了です。

I am deploying the GridEngine cluster (project version 2.0.12) under proxy-enabled environment. First, I added proxy setting in cloud-init as below and deployed, but it failed with the message below.

---

        :

• ./install.sh --install-python3 --venv /opt/cycle/gridengine/venv
  which: no python3 in (/sched/sge/sge-2011.11/bin/linux-x64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
  WARNING: Running pip install with root privileges is generally not a good idea. Try __main__.py install --user instead.
  Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f28fa89eeb8>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/virtualenv/
  :
  Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f28fa89e198>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/virtualenv/
  Could not find a version that satisfies the requirement virtualenv (from versions: )
  No matching distribution found for virtualenv
  EXCEPTION: bash[setup cyclecloud-gridengine] (gridengine::scheduler line 488) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'

---

Therefore, as a workaround to avoid the situation above, I did the following (1) to (5) instead.

(1) I added proxy setting in Cloud-init tab for the cluster setting on cyclecloud portal.

#!/bin/bash
export https_proxy="http://10.0.4.4:3128/"
export HTTPS_PROXY="http://10.0.4.4:3128/"
export http_proxy="http://10.0.4.4:3128/"
export HTTP_PROXY="http://10.0.4.4:3128/"
export NO_PROXY="127.0.0.1,localhost,.core.windows.net,10.0.0.0/16,10.0.1.0/24,10.0.2.0/24,10.0.3.0/24,10.0.4.0/24"
export no_proxy="127.0.0.1,localhost,.core.windows.net,10.0.0.0/16,10.0.1.0/24,10.0.2.0/24,10.0.3.0/24,10.0.4.0/24"
cat <> /etc/environment
https_proxy="http://10.0.4.4:3128/"
HTTPS_PROXY="http://10.0.4.4:3128/"
http_proxy="http://10.0.4.4:3128/"
HTTP_PROXY="http://10.0.4.4:3128/"
NO_PROXY="127.0.0.1,localhost,.core.windows.net,10.0.0.0/16,10.0.1.0/24,10.0.2.0/24,10.0.3.0/24,10.0.4.0/24"
no_proxy="127.0.0.1,localhost,.core.windows.net,10.0.0.0/16,10.0.1.0/24,10.0.2.0/24,10.0.3.0/24,10.0.4.0/24"
EOF
echo "proxy=http://10.0.4.4:3128" >> /etc/yum.conf
cat <> /etc/wgetrc
http_proxy=http://10.0.4.4:3128/
https_proxy=http://10.0.4.4:3128/
EOF2

(2) Next, I manually added proxy setting to "install.sh" in "cyclecloud-gridengine-pkg-2.0.12.tar.gz" on cyclecloud server.

---

#!/usr/bin/env bash

export https_proxy="http://10.0.4.4:3128/"
export HTTPS_PROXY="http://10.0.4.4:3128/"
export http_proxy="http://10.0.4.4:3128/"
export HTTP_PROXY="http://10.0.4.4:3128/"
export NO_PROXY="127.0.0.1,localhost,.core.windows.net,10.0.0.0/16,10.0.1.0/24,10.0.2.0/24,10.0.3.0/24,10.0.4.0/24"
export no_proxy="127.0.0.1,localhost,.core.windows.net,10.0.0.0/16,10.0.1.0/24,10.0.2.0/24,10.0.3.0/24,10.0.4.0/24"

INSTALL_PYTHON3=0
INSTALL_VIRTUALENV=0
INSTALL_PIP=0
INSTALLDIR=/opt/cycle/gridengine
VENV=$INSTALLDIR/venv
OS=$(awk -F= '/^NAME/{print $2}' /etc/os-release | awk '{print tolower($0)}')
:

(3) Next, I added following line in the template file.

---

                       :
    [[[cluster-init cyclecloud/gridengine:default]]]
   [[[cluster-init gridengine:default:2.0.12]]]
                         :
    [[[cluster-init cyclecloud/gridengine:scheduler]]]
   [[[cluster-init gridengine:scheduler:2.0.12]]]
                         :
    [[[cluster-init cyclecloud/gridengine:execute]]]
   [[[cluster-init gridengine:execute:2.0.12]]]
                         :

---

(4) Next, I ran the commands below to register the project to CycleCloud server.

$ cyclecloud delete_template ge-proxytest01
$ cyclecloud project upload azureuser-storage
$ cyclecloud import_template -f templates/gridengine.txt

(5) I deployed the cluster from cyclecloud portal. Finally, the cluster was deployed without python error.

However, I really feel that this is not a good way because I manually modified "install.sh" in "cyclecloud-gridengine-pkg-2.0.12.tar.gz" to add proxy settings. However, as "install.sh" is archived in ""cyclecloud-gridengine-pkg-2.0.12.tar.gz", this is the only way that I managed to solve the situation.

Are there any good way to reflect proxy setting without changing "install.sh" in "cyclecloud-gridengine-pkg-2.0.12.tar.gz"? under proxy-enabled environment? I would appreciate for the feedback.

Thanks.