9.2.1 as splunk_deployment_server doesn't start

Reproduce the 9.2.1 deployment-server startup using the logged Ansible tasks. Start with splunk_common/tasks/wait_for_splunk_instance.yml and disable_popups.yml, then trace the reported license-slave and Splunk API failures. Done means the container starts without these task failures, the license slave is set, and popups are disabled.

Docker container docker doesn't start well:

TASK [splunk_common : Determine available licenses] ****************************
ok: [localhost] => (item=splunk.lic)
Thursday 18 April 2024  16:01:21 +0000 (0:00:00.151)       0:02:04.761 ********
Thursday 18 April 2024  16:01:21 +0000 (0:00:00.125)       0:02:04.886 ********

TASK [splunk_common : Set as license slave] ************************************
included: /opt/ansible/roles/splunk_common/tasks/set_as_license_slave.yml for localhost
Thursday 18 April 2024  16:01:21 +0000 (0:00:00.108)       0:02:04.995 ********

TASK [splunk_common : Wait for the license master] *****************************
included: /opt/ansible/roles/splunk_common/tasks/wait_for_splunk_instance.yml for localhost => (item=(censored due to no_log))
Thursday 18 April 2024  16:01:21 +0000 (0:00:00.078)       0:02:05.074 ********
[WARNING]: Using world-readable permissions for temporary files Ansible needs
to create when becoming an unprivileged user. This may be insecure. For
information on securing this, see https://docs.ansible.com/ansible-
core/2.11/user_guide/become.html#risks-of-becoming-an-unprivileged-user

TASK [splunk_common : Check Splunk instance is running] ************************
fatal: [localhost]: FAILED! => {}

MSG:

The conditional check 'task_response.status == 200' failed. The error was: error while evaluating conditional (task_response.status == 200): 'dict object' has no attribute 'status'
...ignoring
Thursday 18 April 2024  16:01:22 +0000 (0:00:01.262)       0:02:06.336 ********
[WARNING]: Using world-readable permissions for temporary files Ansible needs
to create when becoming an unprivileged user. This may be insecure. For
information on securing this, see https://docs.ansible.com/ansible-
core/2.11/user_guide/become.html#risks-of-becoming-an-unprivileged-user
FAILED - RETRYING: Set node as license slave (60 retries left).
...
FAILED - RETRYING: Set node as license slave (1 retries left).

TASK [splunk_common : Set node as license slave] *******************************
ok: [localhost]
Thursday 18 April 2024  16:09:28 +0000 (0:08:05.958)       0:10:12.295 ********
Thursday 18 April 2024  16:09:28 +0000 (0:00:00.024)       0:10:12.319 ********

TASK [splunk_common : include_tasks] *******************************************
included: /opt/ansible/roles/splunk_common/tasks/disable_popups.yml for localhost
Thursday 18 April 2024  16:09:28 +0000 (0:00:00.244)       0:10:12.564 ********
[WARNING]: Using world-readable permissions for temporary files Ansible needs
to create when becoming an unprivileged user. This may be insecure. For
information on securing this, see https://docs.ansible.com/ansible-
core/2.11/user_guide/become.html#risks-of-becoming-an-unprivileged-user

TASK [splunk_common : GET OptInVersion] ****************************************
changed: [localhost]
Thursday 18 April 2024  16:09:30 +0000 (0:00:01.207)       0:10:13.772 ********
[WARNING]: Using world-readable permissions for temporary files Ansible needs
to create when becoming an unprivileged user. This may be insecure. For
information on securing this, see https://docs.ansible.com/ansible-
core/2.11/user_guide/become.html#risks-of-becoming-an-unprivileged-user

TASK [splunk_common : Disable Popups] ******************************************
changed: [localhost] => (item={'key': '/servicesNS/admin/user-prefs/data/user-prefs/general', 'value': 'hideInstrumentationOptInModal=1&notification_python_3_impact=false&showWhatsNew=0'})
failed: [localhost] (item={'key': '/servicesNS/nobody/splunk_instrumentation/admin/telemetry/general', 'value': 'showOptInModal=0&optInVersionAcknowledged=4'}) => {
    "ansible_loop_var": "item",
    "changed": false,
    "item": {
        "key": "/servicesNS/nobody/splunk_instrumentation/admin/telemetry/general",
        "value": "showOptInModal=0&optInVersionAcknowledged=4"
    }
}

MSG:

POST/servicesNS/nobody/splunk_instrumentation/admin/telemetry/generaladmin********8089{'showOptInModal': '0&optInVersionAcknowledged=4'}NoneNone[200, 201, 409];;; AND excep_str: URL: https://127.0.0.1:8089/servicesNS/nobody/splunk_instrumentation/admin/telemetry/general; data: {"showOptInModal": "0&optInVersionAcknowledged=4"}, exception: API call for https://127.0.0.1:8089/servicesNS/nobody/splunk_instrumentation/admin/telemetry/general and data as {'showOptInModal': '0&optInVersionAcknowledged=4'} failed with status code 400: <?xml version="1.0" encoding="UTF-8"?>
<response>
  <messages>
    <msg type="ERROR">Argument "{"showOptInModal": "0" is not supported by this handler.</msg>
  </messages>
</response>
, failed with status code 400: <?xml version="1.0" encoding="UTF-8"?>
<response>
  <messages>
    <msg type="ERROR">Argument "{"showOptInModal": "0" is not supported by this handler.</msg>
  </messages>
</response>

failed: [localhost] (item={'key': '/servicesNS/admin/search/data/ui/ui-tour/search-tour', 'value': 'tourPage=search&viewed=1'}) => {
    "ansible_loop_var": "item",
    "changed": false,
    "item": {
        "key": "/servicesNS/admin/search/data/ui/ui-tour/search-tour",
        "value": "tourPage=search&viewed=1"
    }
}

MSG:

POST/servicesNS/admin/search/data/ui/ui-tour/search-touradmin********8089{'tourPage': 'search&viewed=1'}NoneNone[200, 201, 409];;; AND excep_str: URL: https://127.0.0.1:8089/servicesNS/admin/search/data/ui/ui-tour/search-tour; data: {"tourPage": "search&viewed=1"}, exception: API call for https://127.0.0.1:8089/servicesNS/admin/search/data/ui/ui-tour/search-tour and data as {'tourPage': 'search&viewed=1'} failed with status code 400: <?xml version="1.0" encoding="UTF-8"?>
<response>
  <messages>
    <msg type="ERROR">Argument "{"tourPage": "search" is not supported by this handler.</msg>
  </messages>
</response>
, failed with status code 400: <?xml version="1.0" encoding="UTF-8"?>
<response>
  <messages>
    <msg type="ERROR">Argument "{"tourPage": "search" is not supported by this handler.</msg>
  </messages>
</response>


PLAY RECAP *********************************************************************
localhost                  : ok=100  changed=8    unreachable=0    failed=1    skipped=61   rescued=0    ignored=1

Thursday 18 April 2024  16:09:33 +0000 (0:00:03.288)       0:10:17.060 ********
===============================================================================
splunk_common : Set node as license slave ----------------------------- 485.96s
splunk_common : Start Splunk via CLI ----------------------------------- 48.93s
splunk_common : Get Splunk status -------------------------------------- 13.20s
splunk_common : Set options in saml ------------------------------------ 10.54s
splunk_common : Update Splunk directory owner --------------------------- 7.10s
splunk_common : Disable Popups ------------------------------------------ 3.29s
Gathering Facts --------------------------------------------------------- 2.94s
splunk_common : Set options in indexAndForward -------------------------- 1.29s
splunk_common : Set options in authentication --------------------------- 1.29s
splunk_common : Cleanup Splunk runtime files ---------------------------- 1.26s
splunk_common : Check Splunk instance is running ------------------------ 1.26s
splunk_common : Remove user-seed.conf ----------------------------------- 1.24s
splunk_common : GET OptInVersion ---------------------------------------- 1.21s
splunk_common : Update /opt/splunk/etc ---------------------------------- 0.90s
splunk_common : Find manifests ------------------------------------------ 0.89s
splunk_common : Check for scloud ---------------------------------------- 0.88s
splunk_common : Set general pass4SymmKey -------------------------------- 0.86s
splunk_common : Set mgmt port ------------------------------------------- 0.78s
splunk_common : Trigger restart ----------------------------------------- 0.78s
splunk_common : Check if /opt/splunk/var/lib/splunk/kvstore/mongo/splunk.key exists --- 0.76s

Summary:

• The conditional check 'task_response.status == 200' failed. The error was: error while evaluating conditional (task_response.status == 200): 'dict object' has no attribute 'status'
• can't set license slave
• disable popups is not working