StackStorm/st2

st2 services should log connection to [coordination] backend

Open

#4,783 创建于 2019年8月30日

在 GitHub 查看
 (3 评论) (1 反应) (0 负责人)Python (5,794 star) (740 fork)batch import
HAenhancementgood first issue

描述

st2 services log error if [coordination] connection was unsuccessful, which is great:

2019-08-30 11:19:54,609 DEBUG [-] Using Python: 2.7.12 (/opt/stackstorm/st2/bin/python)
2019-08-30 11:19:54,609 DEBUG [-] Using config files: /etc/st2/st2.conf,/etc/st2/st2.docker.conf,/etc/st2/st2.user.conf
2019-08-30 11:19:54,610 DEBUG [-] Using logging config: /etc/st2/logging.api.gunicorn.conf
2019-08-30 11:19:54,622 INFO [-] Connecting to database "st2" @ "ignorant-manta-mongodb-ha-0.ignorant-manta-mongodb-ha:27017,ignorant-manta-mongodb-ha-1.ignorant-manta-mongodb-ha:27017,ignorant-manta-mongodb-ha-2.ignorant-manta-mongodb-ha:27017 (replica set)" as user "admin".
2019-08-30 11:19:54,797 INFO [-] Successfully connected to database "st2" @ "ignorant-manta-mongodb-ha-0.ignorant-manta-mongodb-ha:27017,ignorant-manta-mongodb-ha-1.ignorant-manta-mongodb-ha:27017,ignorant-manta-mongodb-ha-2.ignorant-manta-mongodb-ha:27017 (replica set)" as user "admin".
2019-08-30 11:19:56,138 INFO [-] (PID=1) ST2 API is serving on http://0.0.0.0:9101.
2019-08-30 11:19:56,138 INFO [-] Creating st2api: StackStorm v3.2dev as OpenAPI app.
2019-08-30 11:19:57,063 INFO [-] Connected to amqp://admin:**@ignorant-manta-rabbitmq-ha-discovery:5672//
2019-08-30 11:19:57,111 ERROR [-] (PID=1) ST2 API quit due to exception.
Traceback (most recent call last):
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2api/cmd/api.py", line 81, in main
    return _run_server()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2api/cmd/api.py", line 70, in _run_server
    wsgi.server(sock, app.setup_app(), custom_pool=worker_pool, log=LOG, log_output=False)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2api/app.py", line 79, in setup_app
    router.add_spec(spec, transforms=transforms)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/router.py", line 211, in add_spec
    __import__(module_name)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2api/controllers/v1/keyvalue.py", line 436, in <module>
    key_value_pair_controller = KeyValuePairController()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2api/controllers/v1/keyvalue.py", line 61, in __init__
    self._coordinator = coordination.get_coordinator()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/services/coordination.py", line 226, in get_coordinator
    COORDINATOR = coordinator_setup(start_heart=start_heart)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/services/coordination.py", line 199, in coordinator_setup
    coordinator.start(start_heart=start_heart)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/tooz/coordination.py", line 426, in start
    self._start()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/tooz/drivers/etcd.py", line 252, in _start
    encodeutils.exception_to_unicode(e))
ToozConnectionError: HTTPConnectionPool(host='etcd-cluster-client', port=2379): Max retries exceeded with url: /v2/stats/self (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fca9effa850>: Failed to establish a new connection: [Errno -2] No address found',))

However it doesn't log any message if connection was successful to [coordination] backend:

$ kubectl logs ignorant-manta-st2api-fd58c5b8f-4x8gn
2019-08-30 11:22:12,899 DEBUG [-] Using Python: 2.7.12 (/opt/stackstorm/st2/bin/python)
2019-08-30 11:22:12,900 DEBUG [-] Using config files: /etc/st2/st2.conf,/etc/st2/st2.docker.conf,/etc/st2/st2.user.conf
2019-08-30 11:22:12,900 DEBUG [-] Using logging config: /etc/st2/logging.api.gunicorn.conf
2019-08-30 11:22:12,910 INFO [-] Connecting to database "st2" @ "ignorant-manta-mongodb-ha-0.ignorant-manta-mongodb-ha:27017,ignorant-manta-mongodb-ha-1.ignorant-manta-mongodb-ha:27017,ignorant-manta-mongodb-ha-2.ignorant-manta-mongodb-ha:27017 (replica set)" as user "admin".
2019-08-30 11:22:13,194 INFO [-] Successfully connected to database "st2" @ "ignorant-manta-mongodb-ha-0.ignorant-manta-mongodb-ha:27017,ignorant-manta-mongodb-ha-1.ignorant-manta-mongodb-ha:27017,ignorant-manta-mongodb-ha-2.ignorant-manta-mongodb-ha:27017 (replica set)" as user "admin".
2019-08-30 11:22:14,768 INFO [-] (PID=1) ST2 API is serving on http://0.0.0.0:9101.
2019-08-30 11:22:14,768 INFO [-] Creating st2api: StackStorm v3.2dev as OpenAPI app.
2019-08-30 11:22:15,917 INFO [-] Connected to amqp://admin:**@ignorant-manta-rabbitmq-ha-discovery:5672//
2019-08-30 11:22:16,287 INFO [-] Connected to amqp://admin:**@ignorant-manta-rabbitmq-ha-discovery:5672//

It's preferred to log this for informational, consistency and user expectation reasons, as it happens for both MongoDB and RabbitMQ in logs. This will als make it clearer which services really rely on coordination.

贡献者指南