boto_asg problem if an existing ASG has Target Group attached
#53,594 opened on Jun 25, 2019
Description
hi :)
Description of Issue
The root cause of the issue is with the boto library https://github.com/boto/boto/issues/3613 It happens if Autoscaling groups in the AWS account have Target Groups attached to an ASG. The boto's xml parsing fails as the Target Group is not an expected attribute (happens here). I debugged a request to get_all_groups and saw that the XML returned by the raw AWS HTTP API returns correct data but the dict returned by the get_all_groups is malformed - full of None values and invalid structure altogether.
The invalid structure of the dict returned by boto to salt's boto_asg leads to the exception shown below. Since boto seems to be pretty much abandoned a possible solution would be to remove dependencies from boto_asg module to boto.
Setup
generic.sls
{% for dialer_country in countries %}
{% set dialer_name = env.environment + "-" + app.name + "-" + dialer_country -%}
dialer_{{ dialer_country }}:
boto_asg.present:
- name: "{{ dialer_name }}"
- launch_config_name: "{{ dialer_name }}"
- launch_config:
- image_id: {{ app.ami }}
- key_name: {{ env.key_name }}
- instance_profile_name: arn:aws:iam::{{ env.aws_id }}:instance-profile/{{ app.instance_role }}
- security_groups:
- {{ env.security_group_ids.internal_sg }}
- {{ env.security_group_ids.offices_vpn_sg }}
- instance_type: t3.micro
- cloud_init:
boothooks:
'boothook.sh': |
{% set name_ow = {"name_overwrite": app.name + "-" + dialer_country } -%}
{% set extra_data = {"extra_grains": {"dialer": dialer_country}, "app": name_ow} -%}
{{ cloud_init_boothook(env, app, extra_data=extra_data) | indent(width=13, indentfirst=False) }}
scripts:
'salt_bootstrap.sh': |
{{ cloud_init_script(env, app, extra_data=extra_data) | indent(width=13, indentfirst=False) }}
- availability_zones:
- {{ env.region }}a
- {{ env.region }}b
- vpc_zone_identifier:
- {{ app.vpc_subnets.dialer_1 }}
- {{ app.vpc_subnets.dialer_2 }}
- min_size: {{ app.min_size }}
- max_size: {{ app.max_size }}
- desired_capacity: {{ app.desired_capacity }}
- region: {{ env.region }}
- tags:
- key: 'country'
value: {{ dialer_country }}
- key: 'Name'
value: {{ dialer_name }}
propagate_at_launch: True
- key: 'Environment'
value: {{ app.tags.env }}
propagate_at_launch: True
- key: 'Role'
value: {{ app.tags.role }}
propagate_at_launch: True
- order: last
{% endfor %}
Steps to Reproduce Issue
$ salt "*master*" state.sls dialer.generic
master-10-155.eu-west-1a.qa-public:
----------
ID: dialer_qa-v2
Function: boto_asg.present
Name: qa-public-dialer-qa-v2
Result: False
Comment: An exception occurred in this state: Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1905, in call
**cdata['kwargs'])
File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1830, in wrapper
return f(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/salt/states/boto_asg.py", line 524, in present
asg = __salt__['boto_asg.get_config'](name, region, key, keyid, profile)
File "/usr/lib/python2.7/dist-packages/salt/modules/boto_asg.py", line 172, in get_config
ret[attr] = getattr(asg, attr).split(',')
AttributeError: 'NoneType' object has no attribute 'split'
Started: 16:10:08.655691
Duration: 348.307 ms
Changes:
----------
ID: dialer-qa
Function: boto_asg.present
Name: qa-public-dialer-qa
Result: False
Comment: An exception occurred in this state: Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1905, in call
**cdata['kwargs'])
File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1830, in wrapper
return f(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/salt/states/boto_asg.py", line 524, in present
asg = __salt__['boto_asg.get_config'](name, region, key, keyid, profile)
File "/usr/lib/python2.7/dist-packages/salt/modules/boto_asg.py", line 172, in get_config
ret[attr] = getattr(asg, attr).split(',')
AttributeError: 'NoneType' object has no attribute 'split'
Started: 16:10:09.004262
Duration: 217.633 ms
Changes:
Versions Report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
| $ salt --versions-report
Salt Version:
Salt: 2018.3.1
Dependency Versions:
cffi: Not Installed
cherrypy: 3.2.2
dateutil: 2.7.3
docker-py: Not Installed
gitdb: 0.5.4
gitpython: 0.3.2 RC1
ioflo: Not Installed
Jinja2: 2.7.2
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: 0.9.1
msgpack-pure: Not Installed
msgpack-python: 0.4.6
mysql-python: 1.2.3
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.6 (default, Oct 26 2016, 20:30:19)
python-gnupg: Not Installed
PyYAML: 3.13
PyZMQ: 14.0.1
RAET: Not Installed
smmap: 0.8.2
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.0.5
System Versions:
dist: Ubuntu 14.04 trusty
locale: UTF-8
machine: x86_64
release: 3.13.0-116-generic
system: Linux
version: Ubuntu 14.04 trusty