HalfAtATime deployment with an ELB/ASG
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- aws, shell
- Domain
- cloud, devops, infrastructure
Research direction
Locate the deployment scripts that call autoscaling_enter_standby() and review how they adjust the Auto Scaling Group min and desired sizes. Reproduce a CodeDeploy HalfAtATime deployment with more than two EC2 instances, then verify that concurrent standby operations complete without the ASG ValidationError.
Written by the indexing model from the issue text.
Description
We currently use these scripts during deployment to deregister our instances from an ASG during deployment. However, we've discovered what appears to be a race condition when using HalfAtATime deployment in CodeDeploy and using more than 2 EC2 instances in the ASG.
For example, if you have 4 EC2 instances behind an ASG and run a HalfAtATime deployment on them from CodeDeploy, it will run the deployment scripts against 2 of the 4 at the same time. From there, one of them will instantly fail citing:
[stderr]An error occurred (ValidationError) when calling the EnterStandby operation: AutoScalingGroup [DeploymentGroup] has min-size=3, max-size=70, and desired-size=3. To place into standby 1 instances, please update the AutoScalingGroup sizes appropriately.
I believe this is because both EC2 instances are running autoscaling_enter_standby() at the same time. So they both scale the ASG min and desired down to 3 at the same time and then attempt to move into standby, however only one gets in. When the other makes the attempt, it fails because by that time there are 3 in the ASG and you can't take another one away.
Is there any way to adjust the script to account for this issue? Without that, HalfAtATime deployment is pretty useless given that you're limited to just 2 instances in the ASG at a time.
- Dominant language
- Shell
- Stars
- 639
- Forks
- 777
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from aws-samples/aws-codedeploy-samples
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
aws-samples/aws-codedeploy-samples#98 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
All issues in aws-samples/aws-codedeploy-samples
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
community-scripts/ProxmoxVE#17425 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
danielmiessler/LifeOS#2218 ·
-
docs(agents): strengthen the no-backslash-escaped-backticks rule with an issue-creation example Open
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
technical-debt
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
ll7/robot_sf_ll7#9560 ·
-
package-update
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
oSoWoSo/vOid_Community_repOsitory#148 · 1 comment ·