Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ASG updates #23

Open
antoinefalisse opened this issue Nov 21, 2024 · 0 comments · May be fixed by #24
Open

ASG updates #23

antoinefalisse opened this issue Nov 21, 2024 · 0 comments · May be fixed by #24
Assignees
@sashasimkin

Comments

@antoinefalisse
Copy link
Collaborator

@sashasimkin We protect the EC2 from being killed, but not the docker containers that are running there. So if they are still running when ECS tries to kill things, then the containers are forcefully stopped. As a consequence, trials being processed keep status "processing" and scale-in protection stays on meaning the EC2s stay on. We need to change the logic and protect the task instead of the EC2 (or both).

Action items: @sashasimkin will provide instructions about how to solve this.
sashasimkin added a commit that referenced this issue Nov 26, 2024
@sashasimkin
feat: Use ECS task protection instead of EC2 instance protection for …
563887f 
…auto-scaling

This will let tasks complete and not be force-terminated by ECS

fixes #23
@sashasimkin sashasimkin linked a pull request Nov 26, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sashasimkin sashasimkin

Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@sashasimkin @antoinefalisse