โšก Onwuachi Control Plane

Container Resource Patching Runbook

Overview

Procedure for temporarily replacing a resource file inside a running container without rebuilding or redeploying the container image.

This method is intended for controlled, temporary remediation when a permanent image-based fix is not immediately available.

A resource-level patch should be treated as a temporary operational change, not a replacement for updating the underlying image.


Prerequisites


Procedure

1. Connect to the Host

ssh <instance-host>

Obtain the required privileges:

sudo -i

2. Verify the Container Version

Before changing anything, determine exactly which image/version the container is running.

Examples:

docker ps
docker inspect <container-name> \
  --format '{{.Config.Image}}'

If the application exposes its version through logs:

docker logs <container-name> | grep -i version

Do not assume that every container requires the patch.


3. Confirm the Target File Exists

Inspect the container:

docker exec <container-name> ls -la /path/to/resource/

If the container does not have a usable shell, a minimal shell utility may need to be copied into the container.

For example, if BusyBox is available on the host:

docker cp /usr/bin/busybox \
  <container-name>:/tmp/busybox

Then:

docker exec -it <container-name> /tmp/busybox sh

4. Back Up the Existing File

Inside the container:

cd /path/to/resource/

Create a backup:

cp resource-file resource-file-bak

Verify:

ls -l resource-file*

Never overwrite the original file before creating a rollback copy.


5. Copy the Patched File Into the Container

From the host:

docker cp \
  /path/to/staged/patch/resource-file \
  <container-name>:/path/to/resource/resource-file

Verify:

docker exec <container-name> \
  ls -l /path/to/resource/resource-file

If appropriate, compare the file:

docker exec <container-name> \
  md5sum /path/to/resource/resource-file

6. Verify Application Behavior

Determine whether the application automatically reloads the resource.

If it does not, the container/application may need to be restarted:

docker restart <container-name>

Only restart when the operational impact is understood.

Then verify:

docker ps
docker logs <container-name> --tail 100

Confirm the specific behavior being remediated is resolved.


Important Gotchas

1. Container filesystem layouts can differ

Do not assume the target path exists in every container.

If:

cd /path/to/resource/

returns:

No such file or directory

stop and inspect the image/container layout.

The container may be running a different version or use a different filesystem structure.


2. Always create a rollback copy

Before replacing:

cp resource-file resource-file-bak

Without a backup, rollback may require restarting/redeploying the container from its original image.


3. Verify the version before patching

A resource patch may only apply to certain application versions.

Check the running version first and skip containers where:


4. Track every modified container

Maintain a simple record:

HostContainerVersionPatchedReason
<host><container><version>Yes/No<reason>

This makes subsequent cleanup and permanent remediation much easier.


5. Remember that container changes are ephemeral

A file copied into a running container’s writable layer can disappear when the container is replaced.

A later:

docker pull
docker compose up

or equivalent redeployment may restore the original image contents.

Therefore, a manual container patch is not a permanent fix.


Rollback

If the patch causes problems:

docker exec <container-name> \
  cp /path/to/resource/resource-file-bak \
     /path/to/resource/resource-file

Verify:

docker exec <container-name> \
  ls -l /path/to/resource/resource-file

Restart the container if required:

docker restart <container-name>

Then verify application behavior.


Post-Patch Checklist


tmux / nohup

For container patching, tmux is strongly preferred because the operator may need to inspect logs, execute rollback commands, or interact with the container.

Start:

tmux new -s container-patch

Detach:

Ctrl-b
  โ†“ release
d

Reattach:

tmux attach -t container-patch

List sessions:

tmux ls

Do not use nohup for an interactive patch procedure unless the actual operation has been reduced to a non-interactive script that can safely run unattended.

For fire-and-forget work:

nohup ./script.sh > /path/to/out.log 2>&1 &

System Context

โ† Back to Kb