Write the Rollback Plan Before Changing a Service

“We can roll back” is easy to say when a change only exists in a branch. It becomes a harder statement after a service has accepted new writes or a configuration change has reached several machines.

Before changing a service, write a short reversal plan. The act of writing it often reveals that the proposed change contains more than one kind of risk.

Name the starting point

Record the currently deployed revision and the configuration source that actually controls the running service. Do not substitute the latest commit in your checkout. If the host has local configuration, account for it explicitly without copying secrets into the plan.

Identify any recovery checkpoint you need. An application image, a configuration backup, and a database backup solve different problems. Choose according to what the change can alter rather than taking a random snapshot and calling the job done.

Define a trigger

A rollback trigger should be observable. “The service feels wrong” leaves too much room for improvisation. “The representative read-and-write check fails after startup completes” gives the operator a decision point.

Choose a bounded observation period suitable for the service. Do not invent a universal number. A batch worker and a request-serving application become useful on different schedules.

Separate code from data

Reverting an application revision does not automatically reverse data changes made by that revision. Ask whether the previous application can read the new data and whether returning to a backup would discard legitimate writes.

If the answer is unknown, the plan is incomplete. Narrow the change, test compatibility, or make the data transition a separately reviewed operation. Avoid promising a one-command rollback when the data boundary requires a human decision.

Write the smallest executable note

Include the target, current revision, proposed revision, checkpoint identifier, reversal procedure, verification action, and stop condition. Link the established runbook rather than copying a long command sequence that will drift.

State what must remain untouched. A failed web release should not lead to an unrelated database restore or DNS rewrite simply because those controls are available.

Verify after reversal

Rollback is another change. Confirm the running revision, check the service, and exercise the representative action again. Keep the evidence for both attempts so the next investigation can distinguish a bad release from an environmental failure.

A useful rollback plan reduces the number of decisions you must make while something is broken. It does not eliminate judgment, but it puts the important judgment before the pressure.