Emergency Fix in Production

Owned by Kevin Wu
Last updated: Jan 03, 2024
2 min read

In this scenario, a critical problem is detected in the production system, which requires an immediate fix.

Chain of events:

  1. Team D detects a critical problem in the production

    that needs to be fixed quickly and notifies Teams A and C.
     

  2. Team D resolves the problems in the configurations and performs an export of the configurations which are then checked into the Version Control System.
     

  3. If Teams A and C are not currently working on the configurations in question, they can just make an update in their working copy and import the updated configurations in to their respective

    systems to retrieve the fix.
     

  4. However, if Teams A and C are in fact working with the configurations in which the fix has been made, great care must be taken in order to incorporate the fix into their work so it is not accidentally overwritten.

For smaller changes, the easiest way would be to make the same changes again in the development/staging systems.

For larger changes, you may want to use the merge functionality in the Version Control System. However, whether this will work or not depends on the nature of the changes as well as the capabilities of the Version Control System.