Brown Bag: Auto Deploy & "Release A" Release-Information
Recorded walkthrough of automated deployments and the "Release A / Release B" fix-version convention in Jira. Recorded deliberately because teams were just coming online with auto-deployments
| This is current (2026) content. Process mechanics are likely still accurate, though a few personnel/access issues were flagged as open. |
|
Authoritative source
This page is a record of a recorded brown-bag session and is preserved as-is.
For the current, maintained process — including the correct deployment
blackout window and the |
TL;DR
Auto-deploy doesn’t change the ticketing, it reorders it. POs deploy the release themselves; release managers (Eric) create the NGD ticket ahead of time but do not submit until the deployment is done. The NGD subtask gets assigned to the PO who actually did the deploy, marked done, then submitted to CCB as a retro record.
Timing is gated. Deploys wait for a set time (example: 4:30). PO posts in releases channel, tags release managers, then at deploy time tags the SRE team / on-call rotator so SRE is watching/monitoring.
Rollback under auto-deploy is counterintuitive: to redeploy the previous version you go to the next comp tag UP, not back down — because the manifest has to change to trigger a redeploy. If comp 3 is in prod and you push new, rolling "back" is actually comp 4.
Release A/B fix-version convention: Name Jira fix versions service-name: release a (and release b, etc.) and dump the stories in.
You don’t need to know the version number yet; rename to the real version (e.g. 1721) only when you cut it.
| Services that never go to prod (testers, sandbox RPC libs) should NOT have fix versions. |
Auto-Deploy + Release-Manager Process Flow
The Mechanics, In Order
1. Heads-up post
POs know they’re doing an automated deploy that afternoon. Deploys are time-gated (example: 4:30). PO posts in the releases channel, tags release managers, says e.g. "VCAS 123, doing it at 4:30."
2. Pre-stage the NGD ticket
If the release manager has bandwidth, they create the NGD ticket ahead of time but do NOT submit it. Submission only happens after the deployment is over.
3. Assign and mark the subtask
The subtask gets assigned to the PO and marked done. The release manager is just submitting the ticket as a retro record afterward.
These can be queued same-day or done the next day.
4. Deploy-time SRE tag
At deploy time (~4:30) the PO tags the SRE team, or the specific on-call rotator if known, so SRE is ready to watch and monitor as the deploy goes out.
5. Deploy runs
PO pushes the release. Everyone watches, agrees it’s healthy, moves on.
6. Submit the retro ticket
Release manager submits the ticket showing the PO did the deploy. Release manager verifies the ticket has the right version and config tag — flagged specifically because there have been cases of a patch change or config change between cut and deploy.
7. CCB review → close
Ticket goes to CCB, they review, send it back to the release manager, who closes it out and closes the release.
Rollback
POs own the rollback. Since POs push the release, they also do the rollback if something looks bad. However POs are expected to talk it through with the release manager and SRE team before rolling back, unless it’s a full outage. The host explicitly declined to detail the rollback mechanics on camera ("I won’t say how the actual roll back process works" — there have been "some issues").
Release-Manager Side of Rollback
Create an additional subtask recording that they rolled back to the previous version, as part of the NGD package that goes to the platform team.
Approval Rules
Fully down / no traffic getting through: Roll back immediately, don’t wait for anyone. "We’re not going to leave Acheron or VCAS down… we’re not going to wait for someone to check something." Let the release manager handle the logistics/fallout.
Partially broken (not urgent): Talk it through first, get everyone on the same page.
Scheduling side: Technically requires VA Contract Partner’s approval — but "it is as simple as just a Teams message to him." Get the checkmark and go. If he doesn’t respond, tell Ryan and Ryan gets it within a minute. (Noted as a constraint only on the scheduling side, not shared services.)
Shared-services side: "We can just click the button and do it." No approval needed.
| VA Contract Partner performs scheduling-side rollback approval. Single-point approval is a latency risk if both the VA Contract Partner and Ryan Lemire are unavailable. |
Rollback Under Auto-Deploy — The "Next Comp Tag Up" Gotcha
This is the load-bearing technical nuance of the whole session.
Normally a release ticket lists the rollback version as the old version. Under auto-deploy that is not the case.
To redeploy a prior version you go to the next comp tag UP, because the manifest has to change for Argo/Skaffold to redeploy. (verify: tool is Argo CD via Skaffold)
Worked Example
-
Comp 3 in prod now
-
Push new version
-
Need to redeploy the previous one
-
It is Comp 4 now
The Manifest Requirement
The manifest change must be the very last change, in the commit that’s tagged. The pipeline doesn’t just read "what’s in the manifest JSON" vs "what’s the release version" — it verifies the manifest was actually changed to that version in that tagged commit.
So you cannot: . Change the manifest + commit . Separately update the changelog, commit, tag, and push to prod
That sequence won’t deploy.
You can put the tag (not just the version) in the manifest. Mismatches between the manifest text and the tag are a common failure mode — a typo between the two silently fails the deploy.
Reported Real Failures
-
"Sitting there waiting on nothing" — deploy never triggered because the manifest text wasn’t right.
-
Typo between manifest and tag → deploy didn’t go correctly → had to redo, cascading "configuration 10 tags all the way up."
-
People (incl. the speaker) double/triple-check that manifest and tag line up and still occasionally get one wrong and have to redo.
| "Rollback goes to the next comp tag UP, not down" — this is the load-bearing technical fact. The manifest must change in the tagged commit; rollback is comp N+1, not N-1. |
The Black-Box Problem: POs Can’t See Deploy Failures
POs don’t have access to the project, so they can’t see why a deploy failed or didn’t deploy.
Current workaround: ask someone to look or file a ticket.
Quote: "It’s kind of a black box once we make that change and hope it comes out the other side."
Standing Ask: Read Access for POs
Read access for POs to their own deployments would let them self-troubleshoot. Framed half-jokingly as a "parting gift to us" (several people on this call are apparently rotating off). Noted they can’t control whether that access gets granted.
| Check whether PO read access to deployments has been granted — it changes the self-service troubleshooting story. |
"Release A / Release B" Fix-Version Convention (Jira)
Demoed against a real project (used as the model).
When you create a fix version, name it service-name: release a and dump in whatever stories are going into your next version.
You don’t need to know the version number yet.
When you’re ready to cut the version and create the ticket for the release manager (or for auto-deploy), you can either:
* Leave it as release a, or
* Rename it to the real version once you know it (e.g. 1721).
Point of the Convention
Avoid renaming a fix version "a hundred times." Release manager tags it on the release-manager ticket and goes from there.
Release B for Parallel Flexibility
release b gives you parallel flexibility.
Example: Two versions were running for a service — a release a with current changes and a release b already staged in case something needed to go in separately.
If you have something you have to pull over, throw it on a separate release letter.
Minor note: Some projects had lettering tweaked from B and C → A and B so the lettering starts at A.
Services That Never Reach Prod → No Fix Version
Certain services never go to prod (testers are SQA-only; RPC libraries are sandbox-only).
Do not create fix versions for those. A fix version "would be seen by somebody else."
Rule, stated flatly: "Fixed versions are only for a version that is going to prod." Multiple other never-to-prod projects exist with no fix versions. Leaving fix-version blank on those tickets is correct, even though it "feels wrong."
What’s Still Accurate (2026)
-
"Rollback goes to the next comp tag UP, not down" — this is the load-bearing technical fact.
-
Release A/B fix-version convention — still the pattern for staging stories before you know the version number.
-
Non-prod services get no fix version — still the rule.
-
SRE tagging at deploy time — still expected.
-
NGD ticket created before deploy but submitted after as a retro record — still the flow.