VA Data Modernization: A Reversible Path from Legacy to Cloud

Why VA Legacy Migrations Stall

VBA claims processing and VHA clinical systems were built over decades on mainframe and client-server architectures that were never designed to talk to each other, let alone to a modern lakehouse. Some of these systems predate the internet. Documentation is thin, the people who wrote the original COBOL are retired, and the business logic lives in undocumented edge cases that only surface during an audit or a benefits appeal.

The instinct in most modernization programs is to pick the target cloud platform first: stand up the new environment, buy the tooling, and start moving data. This is backward. Every VA modernization effort that has stalled or been restarted did so because the team did not know what it was migrating before it started migrating it. You cannot reconcile what you never profiled, and you cannot promise Veterans continuity of benefits if you are guessing at the source system's actual behavior.

The fix is sequencing: inventory and profile the legacy estate exhaustively before you write a single line of target-state schema. This is slower up front and dramatically faster overall, because it eliminates the rework that comes from discovering mid-migration that a field you assumed was clean is actually three different encodings stacked on top of each other.

Step One: Inventory and Profile Before You Design Anything

Start with a full data inventory: every table, every flat file, every batch job, every interface agreement between VBA and VHA systems, and every downstream report that depends on them. This includes shadow systems, the spreadsheets and Access databases that regional offices built to fill gaps in the official system of record. Those shadow systems often hold the business logic nobody documented anywhere else.

Profile every field for actual content, not documented content. Null rates, distinct value counts, format drift over time, and referential integrity violations all need to be measured against the real data, not the data dictionary. In systems that have been in production for 20 to 30 years, the documented schema and the actual data almost never fully agree, and the gaps are exactly where migrations break.

Map data lineage end to end: where a claims status code originates, what transforms it along the way, and every consumer that reads it downstream. For VBA and VHA, this lineage work also doubles as a control mapping exercise against NIST 800-171 requirements, since you need to know where sensitive Veteran data lives and moves before you can certify the target environment meets the same protection level.

Step Two: Define the Target Lakehouse Around Actual Use, Not Aspiration

Once the inventory is done, define the target lakehouse architecture based on how the data is actually used today, not on a theoretical future state. A lakehouse pattern, raw and curated zones with governed access controls, fits federal data well because it lets you land data close to its native form first and apply transformation and quality rules in a separate, auditable layer, rather than forcing transformation at ingest where errors are hard to trace back.

Design the target schema to accommodate the mess you found in step one, not to erase it prematurely. If a legacy field has three inconsistent formats because of a 2003 system change, the target model should be able to represent all three states with a clear conversion rule, not silently collapse them and hope no one asks about the missing 200,000 records from before 2003.

Build access and identity controls into the target design from day one, not as a bolt-on. For any system touching Veteran PII or clinical data, this means role-based access aligned to least privilege, audit logging on every read and write, and encryption standards that meet or exceed 800-171 control families for access control and audit accountability. Retrofitting these controls after the data has landed is far more expensive than designing for them up front.

Step Three: Migrate in Reversible Waves

Never plan a single cutover for a system that Veterans depend on for benefits or care. Break the migration into waves defined by business domain, not by technical convenience: a wave might be a single claim type, one facility's scheduling data, or one benefit category. Each wave should be small enough to validate fully and reversible enough that a bad wave can be rolled back without touching anything else.

Reversibility means the legacy system keeps running in parallel until its replacement wave is proven, not decommissioned on a schedule set by a project plan. This dual-running period costs more in infrastructure, but it is the only honest way to guarantee mission continuity. A benefits determination or a clinical order cannot be delayed because a migration wave needed a few extra weeks.

Sequence waves from lowest risk to highest. Start with read-only reporting data where an error is inconvenient but not harmful, build confidence and tooling maturity, then move toward the systems that directly drive payments and care decisions. By the time you touch the highest-risk domains, your reconciliation tooling and rollback procedures have already been proven under real conditions.

Step Four: Reconcile Every Record, Not a Sample

Sampling is not acceptable for VA data migration. Every record that moves from legacy to target must be reconciled against the source, field by field, with discrepancies logged and resolved before the wave is declared complete. For high-volume domains this means automated reconciliation pipelines that run continuously during the dual-running period, not a one-time spot check before go-live.

Reconciliation needs three layers: count reconciliation confirming the right number of records moved, value reconciliation confirming field-level accuracy, and behavioral reconciliation confirming that downstream calculations, like a disability rating or a copay determination, produce the same results in both systems given the same inputs. Behavioral reconciliation is the layer most programs skip, and it's the layer that catches the business-logic bugs that count and value checks miss entirely.

• Count reconciliation: source row counts match target row counts within documented, explained exceptions
• Value reconciliation: field-level diffs run on 100% of records, not a sample, with every discrepancy triaged
• Behavioral reconciliation: parallel-run outputs (ratings, payments, eligibility) match between old and new systems
• Sign-off gate: no wave closes until reconciliation logs are reviewed and accepted by a named business owner

How VAERESOURCE Builds This Way

This roadmap only works if every step leaves an audit trail. We build migration pipelines that are fail-closed by default: if reconciliation logic can't confirm a record matches, that record is held and flagged, not passed through silently. That single design choice prevents the quiet data corruption that turns into a benefits dispute or a clinical data error months after go-live.

We keep a human in the loop at every wave gate. Reconciliation reports, exception logs, and rollback decisions go to a named reviewer before a wave is declared closed, because automated tooling should surface judgment calls to people, not make them invisible. Every transform, every mapping decision, and every exception resolution is documented so the migration can be audited years later, which matters for a system that will outlive the contract that built it.

This is deliberately not a fast approach, and we don't pretend otherwise. It is the approach that lets a VA program office tell Congress, the IG, and Veterans themselves exactly what moved, what changed, and why they can trust the result.