Change Metadata Schemas Without Breaking Integrations

Expand the contract, migrate stored values, and verify each consumer before retiring a field

Published Updated

Change Metadata Schemas Without Breaking Integrations. Three flat cyan panels show expanding a field definition, migrating values, and verifying the old field can retire, labelled Expand, Migrate, and Retire, above the official FormKiQ logo.

Treat a document metadata change as a release of a shared data contract. Identify the applications that write and read the field, introduce the replacement alongside the existing representation, migrate the affected records, and verify each consumer before retiring the old field.

Consider a field called vendor that contains supplier names. Replacing it with supplierId changes both the key and the meaning: a name must now resolve to a particular account in the ERP. Search filters, routing rules, access policies, reports, and exports may all depend on the earlier representation. An upload succeeding tells you little about whether those consumers still work.

Define what the field means before changing its shape

Write down the business fact, owner, scope, data type, cardinality, allowed values, and treatment of missing or unknown data. Include whether a value is current reference data or a fact captured when the document was issued. A supplier's current display name and the name printed on an older invoice may both be useful; give them distinct meanings.

A contract includes the behaviour consumers rely on. Google's API compatibility guidance distinguishes source, wire, and semantic compatibility, and treats a rename as removal plus addition. Apply that reasoning to your own document integrations: the same JSON type can still carry a different business fact.

FormKiQ v1.19.2 also supports JSON attributes with nested-field predicates. Moving flat fields into a structured object changes the contract for existing consumers. Test the JSON paths and missing values explicitly. DynamoDB filters those properties after reading candidates; selected Enhanced Search capabilities index JSON in OpenSearch. Storing an object does not create a DynamoDB index for every nested property.

Keep a release record with the old and new definitions, affected sites or classifications, migration scope, owners, and acceptance criteria. A label such as “contract version 2” can identify this application agreement; it should not imply that every connected system automatically versions or migrates its schemas. For schema fundamentals, see document schemas and metadata validation.

Classify the change and its compatibility risk

A release plan for common metadata changes.
Proposed changeWhat to establish before rollout
Add an optional fieldCheck that writers may send it and readers tolerate its presence or absence. Keep existing behaviour for documents without a value.
Make a field requiredUpdate active producers and resolve in-scope historical gaps before enforcing the rule. Define exceptions without inventing misleading default values.
Rename or split a fieldIntroduce the replacement, choose an authoritative writer and compatibility behaviour, migrate values, and obtain consumer acceptance before retirement.
Change a type or allowed valuesDefine explicit conversion and handling for rejected values. Check parsers, filters, routing, and reports against the proposed representation.
Change the business meaningUse a distinct field or explicitly versioned contract. Preserve the earlier fact and explain how the new value was determined.

Adding a field is a useful first step, but compatibility depends on the consumers. A strict schema may reject an unexpected key; a report may treat a missing identifier differently from an empty string. Include these behaviours in the acceptance tests.

Inventory both producers and consumers

Trace the field through intake forms, APIs, extraction mappings, administrator edits, rules, search filters, access decisions, scheduled reports, exports, and event payloads. Assign an owner to each dependency. Include infrequent jobs and older documents that appear only during a renewal, audit, or recovery operation.

Search needs its own check: a field can be written correctly while a saved query still uses the old key or an index has incomplete coverage. Record the affected query patterns and how their result identities will be compared. The composite-key and OpenSearch guide explains why query design belongs in that inventory.

Expand, migrate, and retire in separate steps

  1. 1 · Expand

    Introduce the replacement

    Permit the new representation while supporting agreed existing writers and readers. Document precedence and reject conflicting inputs.

  2. 2 · Migrate

    Move values and consumers

    Convert historical values in controlled batches, resolve exceptions, update integrations, and compare results on mixed records.

  3. 3 · Retire

    Close the compatibility window

    Confirm consumer adoption and migration outcomes, end legacy writes, and approve the remaining retention or removal of old metadata.

Choose one component to own each business fact. If compatibility requires two representations, define how that owner derives and updates them. Two independent writers can produce conflicting values. Record failed updates and reconcile them before treating a document as migrated.

Set an intended transition window, then use evidence to close it. A calendar deadline alone cannot show that an old client has stopped sending the previous key. Measure old-key writes and reads where observable, inspect scheduled consumers, and obtain the owners' acceptance.

Worked example: replace supplier names with ERP identifiers

This fictional migration covers 12,000 supplier documents. The old vendor attribute contains free-form names. The new contract uses supplierId for the ERP account and supplierName for a defined display value. The names and counts below illustrate the process, not a reported migration result or performance target.

  1. 1. Publish the definitions and lookup rules

    The ERP owns supplier identity. The integration resolves a legacy name using approved account data and business-unit context. “Cedar Supply” is ambiguous when it could identify two accounts. Those records enter a review queue instead of receiving a guessed identifier.

  2. 2. Introduce the fields and compatibility path

    Allow the replacement fields before new producers send them. The configured integration looks up the ERP account and writes the selected identifier and display value. A legacy export keeps its documented name-based response during transition. Conflicting old/new inputs are rejected for correction under the agreed rule.

  3. 3. Backfill with a record of each outcome

    An initial pass resolves 11,400 documents, leaves 450 ambiguous matches, and identifies 150 with insufficient source data. The migration records document identity, earlier value, selected account, conversion-rule version, time, and result. It uses a controlled write window or supported concurrency check so a batch cannot silently overwrite a newer correction.

  4. 4. Reconcile the records and consumers

    The initial totals account for all 12,000 documents, but 600 still need a decision. Reviewers resolve or explicitly exclude those cases under the migration scope. The team then tests search, routing, access, reports, and ERP exports on old, migrated, newly created, and corrected documents. Comparing returned document IDs catches errors that equal result counts would hide.

  5. 5. Retire only the agreed legacy behaviour

    After the named consumers adopt the identifier and new writes conform, the owner ends legacy writes and retires the compatibility response. The release record preserves the earlier definition and conversion decisions. Original invoice content remains historical evidence; any removal of old metadata follows the applicable record policy and scope.

Treat access and recovery as release criteria

A fallback search can sometimes improve availability during migration, but an access rule needs an explicit authorisation decision. Test the intended allowed and denied document sets for missing, ambiguous, and conflicting values. Avoid turning “unknown supplier” into broad access or combining old and new rules in a way that expands permissions unexpectedly.

Keep a recovery path that can handle data already written under the new contract. Reverting application code alone may leave new values that an older consumer cannot interpret. Preserve the earlier representation where permitted, define reverse conversion only where it is meaningful, and plan forward repair for lossy or ambiguous transformations.

Before retrying a failed batch, identify which documents changed successfully and which changed again afterward. A repeatable migration should recognise completed work and route conflicts for review. Retain enough change evidence to explain the outcome without copying unnecessary sensitive values into general logs.

Approve one metadata release

  • Define the old and new business facts, types, scope, ownership, and handling of unknown values.
  • Name the owners of producers, consumers, saved queries, access rules, and scheduled jobs.
  • Permit the new fields before producers use them, with explicit compatibility and conflict rules.
  • Account for migrated, rejected, ambiguous, excluded, and newly arriving records.
  • Compare search results and access decisions on historical, new, and mixed records.
  • Test recovery with values already written by the new application release.
  • Record consumer acceptance and the authorised retirement decision.

Retire a field when the evidence supports it

A metadata release is complete when the intended values are reconciled, important consumers use the agreed contract, and exceptions have an accountable outcome. Keep definitions, migration evidence, and recovery decisions together so the next field change can build on a known state.

      Start with FormKiQ Core

      The open-source foundation — API-first, deployable into your own AWS account, and free to use. Right for architecture validation and early implementation.

      Get Started Free

      Deploy FormKiQ Essentials or Advanced

      Production-ready editions for departments and complex workflows. Start with a Proof-of-Value deployment or go straight to production.

      Explore Options

      Plan an Enterprise Rollout

      For governance-heavy environments with residency, sovereignty, assurance, and multi-jurisdiction requirements. Talk to us about the right deployment model.

      Book a Call