How to Mask Data Consistently Across Microservices and Shared Databases

Updated 18 June 2026.

Take a bank with four services: core banking on Db2, loan origination on Oracle, customer identity on PostgreSQL, and document storage on S3. Each service holds a customer_id field and also has its own development team with its own independent non-production environments.

When those teams mask their databases and applications independently, customer_id 12345 becomes four different values across four systems, creating a consistency problem that breaks relationships that are critical for end-to-end testing and AI fine-tuning or training.

And this isn't just an edge case. This is a challenge any enterprise with more than two services sharing an entity identifier will encounter.

---

Why Microservices Make Data Masking Harder

Most data masking tools were built for monolithic databases, with one schema and one masking job. Foreign key relationships live in the same database, so the tool traverses them in a single pass and consistency is never really a problem.

Microservices break that assumption. The data is spread across services running different database engines: PostgreSQL here, Oracle there, DynamoDB somewhere else. Each service is owned by a separate team running masking on its own schedule. And the same entity identifier, such as a customer_id, an account_number, or a policy_id, exists independently in each one, meaning no single masking job sees the full picture.

Random replacement works within a single service, but when two services share an entity identifier, the masked values won't match.

---

The Three Consistency Requirements

Cross-service data masking requires three distinct forms of consistency, and a strategy that satisfies only one or two of them still leaves non-production environments functionally broken. DataMasque addresses all three through a single deterministic masking mechanism so teams do not need to solve each requirement separately.

• Within a service: customer_id 12345 must mask to the same value in every table that holds it: customers, repayments, audit_log, notifications. This is standard referential integrity that DataMasque handles automatically.
• Across services: The masked customer_id in the core banking Db2 database must be identical to the masked customer_id in the loan origination Oracle database, even when the masking jobs run at different times and are triggered by different teams.
• Across environments: Dev, staging, and UAT must all produce the same masked value for the same source value. A bug that surfaces with customer_id 12345 in dev must be reproducible in staging, using the same masked data shape.

All three requirements reduce to one engineering constraint: the masking function must be deterministic. The same input must always produce the same output, regardless of when or where the job runs.

---

How Deterministic Masking Solves the Cross-Service Consistency Problem

DataMasque's run secret mechanism turns masking into a pure function: mask(value, run_secret) yields masked_value. The run secret is a cryptographic seed shared across all masking jobs in an organisation. Any masking job that receives the same run secret will produce the same masked value for the same input, always, with no lookup table to maintain and no coordination required between teams.

There is no processing order dependency. The team running the core banking masking job and the team running the loan origination masking job can execute independently on different days; the outputs will be structurally identical because both jobs received the same seed.

The run secret is never stored by DataMasque. It is injected at masking time from the platform or DevOps engineer's secrets manager. This means that even if someone gains access to the DataMasque configuration, the mapping from original values to masked values cannot be reconstructed.

---

Before and After: What Inconsistent Masking Looks Like

The problem is most visible when you lay four services side by side. With random per-service replacement, customer_id 12345 produces a different masked value in each system. Every cross-service join in your test environments fails.

---

Applying hash_columns in Practice: Cross-Service Consistency

The mechanism for cross-service consistency is the hash_columns parameter in DataMasque's YAML ruleset. It specifies which column value is used as the seed when computing a masked output. Set the same hash_columns value across every service's YAML ruleset, inject the same run secret at job execution time, and the output is identical across all services, regardless of which team ran the job or when.

Here is what this looks like in a DataMasque YAML ruleset for the core banking service (Db2):

Every service masks customer_id with the identical imitate_unique config. Because it's a format-preserving cipher keyed by the shared run secret, and not a per-database random draw, customer_id 12345 resolves to the same value everywhere, and cascades to foreign-key references automatically. Same run secret + same instance secret (or disable_instance_secret: true). Column and table names may differ freely; only the seed values and the mask config must match.

The run secret can be shared across teams using AWS Secrets Manager or your CI/CD platform's native secrets store. Each team's masking pipeline retrieves the secret at runtime and passes it to the DataMasque job. No team needs visibility into another team's masking configuration, just access to the shared secret.

ADP described their experience this way: DataMasque was "integrated into our toolkit to guarantee irreversible masked data in our test environments, offering both agility and simplicity."

---

Consistency Across the Full Data Tier

Microservices rarely store sensitive data only in relational databases. A masking strategy that covers only SQL tables leaves gaps wherever the same entity identifier appears in files, event payloads, or object storage.

DataMasque covers the full range of storage types common in a modern services architecture using the same run secret and hash_columns approach throughout, removing the need for additional tooling.

A document in S3 with a customer_id in its JSON body gets the same masked value as the corresponding row in PostgreSQL, as long as both jobs use the same seed and run secret. One tool, one config, consistent output across the full stack.

---

Event-Driven Architectures and Kafka Topics

To keep entity references consistent in event streams, DataMasque uses in-flight masking: a real-time masking service that is part of the DataMasque platform. It is deployed alongside your DataMasque install and shares its instance and run secrets, applying exactly the same deterministic masking as the batch database jobs. Rather than running a batch job against a table, in-flight masking sits inline in your streaming pipeline. A Kafka consumer, Kafka Streams application, or Kafka Connect transform calls it to mask each event as it passes through.

Because in-flight masking uses the same run secret as your database masking jobs, the masked customerId in an event matches the masked customer_id in PostgreSQL exactly. A downstream consumer that joins event data against a masked relational database sees the same value in both places, with no extra coordination required.

---

Frequently Asked Questions

How do composite keys work with deterministic masking?

DataMasque supports composite hash_columns configurations for entities identified by more than one field: if your entity is identified by a combination of tenant_id and customer_id rather than a single column, set hash_columns: [tenant_id, customer_id] and the masking function uses both values as a combined seed, guaranteeing the same masked output for the same combination every time. Apply the identical hash_columns list in every service's YAML ruleset and the cross-service consistency guarantee holds.

Can deterministic masking work across cloud providers and regions?

DataMasque's run secret and hash_columns mechanism is stateless and has no dependency on network topology, cloud provider, or region, so a masking job running on AWS and a job running on Azure will produce identical masked values for identical input, provided both inject the same run secret at execution time. For example, a job in AWS ap-southeast-2 (Sydney) and a job in Azure Australia East will produce the same output. The run secret is retrieved from each environment's secrets manager independently: AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault. The masked outputs are structurally identical regardless of which cloud provider or region the job executes in, because the computation depends only on the input value and the run secret, not on where the job runs.

How do new services join an existing masked environment?

DataMasque makes onboarding a new service straightforward: write a YAML ruleset for the new service using the same hash_columns configuration as the rest of the organisation's rulesets, then grant the new service's masking pipeline access to the shared run secret in secrets management. The first masking run will produce consistent output with every existing masked environment automatically. There is no backfill process and no coordination required with teams that have already run masking jobs.

---

See how DataMasque handles cross-service masking, or explore related topics:

• Integrating Data Masking into CI/CD Pipelines: An API-First Approach
• What Is Referential Integrity in Data Masking and Why It Matters
• More realistic masked data with DataMasque's Deterministic Masking