Your most valuable data is locked down. Relationship-aware masking changes that. 

‍How to prepare high-fidelity datasets for AI training without exposing sensitive information.

The data problem holding enterprise AI back

Your organization's most valuable data is probably locked down right now. Customer behavior logs, transaction histories, sales and support transcripts -  the data that could sharpen your AI models, accelerate experiments and drive better decisions. Every request to access that data lands with the data team, triggering security reviews, governance checks and careful processing to meet GDPR, HIPAA, CCPA or other privacy requirements. For the business and AI teams waiting on it, that means long delays or datasets that can't be used.

The challenge runs deeper than access alone. The insights AI models depend on rarely live in a single system. They emerge from the connections between structured records and unstructured sources like application logs, call recordings, clinical notes and customer emails. That combination is exactly what makes it so hard to use safely.

The obvious answer is to mask or redact before use. The problem is that most masking tools strip away the relationships the model needs to learn from. You end up with a dataset that is privacy-safe but no longer useful.

Where existing tools fall short

Structured and unstructured data are treated as separate problems. The vast majority of masking solutions handle relational databases with one process and logs, transcripts or documents with a completely separate pipeline. There is no mechanism to keep synthetic identities aligned across both. A customer masked as one identity in a CRM becomes a different identity in a support transcript, or gets removed entirely.

Relationships get destroyed. Pattern-matching tools look for recognisable formats like email addresses or phone numbers and either remove or replace them. NLP-based entity recognition tools try to identify personal information by interpreting sentences. Both approaches act on individual data points in isolation. Neither understands that John Doe in the database and John Doe in the transcript are the same entity, and need to map to the same synthetic identity.

Once cross-system relationships are broken during masking, they cannot be reconstructed. Logs no longer connect to customer records. Transcripts lose their relationship to accounts. The dataset becomes a collection of isolated data points rather than a coherent picture of how customers interact with your business.

Performance at scale is also a  real constraint. Those same NLP tools process each line of unstructured text individually, and even  efficient models take half a second or more per statement. Across millions of log entries or years of transcripts, that makes dataset preparation a costly, multi-day operation - meaning AI teams are stalled waiting to access this data. 

An example: call center transcripts

Many organizations hold years of customer support transcripts - a rich signal for training virtual assistants, automating service workflows and analyzing customer experience at scale. These transcripts reference customer names, account numbers and addresses that tie directly to records in relational systems.

Traditional redaction removes those identifiers but destroys the thread connecting transcript to account to interaction history. The dataset is sanitized but stripped of the relationships that made it useful for AI. 

How relationship-aware masking works

Relationship-aware masking is a data de-identification approach that uses relationships already established in structured data to consistently mask sensitive information across both structured and unstructured data sources. Rather than analyzing each data source in isolation, it anchors unstructured masking to synthetic identities already generated in the database layer, preserving the cross-system connections AI models depend on.

The process starts in the relational layer. When DataMasque masks structured data in systems like PostgreSQL or MySQL, it generates synthetically identical customer data - replacing real customer names, account numbers and other identifiers with realistic substitutes that maintain referential integrity across tables and databases.

When the unstructured data is masked, DataMasque applies the same synthetic identities already established in the database.

If John Doe becomes David Carter in the CRM, David Carter appears in the transcripts, the application logs and any other unstructured source that references that individual. The relationship between systems is preserved, and the context models need stays intact.

What this makes possible for AI and data teams

The practical outcome is the ability to use data that was previously off-limits - not because it lacked value, but because there was no safe path to it. Data teams can give AI and business teams access to high-fidelity data without the security reviews, delays and compromises that come with using production data directly.

With relationship-aware masking in place:

• Training datasets can span structured and unstructured sources without losing cross-system relationships
• Synthetic identities stay consistent across CRM records, transcripts, logs and documents, preserving the signals models learn from
• Datasets can be regenerated quickly and repeatedly, so teams can experiment without re-exposing production data
• Fine-tuning and evaluation cycles move faster because data preparation no longer takes weeks to months
• Compliance and privacy requirements are met without compromising the quality of the dataset

Performance at enterprise scale

Because relationship-aware masking replaces known values using pre-established relationships rather than passing every record through an ML model for detection, large volumes of transcripts, logs and operational data can be processed in seconds rather than hours.

Handling every data type

Relationship-aware masking is the foundation. For data tied to database records - customer names, account identifiers, transaction references - it is the most reliable and performant approach, preserving cross-system relationships at speed.

The harder problem is free-form text with no database relationships and no predictable structure. Medical notes, legal documents and clinical referrals contain sensitive information buried in natural language - a patient referenced throughout a clinical note, two doctors named in a referral letter, roles and identities that patterns alone cannot reliably distinguish. This is where semantic understanding is needed, not just matching.

DataMasque’s AI engine is an optional add-on for exactly these cases. Rather than writing detection patterns, using the AI detect you describe what you want to find in plain language through the same YAML configuration: first name of doctors, last name of patients, date of birth. The engine understands the difference between a doctor and a patient, between a date of birth and a date of visit, and can retain meaningful attributes like age while substituting a new date of birth that preserves data fidelity. It masks consistently across all related record types, so the same synthetic date of birth would be consistent throughout relational data as well as the unstructured document data. 

Not every organization will want to use a model in their masking workflow, and for most enterprise data the combination of relationship-aware masking and algorithmic detection is sufficient. The AI engine is there for teams that need it, entirely separate from the core platform. AI engine runs do take longer than relationship-aware matching - a deliberate trade-off for accuracy on complex, unstructured text.

The engine runs on a foundation model in AWS Bedrock, inside your own cloud environment. No data leaves your infrastructure. DataMasque has also just shipped its own detection engine that runs fully on a GPU instance or on-premise, with no external calls at all - for teams that need complete control over where their data is processed.

Relationship-aware detection with the AI engine can also be  combined with algorithmic detection using checksums, regex and known patterns to identify structured identifiers like credit card numbers or tax IDs with mathematical certainty. For data that follows a predictable format, this gives you precision that probabilistic model-based detection alone cannot match. 

The broader picture

Demand for high-fidelity training data will keep growing as AI programmes expand. Production data will stay off-limits. The teams that move fastest will be those with a data team that can provide synthetically identical customer data at scale, safely and on demand.

Relationship-aware masking is how DataMasque makes that possible - giving data teams a secure, scalable way to unlock their organization's most sensitive data for AI.

If your team is working through the data preparation problem and running into the structured-unstructured alignment challenge, it is worth seeing how DataMasque handles it.

Frequently asked questions

What is relationship-aware masking?

Relationship-aware masking is a data de-identification technique that uses synthetic identities established during structured database masking to consistently replace sensitive information across connected unstructured sources - such as logs, transcripts and documents. Unlike pattern-matching or NLP-based approaches, it maintains the relationships between data across systems, which is essential for AI training datasets.

How is relationship-aware masking different from NLP-based data masking tools?

NLP-based tools treat each data source independently, which breaks consistency across sources before the data ever reaches an AI workflow. Relationship-aware masking keeps identities aligned across structured and unstructured sources from the start. For unstructured text with no database relationships, DataMasque's AI engine add-on handles semantic detection - though this is entirely optional for organizations that prefer to keep models out of their masking workflow. Both methods can also be combined with algorithmic detection using checksums, regex and known patterns to identify structured identifiers like credit card numbers or tax IDs with mathematical certainty. For data that follows a predictable format, this gives you precision that probabilistic model-based detection alone cannot match. 

Does data leave our environment when using DataMasque?

No. DataMasque is self-hosted and runs entirely within your own infrastructure - on-premise or in your private cloud. No data leaves your environment at any point, including when using the AI engine add-on.

‍