🔔 Stay tuned. Data quality is a new capability in dScribe, and we're still expanding it. This article covers the fundamentals. Reach out if you want extra help.
You can now define data quality rules directly in dScribe. Rules are modeled on the Open Data Contract Standard (ODCS), so the quality expectations you capture in the catalog use an open, portable format you can sync with external data quality engines.
Data quality rules in dScribe are always part of a data contract. For more information, see Data Contracts.
What you can do today
Data quality rules let you express what "good" looks like for your data and attach those expectations right where the data is documented:
Define rules without code — build rules through a guided form; no SQL or scripting required (though SQL and custom options are there if you want them).
Apply rules at two levels — on a dataset as a whole, or on an individual dataset element (a column).
Stay tool-agnostic — rules follow the ODCS and are stored in .yaml files so you can execute them in your data quality engines of choice. For example: Soda, Great Expectations or Monte Carlo.
Write back loop - results of executed data quality rules can be written back and displayed in dScribe, so users immediately view data quality scores on data assets in the catalog
Adding a quality rule
Inside the data contract linked to a dataset or data product, go to Data Quality and click Add rule. The Add quality rule dialog walks you through a few choices:
Severity
Set how a failing rule should be treated: Info, Warning, or Error. This lets you separate purely informational checks from the ones that should raise a real red flag.
Rule type
Pick how the rule is defined:
Library — a pre-built metric. Choose a Metric (for example, Invalid Values), a Dimension (for example, Accuracy), and a Threshold (for example, must be greater than 0). This is the no-code path.
SQL — a custom query, for checks you'd rather express yourself.
Text — a documented-only rule: a quality expectation you want recorded in the catalog without an automated check behind it.
Custom — import from an external engine. For example: Soda, Great Expectations or Monte Carlo
Where rules show up
Once saved, a rule appears under Quality rules on the dataset or column it belongs to. If you have set up a sync with your data quality engine, each rule will also show the latest run state with more details.
Schema drift
When you include dataset elements in your data contract's schema, whenever these dataset elements get deleted or renamed in the source, a warning will automatically display in dScribe pointing this out. This helps avoid accidentally breaking datasets that stakeholders might depend on for their own data use cases.
Running the dscribe-dq Python SDK
dscribe-dq is dScribe's Python SDK for running data quality checks directly against your data sources — Databricks, MSSQL, or SAP HANA/Datasphere — and writing the results back to your dScribe workspace. Install it with pip install dscribe-dq. Before running it, make sure the target data product already has data quality rules defined in its ODCS spec in dScribe, and that you have a dScribe API key (found under Settings → API keys) along with the asset's UUID.
The simplest way to get started is with the DScribeDQ client. Instantiate it with your API key, base URL, and asset ID, then call run_validation() with connection details for your source, and pass dq.write_back_to_dscribe() as a postprocessor to push results back into dScribe automatically:
from dscribe_dq import DScribeDQ
dq = DScribeDQ(
api_key="<your-api-key>",
base_url="https://app.dscribe.cloud/catalog/api",
asset_id="<your-asset-id>",
)
ctx = dq.run_validation(
source_configs={
# key must match the server id in the ODCS servers block
"<server-id>": {
"type": "databricks",
"host": "<workspace-host>",
"client_id": "<client-id>",
"client_secret": "<client-secret>",
"tenant_id": "<tenant-id>",
"http_path": "/sql/1.0/warehouses/<warehouse-id>",
}
},
postprocessors=[dq.write_back_to_dscribe()],
)
Each key in source_configs must exactly match the id of the corresponding entry in the asset's ODCS servers block — this is the underlying source/connection's UUID in dScribe, not its display name. You can find it two ways: open the asset in the dScribe catalog, go to the Contract tab, and click into the source listed under Infrastructure — the resulting URL (/catalog/admin/source/<id>) ends in the id you need. Or, view the asset's Data Contract (YAML) directly on the Contract tab and copy the id field under servers:. If a key in source_configs doesn't match any server id in the contract, that source's config is silently ignored during validation, so it's worth double-checking the id if a run doesn't behave as expected.
For automated runs — for example in a CI/CD pipeline or scheduled job — you can skip constructing the client entirely and use the module-level run_validation() function instead, which reads everything from environment variables. This makes it easy to drop into a pipeline without hardcoding credentials in code. Each connector has its own required variables — see the SDK's README for the full list per source type, and the troubleshooting guide if a run fails to connect or write back results.
Where to go next
→ More on data quality is on the way. We'll expand this article as the capability grows — reach out through the in-app chat if there's something specific you'd like to do.
Have a question or can't find what you're looking for? Use the chat icon inside the catalog to reach the dScribe support team.