One platform for data observability across global data divisions

About the client

Global biopharmaceutical company operating across multiple divisions.

The challenge

The client needed a unified way to manage a fragmented data ecosystem, control rising operational costs, and protect data integrity across its divisions.

• No centralized view into the health, quality, and cost of analytical assets
• Siloed monitoring development estimated at three times the cost of a single unified platform
• High risk of misinformed business decisions due to the absence of proactive anomaly detection
• Hundreds of project owners had no automated way to track their own resource use and budget consumption

The solution

UD4D designed a modular, cloud-native observability platform on AWS to centralize metadata and automate observability across the data estate.

• Metadata integration across global Databricks workspaces, the Enterprise Shared Area (ESA), and GPTeal (generative AI program), including definition and ingestion of critical metadata
• Review and optimization of Redshift data models for high-performance analytics in Power BI and ThoughtSpot
• A Trino federation layer over processed observability data to enable cross-source querying
• Four Grafana dashboards for Databricks monitoring and automated daily email summaries of Redshift usage
• AWS IAM and SSO to enforce granular, workspace-specific data isolation and security