1) Signals connected to DataObs context
DataObs
Context Layer
Context Layer
Signals link to datasets, owners, business SLA, lineage impact, business service mapping, AI/tool activity, and sensitive-data context.
How We Work
DataObs delivers OTel-first implementation patterns for data and platform observability. Every engagement is vendor-neutral and designed for operable handover.
OpenTelemetry-first · Vendor-neutral · collect · enrich · route · correlate
OpenTelemetry visual guide for data observability
DataObs uses OpenTelemetry patterns to collect, enrich, route, and correlate telemetry from applications, data pipelines, AI agents, cloud platforms, and security systems.
2) Collector pipeline
Receivers
OTLP · Prometheus · Fluent Forward · Kafka · CloudWatch · APIs · data-quality probes
Processors
Batch · Memory limiter · Resource detection · Attributes · Transform · Filter · Tail sampling · DataObs enrichment
Exporters
Elastic · OpenSearch · Grafana · Datadog · CloudWatch · SIEM · Debug/file
3) Deployment patterns
4) DataObs enrichment
Raw telemetry
- service.name
- trace_id
- duration_ms
- error=true
- dataset=orders_hourly
DataObs enrichment layer
- data_product=orders
- owner=data-platform
- sla=60m
- freshness_lag=124m
- lineage_impact=4 dashboards
- sensitive_data=true
- business_service=payments
- ai_agent=enabled
- mcp_tool=customer_lookup
Actionable event
orders_hourly freshness breach impacts payments dashboard and AI forecasting workflow. Owner: data-platform. Recommended action: check upstream Glue job and schema drift.
These diagrams are simplified reference patterns. The actual design depends on the client's platform, security controls, telemetry volume, and existing observability tools.
Reference architecture for connected observability
After the OpenTelemetry visual guide, this architecture view shows how DataObs operationalizes those patterns across reliability, security, AI activity, and business outcomes.
What connected observability looks like in practice
95%Data product health
98%Freshness status
Open-source · Free to deploy · Managed option available
DPE — Free open-source data platform framework
DPE (Data Platform Engineering) is a free, open-source framework for building observable data platforms from day one. Deploy it yourself using the public accelerators, or engage DataObs to deploy and configure it in your environment.
Two ways to use DPE:
- Self-deploy (free): clone the repo, follow the guides, deploy on AWS or your chosen cloud. Community support via GitHub.
- Managed deploy (paid): DataObs configures, deploys, and hands over DPE in your environment with full observability wired in from day one.
Technologies we work with
Founder-led, implementation-first
DataObs is a founder-led specialist consultancy built from hands-on platform engineering experience including 6 years of Elasticsearch and OpenSearch specialisation, team leadership at enterprise scale, and AWS-native data platform delivery across Elastic, OpenSearch, AWS, SAS platforms, cloud migrations, infrastructure automation, SIEM, and enterprise data operations.
Based in Edinburgh, Scotland · Available for remote contracts and engagements across the UK and Europe.
- Hands-on engineering
- Vendor-neutral architecture
- Practical delivery
- Operable handover
- Open-source tooling · Private delivery patterns