Datadog Now Enriches Logs Dynamically with Observability Pipelines
Security and platform teams need rich log context to investigate threats and meet compliance standards. But that context often lives elsewhere, in Snowflake, Amazon S3, ServiceNow CMDB, or Databricks. Enriching logs after ingestion means duplicating lookups in every downstream tool. The result is slower resolution and higher costs.
Datadog has addressed this directly. Its Observability Pipelines now support centralized Observability Pipelines log enrichment using Reference Tables before data routes to a SIEM, logging solution, or data lake. Teams can attach fresh metadata to events before data leaves their infrastructure, with no manual CSV updates required.
The Enrichment Table processor connects directly to SaaS-hosted Reference Tables. These tables update automatically via integrations, so engineering teams spend less time maintaining datasets. Supported sources include Snowflake for threat intelligence and user profiles, ServiceNow CMDB for asset metadata, Salesforce for customer and billing context, Databricks for anomaly scores, and cloud storage sources like Amazon S3 and Azure Blob Storage for IP reputation lists and allowlists.
For example, a team can enrich logs with threat intelligence from AlienVault feeds stored in Snowflake. Once the Enrichment Table processor matches an IP address field, the log event automatically receives relevant threat data. That enriched log then routes to the downstream tool of choice, Microsoft Sentinel, CrowdStrike, or Datadog BYOC Logs.
Beyond real-time enrichment, the feature supports security investigations involving historical data. Teams can rehydrate and extract archived logs, apply current Reference Table context, and route normalized data to a SIEM. This is especially valuable when threat intelligence or fraud scores have changed after an incident began.
Furthermore, Observability Pipelines log enrichment also improves routing decisions. Instead of static heuristics, teams can use enriched attributes, such as threat classification, environment label, or customer tier, to send high-risk events to a SIEM while routing benign, high-volume traffic to lower-cost storage like Amazon S3.
