Uncovering Adobe’s OpenTelemetry Adoption: A Behind-the-Scenes Look

Adobe‘s Observability Strategy Using the OpenTelemetry Collector

Adobe‘s Chris Featherstone and Shubhanshu Surana discussed the software company’s use of the OpenTelemetry Collector as a central observability hub for tracking massive amounts of data, including metrics, span data, and log data. Featherstone explained that not all of the data gathered flows through his team or the collector, but “it’s a pretty good chunk.” Distributed tracing led his team to OpenTelemetry.

The Challenge of Distributed Tracing

Adobe is made up of various acquisitions, meaning the company has a multitude of opinions about the best cloud, tools, and text editors for the job. “With distributed tracing specifically, that becomes a huge challenge,” Featherstone said. “Imagine trying to stitch a trace across cloud vendors, open source…” This challenge led the company to use the collector, which worked much better than the Jaeger agents they were previously relying on.

Dynamic and Flexible Collector with Multiple Destination Capabilities

The team instruments applications using OpenTelemetry libraries primarily for auto-instrumentation using Java. The team also uses application enrichment which adds Adobe-specific data and enriches pipelines to the collector. They use custom extensions and processors, and the team’s configuration is controlled through GitOps when possible. Featherstone praised the collector for being incredibly dynamic, with extensions that allow multiple destinations with one set of data.

Managing the Data and Ensuring Security

Surana, Adobe‘s cloud operations site reliability engineer for observability, said they do a lot of enrichment and eliminate specific fields that they don’t want to send to the backend. They developed a custom extension using the custom authenticator interface to use a single system to securely transfer data to different backends while still ensuring security for both open source and vendor tools. Featherstone said the team is always looking to improve data quality, such as eliminating data that no one looks at and rate limiting spans at the edge.

Trace-First Troubleshooting

Featherstone explained that Adobe has a ton of east/west services, and troubleshooting through logs can be slow and challenging. They’re trying to pivot more to trace-first troubleshooting by making a lot of effort to make their traces complete. The team is looking into how people troubleshoot and whether their current tools provide the best way to do that.

OpenTelemetry‘s Role in Adobe‘s Observability Strategy

Featherstone praised OpenTelemetry‘s plug-in-based architecture and its ability to send data to different destinations using a single binary. “OpenTelemetry in general feels a lot to me, like the early days of Kubernetes where everybody was just kind of buzzing about it, and it started like we’re on the hockey stick path right now,” he said. “The community is awesome. The project is awesome. If you haven’t messed with the collector yet, you should definitely go check it out.”

Advice for Others Implementing Observability Strategies

Featherstone’s team’s approach to implement observability best practices include developer productivity with the goal to help developers write better code faster. They use GitOps for configuration management, allowing engineers to quickly identify Adobe-specific service identifiers and locate engineering contacts. They also utilize OpenTelemetry libraries when possible for auto-instrumentation and application enrichment to build a dynamic collector that works well with multiple destinations.