Data-Driven Decisioning Employee Experience Redesign

Objective & Scope

Our organization has a significant delinquency in data-driven decisioning, resulting in lost opportunities to incorporate consistent customer feedback into our development processes.

We have assessed the current state of the Product Management (PM) experience through three lenses:

  • Awareness: Most PMs do not test or even monitor their products. Of the 190 FTE PMs expected to have Adobe Analytics accounts, only 63% do, and only 31% of those have logged in within the last 30 days.

  • Process: No onboarding processes exist to educate PMs on the benefits of analytics, tagging etiquette, testing strategies, or how to execute them.

  • Culture: An analytics and testing culture is absent. PMs are task-oriented rather than results-oriented, hindering data-driven decisioning.

While working across Product Domains and Lines of Business, we have uncovered a number of opportunities to streamline and expedite our experience:

  • Partnerships with Adobe Testing and Analytics Teams have identified multiple gaps and inefficiencies in the PM testing experience.

  • Best testing practices and culture from effective Product Teams that embrace testing were used as insights that can be embedded across all domains.

  • Interviewing PMs from external organizations revealed new possibilities that can recalibrate our culture.

  • Establishing a benchmark among competitive institutions and technology organizations helped define and size our opportunity.

Discovery

Design Recommendations

After the implementation of our redesigned Employee Experience, our efforts resulted in a 50% increase in PMs who are testing:

  • Awareness: With the design and delivery of comprehensive trainings and the adoption of integral testing frameworks across domains, PMs reap the benefits of assessing and implementing data, analytics, and testing into all decisions, enabling them to work smarter, not harder.

  • Process: By updating our onboarding experience and E2E product lifecycle processes with monitoring and testing elements, we have baked in strategies into every step of product development.

  • Culture: Moving from a task-oriented mindset to a results-oriented mindset ensures that Data-Driven Decisioning (DDD) is a fundamental value and practice for all PMs.

Data-informed decisioning optimizes our experiences, enabling the delivery of best-in-class products for our clients, and generating significant value for our business

Opportunities for optimization are abundant, with the need for PMs to up-skill to parity with industry standards being clear

While Adobe Analytics is a powerful tool in the hands of our PMs, the pain points they have surrounding it are far too many form their POVs

Key Takeaways

  • Absence of training during onboarding results in confusion about how to use Analytics

  • New PMs are not provided with tagging or reporting archives when they take over

  • PMs don’t know what key metrics they should be monitoring for their products

  • PMs are overwhelmed with their workloads and suffer from task-oriented mindsets

Current state of the A/B Testing Process has multiple opportunities to streamline

Two questions were raised about the A/B Testing Process:

  1. How long does it take to launch a test? → 5-12 weeks

  2. Do we have the right infrastructure? → Yes, we do! The challenge is in our process.

Embedding research and testing tools in our existing process helps us create disruptive, innovative features

Demand for Data-Driven Decisioning Workshops* has increased throughout the year, reaching an audience of 500+, resulting in a 50% increase in PMs who are testing

Once we redesign our practices around Adobe Analytics optimization, the PM experience evolves, and data-driven decisioning comes to life

Key Solutions

  • Comprehensive training for all PMs informs them how to effectively use Analytics

  • New PMs are provided with flow diagrams, tagging, and reporting archives

  • All processes that go into identifying key metrics, building dashboards, setting up alerts, and navigating Analytics are readily available

  • PMs become aware of the benefits of monitoring and shift to a results-oriented mindset

Previous

Strategic Content Design