Evidence Based Management - Changing What Is Measured
Introduction to Evidence-Based Management
In this section, the speakers introduce the topic of evidence-based management and discuss the importance of empiricism in the agile space.
The Role of Empiricism in Evidence-Based Management
- Evidence-based management goes beyond just goals and measures, incorporating empiricism and management.
- Empiricism fills a vacant governance niche in the agile space.
- It is important to understand how changing measures play a crucial role in evidence-based management.
Importance of Changing Measures
This section focuses on why changing measures is essential in evidence-based management.
Reasons for Changing Measures
- Measures may need to be changed if they are no longer relevant to the current goals.
- Performance measurements can unintentionally or intentionally force behaviors that do not align with desired outcomes.
- Introducing new measures can influence behavior, so it's important to assess whether they are helping achieve goals.
- Relevance is another factor for changing measures. Even if a measure changes, it may not impact other desired changes.
Starting Notes on Changing Measures
The speakers begin discussing changing measures by sharing their screen and taking notes on a board.
Initiating Discussion on Changing Measures
- The speakers start noting down ideas related to changing measures on a shared board.
- They emphasize the interconnectedness between empiricism and evidence-based management when considering changing measures.
Reasons for Changing Measures (Continued)
The discussion continues regarding reasons for changing measures.
Additional Reasons for Changing Measures
- Changes in goals often require corresponding changes in measurements.
- Performance measurements may not effectively capture the intended information from knowledge workers, leading to unintended consequences.
- Changing measures can help align behaviors with desired outcomes and ensure relevance to the overall goals.
Considerations for Changing Measures
The speakers delve deeper into the considerations for changing measures in evidence-based management.
Influencing Behavior and Ensuring Relevance
- Introducing new measures can influence behavior, so it's crucial to assess whether they are helping achieve desired outcomes.
- Relevance is key when evaluating measures. Even if a measure changes, it may not impact other desired changes.
- An example is given regarding flow metrics and how focusing solely on throughput may not necessarily increase current value or innovation.
Personal Example of Changing Measurements
The speakers provide a personal example related to changing measurements in the fitness industry.
Applying Changing Measurements in Personal Life
- A scenario is discussed where someone initially focuses on weight loss as a measurement but later shifts their focus to strength training.
- Continuing to measure weight loss despite the change in focus may detract from achieving the goal of getting stronger.
- This example highlights the importance of aligning measurements with current objectives.
Obsession with Irrelevant Metrics
The speakers discuss organizations' obsession with irrelevant metrics and the difficulty of letting go of established measurements.
Obsession with Irrelevant Metrics
- Organizations often become fixated on specific metrics that may be completely irrelevant due to other objectives.
- Despite recognizing irrelevance, organizations struggle to let go of established measurements due to sunk cost fallacy.
- Shifting from a management of people mindset to a management of work mindset requires letting go of individual performance metrics, which can be challenging.
Difficulty in Letting Go of Established Measurements
The speakers elaborate on the challenges of letting go of established measurements in organizations.
Challenges in Letting Go
- Once measurements are ingrained in an organization, it becomes difficult to change them due to the sunk cost fallacy.
- Transitioning from individual performance metrics to a focus on work management is particularly challenging.
- Organizations need to shift their mindset and prioritize relevant measures aligned with their current goals.
Timestamps may not be exact due to limitations in processing natural language.
Working with Knowledge Workers and Metrics
The speaker discusses the challenges of measuring the performance of knowledge workers and the relevance of metrics in evaluating their success.
Challenges in Measuring Knowledge Workers' Performance
- It is difficult, if not impossible, to measure the performance of knowledge workers based on product success.
- The speaker questions whether there are any flags or indicators that suggest when existing measurements are no longer relevant.
- Lagging metrics are often used to measure current value and unrealized value, while leading metrics focus on innovation and time to market.
- Establishing a direct causal link between leading and lagging metrics can be challenging in complex environments.
Relevance of Metrics
- If a metric shows real gains but does not result in changes in current value or unrealized value, it may not be a meaningful metric.
- Behavioral influences and changing goals should be considered when evaluating metrics.
- The speaker shares frustration as a product owner in seeking direct correlations between features built and their success in production.
Tilt Towards Key Value Areas (KVA)
- The speaker believes that at any given moment, there is a tilt towards one of the key value areas (KVA) in product development. Companies may lean too much into one KVA at times.
- Regularly assessing where the focus lies among KVAs can serve as a sense-making tool for determining if measurements need to change.
Examples of Changing Measures
The speaker provides examples where changing measures has helped companies improve their performance.
Example 1: Tracking Irrelevant Metrics
- A company had a dashboard with metrics across all KVAs, but some metrics were related to products that were declared end of life. The speaker questions the relevance of tracking these metrics.
Example 2: Wells Fargo's Growth Strategy
- Wells Fargo's goal was to get more people to open accounts, which worked well when they were an underdog trying to capture market share. However, as they already had a significant market share, this growth-focused strategy became less effective.
Importance of Changing Metrics
- Changing measures can help identify areas where current value is not being delivered upon and highlight factors like time to market and ability to innovate that may be hindering success.
- Regularly reviewing and updating metrics is crucial for achieving success in running companies.
New Section
In this section, the speaker discusses the importance of measuring billable time and work coming in to ensure a sustainable business.
Measuring Billable Time and Work
- The speaker focuses on determining the percentage of billable time available in the future rather than just looking at utilization.
- This measure helps determine if there is enough work coming in to sustain the business financially.
- Initially, tracking hourly rate and future hours helped prioritize new clients and training needs.
- However, over time, this metric became counterproductive as it led to a loss of innovation and neglecting holiday time.
New Section
In this section, the speaker discusses how focusing solely on income certainty can lead to burnout and lack of innovation.
Balancing Income Certainty and Innovation
- The speaker shares that prioritizing income certainty started becoming counterproductive as it hindered innovation.
- By solely focusing on income, they lost time for creativity and forgot to book holiday time.
- This behavioral component led to potential burnout.
- As a result, they stopped using this measure as it didn't contribute to creating a sustainable business.
New Section
In this section, the speaker talks about shifting focus from income certainty to measuring available time for innovative work.
Shifting Focus to Innovation
- The speaker stopped using measures related to income certainty and shifted towards tracking available time for innovative work.
- They now monitor their utilization percentage to ensure it doesn't exceed a certain point.
- This shift allows them to prioritize innovation while maintaining basic financial stability.
New Section
In this section, the speaker discusses how they measure their ability to innovate by looking at calendar days and reducing excessive workload.
Measuring Ability to Innovate
- The speaker shares that they started measuring how many days were blocked off on their calendar to assess their workload.
- They realized they had too much on their plate, leading to exhaustion and compromised quality.
- They are now focusing on reducing the number of calendar days booked to create more time for innovation and automation.
New Section
In this section, the speaker emphasizes the importance of measuring innovation rate and its impact on business evolution.
Importance of Innovation Rate
- Both speakers learned through experience that measuring innovation rate is crucial for business growth.
- Initially, they focused on other metrics like classes booked or billable time but found that innovation rate was more relevant.
- Guarding the ability to evolve and expand becomes essential once basic stability is established.
New Section
In this section, the speakers discuss how reducing calendar days can increase their ability to automate tasks and expand.
Reducing Calendar Days for Expansion
- By reducing the number of calendar days booked, there is a potential increase in the ability to automate manual tasks.
- This reduction allows for more time dedicated to expanding and improving processes.
- The speakers highlight the importance of balancing workload with opportunities for growth.
New Section
In this final section, the speakers summarize key takeaways from the discussion on changing measures.
Key Takeaways
- Goals should not be static; regularly re-evaluate what you measure based on goal changes, behavioral influences, and relevance.
- The three measures discussed - goal change, behavioral influence, and relevance - provide insights for re-evaluation.
- The speakers encourage viewers to gain insights from existing measurements and consider alternative measures if necessary.
The transcript provided does not specify the language, so the summary is written in English.