One of the most valuable lessons I learned at Amazon was treating measurement as a first-class concern. Not just measuring things, but investing real engineering effort into ensuring those measurements were accurate and reliable. This principle has shaped my career.

Real Examples of Measurement Impact

Development Team Velocity

One of my most successful implementations was measuring development team velocity - not as a performance metric for reviews, but as an internal tool for self-improvement. This subtle distinction made all the difference. Teams used velocity data to identify opportunities for optimization, accurately forecast their deliverables, and make data-driven decisions about process changes. The key was keeping this metric for the team’s eyes only, ensuring it remained a tool for growth rather than judgement.

System Cost Analysis

Understanding true cost per operation in distributed systems reveals insights that total costs hide. By measuring granular metrics like cost per request, teams can identify optimization opportunities that aggregate numbers miss. This is especially powerful in growing systems, where rising total costs can mask underlying efficiency gains. The key is choosing the right unit economics - a growing business might see total costs increase while actually becoming more efficient per transaction. As a leader, incentivize and reward the efficiency gains.

Marketing Impact

When working with marketing teams, I push for measuring incremental lift with proper control groups. This meant tracking real conversions tied directly to business objectives, not just vanity metrics. While more challenging to implement, this approach provided unambiguous data about what drove business results.

The Resistance to Measurement

Despite its value, few businesses prioritize accurate measurement. Senior leaders have argued “measurement is too expensive” or that “only improvements count for performance reviews.” I’ve witnessed many organizations reject measurements that don’t reflect positively.

The Path Forward

The only way to know you’re getting better is to measure - otherwise, it’s just a feeling. If the measurements aren’t flattering, that’s an opportunity to be humble and see all the room for improvement. Here’s what I’ve learned:

  1. Start small but measure accurately
  2. Focus on metrics that impact business outcomes
  3. Be prepared for initially unflattering results
  4. Stay humble and use the data to guide improvement
  5. Build a culture that values truth over comfort

TLDR

Great measurement isn’t easy - it takes real engineering work and organizational commitment. In my experience, few investments have paid off more consistently.