How to understand technical debt

October 31, 2021

(This is a plog aka powerpoint blog)

Capital, Work and Debt

  • All work requires capital - your resources i.e. people and money
  • All assets require maintenance
    • Well designed and well built assets require lesser maintenance (and vice versa)
    • Corollary = you can skip work now and spend on maintaining more later -Hence this is debt i.e. borrowing from the future
    • Increase in scale causes assets to decline in quality with time
      • Hence, debt accumulates over time by default
  • Work has two basic categories = building and maintaining
    • Building creates more assets (investment)
    • Maintenance depletes capital with no increase in assets (expenditure). Can be -
      • Repair = increases asset quality (so it requires lesser maintenance) = principal on debt
      • Mitigation = counter the failures due lack of asset quality = interest payment on the principal
  • Thus debt is paid in two parts – repair (principal) and mitigation (interest)

Definitions

  • Capital = resources to build or maintain = people and money
  • Assets = everything built and learnt till now
  • Debt = effort required to raise the asset quality to what is required
  • Investment = using capital to build more assets
  • Expenditure = using capital to maintain an asset
  • Repair = using capital to increase asset quality
  • Mitigation = using capital to counter the effects poor asset quality
  • Interest = effort spent in mitigation due to poor asset quality
  • Principal = effort needed to increase asset quality to required level

Examples of interest payments

  • Effort spent in debugging and fixing production defects
  • Additional time required to understand and extend the system
  • Time required to onboard people
  • Effort spent in managing customer expectations due to bad user experience
  • Effort spent in coordinating between various teams
  • Resources spent in over scaling sub optimal systems

Examples of lacking asset quality

  • Badly refactored code
  • Improper separation of concerns
  • Lack of automated test cases
  • Lack of documentation
  • Non performant / ageing infrastructure (like databases, frameworks)
  • Lack of telemetry
  • Lack of proper CI/CD

Virtuous cycle of growth

  • Capital takes time to scale up
    • You can hire and onboard only limited number of people in limited time
    • More people require more management
    • Hiring requires money
  • You cannot default on your debt, interest payments are not optional
    • Total working capital = Total capitalInterest payments
  • Virtuous cycle = More working capital leads to more assets leads to higher growth leads to more capital formation leads to paying off past debts in time and growing faster (and vice versa is also true)

Sinking ship

  • Slow growth = Working capital - Interest payments = Δ i.e. very little productive resources are left for repair or new growth
  • Debt accumulates = Existing debt + Declining asset quality + New assets built
  • Stalled growth = Growth in capital < Accumulation of debt i.e. system is deteriorating more rapidly than the growth in resources required to repair it
  • Bankruptcy = Working capital < Interest payments i.e. all effort not enough even to just somehow run the system

Case for taking debt

  • Debt gives you more power in the present
    • Grow now faster, get more resources, pay off the debt
  • To stay on high growth trajectory and avoid stalling - interest payments remain steady i.e. resources spent on maintenance are constant % of total resources
    • Corollary = if interest payments keep mounting it will lead to bankruptcy
    • Must pay off the principal on debt in time otherwise total interest paid >> principal

    -* Optimum amount of debt → growing debt = total growth (in absolute terms)*

  • Keeping debt in control → principal must be constant and interest on it must be manageable
    • Paying off the principal on time i.e. repairing the system vs mitigating problems for a long time
    • Repairing the older system gives you the room to take debt on new assets

Prioritisation

  • Debts on different items carry different interests (eg. Home loan vs credit card debt)
    • i.e. The cost to repair something might be much lesser or much more than enduring that thing
  • To Do: Make your balance sheet first
    • How much interest are you paying across various items i.e. how much effort is going day to day in mitigating poor assets
      • Refer to examples of interest payments and estimate the ‘true cost’ of it
      • This will also depend on  the dependencies your future roadmap has
    • How much principal do you owe i.e. how much will it cost to fix those assets
    • Your principal will grow over time (because asset quality declines). So catalogue your assets’ mid-life and end of life.
    • Categorize your debt and align it with your roadmap
    • Pay off the more expensive debt first

Types of debt

  • Engineering debt
    • Can affect performance
      • Process failures, over scaling, high load times
    • Can affect development efficiency and quality
      • Unknowns, lack of clarity of scope, inaccurate estimates, too much QA, unsatisfied developers, unable to scale the team
  • Product Debt
    • Low customer satisfaction (due to gap in customer expectation and product)
      • Not knowing what to improve, lack of metrics and their measurement, lack of customer understanding, lack of customer feedback
    • Piling customer commitments (due to lack of ability to execute the roadmap)
      • Sales commitments, competition and market pressure, incomplete execution of product vision
  • Process debt
    • Lack of planning and execution
      • No control over timelines, big gap between delivery expectation and reality
    • Lack of processes for effective communication
      • Lot of time spent in meetings, hard to onboard new people, unable to refer to previous discussions when required, missing stakeholders in key decisions
    • Lack of measurement / data
      • Intuition based decision making, no accountability and retrospective of past decisions

Stages

In which stage you are?

  • No idea about the state of the applications or customer / effort impact
  • Know the impact but don’t know the cause
  • Know the cause but don’t know the solution
  • Know the solution but don’t know how you will execute it
  • Have the solution and the plan but others are not aligned
  • Have the solution, plan and alignment but don’t have the resources
  • Have the resources at the start but get lost during execution due to unknowns, challenges and competing priorities
  • Despite delays and challenges, few iterations, bruised and hurt, you emerge victorious

Advice

  • Engineering requires judgement to prioritise work items
  • Measure -
    • Health – performance, errors, customer impact
    • Effort spent – issues, enhancements
    • Projects – status, dependencies
  • Data is never enough to exactly tell you what to do. Take data-informed (vs data-driven) decisions. Take decisions based on all round evaluation
  • Sense of urgency – if you don’t decide in time, time will decide for you
  • Don’t let any problem become too big to handle
    • Effort and skill required to solve a problem grows exponentially with the problem