The key to any successful startup is close collaboration between product
and engineering. This sounds easy, but can be incredibly difficult. Both
groups may have conflicting goals and different definitions of success that
have to be reconciled. Engineering might want to build a product that is
perfectly scalable for the future with the best developer experience.
Product might want to quickly validate their ideas, and put features out
that will entice customers to pay for the product. Another example that’s
common to see is an engineering-led “engineering roadmap” and a product-led
“product roadmap” and for the two to be completely independent of each
other, leading to confusion for product engineering. These two mindsets
put two parts of your organization at odds. The easy path is to skip the
difficult conversations and operate within silos, coming together
infrequently to deliver a release. We believe that aligning these two
disparate organizations into cohesive team units removes organizational
friction and improves time to value.
How did you get into the bottleneck?
At the beginning of a startup’s journey, aligning is natural because
you are a small team working closely together, and likely the product and
tech leaders had close personal relationships before the company was
founded. The initial startup idea is very strong and as it quickly gains
traction, what to work on next is obvious to all groups. As the company
grows, however, skill-based verticals begin to appear with more layers of
management, and these managers don’t always put the effort in to create
an effective working relationship with their peers. Instead, they focus on
urgent tasks, like keeping the application running or preparing for a
funding round. At the same time, the startup faces a critical juncture where the company needs to
to decide how to best invest in the product, and needs a holistic
strategy for doing so.
Well-run startups are already working in cross-functional product
teams. Some functions will naturally work well together because they fall
under the same vertical hierarchy. An example would be development and
testing — well integrated in startups, but often siloed in traditional
enterprise IT. However, in the scaleups we work with, we find that product
and technical teams are quite separated. This happens when employees align
more with their function in an Activity Oriented
organization rather than with an Outcome Oriented team, and it
happens at every level: Product managers are not aligned with tech leads
and engineering managers; directors not aligned with directors; VPs not
aligned with VPs; CTOs not aligned with CPOs.
Ultimately, the bottleneck will be felt by reduced organizational
performance as it chokes the creation of customer and business value.
Startups will see it manifest in organizational tension, disruptive
exceptions, unchecked technical debt, and velocity loss. Fortunately,
there are some key signs to look for that indicate friction between your
product and engineering organizations. In this article we will describe
these signs, as well as solutions to lower the communication barriers,
build a balanced investment portfolio, maximize return on investment, and
minimize risk over the long term.
Signs you are approaching a scaling bottleneck
Finger pointing across functions
Figure 1: Friction across a typical
hierarchical structure
Team members align themselves with their management structure or
functional leadership as their primary identity, instead of their
business or customer value stream, making it easier for teams to assume
an “us” versus “them” posture.
At its worst the “us vs them” posture can become truly toxic, with little respect for each other. We have seen this manifest when product leaders throw requirements over the wall, and treat the engineering team as a feature factory. They might abruptly cancel projects when the project doesn’t hit its outcomes, without any prior indication the project wasn’t meeting its KPIs. Or conversely, the engineering team continually lets down the product team by missing delivery dates, without warning that this might happen. The end outcome is both sides losing trust in each other.
Engineers often stuck due to lack of product context
When product managers pass off features and requirements without reviewing them with the
engineers (usually within the constructs of a tool like Jira), critical business and customer context can be lost. If
engineers operate without context, then when design or
development decisions need to be made, they must pause and track down the product
manager, rather than make informed decisions themselves. Or worse, they made the decision anyway and
build software based on an incorrect understanding of the product
vision, causing time delays or unused software. This friction disrupts
flow and introduces undue waste in your delivery value stream.
Missed dependencies
When engineers and architects operate with minimal context, the full
scope of a change can be overlooked or misunderstood. Requirements or
user stories lack depth without context. Customer personas can be
ignored, business rules not taken into consideration, technical
integration points or cross-functional requirements missed. This
often leads to last minute additions or unintended disruptions to the
business or customer experience.
Work slipping between the cracks
Tasks slipping between the cracks, team members thinking someone else
will be responsible for an activity, team members nudging each other out
of the way because they think the other team member is operating in
their space, or worse, team members saying “that’s not my job” – these
are all signs of unclear roles and responsibilities, poor communication
and collaboration, and friction.
Technical debt negotiation breakdown
Technical debt is a common byproduct of modern software development
with many root causes that we have
discussed previously. When product and engineering organizations
aren’t communicating or collaborating effectively during product
planning, we tend to see an imbalanced investment mix. This can mean the
product backlog leans more heavily towards new feature development and
not enough attention is directed toward paying down technical debt.
Examples include:
- Higher frequency of incidents and higher production support costs
- Developer burnout through trying to churn out features while working
around friction - An extensive feature list of low quality features that customers quickly
abandon
Teams communicating but not collaborating
Teams that meet regularly to discuss their work are communicating.
Teams that openly seek and provide input while actively working are
collaborating. Having regular status meetings where teams give updates
on different components doesn’t mean a team is collaborative.
Collaboration happens when teams actively try to understand each other
and openly seek and provide input while working.
How do you get out of the bottleneck?
Eliminating the wall between Product and Engineering is essential to
establishing high performing product teams. Cross-functional teams must
communicate and collaborate effectively and they must be able to negotiate
amongst themselves on how best to reach their goals. These are strategies
Thoughtworks has applied to overcome this bottleneck when working with our
scaleup clients.
Identify and reinforce your “First Team”
At its most basic, a product team is a group of individuals who come
together in a joint action around a common goal to create business and
customer value. Each team contributes to that value creation in their own
unique way or with their unique scope. As leaders, it’s important to identify
and reinforce a team dynamic around the creation of value rather than an
organizational reporting structure. This cross-functional product team becomes
a team member’s “first team”. As a leader, when you define your team as your
group of direct reports, you are enabling a tribal concept that contributes to
an “us vs. them” dynamic.
The First Team mindset was defined by
Patrick Lencioni and referenced in many of his works including
The Advantage and The Five Dysfunctions Of A Team: A Leadership
Fable, and while it’s typically used in relation with the
establishment of cross-functional leadership teams as the primary
accountability team rather than organizational reports, the same concept
is applicable here for product teams.
Simply changing your organization’s language, without changing its
behaviors isn’t going to have a measurable impact on your scaling woes. Still,
it’s a simple place to start and it addresses the organizational friction and
larger cultural issues that lie at the root of your performance issues.
Changing the language
The more hands-on an organization is willing to be in breaking
silos, the more likely it is they will be effectively be breaking some
of the implicit ‘versus’ states that have enabled them.
Taking a hands-on approach to moderating the language
used in your organization is a simple first step to breaking down
barriers and reducing friction.
- Referring to your organizational group of practitioners as anything other
than “team” – such as squad, pod, or whatever term fits your organization’s
culture is a small change that can have a strong impact. - Naming your cross-functional product delivery teams,
ideally with the name of their product or value stream, is another simple change that helps
these multi-disciplined individuals adopt a new team identity separate from
their organizational reporting context. - Drop “us” and “them” from the professional vocabulary. Coming up with
alternative terms but keeping the same context of ‘that group over there – which
isn’t this group’ is cheating. We filter ourselves and our language in a
professional environment regularly and this language needs to be added to the
‘not allowed’ list.
Change the behavior
Those of us trying to
change our organizations’ culture need to define the things we want to
do, the ways we want to behave and want each other to behave, to provide
training and then to do what is necessary to reinforce those behaviors.
The culture will change as a result.
Changing an organization’s culture when it isn’t delivering the desired
results is hard. Many books have been written on the subject. By
defining and communicating the expected behaviors of your teams and
their respective team members up front, you set the underlying tone for
the culture you are striving to create.
- Leaders should set a principle and expectation of a blameless culture. When
something goes wrong, it’s a wonderful learning opportunity, to be studied and
used to continuously improve. An example of this is the concept of
a blameless post-mortem. - Set expectations and regularly inspect adoption of desired behaviors. Hold
team members accountable to these behaviors and refuse to tolerate
exceptions. - Orient “team” constructs around value streams instead of functions.
Differentiate “teams” as groups who deliver common value from Communities of
Practices or Centers of Excellence which are typically formed to deepen or
deliver specific competencies such as Product Management or Quality
Assurance. - Acknowledge that some personality types are less compatible than others.
Shift talented people around to find the optimal team synergy.
Define and communicate how your scaleup delivers value
A company is, in many ways, just one large team with one shared goal
— the success of the organization. When product and engineering don’t
have a shared understanding of that goal, it’s hard for them to come to
a collaborative agreement on how to achieve it. To avoid this source of
friction, executives must clearly articulate and disseminate the overall
value their organization provides to its customers, investors, and
society. Leaders are responsible for describing how each product and
service in your portfolio contributes to delivering that value.
Management must ensure that every team member understands how the work
that they do day in and day out creates value to the organization and
its customers.
The goal is to create a shared mental model of how the business
creates value. The best way to do this is highly dependent on the nature
of your business. We have found certain kinds of assets to be both
common and useful across our scaleup clients:
Assets that describe customer and user value
These should describe the value your product and services create, who
they create it for, and the ways you measure that value. Examples
include:
- Business Model Canvas/Lean Canvas
- User Journeys
- Service blueprints
- Personas
- Empathy maps
- Storyboards
- Job forces diagrams
- Product overviews (one-pagers, etc)
Assets that describe your economic model
These should describe the value your company receives from customers,
the cost of creating that value, and the ways in which you measure that
value. Examples include:
- Business flywheels
- Value stream maps
- Wardley maps
- Retention/churn models
- Customer acquisition models
- Customer lifetime value models
- Projected balance sheets and income
statements
Assets that describe your strategy
These should describe why you’ve chosen to serve these customers in
this way, the evidence you used to make that decision, and the highest
leverage ways you can increase the value you create and the value you
receive in return.
- Target customer profiles
- Issue trees
- Impact maps
- Opportunity/solution trees
- Hierarchy diagrams
- Causal loop diagrams
- Other bespoke frameworks and models
Once you have these assets, it’s important to constantly refer to
them in presentations, in meetings, when making decsions, and most importantly,
when there is conflict. Communicate when you change them,
and what made you make those changes. Solicit updates from the
organization. In short, make them part of your normal operations and
don’t let them become wallpaper or another cubicle pinup.
A simple heuristic that we’ve found to understand how successful
you’ve been in this communication is to pick a random individual
contributor and ask them to answer the questions answered by these
assets. The better they can do this without referring to the assets, the
better they will be at incorporating this thinking into their work. If
they don’t even know that the assets exist, then you still have a good
deal of work to do.
The alignment and focus these assets create allow for better
deployment of organizational resources, which enables scaling.
Additionally, they redirect the natural tension between product and
engineering. Instead of unproductive interpersonal friction, you can use
creating and updating these assets as a venue for true collaboration and
healthy disagreement about ideas that strengthen the company.
Create multidisciplinary stream-aligned teams
When a company is just starting out, it often only has one value
stream. But once it grows, it needs to break apart its products and
services into multiple value streams so that individual teams can
assume full ownership of various products or pieces of products. The
best way to do this decomposition is beyond the scope of this article
(we personally are big fans of Team Topologies as a way to think
through it), but some key things to consider are:
- Could this value stream and the products and services it comprises exist as
a separate company (even if it wouldn’t be a highly successful one)? - Can you align your value streams to a specific way that your company creates
value, or to a specific customer or user that the company creates value
for? - How do your different value streams interact with each other?
After defining your value streams, it’s time to bring
multi-disciplined team members together around them — because value creation is a team sport. Ask the leaders
of these value streams to create similar but more detailed versions of
the assets we discussed above, and then determine what skills and
capabilities are routinely needed to deliver and evolve the value of
the product(s) and/or service(s) in the value streams from start to
finish.
Pool those individuals together into an Outcome Oriented team, rather
than coordinating work across Activity Oriented or functional teams. In
Agile IT Organization Design, Sriram Narayam
states, “The more process and indirection there is, the greater the
friction for effective collaboration. By contrast, people within a team
don’t have to schedule meetings to collaborate with each other. They
collaborate continuously and get into huddles (informal, ad hoc
meetings—virtual or face to face) on demand.” While this model helps
reduce latency within outcome-oriented team, it also reduces the
friction among multidisciplinary team members.
Keep in mind that as your company grows, you may need to have
“teams of teams”, with multiple teams aligned around one value stream
and a team of cross-functional leaders for that value stream as well.
As the complexity of your value creation increases, so too does the
criticality of maintaining common purpose across your product delivery
teams.
Product managers and software engineers have a shared
responsibility to understand the needs of the customer so that they
can define and prioritize the work. There isn’t an ideal mix of
product people to engineers; every product is going to have a
different ratio. The important part is to know that both are
responsible for understanding, prioritizing and creating value.
As the product evolves, the needs of the team will evolve as well.
Take regular inventory of the team’s capabilities and empower the
stream-aligned teams to advocate for their own needs. Ensure that the
teams are fully resourced with the staff, skills, information, and
authority to deliver efficiently without unnecessary dependencies on
external resources. Fully sourced, empowered and autonomous product
teams operating as a single entity, regardless of each individual’s
reporting structure, dramatically reducing cross-discipline
friction.
Establish team working agreements at all levels
Ensure every team member knows what role they’re playing
The best teams are those that have negotiated the best ways of working for
themselves. It’s important for organizations to have established sensible
defaults to guide less mature teams on how to work effectively as a team. Even
with established defaults, it’s important that teams have autonomy to decide
which member will take on which responsibilities. This measure of autonomy
leads to greater accountability and a higher level of intrinsic motivation. As
new teams form, codify this working agreement in the team’s common knowledge
repository. During retrospectives, revisit this team working agreement as the
team learns more about each team member’s strengths and weaknesses and
reassign the responsibilities accordingly. This team working agreement becomes
both the social contract of the team, and also a unique “responsibility
fingerprint” that no other team has. As new team members join or rotate
through the team, having a referenceable team working agreement accelerates
integration and reduces time to value during onboarding.
Team working agreements frequently contain:
- Team Name: What is the unique identifier for the team?
- Team Mission: Why does this specific team exist? What value is it expected
to deliver? - Team Metrics: How will the team measure success? Include value creation
metrics, not just activity metrics. - Team Responsibilities: What work needs to be done to ensure success and
which team members are agreeing to own those work items? Organizations can seed
this work with typical activities and recommendations for team responsibility
allocations, but team members are free to renegotiate amongst themselves. - Team Skills: What skills are needed on this particular team to ensure
success? - Team Norms: Guidelines, principles, ceremonies, and/or sensible defaults for
team members to align on how team members are expected to behave, interact, and
make decisions.
Leaders form their own teams
Figure 2: Cross-functional collaboration at all levels
Working agreements are a useful tool for cross functional teams,
but they are also a great tool for aligning cross functional
leaders.
These three holistic view leaders—the head of product, the head
of design, and the head of technology—are obviously very valuable
individually, but in combination you can see their real power.
Executive leaders have a responsibility at a macro level to align
on corporate and product strategies and the subsequent measures of
success. If executives are not aligned on such measures as “desired
investment mix across products”, teams expected to deliver to these
measures are not being set up for success.
Functional line managers in a digital product organization may no
longer direct the day to day efforts of the stream-aligned team – that
responsibility falls to the team and the product manager on that team
– but they still have tremendous value. Functional managers are
responsible for ensuring that they are providing a healthy bench of
skilled players to staff those teams. It’s critical that these direct
managers are aligned on the roles and responsibilities of product team
members to avoid conflict within the product teams.
…team members must prioritize the results of the team over
their individual or departmental needs
Negotiate a balanced product investment mix
Figure 3: A balanced investment mix sweet spot
The diagram above explains the sweet spot of a balanced investment
mix, where we have traded off technical vs product investment. Over
investing with a product feature heavy backlog likely indicates an
underinvestment in technical debt and runway, leading to under-engineered
solutions, whereas over investing with a technology heavy backlog likely
indicates an underinvestment in customer valued features and
over-engineered solutions. It’s very hard to know when the balance is
perfect. It’s likely to change over time as your company grows and
pivots.
An example of under-engineering we often encounter is when a product
has problems with availability. This issue means the development team
has to spend time fighting fires, which reduces focus and affects their
productivity. While this might be sustainable when you are small, if
your customer usage spikes (in hypergrowth), the team becomes
overloaded, and customer experience is affected. That debt repayment
will always come due when your business can least afford it.
A different imbalance results if the technical team does too much
early optimization, and they end up over-engineering. An example of this
is overfitted architectures that are built to handle hundreds of
thousands of users when the company only has ten. When the startup
pivots, a lot of that work ends up being thrown away. There is always a
balance to strike between building the product to be scalable in the
future vs building what you need right now to survive.
The important thing is to be able to spot when this mix is
imbalanced, and be able to correct it. A continuous improvement process
is incredibly important. If a team (at product team level or a
management team) is aware of their shared goal, then a cross-functional
group can assess the balance regularly, and use data as a guide. Some
data will be quantifiable, and some will be more subjective. Information
you can use to guide you includes:
- Collecting individual opinions – does an engineer feel productive and
motivated? Does a product manager think the team is efficient? - Productivity metrics – How fast can we test and build a new feature?
- View of current state and near-term future state – Are we overcomplicating
build out for the sake of future scalability? - Product growth – Do we know how we are progressing towards a product’s goal?
Are there enough analytics, user testing and customer feedback to confirm that
our product investments are paying off? - Trends – As we build more features or increase users, how are metrics
trending? For example, look at metrics like the build time, the lead tiime to deploy
to production, and the amount of incidents. As complexity increases, technical
investment should bring them under control, and reduce toil for developers.
An experienced technologist that has scaled a technical platform is
highly valuable. They can interpret the data, using their intuition to
spot potential future problems, while taking a pragmatic point of
view.
Account for cross-functional requirements
A good product isn’t just a product with the latest features.
- It must be built to be performant, reliable and stable.
- It should be cost efficient – the cost of operating the product shouldn’t
exceed the revenue that the product generates. - The underlying architecture of the product should enable future feature
development to occur quickly and efficiently. - It should have in mind client expansion and be able to scale, without too
much rework. - It shouldn’t put private customer or business data at risk.
These and many other qualities of a product fall under the umbrella
of cross-functional requirements. Failing to account for these
requirements in the interest of getting new features out the door
creates compounding problems.
Some problems are more obvious because you can observe them. They’re
noticeable when a customer complains. Others are only going to be
noticeable over the long term. Martin Fowler talks about the importance of keeping internal quality
high – doing refactoring, creating automated tests, decoupling your
architecture. Early stage companies tend to skip this, for short term
productivity increases. This might be the right decision, but once they
think about adding more teams internal quality has to be addressed, or
long term value creation is forfeited.
Balancing the backlog
Creating a balanced backlog starts with trust, as it’s fundamentally
a negotiation between product and engineering. We recommended that every
product leader works to build a close, collaborative relationship with
their technical counterparts, and vice-versa. There will and should be
many difficult discussions as you work to find balance. A startup has
very limited resources, and often has to make hard trade-offs between
improving the developer experience and building new features.
Productive negotiation depends on transparency, the ability to share
detailed information, and the desire to see the situation from the other
person’s perspective. If a product manager understands the technical
architecture and strategy, they can suggest ideas that are easier to
build. If a technical person understands the reasoning and research
behind a product strategy, they can suggest alternative solutions that
the product person hasn’t thought of, e.g. utilizing ML/AI to solve a
problem.
When negotiating a backlog, startups often find it challenging to
understand the relative impact between potential investments — and
because usage and revenue metrics are easy to obtain and well
understood, work that will impact those is often prioritized, which
pulls the investment mix out of balance. To counteract this, we
recommend finding metrics that allow you to measure the impact of
technical investment as well. Each situation is different, but there are
a number of research-supported, de facto standards shown to improve
long-term productivity that you can use as a starting point.
- Focus on DevOps and DX outcome-driven metrics. Reading maximizing developer effectiveness
is a good starting point. - In the Thoughtworks Scaleup Studio, we have a number of sensible
defaults,
that come from a study of what practices and technologies that successful
scaleups are using. This includes continuous delivery, domain-oriented
microservices, prudent use of technical debt, building experimentation processes
and infrastructure. - Set a non-negotiable quality bar and keeping to it. For example, each
language has a set of good practices that we can easily check our work against, automatically.
Product vs Engineering collaboration approach as you grow
Phase 1
Experimenting
The startup itself is one team.
Preliminary working aggreements and assets to describe mission statement.
Investment mix is heavily oriented towards product investment. Often building to improve knowledge and not a
working product (e.g. throwaway prototypes).
Experiments with different economic models.
Phase 2
Getting Traction
The company starts to split off into sub-teams, still
thinks of itself as “one big team.”
Working agreements become more concrete
Customer value assets are refined and used in onboarding and
orientation. Economic model becomes clearer, but still flexible.
Hire your first non-founder product and engineering leaders.
Investment mix is still heavily product-oriented, focused on
creating a durable product — key foundational investments to support
scale.
Phase 3
(Hyper) Growth
Too large to operate as “one big team”, decomposes into
stream-aligned teams.
Cross-functional teams of leaders are created for middle
management. First platform engineering teams created.
No longer searching for new markets. Investment mix doubles down
on the value your products create.
Customer value, business strategy and economic model assets are
now quite concrete, very slow to change, and designed for
distribution.
Each product and sub-product creates its own value statements and assets as needed.
Phase 4
Optimizing
Leaders have to actively work against becoming siloed along
functional lines.
Team structure starts to change to optimize for maximum autonomy
and agency.
Structures to support skill development and consistency across
functional groups emerge.
Multiple teams created to make work on stream aligned teams more
efficient (platform engineering, product ops, design ops, etc).
Investment mix in core products becomes more focused on technical investment,
including an investment in developer experience.
