Any organization hoping to survive in a technological, globalized
world has to effectively work with data. With 97% of organizations
investing in big data and
AI, the need to
effectively manage this data is more critical than ever. Data Mesh
helps companies to generate valuable insights and live up to the
promise of becoming data driven organizations.
The Data Mesh paradigm is founded on
four principles:
- domain-oriented decentralization of data ownership
and architecture - domain-oriented data served as a product
- self-serve data infrastructure as a platform to enable autonomous,
domain-oriented data teams - federated governance to enable
ecosystems and interoperability
Data mesh is a decentralized socio-technical approach to remove the
dichotomy of analytical data and business operation. Unfortunately, as
with many socio-technical approaches, many organisations struggle to align
on the goals and strategy that is really needed to make a data mesh
transformation a success. The transformation to a data mesh is hard. It
represents a paradigm shift in how teams are organized, how work is
prioritised and how to apply the newest data related technology advances.
This is a wide reaching change that impacts people across the whole
organization.
We have been part of several Data Mesh transformations, across a wide
variety of organizations. We have also exchanged and learned from
Thoughtworkers around the world who have been involved in many more.
We have seen two common challenges facing Data Mesh transformations:
technology focused and big design upfront.
In one approach, teams start building data products without aligning
their work with higher level goals. This approach (technology focused) is
based on a genuine thinking that Data Mesh is all about technologists and
new ways of collecting, transforming and serving data. The main goal of
Data Mesh is to unlock the value of data at scale, and while technology
is a key part of this, alignment with business goals is critical for
success.
To start from the technical side without being very aligned with the
business strategic goals is like packing for a holiday without knowing
where you are going. Don´t do it. Make sure there is an initial alignment
before starting the journey.
Another common approach is to do a deep dive analysis before starting
with Data Mesh implementation. This is Big Design Up Front, with long
periods for analysis and design, often a few months. The result: lots of
diagrams and plans for the path going forward.
Diagrams and plans are great, but the value comes from making them
real and delivering value to the users and the business. A key concept of
Data mesh is product thinking, where we incrementally evolve our products
based on user needs and real usage. Our approach to the data mesh
transformation as a whole also needs to follow this evolutionary
approach, with short feedback loops.
The Data Mesh Accelerate Workshop
At Thoughtworks, our response has been a process called Data Mesh
Accelerate. We have put together a good sequence of activities to set an
initial direction, to build understanding and to create strategic
alignment. It provides enough initial analysis and design to get started,
and a good balance between technology and business alignment.
The Data Mesh Accelerate workshop provides a series of activities
focusing on collaboration, the capture of the current state and the
mapping of aspirations. The main goal is to build the initial alignment
across stakeholders, to understand what kind of Data Mesh outcomes the
group is aiming at, and explore the process for identifying, designing
and building data products.
Leaving the workshop, stakeholders should have a shared understanding
of Data Mesh concepts, what the journey ahead looks like and alignment on
the next steps. The Data mesh workshop represents the first step on an
organization’s data mesh transformation, and while there are many steps
ahead, we have found that taking the time to make the first step
successful, pays off massively for the journey ahead.
Where the accelerate workshop fits in
The Data Mesh Accelerate Workshop is one part of a data mesh
transformation. To kick off a data mesh transformation you start by
selecting an appropriate domain to work with, then you go from the
vision to the use case (accelerate), then you identify the data
products, platform and organizational changes needed to support that use
case (discovery and inception) before actually building the data
products and adding them to the data mesh.
The Data Mesh Accelerate workshop fits in when you have got the domain
stakeholders (business and technical people) interested and available
for a few hours to kick start a successful Data Mesh transformation.
It provides the means to get everyone aligned on the initial path.
Prep work
Facilitating any workshop, let alone a multi-day workshop like Data
Mesh Accelerate, requires preparation to make it a successful
experience. The sub-sections below explore what you need to do before
the day of the workshop.
Materials, tools and logistics
At the most basic level you will need to arrange the material and
tools (or their digital equivalents) and logistics required for the
workshop:
- Gather Post-Its and pens
- Prepare collaboration boards (see below)
- Give access to the needed tools (video-conferencing, online boards
etc.) - Book a room and time in the participants calendars
Preparing the audience
Along with understanding what the workshop is about, each
participant needs to know what is expected from them. It is your job to
help clarify it as well as to make sure the right people are invited to
the workshop.
You should communicate the goal of the workshop and where it fits
with the context of the group going through it and the organization.
You have to put it in context on what has happened before and what will
happen after it. You can achieve this in several different ways:
- Send a short introductory email
- Share a explanatory video
- Schedule a conference call
- Share a link to this article and to Zhamak’s
articles - Schedule one-on-ones with participants
Besides preparing the audience, you will be gaining an insight into
how technically skilled and familiar with data mesh the participants
are. This will also help you adapt the content to their specific
needs.
Agenda management
Every workshop needs an agenda. Clarifying the agenda and inviting
people well in advance will help put your participants at ease and help
them find times in their busy schedules.
In the next section is a typical agenda we have been using. We
recommend you start from there and adjust it according to your specific
context and needs.
Preparing specific activities
Each activity needs a working area with its instructions. These are
described in the next section. But it is your job to prepare these
areas for your workshop. In a face to face setting, you might prepare a
few flipcharts, while if you are remote, you will arrange the template
on your collaboration tool of choice. We describe the activities in
terms of their outcomes and their step-by-step, leaving you to adapt to
your specific context and setting.
Please find below a sample agenda and the explanation for each of
the activities.
Here is a link to a MURAL template
following the above agenda.
The sample agenda lays the workshop out as four afternoons. We typically
go with such an agenda when running the workshop remotely. It avoids
videoconferencing fatigue and makes it easier for participants to fit it in
their busy agendas.
Whenever we run this workshop in person we do it in two consecutive days.
We reserve a good conference room and fill the boards and walls with many
post-is as we go through the activities.
A Sample Agenda for a data mesh accelerate workshop
Kick-off
The week starts with a kick-off, followed by a sequence of intense
sessions and ends with a review. The agenda and expectations for the
week are covered on the kick-off. The results obtained from the
workshop are presented in the review.
Step by Step
- Ask the main sponsors of the organization to open the workshop with a
speech about the importance of this engagement. - Make a brief presentation about the intention of the overall workshop,
its agenda, and each activity. This is also your chance to bring attention to
the ground rules (see below). - Ask everyone to quickly introduce themselves (tip: besides name and role,
it is nice to ask people to share something else, such as favorite food or
vacation spot).
The kick-off should end with a clear understanding of the goal for
the workshop and the upcoming activities. It may be the first time some
people get to work with each other, therefore the importance of the
tip, making some space for people to connect to each other.
Setting clear ground rules, such as switching off phones or
expecting people to be open to other perspectives will help
participants understand what is expected of them and help the whole
workshop run smoothly.
Data Mesh Four Principles and challenges ahead
The four principles of Data Mesh activity has two main goals: (1)
build a shared understanding of the four principles of Data mesh —
Domain Ownership, Data as a Product, Self-serve Data Platform and
Federated Computational Governance –, and (2) foster an open
conversation about the challenges to implement each of these in the
current context of the organization and the group.
Step by Step
- Share a quick introduction to the four principles
- Ask the participants to share the threats to applicability for each
principle: “What are the challenges in the way for us getting there?” - Have a conversation about the challenges ahead
It is important to get everyone on a similar level of understanding
about Data Mesh and its principles. Someone with a good understanding
of data mesh should give a short presentation to the audience, covering
at a minimum:
- The “Why” of Data Mesh, what problems it aims to solve.
- The Four Principles of Data Mesh, explaining the reasoning and details of
each.
After sharing the information, give the audience time to clarify,
share and ask questions about the principles.
Ask the participants to share their previous experiences, concerns
and where they see the challenges ahead for each of the four
principles. Talking about the challenges helps the group get an initial
feeling about the areas to pay closer attention to, especially in the
beginning of a transformation driven by data.
“If we were to start applying this principle tomorrow, what
challenges would we face?”
As participants are sharing their perspectives, ask the rest of the
group to compare their perspectives to those being presented by asking
three questions; what was the same, what was different and what
surprised them. Asking
these questions is a great way to compare and contrast the different
perspectives in the group, uncovering similarities and blind spots.
By exploring the challenges faced by the organisation and the pain
points raised by the participants, you are able to understand the
current context and frame subsequent discussions appropriately. Given
that Data Mesh aims to address common failure modes experienced by data
organisations, understanding the specific challenges faced by your
organisation is a great place to start!
Data Mesh Nirvana
The goal of this activity is to understand what good looks like in
the context of Data Mesh by creating a nirvana statement that clearly
articulates an ideal future state. This activity helps the group look
forward and provides an opportunity to introspect on some of our
current pain points.
“Nirvana is that special place, that end state of perfection (many
times not achievable, but important to aspire for). It is very
important to align and understand what nirvana is, for this group of
people and within this organizational context.”
Step by Step
- Break the participants into smaller groups.
- Have each group create a nirvana statement, then present it back to the
larger group. - Combine to create a shared nirvana statement.
Breaking up the participants into smaller groups helps facilitate
lively discussion. People increase their participation when working in
smaller groups. We find that groups of 4-6 participants made active
discussion easier.
Give each group the same instructions: to write a short statement
about the ideal future state for the Data Mesh transformation. Stickies
with the prompt “Our Nirvana is…” can help kick off this
discussion.
After the groups have created their nirvana statements, reconvene
and have a volunteer from each group read out their statement and share
the thinking behind it.
The final task is to combine the different statements into a single
shared nirvana statement. We recommend a
fishbowl
approach to stimulate discussion and collaboration. By the end of this
process, you should have a single nirvana statement that everyone in
the group agrees represents a desirable goal.
It is very interesting to see how this simple question gets
participants very engaged and then aligned: what is our nirvana? Once
the group is in agreement about that, it’s much easier to clarify the
steps towards it.
Before talking about options, actions, initiatives, or any execution
task towards achieving something big, it is essential to align on that
high level goal.
4 Key Metrics
The four key metrics activity has two main goals: (1) to foster a
conversation about the current state of the four key metrics, and (2)
to make it visible where the team aims to get to.
The book
Accelerate
and the associated
research demonstrates
the importance of the 4 Key
Metrics
for achieving great organizational and software delivery performance.
The 4 Key Metrics are Lead time, Deploy frequency, Mean Time to Restore
(MTTR) and Change fail percentage.
Tell me how you measure me and I will tell you how I behave
Step by Step
- Share details of the 4 Key Metrics and have a discussion about their
importance. - Ask the participants to share where they consider they are today on each
of these 4 metrics - Discuss future expectations around the 4 Key Metrics.
Start by introducing the 4 Key Metrics so that participants are familiar
with their usage and the research behind them.
Within the workshop participants there should be representatives of
different areas and different roles. This diverse group of people is needed to
build a complete understanding of where they are for each of the metrics.
Start at the first metric, lead time, for example. Ask the participants
to share where they consider they are today as per the Lead time metric:
- More than six months
- One to six months
- One week to one month
- One day to one week
- Less than one day
- Less than one hour
As the dots are gathered, ask the participants to share the stories behind
the dots. The outliers in particular can offer valuable insights into the
experience of the teams.
To close the activity, you should foster a conversation about how the
participants think they could improve on these metrics over the next year.
Even though the group is not yet deciding exactly how to get there, it is
important to open the conversation and pay attention to the effort ahead of
them. To kick off this conversation, you could say: “consider a year from now,
where do you believe we could be at for each of these four metrics?”
Metrics bring an important conversation about desirable outcomes and how to
measure progress. But even more, it raises everyone’s attention to the current
state and to a desirable future state, including an initial conversation about
how far away the desirable state is. This activity brings tech and business
together, not only by having a common goal, but also by looking at progress
via similar (measurable) lenses.
Objectives and Key Results
This activity’s goal is to align on the organisation’s main objectives and
start a conversation about measurable progress towards success. There are many
ways to have this conversation. In this article, we have chosen OKRs –
Objectives and Key Results, but building a Lean Value
Tree is another
effective approach that links business vision to our daily work. Of course, if
your organisation is already using a goal/objective framework, you might be
able to save some time by pulling in your last set of objectives from
that!
Objectives and Key Results (OKRs) are a
goal-setting framework that helps a group of people define goals — or
Objectives — and then track the outcome via quantitative metrics, the Key
Results. OKRs has been around since the 1970s from Intel, and many popular
organizations, such as Google, Oracle, Twitter, LinkedIn and Dropbox have
adopted them.
Objectives (Where do we want to go?) and Key Results (How do we know if we
are getting there?)
Step by Step
- Identify the high level objectives of the organization
- Brainstorm key results that can measure progress towards those
objectives - Iterate over the OKRs as a group
OKRs can be tricky to write well. Are the objectives really worthwhile
outcomes or are they outputs? Do the Key Results really capture what is
needed to deliver on the Objectives? Avoid common
mistakes
by giving the participants time to refine their OKRs.
Identifying Objectives and Key Results helps make the journey ahead more
tangible. As we move ahead and start to define the (data) products we intend
to build, keeping these OKRs in mind helps bridge the gap between outcome
and output.
Explore the Use Cases
Having identified your Objectives in the previous step, you now turn your
attention to how you can actually deliver on those goals. Use cases are a
translation of the desired outcome for a customer or a user. In the context of
analytical data, use cases are usually either:
Efficiency gains
e.g. Forecast service usage to optimise efficiency
or
Experience improvements
e.g. Recommend a service to users based on their past
purchases
This activity is the moment for the group to brainstorm and align on Data
Mesh Use Cases. To help focus the discussion, we provide the following Use
Case template:
We believe that <this Use Case> will help achieving
<this GOAL>
Step by Step
- Divide the participants into groups and ask them to brainstorm use cases,
using the template above. - Have each group present their use cases to the wider group, followed by
discussion and clustering - Refine and select Use Cases
You should kick off this activity with a definition and discussion of Data
Mesh Use Cases. This will make the brainstorming more productive. At this
stage, you want to gather as many ideas as you can, because you will filter
down later.
When the groups return, have a volunteer for each group present the use
cases they identified and open up to a wider discussion. This is a great
opportunity to combine (or cluster) similar use cases and discard
non-analytical use cases.
Once you have a set of analytical use cases, you need to decide which ones
to take forward to the next activities. If you have a few Use Cases, you might
take all of them to the next activity. Although if there are too many, you
should prioritize them. There are quite a few prioritization
techniques, choose the
one that fits your context.
“Build it and they will come” is a pitfall that many transformation efforts
fall into, so by keeping the Use Cases connected to the Objectives previously
identified, via the presented template, ensures that the work is aligned with
the high level engagement outcomes.
Discovering Data Products
Data Mesh applies Product Thinking to remove friction, deliver value and
truly delight our data users. Data
Products
are the architectural quantum of Data Mesh, the “smallest unit of architecture
that can be independently deployed with high functional cohesion, and includes
all the structural elements required for its function.” (definition from
Evolutionary Architectures).
These Data Products represent the building blocks of Data Mesh and in this
activity, we identify Data Products that can help us satisfy the Use Cases we
identified above. The interaction of one or more Data Products fulfills a Use
Case.
Having a small set of Use Cases in the previous activity, you now need to
start comprehending, proposing and mapping the relationship between Use Cases
and Data Products.
Step by Step
- Identify Data Products for the selected Use Cases
- Identify sources and consumers for each data product
- Map the connections between the data products
“If you were to hire a data product to help address this Use Case, what
would its job be?”.
Jobs to be done
(JTBD) is a
framework created by Clayton Christensen for understanding customers and their
motivations for adopting a new product or service. For defining a Data
Product, we suggest you follow the same approach. The simple question above
helps the participants start a discussion about how to put their data to good
use.
Once you have identified the data product’s job, you can dig a little
deeper by asking what data would be needed to actually carry out that job.
This may include existing master data or operational systems, manual data
entry or even other data products.
With the sources of the data product identified, you should ask who would
actually be interested in consuming this data. This may include particular
people, roles, other data systems or other data products.
Part of the vision behind Data Mesh is a mesh of reusable, interoperable
data products. In the final part of this activity, you want to map out how the
data products interact with each other. Wherever a data product consumes from
another data product, draw that link on the board. At the end of this
activity, you should start to see a
data product interaction map
Identifying Data Products, their sources, their consumers as well as the
connections between them are an essential part of a successful Data Mesh
transformation. At this stage of the workshop, the group will have connected
very important pieces of the puzzle for the first few Use Cases.
Wrap up and review
Wrapping up the workshop is important in order to provide a sense of
closure to the experience, review the progress made, and initiate a
conversation around next steps. The accelerate workshop will feel much more
complete when you give a short summary at the end and provide some context for
next steps.
You should wrap up the workshop by summarizing the takeaway points and the
connections from the different activities of the workshop. Some topics you may
want to cover in the review include:
Step by Step
- The goals, use cases and data products identified during the workshop
- Any interesting observations, learnings or recommendations
- Details of next steps
It may also be helpful to invite external stakeholders, who may have an
interest in the outcomes of the workshop to this session.
After the review, there are a few activities that we recommend wrapping up
with:
Futurespective
Running a short futurespective activity like Future LinkedIn
Posts helps the
participants visualise where this journey will take them.
Check Out
If time permits, a short check-out activity like One Word Before
Leaving can be a
great way to share experiences and gather feedback from your participants
before bringing the workshop to a close.
With your last activities done, you can bring the workshop to a close as
you look forward to kicking off your data mesh transformation!
Closing thoughts
The Data Mesh Accelerate workshop is an intensive workshop, bringing
together participants from across your organization to build a shared
understanding of Data Mesh and alignment on the next steps.
Things are moving really fast in the technical and business data
world. Workshops and activities such as the ones we presented here are
the means to bring people together, align and plan for a successful
transformation.
We and many of our colleagues have been using what we shared here to
achieve great results. We hope these work for you as well and you use it,
adapt it, add to it and keep on sharing with our great data
community.
