Any organization hoping to survive in a technological, globalized
world has to effectively work with data. With 97% of organizations
investing in big data and
AI, the need to
effectively manage this data is more critical than ever. Data Mesh
helps companies to generate valuable insights and live up to the
promise of becoming data driven organizations.
The Data Mesh paradigm is founded on
four principles:
- domain-oriented decentralization of data ownership
and architecture - domain-oriented data served as a product
- self-serve data infrastructure as a platform to enable autonomous,
domain-oriented data teams - federated governance to enable
ecosystems and interoperability
Data mesh is a decentralized socio-technical approach to remove the
dichotomy of analytical data and business operation. Unfortunately, as
with many socio-technical approaches, many organisations struggle to align
on the goals and strategy that is really needed to make a data mesh
transformation a success. The transformation to a data mesh is hard. It
represents a paradigm shift in how teams are organized, how work is
prioritised and how to apply the newest data related technology advances.
This is a wide reaching change that impacts people across the whole
organization.
We have been part of several Data Mesh transformations, across a wide
variety of organizations. We have also exchanged and learned from
Thoughtworkers around the world who have been involved in many more.
We have seen two common challenges facing Data Mesh transformations:
technology focused and big design upfront.
In one approach, teams start building data products without aligning
their work with higher level goals. This approach (technology focused) is
based on a genuine thinking that Data Mesh is all about technologists and
new ways of collecting, transforming and serving data. The main goal of
Data Mesh is to unlock the value of data at scale, and while technology
is a key part of this, alignment with business goals is critical for
success.
To start from the technical side without being very aligned with the
business strategic goals is like packing for a holiday without knowing
where you are going. Don´t do it. Make sure there is an initial alignment
before starting the journey.
Another common approach is to do a deep dive analysis before starting
with Data Mesh implementation. This is Big Design Up Front, with long
periods for analysis and design, often a few months. The result: lots of
diagrams and plans for the path going forward.
Diagrams and plans are great, but the value comes from making them
real and delivering value to the users and the business. A key concept of
Data mesh is product thinking, where we incrementally evolve our products
based on user needs and real usage. Our approach to the data mesh
transformation as a whole also needs to follow this evolutionary
approach, with short feedback loops.
The Data Mesh Accelerate Workshop
At Thoughtworks, our response has been a process called Data Mesh
Accelerate. We have put together a good sequence of activities to set an
initial direction, to build understanding and to create strategic
alignment. It provides enough initial analysis and design to get started,
and a good balance between technology and business alignment.
The Data Mesh Accelerate workshop provides a series of activities
focusing on collaboration, the capture of the current state and the
mapping of aspirations. The main goal is to build the initial alignment
across stakeholders, to understand what kind of Data Mesh outcomes the
group is aiming at, and explore the process for identifying, designing
and building data products.
Leaving the workshop, stakeholders should have a shared understanding
of Data Mesh concepts, what the journey ahead looks like and alignment on
the next steps. The Data mesh workshop represents the first step on an
organization’s data mesh transformation, and while there are many steps
ahead, we have found that taking the time to make the first step
successful, pays off massively for the journey ahead.
Where the accelerate workshop fits in
The Data Mesh Accelerate Workshop is one part of a data mesh
transformation. To kick off a data mesh transformation you start by
selecting an appropriate domain to work with, then you go from the
vision to the use case (accelerate), then you identify the data
products, platform and organizational changes needed to support that use
case (discovery and inception) before actually building the data
products and adding them to the data mesh.
The Data Mesh Accelerate workshop fits in when you have got the domain
stakeholders (business and technical people) interested and available
for a few hours to kick start a successful Data Mesh transformation.
It provides the means to get everyone aligned on the initial path.
Prep work
Facilitating any workshop, let alone a multi-day workshop like Data
Mesh Accelerate, requires preparation to make it a successful
experience. The sub-sections below explore what you need to do before
the day of the workshop.
Materials, tools and logistics
At the most basic level you will need to arrange the material and
tools (or their digital equivalents) and logistics required for the
workshop:
- Gather Post-Its and pens
- Prepare collaboration boards (see below)
- Give access to the needed tools (video-conferencing, online boards
etc.) - Book a room and time in the participants calendars
Preparing the audience
Along with understanding what the workshop is about, each
participant needs to know what is expected from them. It is your job to
help clarify it as well as to make sure the right people are invited to
the workshop.
You should communicate the goal of the workshop and where it fits
with the context of the group going through it and the organization.
You have to put it in context on what has happened before and what will
happen after it. You can achieve this in several different ways:
- Send a short introductory email
- Share a explanatory video
- Schedule a conference call
- Share a link to this article and to Zhamak’s
articles - Schedule one-on-ones with participants
Besides preparing the audience, you will be gaining an insight into
how technically skilled and familiar with data mesh the participants
are. This will also help you adapt the content to their specific
needs.
Agenda management
Every workshop needs an agenda. Clarifying the agenda and inviting
people well in advance will help put your participants at ease and help
them find times in their busy schedules.
In the next section is a typical agenda we have been using. We
recommend you start from there and adjust it according to your specific
context and needs.
Preparing specific activities
Each activity needs a working area with its instructions. These are
described in the next section. But it is your job to prepare these
areas for your workshop. In a face to face setting, you might prepare a
few flipcharts, while if you are remote, you will arrange the template
on your collaboration tool of choice. We describe the activities in
terms of their outcomes and their step-by-step, leaving you to adapt to
your specific context and setting.
Please find below a sample agenda and the explanation for each of
the activities.
Here is a link to a MURAL template
following the above agenda.
The sample agenda lays the workshop out as four afternoons. We typically
go with such an agenda when running the workshop remotely. It avoids
videoconferencing fatigue and makes it easier for participants to fit it in
their busy agendas.
Whenever we run this workshop in person we do it in two consecutive days.
We reserve a good conference room and fill the boards and walls with many
post-is as we go through the activities.
A Sample Agenda for a data mesh accelerate workshop
Kick-off
The week starts with a kick-off, followed by a sequence of intense
sessions and ends with a review. The agenda and expectations for the
week are covered on the kick-off. The results obtained from the
workshop are presented in the review.
Step by Step
- Ask the main sponsors of the organization to open the workshop with a
speech about the importance of this engagement. - Make a brief presentation about the intention of the overall workshop,
its agenda, and each activity. This is also your chance to bring attention to
the ground rules (see below). - Ask everyone to quickly introduce themselves (tip: besides name and role,
it is nice to ask people to share something else, such as favorite food or
vacation spot).
The kick-off should end with a clear understanding of the goal for
the workshop and the upcoming activities. It may be the first time some
people get to work with each other, therefore the importance of the
tip, making some space for people to connect to each other.
Setting clear ground rules, such as switching off phones or
expecting people to be open to other perspectives will help
participants understand what is expected of them and help the whole
workshop run smoothly.
Data Mesh Four Principles and challenges ahead
The four principles of Data Mesh activity has two main goals: (1)
build a shared understanding of the four principles of Data mesh —
Domain Ownership, Data as a Product, Self-serve Data Platform and
Federated Computational Governance –, and (2) foster an open
conversation about the challenges to implement each of these in the
current context of the organization and the group.
Step by Step
- Share a quick introduction to the four principles
- Ask the participants to share the threats to applicability for each
principle: “What are the challenges in the way for us getting there?” - Have a conversation about the challenges ahead
It is important to get everyone on a similar level of understanding
about Data Mesh and its principles. Someone with a good understanding
of data mesh should give a short presentation to the audience, covering
at a minimum:
- The “Why” of Data Mesh, what problems it aims to solve.
- The Four Principles of Data Mesh, explaining the reasoning and details of
each.
After sharing the information, give the audience time to clarify,
share and ask questions about the principles.
Ask the participants to share their previous experiences, concerns
and where they see the challenges ahead for each of the four
principles. Talking about the challenges helps the group get an initial
feeling about the areas to pay closer attention to, especially in the
beginning of a transformation driven by data.
“If we were to start applying this principle tomorrow, what
challenges would we face?”
As participants are sharing their perspectives, ask the rest of the
group to compare their perspectives to those being presented by asking
three questions; what was the same, what was different and what
surprised them. Asking
these questions is a great way to compare and contrast the different
perspectives in the group, uncovering similarities and blind spots.
By exploring the challenges faced by the organisation and the pain
points raised by the participants, you are able to understand the
current context and frame subsequent discussions appropriately. Given
that Data Mesh aims to address common failure modes experienced by data
organisations, understanding the specific challenges faced by your
organisation is a great place to start!
Data Mesh Nirvana
The goal of this activity is to understand what good looks like in
the context of Data Mesh by creating a nirvana statement that clearly
articulates an ideal future state. This activity helps the group look
forward and provides an opportunity to introspect on some of our
current pain points.
“Nirvana is that special place, that end state of perfection (many
times not achievable, but important to aspire for). It is very
important to align and understand what nirvana is, for this group of
people and within this organizational context.”
Step by Step
- Break the participants into smaller groups.
- Have each group create a nirvana statement, then present it back to the
larger group. - Combine to create a shared nirvana statement.
Breaking up the participants into smaller groups helps facilitate
lively discussion. People increase their participation when working in
smaller groups. We find that groups of 4-6 participants made active
discussion easier.
Give each group the same instructions: to write a short statement
about the ideal future state for the Data Mesh transformation. Stickies
with the prompt “Our Nirvana is…” can help kick off this
discussion.
After the groups have created their nirvana statements, reconvene
and have a volunteer from each group read out their statement and share
the thinking behind it.
The final task is to combine the different statements into a single
shared nirvana statement. We recommend a
fishbowl
approach to stimulate discussion and collaboration. By the end of this
process, you should have a single nirvana statement that everyone in
the group agrees represents a desirable goal.
It is very interesting to see how this simple question gets
participants very engaged and then aligned: what is our nirvana? Once
the group is in agreement about that, it’s much easier to clarify the
steps towards it.
Before talking about options, actions, initiatives, or any execution
task towards achieving something big, it is essential to align on that
high level goal.
4 Key Metrics
The four key metrics activity has two main goals: (1) to foster a
conversation about the current state of the four key metrics, and (2)
to make it visible where the team aims to get to.
The book
Accelerate
and the associated
research demonstrates
the importance of the 4 Key
Metrics
for achieving great organizational and software delivery performance.
The 4 Key Metrics are Lead time, Deploy frequency, Mean Time to Restore
(MTTR) and Change fail percentage.
Tell me how you measure me and I will tell you how I behave
Step by Step
- Share details of the 4 Key Metrics and have a discussion about their
importance. - Ask the participants to share where they consider they are today on each
of these 4 metrics - Discuss future expectations around the 4 Key Metrics.
Start by introducing the 4 Key Metrics so that participants are familiar
with their usage and the research behind them.
Within the workshop participants there should be representatives of
different areas and different roles. This diverse group of people is needed to
build a complete understanding of where they are for each of the metrics.
Start at the first metric, lead time, for example. Ask the participants
to share where they consider they are today as per the Lead time metric:
- More than six months
- One to six months
- One week to one month
- One day to one week
- Less than one day
- Less than one hour
As the dots are gathered, ask the participants to share the stories behind
the dots. The outliers in particular can offer valuable insights into the
experience of the teams.
To close the activity, you should foster a conversation about how the
participants think they could improve on these metrics over the next year.
Even though the group is not yet deciding exactly how to get there, it is
important to open the conversation and pay attention to the effort ahead of
them. To kick off this conversation, you could say: “consider a year from now,
where do you believe we could be at for each of these four metrics?”
Metrics bring an important conversation about desirable outcomes and how to
measure progress. But even more, it raises everyone’s attention to the current
state and to a desirable future state, including an initial conversation about
how far away the desirable state is. This activity brings tech and business
together, not only by having a common goal, but also by looking at progress
via similar (measurable) lenses.
