Meeting Mollie’s Data Demands, Part 1: Specialisation
Since I joined Mollie three years ago, demand for data has exploded across every dimension imaginable:
- The company has grown in size from 100 to more than 700 employees
- More and more teams want to use data to inform decision making
- Several new products have been launched, all synthesising valuable data
- The complexity of questions asked of our data is growing all the time
- Expectations around data quality & freshness are (rightfully) increasing
To me this sometimes feels like being asked to get better at taking penalties while the goal is shrinking, the ball is getting bigger, and you are being moved further and further away from the net.
Suffice to say, these demands have posed significant challenges for Mollie’s Data Department and in particular, my team, formerly known as ‘BI & Analytics’.
To help address these challenges, we split our team members into two separate, more specialised functions — Analytics Engineering and Business Intelligence.
In this blog post (part 1 of 2) I’ll introduce these new functions and explain the important role that they each play for Mollie.
🤷🏽♀️ What is Analytics Engineering?
For those not familiar with this relatively new discipline, analytics engineering is about transforming raw data sources into clean, reusable datasets which form the foundations of an organisation’s analytical capabilities.
What’s different about analytics engineering compared to other disciplines historically tasked with data transformation is it actively incorporates well-established best practices from the world of software engineering.
For example, these include:
- Modular design: by breaking code into smaller, modular pieces — each with more targeted functions — code becomes easier to digest and thus debug, all while encouraging reusability.
- Development workflows: code is managed in a git repository, which creates an auditable trail of changes and allows analytics engineers to vet each others’ work more easily before deploying to production.
- Testing & validation: assertions are made about raw data sources and transformed datasets exposed to consumers — for example that a primary key must be unique, or that an aggregation must fall within a specified range.
- Don’t repeat yourself (DRY): the DRY principle aims to eliminate code that is duplicated and redundant — which could be described as WET (write everything twice 🤡) — thereby improving maintainability and efficiency.
⚖️ Balancing domain knowledge with technical craft
As Mollie products grow in complexity, so too does the need for the specialisation of our analytics engineers. Not only do team members require a deep understanding of raw data sources, they must also have a solid grasp of the questions asked of the data and the nuances around them. This is most easily achieved through close collaboration with the product teams responsible for creating Mollie’s data.
Alongside this, analytics engineers must work closely with each other to share technical knowledge and ensure that our development standards are upheld equally across data from every corner of the business.
To balance these competing requirements, Mollie’s analytics engineers have adopted a hybrid structure, similar to the concept of ‘chapters’ in the Spotify model.
In this arrangement, team members form a functional unit which meets regularly to refine, plan, and review work. Individual team members then specialise within different parts of the business, as shown below:
An additional benefit of this hybrid structure is that analytics engineers can act as the ‘voice of data’ in product teams.
Firstly, this means raising the alarm when we see issues around raw data quality so that they can be fixed at the source.
Secondly, analytics engineers can actually influence the product development process to ensure the right data is made available and in a format that lends itself nicely to analytics. This has recently started to take place during the development of Mollie Connect for Platforms and the integration of in3 payments, but we aim for it to become the status quo as this new structure becomes more established.
🤷🏻♂️ What about Business Intelligence?
Despite the limelight placed on Analytics Engineering in this post, it’s worth highlighting the equally important role played by Mollie’s business intelligence analysts.
This function acts as a centre of excellence for Looker, owning performance optimisation, the development of best practices, and providing training to different groups of users.
Team members are also the proud owners of our ‘core’ Looker model, which contains the business logic underpinning Mollie’s most important and sought after data — covering topics like payments, customer onboarding, customer balances, and partnerships.
In Looker, we use a hub & spoke architecture, allowing data analysts embedded in verticals such as operations, finance and marketing to utilise and build on top of this core business logic. This could be through supplementing it with data from their own use cases (e.g. targets) or creating their own department-specific metrics, for example.
Importantly though, our Looker projects have been architected in such a way that prevents vertical data analysts from readily making changes to core business logic.
This approach helps preserve the integrity of core data, whilst ensuring that everyone looks at it in the same way, regardless of which part of the business they may sit in.
💕 Teamwork makes the dream work
Even though Business Intelligence and Analytics Engineering at Mollie are separate roles with distinct functions, their work is tightly entwined.
Our BI Analysts rely on Analytics Engineers not only for the supply of actual data, but also for key metadata (like column descriptions) which help give data meaning, and model configurations (like partitioning and clustering) which largely determine downstream performance.
In this sense, the two functions are strategic partners — and it is for this reason that the team’s long-term objectives around performance optimisation and data quality are shared across both disciplines.
🔮 What’s next?
Alongside the organisational changes described in this blog post, we’ve been busy implementing dbt to improve our capabilities around data transformation — more on that soon, in part 2 of this post.
As always, thank you for taking the time to read — I hope you found it helpful and don’t forget to keep an eye out for upcoming vacancies in Mollie’s Data Team!
