MLOps Principles in Practice: Continuous improvement

This is the third instalment of a multi-article series exploring a set of MLOps principles that we practice at Relative Insight.

In terms of prime principles, there are three main things that motivate us at Relative regarding MLOps. These are motivated both by how we can create a safe and creative space for our technical team to thrive, and also by how we can best deliver novel functionality at the best price for our customers. They are:

Communication
Continuous improvement
Conscientious pricing

These practices are key because they operate at multiple levels – they impact developer well-being, product health, customer experience and more.

In the previous article, we addressed the importance of feedback and communication. It therefore makes sense to look at a tangible benefit of feedback which is important enough to merit its own piece. That is the principle and practice of continuous improvement.

If you’re able to gather the feedback and act on the information, you’re already well on the path of continuous and iterative improvement.

Of course, most people, companies, and tech teams are going to say that they want to improve and that there’s always room to grow. So really, this is all about reducing the time of the feedback cycle, so much so that you move from step-change improvements on a termly cycle to continuous, minimal improvements that happen so often that it’s near impossible to count them.

As before, there are ways to do that operationally, focused on an angle with how you deploy your ML models, and your software. But there is also the set of cultural practices that every team can employ – and will enable you to put data at the center of your organizational decision-making and best create an environment for team members to thrive.

The pathway to continuous improvement is the feedback loop.

Imagine how difficult it would be if Google Maps (or your navigation software of choice) only gave you directions once per hour. Or if a lighthouse only shone its light once per night? Feedback is the core to knowing whether the direction you’re going is the right one.

MLOps and cultural continuous improvement

On a cultural basis, building a space where feedback can be heard, listened to, and acted upon boosts engagement, ownership and provides an opportunity for bottom-up agency and leadership. Demonstrating that team members will be listened to, and that concerns raised carry weight also helps build a greater level of psychological safety and a groundwork for common respect. However, the most valuable thing that comes directly from this, is the ability to put your product at the center of your engineering work.

Foster a feedback-driven and continuously improving culture to align product feedback seamlessly. Actively digest and thoroughly explore feedback to identify the optimal changes that best address user needs. Engineers, responsible for the software offering, play a key role in this exploration, striking a balance between maintainability and feasibility. This approach ensures the end solution is not only viable but also tailored for effective use.

Part of that assessment comes from the fact that such a culture allows for repeated and continuous testing with internal and external stakeholders – and so you see your “lighthouse beacon” before running into the sharp rocks of a misaligned software release.

Implementing such a culture, and gaining the benefits described requires that the processes in place are democratized and accessible. Some will be most comfortable giving and receiving written feedback, others will prefer verbal feedback via one-to-one or small group sessions.

For example, Betterworks provides a detailed set of guidelines which are a good starting point for trying to do such a thing, and offers some tooling suggestions as well as processes.

The power of Agile

However, another crucial aspect, often overlooked in discussions about continuous feedback and the earlier communication article, is the necessity of empowering everyone to contribute. This holds true from both technical and cultural perspectives. Overcoming this challenge can be difficult, and it often receives comparatively little attention when seeking resources online.

Sarah Mei, in her excellent talk, “The Power of Agile” points out that Agile ignores power dynamics, and assumes that as an unstated principle, everyone in a team feels as empowered as everyone else to contribute.

This is fundamentally often untrue, due to the many cross-sectional power imbalances that exist in society. While her focus in this talk is on the dynamics of pair programming, the key takeaways apply here as well. If there is one thing to take away from this think-piece, it’s this. Because if you or your colleagues aren’t in a position to be able to collaborate and contribute, you’ll only end up improving your technical and cultural techniques for those who are already in a position of privilege and power – exacerbating any imbalances which exist and causing disaffection.

Examples of continuous culture improvement

Creating a process focus group, who are flag-bearers for reviewing and removing stale and unneeded processes (like that meeting which everyone loathes, gets little value from, but still attends) and implementing new ones when appropriate.
Utilize process RFCs to articulate, review, discuss, and share proposals for changes, and then take action based on the outcomes.
Run an innovation lab. Create an innovation lab, a dedicated space where employees can experiment and explore new ideas. The innovation lab can serve as a playground for employees to test new technologies, products, and services. The innovation lab can foster a culture of experimentation and risk-taking, where employees can learn from their successes and failures, in a safer environment and continuously improve.
Hosting a hackathon. Running an event where teams collaborate to solve problems or create new products or services. Hackathons are an excellent way to encourage cross-functional collaboration and idea generation. Hackathons can lead to breakthrough innovations and help employees develop new skills and knowledge, they also help to remove silos from groups and spread a greater awareness of different perspectives across a team and the problems faced by each. This helps our collective technical understanding, and also builds contextual empathy which makes collaborative decision-making easier when faced with competing priorities.

But of course, at your core – you don’t want to be following some ruleset or process handbook. Look at the people who make up your team. Work and listen with them to see what they need, and what works for your collective situation.

Technological Continuous Improvement

In terms of implementing continuous improvement in a tech team, there’s little new to contribute after 20+ years of back and forth in the Agile community. But using MLOps is the pathway to building an agile ML model. It requires the entire end-to-end data and ML pipeline to be resilient, cloud-efficient, and accessible to end-users and maintainers as a reproducible software product. This is what MLOps is all about.

But here are some key takeaways that particularly pertain to the way we at Relative Insight want to work with our software.

Iterative is Inexpensive

The smaller an incremental change is to your codebase, the cheaper it is. It takes less time to plan for, less mental overhead to develop and is easier to review.

The fastest changes of all are ones that are automated away from a developer entirely. DevOps allows for things like automated dependency updates and security patches, and you can apply the same ethos of automation within MLOps to continuously test a new version of a model against your production instance – ensuring your best option is always in the hands of your users. This allows software engineering teams to focus on larger tasks without distraction while maintaining product excellence at a lower cost, both cognitively and financially.

Small changes mean shipped software

Related to the above, the faster a change is shipped, the sooner it will end up in the hands of a customer. Reducing the lag-time between a piece of software being written and being used “in anger” helps to ensure that there isn’t a drift between the needs which motivated the design of the software and the needs of the current customer. After all, everyone changes, and so do our expectations from a piece of technology. If you can’t get it out of the door quickly, then the greater the chances of it no longer being relevant by the time it actually goes out. 

Iterative is Impactful

If you have a system where these small changes are deployed as soon as they’re made, you can quickly see the impact of the improvement. This helps keep changes aligned with the product, and helps a developer to see the impact of the work they’re contributing to. When working fully automatically, you can combine the asynchronous automated communication described before to effectively measure your changes.

By following these steps, you’re basically listening to Dave Thomas’s advice for agile software development.

Find out where you are
Take a small step towards your goal
Adjust your understanding based on what you learned
Repeat

The majority of ML models never reach production. By using MLOps, you can build agile ML models and get them into the hands of the people who need them. Trying and experimenting and understanding how to fix real problems is far more impactful than being able to increase a model’s accuracy by 2% on a test dataset.

Small changes reduce tech debt

One of the tenets of the Agile manifesto is:

“Continuous attention to technical excellence and good design enhances agility.”

If you take a “little and often” approach to refactoring a codebase, you make the code more maintainable and in turn, this can help reduce burnout and frustration for those working on the code. Strive for a goal where the codebase appears as if written by a single person, despite having multiple contributors. Achieving this suggests the implementation of code-linting, reformatting, and a consistent framework, effectively minimizing cognitive load—automating routine tasks—and enabling developers to concentrate on building a feature-rich and robust service. This also mitigates the need for big and expensive rewriting efforts which have no impact on the end user’s experience directly.

For a highly specialized codebase like that of an ML system, the same rules apply – it makes it easier to onboard new faces to the team, and means that a data science team can focus on a field which is growing and changing faster than any other.

It is clear from the above that one facet of MLOps is a framework that allows you to combine the massive potential of AI/ML with the agile software development principles which are so prevalent today. It is also apparent that building an ML system as a “one and done” process will leave you behind those who are able to be responsive and proactive with their software offering.