Skip to content

DataDelver

Exploring the labyrinth of Data Science, Machine Learning, and MLOps one delve at a time.

Delve 1: The (Hidden) Danger of Notebooks in Production

Banner

"Coming together is a beginning. Keeping together is progress. Working together is success." - Henry Ford

When Good Intentions go Awry

At many points in my career I have come across the topic of deploying code related to machine learning models in the form of Jupyter Notebooks. Often, the push towards this idea comes from a place of good intentions, of speeding up the the model deployment process or enabling better access to and understanding of the production environment by data scientists. However, despite the good intentions, this approach has in my experience created an environment of quite negative effect for the engineering teams asked to maintain these systems. In this delve, I will share my own personal experiences on working with notebooks in production systems, some of the ways I have observed them creating unnecessary friction between data scientists and ML engineers, and reflect how I think notebooks can be used as part of a healthy production system.

Delve 0: Hello Labyrinth (World)!

Banner

It seemed so daunting, "I need to make this model work!", "Running this in a notebook isn't good enough, we need to drive live site traffic against this!", "All of this data is bad!".

Entering the Labyrinth

Welcome to my blog data delver! I'm so glad you found your way here! If you're like me, when you first started out with data science and machine learning, you may have been feeling overwhelmed. With so many different concepts to learn it may have seemed as if there was an insurmountable labyrinth of information ahead of you, with no clear path towards mastery and practical application. Fear not! For you have found a resource which shall aid you in your own quest to navigate the maze.