ML engineer role explained: Daily tasks & team collaboration

What does an ML engineer actually do on a product team?

Most job descriptions for ML engineers are written by people who have never worked alongside them and know little about their chores. They list the tech stack — Python, PyTorch, TensorFlow, etc. — and vague responsibilities like building and deploying ML models. But usually, there is no mention of what the person does during the workday, what they deliver besides the model itself, and how to understand whether the work is going well.

The lack of clarity is costly. Leaders approve budgets without fully understanding what work needs to be done around the ML model. They expect quick results without accounting for data, infrastructure, and integration challenges. The result is often disappointing.

According to MIT's Project NANDA, 95% of enterprise generative AI pilots didn't produce measurable returns. In many cases, the reason wasn't that the models were incapable but that organizations underestimated everything required to turn those models into working products.

So, what does an ML engineer do on a product team? This piece answers that across a sprint, across the model lifecycle, and across the day-to-day interactions with team members and business stakeholders that determine whether an AI initiative succeeds.

Key specifics of ML engineering work

At its core, the job of an ML engineer suggests taking a business problem and solving it through building a machine learning system that makes useful predictions or decisions from data. Yet, to build such a system is just half the battle. The other half is to keep it delivering value after going live.

The following aspects separate ML engineering from traditional software engineering.

The output is mostly probabilistic

When a backend engineer delivers a feature, it's usually clear whether it works or doesn't. Users can log in, access a page, or make a payment. If something is broken, it is often visible at once.

Machine learning systems don't work that way. When an ML engineer ships a model, a simple pass-or-fail outcome isn't applicable. A fraud detection model might identify 89% of fraudulent transactions. A recommendation engine might increase conversions by 7%. An AI assistant might answer 98 out of 100 questions correctly.

The question isn't whether the model works but whether it performs well enough to justify its cost and support business goals. That's why "good enough" is often as much a business decision as a technical one. That's also the reason why ML engineers communicate a lot with product and business stakeholders, discussing metrics, trade-offs, expected outcomes, and improvements.

Experiments proceed implementation

With requirements in place, a traditional software engineer may immediately proceed toward implementation.

ML engineers, in turn, need to analyze the available data, identify useful signals, formulate hypotheses, and run experiments before writing any production code. This way, teams define which ideas are viable and which aren't.

Progress at this stage is often invisible from the outside. Weeks of investigation and experimentation may result in a conclusion that none of the proposed approaches is likely to succeed. Yet that finding is also valuable because it prevents the team from investing months in the wrong strategy.

The system rots without ongoing maintenance

ML solutions work in an unstable environment where customer behavior evolves, markets shift, regulations change, and new data sources appear.

Without ongoing maintenance, a model trained on historical data gradually becomes less accurate. Therefore, after deployment, ML engineers monitor model performance, investigate unexpected behavior, and retrain models when necessary to make sure the system keeps on producing reliable results as the real world changes.

So, an ML engineer's job is to take an ML solution from theory to production, integrate it into the product, and ensure it stays accurate and helpful long after launch.

What does a machine learning engineer do day to day?

Let's go further and find out what ML engineers' hours go for. In general, there are five main activities, which aren't sequential phases and may run concurrently.

Data work (20–30% of time)

Across the industry, data professionals report spending about 45% of their time just getting data ready before modeling can begin. ML engineers aren't data scientists, but they also work a lot on data, exploring datasets, building and maintaining pipelines for training and evaluation, and spotting data problems that reduce model performance. ML engineers usually find the data issues nobody else noticed, often well before they show up as bad predictions in production.

Experimentation (25–35% of time, higher early)

ML engineers spent quite a lot of time figuring out what works best. They test different models and approaches, run experiments, and compare results, gradually defining the most viable option.

All results — both positive and negative — are thoroughly documented so that the team does not return to dead ends later. As new findings become available, ML engineers modify feature engineering strategies or model architectures.

Though experimentation produces no shippable code, it yields a wealth of valuable knowledge. This makes it one of the core machine learning engineer responsibilities.

Engineering and integration (20–30% of time, higher close to deployment)

Once the model is ready, ML engineers put it into a system. They turn prototype code into production-ready software, expose the model through services or APIs, and make sure it can work flawlessly under real-world conditions.

To ensure the model works well as part of the broader system, ML engineers collaborate a lot with BE and FE developers when integrating the model to data sources, APIs, business logic, and UIs.

Monitoring and maintenance (10–20% of time)

After the model is deployed, monitoring and maintenance are the key activities. Using dashboards and alerts, ML engineers monitor production metrics to spot the first signs of performance deterioration.

All the detected anomalies and data drifts are carefully investigated, and models get retrained when new data becomes available or business conditions change.

Thanks to this ongoing work, the model is kept accurate and reliable.

Cross-functional collaboration (15–25% of time, steady)

Throughout the project, ML engineers work with a wide range of stakeholders. Apart from the above-mentioned BE and FE developers, they communicate a lot with PMs, business stakeholders, data scientists, and leadership.

Also, an ML engineer in a product team acts as a translator between the business and the technology. They translate business goals into model objectives as well as translate model behavior, limitations, risks, and trade-offs back into the language that non-techy people on the team can understand.

The mix of these activities changes as the project evolves.

At the beginning, most of the time goes into understanding the data, testing ideas, and defining whether the problem can be solved with machine learning. After a viable approach is found, engineering and integration take up a larger share of the work. After deployment, focus is on monitoring and maintenance.

The table below shows how an ML engineer's time is distributed at different stages of a project. Yet, the represented numbers shouldn't be treated as a precise formula.

Please note that the table has been prepared based on the assumption that the ML engineer team structure includes all the supporting roles.

Take away a data engineer, and the time spent on data work can climb above 50%. Seemingly, take away MLOps support, and ML engineers will spend much more time on deployment, infrastructure, and monitoring.

That's why different ML engineers can have very different duties and workloads depending on the team structure.

ML engineer role in a software product team

An ML engineer never works alone. They sit in a web of dependencies with other roles, and understanding where the ML engineer's work begins and ends impacts whether the whole team functions efficiently.

Below are four relationships that matter most.

ML engineer ↔ data scientist

In some companies, the two roles are interchangeable, but in well-structured teams, the distinction between ML engineer vs. data scientist is clear.

A data scientist is closer to a researcher. They explore data, test hypotheses, and experiment with different modeling approaches to understand what might work well. Their output is a prototype or a set of insights.

An ML engineer takes these insights and the most promising approach and makes it run reliably in production.

So, data scientists explore and prototype, while ML engineers productionize, scale, and maintain.

Yet, in small teams, one person often takes on both responsibilities, though the skill sets are different.

ML engineer ↔ data engineer

Data engineers create the foundation ML engineers depend on: pipelines, warehouses, and data quality layers.

A common failure pattern shows up when there is no data engineer, and the ML engineer spends most of their time building ETL from scratch. As a result, model work becomes secondary. In fact, that's paying a specialist rate for plumbing.

If your ML engineer hand-writes ingestion pipelines, the team structure is off, and you either need a data engineer or you need to narrow the scope.

ML engineer ↔ backend/software engineer

At the point where a model becomes a product feature, ML engineers closely collaborate with software engineers. Their co-working is essential because the former understand how the model behaves, and the latter understand the production system.

The handoff is usually a trained model and inference logic, often as an API or service, on one side and integration into the product architecture on the other.

When that handoff isn't structured, the model works in isolation and falls over under real latency and load. That's why most production issues in ML systems aren't model problems but integration issues.

ML engineer ↔ product manager

This is one of the most important relationships and also one that is often left to chance.

The PM defines the business goal, and the ML engineer turns it into an optimization target and a measurable definition of success.

Things often go wrong when the goal sounds clear but isn't actually specific enough. For example, "better recommendations" isn't a usable objective on its own. There is still a need to define what "better" means in a particular situation: more clicks, higher conversion, longer sessions, fewer irrelevant results, or something else.

Let's take a spam classifier. First, a team has to decide what is more important: catching all spam (recall) or avoiding false positives (precision). This is a product decision with technical consequences.

When this conversation doesn't happen early between PMs and ML engineers, there's a real risk of optimizing the wrong thing.

Deliverables an ML engineer brings and how to evaluate them

Want to know how to work with ML engineers and evaluate their performance? Start by finding out their exact output and how it should be measured. Spoiler: traditional software metrics like the number of pull requests, story points, or deployment frequency are misleading in ML work. They can undervalue really important work and miss really poor work.

Here's what actually comes out of the role and how to assess each artifact.

One thing that becomes clear from the table is that a model is truly done when it meets an agreed performance threshold, runs in production, is actively monitored, and has a plan for retraining as data and business conditions evolve.

Treating a high-accuracy notebook model as complete work is a common evaluation mistake.

Importance of the experimentation phase, even though nothing is shipped

The experimentation phase is often the most uncomfortable part of an ML project for managers and stakeholders.

Weeks can pass without a visible output: no new features, no production deployment, nothing that can be shown as progress on a roadmap. From the outside, it may seem like stagnation.

In reality, a great deal of work is happening:

• An ML engineer is learning what the data can and can't support.

• Failed experiments are eliminating approaches that would have failed later in production for far more money.

• The results are serving as the evidence base for every following architecture decision.

Yet, there are two ways teams get this wrong.

Some teams rush through experimentation and move to deployment after minimal testing, facing major flaws in production. Others keep on testing endless approaches without ever arriving at a decision.

The best approach is somewhere in the middle. The team identifies 3–5 most promising options, evaluates them against representative data, and agrees upfront on the criteria that will determine success or failure.

Here is how to manage the experimentation phase without micromanaging:

1. Time-box it. For instance, give a team four weeks to test three approaches and make a go/no-go call against pre-agreed criteria.

2. Ask for experiment logs, not demos. The key deliverable is documentation of what was tested and what was learned. Demos are secondary at this stage.

3. Define "good enough" before experimentation starts. Without a shared definition of success, teams can spend months improving a model without getting closer to a business decision.

The temptation to shrink the experimentation phase is understandable, especially when deadlines are tight. But it often backfires. By saving a few weeks here, you will later pay much more for the failure in production.

Production reality — What happens after the model deployment

After model deployment, the work of another kind starts. Now, all efforts are put into ensuring the models continue to perform well in an ever-changing environment.

In practice, production ML comes down to three challenges:

• Data drift

User behavior, product features, market conditions, and external factors all change over time in ways that are hard to predict. Therefore, a model trained on last year's data slowly becomes less reliable just because the world is no longer the same.

After deployment, one of the ML engineer's duties is to monitor for this drift, understand what has changed, and decide if the model needs to be retrained or adjusted.

• Model versioning and rollback

A newly trained model can work worse than the one it replaced. When that happens, there should be a way to roll back quickly to a previous version. This is possible thanks to model versioning that lets teams safely and easily switch between model versions when needed.

• The feedback loop

The best ML systems improve by learning from their own predictions. For this purpose, a feedback loop needs to be introduced. This mechanism tracks whether predictions were correct and feeds that signal back into future training cycles.

This is rarely automatic, and the ML engineer has to design what gets collected, how it's stored, and how it gets usable for model improvement over time.

One of the most common mistakes is treating deployment as the end of the ML work. If no one is responsible for model maintenance in production, it quietly degrades, and you learn about this only when users are already affected.

That's why production ML should always include ongoing engineering capacity. If you consider staff augmentation for AI projects don't end engagement right after deployment unless you have someone on the team to take care of the model.

Effective vs ineffective ML engineers

By some estimates, 80% of AI projects fail. Reasons vary, but in this setup, it's better to have really skilled ML engineers who can mitigate risks.

To understand whether an ML engineer is capable enough, you need to know how they work with ambiguity, communicate decisions, and handle production reality.

Signs of a skilled ML engineer:

• Pushes back on vague requirements and clarifies what success means before starting experimentation

• Documents both failed and successful experiments

• Reports results in business terms, not just F1 scores

• Flags data-quality problems early instead of quietly working on them

• Involves a PM in trade-off discussions rather than making decisions alone

Signs of a weak ML engineer:

• Provides impressive notebook results with no clear path to production

• Optimizes for technical elegance over the business objective

• Cannot explain the failure modes or limits of the model in plain language

• Treats deployment as someone else's duty

• Discovers data quality issues late, although they were visible earlier in the dataset

Top ML engineers have strong technical skills as well as product thinking and communication discipline to work effectively in a cross-functional team.

Building a good model isn't enough. Engineers should also be able to explain what it does, how it does it, and why they'd recommend its implementation. Candidates who can't do this may still be strong researchers, but they often struggle in product-focused ML environments.

Conclusion

So, what does an ML engineer do? Data handling, experimentation, engineering, monitoring, and collaboration are all within their responsibilities. The balance between these activities changes depending on the stage of the project, but none of them ever fully disappear.

The output of ML engineers includes not only the model itself but also experiment logs, evaluation reports, a product system, and monitoring setups that ensure accurate work over time.

Once the role specifics are clear, common misunderstandings around ML teams disappear, from unrealistic timelines to unclear ownership and misaligned expectations.

Devico works with product companies by providing ML engineers with production deployment experience. Beyond that, we also help teams structure roles and interactions so that models reach production.

If you're building AI/ML capability into your product, we're happy to help with both hands-on resources and guidance on team structure.