Why Your Data Science Project is Likely to Fail

Dr. Gideon Rosenthal
June 26, 2019

This is the first in a series about ensuring the success of enterprise AI. See Post 2: Defining the Problem

Data science is the “new black” – with businesses across all industries looking for ways to leverage AI and machine learning to segment customers, predict events, automate processes – and improve the bottom line.

But is it effective?

Stated differently: Has data science succeeded in increasing productivity or helping businesses make better strategic decisions? Shockingly, the answer – at present – seems to be a resounding no.

According to Gartner analyst Nick Heudecker‏, as much as 85% of all data science projects fail. That means a mere 15% of projects achieve their goals.

Here at Data Science Group (DSG), we’re witness to the challenge – as, oftentimes, organizations approach us for help only after their independent efforts at integrating AI for enterprise have hit the proverbial brick wall.

10 Ways to Ensure the Success of Enterprise AI

Despite the fact that most data science projects fail, here at DSG our team has managed dozens of projects successfully in the last five years.

In fact, DSG enjoys a success rate of over 90%. So what’s the secret behind our success?

Based on our experiences, we’d like to share with you the following 10 steps that will help you hit the ground running and achieve your data science goals. But buyer beware: There are real pitfalls to avoid; taking shortcuts in any of these areas can put your project in peril.

  1. Define your problem with care: Set theoretical and operational business goals with measurable KPIs (key performance indicators) or KRIs (key risk indicators); then review and refine your goals.
  2. Choose the right proxy: More times than not, your actual KPI or KRI is not within reach. A proxy problem needs to be defined that has some relationship to the real need. Proxy definition requires in-depth knowledge of both the problem and possible algorithmic solutions.
  3. Define the stakeholder: Many times, there is misalignment between the person who sets the goal and the one who judges the outcome. Who ultimately decides if the project was a success?
  4. Make sure you have relevant data: Your data must be relevant to the KPIs or KRIs you defined. If you do not have access to relevant, high-quality data, the project is doomed before it begins.
  5. Check your data: Is there a chance your training set has been “polluted” by data leakage? Data leakages create wrongful generalizations and inferences that have an impact on AI models.
  6. Assess if the model is generalizable: Can your model be used with real-world data? Many techniques exist for cross-validation and measuring uncertainty, but high-level expertise and in-depth knowledge are essential in order to select the right techniques.
  7. Hire the best: Good data scientists are in high demand, and recruiting isn’t easy. In addition, many prefer not to be tied down to a single company; they would rather be in an environment where they have a diverse range of projects, learn from the best, and develop professionally.
  8. Work with a team: Different issues require different kinds of expertise. No single data scientist knows everything, and effective implementation requires teamwork.
  9. Prepare for production from Day 1: Maintain best practices throughout the research phase – to avoid a setback when you’re ready to move to production. Develop production-grade code and work with an integrated team of data scientists, ML engineers, software engineers, and DevOps.
  10. Monitor performance over time: Monitor the algorithms in the production environment and adapt AI models as necessary, so performance doesn’t degrade over time.

Careful Development & Implementation Gives You the Results You Need

Mitigating and solving challenges related to project development & implementation requires working with an experienced data science R&D team that can assist with both research and production.

At DSG, we’re here to help you avoid costly mistakes – saving you time and resources while maximizing your success in the data science journey. To learn more, contact us.

This is the first part in a series of posts about why data science projects fail. To receive the next post to your Inbox register here.