Tag Archives: Salesforce.com

Data decay: The WHY behind why your CRM data sucks.


  • “Wrong phone number”
  • “The company was acquired”
  • “Wrong title”
  • “I have duplicates”
  • “The data is old”
  • “The emails don’t work”
  • “My CRM data sucks”
  • “I just called someone who’s been dead for a year”

If you’ve heard any of the above comments or something similar; you have a CRM problem.  Are their solutions to these problems?  Yes, however; today I will give you a deeper understanding of WHY the problem occurs.  Many good vendors exist to solve the problems listed above.   I want to arm you with a deeper insight, the WHY.

If you understand the WHY, you will be able to:

  • Have a deeper understanding to the nature of the problem
  • Remove unrealistic expectations (solve the problem, don’t chase a rainbow)
  • Define best practices to minimize bad data
  • Be informed when choosing a vendor (flashy interface does not solve THE PROBLEM)
  • Understand how your CRM decisions effect CRM data
  • Help you be an advocate for change management within your organization
  • Make you a more informed client (some vendors will like this, others will not)

So what is the WHY?

Short answer:

Contact data decays

If you have a short attention span, if you are brilliant, or have limited reading time, we are done here.  That’s all you need and you know what I am going to say in the long answer.  Thanks for reading.

Long answer:

First, let’s establish a baseline from the US Department of Labor.

Employee attrition - base dec

The national average tenure across all jobs in the US is 54 months.  That breaks down to 1.85% per month of job attrition.  For high-demand IT workers the tenure is shorter with 3% monthly job attrition rate.  The rate for Silicon Valley start-ups is almost ridiculous with the average tenure being just over a year.*

*Not from DOL. garnered from several Venture Capital blogs…take it as an extreme example.

A full year of data decay – base factors

A month at a glance does not show the full picture when factored across an entire year.  Look at the picture across a year’s time.

Employee attrition - base decay by year
When reviewing an entire year, data will decay at about 12%.  However, that does not take into account many additional factors including:

  • Change in title, promotion
  • Change in working location
  • Change of phone number
  • Add mobile phone number
  • Change of department
  • Change of area code
  • Change of email format
  • Change in email domain
  • Change in company name
  • Change in company website
  • Merger or acquisition

While I did my best to outline all the factors, there are some that I probably missed.  What I wanted to show is a picture of the WHY data decays.  For the past 10 years, I’ve been immersed in data, the acquisition of data, and the analysis of data.  From this experience, I know that data decays somewhere in the vicinity of 2.8% to 5.5% per month.  This is based on:  (1) Hand verified data-sets that I’ve been building and maintaining for 4 years (2) Poll-based responses from 5 years of live training webinars and (3) General industry sentiment.   When adding in all factors, the picture changes significantly.

 A full year of data decay – all factors

Employee attrition - decay across all factors

Don’t make the “additive mistake” 5.5% decay per month does not equal 66% over 12 months (it’s actually 36%).  You must add each month and it’s data age and then factor in the average data loss across a year.  It’s still not a pretty picture.  In 2011-2012, the numbers are sitting at the high end of the continuum, somewhere in the 5 – 5.5% range. This means that you can expect data decay to be sitting somewhere around 36% a year.

The implications for CRM data health

So what is the health status of your CRM data?  Using the Estimator, find out where you stand.  Start by finding your CRM Data Update Cycle.  This is how often you go through a CRM full data update.  Next, based on what you know about the sector you sell into, pick a Monthly CRM Data Decay Rate.  As a guideline, in 2011-2012 the average is 4.5-5.5%.

CRM data health estimator

Based on the 2011-2012 numbers, to stay under the 10% threshold, the CRM should be refreshed every 60-90 days.

“Hold on Donato”, I get this statement often, “I’m updating contact information in my CRM all year long”.   My response is a question: What percent of all accounts in your CRM do you connect with each year and update their contact info?  Typically the response is somewhere in the 15-20% range.  Again, you must apply the data decay principal to data updated over a year time.  For example, if you connect with 1200 accounts per year, 100 per month, then at the end of the year, the first month’s data is 11 months old,  or somewhere in the neighborhood of 33% outdated.   If you are doing a fully updating contact information (name, title, email, phone, company URL company phone, etc) , each time you connect with an account, then you can give yourself a 5% data health increase.

This begs a question: Who should be in charge of updating CRM contact data?  If you expect sales to do it, I have bridge to sell you in New York.  My answer is Marketing or IT, based on how your organization is structured.

Where does this leave us?  At this point you should have a solid understanding of the WHY behind data decay.  In addition, you should have a more accurate understanding of your CRM Data Health.  Lastly, you should want to take action.  In this blog, I want to leave it as pure educational vs recommend vendors.  Vendors come and go, the percents of data decay may fluctuate, but the concepts should hold true for some time.

Questions to ask prospective vendors:


There are many products out there that can solve CRM data issues.  Solutions include data companies, deduplication vendors, offline data cleanse services, preventative data-technology and data import technologies.  Here are a set of questions to prepare yourself with.

  • How old is your data?  Just because you are buying data, does not mean that is it fresh.  Data vendors suffer from the same data decay as everyone else.   If they guarantee it updated every 6 months, that means it’s somewhere between 15 and 20% outdated the day you get it.
  • What is the source of your data?  If you’ve experience crowd sourced data, you’ve felt the pain.  Make sure the source of the data is consistent with what you need for your sales & marketing model.
  • Data standards: how will you deliver the data?  As an example, there are over 20 ways to write “The Container Company Inc”.  Which way will the company names, URL’s, names, titles, emails and phone numbers be delivered?  Will it match the data standards that you have?  I am shocked that you will still see data coming from large data providers in ALL CAPS.
  • Transparency:  Can I see the source of the data?  Not the same as where it came from.  True data transparency gives you the source of each piece of data.
  • Record by record data age?  Does each record show when it was last updated.  This is a tremendous confidence booster for sales.  Ask yourself: why are sales reps excited by new leads?  Answer: they are fresh.
  • How to prevent future duplicates?  If you don’t stop new duplicates from entering your system, you will have to keep doing it over and over again.
  • How to handle historic duplicates?  Stopping future duplicates is important, but full CRM data health means cleaning up the historic data.
  • Can you consult and help me with data standards?  If they can’t guide you here, for free, something is wrong.  This is the absolute first thing that needs to happen.  Pick a data plan.
  • How to enforce data standards?  Technology can enforce data standards so people don’t type in the wrong stuff.  How do they solve this?
  • How to import lists AND conform them to my data standards?  Ok, you just cleaned your CRM and now you have a list from a Data vendor in ALL CAPS…how will this be handled?
  • How to prevent human data-entry errors?  If you give them the opportunity, people will type anything into the CRM.  What is the solution?
  • Can you help with change management?  Changing the way a company works with data takes experience.  Are they thought leaders or product peddlers?  Can you show me the bumps in the road?  What is the next big thing?  What should I be planning for?

I hope you now have a solid understanding of data decay and it implications.  Understanding the issue is the first step towards instituting change.  If you get one of the vendors on the horn, be nice, you already have answers to questions that they are not asking.  They may also be excited to have a well-informed prospect as a customer.


Donato Diorio is a writer, blogger, speaker and the Founder & CEO of Broadlook Technologies, a company that specializes in CRM data health.  He can be reached at ddiorio at broadlook dot com.


CRM’s are not built to keep your data clean

Why can’t my CRM vendor keep my data clean?

Nearly every CRM company and internal corporate IT department has taken a stab at solving the problem of keeping your data clean. You may have heard of it referred to as CRM Deduping, CRM Dedupe, CRM Deduplication as well as Data normalization.

Unfortunately, no one has done it right! Not Oracle, not Salesforce.com, not Microsoft and not SAP. Why?

Think about it: Who decides the standard? Do you want saleforce.com or Microsoft dicatating the format for data storage that you use? Of coarse not, every business model has unique needs. While one business may prefer storing “The Container Company” another may prefer “Container Company, The”. It would be suicide for a vendor to enforce a single set of rules on all their clients. So the problem is not necessarily ignored, but it is accepted as status quo.

Why didn’t my CRM sales rep tell me this?

Since the issue is accepted as status quo for the industry, it is simply not addressed. Sales reps are not in the habit of pointing out a problem that they do not have a solution for. If the issue is addressed, typically it is periodical crm deduplication process.

CRM Dedupe only treats the symptom

If you think of CRM duplicates as the symptom of a disease, how do you cure the disease vs. treating the symptom? The answer is to never allow duplicates into your CRM system in the first place. The first start in that process is developing a CRM Data Plan.

CRM Deduplication services are a profit center

Your dirty data is another company’s profit center. Many of the major CRM providers have professional services that you can pay to periodically clean your CRM. While these services typically do a great job, they do not solve the issues with inefficiency and lost revenue *between* the data cleansing cycles. Basically, every time you fix your CRM, the cure is short-lived.

Why didn’t I see this coming?

Don’t be hard on yourself. Think about it. When you buy a CRM, it is usually empty. If you import dirty data from an old CRM, the new CRM will be dirty. The problem is not the result of you forgetting to turn on a feature in the new CRM. Massive duplicates and miskeyed data does not become a problem until after you start to use the CRM. When a CRM is implemented, it is typically at that time the buyer realizes there is a huge problem with the information.


Without a systematic way to start with and keep information clean, duplicates will be introduced into your CRM. Good for professional service firms; Bad for you and your CRM. The best solution is not treating a symptom, but curing the disease. The first step to curing the disease is creating a CRM Data Plan… which will be the next blog post.

Next post:What is a CRM Data Plan