Life is good.
Clients are feeling better, and business is getting back to normal.
This too shall pass.
But the problem with locking overrides is that they stay locked.
Days or weeks or longer go by and everything seems ok.
The vendor corrects the root cause of their problem and continues to operate as they were, and then there is an update.
The data the company has corrected and locked is about to change.
That new data is fed through correctly by the vendor, but it hits the override lock and never gets entered into the system.
So, once again, the same data are wrong – and now it is all on the enterprise.
The vendor provided the correct update, but the company’s overrides kept it from going though.
They’re back in the hot seat.
Managing Overrides Like the other steps in this evolution, this is also solvable.
Enterprises learn that they need to manage their overrides by making sure they are still valid and correcting them when they’re not.
“Aging” overrides and applying increasing pressure on vendors as the need for a particular override persists helps ensure that these corrections aren’t forgotten.
The first step of this aging process is to set context-appropriate flags on all overrides so that the locks are removed when the vendor has corrected the feed.
An enterprise managing tens, hundreds, or thousands of overrides cannot supervise each one manually, so database administrators are responsible for grasping the semantics of the data and writing a lock script appropriate to the context.
If the entry in question can only have one value and that value is relatively static (think address or employer or product identifier), the flag can be set to check for an identity between the incoming value and the correct value set by the override.
If they match, the lock is removed, and the feed continues normally.
If the value of the problematic entry is volatile, however, other logic must be put in place to determine whether the vendor feed issue has been resolved.
The reference check and reasonableness check, described below, are examples of how data stewards might monitor for a dynamic value.
(These are just a few common methods for setting flags; circumstances may require any number of others.
) Next, the enterprise’s data team needs to ensure that stakeholders stay informed of the issue ongoingly if it isn’t resolved right away.
Establish a grace period, and once it’s elapsed, have regular notifications sent to relevant departments and perhaps even the vendor itself.
This way, the issue won’t languish in obscurity and never get fixed.
Let’s imagine, however, that an override grows to be weeks or even months old when one day the flag conditions are met and the lock removed.
This turns off the override notifications, and its days before it surfaces that the value is wrong yet again.
After some investigation, the data team determines that the feed was never fixed in the first place, that the incoming value only met the flag criteria by chance.
Mistakes like these only cause extra work, so it’s important that admins place expiration dates on override flags’ automatic unlocking functions.
When an error has persisted for so long, it’s important for a person to confirm that it was indeed fixed before unlocking the override manually.
The older the override, the greater the chance that an incoming value will be to meet the flag criteria by happenstance.
The point is, data overrides are not a set-it-and-forget it mechanism.
They need to be managed, or they will cause more harm than good.
When overrides are developed, monitored, and reported on as part of a successful data quality program, it’s possible for enterprises to mitigate complications that arise from using vendor data, more effectively manage their data suppliers, and minimize undue cost to the company and its clients.
.