Since the silos in your organization are always likely to grow, consider a per-connector pricing model as a negative as well.
Generic Vendor Everyone has at least one or two niche systems that are not covered by the usual crop of connectors and probably will be low on the priority list for a vendor.
Ensure the products you are assessing have generic connector options so you can build your own.
Security and Control If you are dealing with sensitive data and need full control of the data processing infrastructure and its security architecture, a SaaS vendor might not be for you.
You can still take advantage of a cloud data warehouse.
Pricing It’s tough to estimate just how much data you have or will have in the future, and this shouldn’t be a limiting factor.
Take a careful look at vendors who use esoteric utility metrics.
Once you have centralized the data, it must be cataloged in its raw form so authorized users can access and find what they need with ease.
This is also the best time to lock away any of those data sets that are too hot to handle or anonymize data sets.
While this solves the problem of data siloed across source systems, it doesn’t address the problem of format-siloed data.
In other words, data that cannot be brought together because of the way it is collected and stored.
A process needs to take place to transform this data into a common format.
Step #2: Transform data to get the most meaningful insights for your business — and, make sure everyone can do this Once the data has been dropped into a cloud data warehouse, data lake or whatever the most en vogue term for a big ol’ data store is, most data pipeline vendors just leave you to it.
And by ‘leave you to it’ I mean they rely on one of your technical departments to build some big, slow, difficult to maintain SQL or other pipeline routines that turn the data into something useful.
This approach does NOT scale with your ambitions to be a data-driven organization.
It’s critically important to make sure the data and the ability to work with it is accessible across an organization.
Sure, some data might be used for critical management reports that the business relies on powered by centrally managed ETL’s, but far more of it is most useful for everyday tactical decisions.
The ability to transform that data should be a skill that is available right across an organization.
Every department should have access to the best visualization and business intelligence made possible by a simple yet powerful data transformation capability.
ETL tools have traditionally been the preserve of the hard-core data professional, but this is changing faster than ever.
A new wave of data transformation technology is fast enough and simple enough to be used by a wide range of business professionals – from software engineers to marketing analysts alike.
Conclusions On balance, the systems that lead to having many data silos are a good thing; they indicate a business has the autonomy to choose the best systems in each department.
This should make the business more efficient overall.
However, the business needs data from all these systems.
If there is one thing that EVERYONE in an organization who can purchase a SaaS system (no matter how small or cheap) needs to know it’s that the data collected by those systems must be available; any system that hoards its own data in the cloud is a non-starter.
Dealing with those data silos is not nearly as painful as you might think.
There are a large number of extract and load tools that can centralize your data.
Forget these as they only solve half of your data silo problems.
To overcome the more painful problem of data transformation look to the cloud-ISVs, the next Salesforces, that are developing and evolving in the name of customer obsession.
Sign up for the free insideBIGDATA newsletter.
.