Matt Casters on Data Integration

Being part of the ICT BI community for a long time you tend to be aware of the problems that are still waiting to be solved in a lot of companies.

The article at the Register titled “Too many users fending for themselves on BI” again makes it painfully clear that we (the ICT BI guys) still have a long way to go before we finally bridge the gap between ICT and the Business User.

In the past, when I was still a well dressed BI consultant, I have many times been in situations where an ICT manager would hire me to save him from the “angry users mob“.  A lot of times it’s exactly as stated in the article: users are asking and fending for themselves and the ICT dept and at the same time the ICT department is doing it’s best to keep all this under control by denying as many requests as possible.
At this point, FWIW, I would also like to offer my view on a few the causes of this problem:

• ICT is not doing enough in terms of requirements analyses and not executing fast enough to keep these requirements relevant.
• After all these years, after sooooo many Kimball modelling success stories, no slowly changing dimensions and stars are being built.  There is always one hot-shot in every organisation that will claim they “need it to be 3rd normal form”.  In all (100% and at least 10) of the emergency “build it now or we’re dead” cases that I have experience with, there was no real data warehouse in place.
• BI Software in general is too expensive to acquire and maintain.  OK, I should have said “was too expensive to maintain” here :-)  As a result companies expect a lot, but it’s just software, not a solution.  You can’t buy a warehouse.  On top of that, most of the time, the user creating reports for himself only works for the very simple cases.  Self-service BI is for the most part a myth created by the established BI vendors to sell more software.
• Companies are not listening when you say (see the article!) that they need liasons people to communicate.  It reduces the gap, it’s good, it works, OK?  It doesn’t finish when a solution is in place.  Corporations change, people change, views change… BI requirements change and most certainly ICT changes very rapidly.  It only makes sense to do some coordination here!

I was glad to see that my views on the situation are reflected, but at the same time I was somewhat sadned by the fact that the more things change, the more they stay the same…  As the lead developer of an ETL tool, I have to acknowledge that not everything can be handled by technology.  This is probably the main cause of frustration for lots of ICT people out there in the field right now
Until next time!

Matt

3 Comments »