Print
Discuss
Send to friend
RSS feeds
IT Week: SeeWhy has been very vocal about the need for a new approach to business intelligence, or BI 2.0 as you call it. Why is this?
Charles Nicholls: Businesses want to build BI reporting into operational processes to help them optimise those processes, but traditional BI is just not very well suited to this environment. Operational processes are tactical rather than strategic – they are dealing with issues such as how do I know this shipment will arrive on time? How do I build BI information into the process to make changes as required? The traditional data warehouse approach of extract, transform and load the data, then analyse it and report on it, provides great data for making long-term strategic decisions, but it doesn’t help you get the data faster so it is not suited to supporting these operational processes where you need the data quickly.
So, you think there is too much latency in getting results from traditional BI systems?
Yes. But on top of the data latency issue a lot of the tactical data you require isn’t even in the data warehouse. It will include aggregated data, like a store’s sales, but it often won’t have the granular operational data, such as information on each individual sale. If you want this granular data in the data warehouse it increases exponentially the volume of data and creates major load-time issues. As a result, very few firms get intra-day sales figures and are often left trying to make operational decisions based on end-of-day sales figures at best.
How does this affect corporate performance?
If you take the retail example, most stores will have a 2pm order cut-off point to order stock for the next day. If your reports are done overnight and you get a peak in demand in the morning, you won’t find out until the next day. That means you won’t get more stock in until the day after that and you’ll be out of stock for a day. A recent report from AMR Research found that the average retailer currently takes five weeks to detect a change in demand and a further five days to do something about it. That’s because the way replenishment is managed today is often through grouping similar stores together, analysing their aggregated data, reporting on it and then making decisions.
How does SeeWhy believe firms should tackle this problem?
What we aim to do is measure demand at much lower levels of granularity for every point of sale. That allows you to drive much smarter processes and increase sales. We can detect whether something is selling faster than usual and integrate with the ordering system to change the processes to adapt to the increased demand.
How can this level of granular reporting be achieved?
We have a very different approach. We have concluded that trying to make extract, transform, load, query and report go faster isn’t going to work – there are so many stages that the inefficiency is huge. In that process you have to undertake at least three queries before the report reaches the end-user and it is these queries that take all the time and computing power. We take more of a service-oriented architecture (SOA) approach. Traditional BI takes a database-oriented architecture approach but firms don’t build systems using database-oriented architectures anymore. SOA means that your reporting tools’ integration is not with one database but ideally with an enterprise service bus (ESB) or middleware layer that is managing all the data flowing around. We tap into the ESB and find the events, the anomalies, in the data flow.
What is the practical benefit of this architectural approach?
In the new world of BI 2.0, as we have coined it, it is not a case of looking at a report and finding the issues. Our approach automatically finds the event and that can be used to automatically trigger the appropriate change in operational processes. We can check everything as it goes through the ESB – as opposed to a data warehouse approach where you start at the top and drill down looking for the level of granularity that the analyst simply doesn’t detect unless they get a tip-off. For example, they’ll be able to tell one week had a fall-off in sales of a certain product, but they won’t have the granularity to work out it was due to stock shortfalls unless the store rings up and says, “We’ve run out of this”. Our approach will automatically find events that are out of the ordinary against historical data or future targets, such as peaks in demand, as they happen.
And can this only work in an SOA environment?
No. The software can act on any stream of events, and that can be found in web traffic, ESBs, middleware, or be pulled out of a traditional data warehouse if required. That stream then goes into an in-memory calculation engine that allows you to do the analysis fast, rather than through a series of relational database queries, as it is those that slow everything down.
Can you give an example of how this event-based approach works?
We worked with one customer recently to detect when individual customers were having a bad day and not ordering as much as they normally would. This could be because they were trying out a rival, so the company was able to trigger a promotional offer or a sales call to entice them back before they even switched. We also deployed the system across [the drinks company] Diageo’s supply chain. The company had a problem in that 98 percent of shipments of Guinness left its facilities on time, but 50 percent arrived late. By tracking every shipment and looking for events we could find out when something was going to be late and work out where the problems were. The shipping department would know on the third day of the month if it was not going to hit delivery targets and take action, whereas in the past it only knew it had a bad month after it was com pleted. This level of insight really changes the way you work.
How tightly can you integrate these event-based reports into the operational process?
It varies based on the process and the customer. We can either offer manual alerts, as we did with Diageo’s shipping department, or integrate into a BPM tool to initiate automated corrective action as soon as an issue arises. For example, if a customer is not spending it can trigger an anti-churn email to them or drop their details into the task list of an outbound call-centre agent.
What if organisations still want to get at the strategic, longer term information delivered by traditional BI systems?
We put the data we ascertain from the event streams into a relational database for back-up purposes, but that also allows firms to analyse it further, if they require, using traditional BI tools. We see this as offering a good fit with the traditional data warehouse environment. The functionality we are offering simply can’t be done with a data warehouse, but if you are dealing with strategic issues, such as complying with regulations, then a data warehouse is still valuable. We see this complementing data warehouses – it is not a case of asking firms to rip out their existing BI infrastructure.
del.icio.us
Digg this
reddit!
