Mini-Post -- IBM IOD -- Consider Informix For Time

One more IOD mini-post, and then I'm out of time (pun intended).

It was nice to see the Informix folks again, and for all their past database innovation I owe them some mention.  They introduced object-relational, among other things, plus "blades" to provide specific performance speed-up for particular querying tasks, and they even (although I never saw it implemented) proposed a relational feature that eliminated the sort step.

In today's IBM, their impact is necessarily muffled, although they still have the same virtues they always had:  An indirect channel untouchable by Oracle that services the prevalent medium-sized businesses of Germany and the new SMBs of China, among others; mature object-relational and relational technologies with scalability in certain query types, and with administrative simplicity relative to the Oracles of the world; and mastery of time-series data.  As I understand it, their near-term plans involve attempting to expand these strengths, e.g., by providing "edge" Web streaming-data processing for SMBs.

From my viewpoint, fairly or unfairly, the place the general user public ought to pay attention to Informix is in handling time-series data.  In recent years (again, this is my imperfect understanding) Informix has expanded its intertemporal "blade" technology to provide compression of (and hence very rapid querying of) time-series data via a process very similar to that of storage companies' dedupe "windows" -- taking a certain time frame, noting that only changes need to be stored given and original value, and hence compressing the time series data by (in storage's case, and probably in Informix's) 70-90%. I should also note that Informix can take advantage of IBM DB2 BLU Acceleration to further compress and speed querying of 1-to-2-column-per-query time-series data -- it's an earlier version of BLU, apparently, but one tuned more, and Informix provides a simple "fork" when it's time to go columnar.

How would I suggest any enterprise -- not just present Informix customers -- could use this time capability?  Well, let's face it, more and more data from the Web has part of its value as assessing behavior over time -- social network analysis, buying patterns, sensor data, the buying process. For this type of analytics, I would argue that neither the streams processor up front (can't store enough historical data) nor the data warehouse (not optimized for time-series data) is the best fit.  Rather, I would suggest a "mart" running Informix parallel to the main data warehouse, storing only time-series data, with a common front-end for querying -- somewhat akin to what I have seen called an "operational data store", or ODS (I tend to use the term ODS for something quite different, but that's a whole other conversation).

This would be of value not merely because it provides better performance on queries on time-series data.  Let's face it, time-series data has up to now not been considered worthy of separate consideration by today's enterprises.  And yet, our understanding of the customer should be far more enriched by understanding of customer processes and changing customers than it has been.  Creating such an Informix ODS would at least start the IT-business-exec conversation.

Just a thought ... and hopefully a timely one :)  Ttfn.

Mini-Post -- IBM IOD -- The Customer-Driven Big Data Enterprise

I have been challenged by several people as follows:  If you think that users are not spending as much as they should on Big Data because they don't see it as a process, but rather as a series of one-shot "big value-add insights", what then is the process they should create?  I don't pretend to have all the answers, but here, off the cuff, are my thoughts.

My first reflex in answering these questions is to recommend something "agile" (my definition, not the usual marketing hype).  However, in today's un-agile enterprise, and particularly dealing with un-agile senior executives, that won't work.  Btw, there's a wonderful phrase for this kind of problem, that I credit to Jim Ewel of Agile Manifesto fame:  HIPPO.  It stands for the HIghest-Paid Person in the rOom, and it refers to the tendency of decision-making to be made according to the market beliefs of the HIPPO rather than customer data. Still, I believe something can be salvaged from agile marketing practices in answering the question -- the idea of being customer-data-driven.

Next, I assert that the process should have a Big Data information architecture aimed at supporting it. If we are to use Big Data in gaining customer insights, then our architecture should allow access to and support integration of the three types of Big Data sources:  Loosely, (1) sensor-driven (real-time streams of data from Web sensors such as GPS tracking and smartphone video), (2) social-media (the usual Facebook/Pinterest sources of customer interest/interaction unstructured data), and (3) the traditional inhaled in-house data that tends to show up in the data warehouse or data marts.

The process itself would be one of iterative deeper understanding of the customer, equal parts understanding the customer as he/she is now (buying procedures/patterns plus how to chunk the present and potential parts of the market) and where he/she is going (changes in buying behavior, new technologies/customer interests, evolution of present changes via predictive analytics carefully applied -- because agile marketing tells us there's a danger in uncritical over-application of predictive analytics).  The process would be one of rapid iteration of customer-focused, Big-Data using insights, typically by data scientists, often feeding the CMO and marketing first, as befits the increased importance in today's large enterprise of the CMO.

What I suggest you have in this process is a Big-Data-focused, customer-insight-driven enterprise-driving analytical process.  Or, for short, the "customer-driven enterprise."  As in, Big Data for the customer-driven enterprise.