Does Data Virtualization Cause a Performance Hit? Maybe The Opposite

In the interest of “truthiness”, consultants at Composite Software’s Data Virtualization Day last Wednesday said that the one likely tradeoff for all the virtues of a data virtualization (DV) server was a decrease in performance.  It was inevitable, they said, because DV inserts an extra layer of software between a SQL query and the engine performing that query, and that extra layer’s tasks necessarily increases response time. And, after all, these are consultants who have seen the effects of data-virtualization implementation in the real world. Moreover, it has always been one of the tricks of my analyst trade to realize that emulation (in many ways, a software technique similar to DV) must inevitably involve a performance hit – due to the additional layer of software.

And yet, I would assert that three factors difficult to discern in the heat of implementation may make data virtualization actually better-performing than the system as it was pre-implementation.  These are:

·         Querying optimization built into the data virtualization server

·         The increasingly prevalent option of cross-data-source queries

·         Data virtualization’s ability to coordinate across multiple instances of the same database.

Let’s take these one at a time.

DV Per-Database Optimization

I remember that shortly after IBM released their DV product in around 2005, they did a study in which they asked a bunch of their expert programmers to write a set of queries against an instance of IBM DB2, iirc, and then compared their product’s performance against these programmers’.  Astonishingly, their product won – and yet, seemingly, the deck was entirely stacked against it.  This was new programmer code optimized for the latest release of DB2, based on in-depth experience, and the DV product had that extra layer of software. What happened?

According to IBM, it was simply the fact that no single programmer could put together the kind of sophisticated optimization that was in the DV product. Among other things, this DV optimization considered not only the needs of the individual querying program, but also its context as part of a set of programs accessing the same database.  Now consider that in the typical implementation, the deck is not as stacked against DV: the programs being superseded may have been optimized for a previous release and never adequately upgraded, or the programmers who wrote them or kept them current with the latest release may have been inexperienced.  All in all, there is a significant chance (I wouldn't be surprised if it was better than 50%) that DV will perform better than the status quo for existing single-database-using apps “out of the box.”

Moreover, that chance increases steadily over time – and so an apparent performance hit on initial DV implementation will inevitably turn into a performance advantage 1-2 years down the line. Not only does the percentage of “older” SQL-involving code increase over time, but the need for database upgrade (as, for example, upgrading DB2 every 2 years should pay off in spades, according to my recent analyses), means that these DV performance advantages widen – and, if emulation is any guide, the performance cost from an extra layer never gets worse than 10-20%.

The Cross-Data-Source Querying Option

Suppose you had to merge two data warehouses or customer-facing apps as part of a takeover or merger.  If you used DV to do so, you might see (as in the previous section) the initial queries to either app or data warehouse be slower.  However, it seems to me that’s not the appropriate comparison.  You have to merge the two somehow.  The alternative is to physically merge the data stores and maybe the databases accessing those data stores.  If so, the comparison is with a merged data store for which neither set of querying code is optimized, and a database for which one of the two sets of querying code, at the least, is not optimized. In that case, DV should have an actual performance advantage, since it provides an ability to tap into the optimizations of both databases instead of sub-optimizing one or both.

And we haven’t even considered the physical effort and time of merging two data stores and two databases (very possibly, including the operational databases, many more than that).  DV has always sold itself on its major advantages in rapid implementation of merging – and has constantly proved its case.  It is no exaggeration to say that a year saved in merger time is a year of database performance improvement gained.

Again, as noted above, this is not obvious in the first DV implementation.  However, for those who care to look, it is definitely a real performance advantage a year down the line.

But the key point about this performance advantage of DV solutions is that this type of coordination of multiple databases/data stores instead of combining them into one or even feeding copies into one central data warehouse is becoming a major use case and strategic direction in large-enterprise shops. It was clear from DV Day that major IT shops have finally accepted that not all data can be funneled into a data warehouse, and that the trend is indeed in the opposite direction.  Thus, an increasing proportion (I would venture to say, in many cases approaching 50%) of corporate in-house data is going to involve cross-data-source querying, as in the merger case. And there, as we have seen, the performance advantages are probably on the DV side, compared to physical merging.

DV Multiple-Instance Optimization

This is perhaps a consideration more suited to abroad and to medium-sized businesses, where per-state or regional databases and/or data marts must be coordinated. However, it may well be a future direction for data warehouse and app performance optimization – see my thoughts on the Olympic database in a previous post. The idea is that these databases have multiple distributed copies of data. These copies have “grown like Topsy”, on an ad-hoc, as needed basis.  There is no overall mechanism for deciding how many copies to create in which instances, and how to load balance across copies.

That’s what a data virtualization server can provide.  It automagically decides how to optimize given today’s incidence of copies, and ensures in a distributed environment that the processing is “pushed down” to the right database instance. In other words, it is very likely that data virtualization provides a central processing software layer rather than local ones – so no performance hit in most cases – plus load balancing and visibility into the distribution of copies, which allows database administrators to achieve further optimization by changing that copy distribution. And this means that DV should, effectively implemented, deliver better performance than existing solutions in most if not all cases.

Where databases are used both operationally (including for master data management) and for a data warehouse, the same considerations may apply – even though we are now in cases where the types of operation (e.g., updates vs. querying) and the types of data (e.g., customer vs. financial) may be somewhat different. One-way replication with its attendant ETL-style data cleansing is only one way to coordinate the overall performance of multiple instances, not to mention queries spanning them.  DV’s added flexibility gives users the ability to optimize better in many cases across the entire set of use cases.

Again, this advantage may not have been perceived (a) because not many implementers are focused on the multiple-copy case and (b) because DV implementation is probably compared against performance against each individual instance instead of or as well as against the multiple-instance database as a whole. Nevertheless, at least theoretically, this performance advantage should appear – and especially because, in this case, the “extra software layer” should not typically add some DV performance cost.

The User Bottom Line:  Where’s The Pain?

It seems that, theoretically at least, we might expect to see actual performance gains over the next 1-2 years over “business as usual” from DV implementation in the majority of use cases, and that this proportion should increase, both over time after implementation and as corporations’ information architectures continue to elaborate. The key to detecting these advantages now, if the IT shop is willing to do it, is more sophisticated metrics about just what constitutes a performance hit or a performance improvement, as described above.

So maybe there isn’t such a performance tradeoff for all the undoubted benefits of DV, after all.  Or maybe there is.  After all, there is a direct analogy here with agile software development, which seems to lose by traditional metrics of cost efficiency and quality attention, and yet winds up lower-cost and higher-quality after all.  The key secret ingredient in both is ability to react or proact rapidly in response to a changing environment, and better metrics reveal that overall advantage. But the “tradeoff” for both DV and agile practices may well be the pain of embracing change instead of reducing risk.  Except that practitioners of agile software development report that embracing change is actually a lot more fun.  Could it be that DV offers a kind of support for organizational “information agility” that has the same eventual effect: gain without pain?

Impossible. Gain without pain. Perish the very thought.  How will we know we are being organizationally virtuous without the pains that accompany that virtue?  How could we possibly improve without sacrifice? 

Well, I don’t know the answer to that one.  However, I do suggest that maybe, rather than the onus being on DV to prove it won’t torch performance, the onus should perhaps be on those advocating the status quo, to prove it will.  Because it seems to me that there are plausible reasons to anticipate improvements, not decreases, in real-world performance from DV.

Information Agility and Data Virtualization: A Thought Experiment

In an agile world, what is the value of data virtualization?

For a decade, I have been touting the real-world value-add of what is now called data virtualization – the ability to see disparate data stores as one, and to perform real-time data processing, reporting, and analytics on the “global” data store – in the traditional world of IT and un-agile enterprises. However, in recent studies of agile methodologies within software development and in other areas of the enterprise, I have found that technologies and solutions well-adapted to deliver risk management and new product development in the traditional short-run-cost/revenue enterprises of the past can actually inhibit the agility and therefore the performance of an enterprise seeking the added advantage that true agility brings. In other words: your six-sigma quality-above-all approach to new product development may seem as if it’s doing good things to margins and the bottom line, but it’s preventing you from implementing an agile process that delivers far more.

In a fully agile firm – and most organizations today are far from that – just about everything seems mostly the same in name and function, but the CEO or CIO is seeing it through different eyes. The relative values of people, processes, and functions change, depending no longer on their value in delivering added profitability (that’s just the result), but rather on the speed and effectiveness with which they evolve (“change”) in a delicate dance with the evolution of customer needs.

What follows is a conceptualization of how one example of such a technology – data virtualization – might play a role in the agile firm’s IT. It is not that theoretical – it is based, among other things, on findings (that I have been allowed by my employer at the time, Aberdeen Group, to cite) about the agility of the information-handling of the typical large-scale organization. I find that, indeed, if anything, data virtualization – and here I must emphasize, if it is properly employed – plays a more vital role in the agile organization.

Information Agility Matters

One of the subtle ways that the ascendance of software in the typical organization has played out is that, wittingly or unwittingly, the typical firm is selling not just applications (solutions) but also information. By this, I don’t just mean selling data about customers, the revenues from which the Facebooks of the world fatten themselves on. I also mean using information to attract, sell, and follow-on sell customers, as well as selling information itself (product comparisons) to customers.

My usual example of this is Amazon’s ability to note customer preferences for certain books/music/authors/performers, and to use that information to pre-inform customers of upcoming releases, adding sales and increasing customer loyalty. The key characteristic of this type of information-based sale is knowledge of the customer that no one else can have, because it involves their sales interactions with you. And that, in turn, means that the information-led sell is on the increase, because the shelf life of application comparative advantage is getting shorter and shorter – but comparative advantage via proprietary information, properly nurtured, is practically unassailable. In the long run, applications keep you level with your competitors; information lets you carve out niches in which you are permanently ahead.

This, however, is the traditional world. What about the agile world? Well, to start with, the studies I cited earlier show that in almost any organization, information handling is not only almost comically un-agile but also almost comically ineffective. Think of information handling as an assembly line (yes, I know, but here it fits). More or less, the steps are:

1. Inhale the data into the organization (input)

2. Relate it to other data, including historical data, so the information in it is better understood (contextualize)

3. Send it into data stores so it is potentially available across the organization if needed (globalize)

4. Identify the appropriate people to whom this pre-contextualized information is potentially valuable, now and in the future, and send it as appropriate (target)

5. Display the information to the user in the proper context, including both corporate imperatives and his/her particular needs (customize)

6. Support ad-hoc additional end-user analysis, including non-information-system considerations (plan)

The agile information-handling process, at the least, needs to add one more task:

7. Constantly seek out new types of data outside the organization and use those new data types to drive changes in the information-handling process – as well, of course, as in the information products and the agile new-information-product-development processes (evolve).

The understandable but sad fact is that in traditional IT and business information-handling, information “leaks” and is lost at every stage of the process – not seriously if we only consider any one of the stages, but (by self-reports that seem reasonable) yielding loss of more than 2/3 of actionable information by the end of steps one through six. And “fixing” any one stage completely will only improve matters by about 11% -- after which, as new types of data arrive, that stage typically begins leaking again.

Now add agile task 7, and the situation is far worse. By self-reports three years ago (my sense is that things have only marginally improved), users inside an organization typically see important new types of data on average ½ year or more after they surface on the Web. Considering the demonstrated importance of things like social-media data to the average organization, as far as I’m concerned, that’s about the same as saying that one-half the information remaining after steps one to six lacks the up-to-date context to be truly useful. And so, the information leading to effective action that is supplied to the user is now about 17% of the information that should have been given to that user.

In other words, task 7 is the most important task of all. Stay agilely on top of the Web information about your customers or key changes in your regulatory, economic, and market environment, and it will be almost inevitable that you will find ways to improve your ability to get that information to the right users in the right way – as today’s agile ad-hoc analytics users (so-called business analysts) are suggesting is at least possible. Fail to deliver on task 7, and the “information gap” between you and your customers will remain, while your information-handling process falls further and further out of sync with the new mix of data, only periodically catching up again, and meanwhile doubling down on the wrong infrastructure. Ever wonder why it’s so hard to move your data to a public cloud? It’s not just the security concerns; it’s all that sunk cost in the corporate data warehouse.

In other words, in information agility, the truly agile get richer, and the not-quite-agile get poorer. Yes, information agility matters.

And So, Data Virtualization Matters

It is one of the oddities of the agile world that “evolution tools” such as more agile software development tools are not less valuable in a methodology that stresses “people over processes”, but more valuable. The reason why is encapsulated in an old saying: “lead, follow, or get out of the way.” In a world where tools should never lead, and tools that get out of the way are useless, evolution tools that follow the lead of the person doing the agile developing are priceless. That is, short of waving a magic wand, they are the fastest, easiest way for the developer to get from point A to defined point B to unexpected point C to newly discovered point D, etc. So one should never regard a tool such as Data Virtualization as “agile”, strictly speaking, but rather as the best way to support an agile process. Of course, if most of the time a tool is used that way, I’m fine with calling it “agile.”

Now let’s consider data virtualization as part of an agile “new information product development” process. In this process, the corporation’s traditional six-step information-handling process is the substructure (at least for now) of a business process aimed at fostering not just better reaction to information but also better creation and fine-tuning of combined app/information “products.”

Data virtualization has three key time-tested capabilities that can help in this process. First is auto-discovery. From the beginning, one of the nice side benefits of data virtualization has been that it will crawl your existing data stores – or, if instructed, the Web – and find new data sources and data types, fit them into an overall spectrum of data types (from unstructured to structured), and represent them abstractly in a metadata repository. In other words, data virtualization is a good place to start proactively searching out key new information as it sprouts outside “organizational boundaries”, because they have the longest experience and the best “best practices” at doing just that.

Data virtualization’s second key agile-tool capability is global contextualization. That is, data virtualization is not a cure-all for understanding all the relationships between an arriving piece of data and the data already in those data stores; but it does provide the most feasible way of pre-contextualizing today’s data. A metadata repository has been proven time and again to be the best real-world compromise between over-abstraction and zero context. A global metadata repository can handle any data type thrown at it. For global contextualization, a global metadata repository that has in it an abstraction of all your present-day key information is your most agile tool. It does have some remaining problems with evolving existing data types; but that’s a discussion for another day, not important until our information agility reaches a certain stage that is still far ahead.

Data virtualization’s third key agile-tool capability is the “database veneer.” This means that it allows end users, applications, and (although this part has not been used much) administrators to act as if all enterprise data stores were one gigantic data store, with near-real-time data access to any piece of data. It is a truism in agile development that the more high-level the covers-all-cases agile software on which you build, the more rapidly and effectively you can deliver added value. The database veneer that covers all data types, including ones that are added over time, means more agile development of both information and application products on top of the veneer. Again, as with the other two characteristics, data virtualization is not the only tool out there to do this; it’s just the one with a lot of experience and best practices to start with.

And, for some readers, that will bring up the subject of master data management, or MDM. In some agile areas, MDM is the logical extension of data virtualization, because it begins to tackle the problems of evolving old data types and integrating new ones in a particular use case (typically, customer data) – which is why it offers effectively identical global contextualization and database-veneer capabilities. However, because it does not typically have auto-discovery and is usually focused on a particular use case, it isn’t as good an agile-tool starting point as data virtualization. In agile information-product evolution, the less agile tool should follow the lead of the more agile one – or, to put it another way, a simple agile tool that covers all cases trumps an equivalently simple agile tool that only covers some.

So, let’s wrap it up: You can use data virtualization, today, in a more agile fashion than other available tools, as the basis of handling task 7, i.e., use it to auto-discover new data types out on the Web and integrate them continuously with the traditional information-handling process via global contextualization and the database veneer. This, in turn, will create pressure on the traditional process to become more agile, as IT tries to figure out how to “get ahead of the game” rather than drowning in a continuously-piling-up backlog of new-data-type requests. Of course, this approach doesn’t handle steps 4, 5, and 6 (target, customize, and plan); for those, you need to look to other types of tools – such as (if it ever grows up!) so-called agile BI. But data virtualization and its successors operate directly on task 7, directly following the lead of an agile information-product-evolution process. And so, in the long run, since information agility matters a lot, so does data virtualization.

Some IT-Buyer Considerations About Data Virtualization Vendors

To bring this discussion briefly down to a more practical level, if firms are really serious about business agility, then they should consider the present merits of various data virtualization vendors. I won’t favor one against another, except to say that I think that today’s smaller vendors – Composite Software, Denodo, and possibly Informatica – should be considered first, for one fundamental reason: they have survived over the last nine years in the face of competition from IBM among others.

The only way they could have done that, as far as I can see, is to constantly stay one step ahead in the kinds of data types that customers wanted them to support effectively. In other words, they were more agile – not necessarily very agile, but more agile than an IBM or an Oracle or a Sybase in this particular area. Maybe large vendors’ hype about improved agility will prove to be more than hype; but, as far as I’m concerned, they have to prove it to IT first, and not just in the features of their products, but also in the agility of their data-virtualization product development.

The Agile Buyer’s Not So Bottom Line

Sorry for the confusing heading, but it’s another opportunity to remind readers that business agility applies to everything, not just software development. Wherever agility goes, it is likely that speed and effectiveness of customer-coordinated change are the goals, and improving top and bottom lines the happy side-effect. And so with agile buyers of agile data-virtualization tools for information agility – look at the tool’s agility first.

To recap: the average organization badly needs information agility. Information agility, as in other areas, needs tools that “follow” the lead of the person(s) driving change. Data virtualization tools are best suited for that purpose as of now, both because they are best able to form the core of a lead-following information-product-evolution toolset, and because they integrate effectively with traditional information-handling and therefore speed its transformation into part of the agile process. Today’s smaller firms, such as Composite Software and Denodo, appear to be farther along the path to lead-following than other alternatives.

Above all: just because we have been focused on software development and related organizational functions, that doesn’t mean that information agility isn’t equally important. Isn’t it about time that you did some thought experiments of your own about information agility? And about data virtualization’s role in information agility?

The Other BI: Composite CIS/Studio and Agile BI

This blog post highlights a software company and technology that I view as potentially useful to organizations investing in business intelligence (BI) and analytics in the next few years. Note that, in my opinion, this company and solution are not typically “top of the mind” when we talk about BI today.

The Importance of the Composite Software Technology to BI

Today’s enthusiasm for “agile BI” that often consists of slapdash applications of SCRUM to rapid roll-out of new analytics applets obscures the fact that business agility, not to mention New Product Development (NPD) agility, involves applying more of agile methodologies than a sprint, and using agile methodologies in many more areas than software development. However, this is where the market is right now. The issue at hand is not so much the better application of business agility in the future, as applying today’s “agile BI” more effectively in the next 2-3 years. This is best done by recognizing one key fact: infusing the organization’s BI with agility is not primarily about development agility – it is about information agility.

Speeding up an inflexible business process via targeted analytics simply doubles your investment in and commitment to an information-handling process that needs instead the ability to fundamentally change. As I have noted in past posts, surveys show that in the typical process of receiving data and using analytics as part of the process of turning it into the right decision by the right decision-maker, or the right change in the business, the biggest problem is not that information is delivered too slowly. The fundamental problem is (a) that “leaks” at every step in the process result in more than 2/3 of the actionable information being lost along the way, and (b) the inflexibility of the process – specifically, its lack of openness to new types of data as they arrive – keep making the “leaks” worse.

Today’s best technology for dealing with (a) and (b) is something now called “data federation” or “data virtualization”. I won’t repeat the long litany of benefits to be expected from data virtualization; here, I will simply note that data virtualization, like master data management, provides one view of related data across the enterprise to the developer and the end user, and, unlike master data management, it can actively reach out, within or beyond the enterprise (as in Composite Discovery), to discover new data sources that can likewise be part of that global view of potential information. The one view of data tackles problem (a) at several critical points by ensuring that everyone can potentially see all data, and that analytics tools can be potentially applied to all useful data. The discovery feature tackles problem (b), if used effectively, by constantly refreshing the actionable data from, let’s say, new social-media data sources not presently covered by BI – as some Composite solutions do for Big Data.

Solving problem (b), of course, improves the organization’s information agility, not just its development agility. So the quick and effective method of tackling information agility in the here and now is to make a data virtualization development tool part and parcel of agile BI projects. Tools from folks like Composite Software, Tibco, Denodo, IBM, and SAP/Sybase are among those that would seem to fit the bill.

The Relevance of Composite Software to BI

Surprisingly enough, for a while around five years ago data virtualization’s predecessor, Enterprise Information Integration, was a hot topic in BI, as a way of extending data warehousing to operational data stores in order to query certain key new data in “near real time.” The primary results of this “fad” were to finally and definitively establish a major market for data virtualization, as well as to cement BI as Composite Software’s core market. In other words, unlike most or all other data-virtualization vendors, Composite Software has ten years of experience with real-world data virtualization at medium to large scales, five years of experience with BI at the same scales, a demonstrated ability to combine with other flexible BI tools such as MicroStrategy, and a focus on handling more “complex” needs that translates naturally to the more complex in-depth analytics typical of social-media and related Big Data analytics.

It may reasonably be objected that Composite, as well as other data virtualization vendors, has shown little or no interest in agile software development methodologies up to now, and therefore use of data virtualization solutions such as Composite’s may tend to slow down agile BI development efforts. Again, I would argue, as someone who has been touting agile development nearly since its inception twelve years ago, that this misses the point on two counts. In the first place, the rule of thumb with agile tools is “lead, follow, or get out of the way.” Where a SCRUM-supporting tool leads by urging developers towards agile best practices, and a really good refactoring tool follows in its wake by adding flexibility to the code thus produced, there is a third category of tools that simply automates and simplifies the necessary tasks of software development, while not getting in the way of the developer – and that is what something like Composite Information Server (CIS) with Composite Studio does for data-accessing functions in agile-BI programs.

In the second place, data virtualization such as Composite Software’s comes with a built-in gain in any agile-BI program’s information agility, which is often just as valuable as increased analytic-program implementation speed. Out of the box, the developer is assured of not missing a key in-enterprise data source. More subtly, the developer is made aware of other data sources than the one he or she may initially have targeted, which means that, as should be expected in agile development projects, new and unexpected features with value-add surface in the middle of the project.

In other words, while Composite Software may be in the middle of the pack when it comes to agility applied to agile-BI software development, it has long taken a leadership role in the features of data virtualization that really matter to agile-BI users, including BI services expertise, discovery, and complex data virtualization.

One final caveat appears to be going away. Since their inception, data virtualization tools have been oriented towards users above a certain size of medium-sized organization. Obviously, much of the action and excitement around agile BI has centered around its use in the public cloud by SMBs, who traditionally don’t deal with lots of various data sources that could use virtualization. However, what these SMBs are finding is that some public-cloud data is inherently “multi-organization”, and therefore requires federation and virtualization – especially since many of these same SMBs are now using multiple public clouds for their analytics. Sooner or later, Big Data has to be combined with ERP data on the cloud, and then with customer data on salesforce.com or wherever, and so on. As a result, even these SMBs are probably going to find data virtualization and Composite-type technology useful in their agile-BI efforts sooner rather than later.

Potential Uses of CIS-Type Agile BI for IT

Until recently, data virtualization in the enterprise has typically been done on a per-project basis, with the result that spreading it across all in-enterprise data is typically a work in progress. Obviously, this limits a little the usefulness of data virtualization in agile-BI development in the near future; but it is also an opportunity, in those enterprises that qualify, to kick-start the “global metadata repository” effort into high gear. Slap a data virtualization engine and development tool like CIS with Composite Studio into the agile-BI development team’s toolset, and have them scream to you about needing it to cover more data. That will get corporate’s attention, fast.

For those organizations that haven’t used data virtualization before, especially the SMBs, using data virtualization on things like Big Data is an excellent opportunity to avoid the arrows that the large-enterprise data-virtualization pioneer users faced. More specifically, if you start out by federating key customer Big Data with ERP on the cloud, private or public, you have already created a repository that covers most of your likely business-critical data in the next 2-3 years, and so you will not have to retrofit it later on.

A third, often underestimated, use of data virtualization is as a tool to support the new “business analysts” on the corporate side who are also doing agile analytics. A recent survey cited by Sloan Management Review suggests that these now view IT as slower and lower than dirt. Give them and support a tool that allows them to go out and discover new data sources, and I would not be surprised if they responded, like the woman conducting a choir in the movie Love Story, with “That is absolutely, stupendously, incredibly – OK.” That is, they won’t kiss you, but they will make a big shift towards tolerating you.

However, the use of CIS-type data virtualization that is dearest to my heart, and should be dearest to yours if you ever want to accomplish business agility rather than just BI “agility,” is applying its discovery features constantly, out on the Web, embedded in your analytics applications, to find and bring in-house new data and new data types. One key finding of business surveys is that only half of respondents find out about new key data heralding new market trends on the Web in less than half a year. Data virtualization can cut that to a day. You see, business agility isn't about reacting quickly and effectively to changes in your environment that you happen to find out about because they show up eventually in your customer complaints; the critical success factor is finding out all the changes that you can, as they happen out there, and then reacting quickly and effectively.

The Bottom Line for IT Buyers

Over the next few years, IT buyers should be viewing the acquisition and use of data virtualization tools in agile BI, among other things, as a no-brainer – but they don’t. Those smart IT buyers who do, however, will find that a pre-short-list is not necessary: on closer examination, the use cases almost present themselves. At any rate, that has been users’ experience in data virtualization’s original core markets such as the armed forces in the government – another Composite installed base.
However, for those constrained by budgets even in their strategic analytics initiatives, Composite and others may well have to go on a “pre-short list.” This should be of a peculiar type, however. The IT buyer should not passively wait for corporate to give IT a mandate that requires data virtualization. Instead, the IT buyer should be actively looking for the opportunity to introduce or extend agile-BI data virtualization, like the politician looking in every budget process or special spending initiative for a way of cutting taxes.

Composite Software is one among several who should be on that short list, as well as other data-virtualization-focused firms like Denodo and Tibco, not to mention the data federation solutions of IBM, Oracle via its BEA WebLogic Liquid Data acquisition, Red Hat/Metamatrix, and SAP/Sybase. Composite Software’s edge over most of these is simply that, over the last 10 years, Composite has established its experience and a leadership role in features and services useful to agile BI, and is among the most vocal about their plans to support Big Data and agile BI. Since that leadership role hasn’t diminished over the last three years, Composite Software will probably maintain its position at or near the top of agile-BI data virtualization short lists over the next 2-3 years, as well.

The bottom line for IT buyers of Other agile-BI data virtualization in general, and Composite Software’s CIS and Composite Studio in particular, is this: buy it right now or buy it a year from now, use it for BI agility or use it for business agility, but buy it and use it for some agile-BI initiative sometime in the next 2-3 years. Like an agile development methodology, it looks like it might cost more and cut into the bottom line, but it repeatedly costs less and delivers more revenue – because, used right, it improves your information agility.