In-Memory Computing Summit 1: The Database Game, The Computing Game, Is Changing Significantly

So why was I, a database guy, attending last Monday what turned out to be the first ever In-Memory Computing Summit – a subject that, if anything, would seem to relate more to storage tiering?  And why were heavy-hitter companies like Intel, SAP, and Oracle, not to mention TIBCO and Hitachi Data Systems, front and center at this conference?  Answer:  the surge in flash-memory capacity and price/performance compared to disk, plus the advent of the Internet of Things and the sensor-driven web, is driving a major change in the software we need, both in the area of analytics and operational processing.  As one presentation put it, software needs to move from supporting Big Data to enabling Fast (but still Big) Data.

In this series of blog posts, I aim to examine these major changes as laid out in the summit, as well as their implications for databases and computing IT.  In the first post, I’d like to sketch out an overall “vision”, and then in later posts explore the details of how the software to support this is beginning to arrive.

The Unit:  The “Flat-Memory” System

In an architecture that can record and process massive streams of "sensor" data (including data from mobile phones and from hardware generating information for the Internet of Things) there is a premium on "stream" processing of incoming data in real time, as well as on transactional writes in addition to the reads and queries of analytical processing.  The norm for systems handling this load, in the new architecture, is two main tiers:  main memory RAM and "flash" or non-volatile memory/NVRAM (approximately 3 orders of magnitude).  This may seem like hyperbole when we are talking about Big Data, but in point of fact one summit participant noted a real-world system using 1 TB of main memory and 1 PB of flash.

Fundamentally, flash memory is like main-memory RAM:  more or less all addresses in the same tier take an equal amount of time to read or change.  In that sense, both tiers of our unitary system are "flat memory", unlike disk, which has spent many years fine-tuning performance that can vary widely depending on the data's position on disk.  To ease the first introduction of flash, it provided interfaces to CPUs that mimic disk accesses and therefore make flash's data access both variable and slow (compared to a flat-memory interface).  Therefore, for the most part, NVRAM in our unitary system will remove this performance-clogging software and access flash in much the same way that main-memory RAM is accessed today.  In fact, as Intel testified at the summit, this process is already underway at the protocol level.

The one remaining variable in performance is the slower speed of flash memory.  Therefore, existing in-memory databases and the like will not optimize the new flat-memory systems out of the box.  The real challenge will be to identify the amount of flash that needs to be used by the CPU to maximize performance for any given task, and then use the rest for longer-term storage, in much the same way that disk is used now.  For the very largest databases, of course, disk will be a second storage tier.

The Architecture:  Bridging Operational Fast and Analytical Big

Perhaps memSQL was the presenter at the conference who put the problem most pithily:  in their experience, users have been moving from SQL to NoSQL, and now are moving from NoSQL towards SQL.  The reason is that for deeper analytical processing of data such as social-media whose value is primarily that it's Big (e.g, much social-media data) use of SQL and relational/columnar databases is better, while for Big Data whose value is primarily that it's fresh (and therefore needs to be processed Fast) SQL software causes unacceptable performance overhead.  Users will need both, and therefore will need an architecture that includes Hadoop and SQL/analytical data processing. 

One approach would treat each database on either side as a "framework", which would be applied to transactional, analytical, or in-between tasks depending on its fitness for these tasks.  That, to me, is a "bridge too far", introducing additional performance overhead, especially at the task assignment stage.  Rather, I envision something more akin to a TP monitor, streaming sensor data to a choice among transactional databases (at present, mostly associated with Hadoop), and analytical data to a choice among other analytical databases.  I view the focus of presenters such as Redis Labs on the transactional side and SAP and Oracle on the analytical side as an indication that my type of architecture is at least a strong possibility.

The Infrastructure:  If This Goes On …

One science fiction author once defined most science fiction as discussing one of three “questions”:  What if? If only …, and If this goes on …  The infrastructure today for the new units and architecture is clearly “the cloud” – public clouds, private clouds, hybrid clouds.  With the steady penetration of Hadoop into enterprises, all of these are now reasonably experienced in supporting both Hadoop and SQL data processing.  And yet, if this goes on …
The Internet of Things is not limited to stationary “things”.  On the contrary, many of the initial applications involve mobile smartphones and mobile cars and trucks.  A recent NPR examination of car technology noted that cars are beginning to communicate not only with the dealer/manufacturer and the driver but also with each other, so that, for example, they can warn of a fender-bender around the next curve.  These applications require Fast Data, real-time responses that use the cars’ own databases and flat memory for real-time sensor processing and analytics.  As time goes on, these applications should become more and more frequent, and more and more disconnected from today’s clouds.  If so, that would mean the advent of the mobile cloud as an alternative and perhaps dominant infrastructure for the new systems and architecture.

Perhaps this will never happen.  Perhaps someone has already thought of this.  If not, folks:  You heard it here first.

A Naive Idea About the Future of Data Virtualization Technology

On a weekend, bleak and dreary, as I was pondering, weak and weary, on the present state and future of data virtualization technology, I was struck by a sudden software design thought.  I have no idea whether it’s of worth; but I put it out there for discussion.

The immediate cause was an assertion that one future use for data virtualization servers was as an appliance – in its present meaning, hardware on which, pretty much, everything was designed for maximum performance of a particular piece of software, such as an application, a database, or, in this case, a data virtualization solution.  That I question:  by its very nature, most keys to data virtualization performance lie on the servers of the databases and file management tools data virtualization servers invoke, and it seems to me likely that having dedicated data-virtualization hardware will make the architecture more complicated (thereby adding administrative and other costs) to achieve a minimal gain in overall performance.  However, it did lead to my thought, and I call it the “Olympic database.”

The Olympic Database

Terrible name, you say.  This guy will never be a marketer, you say.  That’s true. In fact, when they asked me as a programmer for a name for Prime Computer software doing a graphical user interface, I suggested Primal Screens.  For some reason, no one’s asked me to name something since then.

Anyway, the idea runs as follows.  Assemble the usual array of databases (and Hadoop, yada).  Each will specialize in handling particular types of data.  One can imagine splitting relational data between that suited for columnar and that not so suited, and then applying a columnar database to the one and a traditional relational database to the other, as Oracle Exadata appears to do. But here’s the twist:  each database will also contain a relatively small subset of the data in at least one other database – maybe of a different type.  In other words, up to 10%, say, of each database will be a duplicate of another database – typically, the data that queries will typically want in cross-database queries, or the data that in the past a database incorporates just to save time switching between databases.  In effect, each database will have a cache of data in which it does not specialize, with its own interface to it, SQL or other.

On top of that, we place a data virtualization server.  Only this server’s primary purpose is not necessarily to handle data of varying types that a particular database can’t handle.  Rather, the server’s purpose is to carry out load balancing and query optimization across the entire set of databases.  It does this by choosing the correct database for a particular type of data – any multiplexer can do that – but also by picking the right database among two or several options when all the data is found in two or more databases, as well as the right combination of databases when no one database has all the data needed.  It is, in effect, a very flexible method of sacrificing some disk space for duplicate data for the purpose of query optimization – just as the original relational databases sacrificed pure 9NF and found that duplicating data in a star or snowflake schema yielded major performance improvements in large-scale querying.

Now picture this architecture in your mind, with the data stores as rings.  Each ring will intersect in a small way with at least one other data-store “ring” of data.  Kind of like the Olympic rings.  Even if it’s a terrible name, there’s a reason I called it the Olympic database.

It seems to me that such an Olympic database would have three advantages over anything out there:  specialization in multiple-data-type processing in an era in which that’s becoming more and more common, a big jump in performance from increased ability to load balance and optimize across databases, and a big jump in the ability to change the caches and hence the load to balance dynamically – not just every time the database vendor adds a new data type.

Why Use a Data Virtualization Server?

Well, because most of the technology is already there – and that’s certainly not true for other databases or file management systems.  To optimize queries, the “super-database” has to know just which combination of specialization and non-specialization will yield better performance – say, columnar or Hadoop “delayed consistency”.  That’s definitely something a data virtualization solution and supplier knows in general, and no one else does. We can argue forever about whether incorporating XML data in relational databases is better than two specialized databases – but the answer really is, it depends; and only data virtualization servers know just how it depends.

The price for such a use of a data virtualization server would be that data virtualization would need to go pretty much whole hog in being a “database veneer”:  full admin tools, etc., just like a regular database. But here’s the thing:  we wouldn’t get rid of the old data virtualization server. It’s just as useful as it ever was, for the endless new cases of new data types that no database has yet combined with its own specialization.  All the use cases of the old data virtualization server will still be there.  And an evolution of the data virtualization will accept a fixed number of databases to support with a fixed number of data types, in exchange for doing better than any of the old databases could in those conditions.

Summary

Ta da! The Olympic database! So what do you think?