Advanced Data Virtualization Differentiation

Where's the Beef? - http://bit.ly/fL0WkP

Author: Michael Zuckerman, CMO, Queplix

What makes advanced data virtualization unique? How is this really different than basic data virtualization? Than ETL with web services? Good questions. Where's the beef?

All of the basic capabilities of basic data virtualization are there with advanced data virtualization - that's a given. This includes four basic components, worth reviewing again since my previous post on advanced data virtualization. Even at this level Queplix has significant differentiation:

(1) Data is virtualized. All data from all the sources you connect to appear accessible from our system - one place.

(2) Data is abstracted. There is now a consistent object format to get to and use this data regardless of the underlying application and database that holds it. Other products perhaps only manage relational well most of the time. They tend to "cludge" access to other data formats through Java API's or table functions in the from clause of a SQL statement. This is much more difficult for you. In Queplix technlogy all relational, object and XML area easily accessible from various Queplix systems in one virtualized dashboard. Anything and everything.

(3) Data is federated. The system gets you the basic connectivity to pull together pieces and move them to other places, such as BI domains, data warehouses or other applications.

(4) Data is presented in various "views" to the consuming applications. They can be BI, applications (for data integration) and more. Web mashups have been a popular application for basic data virtualization.

If basic data virtualization is what you need I'd suggest you try TEEID. Go to http://www.jboss.org/teeid and download data virtualization. You are good to go for many applications.

Advanced data virtualization has many more powerful capabilities. Lets look more closely at them:

(1) Advanced data virtualization uses one or more persistent metadata servers. This is not middleware. The core of VirtualETL™, CloudETL™ and Virtual Data Manager™ is a persistent metadata server. The basis of advanced data virtualization is a persistent metadata directory. There is more – this is really a persistent metadata server with additional workflow for harmonization, security and intelligent data management. This means that you can start with a simple integration between two applications and then extend those applications, pursuant to security permissions, to additional applications. This is not a point-to-point architecture. This is a 1:Many architecture (hub and spoke) and brings tremendous advantages in scale and power. This enables the low end of an architecture that scales from one, two or three integrations to solve a much larger data integration and master data alignment challenge. This is the baseline of addressing the larger vision. It starts with the right architecture. That's the first place basic data virtualization and ETL-with-web-services really breaks down - they don't have the right architecture.

(2) Speed. Persistent metadata servers are very fast, up to 1000 times faster than a relational database engine. That’s because our engines don’t hold the data, only information about the data that let us use it. That’s why we can scale to support and manage problems at the scale of master data management. That's how it all works. We don't spend endless hours caching mountains of files ... only the records (objects) we need to touch.

(3) Globalized Data. Basic data virtualization has only the notion of an abstracted view. Pieces of one or two things are abstracted virtually at a third consuming application. Advanced data virtualization creates globalized data. A metadata component that represents and aligns entities across the various applications to be integrated. It is a powerful concept. A true abstraction. It can support automated data synchronization (Data Harmonization™) or support a highly simplified interface for consuming applications such as business intelligence or enterprise search. The possibilities are endless because this enables an architecture that provides broad advantages across multiple classes of applications.

(4) Automated Server. Powerful capabilities set up connectivity, with requisite security permissions, integrate the applications and create global entities in an almost completely automated fashion. Automate or die! Manual SQL coding, wire diagram creation and messy templates are 15 years old - don't waste your time with this stuff.

(5) Application Software Blades. Application software blades are very intelligent points of data connectivity and security. With applications such as SAP®, PeopleSoft® and Siebel® they are very application aware – we understand the details of internal structures and quickly abstract and align the key data fields so that you can integrate them with other applications. Application software blades are bi-directional and can support the movement of select data elements through harmonization if required. Application software blades can cut 50% of the entire coding and set up out of your costs. This is a huge advantage over legacy ETL-with-web-services and basic data virtualization technology.

(6) Object Model - Step Into the New World. Persistent metadata server are 100% based on the lastest object technology. We convert the representations of the key metadata from the enterprise applications into objects. It doesn’t matter if the source is flatfile, XML, relational (Oracle, Sybase, Microsoft, IBM-DB2, MySQL...), object, MongoDB, MarkLogic, CouchDB, Hadoop, BigTable, Hypertable or VoltDB – it all gets viewed in the object paradigm. Objects scale easier, are easier to manipulate and provide huge advantages in building out a simple user interface with significant power. This is the future - even of data warehousing. Get with the program. That's why we don't need to use SQL - it wasn't made to deal with all of this efficiently nor effectively!

(7) 3 Layer Security Model. The key to deploying an enterprise wide technology is to let the user, without exception or qualification, maintain their current security policy and enforce it. This isn’t just simple data security. We aren't protecting an oil pipeline for data. This is the authentication pass thru for user and groups. Tied to your applications. RSO, SSO – we can support it all. We have an Application Software Blade for LDAP. We’ve also enabled the user of an optional, 3rd layer of security to protect special sets of global entities. By combining information in new ways, perhaps to support a business intelligence application, you may present new and valuable information and we given you an additional way to protect it, at your option. You won't find any of this anytime soon in ETL-with-web-services or basic data virtualization.

(8) Data Harmonization. The key element of the vision that advanced data virtualization supports is the notion that a simple integration between two applications and an enterprise wide deployment of master data management are two ends of the same problem. Advanced data virtualization is the first technology to view this as one challenge and provide a solution that cost effectively scales form the low end to the high end. Each stage of the implementation is more cost effective than any competitive alternate and brings the advantages that come with the full vision. Our Data Harmonization™ allows two or more applications to synchronize data automatically. This is an optional and highly automated process that is configurable from being near-time to real-time, to scheduled or even staged depending on the needs of the consuming business organization. It is easy-to-use and highly automated.

(9) Networked Architecture That Can Scale Up To Virtual Master Data Management. Advanced data virtualization uses the power of our persistent metadata server to align different data models to get your systems all working together. Our Data Integration Management Protocol positions our Virtual Data Manager instances as distributed components of a multiprocessor system - but they are all working together. All powered by advanced data virtualization. No one does anything like this - the power is in the architecture. Advanced data virtualization does this without disruption, expensive and unnecessary change. You can implement effectively for an SMB enterprise yet scale up the largest problems in a global enterprise. We do this much better than than any legacy MDM architecture. We have customers integrating together over 50 for data synchronization - can your reference architecture do that?

(10) Data Quality and Data Governance. Any integration for business intelligence, data integration between applications, data warehouse design or master data alignment/management includes considerable time and resources to cleanse, match and semantically analyze data. There are tools within tools to do this. Advanced data virtualization makes it easy. You can find you data quality issues and eliminate them. You can set up automated components to normalize your data across multiple applications. One system - one platform - one continuum.

Welcome to the new world!

Tweet This: Advanced Data Virtualization Differentiators: Where's the Beef? - http://bit.ly/fL0WkP

Follow Queplix on Twitter at: http://www.twitter.com/QueplixCorp

Resource library: For more information about data virtualization and application integration, visit Queplix Resource Library - http://bit.ly/fSkNMD

For parties interested in Queplix, data virtualization, data integration, data integration software, master data management, business intelligence, composite software