Making Data the Differentiator

Forward-Thinking Companies Use Aggregation and Analytics to Create a Competitive Edge

Shopzilla, the online shopping portal, gives Internet shoppers a quick way to find the products they want at the best price. But the company does more than just supply Web-based comparison shopping. It also provides its merchant partners with a system that allows them to conduct surveys of customers who buy from their sites.

Shopzilla’s merchant customers use these survey results to drive operational decisions. They can make special offers to shoppers who report an unsatisfactory shopping experience during a point-of-service survey, which can help build customer loyalty. Or they might offer special pricing to customers who have purchased the same product more than once.

Knowing which actions to take requires rapid access to customer survey data and the ability to run fast, complex ad hoc queries.

For these companies, and Shopzilla itself, business intelligence has become their lifeblood. The data they collect, often from multiple sources, must be aggregated and analyzed if it is to provide value. That requires data warehousing and analytics solutions that ensure:

• The capacity to handle increased query demand, especially for ad hoc queries
• Access to hundreds or thousands of simultaneous users
• Reduced data latency, sometimes providing nearly real-time updating
• The ability to grow with increasing data volumes
• Lower hardware, administration and maintenance costs
• No downtime
• High levels of security

The Right Tool for the Right Job
These demands aren’t easy to achieve, and in many cases, companies are using the wrong solutions.

Most often, businesses turn to the familiar relational database management system to handle their data analytics challenges. The problem with using these systems for analytics is that complex queries on large volumes of data run very slowly. This is because RDBMS solutions are designed for transactional, rather than analytic environments. They store data in tables by row, where each row is effectively equivalent to a transaction.

“Whenever you access data from a conventional database, you read each row in its entirety, regardless of the actual fields that you are interested in,” writes Philip Howard of Bloor Research in the paper Sybase IQ Is Well Positioned to Compete with Both the Traditional and Appliance Vendors.  “In practice this might mean reading a 3,000 byte record to retrieve just 20 characters of data.”

As a workaround, many DBAs (database administrators) tune these transactional databases to speed up results. But this only works for canned queries, and it requires significant administrative time and investment. When it comes time for ad hoc requests across large volumes of data, these systems still slow to a crawl.

Another common option is data warehouse appliances. These solutions are designed for analytics, so they offer significant speed advantages over transactional databases for individual queries.

However, they also have some major flaws. First, they run on proprietary hardware, which means the entire system must be replaced when the hardware becomes obsolete. They also have very limited ability to scale up to support multiple users. Multiple queries from hundreds or thousands of users can choke the systems. Finally, DBAs are generally limited in the amount of tuning they can do for specialized applications, and these appliances offer little support for text analytics.

For companies that depend on fast, ad hoc query responses by many users accessing large volumes of data, a different solution is needed. Sybase IQ offers that solution.

A Column-Based Approach
Sybase IQ takes a fundamentally different approach to data analytics. It stores and accesses data in tables by column. This serves to automatically index the entire database, because query selection criteria are defined by column. This greatly reduces the amount of data that needs to be read to respond to a query, and dramatically speeds up response times.

Despite this change in the processing structure of the data, Sybase IQ works exactly like other relational databases from an administrative and management point of view. It runs on standard hardware and operating systems. It is accessible via standard SQL. And its database schemas are generally the same as those of a transactional database.

Because of these advantages, Sybase IQ supports extremely fast query speeds without limiting:

• The amount of data available for analytics
• The number of concurrent users and queries
• The launching of new applications
• The complexity of queries
• The number of access windows to data

High Return on Investment
In addition to its edge in query response times, Sybase IQ offers customers a far better return on investment than other data warehousing and analytics solutions. Because of its speed, it requires fewer hardware resources than other options. Its column-based approach also reduces the need to pre-aggregate data and tune the database, saving significant administration time.

Furthermore, Sybase IQ’s nonstop availability eliminates costly downtime, which can frustrate customers who rely on access to the data. Sybase IQ also offers easy migration from current systems, and provides excellent security options, including column-level encryption, to protect data.

Superb Results
For companies that rely on analytics for a competitive edge, Sybase IQ’s advantages translate into real results. Shopzilla selected Sybase IQ to upgrade its merchant customers’ access to survey information. The results were dramatic. The company saw query times reduced by 99 percent, achieved lower administration costs and greatly improved customer satisfaction. “The customers say it’s incredible,” said Henri Asseily, chief technology officer and co-founder of Shopzilla. “They are blown away by what they can do.”

Shopzilla is not alone in reporting these improvements. LoanPerformance, a major provider of risk management, financial analysis and regulatory compliance services, was facing major slowdowns in performance as its databases grew. Customers were complaining about extended wait times for information, as 20 percent of queries took more than 5 minutes and 5 percent timed out.

After adopting Sybase IQ, LoanPerformance saw average query speed increases of eight to 100 times over their previous system. The number of queries that took more than 10 minutes also declined from 16 percent to less than 1 percent. “This is an across-the-board upgrade that provides a whole new, entirely different experience for users—especially power users,” said Dan Feshbach, chief executive officer for LoanPerformance.

Because of its unique column-based architecture and other powerful features, Sybase IQ is an ideal solution for businesses that depend on data aggregation and analytics solutions to grow revenue and provide differentiation. When they face large numbers of ad hoc or complex queries, many concurrent users, active warehousing situations and large data sets, Sybase IQ delivers the performance they require.

Sybase and its partners also offer a host of solutions to common data warehousing problems, such as data transformation requirements, to support a customer’s use of Sybase IQ. “We’ve built an ecosystem of choices that can be tailored to meet specific customer business needs,” says Lisa Dreyer, director of product marketing for Sybase. These include a built in ETL function, which can pull data from many sources, transform it and load it into Sybase IQ. Also contributing to the solution is PowerDesinger, Sybase’s market leading data and enterprise modeling tool. PowerDesigner contributes tools that streamline the migration to—as well as the development and maintenance of—Sybase IQ through comprehensive forward and reverse engineering techniques.

Because of its unique features and complementary products, Sybase IQ is fast to implement and easy to support. This, combined with its column-based architecture, make it a perfect solution for many data warehousing and business intelligence needs.