Product Guide: Chart your Web site's success

Product Guide: Chart your Web site's success

In the late 1990s, Web analytics packages did a respectable job crunching server logs and uncovering broad Web site trends such as page views or user clickstream behavior. Today the focus has shifted to business reporting -- pinpointing the effectiveness of promotional campaigns, measuring ROI, and analyzing processes -- and to delivering those facts to content owners in a clear manner so that the appropriate corrective measures can be put into motion.

In the late 1990s, Web analytics packages did a respectable job crunching server logs and uncovering broad Web site trends such as page views or user clickstream behavior. Today the focus has shifted to business reporting -- pinpointing the effectiveness of promotional campaigns, measuring ROI, and analyzing processes -- and to delivering those facts to content owners in a clear manner so that the appropriate corrective measures can be put into motion.

Such solutions can help you determine, for example, how the placement of a product on your home page affects sales; which e-mail campaign best persuades recipients to make purchases; or whether a pay-per-click keyword campaign is driving the right customers to your site. Moreover, these tools help you identify customer support issues so that you can post solutions to your site, rather than leaving your call center staff to contend with frequently encountered, easily solved troubleshooting issues.

Maintaining an effective Web site requires matching your site's business objectives with the appropriate analytics solution. To help you find that solution, I evaluated four of the top applications, which cover both hosted and in-house architectures. They include Coremetrics 2005, NetIQ WebTrends 7.1 Enterprise Edition, Omniture SiteCatalyst 11, and WebSideStory HBX On-Demand Web Analytics 2.5.

Companies that use the Web to gather qualified marketing leads or to provide visitors with support will find that HBX On-Demand and WebTrends deliver the best overall value. Melding technical performance data with content reports, both solutions help you recognize problems with your site and measure how well your online processes work.

The Web is also an enormously effective commerce tool, returning as much as US$25 for every dollar invested in retaining clients, according to The Boston Consulting Group. All four products aim at helping you understand your most valuable visitors, and they each come close to the mark. Because visitor segmentation and tracking are so complex, companies in vertical markets such as retail or financial services will be best served by a tailored solution such as Coremetrics.

In evaluating these products, I first considered implementation, paying particular attention to each solution's approach to adding tracking information to Web pages and the amount of information the solution retained. Next, I focused on usability, scrutinizing each system's navigability and the readability of its dashboards.

I also examined the depth of each system's analysis: I looked for active segmentation, campaign analysis, and funnel or scenario analysis. But it isn't enough to have these features. Reports must be formatted for the intended audience, such as sales figures that can be rolled up over several sites for executives. If a particular report isn't included, there must be a way to generate an ad hoc report or to access a report builder.

Another important consideration -- especially in this age of personalization and one-to-one marketing -- is the ability to meld Web data with, say, a CRM application or a third-party e-mail system. Therefore, I assessed the various ways collected visitor information can be reused.


Foremost, Coremetrics 2005 is an online marketing system for selected vertical markets, including retail, financial services, and travel services. Not only does this hosted solution provide a wealth of statistics about Web customers, but its ad hoc reports help e-commerce executives understand customer actions and thus improve business.

From the outset, I was impressed with Coremetrics 2005's implementation. There's little work needed to place JavaScript page tags, called SmartTags, nor is there any ongoing maintenance. For example, the tags automatically sense the marketing campaign that sends visitors to specific Web pages. When testing Coremetrics' competition, I often had to predefine events or customer segments before I could run reports.

Another Coremetrics 2005 advantage, called LIVE (Lifetime Individual Visitor Experience), stores a complete record of every click and activity for every customer and visitor to your Web site. In contrast, other products sometimes amalgamate this data, thus losing most details.

This solution's browser interface conveniently groups reports -- including top-line metrics, marketing, site analysis, and segmentation -- within main drop-downs. Plus, when users log in they see a snapshot graph of how the site is performing. For instance, a retail client might see month-to-date sales compared with targets, as well as projections for the remainder of the month. I easily set up these charts by making a few menu selections.

Although not a capability exclusive to this product, Coremetrics' LIVEview browser plug-in allowed me to quickly switch the type of data superimposed on a Web page. Readouts range from page views for selected time periods to sales and number of orders.

I found this solution's marketing and merchandising analysis especially complete. Using the Marketing Management Center module, I displayed reports that broke down channels, such as search and e-mail. With a few clicks I drilled into each channel, getting information such as referrals by Google and Overture search (both paid and natural), sales that were the result of an e-mail campaign, and average order value. These prepopulated reports that summarize site performance can be downloaded to Excel or be e-mailed on predetermined schedules.

Yet where Coremetrics really outclasses other products is in its LIVE profiles. Using a wizard, I created a new customer segment on the fly based on RFM (recency, frequency, monetary) analysis. After a few steps I was able to see all visitors who had returned three or more times to the site, had purchased more than $300 in electronic equipment, and had entered the site via a Google keyword campaign. I could even drill down to an individual visitor and see what they did, click by click.

For this and other tasks, Coremetrics includes Virtual Analyst. This souped-up help function gives you very detailed instructions and answers to questions -- much as you'd get during a live conversation with a member of the company's professional services staff. In testing, this expertly explained how to create intricate segment reports and what these reports mean.

For retailers, customer data is invaluable -- and Coremetrics 2005 appears to have the best method for employing LIVE repositories for both cross-selling and multichannel marketing. Although I didn't test these features in depth, several Coremetrics clients report that the system uncovered cross-selling opportunities beyond the ordinary.

In the multichannel area, Coremetrics formats LIVE data so that it can be used by third-party applications. For example, you can tie into CRM applications from Responsys to create targeted e-mail campaigns.

Coremetrics isn't sold primarily to content site owners, although it does have solid tools that developers can use to optimize sites, such as path analysis, which can be used to find out whether visitors bail out on a transaction because a form is too difficult to use. Coremetrics 2005's ROI comes from exceptional marketing and merchandising features that enable retailers and other e-commerce site managers to proactively reach out to customers.

NetIQWebTrends 7.1 Enterprise Edition

For marketers and content owners to act on Web performance data, the data must be easy to view and digest. To that end, WebTrends includes visual aids such as charts of clickstream paths and scenario analysis. The package also offers several ways to mine data dynamically, including a fine Excel plug-in that permits pivoting Web and offline information. Released in late January, Version 7.1 has improved WebTrend's SmartView browser plug-in for evaluating page links. The updated iteration also integrates better with e-mail marketing services, and it delivers more detailed audience-segmentation data.

WebTrends is easy to set up. You just update Web pages with tracking tags that identify campaigns and other attributes you want to monitor. If it's installed in-house, you also load a server plug-in.

Administrators control rights to templates and thus determine which reports users see when they log in to the WebTrends portal. Because WebTrends desktop supports multiple profiles, I was able to analyze different portions of my Web site, such as the software support area. WebTrends arranges reports in an intuitive "table of contents" format that clusters, for example, all online sales reports under the Commerce heading.

Introduced in Version 7.0, Campaign Drilldown gave me a detailed look at campaign performance, showing me statistics such as the number of visitors from a printed ad. Reports include the number of unique visitors per month, conversion metrics, and revenue information. WebTrends also distinguishes between paid search engine campaigns and organic search. Indeed, the product does one better: It shows your search engine ranking for various keywords versus that of your competition. Yet WebTrends not only reports your position but also the ROI for those keywords. For example, you could be ranked No. 3 for a keyword but actually derive more revenue than if you were in the top spot.

To improve conversion rates, I used Scenario Analysis, one of the most complete navigation analysis tools included in this test. I easily saw how people navigated to -- and through -- a registration process and self-service resolution. Significantly, I was able to find out not only where people abandoned the process but also where they went after that point, which helped me fix problems with forms. Unlike Omniture SiteCatalyst, WebTrends allowed me to see where visitors entered my site from e-mail links, helping me accurately gauge the effectiveness of my e-mail campaign.

In the e-commerce area, Product Drilldown summarized order and revenue figures for broad product groups and then allowed me to dig into each category with a simple click -- down to individual SKUs. Calculations such as GMROI (gross margin ROI) should be especially useful to marketing executives.

One of the best features, SmartReports for Microsoft Excel, integrates offline marketing and product cost data with live Web information. This plug-in automatically creates pivot tables, and with one click, you can update tables with live data. After I customized the spreadsheet and stored it on a shared network drive, a scheduler function updated figures automatically throughout the day. The result: Finance and marketing colleagues received needed data without having to access WebTrends directly.

WebTrends 7.1 enhances the product's SmartView browser plug-in. For example, much better segmentation data such as buyers vs. nonbuyers appeared over links on Web pages. Moreover, this same overlay capability now works with HTML messages sent using ExactTarget, so you can quickly see performance metrics for each link in an e-mail.

WebTrends' flexibility -- for example, it can analyze the log files of every major Web server -- comes with a price. You must set up tracking codes and configure campaigns. That effort aside, WebTrends 7, and now 7.1, proves that the company is once again taking the competition -- and client needs -- very seriously. Custom reports are much easier for users to develop, and they don't rely on Excel, which is necessary in WebSideStory. Just as important, WebTrends helps you easily drill down into data to really understand customer actions.

OmnitureSiteCatalyst 11

Metrics are the staple of Web analytics. But many products' canned reports lack the details a particular manager needs -- and may require developer skills to modify. Omniture's SiteCatalyst 11 takes an opposite "white box" approach. You get stock reports such as most-popular pages and e-commerce revenue, but the system's strength is that it opens all collected data to end-users, permitting them to build custom reports relevant to their area of interest in near real time.

Despite being packed with options, SiteCatalyst's portal-like Web interface is well organized. For example, the primary Commerce, Traffic, and Paths reports are shown prominently in the main navigation window.

My testing kicked off where the product focused most, e-commerce. I especially liked Campaign Manager, which gave me a snapshot of active e-mail and online marketing endeavors along with how much revenue each generates. A downside is that you must preplan these campaigns so pages you want to analyze have the appropriate tags.

An advantage of SiteCatalyst, however, is that you can immediately drill down (using a pop-up menu) to further examine these figures, including what customers bought and the length of purchase cycles. This information isn't quite as granular as that given by Coremetrics, but it's very valuable nonetheless. Furthermore, using SiteCatalyst Report Wizard, I easily created new reports, such as which sales were the result of paid search engine placement and which of natural searches.

Because there are 180,000 report permutations, saving and sharing your work is crucial. I had no trouble bookmarking my custom analysis and adding it to a dashboard. Moreover, I created real-time gauges based on sales thresholds. You can use gauges to quickly refer to a "balanced scorecard" view of site performance and the effect of actions. The system also e-mails alerts when limits are crossed and allows you to roll up information into company-level dashboards suitable for executives.

Another common e-commerce need, building conversion funnels, requires little more than dragging and dropping pages into a report template. What's nice is that you can compare performance over time or look at data from other dimensions without creating variables or performing other implementation work.

SiteCatalyst held up well in noncommerce activities, too. I used the same drag-and-drop interface to monitor e-mail responses, including seminar registration. If you want more precision -- for example, to monitor individual press releases -- you can create custom variables for those pages.

SiteCatalyst integrates well with Microsoft Excel using a direct ODBC connection. This allows a financial analyst to have information about each day's site revenue automatically loaded in a spreadsheet, freeing them from having to log in to SiteCatalyst. The same process allows you to join data from offline sources, such as billboards and direct mail, with Web performance to get an overall picture of marketing effectiveness.

Although not unique, this product's click map overlay -- an Internet Explorer plug-in that transposes data onto the Web page -- will prove useful to content producers. It quickly told me how many people clicked on links and their relative importance. You can also see how many orders come linked and the conversion rate.

Omniture SiteCatalyst 11 is very accessible to nonexperts. Reports are available from the Windows Start menu or Excel. Plus, site and data from other sources is revealed in enhanced dashboards, giving executives an inclusive view of business activities. The system's segmentation and cross-sell features are well done, so users don't have to spend a lot of time analyzing data. Rather, they can quickly spot problems and make corrections.

WebSideStory HBX On-Demand Web Analytics 2.5

Based on the number of enterprise customers, WebSideStory is the largest hosted analytics provider and the first of the pure-play vendors to go public. HBX On-Demand Web Analytics, which captures and reports online visitor and customer activities, is a big part of that success. Although there's danger of not adequately serving these diverse reporting needs (and indeed the product has fallen behind competitors on occasion), Version 2.5 has few failings.

Managers logging in to the HBX service first see dashboards with easy-to-read gauges that measure banner ads, keywords, and other digital marketing KPIs (key performance indicators). It's just as easy for Web developers to implement tracking tags and for marketers to use other parts of the application. So you get the benefit of powerful analytics but in an uncomplicated form.

Because a lot of enterprise clients rely on WebSideStory for content analysis, I began my test by analyzing how visitors navigated my site. A highly customized funnel allowed me to specify page flows and even the specific form field where people abandon a process, such as a seminar registration. Additionally, HBX's special Event Sequence query allowed me to quickly examine the entry and exit points for pages, along with the referral source for those pages.

At a deeper level, reports clearly showed which pages visitors viewed most often and how frequently they visited them. I liked the way HBX allowed me to analyze my site's content according to business organization. For example, I easily grouped visitors to a services area and determined whether they were also going to a section of hardware offerings.

HBX's e-commerce analysis gives merchandise managers real-time reports in an effort to help them increase sales. I easily spotted not only the product categories and brands that sold the most but also those that responded best to promotions such as paid search placements. I could then use this information to build a strategy for selling similar items. Of note, HBX autosenses campaigns (such as pay-per-click keyword buys and banner ads), so managers don't have to perform much system maintenance. Furthermore, HBX provides a number of cost models, including cost per click and cost per acquisition.

Similarly, HBX allows active segmentation. That is, from the browser user interface I dynamically created a segment of users from the United Kingdom who browsed for a solution on international banking but didn't sign up for a local seminar. I then targeted members in that group for a direct mail campaign. This feature adds approximately 15 percent to the service's cost, which is a very good deal compared with the 20- to 100-percent premium others charge for similar segmentation.

HBX On-Demand Web Analytics' mature report-builder Excel plug-in proved to be the best of this group. It allowed me to access every bit of stored data using a wizard, run reports immediately or update figures on a set schedule, and e-mail reports to executives. The company also offers a suite of APIs that allow virtually any third party to access visitor profile data. For instance, the internal search function works with the Atomz hosted service, which enables you to see which terms visitors seek. Furthermore, HBX integrates with, allowing you to follow up on sales leads.

For the price, HBX On-Demand Web Analytics proved to be the best overall choice. It easily harvested great detail about visitors and turned that data into detailed commerce reports, campaign analysis, and navigation measurements. In particular, usability shone, as did the service's cross-channel integration with CRM tools.

Web analytics picks

You won't go wrong with any of these products, but for general marketing needs, WebSideStory HBX On-Demand and Omniture SiteCatalyst are top picks. They combine outstanding presentation, real-time reporting, and data warehousing, plus they approach Coremetrics on the technical side, giving you almost immediate access to a large warehouse of historical data.

WebTrends has overcome stumbles in its earlier versions. Version 7.0 makes it easier to create reports and use the interface, and it includes greatly enhanced e-commerce functionality. Version 7.1, which was released as testing concluded, puts WebTrends even closer to the top with even better visual analysis.

Coremetrics is my pick for the vertical markets it tackles, and it will be a product to watch if the company takes a more general approach. In this commerce area, WebSideStory is surprisingly strong (a quality that gives it the overall top score), with WebTrends 7.1's improved marketing and integration earning it a close second-place tie. Putting Web analytics to the test

What do enterprises ask of Web analytics packages? Here's a list of requirements vendors most often see in RFPs (requests for proposals). These specs formed, in part, the testing checklist for this roundup and can help you decide what's most important in your solution.

User interface -- Provides flexible calendaring and trending -- Matches calendar to fiscal calendar -- Delivers flexible dashboards for key performance indicators -- Exports to Microsoft Excel -- Distributes reports throughout the organization via e-mail on scheduled basis

Site analysis -- Ties page traffic to conversion -- Allows users to group tens of thousands of pages into client-defined content categories -- Presents visitor fallout in a defined path -- Shows paths of specific segments or groups of users

Merchandising analysis -- Groups products into client-defined merchandising hierarchy -- Analyzes cross-sold items within session and across session -- Tracks individual search terms used by visitors tied to conversion -- Segments users based on products browsed, abandoned, or sold and extracts e-mail addresses of segments for retention marketing purposes

Marketing analysis -- Tracks marketing activity across various "attribution windows," for example, visitors who clicked through and converted 14 days later -- Differentiates between paid and natural search -- Automatically classifies links without Excel integration -- Evaluates lifetime revenue of visitors acquired from any link or campaign on the fly

Partnerships -- Works with marketing-campaign management vendors to enable automated tracking of marketing campaigns -- Collaborates with e-mail service providers to enable targeted e-mail campaigns on the basis of site behavior -- Operates with survey vendors to provide pathing analysis based on psychographic behaviors -- Partners with rich-media companies for tracking behavior in online catalogs integration -- Uploads marketing cost, impression, and open rate data from third-party source s- Integrates product cost data for analysis of net sales and margin -- Merges of line purchase behaviors to deliver cross-channel analysis -- InfoWorld (US)

Join the CIO New Zealand group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments