At 10,000 feet, Web analytics attempts to answer questions about what your Web visitors are up to, and why your organization should care. This post attempts to look at the who and the how of Web analytics, including the rewards and barriers to action. It also reviews some practical tips on where to begin, along with some ideas on what a model dashboard could look like (because presentation matters).
From the point of view of Web visitors, Web analytics attempts to answer questions such as:
- Why are they coming?
- Where are they going?
- Where are they coming from?
- What are they doing?
From the point of view of the organization, questions cover:
- Why do we care?
- What are our goals?
- Is it working?
- What should we do?
The problem and the challenge is that Web analytics gives you an ocean of data. Frankly it is overwhelming. The data is rich and deep, and only a click away. But what does it mean?
THE WEB ANALYTICS ACTIVITY CYCLE
In an ideal world the Web analytics activity cycle has four parts.
Discovery. First, it’s all about discovery. In other words, what is the business/Web mission, purpose and/or goals? How mature is your Web analytics team – the who, what and how about your program. A good discovery session also explores the organization’s pain points and challenges – do you have agreement on priorities?
Translate Goals into KPIs. After you’ve gotten your team on the same page about these issues, its time to translate your Web goals into Key Performance Indicators, or KPIs as they are commonly called in the industry. I’ll discuss some of my favorites deeper into this post. There are some interesting resources out there to see what others use, including the KPI Library.
Setup and Configure Tools. With your KPI picks in place, then it is time to set up and configure your tools. A free place to start is Google Analytics, paired with a heat-map tool called Crazy Egg. If you are feeling green, Google Analytics offers some good online training. There is also a good screencast describing how to use Google Analytics produced by Beth Kantor that I recommend. To meet vendors and practitioners in the Web Analytics space, consider joining the Web Analytics Association which has begun offering a Web Analyst certification exam and offers regular Webcasts for members.
As a Web analytics manager be ready to offer some training to those on your team about the basics to set expectations. Also be sure to get a historical baseline so you can compare your growth to something real. When you configure your tool, make decisions about whether you are going to count internal traffic or configure the analytics platform to exclude internal visits and just count external customers.
There are a lot of handy tools for Website analysis out there, some of which are free and others that charge thousands of dollars a month for very rich data streams. What has been a real game changer, however, is Google Analytics (formerly known as Urchin before Google acquired it). When Google released the free version to help create a class of informed customers who are (hopefully from Google’s perspective) purchasing ads on the search engine, it unleashed the ability for any Web site owner to see an incredibly deep amount of behavioral data about online visitors. The Google Analytics dashboard lets you see site usage, a geographic map overlay, a content overview and traffic sources among other things. In fact, the map lets you click into a state and even view which cities are sending visitors to your pages.
Analysis. The last and most important part of the activity cycle is the analysis. Simply sending a spreadsheet or dashboard link won’t do if you are serious about KPIs. Instead, pay careful attention to trending data — current versus baseline data. Establish your insights from the data, weigh the impact of making changes and manage for quick wins and long-term focus alike.
ADOPT AN ACTION ORIENTATION
Before we go further, its important to discuss the elephant in the room. You must recognize and grapple with analysis problems. What are you measuring? Is it subject to misinterpretation? Or do you struggle with a hurricane of demand for data that swamps your ability to do analysis to begin with?
You may also suffer from some management headaches. Your executives may have low engagement with your activities. It could be that your rhythm is to give them steady reports by email and there is simply never any action on the data. Missed opportunities are frustrating. And if you are ultimately in charge of Web analytics, you will have to set priorities for your staff’s time and resources because nothing is ever available in an unlimited supply, in spite of good intentions to deliver the gold.
If you run your Web analytics work out of your marketing department, chances are your job is very complex. A McKinsey study shows how these marketing manager/executives spend their time on a variety of Web related activities. “An explosion of customer segments, products, media vehicles and distribution channels have made marketing more complex, more costly and less effective,” the authors wrote. The time study shows each day your Web related activities include:
- 15% – on Web development and maintenance work
- 10% – on search engine marketing
- 5 % – on search engine optimization
- 5% – on email marketing and direct mail
If you choose to focus your limited attention span on Web analytics, rewards await.
You gain deeper understanding of your site visitors. You can improve user experience. You get the opportunity to make smart decisions and improve resource allocation. And you get the ticket to a greater return on your Web investment. These rewards come only if you have adopted an action orientation.
An action orientation for your KPI program starts with developing your selection criteria. The KPIs must be relevant, timely and instantly useful. Resist the urge to showcase everything and start small. For example, just five data-points — traffic/reach, search, satisfaction, actions and outcomes — can be a satisfying beginning to your work. Once you have the KPIs locked in, the next critical step is scheduling reports. I recommend not less than quarterly. Reports should also keep the audience in mind and be edited for their particular needs and attention span. Finally, do not neglect meetings. Sending out an email link will not do if you want to a reputation for action.
The action orientation follows a cycle of its own — identify the problem, the metric, your hypothesis, fix the problem, test the fix, analyze the results and take action to modify the fix as needed. Then the cycle starts anew.
MAKE THOSE KPI REPORTS AUDIENCE-CENTRIC
Making your KPI reports audience centric is one of the keys to success. At the AF Portal, for example, we identified three key groups who needed regular KPI reports.
The first group – senior leadership – received a culled down executive size report of no more than three metrics. The idea was to have everyone on the same page with something that was easily digested and summarized and could fit on an index card in a pocket if desired. For our executive KPIs we selected reach, search and satisfaction.
The second group was content managers for the 20 Major Commands at the USAF. These team members had responsibility over the growth and maintenance of the AF Portal and its adoption at the headquarters command and the various Air Force Bases at that command. They also worked to disseminate policy, seed adoption and all the other day-to-day operational details of running their Intranet. For this group, we identified seven KPIs that mattered. They would get these metrics on a routine schedule. These KPIs included the first three the executives got, plus four more: top entry (to capture popularity), feedback, page influence (to get at top editorials/stories), and savings (to monetize the top ten downloaded documents, which helped the team get their arms around return on investment and changing the business of publishing content).
The third group was the AF Portal team, responsible for the day-to-day operations of the enterprise effort. Here, team members got all seven metrics that the Content Managers got, along with three more: adoption (to capture registered users), loyalty (to get at time per page and trends) and help desk (where we pinpointed the metric to problem types only, although other help desk metrics were circulated to another team more focused on that part of the effort).
In identifying the KPIs that you will share across your organization, it is vital to link it to strategic questions that get at the overall goals and challenges your organization faces and answer the burning questions you have about whether you are effective. For example, a set of conceptual KPIs for one organization included the following five:
- Reach: does our audience know about us.
- Relevance: are we providing what our key audiences want?
- Packaging: is the information we offer in a consistent and usable format?
- Access and Collaboration: are staff experts made available and used?
- Quality: are we delivering a superlative Web experience?
LEAP INTO THE FUTURE
If you are in charge of your organization’s Web analytics program, and you want to leap into the future, you must negotiate agreements and hold routine voice-of-the-customer meetings. The agreements should cover the format of your reports, who owns which metric (spread the love!) and agreement on the audience segments who will receive your metrics reports. The meetings will be critical to your success. Do not fall into the rut of pressing the send button and thinking your job is done. Hold meetings not less than quarterly with all the right stakeholders invited.
One last part of your job which will impact your success is how you choose to present your data. Style matters.
It is important to include trending, so users of the data can compare results over time. The reports goals should be to make the data actionable, hold owners accountable, improve take-aways and seed discussions that will help you and the organization reach decisions about how to manage your Web site. I have found the best format to reach these goals is a quad-chart style dashboard. This is a tip I picked up at a Web analytics conference from a presentation led by Avinash Kaushik, then with Intuit and now with Google.
Avinash’s model dashboard is easily created within a Powerpoint slide. The headline is the name of your metric. At the top, you include a red-yellow-green dot to show the health of this metric, along with the name of the person in your organization responsible for managing the health of this part of your Web operation. The quad chart itself includes these four parts: in the upper left you present the data in some sort of chart format ideally. In the upper right corner you explain in plain English what that data is telling you – your insights. In the lower left corner you write your bullets for what you are recommending as actions based on the data’s story. And finally in the lower right corner you remind the team what actions you SAID you would do at the last meeting and what the status is of where you are on those changes and action items.
I like this dashboard format a lot. It simplifies the data into a complete story-board that helps the team talk about it and take corrective steps if necessary. It holds the owners feet to the fire – nobody wants to be a red-dot for long. And it helps everyone see clearly what the to-do list is (or can be), new and old.
PUT YOUR KPI QUAD-CHART INTO ACTION
Putting this into action at the AF Portal, we started with surfacing the key questions we wanted to use the data to measure.
- Are we paying attention to the right content? Which top-level navigation button gets the most clicks?
- What is the status on migrating applications (a key business driver for the AF Portal – that applications move to a common platform for which the AF Portal was the presentation layer)?
- Are we focused on applications our users care about?
- Are the complaints about search justified?
- Of all the new functionality we are introducing, which features are getting used and where must we focus our outreach efforts?
Our KPI on traffic and reach answered question one. We showed our team the top six tabs using unique visitor data and page view data, in raw numbers and percentile format. The results showed us that although most of the team’s effort went to building out the organization pages, the top visits to the AF Portal were on the customizable workspace pages followed by the LIFE pages which HQ’s managed. The first time this quad chart debuted, the action items were about seeding discussion on how the team “flying blind” in managing its resources and expectations for growth prior to receiving regular KPI reports.
A second KPI on traffic, focused on applications, helped us address our question about migrating applications. By clustering the applications into master categories by application type – self-service, mission-based, training or content related – we were able to shine a light on the success of the drive toward self-service applications as a replacement for many walk-in, phone-in services at the USAF. A need to look at the way the feedback loop operated with these self-service applications was also brought into focus by the data showing their popularity. With regard to the training application group, the need to simplify and unify how users found these tools was an active discussion item which emerged from this presentation of the data.
Looking at the search KPI was also instructive. Because it was a distributed publishing model at the AF Portal, we were able to isolate the business unit which owned the content which was showing up on most of the searches and work with them to optimize search. The results validated that search quality was a valid complaint and spurred action to give better tips and training to the content manager community.
Our metric on adoption showed the raw tally of how many users subscribed to various features new to the AF Portal platform. Marketing was inconsistent: some new features received no promotion while others were released with great fanfare. The need to do more consistent and full outreach on these capabilities came into focus. In addition, it sparked a discussion on tracking measurable goals for adoption and the need for a communications plan. One diamond in the rough that we saw was not adequately embraced at the time was Really Simple Syndication, which was relatively new at the time.
SUMMARY
When starting up a KPI program, Web Analytics managers can easily drown in an ocean of data. A well-constructed KPI effort is your lifesaver. If you are already in the game, this post was designed to suggest some new tricks to add to your current metrics routine.
What KPI program are you ready for? Whether a novice or an old-hand, this area of website management promises to continue to develop and mature.
NOTE: All of the quad-chart style dashboard slides covered in this post are illustrated in my Slideshare KPI presentation embedded below.