"We have the data, we have the pretty reports. But we do not have an educated team who can take the resulting statistics and turn them into meaningful plans. The process of creating Web strategy and executing on Web tactics has gone missing."
Besides being fun to say and a legitimate excuse for me to wear my "Mind The Gap" hat purchased the last time I was caught in the rain in Covent Garden, the Feedback Loop Gap is the most pernicious problem faced by those tasked with running and analyzing the success of large, sophisticated Web sites. It all begins the day a business person stumbles upon a Web server log.
Your Web server generates a mountains of data collected in your server logs every day. The assumption is that this information is generated to help you determine how well you're doing. Nice thought, but not the case.
Log files are merely the result of good, solid engineering. If you create a system that does something, it's a good idea to record what happened. That way you've got something to look at when things go wrong. Log files are, therefore, the results of a Web server doing its job, and not a formal effort to capture valuable business intelligence.
The business management side of large companies went to the Web-heads and asked for reports. The log files were available and delivered. Ever since, mankind has been trying to interpret log files like astrologers, phrenologists, and readers of the I-Ching. Log files do, indeed, contain more data than, say, the configuration of the bumps on your head, but it's a matter of torturing usable information out of them.
Here then, is the log file torture process practiced by multitudinous multinational corporations:
1. Capture Log Data
Everybody's happy to lasso the data that Web servers spit out on a daily basis. It's just a matter of file storage - rather mundane, actually. And it gives everybody hope.
2. Cleanse the Data
Some data are useful, some are not. Some are informative, some are confusing. It usually requires a couple of rounds of attempted comprehension before the dirty data become obvious and can be expunged. Traffic from within your own company and visitations by spiders and robots are the first to go. Then you need to decide if your one-page visits are worth the bytes they're stored in.
3. Report
Nice, crisp, colourful charts and graphs adorn the walls of technical and marketing offices. "Look! We have an active, interactive Web site and people are doing things there!" These charts are pretty to look at. They are proof that people are paying attention. They are useless.
4. Analyze
At some point, one of the technical committees squints at a series of reports over a several months period and begins to wonder why some numbers are going up, some are going down, while others do not change in the slightest. They wonder what external forces are at work. They scratch their heads and come up with seventeen technical explanations for the change over time of the appearance of the pie charts, bar charts and diagrams. Twelve of these explanations are interesting and two of these ideas are worth chasing.
5. Deliver
Convinced that they have discovered something important, the technical team calls a meeting with their business compatriots and presents the findings. They expound on the statistics and the resulting graphical depictions in terms of how many people showed up when, how often, for how long, and to what end. The marketing staff are thrilled to receive actual data - hard evidence that the work they've been doing has had an effect.
The End.
Notice something missing?
Several things in fact?
Here's what I looked for and found only in the rarest organizations:
6. Interpret
Technical people and business people gather enough knowledge about Web statistics to actually understand what the charts, graphs and diagrams really mean.
7. Plan
Based upon a thorough understanding of the reported results, a team made up of information systems people, Web designers and business people map out the changes necessary to improve usability, increase customer satisfaction, and boost revenue. Armed with an understanding of site visitors' desires and frustrations, new designs, new content, and new applications are devised and developed.
8. Execute
Select the best options which will result in the greatest impact with the lowest investment in time or money. Secure upper management support. Create a timeline. Instill ownership. Charge ahead.
9. Repeat
Capture, measure, and analyze the results of the changes and incorporate those results in the next planning cycle. Ad infinitum.
The problem facing most companies falls between items 5 and 6: the dreaded Feedback Loop Gap. We have the data, we have the pretty reports. But we do not have an educated team made up of technical and business people who can take the resulting statistics and turn them into meaningful plans. Feeding Web server statistics, customer satisfaction surveys, sales data, etc. back into the process of creating Web strategy and executing on Web tactics has gone missing.
The technical side doesn't have a view into of the company's business drivers and the business people don't know a) what the data mean, nor b) what valuable information might additionally be available.
We need to sufficiently educate those controlling the marketing and customer service budgets so that they can interpret the data they get. At best, they will understand the metrics reports well enough to make well-informed plans for the next step *and* instrument the execution of those plans so that the right information is gathered with each iteration: the Feedback Loop.
We can only hope. In the meantime: Mind the Gap.
Thanks Jim
Jim Sterne is our first guest blogger on adCenterCommunity.com - among other accolades he produces the eMetrics Marketing Optimization Summit and is Founding President and current Chairman of the Web Analytics Association
Get Social: Join us on our Facebook Page & follow us on Twitter: http://Twitter.com/adCenterBlog