Here’s a scenario I come across much more frequently than I want to:  In a meeting discussing a new marketing campaign at a medium sized corporation, a typical marketing manager will say something like “We should put the call to action above the fold and to the right of the graphic”. My response (other than a silent internal sigh) is always along the lines of “How do you know that’s better?”

All too often their answer will come down to one of three things, either “I read somewhere that…”, or possibly “At my last company we…”, or my favorite, “Our customers want…” All of these answers are bogus, probably wrong and quite possibly harmful to the potential success of the campaign and also the company’s business. If they were really being honest, in most cases what this person should really say in response to my question is “I don’t know.”

The reason is that many otherwise smart people don’t use hard data to verify their assumptions. They don’t test those assumptions by running comparisons of the different possible approaches, or they don’t use the data they collect usefully, and they almost always don’t know their customer audience as well as they think they do.

The reality is that people are fickle and unpredictable, particularly customers, and particularly customers on the Web. What worked for one company and one product might not work for another; what worked last month might not work today; and how you think users will behave is not always intuitively obvious. That’s why it’s critical to always test, measure, and analyze any assumptions you make about your customers.

Of course in reality there are certain general principles about website layout and campaign management which hold true in most situations, most of the time (particularly with large, well-defined audiences), and experienced web marketers and designers know this. But it really doesn’t hurt to be sure. More than once I’ve seen seasoned web professionals eat their words when an analytics report disproved their pet theory… They didn’t know what they didn’t know, and some cases what they didn’t know can (and does) hurt them.

Part of the problem is that most marketing folks aren’t scientists and aren’t trained in the use of the scientific method, so the idea of constructing an experiment to determine the best ways to persuade their customers either doesn’t occur to them or is sufficiently daunting a task that they don’t know how to approach it effectively.

Another, bigger problem is that many of the software tools out there supposedly designed to help users test and measure website results are complex and ugly, and are really aimed at highly-skilled technical users with statistical and programming expertise (not a description of the average marketing manager). If it isn’t easy and obvious to obtain the analytics you need to make good decisions, then you’re probably making bad ones.

What’s needed in many companies and marketing departments is a mindset that it’s great to be creative and think outside the box, but always test (design A/B tests for all your assumptions), measure (using web analytics), and analyze (use analytics software that’s easy to use and presents information in a digestible form). Most importantly, use the analyses you create to drive your collective marketing and website decision making, so you’re always on solid ground and maximizing the effectiveness of the dollars you spend on both.

Because otherwise what you don’t know can hurt you… and it will, if you’re not careful.