In my update on the 2008 Tribalization of Business study on business communities that we are doing with Deloitte and The Society for New Communications Research last week - I pointed out how some companies are totally misaligning their measurements of community effectiveness with their goals.

As you will see from the slides, many companies measure effectiveness by looking at page views and time spent on the site. Yet not one company listed ad revenue as a goal for the community - which is what page views and time spent on the site would be good for. Let’s assume that your goal is to have a support community - one in which people can help one another or get help from some your employees. If you could deliver the support in a way that never required people to come to your site, you would still achieve your goals. In fact, if you build your community so that people do not have to come to it, chances are that you will have more people participating in it. There are only so many destinations that a person will visit on a regular basis, and chances that your business community becomes one of them are fairly slim.

Another interesting wrong-headed metric-related finding from the study is that a majority of respondents found that “getting people to engage” was one of the biggest obstacles to making a community work. Now if you have a small community, chances are that you could get a fairly high engagement rate. The larger your community becomes, however, the more its profile will resemble that of large public communities - 1% of hardcore contributors, 10% of active users and 80-90% of lurkers. Now does that mean that the lurkers do not get value from your community? In the case of the customer support community, lurkers who do not contribute could still find the help they need and feel better about you than if they had not found it and also save you the cost of a call into the call center. So measuring community effectiveness by measuring engagement is just not a representative metric of community success.

Now the real issue with all this is that if you have a community development team who is being measured by those wrong-headed metrics, they will invariably develop bad behaviors in order to maximize these metrics. They could in fact develop community features that will stand in the way of success for your communities, or close down communities that are in fact doing really well.

If you missed it, there is a dynamic conversation on managing communities going on right now…Chris Brogan kicked it off and Nancy White wrote some interesting musings and also kept track of many of the other interesting links.

I feel both exhausted and encouraged from this week’s Conference Board conference on Measuring Marketing Effectiveness. Exhausted because the data shows that despite so much talk for so many years about the need for measures and ROI , we marketers have made very little progress over the past 10 years.

A 2007 ANA study found that just 11 percent surveyed said they are very satisfied or satisfied with their ability to determine marketing ROI. A soon-to-be released Conference Board study found that none of the companies surveyed feel as though they’ve “arrived” at figuring out a good way to measure marketing.

Exhausting, too, because creating approaches that provide insights and guide planning - vs. simply measuring tactics — is hard, scientific work. Companies with successful measurement systems, like Eli Lilly, Unilever, MetLife, said it takes at least three to four years to begin making real progress.

The only measure that may matter?

What was encouraging, however, is that marketing measurement innovators believe one approach is particularly valuable: measuring customer preference or perceived value, which are leading indicators of revenue, profits, and cash flow. (In other words, a measure that helps you manage and satisfy the CEO and CFO AND see glean insights to help manage vs. simply measure marketing.)

Don Sexton, professor of marketing at Columbia University believes that this is the most effective measure, yet is missing from nearly every list of marketing measures. (FYI: Don is releasing a book on the topic this fall.)

Other takeaways:

Relationship preference matters as much as product preference

Mark Kershisnik of Eli Lilly believes (has has the data to back it up) that equity can provide a measurement of both investment and performance, and the way to measure equity is by assessing product brand preference AND relationship preference.

I found this especially interesting as so many marketers focus exclusively on product preference, yet customers make decisions, particularly in the B2B landscape, on relationship factors like trust, likability, innovation.

Most common measures are meaningless: lagging indicators vs. leading indicators

Most of the common marketing metrics are, well, useless. Awareness, mind share, perception, recognition, recall, share of market, loyalty, purchase intention, cost per click, etc. may be easy to measure, but they don’t connect to business value nor do they provide indicators of what to do differently to improve performance. They are lagging indicators measuring past performance rather than leading indicators that can help diagnose where to improve brand and relationship preferences and how to monitor progress of achieving marketing objectives.

Focus on just a few things

Many marketers try to measure too many things - 30 or 40 factors. It’s impossible to properly assess that many factors - or have the resources to work on improving that many factors. Many of the speakers recommended focusing on just 3 - 4 product preference factors and 1 -2 relationship preference factors.

Getting on the same page crucial to success

All of those firms with successful measurement strategies have educated their entire leadership team so that everyone has a shared definition of marketing, marketing value, measures and metrics.

The CFO’s mantra

Kamal Sen, director of business analytics and strategic planning for Unilever in Asia, Africa, Middle East and Turkey, offered what the CFO really cares about

• Sales is vanity.
• Profits are sanity.
• Cash is reality.

I was invited to attend The Conference Board meeting on Marketing Effectiveness yesterday. The main theme focused on how companies keep their marketing departments accountable.

Surprisingly, but as most research shows, a majority of companies are nowhere near being able to hold their marketing departments accountable. Not only are some companies measuring the wrong things, a majority of them have no ability to measure anything at this stage. Columbia Business School Professor and Conference Chair Don Sexton further noted that the lack of progress in marketing effectiveness is also visible in the budget process. Research shows that 54% of companies set their marketing budgets based on historical data – clearly indicating that they have no clue on the effectiveness – and only 20% of marketing executives report being able to forecast the impact of a 10% budget cut.

Now in some cases it’s no wonder than companies cannot measure their marketing effectiveness. A new study by The Conference Board, which will be released later this summer, found that half of the companies which reported no progress with marketing ROI had nobody assigned to the task. Duh…

According to Kevin Clancy, another keynoter at the conference, most marketing programs have an ROI that is zero or negative, which could of course be another explanation for why no progress in measuring marketing effectiveness has been made in the last 40 years - people in charge don’t want that dirty secret to be exposed.

One Fortune 50 company which had a few representatives at the meeting had a very interesting approach to marketing measurement, although one that was running into cultural and political barriers. They recognized that marketing is indeed a large multi-variable complex system that needs to be measured as such. You cannot measure the impact of a campaign without also keeping pricing, packaging, and competitive changes into the mix, just to name a few variables. So they hired a bunch of PhD’s in systems dynamics and operational research to measure marketing as a complex system. Unfortunately, and because they lack marketing expertise, marketers are resisting cooperation with them at this time.