I agree that measuring sentiment by hand is a relatively straighforward (if tedious) task (though there’s plenty of ambiguity involved even there), as well as the fact that negative sentiment, in this case, likely correlates with poor performance (as well as a suppressed brand affinity), my larger point here is that metrics like conversation volume (however you label it) even with sentiment are only telling a very small part of the story.

The difference between what people say in a low-barrier environment (like Twitter or Facebook) and what they actually do can be quite large. What matters is the effect (either perception, behavior, or both) that was created in your target audience. If you’re only measuring the conversation, then you can only guess what the result was (and my experience indicates that we’re all pretty bad guessers). More importantly, you won’t be able to compare the results of a social program with that of anything else, so you won’t be able to tell what money that you are spending is paying off, and what is just hot air.

Good luck at SWSX!

Unlike mainstream media, which attempts to stay neutral and often buries it’s POV between the lines, the social media content that makes up the vast majority of this so-called “buzz” is generally very easy to gauge manually. A tweet that says “What was GroupOn thinking?” may baffle automated sentiment analysis software but anyone past a 6th grade education will immediate recognize it as negative. To measure sentiment in a statistically relevant way, you really only need a sample set of about 500 mentions, which will take an intern about two hours to score accurately. Everyone is looking for an easy way to do this but the hard way just isn’t that hard.

“Buzz,” as it’s now commonly used, is really just a cheap way to label volume of mentions. It made sense 10-15 years ago when it was impossible to get the actual data but now that it sits in our hands I don’t think there’s any excuse to keep using it.

The GroupOn spots created a high volume of negative sentiment towards a brand that was making tremendous progress in awareness and that’s why, in my opinion, the Super Bowl executions were a failure.

Coincidentally, this is one of the topics I’m discussing at my SXSW panel next weekend. I’m going to attempt to make a case that “buzz” is one of the words that no longer has a place in digital marketing. Wish me luck.