Tag Archives: Klout

Measuring the Immeasurable

Posted on by
6

My post last week about the shortcomings of Klout got several thousand views and generated quite a bit of discussion. it also got me several e-mails from companies that claim to have built a better mousetrap than Klout. I haven’t reviewed these tools in detail just yet, but it appears that influence is a red-hot topic in PR and marketing circles right now.

Influence measurement is a natural evolution of conversation monitoring, a discipline that’s personified by Salesforce.com’s Radian6 tool and dozens of competitors. Monitoring is a solid practice that can keep you in touch with the topics and brands people are discussing online. Most tools now also provide some degree of sentiment analysis, which attempts to derive attitudes from comments. Sentiment analysis is devilishly difficult to get right, however. If a teenager calls something “sick,” it’s a compliment. Coming from a 50-year-old, it’s an insult. Most experts I’ve spoken to on this topic say that sentiment analysis tools are at best 70% accurate.

Circle of Influence

Topaz Partners has developed the "Circle of Influence" to depict the different factors that go into decision-making (Click to enlarge). TopazPartners.com

This isn’t stopping vendors from tackling the even more complex issue of Influence analysis. This goes beyond sentiment analysis to attempt to determine a person’s ability to drive action. The problem is that there are lots of variables and intangibles to influence that resist being boiled down to a single number. For example:

  • What is action? A “like” or retweet is a form of action, but not necessarily one that leads to a decision.
  • Online actions have different gravity depending upon the stakes and the effort involved. Writing a comment takes more effort than clicking a “like” button. Posting a blog entry referencing someone else’s words is more involved than writing a comment.
  • Which actions really matter? I have yet to see a tool that can correlate influence with purchases or donations with any degree of certainty. We assume that conversation about a topic influences decisions, but are they the decisions we want? A lot of people have been talking about Hewlett-Packard lately, but I doubt it’s driving profitable sales of HP products.
  • Influence is contextual. If I’m considering buying a Yamaha stereo and find a blog entry from someone who exhibits deep knowledge of the model I’m considering, that person may have a disproportionate influence on my decision, regardless of the number of followers or subscribers he has. The weakness of most influence analysis tools is that they abstract broadly, looking at things like reach and amplification. However, decisions are more likely to be influenced at a micro, rather than a macro level.

One of the most illuminating books I’ve read on this topic is Influencer Marketing by Duncan Brown and Nick Hayes. The authors argue that the influence of media in general, and social media in particular, is greatly overrated. They count no less than 50 kinds of influencers, ranging from resellers to academicians to government officials. Most of them have little or no online visibility, but their knowledge, leverage and/or connections make them enormously influential. What’s more, the larger the purchase, the greater their influence tends to be.

I don’t agree with everything Brown and Hayes say, but I commend them for resisting the urge to oversimplify. Their basic message is that influence and audience are two different things. Celebrities can have huge audiences but little power to affect decisions. Conversely, people with very deep knowledge can have small audiences and great influence. Seth Godin said it well: Small Is the New Big.

In the mainstream media world, audience was associated with influence because we had few tools to understand the true dynamics of decision-making. Our natural tendency is to apply this same metric to online conversations. The danger of this approach is that social media is more about quality than quantity. In the same way that early automobiles had steering mechanisms that mimicked reins, we are applying old assumptions to a new medium. I’m not saying that influence measurement tools are inherently unreliable, but they are attempting to measure what may be immeasurable. Just be skeptical.

The Trouble with Klout

Posted on by
38

Estimating influence is a delicate balance of art and science. People are drawn to quantitative methods because scores are easy to understand. The downside of reducing influence to a number, though, is oversimplification.

Paul Gillin's Klout InfluenceLately, I’ve been looking at Klout, the popular new tool that bills itself as “The Standard” for influence measurement. The more I look at it, the less I like it. Klout’s weaknesses have not stopped it from amassing an impressive list of more than 3,000 business customers and from being incorporated into popular applications like HootSuite as a standard metric. It is “the emerging standard” for measuring influence online, said Klout Marketing Manager Megan Berry in a podcast interview with Eric Schwartzman last month. I just hope those clients aren’t taking this metric too seriously.

Beyond Followers

Klout attempts to determine influence metrics by looking at a person’s online activities and the actions of others that result from them. The thinking is that influence isn’t a matter of how much you say as much as the impact your words have on others.

Many people have a Klout index and don’t know it. The service crawls Twitter and ranks members automatically. If you want to grow your score, you can log in to the site and give it a bunch of information about your online activities. I spent 15 minutes on Klout registering my social networks and grew my score 10 points on the spot. This is a major flaw in Klout, but more on that later.

Klout uses a proprietary algorithm to estimate influence based upon comments, retweets, @replies and mentions, among other things. The company isn’t very transparent about how it calculates the score, and with good reason. The algorithm is a competitive asset and disclosure would inevitably invite people to manipulate the system.

The downside of opacity is confusion. By revealing so little about how its ratings are calculated, Klout essentially asks customers to put their faith in the service to do the right thing. This is dangerous, given Klout’s flaws. Nevertheless, the score is a public record that anyone can see, and its influence is growing to the point that Klout scores are now reportedly showing up on resumes.

The Shirky Effect

Clay ShirkyThe problem is that some of the ratings are nonsense. For example, my Klout score (66) is modestly higher than Clay Shirky‘s (60) and significantly higher than Marc Andreessen‘s (42). This is ludicrous. Shirky (right) is the author of two influential books about online sociology and has been a thought leader on the Internet since the mid-90s. Andreessen (below left) invented the browser, cofounded Netscape and is one of the fathers of the modern Internet. Both are sought-after speakers and the subject of extensive Wikipedia articles. Yet Klout says I have more influence.

Marc AndreessenThe problem is that neither of these brilliant innovators plays by Klout’s rules. They aren’t active on Twitter and they don’t have Klout accounts. The fact that a single post on Shirky’s blog can draw more than 1,200 comments or that Andreessen’s occasional writings appear in The Wall Street Journal is of no consequence. Klout doesn’t monitor either of those outlets.

Klout’s bigger flaw is that its scoring system is tied to membership. The more you tell Klout about you, the higher your score is likely to be. This linkage fundamentally undermines the quality of the service. In effect, Klout pays you to endorse its service by rewarding you with a higher rank. If Google did that, Congress would be holding hearings.

A Million and One Improvements

Klout admits that its methodology isn’t perfect. In the interview with Schwartzman, who is the co-author of my B2B social media marketing book, Megan Berry said the company has “a million and one” improvements it wants to make. Schwartzman pressed Berry hard on shortcomings in the Klout methodology, and her responses were a weak defense. In essence, Klout treats every social network the same and all interactions equally, she said. A retweet, which is a one-button operation, is just as good as a thoughtful commentary on a blog. Except that Klout doesn’t currently monitor blogs, other than those on Google’s Blogger service. That must be one of the million-and-one improvements in the pipeline.

Megan Berry on KloutA comparison of Berry’s and Schartzman’s Klout profiles showcases the service’s flaws.Berry’s Klout score as of this writing is 70, while Schwartzman’s is 60. Barry does have a couple of thousand more Twitter followers than Schwartzman, but she said Klout ignores follower metrics as meaningless. Berry is very active online, but not nearly as active as Schwartzman.  Her blog has been updated eight times this year while Schwartzman has posted 36 episodes of his popular On the Record…Online podcast and more than 30 entries on his Spinfluencer blog. Berry contributes occasionally to Huffington Post and Mashable, but Schwartzman is also active outside his own channels, contributing to Social Media Today and For Immediate Release. Schwartzman has 44 recommendations on LinkedIn, while Berry has three.

Eric Schwartzman on KloutAs far as I can tell, there are two principal reasons why Berry outscores Schwartzman on Klout. One is that she knows the system. She has at least a vestigial account on every social network that Klout cares about, whereas Schwartzman limits his activities to fewer outlets. Berry also tweets regularly on behalf of her employer, giving her Twitter account a Klout halo effect that attracts retweets and @replies.

My intention isn’t to pick on Megan Berry. She’s obviously a bright young woman who’s very savvy about social media. However, there’s nothing I can find that qualifies her as significantly more influential than the veteran Schwartzman, not to mention Marc Andreessen.

In her interview with Schwartzman, Berry described Klout as “[Google] PageRank for people.” In my opinion, it’s got a long way to go. Klout has some utility as a way to compare the online presence of active social media users, but measuring influence is much more complicated than counting retweets and Foursquare tips. Klout is betting that it can use its metrics to entice (coerce?) people to join its social network, which it can then monetize through advertising. The link between membership and Klout score is a disturbing weakness. Proceed with caution.