I’m confused. But maybe… that’s the point.
If you’re new to this experiment, the question I asked was simply, “Does tweeting at a conference affect your Klout Score?” (For more background, view the original blog post.)
Why do we care? As association communicators, we
want need to know the influencers of our industry. And key to that, is knowing who is sharing relevant industry information and what their effective reach is.
Associations also typically have a cycle of knowledge creation focused around annual meetings and other conferences. And a few key questions that can be asked include “What is the shelf-life of this knowledge?” and “How can we spread this knowledge beyond the most engaged members that are here?”
What better way than to test this in our own industry by measuring the statistics provided by Klout.com – a service using social media analytics to measure one’s influence across various networks and bills itself as the “Standard for Influence.”
So given our industry’s biggest conference of the year. With lots of learning, networking and information sharing going on. What effect did all this tweeting, Instagramming, sharing and such have on our supposed influence?
The answer is… not much.
14 people participated in the experiment (admittedly, not statistically huge)
The average pre-conference Klout score was 56.29 (out of 100)
The average post-conference Klout score was 57.92 with a net shift of 1.63
The biggest score increase was 8.63 points by Linda Chreno (@lindachreno)
The biggest score decrease -2.21 points by Pamela Strother (@SponSpecialist)
Overall, this represents a pretty small shift among the participants. However when looking at a few specific scores, some interesting data points pop out.
The Klout Style of 86% of the participants changed after the conference
Did that burst of activity make Klout think we changed our personality?
The amount of Twitter activity at ASAE 12 seemed to have no influence on the score.
I sent approximately 60 tweets over the 3 days I was at the conference and my score went down 1.18 points
Sandra Giarde (@sgiarde) sent approximately 62 tweets during her time at the conference and her score went up 6.36 points
Maddie Grant (@maddiegrant) sent almost 100 tweets and her score went DOWN 1.08 points
So what do I take away from this? What should we keep in mind?
Social influence must be measured over the long term
This experiment was conducted around our industry’s biggest event of the year and captured a blink in the timeline of our online interactions. Yes, new influencers may emerge during an event like this. But when you have an established group of social influencers, the overall makeup of the social sphere isn't going to change radically over one event.
Interestingly, while the small results show that a flurry of activity doesn’t move the needle too much in the positive direction, a lack of activity will negatively impact one’s Klout score. I experienced this when I was on vacation in June and was off of social media for two weeks. Other participants commented on the same experience.
A single “measure of influence” doesn’t capture true influence
My experiment participants represent some of the most prolific commentators in associationland including Maddie Grant (@maddiegrant), Jay Daughtry (@chatterbachs), John Chen (@johnyschen) and more. Yet, the net shift was “meh.”
Even more important, is that this single measurement is incomplete. The best example is to compare my influence that of ASAE's CIO Reggie Henry (@reggiehenry).
The chart below shows how our two Klout scores changed over the past 30 days (my score is red, his is light blue).
Klout Scores of @rvanhilst and @reggiehenry
Note that I actually had a HIGHER score than Reggie before ASAE Annual and after 30 days he and I are neck and neck. According to Klout we are even influential on the same topics of media, marketing, technology, speakers and smartphones
So I’m calling “bullshit” on this one. First, there is no way I have ever come close to being as influential Reggie (let’s face it, he is “Mr Technology” in our world). Second is the fact that he presented at ASAE annual and was quoted, retweeted, photographed and warmly greeted by all who know him. I merely attended and tweeted other people’s thoughts.
Yet, he and I have the same level of social influence? I don’t think so.
Continue to “experiment in the open”
The social media sphere of associationland continues to evolve. From the first bloggers huddled together years ago at ASAE Annual to today’s tweet-ups and people meeting for the first time IRL (“In Real Life), the tools and techniques for social engagement continue to change.
Some of the comments from my fellow experimenters reflect these shifts:
“The growth in Tweeting from ASAE11 to 12 seemed like a big jump in the diversity of ‘conversations’ happening in any given moment. It was moving so fast it was hard to jump in.”
“Coolest thing for me was finding and meeting all of the people that I have been tweeting with for the past two years and meeting them in person. Also having a conversation in more than 140 characters.”
As change agents for our industry and our organizations, we have been given a laboratory to play in. We should embrace it and be willing to try new things.
And I think that is the best conclusion that can be drawn from this experiment.
Note & Disclaimer: As some people pointed out, Klout changed the scoring method during the experiment. For everyone that participated, I looked up their profile on Klout.com and recorded their scores on August 7 and August 21 capturing the 2 weeks around the conference. I overlaid it with their feedback and participation data to get these results. Also, Klout is now rolling out their new look. So finding users and this data again will be difficult. So take this all as "one bloggers opinion based on directional data." But I still think it's pretty accurate.
I guess it also means the other lesson learned is to be ready to adjust your plans when these “free” tools change things on you.