Stephen wrote an excellent post demystifying Twitter Analytics for PR practitioners a few weeks ago, and I thought it would be fun to look at some of the numbers in more detail. So with Stephen's help and the willing participation of a few friendly volunteers, I began my research. What I found was interesting and -- I hope -- instructive.

By Mat Morrison

Reach vs Impressions

Perhaps the most important lesson: don't confuse impressions with reach. Lots of people do, but it could be a dangerous mistake when it comes to your own analysis and measurement.

"Reach" measures the number of unique viewers who were exposed to your messages, while "Impressions" count the total number of exposures. (Frequency represents the average Impressions per viewer, and Impressions = Reach x Frequency.)

Unlike Facebook, say, Twitter doesn't currently report on Reach in either its Analytics or its Ads platform.

While it's almost certain that Reach and Impressions track closely (i.e. as Impressions increase, so will Reach) we still don't know for certain what sort of multiples are involved.

This means, for instance, that we can't usefully make comparisons between Facebook Reach (a hot topic in Q1 this year) and Twitter Reach - because we don't know what it is, only that it's less than our Twitter Impressions.

Replies vs Standard Tweets

Twitter calls any message on its platform that contains someone's screenname a mention; and a mention that starts with a screenname is a Reply. Replies have some odd characteristics of which -- as seasoned Twitter professionals -- you're all no doubt aware. But for everyone else, it's worth saying, "your public Replies aren't as public as you think."

Unlike ordinary Tweets, which can be displayed on all my followers' timelines, my Reply will - in general - only be shown up on the timeline of the person to whom I address it.

If we share mutual followers, they'll be able to see the Reply in their timelines as well.

My Replies will be visible on my profile page (under the "Tweets & replies" tab) should anyone check. And it will turn up in Social Listening Tools like BrandWatch and Netbase. But that's not really how normal people use Twitter.

What all this means is that most Replies receive far fewer impressions than standard Tweets. And, if (like me -- or most brands) you spend a lot of time replying to people, then those Replies are going to skew your data very heavily, leading you to underestimate your impressions per Tweet. And that could be a disaster when it comes to assessing and planning your Twitter activity. But how much of a disaster?

Five of us decided to pool our data; the results (shown below for June/July 2014) were interesting.

	followers ('000s)	mean impressions (tweets)	mean impression (replies)
@damienmulley	16	1,136	143
@wadds	14	1,097	87
@whatleydude	12	1,090	144
@mediaczar	6	709	74
@paulfabretti	5	673	97

Firstly; while there seemed to be a strong predictive relationship between follower counts and impressions on standard Tweets (R² = 0.93), as we might expect, this doesn't hold true for Replies (R² = 0.40)

As Stephen has noted, impressions seem to come in around the 10% mark. As noted above, though -- please don't confuse this with Reach; to make it a bit easier to avoid temptation, I've converted his percentage to "impressions per 1,000 followers."

	followers ('000s)	impressions/'000 followers
@damienmulley	16	72
@wadds	14	83
@whatleydude	12	94
@mediaczar	6	120
@paulfabretti	5	97

Here, it seems that impressions/1000 followers may actually decrease as followers increase (R² 0.99). here are all sorts of possible contributory or confounding factors here, so I wouldn't want to propose this as a rule, yet -- just as a potentially fertile area for research.

Stephen wanted to know about engagement rates; so we've looked at the average daily engagement[1] (total daily engagements / total daily impressions) over the period.

	followers ('000s)	mean engagement (tweets)	mean engagement (replies)
@damienmulley	16	3.23%	4.58%
@wadds	14	1.99%	4.66%
@whatleydude	12	2.26%	3.95%
@mediaczar	6	2.32%	4.43%
@paulfabretti	5	2.57%	4.33%

The first thing to notice is that Replies get much more engagement per impression (somewhere between 1.5x and 2x) than standard Tweets. This is only to be expected; if I'm engaging in a conversation with you, you're far more likely to Reply or even just favourite my message.

The other thing to notice is that there doesn't seem to be much of a relationship between follower count (or mean impression count) and engagement. Either we don't have enough data to spot a relationship here, or -- quite possibly -- there isn't one. On Facebook, we often see what appears to be a positive feedback loop, where more engaging posts receive more impressions. The absence (currently) of editorial algorithms on Twitter seem to mean that this effect doesn't hold true here.

Caveat

Of course, the sample here (friends of Stephen's, only 5 people) is laughably small -- so please take most of what I've said with a humongous pinch of salt.

Except the bit about removing Replies from your analyses; that's just common sense. To give you an idea of how bad the problem is, here's a good headline figure: of the 12,600 Tweets we analysed, 7,200 were Replies -- that's 6 Replies for every 4 broadcast Tweets. Imagine how that's going to skew your results...

For those of you who are interested in reproducing this sort of research for your own clients, I've created a screencast tutorial.

I do hope it makes sense. It's the first one I've done (I'm trying to teach myself how to use the format) so please do let me know what you think. And of course, if you'd like to invite me in to have a chat about how you can do all this for yourself -- you know where to find me.