Social media monitoring: absolutely pointless

Culture / Digital / Geek / Innovation / Public Relations / Social Media

Tags: digital measurement, Measurement, Measurement tools, monitoring social media, monitoring tool, Public Relations, social analysis, Social Media, social media alerts, social media monitoring, social media ROI, social media tools, social ROI

July 27, 2010 View Comments

Social media monitoring is a complete waste of resources. Total waste of money, time and understanding.

Yep.

You know why?

I’m guessing by now you’re either really intrigued or really angry. Hopefully both.

Social media monitoring is completely useless without context or clear outputs.

What are outputs? Your next steps once you capture something.

What is context? Context means understanding the data, turning data to information. Something meaningless to something useful.

Please read on, let me elaborate.

Let’s take an example: you’re on the board of BRAND Z, you decide that after fifteen years of traditional activity you should probably look into social media, but where do we start?! ARRGGHHH!!! TOO MUCH INFORMATION??!!!

*ring ring*

“Oh, hello social media monitoring service! You only charge £7 a month? And I get shiny graphs?! Where do I sign?!!!”.

Here’s why I’m so irked.

(Now my next point is going to irritate you if you’ve read a few of my last posts because it was born from the Nassim Nicholas Taleb book Fooled by Randomness. Which I appear to quote for everything. Even in Tesco.)

Fooled by Randomness is largely about the unpredictable being predicted and the uselessness of models, and Taleb’s context is the financial markets. Which doesn’t sound too transferable to ‘digital’ but a few chapters in he begins to discuss the probability of successful analysis at different rates for monitoring share price and how minute changes shouldn’t effect your overall view of the investment opportunity – basically, always take a long view on big issues.
Think about this in terms of a boat on the waves, just because it’s at the bottom of a wave doesn’t mean it’s going to sink.

So Taleb then pulls together a probability of success at different rates table;

One year – 93% probability of successful analysis
One quarter – 77% probability of successful analysis
One month – 67% probability of successful analysis
One day – 54% probability of successful analysis
One hour – 51% probability of successful analysis
One minute – 50.17% probability of successful analysis
One second – 50.02% probability of successful analysis

The point that he’s trying to make is that people are effected too much by minute by minute changes and cannot comprehend the bigger picture. The same can be said for monitoring; of course it’s important to look at ‘real time’ alerts and understand what’s being said, but it’s much more important to understand how this fits into the wider context.

If you cherry pick blog posts about BRAND Z without giving them context you’ll have an almost completely random view of the brand – monitor every blog post and hold monthly reviews and you’ll begin to gather insight, do this for a year and you’ll have a much more clear idea about your reputation online.

The point of this post is not to make you angry, but to highlight that monitoring only works if two criteria are fulfilled;

a) There are a set of clear next steps to take – listening is fine if you know how (or if) to react

b) Always look at the macro, not the micro and always, always, ALWAYS give your monitoring context

I know this is quite a long, ranty post, but if we’re to move the ‘digital’ industry along we need to be be more intelligent about this stuff.

*quick update: this post has also been cross-posted on The Wall so there may be some interesting comments there too!*

Steve Cranston

Really solid thinking. Without context we're monitoring chatter and noise, for the most part. This is largely why the qualitative and human analysis Synthesio provide comes closer to what Brand Managers really need in order to take actionable steps. Bravo, nice post.
Steve
Jon Moody

Indeed, none of us is as smart as all of us (or at least some of us together). Despite the controversial headline, a good part of the value of the post (like many) is in the subsequent contributions.
thomasmessett

To my mind: nothing is useful without context, if I tell you I have sold 3 laptops today is that good or bad? Well without any context it means nothing, but if I tell you that the last week I have sold 1 a day then suddenly it means more, if I then tell you that the average for my whole region of salespeople for the last 6 months is 0.7 a day it looks even better. Context. As people we are always seeking out context, we compare ourselves, our peers, our business and many other things, we compare ourselves to our own goals, where we were a year ago, where our friends are etc. etc. We always seek context. That is not a problem. I used to conduct demonstrations of Radian6 for 6Consulting, I rarely met a brand or agency in that time who didn't want context (well there were a few, but as a general rule) both in terms of comparative analysis and in terms of long term trends.

However SMM does have many problems, to my mind a couple of these are with really understanding data, data integrity and over reliance on machines. Very few people in this industry truly understand regression analysis or much beyond basic cause and effect, too few know how to set up full Boolean searches or how SMM tools capture data and SMM tools themselves have a very long way to go, for example: the same terms entered into different tools will often capture very different results and trends due to different coverage strengths of the tools themselves, worse than this: if the person setting up the tool does not have an intimate understanding of it's strengths, weaknesses and mode of data collection you end up with a very messed up dataset from the start. Also, Spam blogs and spam tweets often create huge false positives and problems with getting to really good, relevant info – this is fine if you have a human hand to go through and check datasets, but few people budget properly for this, hence: over reliance on machines.

Too many agencies, and brands, are just "phoning it in" - buying tools they don't really understand and extrapolating results based on flawed data. They all seek context and comparison but when the basics of data integrity, consistency, system setup and statistical methodology do not sit behind that then of course they end up producing crap.

In terms of taking next steps: well I agree more with this one, what use is good info if it's not acted upon?

I guess, to summarise: I don't feel this post really hit at the heart of the many problems and areas of development needed in this industry, I feel it was a little to obvious and simple to really warrant the sensationalist title, I think we all need to openly debate a number of issues if we really want to improve social media monitoring, my rambling comment has covered just a few of these issues, what do you think?
Jed Hallam

Hi Tom,
Thank you for taking the time to bash your thoughts, I (genuinely) really, really appreciate it and it's great to get another perspective on this post.

First of all, let me just say that I completely agree with you - SMM is absolutely littered with problems (if you look over my last post you'll find *another* rant about automated sentiment analysis). Where I would add to your point about understanding within the industry (for industry, read comms) is that very, very few people understand social media, let alone measurement, let alone data reliability, let along regression analysis or support vector machines. You're in a very fortunate position as you *do* know all of this, and much much more by the sounds of things.

I'd also like you, and everyone that reads this post from this point to see this as an opening gambit, a prologue. There are many, many flaws with measurement (on the side of the measurement tools, the agencies that use them and the clients that demand them) and I WILL be writing much more on these issues - this wasn't a dumbed down post, it was an introduction. Yes, the title was on the edge of link bait, but so what? In a sea of comms. bloggers regurgitating the same posts week in week out I wanted to say something fresh (while you'd disagree that this is fresh, to many it is - see the comments pointing this out) that would get people talking.

Open debate will only happen when people realise that there's a problem, and I hope that people like KD Paine and Jason Falls will help to bring these issues to the forefront.

Again, thank you for commenting :-)

Jed
Dhiren

Liking the Taleb illustration. I agree with your points Jed. I've been helping a number of clients develop a framework for what they should and shouldn't monitor.

I'm integrating macro monitoring with web analytcis and other relavant metrics.

All this data is new to many clients and I think qualified analysts now need to wade in, separate the wheat from chaff, and create some context.

A lot of the low budget tools seem lack customisation and advanced data mining capabilities. As time goes on we'll start to see a lot of these substanceless tools end up in analytics graveyards.
Jed Hallam

I totally agree, analysts and clued up social media folk (but properly clued up) need to help educate clients and their teams on what data is reliable, relevant and needed and what isn't.

Integrating macro monitoring is key to any online reputation measurement - but rarely is!

I hope that there's the level of Darwinism that you're predicting ;-)
Matthew Cain, Trufflenet

Jacqui's point is a good one. There are lots of things that can be automated but you can't replicate the level of understanding of well-trained analysts.
Jacqui Hill

Hey Jed,

Great post - the tagline did exactly what it should've done, had me ready to 'defend, defend' while waiting for the page to load, then lowering my heart rate as I read though the post :)

I cannot support your opinion strongly enough - while the idea of 'real-time' monitoring is an exciting concept, with some value, where listening can really come into it's own is where it is given the time to contextualise itself and be analysed thus.

Our analyst team use our listening tool to gather appropriate data etc. but then are skilled in taking this data and putting in in contaxt to provide actionable insights. Any data read in a silo loses its importance, and a dashboard can only tell you so much. Human analysis adds understanding to listening.

Thanks again for the post!

Jacqui (@JacquiHill)
1000heads
Jed Hallam

Hi Jacqui,

I was hoping that most people would feel like you did ;-)

As one of your colleagues, Tom, has pointed out above, it's really important that we (as an industry) start to talk about measurement in a meaningful way rather than trying to make a quick buck - building internal tools that are tested and verified by analysts will be a big part of that in my eyes.

Thanks again for commenting!
Matthew Cain, Trufflenet

Monitoring is a pretty unhelpful concept because it's so passive. Busy marketeers and corporate communicators usually have lots of data already and often struggle to see the value in something which appears to be the modern equivalent of press clippings.

The key for social media practitioners (internaly and in agencies) is demonstrating why the social media insights and important and what action can/should be taken as a consequence.
Elyse

Jed,

Great thoughts in your blogpost …and nice hook in the headline too ;) So many companies are still trying to figure out what to do with all of the data available in social media monitoring. Too few companies have good social data strategies in place and I fear that many are hoping that they can just push a button and the graphs in their software will magically tell them what insights to draw from the reports.

While I think there are definite use cases for monitoring on a daily basis, to do a true analysis, you do need to compare across time (months, or even years of data). Even beyond that, marketers also need to able to cross analyze against data outside of social media (your customer databases, market research, web analytics, campaign analytics etc). I think this need will continue to push social media monitoring away from being a siloed software solution and will move towards fully integrated marketing/PR solutions . We’ve already seen a lot of acquisitions and mergers in the industry in the last year that add a lot of complementary functionalities to existing SMM platforms and I think this will continue as companies begin to wrap their heads around social metrics and get a better understand of exactly what they need.

Elyse DeVries (@elyse_d)
Alterian
Tom Malkin

Enjoyed reading your comments. For effective context, it's so important to obtain deeper social media insights into “the story” of an entire product category (let alone a brand or product) and then monitor how brands and products can be measured against each other over time on key issues that drive decision making for the competitive set. Thought you would appreciate case studies illustrating this unique capability. Check out the Tiger Woods and ARF-Starcom 360 Measurement case studies and contact me at tom@geeyee.com or at 877-GEEYEE-1 if you’d like to explore further:

Tiger Woods: https://www.geeyee.com/blog/2010/05/may-2010-tiger-woods-case-study
ARF-Starcom: https://www.geeyee.com/blog/2010/05/arf-360-measurement-workshop-at-starcom-mediavest-group
Andreana Drencheva

Jed,

Great post.

Data and numbers are valuable only when you can understand them within the context in which these numbers and data were taken from. And that takes time, because changes in behavior take time. The fact that Brand X has a mention in a Blog Y today means nothing if no one else talks about Brand X. The question shouldn't only be why Blog X mentioned the brand, but also why blogs A, B, C, D, etc. didn't.

Thank you!
Jed Hallam

Exactly!
AmberNaslund

Hi Jed -

Nice insights (and clever headline bait). :)

One of the key bits of *any* analysis in business is the trend line perspective. For example, understanding the impact of a campaign isn't necessarily going to happen in the hours or even days after the launch. It's a matter of looking at month over month and year over year comparisons to see what moved the needle, and what didn't.

Data of any kind is only valuable when it's contextual, which is why the struggle for the "ultimate metrics" in social media is such a toughie. The metrics (and to your point, monitoring data) that ultimately matters to your business completely depends on the goals at hand, and the steps you're taking to execute on your strategy. You can't possibly extract any insights that are valuable until you ask the important question: "Why does this matter to ME?"

Thanks for sparking some good conversation.

Cheers,
Amber Naslund
Radian6
Jed Hallam

Thanks for commenting Amber, some great points - especially about 'ultimate metrics', which is a particularly sore point for me (long story)!
40deuce

Hey Jed,

I originally clicked the link thinking that I would read this post and then despite your point of view, but after reading I feel no need to. In fact, I actually agree with you.
Social media monitoring means nothing if it has no context. Software like Sysomos can bring you all the information and numbers you could ever want, but those numbers are meaningless if you don't know what kind of context to connect them to.
The real power in social media monitoring is being able to take that information and numbers and see what they mean in a larger sense. What can you learn from those numbers and turn them into? How can those insights help you in the future? That is what social media monitoring is good for.

Cheers,

Sheldon, community manager for Sysomos (http://sysomos.com)
Jed Hallam

Hey Sheldon,

It was intended to make people feel a little angry (initially) but I'm glad you came around!

Numbers mean nothing without the frame around them to give context. Something that's equally interesting is the number of social media monitoring agencies that have picked this up - some good monitoring going on there ;-)
Barry A. Martin

We employ a pile of free solutions cobbled/aggregated and processed through various RSS readers. We read up daily, over a cup of coffee (or digestif) and largely treat it like a personal newspaper.

We do this for two reasons.
1. It's nice to be able to react to timely information. Being social is about being useful and we (and our clients) might be in a better position to pint out meaningful news.

2. We abandoned a 'campaign' mentality years ago, when Marty Neumeier explained (in The Brand Gap) that you can influence what people think of you but you certainly don't control it. Instead, we've worked towards making our client's marketing about helping people work on things they were already interested in. We're constantly working on projects with our client's clients, and Social media is a bit of a lifeline that way. We're not inferring 'how we're doing' on the grand scale–like real estate or casinos, you only realize your winnings when you crystalize them by quitting the game.

My point is there's plenty of benefit to be derived from checking in regularly.
Per you context suggestion, I'd characterize that as your brand or communications strategy. Know what you're trying to do first, then figure out how social networks and platforms can facilitate/add to your effort.
And knowing what your're doing and who could care about it has a funny way of making it easier to figure out how to act on what you hear as well.
Jed Hallam

Hi Barry,

Thanks for commenting with such gusto! I certainly agree with you, I think there's a definite need for daily monitoring - especially for customer relations and media purposes - but my point would be that there's a bigger problem at hand which is how people use that information, rather than how frequently they collect it.

Any agency worth its salt advises clients on short term and long term goals, but reputation management is a long term goal - especially online.

Again, thanks for taking time to comment :-)
tomob

As someone who spends a lot of time talking people out of daily monitoring I can appreciate this blog post.

When you find yourself explaining to management why Verizon net sentiment went down 00.25% from Wednesday to Thursday - then you are just making stuff up.

With few exceptions, the world does not move that fast.

Tom O'Brien
@tomob
Jed Hallam

Thanks Tom! What's been great about this has been the amount of people that agree - it restores my faith in marketing a little ;-)

I think that people presume that because we can track every conversation online that each one has an impact - but these conversations have always taken place (offline) and we didn't have to react!

I hate to use this awful cliche, but context is king when it comes to monitoring.
Synthesio

NOW I see why Jon agreed with this post :)

Actually Synthesio - not sure if you've heard of us? - provides monitoring dashboards because we realize that almost everyone wants to be able to check at any moment "how they're doing", but we agree with you in that you have to be able to take a step back and take a look at the big picture to get an idea of how things have changed over time. Monitoring every single second is like watching the stock market every minute of the day ; the changes may seem far more drastic than they actually are, or far smaller than they actually are.
The other day, even, I was reading in Nudge (by Thaler and Sunstein) that people that check their stock investments every day are likely to be restless and disappointed, while those that check them after 10 years will see that their investments have grown. Clearly, brands should pay attention more often than once every 10 years, but it makes you think about the long-term benefits that will come of connecting with people, and how the brand has ultimately evolved over a period of time, in terms of sentiment, volume of mentions, influence of sites where the brand is mentioned, etc.
Nice post, Jed, and nice to find your blog.

Michelle @Synthesio
Jed Hallam

Hi Michele, thanks for taking the time to comment - it's great to hear that you guys are looking at monitoring from a pretty ethical and sensible stand.

Nudge is an excellent book (although *still* haven't finished it!) and I'd recommend Taleb's other book too if you're into that type of thing, it's called The Black Swan.
Jon Moody

Thanks for this - you've encapsulated well what I've been ranting on about for a few years now - the need to see the big picture, for a representative analysis and most importantly of all, for insights that lead to action.
Jed Hallam

Exactly - and I think until we see brands sharpen up their knowledge and consultants open up we'll still be facing this for years.
Drew Spencer

You make some very good points in this post. It's not about monitoring, but insight.
Jed Hallam

Exactly Drew - I think one of the biggest concerns I have is that a lot of people sell fear, rather than advice and counsel.

Jed Hallam

Social media monitoring: absolutely pointless

Things I write about

Search

Old stuff