The only way to know if you’re successful is through effective evaluation.
Analytics can tell you a huge amount about your content (and your audience).
Beware of vanity metrics and reading the wrong signals.
Up until this point we have focused on the production of content. Perhaps you’ve made a powerful video, built a digital audience and published your video on your social media channels. You might think that it’s now time to put your feet up – but sadly it’s not time to relax yet.
Evaluation, based on credible data, is the only way you’ll ever know if what you’re doing is actually working. Think about the rise of digital advertising. In the past, if you put an advert in the local newspaper or ran an expensive TV ad, you would have no real way of measuring how effective it was. It would have been possible to measure increases in sales around the time of advertising campaigns, but certainly nothing much more sophisticated than this. You were also very limited in your ability to target your advertising to particular people. With digital advertising you can do both – accurately target specific audiences and measure how popular your advert is with them. It’s not that surprising, therefore, that advertising spend on ‘traditional’ media is dramatically falling.
Detailed evaluation allows you to measure a very wide array of different things about your content. It could range from the number of views your content has had to detailed audience insights.
Proper evaluation is something you have to build into every content and campaign plan you create. Deciding which metrics to measure and what your targets are should be second nature. Evaluation is also important if you’re running a team of content producers. Without detailed metrics, you’ll be unable to provide guidance and insight on what’s working. Evaluation also lets you spot audience trends that you can then use. You might discover people are interested in a subject you hadn’t previously considered, thanks to your audience insights.
Choosing the right metrics is arguably the most important part of evaluation. A video may quickly gather a vast number of initial views. But the analytics tell you that because it has a very compelling opening shot, people are watching for a few seconds and then switching off. The initial video views may be high, but only 2 per cent of viewers watched all the way to the end. Was the video therefore successful? If you had picked initial views as your metric, the video would have been an enormous success (according to your plan). Sadly, though, it clearly isn’t effective – most people watched for only a few seconds.
In this situation, a better metric would have been to measure 10 or 30 second views. This number gives you a more accurate picture of how many people watched in a meaningful way. You can also use this information to improve your content in future. If you saved the crucial bit of information until the end of the video, the overwhelming majority of people who watched would have missed it. So from now on, you know to put the vital information at the start of the video.
Demonstrating to an employer that you can evaluate content effectively complements your content production skills. Too often people focus on the production of the content itself, but by demonstrating you can evaluate too you’ll stand out from the crowd.
But what is proper evaluation? Because of the wealth of information available to you, it’s all too easy to get swamped with numbers. Good evaluation should:
measure the success (or failure) of your content against your objectives;
provide insight you can use to improve your content in future;
measure inputs as well as outputs;
be transparent.
We’ll now discuss each of these points in detail to explain why each one matters.
Demonstrating that your content is effective isn’t just important for your own satisfaction. You may have to show your digital editor why the content you have produced is popular with audiences. You may produce content for a commercial client and need to deliver against set objectives. If you are running a public sector or charitable campaign, you have to evaluate your content to show it’s an effective use of money.
Let’s cover the fundamentals of what good evaluation should be.
Before producing content as a one-off or as part of a long-term campaign, it’s important to set objectives. These don’t need to be complicated or numerous, but you need to define success. Sometimes these objectives will be obvious – if you want to produce advertising content that gets a certain number of people clicking through to a website, the number of clicks is the most important objective to measure. You need to be specific in your evaluation: it’s tempting just to think ‘if my content is popular then it’s done the job’. Setting objectives for you and your team makes sure you are focusing on what your content is meant to achieve.
The SMART criteria can help you set effective objectives. They are shown in the table.
Table 6.1 SMART objectives for measuring content effectiveness
SMART | Not | |
|---|---|---|
Specific | We want to reach 18- to 21-year-old men interested in fitness | We want to reach young people |
Measurable | Get 20,000 video views | Inspire people |
Achievable | We got 20,000 engagements before so we want 25,000 this time | Wouldnt it be great if we got a billion visits |
Relevant | We are launching a new website so we’re measuring unique visitors | We are launching a new website so we’re measuring retweets |
Timely | The content has to get more views than our average in 24 hours | As long as it gets the views at some point were happy |
By getting into the habit of using SMART objectives you’ll ensure you’re on the right track when it comes to producing content. Regularly reviewing these objectives is also recommended as your goals may change over time.
This book certainly doesn’t have all the answers, and what’s right for one audience can be very different for another. If you are working in a team, it’s especially important that you regularly review your work and how it’s resonating with your target audiences. ‘You produced a really beautiful design’ may be flattering, but how useful is it?
Effective evaluation should include valuable insight into what you can do better next time. Every review or evaluation document should include a ‘lessons learnt’ section. It doesn’t have to be rocket science – a simple lesson is still a valuable one. For example, you may be running a long-term campaign involving lots of different media types. Once it’s complete you review the analytics, which show that animations were generally watched all the way through, while the audience switched off videos after a few seconds.
So the simple lesson learnt would be to produce more animations next time, as they were more useful to the target audience. This also neatly illustrates that evaluation doesn’t have to be done at the end of the campaign. If you have the time, it’s best to do it on an ongoing basis. Why wait until you’re done to improve your work?
Finally, it’s worth sharing these insights as widely as you can. In the world of digital content, people are always sharing ideas about what does or doesn’t work. It’s also a great way to raise your profile within your industry if people start to see you as a reliable source of fresh insight.
It’s all well and good measuring whether the content you’ve made is effective or not. But how much time did it take to produce? What was the cost?
If you’re running your own business, this should be second nature. You’ll be used to measuring the cost to profit benefit to everything you do, and content shouldn’t be any different. Evaluate how much of your resources your content has taken. It’s not worth doing this for everything you make (unless it’s highly complex), but it’s certainly worth reviewing.
This is critical if you’re managing a team. You have to monitor how much of their time content takes to produce and whether it’s worth it. Why get two people to produce a complex piece of storytelling over a number of days if the audience prefers a quick video interview filmed and broadcast live? Effective content isn’t just material that’s popular with your audiences but also content that doesn’t place a heavy burden on your limited resources.
This can be difficult. You may put your heart and soul into a powerful piece of content that then falls completely flat with your audience. It’s never easy to admit failure or that you’ve made a mistake. But evaluation works both ways – it gives you a solid foundation to promote your successes but also highlights where you haven’t achieved your goals. You shouldn’t take this personally, and by identifying the lessons learnt you still learn and grow.
Hold regular meetings with your team to review the content you’ve produced that week or that month. If everyone is transparent about what worked and what didn’t, you can ensure your long-term success as you’ll be able to absorb new findings quickly.
There are a number of different terms that are used in digital evaluation. It’s important to understand the nuance of each one so that you are confident in reviewing your work. Unhelpfully, different social networks or tools use slightly different measurements. Reach on Facebook is not the same as impressions on Twitter, although they may look similar.
In this section you will find an explanation of the most commonly used phrases as well as guidance on which metrics are good and bad for measuring.
Bounce rate is a metric used for websites. It’s the percentage of people who leave your website after looking at only one page. Generally you want this to be as low as possible. If you have a high bounce rate, it suggests that people find your site either boring or unhelpful.
There is not a set ‘good’ bounce rate – this is another metric you should measure over time and try to lower. Generally it would be extremely hard to get it below 25 per cent, while over 60 per cent would be bad. If you find that your site does suffer from a high bounce rate, you can try improving the content, improving the design to make it easier to navigate or check you are targeting the right audiences. Speed is also a big deal – if your website loads slowly and has lots of intrusive popups, people will simply leave.
Use it for
seeing if your website is working properly for your audience.
Avoid
ignoring this metric.
Clickthroughs are the number of people who visited your website from your content. It could simply come from an e-mailed or tweeted link, an embedded ‘visit us’ button in your video or from an advertising campaign. These are expressed as either the number of clickthroughs or the clickthrough rate (CTR), which is the number of clicks as a percentage of the people who saw the content.
This is a very important metric, especially if you’re running a paid-for advertising campaign. If you’re paying to try to boost traffic to your website, this is the number you should be keeping an eye on. A low CTR would suggest the content isn’t clear or compelling. A high CTR naturally says the opposite.
Use it for
measuring how effective your content is at sending traffic to your website.
Avoid
only measuring it when you are running an advertising campaign. It’s worth checking how good your content is at sending traffic to your site even when you aren’t promoting it.
Engagements is a very important metric – it shows you how many people responded to your post or content. This includes adding a comment, liking it, sharing it, following you as a result of seeing it, or negative things such as reporting or unfollowing you. It shows how interesting your content is, as people have taken the time to respond in some way.
Having a very large number of engagements may not necessarily be a good thing. If you post content that your followers absolutely loathe, yes, you’ll have an incredible engagement rate, but it doesn’t mean you have a happy audience. Generally though, you want your audience as engaged as possible, but make sure you review what those engagements are to ensure you’re on the right track.
Use it for
analysing how interested your audience is in your content. Look into the stats in detail to see how people respond differently to your content.
Avoid
assuming all engagements are positive.
The engagement rate is the number of engagements as a percentage of impressions. So if you had 1,000 impressions and 100 engagements, your engagement rate would be 10 per cent. There is no global average – you have to figure out what yours is over time and then try to improve on it. Generally though, 1–2 per cent is a good rule of thumb.
Use it for
setting a benchmark to measure how your content performs and see how your audience becomes more or less active over time.
Avoid
assuming that all the engagements that make up your engagement rate are positive.
The number of times a post or piece of content has been seen. This isn’t the number of people who have seen a post, as someone could see the same content a number of times. This is often the largest number your metrics will show you.
Use it for
a way to measure the overall distribution of your content and awareness of your brand.
Avoid
using it to demonstrate how effective your content is. Many people may have seen it, but if none of them actually view it, it isn’t effective at all.
Social media is funded through advertising. If you’re running a page on social media, you will have the option either to post normally or to pay to boost the post to more people. In your analytics you will see this separated out as organic, ie those that were not boosted, and as advertising and paid posts that were. A general rule of thumb is that it’s sensible to boost content that’s already performing well organically with your audience, as you know it’s popular.
Pageviews are similar to impressions. It simply means how many times your website has been looked at. Note that it is different from uniques. Pageviews is the number of views, not the number of people looking at the page. So if you have a popular news site that people check daily, your pageviews will be much higher than your uniques.
Use it for
measuring how popular your site or parts of it are. You can also use it to demonstrate the site’s worth to advertisers and how compelling your content is.
Avoid
using it to measure how effective specific pieces of content are.
Reach is simply the number of people who have seen a piece of content. This is different from impressions – one person may view a post ten times but that still counts as one reach measure. It’s generally slightly lower than impressions owing to repeat views from your followers. You should measure how much your reach is increasing (or decreasing) over time to see how widely your content is being seen.
Use it for
a way to measure the overall distribution of your content and awareness of your brand.
Avoid
using it to demonstrate how effective your content is (as for impressions).
Sentiment is the measurement of what people think of your content, brand or subject. Obviously, it would take an incredibly long time to manually go through every comment and figure out what percentage is positive or negative. Thankfully, there are a number of powerful analytical tools that can do this automatically by studying the language people use. This, combined with your engagement metrics, gives you a much more detailed picture of how your content is performing.
Word clouds are often created by these tools. They show you what phrases people use along with your content. This can help you identify trends and what your audience is interested in. Sadly, sentiment analysis isn’t that simple – despite the power of analytical tools, they are not perfect. For instance, they cannot detect sarcasm and often tag such comments as positive. Therefore any sentiment analysis should be combined with some human analysis. It can be as simple as someone checking any automated sentiment analysis and making sure it’s got things right. Categorize comments or content into supportive, negative and indifferent to get an idea of what your audience’s mood is like.
Use it for
learning more about what your audience thinks and what subjects they are passionate about.
Avoid
taking raw sentiment analysis data at face value.
Uniques are sometimes also called users. This number can be a confusing one, but it’s as close as you can measure to the number of people visiting your website. What makes life tricky is that most people have multiple devices and browsers. For example, you may have a work computer, a personal tablet and a smartphone. By the end of the day you may have visited your favourite news site on all these devices. The analytics would measure this as three uniques or users – despite you being one person!
Use it for
measuring how many people visit your website (in an imperfect way), tracking the growth of your audience, and comparing it to the pageviews to see how many people are returning to your site.
Avoid
thinking it equates to the exact number of people visiting your site. It’s essential you measure it alongside pageviews too, so you can see how loyal your audience is.
You may think this does what it says on the tin – how many people have seen your video. Sadly, it’s not that simple. What actually counts as a video view? Different platforms measure views in different ways. When Facebook announced their new video player, they measured three seconds as a view. Most videos on Facebook autoplay, so it’s likely that each one would get three seconds of view time as people scroll past, so it’s not a helpful number. YouTube measures around 30 seconds as a view. Therefore it’s important to check carefully how the platform you’re using measures views.
If your platform provides it, keep a close eye on retention – how long people are watching. This is usually depicted as a graph. If people drop off suddenly at a particular place in the video, try to establish why.
Use it for
measuring how many people watched your video.
Avoid
just measuring the initial view number. Look at how many people watched for 10 or 30 seconds.
When it comes to evaluation, the biggest mistake most people make is simply not doing any. So by doing even the simplest analysis of your content you’re already doing well. As this section covers, it’s important to get into the habit of regularly reviewing your and your team’s content to make sure you know it’s working properly.
But it’s easy to spend a lot of time evaluating, so you want to make sure you avoid some of these mistakes.
When you first see the stats for your digital content, it’s completely natural to look for the biggest numbers. If you’ve published a video on social media, your reach will be higher than the number of video views – and certainly the 10- or 30-second views. The latter is more meaningful, but it feels nice to promote the big number to people you work with.
It’s important to resist this temptation. You ideally want all your stakeholders to understand the importance of choosing the right metrics. If you’ve trained them properly, they will see through your ploy to pick the big number.
You should be especially vigilant about vanity metrics when there’s money involved. Unscrupulous digital agencies may try to show you just the biggest numbers to make themselves look good. This can be avoided by setting SMART objectives and being clear with other organizations about what metrics matter.
Vanity metrics can have a place, though – if you’re trying to convince a busy, senior manager of the value of digital content, sometimes you have to dazzle with big numbers rather than explain the nuances of digital measurement. But use them very sparingly.
With the market for digital analytics growing rapidly, there is a wide range of extremely powerful analytical packages out there. These hoover up vast amounts of data, can perform highly complex analysis and visualize data in an appealing way.
Despite what their salespeople will tell you, these packages require a lot of training and skill to use effectively. They are also very expensive. Many organizations or teams see them as a shortcut to brilliant evaluation, but without plenty of training this can be a waste of time. They are often too powerful for what most teams need to actually measure, and with their complexity it’s hard for everyone to understand the reports. Clearly, though, they have a place, especially if you can assign dedicated evaluators to manage them, but the free analytics provided by the social networks or websites will do the job for the vast majority of people, so think carefully before investing in an all-singing, all-dancing evaluation package.
When you want to do any meaningful user research, it’s important you gather enough data to make it as accurate as possible. For example, if you want to survey your audience to find out what content they want more or less of, you won’t get much meaningful data by asking ten people. As a bare minimum you should always try to survey at least 1,000 people. This can obviously make it harder and more expensive, but you can trust the data that way. If you want to increase the number of people taking part in your research, you may want to offer an incentive or outsource to a specialist company.
Meaningful user research should be the foundation of any digital project. If you are building a new website, launching a new media brand or developing an app, you can’t just assume what the users will want and how they will use it.
It’s easy to get swamped by spreadsheets. This is a similar mistake to using overly complex analytical tools. By just harvesting large quantities of data you’re not helping anyone. If you’re going to capture data, have you planned how you will analyse them? Also consider whether you need to bring in any specialists to help evaluate the material.
Digital content provides a rare mix of the artistic and the scientific – you can produce beautiful content that tells a powerful story while also subjecting it to rigorous evaluation. You may want to consider specializing as an evaluator if it’s something you enjoy and have a talent for. People with solid evaluation skills are in big demand and you can learn a lot of the skills yourself.
It’s worth being comfortable with evaluation skills no matter what aspect of digital content you’re interested in, but don’t discount it as a great career choice in itself. So what overall skills would you need?
Organized and strategic
How are you going to solve people’s evaluation problems? You’ll need to produce and carry out a successful plan.
Listening and observing
What people do and what people say are often two different things. Knowing how to analyse the difference is important.
Analytical and objective
You’ll inevitably be working with a lot of data. You’ll have to be able to study them and find valuable insights from them too.
Communication and collaboration
You could answer that these are crucial skills for any role. And you’d be right. But it’s not good enough just to understand the data – you have to work with people with different skills from many different teams. Can you explain the data clearly? Can you repurpose them for different needs?
Empathy
This goes a long way in understanding people.
So what evaluation roles are there? Here are some examples.
Advertising companies, digital agencies and in-house marketing teams usually have a large digital analysis team. You’ll be measuring the effectiveness of campaigns and paid-for advertising, and conducting audience research. You should spend the time to become a Google Analytics pro as well as studying one other large analytics package. Your employers would want you to demonstrate whether adverts are performing well, how customers (or potential customers) are discovering and interacting with marketing material, and how marketing content can be altered to appeal to more people.
You’ll need a range of skills, including SEO and HTML knowledge, on top of your content skills. You’ll also spend a lot of time performing audits for clients or colleagues. Not only do you need to be able to perform these audits, but you also need to be able to convey your findings clearly.
User research is a complex skill and one that’s highly in demand. You’ll need to learn how to conduct user testing, large-scale surveying or interviews, how to work with designers to ensure user interfaces work properly, and overall usability testing.
The best user research isn’t just an initial fact-finding exercise. It should be an ongoing process to fine-tune digital products and services.
A huge range of organizations want to hire user researchers, which would give you a lot of choice in your career. You would generally be in demand from digital development companies, government agencies that provide digital services and large companies with a big digital presence.
Becoming a data scientist is academically challenging and would certainly require a qualification in a field such as computer science. These professionals can capture, sort and analyse vast quantities of highly complex information. It’s not just academic ability you’ll need, but also business nous. Knowing how your work fits into the overall goals of the organization will help you succeed. You’ll need coding ability as well as database expertise. It’s a challenging field, but a highly rewarding one: if you get a role as a data scientist, you could be working on AI research at a leading tech company.
The transparency of digital analytics can make plenty of people nervous. Knowing that all the digital content you produce can be analysed in a wide variety of ways can make it feel as if any mistakes you make will be out in the open for everyone to see.
It’s important to acknowledge this worry that people working for you may have. Of course, it’s only right that thorough evaluation should underpin the content you produce. But this doesn’t mean trying to catch people out. A sense of creativity and risk taking is crucial to any digital content team, a result of which means that inevitably some content will miss the mark.
Evaluation is actually an excellent way to encourage your team and foster some friendly competition. This book has recommended regular content meetings a number of times, and sharing the data on how content has performed should be a central part of them.
Here are some suggestions on how you can do this.
Schedule a regular update with your team which uses your analytics to inform the discussion. People should share what worked and what didn’t, identify what lessons were learnt and explore whether any surprises were found. If you have access to evaluation tools, it’s worth putting them on a screen for all to see while the discussion takes place. It’s often good to feature a set number of productions each week, so people have something to aspire to. You shouldn’t just highlight the most popular content but also reward creativity, taking a risk, or thinking quickly in a fast-moving situation.
Digital evaluation should be shared as widely as possible. It’s worth hosting brief highlight sessions with your wider department or organization if possible. Not only does this let you and your team plug the great work you’ve been doing, but it also allows as many people as possible to learn from your work. Everyone benefits from having a greater understanding of your audiences and the methods that successfully appeal to them.
You probably don’t want to air your metaphorical dirty laundry at these updates – you can save that for the team-only meetings – but you should be as open as possible. The lessons you have learnt and the insights you have gained are particularly valuable discussion points for these sessions.
If possible, it’s a worthy investment to put some screens around the office showing analytics, content that’s currently being distributed, and what your audience are discussing. In newsrooms it’s absolutely worth sharing what the most popular/discussed articles currently are. Likewise in PR offices or communications offices, keeping a word cloud or other form of live data displayed has the benefits of giving people immediate insight and reminding them that these tools are available to use if they need them.
You don’t necessarily need fancy analytical tools. You can simply design your own graphics and update them on, say, a weekly basis.
In a large, complex organization you obviously can’t share all your successes and insights face to face. You may consider sharing a regular e-mail bulletin about your most successful content with relevant stakeholders. Or if you’re feeling really digital, update your internal social media or chat apps.
You might consider producing a regular bulletin that anyone can sign up to. This is a great way to improve your and your organization’s reputation for digital leadership – although obviously be careful about what information you share.
As always, rewards are a brilliant way to motivate people. You could have a trophy that changes hands every week, vouchers for coffees, or even go back to school with some gold stars.
This chapter has demonstrated why it’s important to evaluate, how to evaluate consistently to maintain standards, and how to specialize in the field. Good evaluation doesn’t have to be complicated or require hefty qualifications.
Now that we have covered the production and measurement of digital content, we will move onto arguably the most important method of distributing it – social media.