Fighting flawed metrics in public relations

Advertising Value Equivalent (AVE) festers as a public relations metric. It's changing but slowly.

“Worked in media measurement for all his life, achieved nothing. That’ll be on my tombstone," joked Richard Bagnall.

We’d met for lunch and were talking about the ongoing need for education to drive up standards in public relations measurement.

It was a throw away comment but it's rooted in a truth that a measurement approach which is proven to be flawed continues to have a stronghold on public relations.

Bagnall is the chair of the Association of Media and Evaluation Companies (AMEC) and the European boss of Prime Research. Alongside CEO Barry Leggetter he’s at the forefront of driving new measurement standards in organisational communication.

AMEC's own data suggests that around 20% of organisations worldwide still use Advertising Equivalent Value (AVE). A PRCA survey recently put the figure at a third of organisations in the UK.

What is AVE?

AVE attributes an ad value to earned media content secured by a public relations practitioner.

An arbitrary multiplier is often applied, justified on the basis that editorial content has greater credibility and is valued more by consumers than ad space.

It's an entirely flawed approach.

In addition to AMEC it has been denounced by the Chartered Institute of Public Relations (CIPR), Institute for Public Relations (IPR), Public Relations Communications Association (PRCA) and Public Relations Society of America (PRSA), and others.

AVEs have no relation to a campaign's objectives, messages, or sentiment, let alone organisational outcomes but that doesn't stop them continuing to be used as a public relations metric.

Online monitoring firm Meltwater published a white paper written by public relations practitioner and Forbes contributor Robert Wynn last year promoting the use of AVEs as a valid metric. It was widely decried throughout the public relations profession worldwide.

The white paper has subsequently been deleted from Meltwater’s website.

Barcelona Principles

AMEC's annual conference in Barcelona in 2010 set out the Barcelona Principles, a series of a set of seven guidelines industry to measure the efficacy of public relations. They were updated in 2015.

1. Goal setting and measurement are fundamental to communication and public relations
2. Measuring communication outcomes is recommended versus only measuring outputs
3. The effect on organisational performance can and should be measured where possible
4. Measurement and evaluation require both qualitative and quantitative methods
5. AVEs are not the value of communications
6. Social media can and should be measured consistently with other media channels
7. Measurement and evaluation should be transparent, consistent and valid

The public relations profession has spent the subsequent seven years rallying behind AMEC's initiative.

Measure organisational performance, not media

The public relations measurement business has been built on traditional media metrics based on content evaluation. At best these are poor proxies for performance.

Metrics such as AVE, reach or impressions have no relationship to any metric that a business would recognise. They use an entirely different set of criteria based on outtakes and outcomes.

As practice shifts from traditional media as the primary form of public relations engagement, to a variety of digital and social media, the industry is getting smarter at measurement but progress is slow. Paid, earned, owned and share media all leave an audit trail that can be interrogated.

Our opportunity is to define robust objectives, outputs, and outcomes for a campaign, and then to overlay AMEC’s Barcelona Principles and measurement framework.

Integrated Evaluation Framework

In 2016 AMEC brought together academics, measurement experts, public relations practitioners and organisations to create an Integrated Evaluation Framework that would work across all integrated communications.

The Framework shows how to implement Barcelona Principles 2.0, linking organisational objectives to communication objectives, to outputs, outtakes, outcomes and organisational impact.

• Objectives - Like all good measurement, it should start with clear organisational objectives. These can come in many different forms, whether they be awareness, advocacy, adoption or demand related. Following on from organisational objectives, is communication objectives. These should reflect and mirror the organisational objectives.
• Inputs - This covers two important areas: first, to define the target audiences of the campaign; and second, the strategic plan and other inputs such as describing some of the situation analysis, resources required and budgets.
• Activities - This section is outlining what activities were carried out, any testing or research, content production etc. Importantly, the tool recognises the importance of paid, earned, shared and owned (PESO) and gives users the ability to tag accordingly.
• Outputs - In outputs, this covers the core measures across Paid, Earned, Shared and Owned (PESO) media. For example what was the reach of the paid advertising, how many visitors to the website, how many posts, tweets or retweets, how many people attended the event, and how many potential readers of the media coverage. This is quantitative and qualitative measures of outputs.
• Outtakes - this refers to the response and reactions of your target audiences to the activity. How attentive were they to the content, what was their recall, how well understood is the topic, did the audience engage with the content or did the audience subscribe to more information.
• Outcomes - measure the effect of the communications on a public. Have the target audience increased understanding, has it changed their attitude to the topic, has it increased trust and/or preference, has it had an impact on the intention to do something (e.g. trial, subscribe, register) or increased online advocacy.
• Impact - This final section is where impact on the organisational objectives is evaluated. So here the tool is looking to cover reputation improvement, relationships improved or established, increase in sales or donations, change in policy, or improved social change. This is a clear demonstration of business outcome and link to organisational objectives.

The Integrated Evaluation Framework was launched last year with a comprehensive website of resource material and an interactive tool, supported by Lewis PR, to steer practitioners through the process.

The framework has become a standard at Ketchum. Every conversation around measurement within the business is framed around the Integrated Measurement Framework.

Best practice from the UK Government Communication Service

In the UK the Government Communication Service (GCS) has published an online measurement toolkit and case studies that are available for anyone to download.

GCS has adapted the AMEC Integrated Evaluation Framework for use within government. The toolkit contains a planning model and measurement framework for media, digital, marketing, stakeholder engagement and internal communication.

The most useful area of the toolkit is a series of case studies. Each one is a worked example of a government campaign using the GCS measurement framework.

You’ll find example measurement frameworks reviewing campaigns on exporting, plastic bag charging, maternity safety, local road maintenance and energy switching.

The leadership of GCS under the direction of its executive director for communications Alex Aitken and its Evaluation Council in openly sharing its framework should be applauded.

Anyone working in public relations should study the toolkit as an example of best practice.

Tackling the supply chain

“The public relations and communications industry has made enormous progress in eliminating AVEs over recent years, much of it in partnership with AMEC’s strong leadership,” said Francis Ingham, director general, PRCA.

My personal view is that monitoring companies such as Cision, Gorkana, Kantar Media, Meltwater, and Vuelio could accelerate progress if they were prepared to stop supplying AVE metrics as a default metric within reports, alongside other metrics.

Agencies such as my own firm Ketchum teach practitioners that AVEs are flawed and have no place in a modern public relations campaign or planning process. Teachers on public relations courses at colleges and universities do the same.

But then monitoring vendors provide web based solutions that calculate AVE data as a default.

Measurement companies universally argue that they only supply AVE data because clients demand it and that if they didn't supply it, another monitoring vendor would.

AMEC and the CIPR take a similar stance saying that the market needs to solve the issue rather than laying the blame on an area of the supply chain.

“Measurement and monitoring firms are merely responding to market demand. The problem is the lack of professionalism among residual groups within the industry who have failed to move on from AVEs,” said Jason MacKenzie, President, CIPR.

“We all know that peddling AVEs is like selling a fairy story. AVEs belong in the dustbin of history: their continued use damages the reputation of public relations.”

Francis Ingham at the PRCA goes further.

“If measurement and evaluation companies stopped providing AVEs it would kill them overnight. I hope that they will rise to the challenge and do so.”

Postscript

I’m going to end this story where I started with Richard Bagnall.

He was clearly exaggerating to make a point when he said that he had achieved nothing in his career. But the point about the slow death of AVE’s is real. He has been actively campaigning against them for 21 years.

Good progress has been made in the past seven years. Agencies such as Lewis and Ketchum; and communication teams such as the UK Government Communication Service are committed to new forms of measurement.

Shifting the final 20% of the market is going to require a concerted effort by everyone in the communications and public relations business.

“The educational battle will take time, but as our numbers show, it is being won,” adds Bagnall.

It may not take a lifetime but it will likely take a generation.