Author: James Deane, May 12 2014    The World Press Freedom Day conference took place at the UNESCO headquarters in Paris earlier this week. Its theme - media freedom and the post 2015 development framework - added important heft to efforts to get these issues into the final declaration that will be agreed by the UN next September.

But one problem which the media support community can more immediately help with is measurement.

The UN High Level panel report published last year suggested that a goal on honest and accountable government and effective institutions be established.  A component of it, the panel recommended, should be a target around ensuring “people enjoy freedom of speech, association, peaceful protest and access to independent media and information". Another is to "guarantee the public's right to information and access to government data".

The Paris conference spent a good deal of useful time affirming how important this goal was.  What it didn’t solve is the many obstacles which stand in the way of its inclusion. Some of these - such as the intense suspicion of such a goal by many governments - will need to be solved by argument, advocacy and politics and those efforts are gathering pace. The measurement challenge, this conference aside, seems barely to have been recognised by those advocating for these issues.

We can’t honestly ask anyone to take this goal seriously until we come up with a clear system through which progress against the goal can be measured. This is fiendishly difficult. When we talked about this in Paris, much of the conversation was taken up with the problems of measurement. We need to move quickly to some solutions. This is a proposal. If there are better ones, let’s hear them.

First some criteria

Any goal or target needs to be simple, easily understood by those it is designed to benefit, and credible to development actors, media organisations and, above all, people. It needs to be capable of being applied in all countries and all contexts. The whole purpose of the post MDGs is to capture the world’s imagination. Any target or measurement indicator that fails to do this probably won’t cut it.

It needs to reinforce and if possible strengthen, rather than weaken, existing international norms and treaties, including Article 19 of the UN Declaration on Human Rights.

The process for measuring the goal also needs to command widespread credibility.  That means it cannot easily be drawn from existing media freedom indices – like Freedom House - for the simple reason that even democratic developing countries like India or Brazil will not have a formal UN goal measured for them through a process in which they have no buy-in.

It needs - and here there are special challenges for anything concerning media and communication - to be change resilient and change relevant.  The transformation of media and communication environments promise to be even more dramatic over the next 15 years than it has over the last 15.  Facebook was founded in 2004, Twitter in 2006 and its Chinese counterpart, Weibo in 2007.  None of them was even a twinkle in the eye of their inventors when the original Millennium Declaration was signed.

The indicator also needs to be comparable across countries, so performance of different countries can be held up to scrutiny.

I would also argue that measurement focuses on what people think. A central thrust of the post-2015 development framework is likely to, and should, focus on a people-centred approach. The measurement should focus on people, not just on the media.

There are other criteria (and similar criteria to these written in more technical language) which the UN has set out, but these suffice for now.

Most discussions to date on how to measure improvements in media or media freedom tend to involve a whole series of different indicators.

There are excellent examples, such as the UNESCO Media Development Indicators, but this lists many different types of indicator which are not amenable to being consolidated into something simple. They were also explicitly designed not to generate indicators that could create comparisons across countries. There are also datasets from people like the ITU[RS1]  on access to different technologies, including access to radio and television, but they don’t address whether media is independent or free.

So we need something new.

I think the goal should be something like, by 2030, all people on the planet enjoy freedom of expression and have access to independent media.

I think a second target, which also caters to broader issues of access to information, should be “all people have access to independent information which they believe better enables them to take a full part in their society and to make informed decisions in their own interest”.

The media target would be measured simply by carrying out nationally representative surveys asking people three questions which would in turn amount to a composite score.

    “Do you believe that the media in your country is free and independent?”
    “Do you believe the media in your country works in the public interest?”
    “Do you trust the media in your country?”

First reactions

I put this proposal to the Paris conference and several problems were highlighted.

It is a perception-based indicator and it is not clear that all people would have shared definitions or concepts of “free”, “independent”, “public interest”, or even “media”.

My research colleagues at BBC Media Action, who spend a lot of time working out how to accurately measure such things, point to similar concerns, but they and other researchers are very good at solving such problems. I’m not convinced these issues are insuperable.

As for the perception problem - I feel there is room in a set of indicators focused on advancing human progress for measures asking humans if they feel progress is being made. Perception measures are frowned on by much of the research community, but unlike other areas of development (such as reductions in maternal mortality), if someone doesn’t perceive themselves as having freedom of speech or even access to independent media, I would seriously doubt any other measure that suggests that they do.

I have my own concerns.

It may be that some closed regimes where media freedoms are constrained may score more highly than much freer, messier, noisier media environments which are, in fact, playing a much more important part in, for example, holding government to account.

Certainly some of our research suggests that the more people know (through a diverse rather than a monolithic media landscape), the more people question and the less people trust.  We may need to build in some additional indicators, such as plurality or diversity of media, although this immediately makes the indicator more complex. And, of course, adding a question like, ‘Can you speak freely in your country?’ cannot be answered honestly in a country without such freedom of speech.

Other approaches

There are also alternatives to this proposal, where a framework approach is taken - Article 19 has done important work on this.

This would bring together a whole set of indicators (eg where freedom of the media is enshrined in law, where journalist safety is protected, where regulatory environments provide for freedom of the media etc) and on balance I would prefer such an approach.

I am just very unsure how such a framework will be agreed for the post-2015 process, how it would be crystallised into a very small number of easy to understand measures, and even less sure that it would meet the criteria I suggest need to be met.

Over to you

So this is a proposal to be shot down in flames by people - including my many BBC Media Action research colleagues - who are more expert in measurement than me and replaced with something better.

But we need to get cracking. We can’t make any progress on this unless we start thinking of solutions, not just the problems, of measurement. Answers on a comment below this blog please, not on a postcard.

And a condition for those shooting down these ideas - you do need to suggest something better!

Click here to access this BBC Media Action blog.
Image credit: BBC Media Action

BBC Media Action
BBC Media Centre, MC3A, 201 Wood Lane
W12 7TQ
United Kingdom (UK)
Phone: 44 (0) 20 8008 0001
Fax: 44 (0) 20 8008 5970