The World
Press Freedom Day conference
took place at the UNESCO headquarters in
Paris earlier this week. Its theme – media freedom and the post 2015
development framework –
added important heft to efforts to get these
issues into the final declaration that will be agreed by the UN next

But one problem which the media support community can
more immediately help with is measurement. 

The UN
High Level panel report published last year
suggested that a goal on honest
and accountable government and effective institutions be established.  A component of it, the panel recommended,
should be a target around ensuring “people enjoy freedom of speech,
association, peaceful protest and access to independent media and
information". Another is to "guarantee the public's right to
information and access to government data".

The Paris conference spent a good deal of useful time
affirming how important this goal was. 
What it didn’t solve is the many obstacles which stand in the way of its
inclusion. Some of these– such as the intense suspicion of such a goal by many
governments – will need to be solved by argument, advocacy and politics and those
efforts are gathering pace. The measurement challenge, this conference aside,
seems barely to have been recognised by those advocating for these issues.

We can’t honestly ask anyone to take this goal seriously
until we come up with a clear system through which progress against the goal
can be measured. This is fiendishly difficult. When we talked about this in
Paris, much of the conversation was taken up with the problems of measurement. We
need to move quickly to some solutions. This is a proposal. If there are better
ones, let’s hear them.

First some criteria

Any goal or target needs to be simple, easily understood
by those it is designed to benefit, and credible to development actors, media
organisations and, above all, people.
It needs to be capable of being
applied in all countries and all contexts. The whole purpose of the post MDGs
is to capture the world’s imagination. Any target or measurement indicator that
fails to do this probably won’t cut it. 

It needs to reinforce and if possible strengthen, rather
than weaken, existing international norms and treaties
, including Article
19 of the UN Declaration on Human Rights. 

The process for measuring the goal also needs to command
widespread credibility.  That means it
cannot easily be drawn from existing media freedom indices – like Freedom House
- for the simple reason that even democratic developing countries like India or
Brazil will not have a formal UN goal measured for them through a process in
which they have no buy-in.

It needs – and here there are special challenges for
anything concerning media and communication – to be change resilient and change
The transformation of
media and communication environments promise to be even more dramatic over the
next 15 years than it has over the last 15. 
Facebook was founded in 2004, Twitter in 2006 and its Chinese
counterpart, Weibo in 2007.  None of them
was even a twinkle in the eye of their inventors when the original Millennium
Declaration was signed.

The indicator also needs to be comparable across
, so performance of different countries can be held up to

I would also argue that measurement focuses on what
people think
. A central thrust of the post-2015 development framework is
likely to, and should, focus on a people-centred approach. The measurement
should focus on people, not just on the media. 

There are other criteria (and similar criteria to these
written in more technical language) which the UN has set out, but these suffice
for now.

Most discussions to date on how to measure improvements in
media or media freedom tend to involve a whole series of different

There are excellent examples, such as the UNESCO
Media Development Indicators
, but this lists many different types of
indicator which are not amenable to being consolidated into something simple. They
were also explicitly designed not to generate indicators that could create
comparisons across countries. There are also datasets from people like the ITU[RS1] 
on access to different technologies, including access to radio and television,
but they don’t address whether media is independent or free.

So we need something new.

I think the goal should be something like, by 2030, all
people on the planet enjoy freedom of expression and have access to independent

I think a second target, which also caters to broader issues
of access to information, should be “all people have access to independent
information which they believe better enables them to take a full part in their
society and to make informed decisions in their own interest

The media target would be measured simply by carrying out
nationally representative surveys asking people three questions which would in
turn amount to a composite score.

  1. “Do you believe that the
    media in your country is free and independent?”
  2. “Do you believe the media
    in your country works in the public interest?”
  3. “Do you trust the media in
    your country?”

First reactions

I put this proposal to the Paris conference and several
problems were highlighted. 

It is a perception-based indicator and it is not clear that
all people would have shared definitions or concepts of “free”, “independent”,
“public interest”, or even “media”. 

research colleagues at BBC Media Action
, who spend a lot of time working
out how to accurately measure such things, point to similar concerns, but they
and other researchers are very good at solving such problems. I’m not convinced
these issues are insuperable.

As for the perception problem - I feel there is room in a
set of indicators focused on advancing human progress for measures asking
humans if they feel progress is being made. Perception measures are frowned on
by much of the research community, but unlike other areas of development (such
as reductions in maternal mortality), if someone doesn’t perceive themselves as
having freedom of speech or even access to independent media, I would seriously
doubt any other measure that suggests that they do.

I have my own concerns. 

It may be that some closed regimes where media freedoms are
constrained may score more highly than much freer, messier, noisier media
environments which are, in fact, playing a much more important part in, for
example, holding government to account. 

Certainly some of our research suggests that the more people
know (through a diverse rather than a monolithic media landscape), the more
people question and the less people trust. 
We may need to build in some additional indicators, such as plurality or
diversity of media, although this immediately makes the indicator more complex.
And, of course, adding a question like, ‘Can you speak freely in your country?’
cannot be answered honestly in a country without such freedom of speech.

Other approaches

There are also alternatives to this proposal, where a
framework approach is taken – Article
19 has done important work on this

This would bring together a whole set of indicators (eg
where freedom of the media is enshrined in law, where journalist safety is
protected, where regulatory environments provide for freedom of the media etc)
and on balance I would prefer such an approach. 

I am just very unsure how such a framework will be agreed
for the post-2015 process, how it would be crystallised into a very small
number of easy to understand measures, and even less sure that it would meet
the criteria I suggest need to be met.

Over to you

So this is a proposal to be shot down in flames by people – including
my many BBC Media Action research colleagues – who are more expert in
measurement than me and replaced with something better. 

But we need to get cracking. We can’t make any progress on
this unless we start thinking of solutions, not just the problems, of
measurement. Answers on a comment below this blog please, not on a

And a condition for those shooting down these ideas – you do
need to suggest something better!


Related links

Media and the post-2015 development framework: the case builds

All blogs
from James Deane

Follow BBC Media Action on Twitter and

Go back to BBC Media Action