Valuing the impact of cultural activities on our wellbeing (part 2 of 2)

This two-part blog post is a follow-up to a series of three articles about Outcomes Based Accountability (OBA) and the Programme for Government (PfG) draft Outcomes Framework.

Yesterday, part 1 of this blog article considered the changes to culture and heritage related outcomes in the draft PfG, published in January 2021. Part 2 now explores different approaches to valuing culture and heritage and considers the potential of these metrics as methods for scrutinising the value of future culture and heritage policies.

A new framework for valuing culture and heritage

Some of the difficulties of valuing culture and heritage have been examined by the Department of Digital, Culture, Media and Sport (DCMS). One DCMS initiative resulted from discovering that The Green Book, which contains guidance issued by HM Treasury on how to appraise policies and programmes, currently had no guidance specific to culture and heritage.

There is growing real-world demand from policymakers and practitioners to monetise the welfare impacts of cultural policies, and to quantify culture and heritage goods and services in a way that aligns with cost benefit analysis and business cases in other sectors.

To help improve the articulation of the value of culture and heritage, in January 2021, DCMS published a new framework for valuing cultural and heritage assets. An aim of this approach was to translate changes resulting from increased engagement in culture and heritage into a monetary value, to support evidence based decision making.

The Culture and Heritage Capital Framework was based on a model developed in 2018 by the Department for Environment, Food and Rural Affairs. It aimed to demonstrate how the benefits or ‘flows’ from culture and heritage assets (stocks) can contribute to the achievement of outcomes, both at an individual (interventions) and population (pressures) level.

The DCMS report noted that although ‘there is a wealth of literature aiming to value different aspects of culture, art and heritage, there exists a lack of a formalised approach’. DCMS commissioned a Rapid Evidence Assessment (REA) of different approaches to valuing culture and heritage. Although a number of further areas of research were also identified, the main methods of valuation included a mix of ‘willingness to pay’ and ‘wellbeing’ metrics:

Source: DCMS Rapid Evidence Assessment: Culture and Heritage Valuation Studies

Subjective wellbeing (SWB) was one of the indicators used in the 2016-2021 PfG. In Northern Ireland, four subjective wellbeing questions are asked each year as part of the Continuous Household Survey.

The researchers of DCMS’ Rapid Evidence Assessment suggested that measuring the impact of engagement with culture and heritage on a person’s subjective wellbeing (SWB) is a promising approach as it can encapsulate the health, educational and economic benefits. However, it is not well documented in the academic literature. In order to monetise this measurement, it needs to be combined with data about the life satisfaction an individual gets from a change in income.

Valuing subjective wellbeing with income compensation

In 2010, DCMS led the Culture and Sport Evidence (CASE) programme and used data from the Taking Part survey to develop new methods of scrutiny and management, using subjective wellbeing (SWB).

Similar to the recent DCMS study, the 2010 research aimed to understand ‘…what drives people to engage, what the impacts of that engagement might be, and how we might value that engagement for economic appraisal.’ The reason was ‘a fundamental issue for culture (…) policy is to understand whether the use of government funding to promote engagement in culture (…) is value for money.’

The use of SWB measures offers a way to overcome the lack of evidence on the value of engagement in culture (…). Compared to a traditional economic approach, which focus on people’s willingness to pay, the SWB approach also has the potential to value engagement in a way that was much more relevant to the sector’s key strength – enhancing people’s quality of life.

Controlling for factors such as demographics, socio-economic status etc., an income compensation (IC) approach was combined with subjective wellbeing as follows;

1. Survey data is used to estimate how a person’s wellbeing changes when they engage in culture;
2. The increase in wellbeing caused by an increase in income is also estimated;
3. A monetary value is worked out by estimating the change in income that would generate the same change in wellbeing associated with engagement in culture.

The example featured in the CASE study estimated that engaging with cultural activities at least once a week generated the subjective wellbeing equivalent of a £9,000 increase in annual household income. This figure is larger than expected and the researchers suggested further work was needed to generate better measures of engagement and factors that impact SWB.

The Commission on Wellbeing and Policy similarly concluded in 2014 that:

The only way forward is through direct measurement of wellbeing and causal models of how it is determined. This is still in its infancy. (…) And, as knowledge accumulates, the evaluation of specific projects in terms of wellbeing outcomes will become increasingly feasible.

This sort of information could be used for different scrutiny contexts. For example, to compare the effect of cultural engagement with other policy outcomes such as improved health or reduced unemployment on SWB. It also supports the assessment of value for money of culture and heritage policy options that contribute to outcomes related to health and wellbeing, such as social prescribing. In 2019, the World Health Organisation evaluated over 3000 studies that provided an evidence base of ‘…the role for the arts in the prevention of ill health, promotion of health and management and treatment of illness across people’s lifespans’.

Cross disciplinary approaches to valuing culture – social prescribing

In March 2021, the Millennium Forum in Derry/Londonderry hosted the first National Social Prescribing Day event, which included ACNI funded projects. In its response to the Department of Health’s consultation on its Mental Health Strategy in March 2021, ACNI responded, noting:

To reduce the burden placed on GPs, and in line with the emphasis on prevention and early intervention, the Arts Council would encourage greater investment in, and use of, the social prescribing model. This would include arts provision as a central resource.

We would advocate for person-centred care and in particular the role of the arts to enable people to live full and meaningful lives. This resonates particularly strongly in support for older people in care home settings.

Further research into the use of Subjective Wellbeing (SWB) and engagement with culture and heritage to measure progress towards PfG outcomes related to health, education, community and the economy are potential issues for further consideration.

Public services creating new value  

In 2020, the BBC commissioned researchers from UCL to prototype a new evaluation framework to measure its value. The researchers set out to develop a methodology that moved away from thinking of the BBC as a publicly funded cultural institution that solely exists to fix market failures. They cited the BBC’s development of the iPlayer as an example where public money was invested in a new value creating development.

In 2004, the former BBC Trust developed a simple public value test, that was used by decision makers before deciding whether to fund new BBC services. The test asked, ‘what benefits do individual users directly gain from using a BBC service?’ and ‘what value is created for society?’. This two level approach is similar to the two levels of OBA used by the NI Executive – population (society) accountability and performance (individual) accountability. The UCL study proposed that the BBC also considers its impact at the industrial level.

Examples of new value creation indicators proposed by the researchers included the measurement of:

This prototype framework is still at an early stage of development. The concept of how to measure ‘new’ value needs further testing to ensure the additional work involved to collect and analyse data adds proportional value to decision makers’ scrutiny and performance management processes. In the example given, gathering commercially sensitive information about companies’ investment costs would be a considerable challenge. However, the concept of public services creating ‘new’ value, as opposed to solely fixing an existing market failure, is perhaps worth further consideration.

Overall conclusions

The issues raised in parts 1 and 2 of these blog posts can be summarised as follows: first, although Outcomes Based Accountability (OBA) is the Executive’s system of choice for monitoring progress for its Programme for Government, there may be issues with its structure. These include the process of attributing change at both an individual and collective level to specific policies and managing the differences between performance and population accountability measures. Second, there are several different approaches to measuring wellbeing and the contributing impacts of cultural and heritage activities. These require a mix of quantitative and qualitative methods, as well a cross disciplinary approach. Further research is required to gather, analyse and use wellbeing data to meaningfully scrutinise policy and evaluate progress towards outcomes.

The challenge for the Executive, the Assembly and the cultural sector in NI is to ensure that the impact of culture on multiple PfG outcomes is accurately measured and reflected. These different models of wellbeing measurement may be of value in considering how this might be best achieved.