This two-part blog post is a follow-up to a series of three articles about Outcomes Based Accountability (OBA) and the Programme for Government (PfG) draft Outcomes Framework.
Yesterday, part 1 of this blog article considered the changes to culture and heritage related outcomes in the draft PfG, published in January 2021. Part 2 now explores different approaches to valuing culture and heritage and considers the potential of these metrics as methods for scrutinising the value of future culture and heritage policies.
A new framework for valuing culture and heritage
Some of the difficulties of valuing culture and heritage have been examined by the Department of Digital, Culture, Media and Sport (DCMS). One DCMS initiative resulted from discovering that The Green Book, which contains guidance issued by HM Treasury on how to appraise policies and programmes, currently had no guidance specific to culture and heritage.
There is growing real-world demand from policymakers and practitioners to monetise the welfare impacts of cultural policies, and to quantify culture and heritage goods and services in a way that aligns with cost benefit analysis and business cases in other sectors.
To help improve the articulation of the value of culture and heritage, in January 2021, DCMS published a new framework for valuing cultural and heritage assets. An aim of this approach was to translate changes resulting from increased engagement in culture and heritage into a monetary value, to support evidence based decision making.
The Culture and Heritage Capital Framework was based on a model developed in 2018 by the Department for Environment, Food and Rural Affairs. It aimed to demonstrate how the benefits or ‘flows’ from culture and heritage assets (stocks) can contribute to the achievement of outcomes, both at an individual (interventions) and population (pressures) level.
The DCMS report noted that although ‘there is a wealth of literature aiming to value different aspects of culture, art and heritage, there exists a lack of a formalised approach’. DCMS commissioned a Rapid Evidence Assessment (REA) of different approaches to valuing culture and heritage. Although a number of further areas of research were also identified, the main methods of valuation included a mix of ‘willingness to pay’ and ‘wellbeing’ metrics:
| Contingent valuation: This method asks people to directly report their willingness to pay (WTP) to obtain a specified good, or willingness to accept (WTA) to give up a good.
Choice modelling: Individuals are not directly asked for their willingness to pay, but rather their valuations are derived from their responses to a choice of options. Hedonic pricing: This is a revealed preference method that looks at the influence of a good/service on prices in a related market, for example, house prices. Travel cost: Here, willingness to pay is derived from the amount of time and money people are willing to spend travelling to consume a good or service. Subjective wellbeing (SWB): This asks people directly how they think and feel about their own wellbeing, and includes aspects such as life satisfaction, feeling positive, and whether their life is meaningful. Department of Health and Social Care. Value can be calculated by relating changes to income to changes in wellbeing. Quality Adjusted Life Years (QALYs): This is a measure of health status in terms of the quality of life associated with a state of health, and the number of years for which that health status is enjoyed. Department of Health and Social Care. |
Source: DCMS Rapid Evidence Assessment: Culture and Heritage Valuation Studies
Subjective wellbeing (SWB) was one of the indicators used in the 2016-2021 PfG. In Northern Ireland, four subjective wellbeing questions are asked each year as part of the Continuous Household Survey.
The researchers of DCMS’ Rapid Evidence Assessment suggested that measuring the impact of engagement with culture and heritage on a person’s subjective wellbeing (SWB) is a promising approach as it can encapsulate the health, educational and economic benefits. However, it is not well documented in the academic literature. In order to monetise this measurement, it needs to be combined with data about the life satisfaction an individual gets from a change in income.
Valuing subjective wellbeing with income compensation
In 2010, DCMS led the Culture and Sport Evidence (CASE) programme and used data from the Taking Part survey to develop new methods of scrutiny and management, using subjective wellbeing (SWB).
Similar to the recent DCMS study, the 2010 research aimed to understand ‘…what drives people to engage, what the impacts of that engagement might be, and how we might value that engagement for economic appraisal.’ The reason was ‘a fundamental issue for culture (…) policy is to understand whether the use of government funding to promote engagement in culture (…) is value for money.’
The use of SWB measures offers a way to overcome the lack of evidence on the value of engagement in culture (…). Compared to a traditional economic approach, which focus on people’s willingness to pay, the SWB approach also has the potential to value engagement in a way that was much more relevant to the sector’s key strength – enhancing people’s quality of life.
Controlling for factors such as demographics, socio-economic status etc., an income compensation (IC) approach was combined with subjective wellbeing as follows;
- Survey data is used to estimate how a person’s wellbeing changes when they engage in culture;
- The increase in wellbeing caused by an increase in income is also estimated;
- A monetary value is worked out by estimating the change in income that would generate the same change in wellbeing associated with engagement in culture.
The example featured in the CASE study estimated that engaging with cultural activities at least once a week generated the subjective wellbeing equivalent of a £9,000 increase in annual household income. This figure is larger than expected and the researchers suggested further work was needed to generate better measures of engagement and factors that impact SWB.
The Commission on Wellbeing and Policy similarly concluded in 2014 that:
The only way forward is through direct measurement of wellbeing and causal models of how it is determined. This is still in its infancy. (…) And, as knowledge accumulates, the evaluation of specific projects in terms of wellbeing outcomes will become increasingly feasible.
This sort of information could be used for different scrutiny contexts. For example, to compare the effect of cultural engagement with other policy outcomes such as improved health or reduced unemployment on SWB. It also supports the assessment of value for money of culture and heritage policy options that contribute to outcomes related to health and wellbeing, such as social prescribing. In 2019, the World Health Organisation evaluated over 3000 studies that provided an evidence base of ‘…the role for the arts in the prevention of ill health, promotion of health and management and treatment of illness across people’s lifespans’.
Cross disciplinary approaches to valuing culture – social prescribing
In March 2021, the Millennium Forum in Derry/Londonderry hosted the first National Social Prescribing Day event, which included ACNI funded projects. In its response to the Department of Health’s consultation on its Mental Health Strategy in March 2021, ACNI responded, noting:
To reduce the burden placed on GPs, and in line with the emphasis on prevention and early intervention, the Arts Council would encourage greater investment in, and use of, the social prescribing model. This would include arts provision as a central resource.
We would advocate for person-centred care and in particular the role of the arts to enable people to live full and meaningful lives. This resonates particularly strongly in support for older people in care home settings.
Further research into the use of Subjective Wellbeing (SWB) and engagement with culture and heritage to measure progress towards PfG outcomes related to health, education, community and the economy are potential issues for further consideration.
Public services creating new value
In 2020, the BBC commissioned researchers from UCL to prototype a new evaluation framework to measure its value. The researchers set out to develop a methodology that moved away from thinking of the BBC as a publicly funded cultural institution that solely exists to fix market failures. They cited the BBC’s development of the iPlayer as an example where public money was invested in a new value creating development.
In 2004, the former BBC Trust developed a simple public value test, that was used by decision makers before deciding whether to fund new BBC services. The test asked, ‘what benefits do individual users directly gain from using a BBC service?’ and ‘what value is created for society?’. This two level approach is similar to the two levels of OBA used by the NI Executive – population (society) accountability and performance (individual) accountability. The UCL study proposed that the BBC also considers its impact at the industrial level.
Examples of new value creation indicators proposed by the researchers included the measurement of:
| Individual | Societal | Industry |
| Individual behaviour changes e.g. number of households that reduced their use of plastics after Blue Planet II was broadcast | Change in population attitudes e.g. women playing sports as a result of broadcasting more female players on television | De-risking industry innovation e.g. launching iPlayer de-risks investment in internet streaming technology, reducing the investment costs for other companies |
This prototype framework is still at an early stage of development. The concept of how to measure ‘new’ value needs further testing to ensure the additional work involved to collect and analyse data adds proportional value to decision makers’ scrutiny and performance management processes. In the example given, gathering commercially sensitive information about companies’ investment costs would be a considerable challenge. However, the concept of public services creating ‘new’ value, as opposed to solely fixing an existing market failure, is perhaps worth further consideration.
Overall conclusions
The issues raised in parts 1 and 2 of these blog posts can be summarised as follows: first, although Outcomes Based Accountability (OBA) is the Executive’s system of choice for monitoring progress for its Programme for Government, there may be issues with its structure. These include the process of attributing change at both an individual and collective level to specific policies and managing the differences between performance and population accountability measures. Second, there are several different approaches to measuring wellbeing and the contributing impacts of cultural and heritage activities. These require a mix of quantitative and qualitative methods, as well a cross disciplinary approach. Further research is required to gather, analyse and use wellbeing data to meaningfully scrutinise policy and evaluate progress towards outcomes.
The challenge for the Executive, the Assembly and the cultural sector in NI is to ensure that the impact of culture on multiple PfG outcomes is accurately measured and reflected. These different models of wellbeing measurement may be of value in considering how this might be best achieved.