The Department of Business, Energy and Industrial Strategy (BEIS) commissioned CAG Consultants to evaluate the Transitional Arrangements (TA) for Demand-Side Response (DSR), in partnership with Databuild, Verco and NERA Economic Consulting.  The TA forms part of the Capacity Market for electricity in Great Britain, and aims to increase the provision of flexible capacity that can help to balance supply and demand in the electricity grid.  While Phases 1 and 2 of the evaluation covered the first TA auction in 2016/2017, Phases 3 and 4 cover the second TA auction in 2017/18.

How we delivered the project

Our approach to this evaluation is realist and theory-based. We have developed a theoretical framework for the TA, setting out for whom, and in what circumstances (i.e. in what ‘contexts’), the policy is expected to lead to particular reasoning and choices being made (i.e. causal ‘mechanisms’ being activated), leading to desired or undesired policy ‘outcomes’. These realist hypotheses are generally known as context-mechanism-outcome combinations or ‘CMOs’. 

In each Phase, we test the evolving theoretical framework using research evidence gathered through:

• Literature survey and consultation with external stakeholders
• Analysis of the Capacity Market register and TA auction behaviour
• Analysis of cost-effectiveness
• Email surveys and in-depth interviews with TA participants
• In-depth interviews with potential TA participants
• Case study analysis of electricity meter data for TA participants

We use realist contribution analysis to develop a ‘contribution story’ for each participant in the TA (expressed as a refined set of ‘CMOs’), cross-checking this against a set of evidence tests using contribution tracing with Bayesian updating.

Objectives achieved

The evaluation aims to inform BEIS’s future policy on DSR in the Capacity Market. Evidence from Phase 1 of the evaluation provided robust, external support to the changes for the second TA that restricted participation to turn-down DSR only. In addition, it was found that awareness of the CM and the TA was restricted to those already involved in electricity markets and with pre-existing relationships with National Grid or BEIS. Thus, future auctions need to be promoted amongst those with the ability to provide DSR but whose strategic objectives do not necessarily involve the engagement with energy markets.

The Phase 1 evaluation report is available here  and the Phase 2 report will be published shortly by BEIS. Please contact Mary Anderson on ma@cagconsult.co.uk to find out more.