The Cold Spots accountability hole

 

Following on from the previous post on this blog looking at how the non publication of apprenticeship vacancy, starts, registrations and applications data by age will mean an accountability hole when judging the progress of the Careers Strategy and schools guidance documents, this is a sequel post of sorts looking at another data accountability gap that will cause the Careers & Enterprise Company some problems.

The release of the Company’s Cold Spots research in 2015 drew together a number of data sources from other Government departments and quangos to map the weaker and stronger areas of employer engagement focused careers provision across England. This audit was useful as it allowed the Company to focus pilot schemes and target initial provision into the locations that needed it the most.

The Company has recently released a short, 2017 update to that original Cold Spots report that, according to those external data sources, shows a “warming” in career outcomes for young people across England.

coldspots1

This map shows that only one Local Enterprise Partnership area (Thames Valley Berkshire) has regressed and was now returning a higher number of cold spot indicators than in 2015. As the report itself says though, “it is too early to make claims about causality” and this is included for good reason. The original 2015 Cold Spots were based on 9 external data sources

Deprivation indicator:
– % Pupils known to be eligible for and claiming free school meals (FSM)1
2013/14
Employer engagement indicators i.e., “cold spots”
– % Employer establishments who had anyone in on work experience2 in the
last 12 months
– % Employer establishments who offered any work inspiration3 in the last
12 months
Outcome indicators:
– % Pupils attaining 5A*-C GCSE results in England 2013 – 14
– % A-levels entered that are STEM4 2013 – 14
– % STEM4 A-levels that are entered by girls 2013 – 145
– % In sustained apprenticeship destinations post key stage 4 (KS4) 2012/13
– % 16-17 year olds NEET (not in education, employment and training), as
reported by LA in June 2015
– % Employers answering: 16 year old school leavers are “poorly” or “very
poorly prepared” for work
– % Employers answering: 17-18 year olds recruited to first time job from
school are “poorly” or “very poorly prepared” for work

The 2016 update continued to use these sources but now the 2017 update finds itself in a quandary as two of those sources (the two employer returns from the UKCES employer survey data looking at satisfaction of school leaver skills & the offers of work experience and work inspiration activities from businesses) are no longer reporting in the same manner. This is due to the closure of UKCES. The responsibility to continue the survey moved to the DfE but the data gathered will be from a smaller sample size (around 18,000 telephone interviews in the 2016 edition vs over 91,000 telephone and face to face interviews in the UKCES editions) leaving the CEC with a dilemma. They need to both show progress on the continuing funded work both in cold spot areas (opportunity areas in Government speak) across the country but also to show the distance traveled from the starting point since the CEC’s inception this data has to be somewhat comparable year on year.

This leaves the CEC relying on GCSE results data and student destination data which are useful outputs to monitor but are one-sided in focusing on the supply side of students entering the workplace. The views of the demand side from employers would not now be comparable across past years.

Thus the recent publication ends with a consultative call for suggestions on which data points to use to achieve this. The CEC should be wary about using data supplied by employer bodies such as the CBI as, historically, this has been much more scathing on the work readiness of school leavers entering the labour market and much more positive about the contribution of business offering experiences to young people. The UKCES returns told a story of employers being much more satisfied with the employability skills of young people and of a significantly smaller amount of engagement provision with education. So the first stipulation for any new data sources the CEC use, would be that they should be from impartial sources. On the flip side to this coin, data supplied by LEPs should also be considered with an arched eyebrow for they will be keen to champion the success of Government funding in their own patch.

It’s also worth pointing out that the improving “warming” outcomes are in direct opposition to survey results from young people who report a lower number of employer engagements last academic year.

Does asking young people what they actually experienced meet the criteria the CEC is looking for?

Another factor for the CEC to consider is that trends in some of these data points are very much at the whim of changeable Government policy. Putting aside the example of UKCES closing it’s doors, using the number of KS4 leavers in sustained Apprenticeship destinations is commendable but since 2015 the Apprenticeship Levy has reshaped that sector, initially caused a drop in overall numbers of starts and begun to grow the provision that is left towards higher and degree apprenticeships and away from the Level 2 Apprenticeships open to 16-year-old GCSE leavers. The Higher Education Funding review could yet again change levels of tuition fees and so impact the desired destinations of young people. Perhaps the case is being made for the CEC to allocate some of its funding to tender for its own data collections and not be reliant on other arms of the State but, at a scale similar to UKCES level data collection, this would need significant investment.

It is the task of the CEC, to make a quantifiable impact on an area of public policy with multiple inputs and multiple outputs and, with their expanded remit in the Careers Strategy, the number of inputs will only grow. Getting the data points right to measure that impact is proving tricky.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s