Claudia Harris

The outcome measures of the CEC

Now two years since it gained a Chief Executive and began to hire its network of staff, the Careers & Enterprise Company (CEC) released its 2nd Annual Round-up earlier this month. The report updates on the Company’s progress in expanding its networks of employer engagement and building it’s research base.

As with previous communications, the Round-up is a polished document full of the praises and progress the Company has made since it’s inception. I’ve posted previously on the difficulty on pinning the Company down on the exact numbers of Enterprise Advisers they have hired but this document does update an exact number.

cec2

While, on the previous page, the less precise term is used for the number of schools enrolled in the scheme.

cec1

If “more than” 1700 schools are signed up to the scheme does that mean that some schools are waiting for Enterprise Advisers to be matched with?

With the most recent DfE release showing that there are 3401 Secondary Schools and the AoC Key Facts showing 359 Further Education Colleges in England that means around 45% have now been paired with an Enterprise Adviser.

The Round Up continues to outline the progress and plans across four areas:

  1. Building local networks (Enterprise Advisers)
  2. Finding out what works (research)
  3. Backing proven ideas (investment funds)
  4. Providing online CEIAG resources (the Compass rating tool and the forthcoming Enterprise Passport)

Which are all full of detail on the admirable ambitions of the CEC. Much providence is given to the underlying research backing for this type of work provided by the studies from the Education & Employers Taskforce and the Gatsby Standards.

cec3

This reliance on the evidence base is to be welcomed and the Taskforce is clear that quantity of engagements is vital to improved outcomes for young people.

  • Quantity matters: greater volume of school-mediated employer engagement is associated with better economic outcomes, demonstrating relationships between the number of school-mediated teenage engagements with employers recalled by young adults and significantly reduced incidence of being NEET.

What isn’t mentioned in the Round Up though is any judgement on the quality of those interactions which their networks have enabled. The Taskforce is also clear that this is a factor in the value of the outcomes achieved by young people

  • Quality matters: more highly regarded employer engagement is associated with better economic outcomes. Analysis presented here shows a consistent relationship between higher regard for school-mediated provision and adult economic outcomes.  It suggests that the instincts of young adults were right: that the schools had prepared them better than comparator peers. Wage premiums in excess of 20% are found linked to higher volumes of employer engagement activities described, in general terms, as having been helpful.

More provision matters but more provision students remember as being helpful really matters. Which shows that, as any practitioner who has run an event with a before & after student view questionnaire will know, evaluation of provision is a vital step in ongoing quality control. While this is something individual providers and organisations will (should) be doing to monitor their own impacts, it is not mentioned that the CEC is collating or monitoring this feedback.

This lack of information on the quality of provision is a hint at the lack of wider absence from the Round-up of outcomes for students, whether gleaned from qualitative or quantitative data.

In the announcement of its formation, the DfE said that the new Company would fulfill a number of remits including

  • provide feedback to government on how well young people are being prepared for work

This was expanded on by the then Education Secretary Nicky Morgan, who at an Education Select Committee appearance on the 9th September 2015, said the Company would be judged by “asking young people at the end of this academic and going forward, where they aware of all the options, when did they receive advice, who came into their school to tell them about all the options” and by asking employers “are young people more work ready, more aware of the options that are out there” and listening to employer feedback. In an answer to Ian Mearns MP, she then said that “more emphasis on destination data, tracking where pupils go” would be a key indicator. When asked if she would return to the Committee with evidence of progress from the CEC Morgan promised to return within the Parliament. Of course since September 2015 a lot of water has passed under the political bridge including Cabinet reshuffles, Brexit and a fluffed General Election. With, at the time of writing, a new Education Select Committee chair and membership waiting to be elected, one of the items in a hugely packed education sphere competing for attention from the Committee should be to ask for this promise to be followed up by Morgan’s successor.

There have also been questions in the House to the DfE Ministerial team about the progress of the CEC in meeting its remit. Firstly on 25th January 2016, Sam Gyimah fielded questions with the claim of “significant progress” as evidenced by the hiring of enterprise advisers, the launch of a fund and the forthcoming (then and still now) enterprise passport. There was also the promise of a Careers Strategy in the (for what it’s worth, ahem, still waiting) Spring to further assist schools in their work with the CEC. Again none of this includes monitoring or evaluating the outcomes of any of this work.

On 7th March 2016, Gyimah again took questions on the CEC and again claimed “excellent progress in opening up schools to the world of work.” As well as taking a swipe at Careers Advice, Gyimah promoted the CEC Mentoring scheme and that “every school will have an “Enterprise Adviser.” The session passed with no information on measuring outcomes for students.

The CEC has also appeared in front of MPs during a session of the Sub-Committee on Education Skills and the Economy on the 26th March 2016. With Sam Gyimah unable to attend because of illness, Claudia Harris (alongside Ofsted and the National Careers Service) took questions from a bunch of fairly unprepared MPs who had not heard of the CEC’s work on cold spots or on Government’s own research on employer engagement. The MP’s mainly focused on the “umbrella” work of the CEC to raise engagement provision in areas where this was not happening, on understanding the structural layout of the CEC and the National Careers Service and testing the potential overlaps between the two. Harris was asked about the quality assessment of the work of Enterprise Advisers and promised that schools and colleges will be surveyed on their views of the work of their Advisers. The most important question on the outcomes for students comes at 16.52 in the link above. Harris says the CEC will be measuring 3 outcomes:

  1. Penetration – the numbers of pupils and schools involved
  2. Satisfaction – asking schools if this provision is helping
  3. Impact – working from a baseline in every school, the CEC will monitor how provision has increased

Then Catherine McKinnell MP asks a vital follow-up question “Is there not a risk that there will be a focus on quantity rather than quality,” to which Harris offers

  1. A literature review conducted by the CEC looking at the effectiveness of mentoring as an employer engagement activity so directing funding what works
  2. A series of “deep dive” focus groups where representatives from the CEC will speak to students on their views of engagement provision they have attended and what help it offered them

The absence of two of the outcomes mentioned by Morgan in her session is noticeable. The omission of feedback from employers and student destination statistics is perhaps wise as these are outcome measures not wholly in the control of the CEC and those with conflicting data points with no clear definition from Government on what would be measured. Would a reduction in the of 16-19 NEETS be a plus mark for the CEC or a rise in employer satisfaction of school leaver skills be evidence of the impact of provision? And from which survey source would this be, those conducted by Government or those conducted by business? Or would the only satisfactory judgement be made by the sort of longitudinal research conducted by the Education & Employers Taskforce? This lack of clarity of definable targets continued in a further Committee session (27th April 2016) with the witnesses Nick Boles MP and Sam Gyimah MP where the conversation on quality monitoring of the CEC is sidetracked onto the Dfe Statutory Guidance.

Having to scour Select Committee archives for definitions of the student focused outcome measures of the CEC is indicative of the lack of clarity from the DfE around this issue. If we take Morgan’s comments (as the initalising Secretary of State) as gospel then achieving the tasks set of assuaging the concerns of business and reshaping destination statstics will be no mean feat for the CEC to achieve. Only today the CBI released it’s 2017 Annual Skills Survey. The results include businesses views on the workplace skills of school leavers

cbi1

with much to be improved upon and contained the fact that only 21% of businesses “are currently aware” of the activities of the CEC. The fear must be that this is a taskmaster who will never be satisfied.

The ultimate quality of the enabling and linking work the CEC delivers will be decided by those volunteers, staff and practitioners on the ground organising and running the face to face provision with young people. Through its short operating period so far, the CEC has focused on the growth of its structure and operations as evidence of its progress. Soon this attention should change though onto the impact of provision and student outcomes to evaluate that public investment committed to the Company.

 

Advertisements