enterprise advisers

The outcome measures of the CEC

Now two years since it gained a Chief Executive and began to hire its network of staff, the Careers & Enterprise Company (CEC) released its 2nd Annual Round-up earlier this month. The report updates on the Company’s progress in expanding its networks of employer engagement and building it’s research base.

As with previous communications, the Round-up is a polished document full of the praises and progress the Company has made since it’s inception. I’ve posted previously on the difficulty on pinning the Company down on the exact numbers of Enterprise Advisers they have hired but this document does update an exact number.

cec2

While, on the previous page, the less precise term is used for the number of schools enrolled in the scheme.

cec1

If “more than” 1700 schools are signed up to the scheme does that mean that some schools are waiting for Enterprise Advisers to be matched with?

With the most recent DfE release showing that there are 3401 Secondary Schools and the AoC Key Facts showing 359 Further Education Colleges in England that means around 45% have now been paired with an Enterprise Adviser.

The Round Up continues to outline the progress and plans across four areas:

  1. Building local networks (Enterprise Advisers)
  2. Finding out what works (research)
  3. Backing proven ideas (investment funds)
  4. Providing online CEIAG resources (the Compass rating tool and the forthcoming Enterprise Passport)

Which are all full of detail on the admirable ambitions of the CEC. Much providence is given to the underlying research backing for this type of work provided by the studies from the Education & Employers Taskforce and the Gatsby Standards.

cec3

This reliance on the evidence base is to be welcomed and the Taskforce is clear that quantity of engagements is vital to improved outcomes for young people.

  • Quantity matters: greater volume of school-mediated employer engagement is associated with better economic outcomes, demonstrating relationships between the number of school-mediated teenage engagements with employers recalled by young adults and significantly reduced incidence of being NEET.

What isn’t mentioned in the Round Up though is any judgement on the quality of those interactions which their networks have enabled. The Taskforce is also clear that this is a factor in the value of the outcomes achieved by young people

  • Quality matters: more highly regarded employer engagement is associated with better economic outcomes. Analysis presented here shows a consistent relationship between higher regard for school-mediated provision and adult economic outcomes.  It suggests that the instincts of young adults were right: that the schools had prepared them better than comparator peers. Wage premiums in excess of 20% are found linked to higher volumes of employer engagement activities described, in general terms, as having been helpful.

More provision matters but more provision students remember as being helpful really matters. Which shows that, as any practitioner who has run an event with a before & after student view questionnaire will know, evaluation of provision is a vital step in ongoing quality control. While this is something individual providers and organisations will (should) be doing to monitor their own impacts, it is not mentioned that the CEC is collating or monitoring this feedback.

This lack of information on the quality of provision is a hint at the lack of wider absence from the Round-up of outcomes for students, whether gleaned from qualitative or quantitative data.

In the announcement of its formation, the DfE said that the new Company would fulfill a number of remits including

  • provide feedback to government on how well young people are being prepared for work

This was expanded on by the then Education Secretary Nicky Morgan, who at an Education Select Committee appearance on the 9th September 2015, said the Company would be judged by “asking young people at the end of this academic and going forward, where they aware of all the options, when did they receive advice, who came into their school to tell them about all the options” and by asking employers “are young people more work ready, more aware of the options that are out there” and listening to employer feedback. In an answer to Ian Mearns MP, she then said that “more emphasis on destination data, tracking where pupils go” would be a key indicator. When asked if she would return to the Committee with evidence of progress from the CEC Morgan promised to return within the Parliament. Of course since September 2015 a lot of water has passed under the political bridge including Cabinet reshuffles, Brexit and a fluffed General Election. With, at the time of writing, a new Education Select Committee chair and membership waiting to be elected, one of the items in a hugely packed education sphere competing for attention from the Committee should be to ask for this promise to be followed up by Morgan’s successor.

There have also been questions in the House to the DfE Ministerial team about the progress of the CEC in meeting its remit. Firstly on 25th January 2016, Sam Gyimah fielded questions with the claim of “significant progress” as evidenced by the hiring of enterprise advisers, the launch of a fund and the forthcoming (then and still now) enterprise passport. There was also the promise of a Careers Strategy in the (for what it’s worth, ahem, still waiting) Spring to further assist schools in their work with the CEC. Again none of this includes monitoring or evaluating the outcomes of any of this work.

On 7th March 2016, Gyimah again took questions on the CEC and again claimed “excellent progress in opening up schools to the world of work.” As well as taking a swipe at Careers Advice, Gyimah promoted the CEC Mentoring scheme and that “every school will have an “Enterprise Adviser.” The session passed with no information on measuring outcomes for students.

The CEC has also appeared in front of MPs during a session of the Sub-Committee on Education Skills and the Economy on the 26th March 2016. With Sam Gyimah unable to attend because of illness, Claudia Harris (alongside Ofsted and the National Careers Service) took questions from a bunch of fairly unprepared MPs who had not heard of the CEC’s work on cold spots or on Government’s own research on employer engagement. The MP’s mainly focused on the “umbrella” work of the CEC to raise engagement provision in areas where this was not happening, on understanding the structural layout of the CEC and the National Careers Service and testing the potential overlaps between the two. Harris was asked about the quality assessment of the work of Enterprise Advisers and promised that schools and colleges will be surveyed on their views of the work of their Advisers. The most important question on the outcomes for students comes at 16.52 in the link above. Harris says the CEC will be measuring 3 outcomes:

  1. Penetration – the numbers of pupils and schools involved
  2. Satisfaction – asking schools if this provision is helping
  3. Impact – working from a baseline in every school, the CEC will monitor how provision has increased

Then Catherine McKinnell MP asks a vital follow-up question “Is there not a risk that there will be a focus on quantity rather than quality,” to which Harris offers

  1. A literature review conducted by the CEC looking at the effectiveness of mentoring as an employer engagement activity so directing funding what works
  2. A series of “deep dive” focus groups where representatives from the CEC will speak to students on their views of engagement provision they have attended and what help it offered them

The absence of two of the outcomes mentioned by Morgan in her session is noticeable. The omission of feedback from employers and student destination statistics is perhaps wise as these are outcome measures not wholly in the control of the CEC and those with conflicting data points with no clear definition from Government on what would be measured. Would a reduction in the of 16-19 NEETS be a plus mark for the CEC or a rise in employer satisfaction of school leaver skills be evidence of the impact of provision? And from which survey source would this be, those conducted by Government or those conducted by business? Or would the only satisfactory judgement be made by the sort of longitudinal research conducted by the Education & Employers Taskforce? This lack of clarity of definable targets continued in a further Committee session (27th April 2016) with the witnesses Nick Boles MP and Sam Gyimah MP where the conversation on quality monitoring of the CEC is sidetracked onto the Dfe Statutory Guidance.

Having to scour Select Committee archives for definitions of the student focused outcome measures of the CEC is indicative of the lack of clarity from the DfE around this issue. If we take Morgan’s comments (as the initalising Secretary of State) as gospel then achieving the tasks set of assuaging the concerns of business and reshaping destination statstics will be no mean feat for the CEC to achieve. Only today the CBI released it’s 2017 Annual Skills Survey. The results include businesses views on the workplace skills of school leavers

cbi1

with much to be improved upon and contained the fact that only 21% of businesses “are currently aware” of the activities of the CEC. The fear must be that this is a taskmaster who will never be satisfied.

The ultimate quality of the enabling and linking work the CEC delivers will be decided by those volunteers, staff and practitioners on the ground organising and running the face to face provision with young people. Through its short operating period so far, the CEC has focused on the growth of its structure and operations as evidence of its progress. Soon this attention should change though onto the impact of provision and student outcomes to evaluate that public investment committed to the Company.

 

The CEC needs more than PR to judge it’s progress

The DfE plan to address shortcomings in school CEIAG provision rests heavily upon the success of the Careers & Enterprise Company (CEC). The work of the CEC is becoming more diversified as it begins to fund other activities beyond it’s original thread of recruiting and organising a nationwide army of Enterprise Advisers corralled by a smaller squadron of Enterprise Coordinators to work with educational institutions. This has required a recruitment drive which, the CEC would say, has gone amazingly well

but, anecdotally,

has struggled to gain traction across the sector.

The number of Enterprise Advisers working with the CEC has been regularly mentioned in their PR communications.

In June 2016, the CEC released their first annual report “Joining the Dots” which stated

cec1

Then, in October 2016, a Government response to a joint Education & Business committee report stated that the CEC had

already appointed 78 coordinators and almost 1200 advisers

and that

Over 700 schools and colleges (in 37 out of 38 Local Enterprise Partnership areas in England) have been helped to develop better careers and enterprise programmes for their pupils

 

By November 2016, this had grown to “more than 1300.”

We have recruited more than 1,300 Enterprise Advisers

In January 2017, the numbers touted were

A third of schools and colleges in the country are currently matched with an Enterprise Adviser – senior business volunteers connecting more than 1,300 schools and colleges

Last week the CEC released another press statement to celebrate the news that the number of Enterprise Advisers had grown

More than 1,300 senior employers from small family-owned firms to global corporations are working with headteachers across the country to help shape career programmes and employer engagement plans since the Company started operations just over 18-months ago.

and that this meant

that the government-backed Company has gone from a standing start to pairing business volunteers with almost half of all secondary schools and colleges in England with a combined population of more than 1.3million students.

which sounds very impressive but is ensconced in the language of public relations. “More than 1,300 senior employers,” “more than 1.3 million students” which isn’t very specific when reporting outcomes of public expenditure.

The statistic there regarding Enterprise Advisers is doubly vague as it refers to “employers” (rather than the November 2016 update which mentions Advisers) which could include (multiple) examples of more than one employee from the same employer offering to volunteer. This would mean that the number of actual Advisers could be a lot more than 1300 yet this figure was also used to refer to the number of Advisers by the CEC in March 2017

Which begs the question, just how many Advisers have the CEC recruited?

This possible figure is also confused by the statement “almost half of all secondary schools and colleges in England” are now matched with Advisers.

In September 2016 there were 325 Colleges in England.

The DfE 2016 annual statistics bulletin shows there are 3401 Secondary schools in England.

secondary schools1

So that’s a total of 3726.

50% = 1863 which is a little more than 1300 which is described as “nearly half.”

If we speculate that the number of actual Enterprise Advisers is 1350 that would equal 36% of schools and colleges in England are matched with an Adviser which is closer to the “third” mentioned in the January 2017 press notice.

This guesswork is just that, guesswork. If the CEC publishes its annual report in June again, it will be a few months until a clear figure is published, with only confusing and contradictory press notices to rely upon.

The CEC is (currently) a publicly funded organisation with a number of different funding announcements comprising its total cash injection

cec2

Earlier this week a much larger pot of Government funding received scrutiny from the Public Accounts Committee when they looked at the National Citizenship Service (NCS) and concluded that it was failing to meet recruitment targets, failing to disclose directors salaries, not providing value for money and failing to determine outcomes for young people for the high level of public investment received. The actual report is forensic. While dealing with a much smaller funding pot to the NCS, the CEC should also receive this sort of scrutiny.