Month: July 2017

More proof that Ofsted is not the white knight for CEIAG provision

One of the repeatedly suggested levers to pull which would improve CEIAG provision in schools is monitoring from Ofsted. Stakeholders, reports and Education Select Committee recommendations have all suggested that a secondary school’s Careers provision should be judged upon by the Inspectorate. This came to pass with the introduction of instruction for Her Majesty’s Inspectors to include comment of the quality of CEIAG provision in the 2013 and 2014 Handbook for Inspectors. This was taken as a positive step in concentrating the minds of school leaders to prioritise their CEIAG provision. Things changed though with the introduction of Ofsted’s short, targeted inspections of schools currently rated GOOD (grade 2) in September 2015.

This changed the way a large number of schools were visited and assessed and is further explained here but essentially it raised fears that, in a short, one day inspection performed by one inspector for a GOOD school that then remained GOOD, CEIAG would not be inspected. The checklist of school duties for a team of four or five inspectors to monitor over a two-day period cannot be the same for one inspector to check in a one day visit. Ofsted’s workflow guidelines for inspecting a GOOD school reflected this with the requirement for a Section 8 inspection to be carried out instead of a full Section 5.

In it’s justification for this change, Ofsted (with some rationale) argued that, in a supposedly self improving school system, an inspectorate should be focusing its resources on where the system needs them most and that would be on schools graded as 3 or 4. This was in line with the previous move to stop inspecting grade 1 schools as a matter of course and only place them under the microscope if substantive concerns were raised. Others would point out that this was an inevitable consequence of an Inspectorate tasked with inspecting growing types of provision and establishments but with a reducing budget.

Earlier this month, Ofsted released both a consultation on how short inspections had worked and the statistics for the numbers of inspections, including short inspections, from September 2016 to March 2017.

Concentrating on secondary schools, a total of 287 short inspections were carried out, of which 201 schools remained GOOD.

dfhpkeovyaaseak

A total of 535 secondary schools were inspected during this period which means that around 37% of secondary school inspections were not even required to pass comment on CEIAG during the academic year.

Ofsted also notes that the number of short inspections that converted to full Section 5 inspections is lower than last year

Twenty nine per cent of short inspections between 1 September 2016 and 31 March 2017 converted to full inspections. In 2015/16, 35% converted.

So, as schools coalesce into the higher grade boundaries but find that OUTSTANDING rating just out of reach, it seems that fewer are undergoing a full Section 5 inspection.

The folks over at Education Data Lab have already done the work in totaling up these numbers to see the complete picture of inspections that have occurred since they were introduced in 2015.

edudatalab1

So, from a total of 530 short inspections of secondary schools, 55% in those 18 months would not have been required to offer a judgement on a school’s CEIAG provision.

The most recent Ofsted Annual Report (2015-1016) shows that more secondary schools than ever are being judged as good or outstanding

ofsted1

a rise of 12 percentage points since 2011.

Both the rise in non converting short inspections and the exemption of inspection OUTSTANDING schools leading to 10 years between inspections in some cases

ofsted2

means that

a) it is increasingly less likely that a school’s CEIAG provision will not be monitored during an inspection

b) it is increasingly likely that a judgement on CEIAG provision contained in an Ofsted report is from a school that previously held a grade 3 or 4

and so then

c) the headline trend of rising numbers of GOOD and OUTSTANDING schools offers no evidence to the quality of Careers provision in schools as the Careers provision in these schools will have either i) not have been inspected during the most recent inspection or ii) been inspected quite a number of years ago.

Under the current framework of inspection, Ofsted is no substantive barometer of the quality of Careers provision in schools now and, if the current trends continue, will only be more out of snyc with provision in the future.

 

 

 

Advertisements

The logical failure of the 2017 CBI Education & Skills Survey

Like any membership lobbying organisation, the CBI support their members to make them look as good as possible and promote the greater value of and worth of business to society. They have their work cut out, surveys show that the general public is distrustful with less than half believing British businesses act ethically, so their annual survey of business leaders (The CBI/Pearson Education & Skills Survey) is a chance to shift the focus elsewhere.

The 2017 iteration is drawn from an online questionnaire completed by 344 employers. (It is worth comparing at the outset this methodology against other recent employer surveys such as the recent DfE Employer perspective survey (which I blogged on here) which was drawn from telephone interviews with over 18,000 establishments across the labour market including non-profit organisations but more on that later).

The scale of employer engagement in education is an important topic with the Careers & Enterprise Company (CEC) tasked with expanding this work to improve CEIAG provision and so prove the research evidence of the benefits to be gained for students. Future qualification and pathway policy is also heavily geared towards gaining employer buy in and engagement. This means that establishing a base point of employer engagement is vital in judging progress and knowing where to target resources. The CEC has already made progress in this area with their Cold Spots research. Using a range of data points and sources, this shows that outcomes for young people (much like HE progression and academic achievement rates) varies greatly across the different regions of the UK.

coldspots1

One of the data points used to model the amount of employer engagement is the UKCES Employer Perspectives Survey 2014. This is the preceding biannual release of the DfE Employer Perspective Survey mentioned above before the closure of the UK Commission for Employment and Skills in March 2017 which the DfE then took over.

That the CEC is using this data on employer engagement and not information published by the CBI is the first hint that the two sources on employer engagement tell very different tales.

The CBI survey offers useful views on the value business leaders places on their required skills from school leavers

cbi2

plus their satisfaction on school leaver skill levels

cbi1

and a plea for more young people to be speakers of foreign languages (page 34). The Employer Perspectives survey meanwhile does not break down employer satisfaction with school leavers into skill areas but by age of recruits (fig 3.7) so comparisons are difficult but, at all ages, it found employers were more satisfied than dissatisfied with the skill levels of their young recruits.

The survey begins to raise eyebrows though with the claims of employer engagement with education including the number of firms that offer work experience. Let’s remind ourselves what the 2017 Employer Perspectives Survey reported,

  • That 65% of employers thought that relevant work experience is a critical or significant factor when taking on a recruit but only 38% of employers had offered a work experience opportunity in the past 12 months
  • There is a huge variation between the sectors that offer work experience (fig 3.9)
  • Only 10% of employers offered work inspiration activities to students

This differs drastically to the findings of the CBI Survey that 81% of employers had some links with education

cbi3

and that these links were “extensive in all parts of the UK.” Not only is this claim widely divergent with the finding from a methodologically more detailed and comprehensive survey, it also undermines the very basis on which the CEC has prioritised its work across the country. Regarding employer links with education, the CBI says there are no cold spots.

The differences continue in the value of work experience with only 23% of businesses reporting that relevant work experience is an “important” factor when recruiting a young person (fig 2.1 pictured above) which is well below the 65% of employers reporting similar in the Employer Perspectives Survey. There is also a lack of consistency of expectations in the CBI results with employers also stating that 54% were not satisfied with school leavers relevant work experience (fig 2.3 pictured above). Why would employers be unsatisfied with something they’ve also deemed not important to recruitment?

The data on the types of provision employers offer through their links with schools is couched in a presentational sleight of hand as the percentages are offered as percentages of those business who have education links, not a percentage of the total businesses. Thus

cbi4

So the CBI is not claiming that 81% of employers provide careers advice talks but that 81% of the 81% with links to education provide careers advice talks. Because we have the total numbers of employers the CBI received responses from (344) we can work this back – 81% of 344 = 279, 81% of 279 = 225, 225 divided by 344 = the CBI is actually reporting that 65% of employers offer careers advice talks. The Employer Perspectives Survey concluded that just 10% of employers offer careers inspirations activities including careers talks.

The differences between the two surveys continue when discussing work experience. The CBI concludes (using the same method above) that around 63% of employers offer work experience placements. The Employer Perspectives Survey reported 38% of employers offered placements and that differences between industries can be stark.

skills report4

The CBI survey also includes business views on both the work of the CEC and the current state of CEIAG provision. They find that the CEC still has plenty of scope to increase their connections with business as only 7% of respondents were engaged with the Company.

cbi5

That 79% businesses were unaware of the work of the CEC is not surprising when you also consider that only 28% of employers are aware of the new GCSE grading system.

cbi6

The CBI though are wholly positive about the need and remit of the CEC

The CBI fully supports their work which has a focus on practical, enabling solutions.

and

Underpinned by sufficient resources, the CEC should play a major role in England in
supporting schools and businesses to develop productive relationships to the benefit of young people.

but the views of the businesses surveyed are extremely negative about the quality of CEIAG provision

cbi7

84% of businesses reporting that Careers advice is not good enough is an overwhelming verdict but also similar percentage to the four previous survey results show in 3.16. The CBI goes strong on its verdict on current Careers provision

These are seriously troubling results. They highlight the urgent need for radical improvement.

This all adds up to a muddled picture offered by the CBI.

They and the employers they surveyed are claiming that 81% of business have links with schools across the country, 65% offer careers advice talks and 63% offer work experience placements. This equates to a large-scale engagement with education yet, it is these same employers from whom 79% had not heard of the work of the CEC. It is from these same employers that less than a third were aware of the introduction of whole new GCSEs and grading systems. The same employers who are engaged with education to offer huge amounts of careers provision but 84% of them also reported unsatisfaction with the Careers advice offered. The solution offered by the CBI to change these views? More engagement with education through the CEC.

Establishing hard quantitative data on employer engagement is not easy as previous studies have shown. Using only limited survey data though can mean results with the failures of logic shown above. The CBI cannot continue to claim that the majority of employers are playing their part in provision only to then be overwhelmingly critical of the scale, quality and outcomes of that provision.

The outcome measures of the CEC

Now two years since it gained a Chief Executive and began to hire its network of staff, the Careers & Enterprise Company (CEC) released its 2nd Annual Round-up earlier this month. The report updates on the Company’s progress in expanding its networks of employer engagement and building it’s research base.

As with previous communications, the Round-up is a polished document full of the praises and progress the Company has made since it’s inception. I’ve posted previously on the difficulty on pinning the Company down on the exact numbers of Enterprise Advisers they have hired but this document does update an exact number.

cec2

While, on the previous page, the less precise term is used for the number of schools enrolled in the scheme.

cec1

If “more than” 1700 schools are signed up to the scheme does that mean that some schools are waiting for Enterprise Advisers to be matched with?

With the most recent DfE release showing that there are 3401 Secondary Schools and the AoC Key Facts showing 359 Further Education Colleges in England that means around 45% have now been paired with an Enterprise Adviser.

The Round Up continues to outline the progress and plans across four areas:

  1. Building local networks (Enterprise Advisers)
  2. Finding out what works (research)
  3. Backing proven ideas (investment funds)
  4. Providing online CEIAG resources (the Compass rating tool and the forthcoming Enterprise Passport)

Which are all full of detail on the admirable ambitions of the CEC. Much providence is given to the underlying research backing for this type of work provided by the studies from the Education & Employers Taskforce and the Gatsby Standards.

cec3

This reliance on the evidence base is to be welcomed and the Taskforce is clear that quantity of engagements is vital to improved outcomes for young people.

  • Quantity matters: greater volume of school-mediated employer engagement is associated with better economic outcomes, demonstrating relationships between the number of school-mediated teenage engagements with employers recalled by young adults and significantly reduced incidence of being NEET.

What isn’t mentioned in the Round Up though is any judgement on the quality of those interactions which their networks have enabled. The Taskforce is also clear that this is a factor in the value of the outcomes achieved by young people

  • Quality matters: more highly regarded employer engagement is associated with better economic outcomes. Analysis presented here shows a consistent relationship between higher regard for school-mediated provision and adult economic outcomes.  It suggests that the instincts of young adults were right: that the schools had prepared them better than comparator peers. Wage premiums in excess of 20% are found linked to higher volumes of employer engagement activities described, in general terms, as having been helpful.

More provision matters but more provision students remember as being helpful really matters. Which shows that, as any practitioner who has run an event with a before & after student view questionnaire will know, evaluation of provision is a vital step in ongoing quality control. While this is something individual providers and organisations will (should) be doing to monitor their own impacts, it is not mentioned that the CEC is collating or monitoring this feedback.

This lack of information on the quality of provision is a hint at the lack of wider absence from the Round-up of outcomes for students, whether gleaned from qualitative or quantitative data.

In the announcement of its formation, the DfE said that the new Company would fulfill a number of remits including

  • provide feedback to government on how well young people are being prepared for work

This was expanded on by the then Education Secretary Nicky Morgan, who at an Education Select Committee appearance on the 9th September 2015, said the Company would be judged by “asking young people at the end of this academic and going forward, where they aware of all the options, when did they receive advice, who came into their school to tell them about all the options” and by asking employers “are young people more work ready, more aware of the options that are out there” and listening to employer feedback. In an answer to Ian Mearns MP, she then said that “more emphasis on destination data, tracking where pupils go” would be a key indicator. When asked if she would return to the Committee with evidence of progress from the CEC Morgan promised to return within the Parliament. Of course since September 2015 a lot of water has passed under the political bridge including Cabinet reshuffles, Brexit and a fluffed General Election. With, at the time of writing, a new Education Select Committee chair and membership waiting to be elected, one of the items in a hugely packed education sphere competing for attention from the Committee should be to ask for this promise to be followed up by Morgan’s successor.

There have also been questions in the House to the DfE Ministerial team about the progress of the CEC in meeting its remit. Firstly on 25th January 2016, Sam Gyimah fielded questions with the claim of “significant progress” as evidenced by the hiring of enterprise advisers, the launch of a fund and the forthcoming (then and still now) enterprise passport. There was also the promise of a Careers Strategy in the (for what it’s worth, ahem, still waiting) Spring to further assist schools in their work with the CEC. Again none of this includes monitoring or evaluating the outcomes of any of this work.

On 7th March 2016, Gyimah again took questions on the CEC and again claimed “excellent progress in opening up schools to the world of work.” As well as taking a swipe at Careers Advice, Gyimah promoted the CEC Mentoring scheme and that “every school will have an “Enterprise Adviser.” The session passed with no information on measuring outcomes for students.

The CEC has also appeared in front of MPs during a session of the Sub-Committee on Education Skills and the Economy on the 26th March 2016. With Sam Gyimah unable to attend because of illness, Claudia Harris (alongside Ofsted and the National Careers Service) took questions from a bunch of fairly unprepared MPs who had not heard of the CEC’s work on cold spots or on Government’s own research on employer engagement. The MP’s mainly focused on the “umbrella” work of the CEC to raise engagement provision in areas where this was not happening, on understanding the structural layout of the CEC and the National Careers Service and testing the potential overlaps between the two. Harris was asked about the quality assessment of the work of Enterprise Advisers and promised that schools and colleges will be surveyed on their views of the work of their Advisers. The most important question on the outcomes for students comes at 16.52 in the link above. Harris says the CEC will be measuring 3 outcomes:

  1. Penetration – the numbers of pupils and schools involved
  2. Satisfaction – asking schools if this provision is helping
  3. Impact – working from a baseline in every school, the CEC will monitor how provision has increased

Then Catherine McKinnell MP asks a vital follow-up question “Is there not a risk that there will be a focus on quantity rather than quality,” to which Harris offers

  1. A literature review conducted by the CEC looking at the effectiveness of mentoring as an employer engagement activity so directing funding what works
  2. A series of “deep dive” focus groups where representatives from the CEC will speak to students on their views of engagement provision they have attended and what help it offered them

The absence of two of the outcomes mentioned by Morgan in her session is noticeable. The omission of feedback from employers and student destination statistics is perhaps wise as these are outcome measures not wholly in the control of the CEC and those with conflicting data points with no clear definition from Government on what would be measured. Would a reduction in the of 16-19 NEETS be a plus mark for the CEC or a rise in employer satisfaction of school leaver skills be evidence of the impact of provision? And from which survey source would this be, those conducted by Government or those conducted by business? Or would the only satisfactory judgement be made by the sort of longitudinal research conducted by the Education & Employers Taskforce? This lack of clarity of definable targets continued in a further Committee session (27th April 2016) with the witnesses Nick Boles MP and Sam Gyimah MP where the conversation on quality monitoring of the CEC is sidetracked onto the Dfe Statutory Guidance.

Having to scour Select Committee archives for definitions of the student focused outcome measures of the CEC is indicative of the lack of clarity from the DfE around this issue. If we take Morgan’s comments (as the initalising Secretary of State) as gospel then achieving the tasks set of assuaging the concerns of business and reshaping destination statstics will be no mean feat for the CEC to achieve. Only today the CBI released it’s 2017 Annual Skills Survey. The results include businesses views on the workplace skills of school leavers

cbi1

with much to be improved upon and contained the fact that only 21% of businesses “are currently aware” of the activities of the CEC. The fear must be that this is a taskmaster who will never be satisfied.

The ultimate quality of the enabling and linking work the CEC delivers will be decided by those volunteers, staff and practitioners on the ground organising and running the face to face provision with young people. Through its short operating period so far, the CEC has focused on the growth of its structure and operations as evidence of its progress. Soon this attention should change though onto the impact of provision and student outcomes to evaluate that public investment committed to the Company.

 

The Crises for Young People

The polarisation of society in aspects of wealth, education, prospects and voting habits used to be aligned firmly along class or location but today a greater divide can be found along generation lines. The old and the young do not have much in common.

The young earned less during their early careers.

They will spend more on rental housing costs while the old benefit from the accumulated wealth in their owned homes and the policies required to redistribute this capital would need a significant shift in the direction of travel across Western economies.

The trust in the young of the welfare state is so low, crowdfunding is their go to option.

Much has been written on the appeal to younger voters of the 2017 Labour Party manifesto with its pledges of scrapping tuition fees, affordable housing and more secure employment.

age20predictor-01

What this means for the life chances for young people is covered in fascinating detail in a new book “The Crises for Young People; Generational Inequalities in Education, Work, Housing & Welfare” by Andy Green, Professor of Comparative Social Science, UCL Institute of Education. A free download, I found it an engrossing read, especially on the challenges facing this generation as they embark on their careers (page 60).

crisesfor youngpeople1

Which is a valuable spark to remember the value of Careers and employer engagement work with young people. It can be a disruptive force for good and enable empowerment for individuals to navigate the tide pushing back against them.