destination measures

The outcome measures of the CEC

Now two years since it gained a Chief Executive and began to hire its network of staff, the Careers & Enterprise Company (CEC) released its 2nd Annual Round-up earlier this month. The report updates on the Company’s progress in expanding its networks of employer engagement and building it’s research base.

As with previous communications, the Round-up is a polished document full of the praises and progress the Company has made since it’s inception. I’ve posted previously on the difficulty on pinning the Company down on the exact numbers of Enterprise Advisers they have hired but this document does update an exact number.

cec2

While, on the previous page, the less precise term is used for the number of schools enrolled in the scheme.

cec1

If “more than” 1700 schools are signed up to the scheme does that mean that some schools are waiting for Enterprise Advisers to be matched with?

With the most recent DfE release showing that there are 3401 Secondary Schools and the AoC Key Facts showing 359 Further Education Colleges in England that means around 45% have now been paired with an Enterprise Adviser.

The Round Up continues to outline the progress and plans across four areas:

  1. Building local networks (Enterprise Advisers)
  2. Finding out what works (research)
  3. Backing proven ideas (investment funds)
  4. Providing online CEIAG resources (the Compass rating tool and the forthcoming Enterprise Passport)

Which are all full of detail on the admirable ambitions of the CEC. Much providence is given to the underlying research backing for this type of work provided by the studies from the Education & Employers Taskforce and the Gatsby Standards.

cec3

This reliance on the evidence base is to be welcomed and the Taskforce is clear that quantity of engagements is vital to improved outcomes for young people.

  • Quantity matters: greater volume of school-mediated employer engagement is associated with better economic outcomes, demonstrating relationships between the number of school-mediated teenage engagements with employers recalled by young adults and significantly reduced incidence of being NEET.

What isn’t mentioned in the Round Up though is any judgement on the quality of those interactions which their networks have enabled. The Taskforce is also clear that this is a factor in the value of the outcomes achieved by young people

  • Quality matters: more highly regarded employer engagement is associated with better economic outcomes. Analysis presented here shows a consistent relationship between higher regard for school-mediated provision and adult economic outcomes.  It suggests that the instincts of young adults were right: that the schools had prepared them better than comparator peers. Wage premiums in excess of 20% are found linked to higher volumes of employer engagement activities described, in general terms, as having been helpful.

More provision matters but more provision students remember as being helpful really matters. Which shows that, as any practitioner who has run an event with a before & after student view questionnaire will know, evaluation of provision is a vital step in ongoing quality control. While this is something individual providers and organisations will (should) be doing to monitor their own impacts, it is not mentioned that the CEC is collating or monitoring this feedback.

This lack of information on the quality of provision is a hint at the lack of wider absence from the Round-up of outcomes for students, whether gleaned from qualitative or quantitative data.

In the announcement of its formation, the DfE said that the new Company would fulfill a number of remits including

  • provide feedback to government on how well young people are being prepared for work

This was expanded on by the then Education Secretary Nicky Morgan, who at an Education Select Committee appearance on the 9th September 2015, said the Company would be judged by “asking young people at the end of this academic and going forward, where they aware of all the options, when did they receive advice, who came into their school to tell them about all the options” and by asking employers “are young people more work ready, more aware of the options that are out there” and listening to employer feedback. In an answer to Ian Mearns MP, she then said that “more emphasis on destination data, tracking where pupils go” would be a key indicator. When asked if she would return to the Committee with evidence of progress from the CEC Morgan promised to return within the Parliament. Of course since September 2015 a lot of water has passed under the political bridge including Cabinet reshuffles, Brexit and a fluffed General Election. With, at the time of writing, a new Education Select Committee chair and membership waiting to be elected, one of the items in a hugely packed education sphere competing for attention from the Committee should be to ask for this promise to be followed up by Morgan’s successor.

There have also been questions in the House to the DfE Ministerial team about the progress of the CEC in meeting its remit. Firstly on 25th January 2016, Sam Gyimah fielded questions with the claim of “significant progress” as evidenced by the hiring of enterprise advisers, the launch of a fund and the forthcoming (then and still now) enterprise passport. There was also the promise of a Careers Strategy in the (for what it’s worth, ahem, still waiting) Spring to further assist schools in their work with the CEC. Again none of this includes monitoring or evaluating the outcomes of any of this work.

On 7th March 2016, Gyimah again took questions on the CEC and again claimed “excellent progress in opening up schools to the world of work.” As well as taking a swipe at Careers Advice, Gyimah promoted the CEC Mentoring scheme and that “every school will have an “Enterprise Adviser.” The session passed with no information on measuring outcomes for students.

The CEC has also appeared in front of MPs during a session of the Sub-Committee on Education Skills and the Economy on the 26th March 2016. With Sam Gyimah unable to attend because of illness, Claudia Harris (alongside Ofsted and the National Careers Service) took questions from a bunch of fairly unprepared MPs who had not heard of the CEC’s work on cold spots or on Government’s own research on employer engagement. The MP’s mainly focused on the “umbrella” work of the CEC to raise engagement provision in areas where this was not happening, on understanding the structural layout of the CEC and the National Careers Service and testing the potential overlaps between the two. Harris was asked about the quality assessment of the work of Enterprise Advisers and promised that schools and colleges will be surveyed on their views of the work of their Advisers. The most important question on the outcomes for students comes at 16.52 in the link above. Harris says the CEC will be measuring 3 outcomes:

  1. Penetration – the numbers of pupils and schools involved
  2. Satisfaction – asking schools if this provision is helping
  3. Impact – working from a baseline in every school, the CEC will monitor how provision has increased

Then Catherine McKinnell MP asks a vital follow-up question “Is there not a risk that there will be a focus on quantity rather than quality,” to which Harris offers

  1. A literature review conducted by the CEC looking at the effectiveness of mentoring as an employer engagement activity so directing funding what works
  2. A series of “deep dive” focus groups where representatives from the CEC will speak to students on their views of engagement provision they have attended and what help it offered them

The absence of two of the outcomes mentioned by Morgan in her session is noticeable. The omission of feedback from employers and student destination statistics is perhaps wise as these are outcome measures not wholly in the control of the CEC and those with conflicting data points with no clear definition from Government on what would be measured. Would a reduction in the of 16-19 NEETS be a plus mark for the CEC or a rise in employer satisfaction of school leaver skills be evidence of the impact of provision? And from which survey source would this be, those conducted by Government or those conducted by business? Or would the only satisfactory judgement be made by the sort of longitudinal research conducted by the Education & Employers Taskforce? This lack of clarity of definable targets continued in a further Committee session (27th April 2016) with the witnesses Nick Boles MP and Sam Gyimah MP where the conversation on quality monitoring of the CEC is sidetracked onto the Dfe Statutory Guidance.

Having to scour Select Committee archives for definitions of the student focused outcome measures of the CEC is indicative of the lack of clarity from the DfE around this issue. If we take Morgan’s comments (as the initalising Secretary of State) as gospel then achieving the tasks set of assuaging the concerns of business and reshaping destination statstics will be no mean feat for the CEC to achieve. Only today the CBI released it’s 2017 Annual Skills Survey. The results include businesses views on the workplace skills of school leavers

cbi1

with much to be improved upon and contained the fact that only 21% of businesses “are currently aware” of the activities of the CEC. The fear must be that this is a taskmaster who will never be satisfied.

The ultimate quality of the enabling and linking work the CEC delivers will be decided by those volunteers, staff and practitioners on the ground organising and running the face to face provision with young people. Through its short operating period so far, the CEC has focused on the growth of its structure and operations as evidence of its progress. Soon this attention should change though onto the impact of provision and student outcomes to evaluate that public investment committed to the Company.

 

Some kind of bliss – on timing for Careers reports

Today saw (yet another) report on Careers work in school added to the library of publications released on the matter. The sub-Committee on Education, Skills & the Economy published a report on the findings of it’s inquiry into the state of CEIAG in schools.  The recommendations and conclusions within retread old ground of those previously recommended by Education Select Committees and takes a lot of direction from the report du jour, the Gatsby Foundation report on Careers.

Some of those detailed recommendations make sense, for example:

We invite the Government, in its response, to set out a comprehensive plan for improving destination data, including the timescales for doing so. This plan should include steps to make the data available in a more timely way and to ensure that they cover a longer period of time, and give more details on how the data will draw on information held by other Government departments. The Government should also consider how best to present its destination data, to mitigate the risk that schools are judged primarily on the number of their students going onto higher education.

 

We recommend that the Government, in its careers strategy, take steps to simplify the delivery of its careers policy at the national level. It should put a single Minister and a single Department in charge of co-ordinating careers provision for all ages, and set out how it plans to rationalise the number of Government-funded organisations delivering careers programmes.

 

We recommend that the Government work with employers and schools to produce a plan to ensure that all students at Key Stage 4 have the opportunity to take part in meaningful work experience.

all get a big thumbs up from me.

Other points such as for the Careers & Enterprise Company to take on the “inspiration agenda” work of the National Careers Service might be good strategic ideas but, as an end facilitator of that provision, I’m more concerned that high quality provision is on offer. How the email invites actually make their way to my inbox doesn’t bother me.

What does concern me though is the sheer unfortunate timing of the whole report and seemingly oblivious to external factors the report (actually written on the 29th June) is.

Firstly the report is published in the post-Brexit maelstrom. We currently have a barely functioning Parliament as both main parties are gripped in their own internal struggles. Getting traction from Ministers caught up promoting their favoured candidates in the Conservative Leadership election will be difficult before the summer recess and, with a General Election a possible blot on the horizon (and so a new Education Secretary) not likely after. If Brexit can scupper an entire White Paper, what hope a report from a sub-committee? Will the report be championed by the opposition? Well, it would need a quick grasp of a new brief from a new Shadow Education MP only a few days into her role after the previous incumbent lasted two days.

The aftershock of the Brexit vote on Government business cannot be underestimated. The Institute for Government rates the Life Chances Strategy (of which the Careers & Enterprise Company is a component) as “delayed” and highly dependent on whoever takes the Tory leadership crown.

Another iceberg in the way of traction is the Chilcot report on the Iraq War. Released the day after the sub-Committee Careers report, it is sure to consume news headlines and, already hard pressed, Parliamentary focus.

Then there is the reliance in the report on Ofsted to monitor CEIAG provision in schools which doesn’t appear to quite realise what’s happening to Ofsted.

We recommend that Ofsted introduce a specific judgment on careers information, advice and guidance for secondary schools, and set clear criteria for making these judgments. The Common Inspection Framework should be amended to make clear that a secondary school whose careers provision is judged as “requires improvement” or “inadequate” cannot be judged to be “outstanding” overall; likewise, a secondary school should be unable to receive an overall judgment of “good” if its careers provision is judged to be “inadequate”.

For context, this academic year has seen a sharp fall in the number of schools Ofsted is actually inspecting due, in part, to a new “targeted” inspection framework. One goal of a “self improving system” is to have this more targeted Inspectorate but the £31m funding “black hole” Ofsted faces over the next four years will drive the inspection framework just as much. Add to this the appointment of a new Chief Inspector from 2017 who will have her own views and priorities and it becomes concerning that relying on an Office for Standards without resources to monitor those standards perhaps isn’t the most effective driver of then improving those standards.

Back in 1997, Kylie Minogue released a track called “Some Kind of Bliss” as the opening to a new direction in her career. An expensive video was shot, indie credibility from the Manic Street Preachers brought in and a whole promotional blitz was planned. Then, on the Sunday before, Princess Diana died, the country had a collective weep and went out in their millions and brought Elton John instead. Kylie’s dalliance with indie was consigned to the musical dustbin. Releasing reports designed to improve CEIAG in the wake of the Brexit vote will have as much impact as when an Australian pop princesses tried to grab onto Britpop’s vanishing coat tails.

Submission to the Education Select Committee follow up Careers inquiry

In a continuing series of posts that should really be tagged “Russell gets ideas above his station,” back in July (which means that Ofsted quote isn’t now from our most recent report) I submitted a response to the Education Select Committee’s call for written evidence for its follow-up session in their Careers Guidance investigation. You can find my submission here (PDF) and the rest of the submissions here.

There are some conflicting views and differing opinions but also plenty of common ground in those submissions about which drivers of change the Committee should recommend the DfE to enact and it will be interesting to see what response the raised profile of CEIAG will elicit from the Education Secretary in her session with the Committee on 3rd December compared to the utter dismissal the previous post holder gave.

July 2014

Submission to the Education Select Committee Careers Guidance Follow Up inquiry.

Russell George (personal submission) Careers Co-ordinator Stopsley High School.

Post holder for 5 years as part of a Work Related and Curriculum team at our 11-16 school. Our most recent Section 5 Ofsted Inspection in 2012 rated the guidance for students moving as “Outstanding” and we were visited as part of the Ofsted “Going in the right direction?” Careers report in February 2013. Following this I have been fortunate to be invited to present at national conferences and events. I also blog at https://fecareersiag.wordpress.com/

Executive Summary:

  • The DfE must realise that asking schools to commit staffing and resourcing to this work requires a significant budget commitment and a dedicated member of staff
  • A scale-able system for organising school/employer interaction already exists in the Inspiring the Future website and any overlap with the forthcoming expansion of responsibilities of the National Careers Service should be avoided
  • The current accountability stick of Destination Measures is not enough to override short term decisions made by institutions to funnel students into or keep them on learning routes for funding reasons. Further detail and onus is required
  • Local Authorities have a central role to play in the collaboration across transition points of all local providers of all routes and must improve their post 14 learner data collection but confusion around their input, role and monitoring powers of FE, UTC’s, Studio Schools and Academies means that many gaps, overlaps and dead ends in curriculum and qualifications have been allowed to flourish

Submission

1. Since the Coalition Government came to power, schools that have committed to succeeding in careers work have taken on two large financial commitments to enable this work to continue at a service level similar to what went before. Firstly, they will have either employed or paid to buy in a Careers practitioner to provide guidance to students and to organise employer facing activities which previously were secured through a free service offered by Connexions. Secondly, those schools which still offer some form of work experience in Key Stage 4 now have to pay a service provider for those placements to still occur. Previously local Education Business Partnerships (EBPs) were centrally funded to perform this work. These standard offers of a Careers program can then be further supplemented with trips to national events such as the Skills Show or the Big Bang Fair and tasters to more localised progression routes but still with the costs of staffing and transport to overcome. More overarching schemes such as Career Academies or Pearson’s Think Future scheme allow schools to buy in a wholly packaged service but this has a cost price that reflects this plus they would still need a nominated co-ordinator in school. This sudden extra demand on school budgets was noted in the recent Gatsby Foundation report “Good Career Guidance1” and the figures they quote for organising each of those activities are, in my experience, realistic. Placing a new requirement on schools and then having Ofsted monitor the implementation of this requirement is sensible but does beg the question that, if Careers, Education, Information & Guidance (CEIAG) work is deemed now to be an accountable aspect of a school’s purpose, should this not be funded as such?

2. Since the updated Careers guidance was released in April 2014, it has been trailed that The National Careers Service will have an extended role to play in encouraging and facilitating school and employer engagement. The Committee should note that a working and highly successful model for this is already in place. The free Inspiring The Future website2 run by the Education & Employers Taskforce allows volunteers from a vast range of organisations and job roles to sign up to offer their time to schools. Schools are then able to contact the volunteers through the site and organise bespoke events from assemblies to group sessions to Careers fairs. Any new offer from the National Careers Service would need to tread carefully so not to needlessly overlap the Inspiring The Future service and find a niche not already catered for. In recent years a raft of organisations and schemes with similar employer/school interaction goals have sprung up such as MyKindaCrowd or National Careers week, employers such as Barclays have set up their own Lifeskills offer, business groups such as Business in the Community and CIPD have their own schemes and regional bodies have grown out of the ashes of EBPs to facilitate this work. For employers wishing to engage with schools this landscape is extremely confusing with all of these competing bodies offering structure and access. This area needs simplification, not the addition of a repetitive new organisation. Without the political will to implement or fund a national structure for school/employer collaboration the Committee should stress to the Government to look to support and expand what is already there.

3. Working in a secondary school without a Sixth Form immediately changes the freedoms and onus of my role. In his evidence session with the Committee on Careers in December 2013, Michael Gove refused to acknowledge the negative impact on CEIAG that the pressures and assumptions placed on students to stay in the school’s own Sixth Form can have. Anybody who has worked in secondary education can tell you this is a palpably false assertion. The DfE has introduced the inclusion of Destination Measure statistics as a monitoring tool to, in some extent, combat this practice but felt that the publishing of a Careers Plan as suggested by the Committee was an administrative step too far. Destination Measures in their current form do not have the accountability clout to alter school’s behaviour in this regard. The forthcoming changes to the accountability system in 2016 have been universally welcomed for placing the progress of every child at the heart of school accountability. Destination Measures must be included in these changes. In the DfE document “Reforming the accountability system for secondary schools3” October 2013 it is stated that a fifth statistic of a sustained destination percentage is desired but that the underlying data must be “robust” enough for this to be including as one of the headline figures schools will be required to publish in a uniform format. The Committee should press the Government to ensure that progress is being made in ensuring the stability of this data from Local Authorities and secure a commitment that these statistics will form a headline part of the new accountability structure. This public accountability would then dovetail with the welcome inclusion of a CEIAG focus in Section 5 Ofsted inspections and the future data to be collected on school alumni employment and benefit outcomes as recently included in the Small Business bill.

4. As well as the impetus on Local Authorities to secure robust 14-19 transition and destination data, further clarification should be requested from the DfE on their requirements to ensure collaboration of education and training providers across Key Stages in their region. A number of recent policies have aims and outcomes which, with local oversight, school led CEIAG can aid. Without local oversight the recent growth in school Sixth Forms at academy schools have the potential to prove poor value for money and increase the pressure on schools to protect revenue streams. Without local oversight the recent growth in UTC and Studio School provision to offer new choices to students at 14 will continue to struggle to enrol students leading to more high-profile closures. Without local oversight, employers will struggle to inform Colleges and training providers of their future labour force needs and the demands of the future localised labour market. Without local oversight, CEIAG becomes a threat to providers conscientious of their reputations and of the competition for enrolment numbers. With local oversight, transparent and informed CEIAG can motivate students to succeed in the routes and paths most relevant to them benefiting all local providers, the regional economy and, most importantly, the students themselves.

  1. http://www.gatsby.org.uk/GoodCareerGuidance
  2. http://www.inspiringthefuture.org/
  3. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/249893/Consultation_response_Secondary_School_Accountability_Consultation_14-Oct-13_v3.pdf

 

What’s the point of Careers quality marks for schools?

 

There is much the Careers community would like to see happen to improve CEIAG provision in schools, the use of Level 6 trained, CDI registered guidance professionals, dedicated funding, a published careers plan for each school and the widespread use of Careers Quality Awards as a road-map towards and validation of improved provision have all been mooted. Not many of these seem to have much chance of ever becoming DfE policy but all (in the eyes of the professional Careers bodies) would achieve in some way what we’re all looking to achieve; they would aid improvement. Or…would they?

In this post, I want to cover some of the issues and unanswered questions I have with Careers Quality marks both as a method of improvement and validation of a high standard of provision. Currently, according to Careers England, there are 12 Quality Awards that have met the Quality in Careers Standard which is the quality award for careers quality awards. The careers rubber stamp king of careers rubber stamps if you will. Some of these 12 awards have their own websites:

http://www.careermark.co.uk/

http://www.investorincareers.org.uk/

but all follow roughly the same procedure – School pays membership fee and gains access to the Standards document > School prepares evidence to meet standards > School sends in evidence for assessment > School pays further fee for assessment day costs > Assessment day > Award of standard > Perhaps two or four years later, re-validation

The reasons usually given in praise of careers quality marks plus my issues with them are:

1. They provide a roadmap for schools to guide them on their journey of CEIAG improvement

The standards documents I have seen from some of the different awards do seem to be very thorough and the evidence base that would be needed to comply with them would be significant but they are documents held behind a paywall, in the case of CareersMark a £90 fee for an annual membership. What value is held in those documents which isn’t found in the DfE guidance documents for CEIAG in schools? The NFER audit and guidance documents? The CDI toolkit? Janet Colledge’s website? These and other resources freely available just a Google search away perform exactly the same function. To justify a pay wall those standards must truly hold some individual value which leads me to point 2.

2. They are validation of not just good practice but practice that works

With the introduction of Destination Measures into league table data and Ofsted inspections schools already inhabit a world where the value of their wider careers work can be judged by outcomes. It’s not perfect but it’s a start and with the planned (postponed but definitely coming at some point) inclusion of more detailed Destination Measures into the forthcoming 2016 accountability changes, the focus on the actual employment and learning outcomes schools achieve for their young people is only going to increase.

With this in mind, a school, paying up to £2000 for a Careers Mark inspection, should be asking, “Is what they’re looking at, not only good practice but practice that has an impact?” Do the activities and CEIAG provision which would meet the Award’s Standards actually have a discernible impact on outcomes? The research work of the Education & Employers Taskforce has raised the bar in this area and is symptomatic of a growing trend in education now that the budget squeeze is really being felt. Policy makers and school leaders want to know what works, they want clear direction on what outcomes will be achieved for what investment. Proof of this can seen in the rise of the ResearchED movement, the spreading work of the Education Endowment Foundation and the rapid uptake and dissemination of the Sutton Trust pupil premium toolkit over the last few years. When school leaders were given a dedicated funding stream for pupils on free school meals, they wanted clear guidance on how to get the most bang for their buck. CEIAG expenditure in schools will be no different so the data needs to show

a) For schools that hold your Quality Award, how much lower is their NEET average over the last 3 years of leavers compared to the national average?

b) For schools that hold your Quality Award, how much lower is their Education Destination not Sustained (using the formula on the DfE Performance tables) percentage over the last 3 years worth of leavers than the national average?

c) Using the data sets utilised by the Education & Employers Task force, how much higher is the average wage of leavers from these schools than the national average 5 or 10 years after they left Year 11?(granted, there may be lag issues with this one)

d) For schools that hold your quality award how higher is the percentage of students securing an Apprenticeship within 12 months of leaving Year 11 than the national average?

Crunch the data for those questions and Careers Quality Awards have a chance of justifying their initial short-term costs to school leaders.

3. They add value to your school with current and future stakeholders

Schools like certificates to hang in their receptions and logos to put on their letterheads. Artsmarks, Investors in People awards, Challenge Awards, Sports Marks and all sorts of other validation proudly adorn Headteacher office walls up and down the country. School leaders feel their institution benefits from holding such awards, that they carry some weight with the public, parents and potential staff and help raise their reputation and standing. Ask any potential parent or member of staff though about what really influences their view on a school (other than results or perhaps the local grapevine) and your answer is likely to be “Ofsted.” Their verdict is near paramount and, as Ofsted are now specifically tasked to monitor CEIAG provision, more schools will* have an official verdict on the quality of their provision to highlight to stakeholders if they so wish

(*at this point some readers may raise their hand and point out the recent Careers England work which showed a majority of inspections last academic year did not include a reference or verdict on CEIAG provision. The solution to this simply cannot be to ask schools to find funds from existing budgets to pay other organisations to do this work but to actually ensure that the inspectorate with the £168m annual budget of public money is doing what it has been tasked to do)

The value that the wider public associate with a verdict from the Ofsted brand (despite the current whirlwind of criticism it is facing from the educational world) simply can’t be matched by a careers quality award.

It is these doubts over the perceived benefit versus the £ cost, the worth of what is actually being validated versus the data already publicly available and what extra opportunities it would bring to my school that have, so far, held me back from journeying along the careers quality mark route.

 

Revised Careers Guidance for FE and 6th Form Colleges

In the interests of completeness, here is the revised Guidance document “Careers guidance and inspiration – Guidance for general further education colleges and sixth form colleges” that was released yesterday.

For those familiar with the corresponding schools careers guidance the document is much of a muchness with the requirements of the duty for independent guidance on all routes to be provided for students in a number of suitable methods. Case studies highlight different ways of achieving this (including utilising organisations such as Career Academies) and Destination Measures are held up as the  method of accountability. The document then runs through a number of online resources that can help achieve Sixth Forms and Colleges achieve these aims.

 

The stories told and not told by school Destination data

A common theme throughout all of the recent commentary on the state of CEIAG in schools has been that the publication of Destination statistics for all schools is a ‘good thing.’ In the modern world, the argument goes, transparency of outcomes for schools should not just rely on qualifications gained by students but on the stability and suitability of the progress those students then make in their first steps beyond the school gates.

With this in mind I wanted to post something concentrating purely on the Destination Data of my own school’s leavers to show how this does and does not offer insight when looking at figures on a school size level.

I’ll be using 4 sets of Destination data to give some context.

Firstly, there is the data currently on the DfE performance tables website. This relates to our 2009/10 leavers.

Second, is the data for the 2010/2011 cohort that is due to be published on the performance table site in June.

So, what to notice between those two? The trend in our numbers to FE seem to be falling while the numbers to Sixth Form College are rising, Apprenticeships are steady and “Destinations not sustained” are falling. The FE and Sixth Form trends have the biggest swing in numbers so could tell the story of a more definitive trajectory. The Apprenticeships and Not Sustained numbers are pleasing but I’m wary of hanging out the bunting because, as you can see from the second table, the numbers of students involved are small. One or two students either way and those percentages alter significantly.

A hugely important factor to bear in mind is that this data is based not on a snapshot but on an extended time period. As the guidance tells us Participation is determined as enrollment for “the first two terms (defined as October to March) of the year after the young person left KS4″ and not sustained destinations are defined as  “young people who had participated at an education destination during the academic year but did not complete the required six months participation.” There is much to commend on the longer term measurement being used here which does more thoroughly test a school’s CEIAG legwork to suitably place their students post KS4. A negative consequence of this more considered approach though is the sheer amount of time that has to be allowed before publication to let the students travel through the system. The most recent set of data above covers students who left us 3 years ago. 3 years can be a lifetime of change in a school with new initiatives, new curriculum, staff turnover, Leadership changes, new priorities and events so to use this to judge that school in the here and now seems to be a little redundant.

The third set of data for our 2011/2012 cohort is from our Local Authority, who, alongside their Youth Service partners, work their way through enrollment lists, phone calls and house visits to get all of the stats which the DfE then utilise in future.

The first thing to notice is that some of the Destination terms are not the same. This immediately causes issues in comparison. Compared to the first two sets of data, the trend away from FE routes and towards Sixth Form (not differentiated between School Sixth Form and Sixth Form College here) reduces but continues. The NEET category (not known in the DfE data) is pleasing again (with the same caveat as above) while the Part Time Education numbers are odd and appear towards the larger end of the local spread (more about this below) but they lead to another concern; any conclusions we draw are only as sound as the data collection and entry job that went before them.

The biggest difference in the data sets is that the Local Authority data is a snap shot taken on the 1st of November 2012, just a few short weeks after the GCSE results. If published then, the immediacy of this data could provide interested parties such as Ofsted or parents much more reactive numbers on which to judge local secondary schools but this immediacy could also cause problems. Any snap measurement could offer a warped view of a reality that would produce very different data if captured on a different date (were the statistics exactly the same on the 2nd of November?) and perhaps not highlight gradual drop out as those learners went through the first term of their KS5 routes. To combat this and to show trends the Authority repeat the exercise in the following April with the same year group and the results of this follow up snapshot for the 2012 leavers are in the columns on the right below.

Clearly the largest change between the November and April is the Part time Education number now reads zero and the number of Apprenticeships has jumped by the same number to 12. How much of this change can be attributed to data entry decisions or to the steady progress of our leavers securing Apprenticeships in the year school would only be known to those with local knowledge of our alumni. It’s a tale not told in the stats.

So, what can we learn from all this data?

1) The considered publication timeframe on the DfE performance tables has both good and bad sides for judging school performance

2) When you drill down to school level, the numbers of actual students involved moving from category to category can be small enough so that only a few students fluctuating between them can significantly impact the percentages

and that

3) Trends in destination growth or reduction for different routes can only be properly identified with multiple data sets over a longer period

If Ofsted and stakeholders such as parents are to get the most out of Destination data in its current form, a considered and measured view and a desire to understand the stories behind the figures really will be required.

 

Me in Centrelink about School Destination Measures

Centrelink is a termly e-magazine from the Centre for Education & Industry at the University of Warwick.

My piece here:

http://www2.warwick.ac.uk/fac/soc/cei/centrelinkmagazine/november2013contents/gettingtogripswithdestinations/

The whole issue is here:

http://www2.warwick.ac.uk/fac/soc/cei/centrelinkmagazine/november2013contents/

and is very much worth a read and your consideration.