External Examiners Induction Event October 2023 video transcript

[Richard Monk] That is great thank you. So just to introduce myself, I am Richard Monk, and I am the Head of Academic Quality at the University of Derby. My team is the Centre for Quality Assurance, and we manage the External Examiner processes. They manage your appointments and various aspects of your engagement with us as a university, so hopefully we will cover all aspects of that this morning, and through the various sessions. So as you'll see there our University Secretary and Registrar June Hughes is going to run through a University overview and some sector challenges and issues that exist at the moment, Paula Bushby who's a Quality Manager within the Centre for Quality Assurance and Professor Ian Whitehead, our Head of Online Learning, will run us through the role of the External Examiner.  Professor Neil Fowler will then deliver a session on learning and teaching and work in train at the University that will be of interest to you as External Examiners.  We will then have a break and then there is an opportunity to move into the breakout sessions.  After that, Chris Bell will be joining us at 11:15 to run through Blackboard and Course Resources, which will be a core part of your engagement with the University, and how you access materials for your role as an External. Then we will move into the breakout Workshop sessions and then come back together at 12:00 noon, where we can address any remaining questions. So, I would now like to handover to June Hughes, our University Secretary and Registrar to talk on her session. Thank you very much.  

[June Hughes] Thank you Richard, and good morning everybody.  Welcome again and thank you for joining this induction session, which we hope will be of value.  I am June Hughes, I am the University Secretary and Registrar here at the University, and over the next 10 or 15 minutes, or so, I thought it would be helpful just to set out some of the sector context particularly relevant to your role as an external with us here at the University.  Some of this you may be aware of but I think it's helpful to draw it together into a bit of a praci.  If there are any questions or comments as we go along please either use the Q&A, that I think we've got live, or in the chat, or indeed raise your hand and I'll pick up any questions in this particular session.  It might not generate a lot of questions but we'll see where we go.  So let's start with the Office for Students, the OfS; sector external quality assessment is via compliance with the OfS conditions of registration the B conditions that have now become famous or perhaps infamous that institutions are aware of.  The University of Derby the same as many other institutions registered under the Higher Education Research Act going back to 2017, actually a majority of Institutions registered from 2018 onwards.  So the OfS back in May/October 2022, so a year or so ago did implement some revised quality and standards conditions which you will have come across in your own institutions, and that does have an impact on the work that externals will undertake.  So you may be aware the standardised Baseline metrics that the OfS use for all provision across levels and delivery, and that's irrespective of location.  So the University of Derby does have partnership activity, we have apprenticeship, work-based learning, all of that comes within the baseline metrics that the OfS will track.  The same quality assessment requires a sampling of student work which is something as an external moderator that you'll want to know has taken place and have a look at; with drill down implications at subject level impact, and interestingly I don't know if you're aware of this the OfS now has powers to impose student number controls, so that's limiting the number of students you can recruit to a particular subject or program area if they're not delivering the positive outcomes to students, so again that's something to be mindful of as you take your role with us.  There are multi-regulatory bodies and compliance and audit visits and that's the Office for Students.  We as an institution are subject to Ofstead compliance; we have initial teacher training which is due this year, and also our apprentiship and FE provision is due an Ofstead visit at some point during the year as well. There's Professional Statutory Regulatory bodies right across the provision that the University has.  And of course we've also got to be mindful of the Competition and Markets Authority, and the requirements that places, particularly on communication to students about changes in programmes, modules, titles etc... The OfS has been exercised over recent years on, if you like, grade inflation, and that's looking at the percentage of firsts and two ones in the sector, and then having a look at the degree algorithm.  We're just at the end of our 22/23 assessment year; we've looked at our degree outcomes have actually returned probably pre-covid levels, slightly lower if anything, so our firsts and two ones are round about the 68% level.  Last year we were 70-71%, so it'll be interesting to see how the sectors moved when that information is published post HESA.  But looking at our degree algorithm and how that is operated is also something for externals to engage with.  As is assessment of English language proficiency, that has become a condition, condition B4.  Perhaps just at this point interesting to note TEF and the outcomes were made public last week, part of a separate process, but does rely heavily clearly on the student experience metrics and the student outcome metrics; whether that's assessment or graduate outcomes.  I'm pleased to say Derby achieved gold, we achieved gold last time, and we've retained the gold rating, so we're very proud of that.  But we're not complacent because there's still quite a lot of work to do to ensure we protect and enhance those standards and achievements for students.  Next slide please.  So continuing with the OfS conditions of registration relevant to your role as externals these are the key things under B1.  So when you're looking at the work, the module detail, the program specifications, the content, the academic experience needs to be up to date, provide challenge, be coherent, effectively delivered and require students to develop relevant skills. So that's incumbent upon us as a University to deliver that, but within your role too as far as you're able look at that, test it based on your own experience with the program teams, with the delivery teams.  B3 is a significant condition this is looking at positive outcomes for students, our continuation rates, our completion and progression - either to further study or graduate level employment; so is the program being offered giving graduates the opportunity to find success in  graduate level employment, and are our students continuing/progressing through the program at a rate which is not only acceptable, but is what we would want for our students.  As I'm sure you will have, we've got areas where continuation is a challenge.  You know we're actively looking at that with program teams at the curriculum design, the assessment strategy and the outcomes for students. Next slide please. Still with the B conditions, and OfS, we have to make sure that students are assessed effectively; that is a critical component for external examiners, that each assessment is valid and reliable, and that academic regulations are designed to ensure that awards are credible.  We have a Regulatory Frameworks Committee that oversees our regulations, and is constantly reviewing those and making sure that they support and enable students in their progression, they don't inadvertently create barriers.  So the regulations ensure effective assessment and proficiency in English language which flicks back to the previous slide where we said that's a further requirement of the OfS.  Relevant awards granted to students that are credible at the point of being granted, compared to those granted previously.  I suppose that comes into sharp focus when we're looking at, you know, we're a University, industry led, we've got a lot of students on placement; the application of the learning, making sure that is current, and links back to the graduate outcomes and, you know, going on to to get that further study or graduate level job. B5 the next condition of registration, and the last one that I refer to within part B; that we only grant awards to those students whose knowledge and skills reflect the sector recognised standards; so that's level 6 in the main obviously, for undergraduate degrees.  So that we've set the standards, we're assessing against those standards, and they're comparable sector recognised awards. I'll continue to the next slide where I just want to say a little bit about the OfS and the QAA quality code.  Again you'll probably be familiar with the fact that the OfS decoupled the regulatory framework from the quality code, and the subject Benchmark statements.  And actually the QAA ceased to be the DQB the Designated Quality Body for England in March 2023, so about six or so months ago.  OfS has assumed those responsibilities, including quality assessment visits.  Anybody that read the recent parliamentary review select committee review outcome of the OfS will have seen the note around they made comment on the OfS being able to undertake that independent quality role, so I think that's something that's still actively under consideration.  It wasn't a particularly popular move in the sector, however that's where it is at the moment, so we have to recognise that and work within it.  The QAA code currently reflects the regulatory regime in England, and the principles are still applicable; three-part structure for external reference point for QA, expectations so again that emphasis on outcomes HE providers managing standards and quality linking back to the OfS conditions, practices effective ways of working that underpin those expectations and being able to demonstrate that and the advice and guidance, and there are 12 thematic areas.  The QAA themselves are reviewing the quality code there will be consultation look out for that revised code expected February 2024, and then they'll update their guidance and revise that over the coming years, over a three-year schedule. Next slide please. I won't read this out from the slide itself but relevant to external examiners and again the OfS conditions emphasis around B5, B4 and B5 actually thresholds for standards, award qualifications, partnership that is one that I've not mentioned in great detail; where we work in partnership with other organisation, effective arrangements to ensure standards can link to PSRBs as well as work-based learning, that we undertake so I think that's important for us to have a look at again s we're an industry-led institution that's very important to us.  And that we use external expertise, as an external examiner you are that pool of external expertise to help us monitor our assessment and classification processes and to ensure that our standards and our assessments are fair and transparent. Moving on to the next slide.  Turning this more internally now away from sector developments, but still linking to the sector University developments and information that will be of interest to you Richard mentioned in in his introduction the Center for Quality Assurance, the team which he leads up and there's some links in there and these slides will be provided to you I'm sure so you can link into those.  The Monitoring and Evaluation strand of the work within the Center for Quality supports external examiners in their role.  But the University has embarked on a project through a program called the Student Success Program.  This is a program that will run over a one to three year basis and we are in the early phases of this the points noted on the slide there we're focusing in on academic regulations review we have as a mentioned earlier we do keep an overview of our academic regulations and we've changed, you know, adapted those over the last couple of years and Professor Neil Fowler will go through some of those changes in his session a little bit later, but because some of that happens organically we think there's a need to take an overview to simplify, to be clearer, and have a more, if you like, coherent thorough review that we hope will take place in 23/24.  You will be briefed as external examiners about the implications of that, any changes that are significant along the way.  Another significant piece of work is our continual monitoring process around reflection on program performance and module performance is being digitised through this Student Success Program.  We've already embarked on that particular theme; there's a lot of work to do; includes automation of assessment verification moderation processes, and we expect that to be implemented in 24/25.  Again that will be subject for briefings with externals, if there are relevant areas that impact on you.  Hopefully it should automate and make things more streamlined for interaction.  And we've introduced a new approach to program design development and approval again Professor Fowler may say something on that a little bit later and our Virtual Learning Environment as part of the Student Success Program there's an upgrade project currently in Blackboard moving to Blackboard Ultra I believe, somebody will correct me if I'm wrong, but estimated to go live in summer 2024.  So you can see over this next 12 months a lot of developments in progress, a lot of new things coming on board, but all connected together; all planned and sequenced in a way that helps the University move forward improves our curriculum, program design delivery, and ultimately outcomes and achievements of students.  And finally University level we have a Strategic Framework and our Strategic Framework takes us through till 2030.  If you've looked on our website you may have seen this yourselves.  We've just gone through a refresh; we've not changed the principles that sit behind that framework, but we have added a fourth pillar and that's focused on industry engagement.  And that industry can be the public sector, health sector, it can be business; it's meant to encompass the broadest range of engagement we have with industry in the region and beyond.  And as I've stated throughout in that activity around placements for students or assessments for students I think it's important just to draw that out for attention at this point.  That's all I wanted to share with you by way of sector insight and updates that I hope have just drawn together for you some pieces you're probably familiar with, and insights into University change on the horizon with some contacts there that you'll find useful I'm sure, to pick up as you go along.  I can't see any questions in the chat I'll just check for hands up, if not unless anybody stops me I'll say thank you for listening I hope you have a successful morning and you enjoy the sessions which follow and the breakouts which Richard referred to and I'm now going to hand on to Paula Bushby Quality Manager within the Center for Quality and Professor Ian Whitehead, our Academic Head of Online Learning.  Thank you very much, and have a good day. 

[Paula Bushby] Thank you very much June.  So good morning all and welcome, as June said my name is Paula Bushby and I'm one of the Quality Managers working within the Centre for Quality Assurance here at the University of Derby and I’m going to be joined later on in this part of the session by Ian Whitehead who's the Academic Head of Online Learning, next slide please.  So just to run through the aims of this part of the session some of you who are here today may already be experienced external examiners, others may be completely new to the role but all of you are new to being external examiners at the University of Derby and so within this session we're hoping to be able to outline some key aspects of what this role entails providing you with an overview of your role as an external examiner but also providing some context of how this might translate into reality. There are quite a few slides in this part of the session and I'm going to be moving through them quite swiftly however as already been stated the slides and the recording of this session will be made available to you afterwards so please don't feel that you have to try and make a note of everything. As already mentioned if at any point during this session you have any questions please do pop them into the question and answer area so that we can address them for you and we'll try to answer as many of those as possible during this session.  Next slide please.  So there's some key Central induction information that will have been made available to you as part of your appointment, some of this may not have landed with you just yet particularly if you've not yet had your local induction with your program leader and if this is the case then this should happen within the next couple of weeks. Much of the information is accessible to you by the external examiner page on the University website and this really should be your go-to location for the most recent versions of any templates or forms and so it's particularly useful when it comes around to writing your annual report or importantly to find out how to claim your fee and expenses.  Next slide please. The University of Derby has a centralised Center for Quality Assurance as you've already heard and you've already met Richard Monk the Head of Academic Quality this morning and you're most likely to have encountered Laura Waldram as it's her and her team who are at the end of the external examiner mailbox and this email address should be your main contact point with the team and they will deal with any queries that you may have about your appointment, fees, expenses,  access to the VLE and matters of that sort.  It's also the email address that you should use to submit your annual report, next slide please.  In addition to those Central contacts you will have some local quality contacts within the Colleges and the Schools that you've been appointed to, hopefully you'll have met some of those already but if not that you'll have plans in place to meet with them with within the next few weeks.  Next slide. If you're new to being an external examiner or if you just want to refresh your understanding there are some key reference points that you might wish to make use of.  There's some practical guidance on external examining provided by the QAA in their advice and guidance documents on external expertise and Advance HE has on their website, a provision of valuable reference sources for external examiners.  Their external examining handbook brings together the outcomes of research and development in the field and it provides evidence supported guidance derived from practicing examiners using case studies to illustrate potential issues that can arise. Hopefully they won't do for you but that where there are issues that can sometimes arise that's quite a good place to refer to, next slide please.  There is also professional development available to external examiners,  this Advanced HE course comes highly recommended by other externals who have attended it, next slide.  So typically your appointment will be for a period of 4 years, in exceptional circumstances and to ensure continuity of coverage there may be the possibility of this being extended by an additional year during this time and particularly as programs change and develop there may be a need to reallocate some of your duties.   Your Role may change to accommodate changes to the program to allow you to take on a more senior position or to become a mentor to a new external examiner and of course you should be fully consulted about any changes and these will be administered by the Centre for Quality Assurance external examiner team.  Of course circumstances can change and it may be necessary for you to resign from your role, we really hope that this won't happen but if it is the case we ask that you provide us and the program team with as much notice as possible so that a replacement can be found, next slide.   

Your key responsibilities as an external examiner are to verify the standard of our Awards and to help to maintain and enhance academic standards including valid and fair assessment processes. In addition to this you may be asked to advise on the development and design of our curriculum and ultimately in this role you will be responsible to our Vice Chancellor as Chair of the University's Academic Board, next slide please.  In order to fulfil this role there are a number of things that you can expect to be advised of and consulted on, you can expect to verify assessment briefs, exam papers and to review a sample of assessed student work and that may include level three and level four modules particularly if they're new or if they are requested by you and you are able to request to review them or if requested by the program team, next slide please. If you haven't already done so, one of the things for you to discuss and confirm with the program leader is the sampling requirements, however this should follow the requirements listed here but with a maximum of 250 pieces of work to include all modes of assessment and to reflect the full range of grades awarded and all delivery modes and locations and I know that when this figure is quoted it can cause a bit of a sharp intake of breath in the room and so just to assure you in the vast majority of cases the sample of work that you'll be reviewing will be nowhere near this size. It really relates to some of our much larger programs but it is good to have that definitive ceiling as a rule the sample size for moderation is calculated as the square root of the total number of students taking the assessment with a minimum number of five, next slide please.  

There are a variety of assessment boards held throughout the year and there is a requirement for at least one external examiner to be present at these and often this will be the college level external however you may find yourself invited to them. Attendance at assessment boards can provide you with really good staff development and it's relatively easily because in the main our boards are still taking place virtually and you can request to attend if this is something that you would like to do so. If you do find that you're not being invited you are able to invite yourself and please do make use of that, next slide. In addition to our standard College boards there are those for students studying at our Academic Partners or through online learning, any requirement for you attend these should have been identified to you within your appointment details and it will be confirmed at the start of each Academic Year. It's worth noting that our online learning holds three assessment boards per year, there's no requirement to attend all of these if you are associated with online programmes or modules but you can if you wish and again these tend to be quite popular for external examiners to attend as part of their CPD, next slide please.  So attendance requirements or visits will be confirmed with you during your local induction with your programme leader, ideally it would be great if these visits can be face to face but they can be virtual if this isn't possible and as already mentioned the majority of our assessment boards are still being conducted online, next slide.  

You will be required to provide an annual report and this needs to be on the latest University of Derby template so please make sure that you're always using the correct template version and so to do that it's best to go to the external examiner page on the University website and to download it from there rather than adapting a previous report. Those templates are regularly updated. Your report is due to be received by us within four weeks of the relevant assessment board and it should be submitted to the external examiner email address please.  You're likely to receive a gentle reminder about this as we head towards the main season for external examiner reports and the incentive for you to submit your report promptly is that your payment won't be processed until that report has been received.  It’s worth remembering that your report will be made available to a wide audience including students and also to our academic partners. You should receive a full response to your report from the programme leader and that should be obtained by you within 6 weeks of receipt of your report and as part of your local induction with the programme leader you should spend some time discussing and agreeing the general and specific expectations of your report, next slide.  We do understand that there may be times and circumstances that can prevent you from being able to fulfill your duties and if this is the case please do let your programme leader know so that we can arrange to have these covered, often this will be by arrangement with another external examiner in a cognate area and just to point out that if you fail to fulfill your duties we do have a policy for escalation and this could lead to your appointment being terminated. Again we hope that wouldn't happen but the main trigger for this would be failure to submit your annual report within those four weeks of the relevant assessment board. And then finally for me is just to let you know that we are continually looking for ways to improve and that we are currently in the midst of a significant piece of work to review our external examiner system, looking at how we use our external examiners for both quality assurance and quality enhancement now and how we might do things differently in the future and this is taking  place alongside the review that June mentioned of our academic regulations.  We've been extremely grateful to our existing external examiners for supporting this review so far through contributing to benchmarking exercises providing us with feedback and for participating in some trial activities and I really hope that you will also be willing to engage with us during this period and please be assured that any changes will be communicated to you well in advance.  And then the other exciting development that June mentioned earlier and that hopefully you'll be hearing about more as this Academic Year progresses is the digitization of several of our quality assurance processes and through this we're intending to simplify and streamline workflows for all involved, including you as our external examiners and within the scope of that project are the processes associated with assessment verification and moderation, so on that note I'm now going to hand over to my colleague Ian Whitehead, thank you.  

[Ian Whitehead] Thanks Paula and good morning, everybody.  As Paula was just saying, a lot of the things I'm just going to draw attention to here will get captured in that digitisation project that we have ongoing at the moment.  But these are the key things really that you can expect to receive from your programme team during the course of the year: so information on each of the modules, the relevant module specifications - so, you'll be able to see what the key module learning outcomes are, and how those relate to the overall progression of the students through the programme learning outcomes. As Paula has just alluded to there, you will get draft assessment briefs for all the modules that are within your remit, and alongside that an assessment verification form, so that you can confirm that the assessments are fair and appropriate and certainly are relevant to the level of study, and they are really effective ways of assessing the learning outcomes.  And we're always really looking for feedback from our externals on that, particularly around making sure that we, inclusive by design, in the way in which we put assessments together thinking about the abilities and the needs of all of our students.  You will, as Paula said before, receive an appropriate sample of marked work from each of the modules that you're working on, alongside a complete results list and information on the standard deviation; and those kind of things are things that we will also discuss and welcome comments from external examiners on at the assessment board; and that's where again we really do kind of get under the covers of how a module is performing, how students are performing, and in particular start to explore where there might be factors that are influencing kind of student progression through the programme.  And we talked before, certainly June mentioned challenges across the university sector, not just here at Derby, around non-continuation, so, looking at where we see those things that might be factors in shaping those outcomes for students.  You will receive a completed internal moderation form for each sample of assessments that you receive, throughout the year any updates to the programme you will receive information on that, and indeed should be consulted as part of those changes as well; and once you've submitted your report, which as Paula said is pretty key to you receiving your payment, you will also receive a detailed response from the Programme Leader, and that that is a key formal part of the of the communications that you'll expect to have with the team.  If we move on to the next slide.  So, that communication transparency is absolutely key, and I'll say a little bit more about what that means in reality in a moment.  I think it is those really detailed good reports and responses and good relationships that we really need to make sure that we're getting that quality oversight, and those kinds of assurances and that we've got that overview the student experience right from end to end, from the start of the programme, right the way through to graduation.  We’ve given lots of information here today, I'm conscious of that, so, if there are things that aren't clear, that maybe are different to what you've done in previous roles or the approach in your own institution, kind of do take the opportunity of the Q&A during the course of the morning to pick those things up.  There's also that email address that Laura Waldram and her team manage, so do use that as well as the various contacts that you'll be given in or have already been given in your appointments letter.  If we move to the next slide, many of you will have been external examiners elsewhere, it won't necessarily be your first time of being external.  If it is your first time you will nonetheless have had relationships with external examiners for your own programmes and you'll know that the really important aspect of the role in the terms of day-to-day mirage the programme is for the external really to be that kind of critical friend for the team, are sounding boards for new developments, or how you manage challenges in terms of design and delivery of the programmes; so, of course there are fundamental quality assurance things that are part key part of the role, and it is using the assessment boards to have those conversations with the team, and to ask challenging questions about performance on individual modules where it's appropriate to do so.  It is to make sure that through moderation we are being kind of fair and consistent in our grading; and also crucially I think, it's not just the grades, but also the quality of the written feedback or verbal feedback, where that's the means of communication to the students, so that there's that real sense of students understanding why they've got the grade that they have, and what they need to improve.  And critically that it's not just students who were perhaps at the lower end of the grading scale who are getting lots of advice on how to improve, but also that the guidance is pushing students at the top of the 2:2 to get into 2:1, top of the 2:1 to get into first, and indeed even giving first class students challenge so that they continue to perform at that level as well.  It is worth emphasizing that this is moderation, so we're not asking you to do second marking or in some cases third marking, if we do have genuine second marking going on some individual modules this is you moderating the overall quality and consistency and standard of the whole marking process, so that that is a key thing to understand, I think.  The reports and responses, that those are key to be our formal kind of record of the communication that goes on between you and the team that you are working with, and certainly that is where ultimately you are giving our Vice Chancellor the assurance around quality and standards on the programme.  There is of course through reporting the opportunity to raise major issues, but it's worth stressing that major issues really are things which are fundamental to the quality and standards of the programme, and things which put us at risk of being compliant with those OfS standards that June was detailing earlier on, or where you're being significant kind of non-compliance with our quality assurance processes within the University of Derby.  So that's the QA side of it; but as I say, it really is that kind of critical friend relationship which is the most important aspect that I found from being an external elsewhere; and it's that opportunity to really get to know the programme team, understand what they see as the standout features, the distinctive features of their programme here, understand the way in which they designing the student learning journey, and being part of those ongoing conversations; getting to meet our students, hopefully; and actually coming on to site to see the student work.  A lot of our programmes, even ones in in quite traditional academic disciplines, have lots of opportunities for students to do public facing work, so it's great if you get the opportunity to come along and be part of seeing that.  But really, as I said, during the course of the next few years you'll see that we've got lots of big plans in the University to review our regulations, to make sure that those are as student friendly as possible; and also to continue to challenge ourselves to maintain that gold TEF outcome; lots of work that we're going to be doing around new curriculum design processes, and those are things that my colleague Professor Neil Fowler is going to talk about now, and he'll probably say a little bit more about how we would like you as an External Examiner Community to be involved in the development of those processes, very much as the critical friend for us, so I will stop there and hand over to Neil. 

[Neil Fowler] Good morning everybody.  I've been hearing the list of things I'm going to be talking about and feel under some pressure now to A, remember all of the things that that have been said, and B, deliver on some of them.  As has been described, my role is Associate Provost for Learning and Teaching which means I've had responsibility for helping the institution to shape the two new Frameworks that we have on screen.  I'll unpack those in a moment,  but I've also helped with the writing of our new Access and Participation Plan to ensure that we we're delivering on the inclusive aspects that we have written herein, and that's important to us, and was an important part of the TEF submission that has been referred to earlier in terms of that whole institutional approach to ensuring social mobility at all stages of the educational pathway, and that is part of what drives us as an institution, and our ongoing approach as June referenced earlier as a university with industry connections; that onward pathway of Learners into whatever their future career may be in whatever industry is that they choose to work within, that making sure that we are adequately preparing them for what that future may look like.  And therefore, I work with colleagues across the institution to help make that the lived reality for our students, and I've been here at Derby now for coming up for two years.  One of the first tasks that I was given upon arrival was to review and update our Learning and Teaching Strategy, and after a little while of mulling that over and talking to colleagues across the institution we binned the idea of having a strategy, and rather we moved to the concept of Frameworks. And you can see the two Frameworks they are that is them in their entirety on the screen.  So they're very simple, four elements in one, five in the other, because we wanted the Frameworks to have a real impact on the lived experience for our programme teams, that they would have agency in interpreting and applying these to the context that they work within; accepting that a university like ours and like many of yours, are incredibly diverse; that there are differences in what is the right pedagogy for different subject areas.  I think it's right that we would teach some of our Arts provision different to how we would necessarily teach some of our engineering provision, or our provision in the humanities or the health domains, they are vitally different to each other, and we didn't want to have a recipe book that says “You must do it like this”, which is how many Learning and Teaching Strategies are created.  But we also recognize the diversity of our Learners.  We have an incredibly diverse learner population; we have significantly greater than average proportion of students who are mature at the point of entry; we have quite a diverse mix in terms of the ethnicity of our students particularly in some of our discipline areas, less so in in others. We have a diversity in the disability mix and the proportion of students with a declared disability of different forms, and therefore the right model should be one that's adaptable to the context of our Learners.  We also wanted the Frameworks to be applicable to all levels and modes of study, so the principles contained within the Frameworks are applicable at undergraduate level, they're applicable at taught postgrad, they also apply to our research degrees approach to doing something - we would still expect these principles to being adhered to or being in quite a different context for PGR.  But they also work across our Campus based provision, our Partnerships provision, our Apprentice provision, our Online provision -the same principles should apply, albeit the way in which they're delivered will be very different according to those contexts because we have different types of engagement with those learners. And so that's what we have.  We don't have a checklist to say “this is what you must do within your programme” rather than these are the things that we expect you to be aligning to and in your engagement with programme teams we would be interested in seeing how that’s being manifest and you know for you as examiners to ask those questions of the programme teams as they apply to individual programmes, as a whole to a module, as a whole but equally to each individual learning event that the students are engaging in.  So how are you building belonging on Tuesday afternoon in the middle of November, is as relevant to how are you building belonging as a whole programme across the lifespan of three years within that provision.  And recognizing that things like those belongings are multiple; that sense of purpose will change, its dynamic across the journey of a programme, what a student thinks they want to do when they arrive on their programme, should, I think in many cases be very different to what they aim to do when they leave that programme because we should be opening their eyes to other opportunities that exist.  So how are we broadening that sense of purpose for our students as they're on the journey.  So much of what you will see, and many of the things that were referred to in sort of the guise of the Student Success program, and the pieces of work that we've got going on, programme design changes, our changes to the continuous monitoring processes, are about us stimulating those discussions; about making the discourse of the how and why we are delivering our programmes part of the day-to-day activity of course, is that it's a continuous process, it's a job that's never done once you've made one lot of changes you come back and you should be looking it again, and as professional and reflective practitioners that's exactly what we should be doing, we should never feel that we've got it perfect and never have to change it again.  Can we nudge on to the next slide, as much to remind me what it is are we going to talk about as is anything else.  Within those Frameworks we have a series of implementation plans of recognizing that the job's not done but we have to start from somewhere; and so, there's already been work done, and here's our plan of things that we will be doing over the coming period.  So, during 22-3 we've been building on module changes, that sort of philosophical discussion about what we're doing, the continuous updating of programmes.  We've changed our module evaluation process, so all modules as I'm sure they do in your own institution, have a review annually, we ask our students to reflect upon their learning experience on each module, and on each programme, and we've stood up a new set of questions that we used, significantly stripped back, and focused on the areas that we believe to be of greatest importance to the success of those programmes, so we created a clearer distinction between questions that make most sense at a module level, and then different questions that we ask at a programme level because they're about the more collective experience that the students have had.  So within your discourse with module teams’, we would hope that they are reflecting upon that student voice, that they're bringing the outcomes of those module evaluation questions into their monitoring and review.  You should expect to have line of sight of the outcomes of those, by all means do ask for them if they're not being shared with you directly.  I think it's an important part of your role as External Examiners to be able to see what the students that those who are going through the modules have to say about the experience at module and programme level.  And we're keen to get some consistency so we can build some longitudinal data. So, we've made the changes to the MEQs and introduced a new programme survey, and we will look to keep that the same for a period of time so we can build up that bank of new data. We've had reference, Paula made reference to enhancements to our monitoring processes, that's coming forward, so we changed some of the focus within the continuous monitoring process and that's going to feed into our digitization processes.  I'll talk a little bit more about programme review methodology on the next slide, hoping that it says it, but if it doesn't, I will pick up on it when I get there. A couple of other parts that that are relevant to draw out just from the slide that we have before us.  We've introduced some language, just actually to help us create a bit of a brand, so Develop at Derby is something that I hope you will come to see being manifest in the modules that you're looking at.  Develop at Derby really is just a brand name that's been added to the range of learner support materials that we have available across the institution. These have been available from our library teams and from our learning support teams for some time, including some of the work that was done by the careers and employability team.  And we've bought those together under a single brand to make it more accessible for our students so that they can see the consistency of what we're doing, and that there's a real push with teams during the course of this Academic Year for them to start to embed those resources within their taught curriculum, so there should be direct reference to aspects, and relevant aspects, of the Develop at Derby resource attached to particular learning tasks, and particularly into assessment tasks; again might be something for you to look for as you're looking at some of those assessment materials and guidance materials given to students to support those assessments: are they being signposted to those resources, are they being picked up in the assignment briefs, and potentially also in the feedback if students have struggled with particular aspects, are they being signposted to the existing resources that will help them to build upon that and perform better in their next assessment.  So, that's something that is very much an ongoing piece of work with the Develop at Derby brand, and it is well established now, and we should be seeing it coming through.  And regulatory change is the last one that I'll pick out from this slide, again partly because it will feature in some of the discussions and some of the quality reports that you would have seen from the last cohorts through, so if you're new to examining you may well see reference to this from the Examiner reports or in the module or program review reports coming into this year.  We made the change a year, or just over a year ago, to move to module aggregate marks, rather than individual component marks being used to determine progression. We did that for modules at level three and four last year, and it's moving through to all levels of study for this year.  So, our old regulations require the student to gain a mark of at least 35 in each component of assessment, as well as a module mark of 40% or more (compensation was available, so you could compensate some modules with a mark of 35) but to pass a module in the normal sense you had to get 35 in each component of assessment and 40 overall, that was changed to using just the straight module aggregate. You needed to get a mark of 40% for the module at levels three and four and that is now the new rule that will be applying for levels five and six from this year moving forwards.  We have recognised that some programmes, because of their nature, because of the particular learning that may be involved or external endorsement regulation (so some of the PSRBs), do require a higher threshold to be set for each component and so we have introduced the capacity for some modules to be identified as still requiring students to reach a  minimum threshold for each component of assessment, so there's a variety of different rules that apply but they're all now within the regulation, they're not derogations from our regulations.  The regulations reflect the fact that for the majority of our modules we are just using a straightforward grading basis; for some modules where there's a PSRB requirement the past threshold still needs to be reached for each component of assessment, so again look out for those within your provision, we are just signing off the last of those applications for this Academic Year, so there'll be some behaviour change coming through as a consequence of that. Can we nudge on one.   

Things that we're doing this year, so already touched on the first two of those, so the expansion of the group pass rules, and introduction of notional minimum threshold regulations.  Paula has made reference to the digitising of the continual monitoring process, including our verification, so the principles will be unchanged, still asking for the same things to be done in terms of ensuring that the assessment that we're using is effective.  We would ask you in in your engagement with the verification process to check that teams have reflected on past performance on those assessments particularly any variability of performance between either different cohorts if a module is shared across different types of provision; have they reflected on differential outcomes that those student groups may have had, but also differentials in outcomes for different demographic groups within there. And we do recognize within our Access and Participation Plan, that you know, we continue to have challenges with respect to the differential outcomes for BAME, and for white students in particular.  There are some differences in socioeconomic background as well that manifest themselves, that's picked up in our new APP with a focus on those students who previously would have received the free School meals.  Rather than using polar or IMD quartiles the free School meals is actually a better differentiator for us to identify the student group's most at risk there.  And we do ask at appropriate levels that consideration is given to those differential outcomes, appreciate that module level that's not always appropriate.  The numbers of students that may fall into groups are often too small to draw meaningful conclusions at a module level, they become more meaningful sometimes at programme level, or even at school subject level, and then we draw down to see what's happening within individual module.  So, it's about being proportionate in our looking at those differential performances, but we would expect teams to be reflecting upon them.  And it's interesting we have about a 20 percentage point difference in, apologies for the phrase it's not one I like, but the good honours rate for our BAME students and white students, so students gaining a first or upper second class degree that actually manifest about a 5 percentage point difference in the average grade achieved by students in those groups, so actually if we focus on the marks that students are achieving we get a better sense of what's going on.  So, looking at average mark on each component of assessment is how we will affect change in the overall Awarding Gap that we're seeing, and again just would sort of mind you to have a look, in the work that students, in the work that colleagues are doing on their modules and a sense of are they reflecting upon those differences in a proportionate and appropriate way for the provision that they're looking at.  Again, there's not a one size fits all part to it.  The slide does draw my attention to our new programme design process; we're moving to a model that that's much more dialogic, philosophically.  The Frameworks are very driven by trying to get programme teams to have agency, and understanding of what they're doing, and it to be a reflective process.  So, our new programme design process is taking that on board, trying to achieve what all so consistently achieve what already happens in our best programme designs of getting the whole team together to engage in a constructive design process supported by appropriate colleagues from elsewhere across the institution to provide support and challenge.  Importantly we're looking at how do we reflect student voice into that, how do we reflect the voices of diversity into those conversations such that programme teams are reflecting on what does this mean, so what does employability mean within the context of this discipline, what are the right reference points, how are we addressing education for sustainable development that you know you would expect to be different in different types of provision because there's different applications that would be relevant.  But we need the teams to be thinking about it within their design. So that new process, we'll be trialling it this year.  So we're going for a relatively soft launch rather than big bang of introducing this, but during the course of this year that is the journey that we're on and we will be looking obviously, as would normally be the case with any programme change, for introducing External Examiners into that dialogue and the externality of that voice to make sure that those design processes, so if any of the provision that you're involved with is involved in either new validation, which will be going down that line, but also programme review and modification, we will move into this model from next academic year, moving onwards that hopefully they're starting to behave in those new ways.  It's a much more dialogic process and they're having the conversations about what will this look like so that's going to be a key one for us to be looking at and further embedding at Derby or part of the work that we're doing.   

I'm not sure if there is another slide.  Sure, there was. There wasn't, good.  I am conscious a couple of questions have popped into the Q&A whilst we're going on, so I'll answer a couple of those whilst I've got the floor, then perhaps we could open out to a little discussion around all of the things that you've heard this morning before we move into our break in proceedings and the next part, but be nice to be able to pick up some other questions or comments if folks have them or if any of the other speakers from this morning have anything that they would like to add.  But just whizzing through the Q&A, the module aggregate pass at the moment is for UG only, we are looking at similar for PG but we're working in a staged process so at moment, those rules apply for undergraduate programmes only. At PG level, there's still an expectation that students will achieve a pass threshold in each component of assessment in postgrad study.  Although that that is part of review/ I would like us to move to aggregate marks at all levels but we're on that Journey, not there yet.  Timeline for activities around external moderation of assessment, when do they usually take place and timings? Paula, can we come to you for that one as it's procedural?  [Paula Bushby] Yes, thank you Neil.  The responses have already been put into the Q&A, Neil, so that really is a discussion to have at a local level. There is a huge amount of variation between programmes depending on intake points and where they're delivered and all sorts of things, so it's a good discussion to have at a local level.  [Ian Whitehead] The important thing to stress is just that it's not necessarily an end of year task, and so the report is the end of year task, but the individual module moderation will be in year activities in preparation for the assessment and progression board, so there's usually a mid-year one for campus programmes and as we explained earlier, we have one per trimester for online delivery.  So, the module level moderation and reporting will be in in sync with those boards, and then your overall report on performance that will be the end of year report that you submit.   

[Richard Monk] I'd like to pass on now to Chris Bell who's joined us. So, Chris is a Senior Learning Technologist in our Digital Learning Team who's going to talk to you about accessing our Virtual Learning Environment and Course Resources, so I'll just hand over to Chris. 

[Chris Bell] Thank you Richard and it is still good morning isn't it to everyone that's joining us virtually today. As Richard has just said I am a senior learning technologist at Derby, so what that role really brings is primarily focused on supporting the use of Blackboard and some of the kind of digital tools that orbit the world of Blackboard at the University. It is a kind of Staff support role with the idea that our students’ digital skills are uplifted, in terms of the systems that we are using and some of the digital practice that we adopt at Derby. So that is my role, but I am here specifically to speak to you today about accessing our virtual learning environment which is Blackboard at the University as an External Examiner. That will be actually accessing Blackboard itself, making sure that you've got appropriate credentials to do so, accessing sample papers, now there's one of two ways in which you will primarily be doing that and then of course leaving feedback on the moderator form within a specific area on Blackboard as well. So really it involves accessing Blackboard, using Blackboard, and then posting content to Blackboard and that's what we're going to cover today. It's not a one-time support offer this, so we are available and the email address was on the previous screen it's DLS@derby.ac.uk (Digital Learning Systems) you can get in touch with us there if you're having any issues with the use of Blackboard, if you've got any issues with usernames and passwords etc. that's more of an IT Services related request that you'll want to be putting in. I'm going to drop both of those links into the chat, hopefully they will show for everyone just so we're aware of how to access those two services, but I'm here representing the DLS side of things today. So to navigate to Blackboard, perhaps confusingly enough we go to a site called Course Resources https://courseresources.derby.ac.uk we used to have a system called Course Resources at Derby we've since moved across to Blackboard, but kept that naming convention. So that's where we go to, a lot of our tutors, a lot of our students will bookmark that page and they know that page is the kind of entry point into Blackboard, and the thing to remember here is to enter your username and password so that will generally be in the format shown on this slide with a unique four numbers in your username. Some of the issues that we've faced in the past with this is if your access, if you're logged in perhaps to your own institutions VLE or a single sign on account then it can get slightly complicated, but if you do encounter any issues with that then clicking on use other account should get you through that issue. But we are on hand at that email address as I've just mentioned if you are struggling with any of that but that's the first port of call being able to access Blackboard and that's https://courseresources.derby.ac.uk . If we can just move on, so in terms of accessing the content on the system I mentioned that there's one generally of two ways in which that happens but the contact that you've got at the University will provide details of where to look and what to moderate and that will generally be a member of the admin team within the college, a programme leader or a module leader, that's usually how this works and the sample of papers will use consist of files uploaded into the modules content collection and I'm going to show you how to access that in a second, that's particularly important because that content collection is also the area that you're going to be uploading that moderator form back into, so that's the kind of area that's The Crucible of activity if you like in terms of accessing sample papers or students work and then pushing that feedback back into that area and it's all done within that same space within Blackboard and within a particular module on blackboard. However, you may also need to access Turn-it-in directly within that particular Blackboard module and that's because often our tutors are leading things like audio feedback for students and that's attached directly to that turn it in submission point, so I'll also show you how to access that as well. But the main thing I think is just getting used to being able to go into Blackboard access that content collection be comfortable with that take the content off there, have a look at it moderate it and then push that feedback back into that content collection area. So that's what I'm going to look at first if we can just move across to the next slide, okay so once you log into https://courseresources.derby.ac.uk Blackboard you'll be faced with a page that looks very similar to the one that's on my screen here, you'll be attached to particular modules that you've been asked to provide feedback in or to and you click on the modules tab on the left hand side, some of you might be familiar with this you might be using Blackboard in other institutions etc. but there will be a module that you can click on when you click into that modules area and that you you'll be only attached to those modules that you're providing that feedback and moderation for. So, you click into that module and then within that module there is going to be an External Examiner folder or a list of submissions to view with Turn-it-in but that's how we access the particular module. So if we just want to move on to the next slide, I'm going to give you a bit more detail about how to access that External Examiner folder within the module itself, so on the left hand menu under an area called course management there's an area, a sub area of that called content collection and you need to click on the module code so the content collection area when you press on it, it just creates a drop- down list you then need to click on the module code in this instance it's a module called 2021 HE test but it probably be in all likelihood a 2023-24 module that makes a little bit more sense than that than that shell module in this instance and it's within there that you'll find samples of work for review and I'm going to at the next slide just talk you through how to access those samples of work within that content collection area, within that particular module area and then how to actually download that to your own computer so you can provide that feedback. So if we can move on to the next slide that would be great, okay so if we do have to go into Turn-it-in, so I mentioned one of two ways we can also do that in a similar way by going into the module itself and then going into course tools but this time we're going to click on turn it in assignments and then we can navigate to the relevant submission point. So within there you will see a submission point for that particular piece of work and then you can select which modules that you're going to review from that list and within there you'll see links to audio feedback etc. if that's been left but also the actual feedback back itself. So that's just how to access turn it in, we'll be looking at how to provide feedback in that External Examiner folder on the next slide, so if we could just move to that that would be great, thank you, okay so within the External Examiners folder we can download the actual work that I've just pointed you toward but we can also upload that feedback as well, so that's the area that we're pulling work from and pushing work into and that's within that Blackboard area that External Examiners area within Blackboard and to do that we go into upload, so there'll be an upload area within that content collection and then you can access your feedback, generally speaking that will be a moderation form that's been provided to you, if there's a number of those that you're pushing back into the system it's probably really useful to zip those files together so add them as just one file and then upload that zip package back into that External Examiners area, so that's the main thing to remember really, we're going into Blackboard we're choosing the module from that module's list within Blackboard, within there'll be an External Examiner folder we're taking the work from there and then we're pushing the feedback back in and that will be in the form of a moderator form either singularly or as a zip package, as a collection of moderator forms or moderation forms. Hopefully that's given you some indication of how to access the External Examiner's folder at Derby, I think primarily the way that you'll be doing this is through that External Examiners folder within the content collection but as I say there may be some instances where we have to go into Turn-it-in to access but that information will be provided to you. So if you do have any questions then I'm happy to answer them but if you've got any issues with passwords, usernames, password reset, passwords expiring etc. use IT Services for everything else use DLS@derby.ac.uk and we'll get back to you really quickly on that email address, it's not a job logging system so if you've got any particular issues with access in terms of where the papers are and how to push that feedback back in then we'll support you through that, so hopefully that's given you some ideas and some indication of how to access student work and then provide feedback in that same area. So, I think that's me done. 

[Richard Monk] That’s great, thank you very much Chris, very much appreciated. I think the next part of the session is to move into the breakout sessions so the team have very helpfully put the lists up of sessions people have booked onto or indicated a preference to attending, the links to the individual sessions are in the chat so you can click on the link and join the session you are intending to go to and the intension after that is to rejoin this sessions at 12.00 for a brief closure and answer any final questions anyone may have. Please feel free to join the breakout session using the links in the chat.  This was to come back together briefly at the end to give everyone the opportunity to ask any sort of final questions you may have. Hopefully you found this morning useful and informative; we'd be very grateful for any feedback you may have on this morning's session, any information about whether you found it useful, things you would have liked to have heard more about, things you may like to have heard slightly less about - we'd be very grateful for any feedback you were able to provide afterwards and my colleagues will send round a feedback form for that purpose. On the screen you can see our general email address if you have any queries that you want to follow up on. Lastly, I'm very grateful to you all for taking the time out to attend this morning and I hope you found it useful.  I also hope you'll find the sessions this afternoon you have with the college teams useful if that is what you are staying on to do. I'll leave this session open for a few minutes just to see if anyone's got any final questions but if not then please feel free to head off and join any sessions you've got this afternoon. Thank you very much for your time and we look forward to working with you over the next few years in your role. 

External Examiners Induction Event October 2023 video

Back to About the External Examiner Role