Game programmer

This document sets out the requirements for end-point assessment (EPA) for the game programmer apprenticeship standard. It explains how EPA for this apprenticeship must operate.

It provides the EPA design requirements for end-point assessment organisations (EPAOs) for this apprenticeship standard. It will also be useful for apprentices undertaking this apprenticeship, their employers and training providers.

The EPA must be conducted by an EPAO approved to deliver EPA for this apprenticeship standard. Each employer should select an approved EPAO from the Education & Skills Funding Agency’s Register of end-point assessment organisations (RoEPAO).

Game programmer is a core and options apprenticeship standard. Apprentices must be trained and assessed against the core and options, either:

• Game programming
• Technology programming

Full-time apprentices will typically spend 24 months on-programme (before the gateway) working towards this occupational standard. All apprentices must spend a minimum of 12 months on programme.

All apprentices must spend a minimum of 20% of on-programme time undertaking off-the-job training.

Before starting EPA, an apprentice must meet the gateway requirements. For this apprenticeship they are:

• The employer must be content that the apprentice is working at or above the occupational standard
• apprentices must submit a portfolio of evidence to underpin the professional discussion
• apprentices must have achieved English and Mathematics at Level 2¹

The EPAO must confirm that all required gateway evidence has been provided and accepted as meeting the gateway requirements. The EPAO is responsible for confirming gateway eligibility. Once this has been confirmed, the EPA period starts. This EPA should then be completed within an EPA period lasting typically for 6 months.

This EPA consists of 2 discrete assessment methods.

It will be possibble to achieve the following grades in each end-point assessment method:

EPA method 1 - project with presentation and questions:

• fail

• pass

• distinction

EPA method 2 - professional discussion (underpinned by portfolio):

• fail

• pass

• distinction

Performance in these end-point assessment methods will determine the overall apprenticeship standard grade of:

• fail

• pass

• merit

• distinction

The EPA will be taken within an EPA period lasting typically for 6 months, starting when the EPAO confirms the gateway requirements are met.

Any supporting material which underpins an EPA assessment method should be submitted at the Gateway.

The apprentice’s employer must confirm that they think the apprentice is working at or above the occupational standard as a game programmer. They will then enter the gateway. In making this decision, the employer may take advice from the apprentice's training provider(s), but the employer must make the decision.

The EPAO determines when all gateway requirements have been met, and the EPA period will only start once the EPAO has confirmed this.

In addiition to the employer's confirmation that the apprentice is working at or above the level of the occupational standard apprentices must meet the following gateway requirements before starting their EPA.

These are:

• achieved English and mathematics at Level 2¹.

• for the project with presentation and questions apprentices must submit: project brief.

• A project brief will be submitted to the EPAO at the gateway, thereby allowing the EPAO to agree the project's subject, title and scope. Following the gateway, the EPAO will confirm the title of the project within 2 weeks of the gateway to ensure suficient scope to meet the KSBs mapped to this assessment method.
• The project brief must scope out the work-based project and should include a summary of the stages to be covered by the work-based project and an overview of the tasks as well as the specific responsibilities and duties assigned and to be undertaken by the apprentice.

• for the professional discussion (underpinned by portfolio) apprentices must submit: portfolio of evidence to underpin the professional discussion.

Portfolio of evidence requirements:

• Apprentices must compile a portfolio of evidence during the on-programme period of the apprenticeship. It should contain evidence related to the KSBs that will be assessed by the professional discussion (underpinned by portfolio). The portfolio of evidence will typically contain 12 discrete pieces of evidence. Evidence should be mapped against the KSBs.
• Evidence may be used to demonstrate more than one KSB; a qualitative as opposed to quantitative approach is suggested. Evidence sources may include: source code,executables, prototypes, screen recordings, project repositories, briefs, meeting minutes, audio clips. Feedback from colleagues and clients may also be included. Video clips should not exceed 10 minutes in total and should be carefully edited to highlight the pertinent features; a video screen capture with a voice-over by the apprentice can be used but the apprentice would need to initially identify themselves on camera before commencing.
• This is not a definitive list; other evidence sources can be included.
• The portfolio should not include refiective accounts or any methods of self-assessment. Any employer contributions should focus on direct observation of performance (for example witness statements) rather than opinions. The evidence provided should be valid and attributable to the apprentice; the portfolio of evidence should contain a statement from the employer and apprentice confirming this.

The EPA period starts when the EPAO confirms all gateway requirements have been met. The expectation is they will do this as quickly as possible.

The portfolio of evidence is not directly assessed. It underpins the professional discussion and therefore should not be marked by the EPAO. EPAOs should review the portfolio of evidence in preparation form the professional discussion but are not requireed to provide feedbakc after this review of the portfolio.

The assessment methods can be delivered in any order.

The result of one assessment method does not need to be known before starting the next.

Overview

A project involves the apprentice completing a significant and defined piece of work that has a real business application and benefit. The project must start after the apprentice has gone through the gateway.

The project should be designed to ensure that the apprentice's work meets the needs of a business, is relevant to their role and allows the relevant KSBs to be the assessed for the EPA. The employer will ensure it has a real business application and the EPAO will ensure it meets the requirements of the EPA, including suitable coverage of the KSBs assigned to this assessment method as shown in the mapping of assessment methods. The EPAO must refer to the grading descriptors to ensure that projects are pitched appropriately.

This EPA method includes 2 component(s):

• project with a project output
• presentation with questions and answers.

Rationale

The rationale for this assessment method is:

Programmers in the games industry are principally engaged in developing software, and each programmer will engineer many different technical systems as part of the development of a complete game. As the project cycle for games is often too long to assess a complete product, this assessment uses one of these technical systems as a practical way of demonstrating software engineering skills and their associated KSBs. Although it isn’t compulsory for this system to be assessed in isolation from its associated product, this may be necessary when the wider product is incomplete or subject to confidentiality requirements. The project and any components must be assessed holistically by the independent assessor when they are deciding the grade for this EPA method.

The project and any components must be assessed holistically by the independent assessor when they are decising the grade for this EPA method.

Component 1: Project with a project output

Delivery

Apprentices must complete a project which may be based on any of the following:

• a specific problem
• a recurring issue
• an idea or opportunity.

The project may also be based on:

• a development of live product where it is appropriate to do so. Where client confidentiatlity requires it the system may be demonstrated in a test harness, or using placeholder/white box assests which conceal the product and/or client.

Typical project titles could include:

• A Third-Person close-Combat mechanic
• An Aeriel Combat Mechanic
• Procedurally Generated Level Content
• Destructible objects
• A Shader-Based Visual Effect
• Parallelisation of an Existing Process.

To ensure the project allows the apprentice to meet the KSBs mapped to this EPA method to the highest available grade, the EPAO should sign-off the project’s title and scope at the gateway to confirm it is suitable.

The project output must be in the form of a software artefact.

The apprentice must start the project after the gateway. They must complete and submit the software artefact to the EPAO after a maximum of 12 weeks. The employer should ensure the apprentice has the time and resources within this period, to plan and complete their project. The apprentice must complete their project and the production of all its components unaided.

The apprentice may work as part of a team which could include technical internal or external support. However, the project output must be the apprentice’s own work and will be reflective of their own role and contribution. The apprentice and their employer must confirm that the project output(s) is the apprentice’s own work when it is submitted.

The minimum requirements software artefact are

• A complete working technical system, or collection of systems which executes and produces a verifiable outcome. Vertification may be achieved through a test harness or other code framework which is not of the apprentice's own creation, but the technical system itself must be all their own work.
• Typically, it would be supported by a code repsoitory evidencing its iterative development by the apprentice over the asesessment period, and not just the final codebase.
• Typically the software artfeact would be a real-time system, but it could also be something which creates data for or from a real-time system.

Component 2: Presentation with questioning

Overview

Apprentices will be required to produce, subit and deliver a presenation to the independent assessor. A copy of the presentation must be submitted to the EPAO at the same time as the project and project output(s); 12 weeks after the gateway.

The independent assessor must have a minimum of 2 weeks to review the software artefact and copy of the presentation in advance of the presentation in order to prepare apropriate questions.

The presentation will provide an overview of the apprentice's project outpu(s). This will be followed by questioning by the indepndent assessor.

As a minimum, all presentations must include:

• An overview of the project
• The project scope (including key performance indicators)
• Summary of actions undertaken by the apprentice
• Project outcomes and how these were achieved.

The presentation should require the apprentice to fully demonstrate the KSBs that are mapped to this assessment method.

Delivery

The presentation will provide an overview of the apprentice's project and the software artefact. Independent assessors will ask questions after the presentation.

The presentation will be arranged by the EPAO in consultation with the employer. It will be presented to an independent assessor on a one-to-one basis, either face-to-face or via online video conferencing. If using an online platform, EPAOs must ensure appropriate measures are in place to prevent misrepresntation.

The presentation and questioning must last 90 minutes This will typically include a presentation of 45 minutes and questioning lasting 45 minutes. The independent assessor has the discretion to increase the total time of the presentation and questioning by up to 10% to allow the apprentice to complete their last point. Further time may be granted for apprentices with additional needs, in-lin with the EPAO's reasonable adjustments policy.

Questions must be asked: The purpose of questioning is:

• To verify that the software artefact was created by the apprentice
• To follow up on any areas of KSbs that the assessor feels haven't been sufficently covered in the presentation.

The independent assessor must ask at least 5 questions. Questions may be taken from an EPAO question bank or be generated by the independent assesor. Independent assessors must use the question bank as a source for questioning and are expected to use their professional judgement to tailor those questions appropriately. Follow up questions are permitted where clairifcation is required. Independent assessors are responsbile for generating suitable follow-up questions in line with the EPAO's training and standardisation process. Quesions must be designed to allow the apprentice to demonstrate the KSBs mapped to the assessment method. Questions must be open and should be holistic in nature to enable the apprentice to cover several KSBs as part of their response.

KSBs observed, and answers to questions must be recorded by the independent assessor. The independent assesor will make all grading decisions.

The independent assesor must use the full time available for questioning to allow the apprentice the opportunity to evidence occupational comptence at the highest level available, unless the apprentice has already acheived the highest grade available.

Those KSBs that the apprentice did not have the opportunity to demonstrate during the project, product output(s) and/or presentation can instead be covered by questioning, although these should be kept to a minimum.

To deliver the presentation, the apprentice will have access to:

• A PowerPoint or equivalent presentation software
• Source control archives for the project or suitable visual evidence of their use
• An appropriate development environment for explaining and demonstrating code
• Computer with internet connection.

KSBs met and ansers to questions must be recorded by the independent assessor. The independent assesseor will make all grading decisions. Apprentices must be given at least 2 weeks notice of the assessment method.

Assessment location

The presentation with questioning must take place in a suitable venue selected by the EPAO for example the EPAO's or employers premises. The presentation with questioning should take place in a quiet room, free from distractions and influence.

Question and resource development

Questions must be written by EPAOs, be relevant to the occupation and assess the KSBs mapped to this project with presentation and questions. It is recommended that this be done in consultation with employers of this occupation. EPAO's should maintain the security and confidentiality of their questions when consulting employers.

Each EPAO must develop a question bank of sufficient size to prevent predictability and review it regularly (at least once a year) to ensure it, and the questions it contain, are fit for purpose.

EPAOs must ensure that apprentices have a different set of questions in the case of re-sits or re-takes. .

• independent assessor EPA materials which include:
  • training materials
  • administration materials
  • moderation and standardisation materials
  • guidance materials
  • grading guidance
  • question bank
• EPA guidance for the apprentice and employer.

Independent assessors are responsible for generating suitable follow-up questions in line with the EPAO's training and standardisation process. Questions must be designed to allow the apprentice to demonstrate the KSBs mapped to the assessment method. Questions must be open and should be holistic in nature to enable the apprentice to cover several KSBs as part of their response.

Overview

In a professional discussion, an independent assessor and apprentice have a formal two-way conversation. It gives the apprentice the opportunity to demonstrate their competency across the KSBs as shown in the mapping.

Rationale

The EPA method is being used because while programmers in the games industry spend most of their time developing software, it is their ability to work productively with a much wider team of creative people that determines their real value to a game studio. As such this assessment allows apprentices to draw upon a wider portfolio of work created after the gateway to relate their KSBs to experiences involving interdisciplinary teamworking and professionalism.

Delivery

A professional discussion must be structured to give the apprentice the opportunity to demonstrate the KSBs mapped to this EPA method to the highest available grade.

Questions must be asked. The purpose of questioning is:

• to explore the apprentice's experience and understanding in which they apply the KSBs in a work context.

The EPAO must give an apprentice 2 weeks notice of the date and time of the professional discussion.

Apprentices must have access to their portfolio of evidence to underpin the professional discussion during the professional discussion.

Apprentices can refer to and illustrate their answers with evidence from their portfolio of evidence, however the portfolio of evidence is not directly assessed.

The professional discussion must last for 90 minutes. The independent assessor can increase the time of the professional discussion by up to 10%. This time is to allow the apprentice to a complete their last point.

For the professional discussion, the independent assessor must ask at least 10 questions.

Questions may be taken from an EPAO question bank or be generated by the independent assessor. Independent assessors must use the question bank as a source for questioning and are expected to use their professional judgement to tailor those questions appropriately. Follow-up questions are permitted where clarification is required. Independent assessors are responsible for generating suitable follow-up questions in line with the EPAO’s training and standardisation process. Questions must be designed to allow the apprentice to demonstrate the KSBs mapped to the assessment method. Questions must be open and should be holistic in nature to enable the apprentice to cover several KSBs as part of their response.

The independent assessor conducts and assesses the professional discussion.

The independent assessor must keep accurate records of the assessment. The records must include the KSBs met, the grade achieved and answers to questions.

The professional discussion can be conducted by video conferencing. The EPAO must have the processes in place to verify the identity of the apprentice to ensure the apprentice is not being aided.

The independent assessor will make all grading decisions.

Assessment location

The professional discussion must take place in a suitable venue selected by the EPAO. The venue could be the EPAO's or employer's premises. The professional discussion should take place in a quiet room, free from distractions and influence.

The assessment method can take place in any of the following locations:

• employer's premises
• other suitable location determined by the EPAO (e.g. assessment centre or training provider)
• via video conferencing

Video conferencing can be used to conduct the professional discussion, but the EPAO must have processes in place to verify the identity of the apprentice and ensure the apprentice is not being aided.

Question and resource development

EPAOs must write an assessment specification and question bank. The specification must be relevant to the occupation and demonstrate how to assess the KSBs shown in the mapping. It is recommended this is done in consultation with employers of this occupation. EPAOs should maintain the security and confidentiality of EPA materials when consulting employers. The questions must be unpredictable. A question bank of sufficient size will support this. The assessment specification and questions must be reviewed at least once a year to ensure they remain fit-for-purpose.

EPAOs will develop purpose-built question banks and ensure that appropriate quality assurance procedures are in place. For example, considering standardisation, training and moderation. EPAOs will ensure that questions are refined and developed to a high standard.

EPAOs must ensure that apprentices have a different set of questions in the case of re-sits or re-takes.

EPAOs must produce the following materials to support the professional discussion (underpinned by portfolio):

• Independent assessor assessment materials which include:
  • training materials
  • administration materials
  • moderation and standardisation materials
  • guidance materials
  • grading guidance
  • question bank
• EPA guidance for the apprentice and employer.

All assessment methods are weighted equally in their contribution to the overall EPA grade.

Performance in the EPA will determine the apprenticeship grade of fail, pass, merit or distinction. Independent assessors must individually grade each assessment method, according to the requirements set out in this plan.

EPAOs must combine the individual assessment method grades to determine the overall EPA grade. Apprentices who fail one of more assessment method will be awarded an overal EPA 'fail

In order to achieve an overall EPA grade, apprentices must achieve:
Any FAIL = Overall FAIL Project PASS + Discussion PASS = Overall PASS Project PASS + Discussion DISTINCTION = Overall MERIT Project DISTINCTION + Discussion PASS = Overall MERIT Project DISTINCTION + Discussion DISTINCTION = Overall DISTINCTION

Apprentices who fail one or more assessment method(s) will be offered the opportunity to take a re-sit or a re-take at the employer’s discretion. The apprentice’s employer will need to agree that either a re-sit or re-take is an appropriate course of action.

A re-sit does not require further learning, whereas a re-take does.

Apprentices should have a supportive action plan to prepare for a re-sit or a re-take.

The employer and EPAO agree the timescale for a re-sit or re-take. A re-sit is typically taken within 2 months of the EPA outcome notification. The timescale for a re-take is dependent on how much re-training is required and is typically taken within 3 months of the EPA outcome notification. The timescale for a re-take is dependent on how much re-training is required and is typically taken within 3 months of the EPA outcome notification.

Failed EPA methods must be re-sat or re-taken within a 6-month period from the EPA outcome notification, otherwise the entire EPA will need to be re-sat or re-taken in full.

Re-sits and re-takes are not offered to apprentices wishing to move from pass to a higher grade.

Where any assessment method has to be re-sat or re-taken, the apprentice will be awarded a maximum EPA grade of Pass, unless the EPAO determines there are exceptional circumstances.

The EPAO must have in place clear and fair arrangements for making reasonable adjustments to the assessment methods for the EPA for this apprenticeship standard. This should include how an apprentice qualifies for reasonable adjustments and what reasonable adjustments will be made. The adjustments must maintain the validity, reliability and integrity of the assessment methods outlined in this EPA plan.

Internal quality assurance refers to how EPAOs ensure valid, consistent and reliable EPA decisions. EPAOs must adhere to the requirements within the roles and responsibilities section and:

• have effective and rigorous quality assurance systems and procedures that ensure fair, reliable and consistent EPA regardless of employer, place, time or independent assessor
• appoint independent assessors who are competent to deliver the EPA and who:
  • have recent relevant experience of the occupation or sector to at least occupational level 7 gained in the last 5 years or significant experience of the occupation or sector
  • meet the following minimum requirements:

    should have three or more years professional experience as a programmer on an interdisciplinary game development team and have released at least one commercial title.

• operate induction training for anyone involved in the delivery and/or assessment of the EPA
• provide training for independent assessors in good assessment practice, operating the assessment tools and making grading decisions
• provide ongoing training for markers and invigilators
• provide standardisation activity for this apprenticeship standard for all independent assessors:
  • before they conduct an EPA for the first time
  • if the EPA is updated
  • periodically as appropriate (a minimum of annually)
• conduct effective moderation of EPA decisions and grades
• conduct appeals where required, according to the EPAO’s appeals procedure, reviewing and making final decisions on EPA decisions and grades
• have no direct connection with the apprentice, their employer or training provider. In all instances, including when the EPAO is the training provider (for example a higher education institution)

Affordability of the EPA will be aided by using at least some of the following:

• utilising digital remote platforms to conduct applicable assessment methods

• assessing multiple apprentices simultaneously where the method of assessment permits this

• using the employer’s premises

Professional body recognition is not relevant to this occupational apprenticeship.