Build systems that collect, manage, and convert data into usable information for data scientists, data analysts and business intelligence analysts to interpret.
This occupation is found in a wide range of public and private sector organisations who work with large data sets including Government departments, NHS, financial and professional services, IT companies, retail and sales and education providers.
The purpose of the occupation is to build systems that collect, manage, and convert data into usable information for data scientists, data analysts and business intelligence analysts to interpret. A data engineer’s main aim is to make data accessible and valid so that an organisation can use it to evaluate and optimise their performance. The role of the data engineer is pivotal to any organisation; it ensures that data pipelines are established to support data scientists and other business stakeholders.
A data engineer will build and implement data flows to connect operational systems, and re-engineer manual data flows to enable scalable and repeatable use. They integrate, support and manage the build of data streaming systems, writing extract transform and load scripts that perform in line with business requirements.
They are responsible for providing high quality, transparent data that enables effective governance and smart business decisions. They will analyse the performance indicators of the data systems that provide clean, regular, and accurate data. A data engineer will understand how data and an organisation’s data architecture is essential to business outcomes.
A data engineer will be able to gather requirements for data solutions, and they demonstrate and articulate data solutions to stakeholders in a way that can be easily understood. Data engineering encompasses a range of activities from collecting data to employing various data processing frameworks, including but not limited to ETL (Extract, Transform, Load), and collaborating with data scientists and other data-centric roles. Data engineers may work in an office or work remotely depending on the sector they work in and location of the employer.
In their daily work, an employee in this occupation will work autonomously or collaboratively with clients, in the business and or data team. A data engineer will work with data analysts, Data scientists and data architects and liaise with other teams and internal and external stakeholders to ensure their data requirements are captured and managed to the specified standard. They will also work closely with machine learning engineer (Ops), software engineers, software developers and technology teams.
An employee in this occupation will be responsible for completing their own work to specification, , ensuring they meet set deadlines. A data engineer contributes towards, engineering designs, plans, execution and evaluation working to time, cost and quality targets. They deliver to the product roadmap and are responsible for meeting quality requirements and working in accordance with health and safety and environmental considerations. They will work according to organisational procedures and policies, to maintain security and compliance.
This is a summary of the key things that you – the apprentice and your employer need to know about your end-point assessment (EPA). You and your employer should read the EPA plan for the full details. It has information on assessment method requirements, roles and responsibilities, and re-sits and re-takes.
An EPA is an assessment at the end of your apprenticeship. It will assess you against the knowledge, skills, and behaviours (KSBs) in the occupational standard. Your training will cover the KSBs. The EPA is your opportunity to show an independent assessor how well you can carry out the occupation you have been trained for.
Your employer will choose an end-point assessment organisation (EPAO) to deliver the EPA. Your employer and training provider should tell you what to expect and how to prepare for your EPA.
The length of the training for this apprenticeship is typically 24 months. The EPA period is typically 4 months.
The overall grades available for this apprenticeship are:
When you pass the EPA, you will be awarded your apprenticeship certificate.
The EPA gateway is when the EPAO checks and confirms that you have met any requirements required before you start the EPA. You will only enter the gateway when your employer says you are ready.
The gateway requirements for your EPA are:
Project with report
You will complete a project and write a report. You will be asked to complete a project. The title and scope must be agreed with the EPAO at the gateway. The report should be a maximum of 3500 words (with a 10% tolerance).
You will have 10 weeks to complete the project and submit the report to the EPAO.
You need to prepare and give a presentation to an independent assessor. Your presentation slides and any supporting materials should be submitted at the same time as the project output. The presentation with questions will last at least 50 minutes. The independent assessor will ask at least 6 questions about the project and presentation.
Professional discussion
You will have a professional discussion with an independent assessor. It will last 80 minutes. They will ask you at least 10 questions. The questions will be about certain aspects of your occupation. You can use it to help answer the questions.
The EPAO will confirm where and when each assessment method will take place.
You should speak to your employer if you have a query that relates to your job.
You should speak to your training provider if you have any questions about your training or EPA before it starts.
You should receive detailed information and support from the EPAO before the EPA starts. You should speak to them if you have any questions about your EPA once it has started.
If you have a disability, a physical or mental health condition or other special considerations, you may be able to have a reasonable adjustment that takes this into account. You should speak to your employer, training provider and EPAO and ask them what support you can get. The EPAO will decide if an adjustment is appropriate.
| Version | Change detail | Earliest start date | Latest start date |
|---|---|---|---|
| 1.0 | Approved for delivery | 11/12/2023 | Not set |
Crown copyright © 2025. You may re-use this information (not including logos) free of charge in any format or medium, under the terms of the Open Government Licence. Visit www.nationalarchives.gov.uk/doc/open-government-licence