Dáil Éireann Debate, Tuesday - 13 October 2020
231. Deputy Donnchadh Ó Laoghaire asked the Minister for Education the date the lines of code containing the error in the context of the leaving certificate calculated grades model were last edited; and if further oversight and tests were carried out by the national standardisation group after the most recent changes were made to the code. [29827/20]
View answer232. Deputy Donnchadh Ó Laoghaire asked the Minister for Education the number of lines of original code, excluding code contained within imported libraries, written by a company (details supplied); if all code was written uniquely for her Department or formed part of a pre-existing product; and if so, the title of the base product. [29828/20]
View answer233. Deputy Donnchadh Ó Laoghaire asked the Minister for Education the programming languages used by a company (details supplied) in developing the system used for calculated grades. [29829/20]
View answer234. Deputy Donnchadh Ó Laoghaire asked the Minister for Education if the leaving certificate calculated grades code was written entirely by a person (details supplied); and if there was an oversight of their work within the company. [29830/20]
View answerI propose to take Questions Nos. 231 to 234, inclusive, together.
The decision to adopt a model of Calculated Grades by my Department was a result of COVID-19, which prevented the State from running the conventional Leaving Certificate Examinations this year. The statistical model underpinning the system of Calculated Grades is a complex and sophisticated bespoke development. Developing a brand-new system for providing examinations results to students in the circumstances of not being able to hold examinations was always going to be hugely challenging. The system was developed specifically for the Irish Leaving Certificate, and within an extremely tight timeframe in order for students to get their Calculated Grades results in time. In light of the intensity of the timeframe for this work, the code was being written, developed and refined on an iterative basis.
The code was written for the system of Calculated Grades by Polymetrika International Incorporated based in Canada. The principal of Polymetrika, Fernando Cartwright is considered an international expert in the field of statistics and psychometrics, the science of educational testing. This was a bespoke development in response to the inability to run the 2020 Leaving Certificate.
Technical details of the Calculated Grades model and standardisation process were published on the date of issue of the Calculated Grades results and are available at https://www.gov.ie/en/collection/90d4a-calculated-grades-document-library/.
The code operationalizing the algorithms in the analysis used a combination of custom-developed ECMAscript, standard numerical and statistical libraries, and SQL. The algorithm implementation used custom-developed code. In all, some 50,000 lines of code were contained within the complex statistical estimation system required to operationalise the standardisation process.
Following the discovery of errors in the way in which junior cycle data had been extracted and used within the model I (the Department) contracted with Educational Testing Services, the world-leading US-based non-profit organisation which specialises in educational measurement, to provide an independent expert opinion on the adequacy of the coding underpinning the standardization process, so as to provide a level of reassurance that students’ results are as intended by the standardization process. ETS evaluated the corrections made to the code and determined that the data extraction works as described in the technical document. It is the professional and expert opinion of ETS that the revision is now correct.
