Facilitating math and language examination – also with speech

College voor Toetsen en Examens

The College voor Toetsen en Examens (hetCvTE) is an independent administrative board, responsible for all exams in secondary education and exams for Dutch as second language (necessary for obtaining a Dutch citizenship).

In the future, math and language exams in vocational education will also be the responsibility of the CvTE. The mission of this board is to guarantee the quality of the exams and to provide circumstances that assure a smooth examining.

Business challenge

The system the CvTE has in place now is capable of planning, presenting and correcting exams. This system is developed over ten years ago not meeting today’s standards and witholding any innovation on future development. The CvTE decided that a new system for digital examining was a logical investment. Further development on the old system would not be cost effective, since the new features CvTE needed, would have a major impact on the overall architecture of the system. Data storage and retrieval and the planning module are the responsibility of DUO (Dienst Uitvoering Onderwijs). The modules to preview, view, correct, review and practice exams with, are Trifork’s responsibility.

Project outline

The most visible improvement is the user interface. For all the tools Trifork created a new interaction and visual design. The new design had to preserve all the things that people liked after ten years of use. In the same time several tools were introduced and embedded in the examining environment; calculator, notepad, magnifier, spell-check, protractor, ruler, marker, special characters, and formula editor.

A major audible improvement is the text to speech (TTS) integration. Trifork integrated Acapela TTS which makes it possible to have all text based content read to you. Trifork supports Dutch and 3 other languages. The TTS switches language automatically when the language within a text changes.

The content of the exams is provided by Cito, and needs to be imported into the assessment platform. The results of each candidate also need to be reported back to Cito, for analysis. The system uses QTI 2.1 to exchange assessments and results. With QTI 2.1 CvTE can accept a solid, well proven and world-wide adapted content standard. This enables them to take in content from various sources and stay independent of their content suppliers.

To prevent cheating, Trifork has developed a client lock-down browser. This browser prevents access to any other application during the assessment. A browser is provided for Windows, Linux and Mac. The assessment system does a browser check when a candidate tries to start an assessment. Access with regular browsers is rejected.

Architecture

The QTI is parsed to HTML in the lock-down browser. Trifork created an unique QTI engine in JavaScript that translates the QTI to HTML5 within the student’s browser. Since QTI describes the entire assessment  including the answers it would impose a security risk to sent this to the browser of the candidate. For this reason Trifork created a rendering filter that strips all the data that should stay secret.

QTI_logo_(small)

The storage and retrieval of questions and given answers is the responsibility of DUO (Dienst Uitvoering Onderwijs). Together with DUO, Trifork has designed a DAL (Data Abstraction Layer), which acts as the interface between the assessment environment and the storage infrastructure. For local development, and for the offline preview environment used by assessment constructors, Trifork has provided an in-memory implementation of the DAL.

The authentication process is part of the logistical application built by DUO. The assessment engine itself does not need to know about user details. However, it does need to be able to restrict access to only authorized candidates. For access security, Trifork has implemented HMAC validation on the URL that launches the assessment engine.

Besides the QTI player in which a student makes his assessment Trifork also created a correction component for the teachers to score manually, and a preview component for test constructors in which they can view the assessment they are creating and all the metadata that comes with it.

Scrum

Although the project was a European tender with a fixed price and fixed date, CvTE managed to keep the requirements specified in de tender on a level that made it possible to fill in the details just before implementation. Some functionality was implemented gradually in several iterations. The product owner made full use of the potential that the scrum process offers and was always looking for functionality that added the most value.

Conclusion

The new system will provide a robust platform to facilitate the growing number of digital exams. Most of the feedback that has been generated during field test and during the real exams in which users encountered the new system are very positive. To maintain this quality the system will be evaluated constantly and will be improved in the years to come. Last but not least the project started in winter 2011 and has been delivered in spring 2013, on time and within budget.

case_cve

 

Website: College voor Toetsen en Examens

Read more: Trifork blog

More information? Contact us!

Read more about : QTI

QTI_logo_(small)