.jpg)
The challenge: exposure to fraud, slow corrections and little visibility of results
ESS examenservices develops and supplies exam products for Dutch secondary vocational education (mbo), covering subjects such as retail, entrepreneurship, sales, fashion, aviation, logistics, interior and home furnishing, and business services. ESS works with Paragin Group, using Remindo for theory exams and Skillsly for practical assessments. Almost every mbo institution in the Netherlands runs exams from ESS, so reliability matters a great deal.
Sylvia Verrips, Manager at ESS, has worked there for more than seventeen years. She describes how exams are delivered:
“Schools choose what suits their vision from our range of theory exams, practical exams, projects, open-book exams and simulations.”
It was the theory exams where ESS ran into a problem for years. They were distributed on paper, and that made them vulnerable.
“Unfortunately, we dealt with fraud fairly regularly. And with theory exams in particular, that is very costly. It also takes a lot of time to put right.”
On top of that, if an error did find its way into an exam, it stayed in circulation for longer than anyone wanted. ESS was also entirely dependent on feedback from schools for any insight into results.
“Schools used to print all the exams at the start of the school year. An error could then sometimes last a whole school year or longer, if the school didn’t download the latest versions in the meantime.”
The solution: digital delivery, a clear overview and less exposure to fraud
ESS decided to move its theory exams fully online with Remindo, which supports the complete assessment cycle. The COVID-19 period sped up the switch, and no theory exams have been issued on paper since.
The deciding factor was the ability to build randomised exams from the item bank.
“The big advantage of Remindo is that you can put together randomised exams. The next student doesn’t get exactly the same set of questions, which already makes fraud less of an issue.”
That approach did raise a new question, from schools and from the Dutch Inspectorate of Education: how do you keep exams equivalent to each other? ESS solved this by grouping items into categories in which every item carries the same weight and is of the same type.
“Every item in one of those groups ultimately has the same value and is the same kind of question. It can’t be that one student gets ten multiple-choice questions and another gets twenty.”
ESS uses Remindo across the whole process around its knowledge exams. The organisation writes the items itself, together with external authors. Alongside the exams, the system now also holds practice banks, so students can get used to the way questions are asked. Signals from the item analysis form the basis for improvement.
“It’s just great that the system alerts you when p-values and Rit values deviate, so you think: I need to do something with this.”
When items deviate, ESS convenes its exam validation committee to look at where the problem lies and how the items can be revised.
For practical assessments, ESS works with Skillsly. Several exam variants have been built into the system.
“The big advantage is that everything is in one place. You have all the components in a single overview. Examination committees can log in and see the status of their students too, and verify the exams that have been taken.”
Choosing the same provider for both knowledge and practical exams was a deliberate decision. When the idea came up to digitise the practical side as well, Sylvia went straight to Paragin Group.
“I think there is real added value in having both products with one supplier. If something comes up, it’s simply easier when it all sits with one supplier.”
For ESS, that means a single point of contact: the same support desk, the same named contacts and one place for questions about both theory and practical exams.
Looking back on implementing Skillsly, she recalls:
“Rolling out practical exams is many times more complex, we have to admit. And scaling works very differently too. With theory exams, it doesn’t make much difference if you scale up to larger numbers. With Skillsly, it makes a big difference.”
Part of that complexity lies outside the system. Many students do their work placements at companies where supervisors can’t simply go online. That is why digital delivery has not been made mandatory for practical exams, while it has for theory exams.
The result: more up-to-date exams and better insight into student progress
The biggest impact is on how current and how good the exam products are.
“The moment we spot an error and correct it, it’s corrected for everyone straight away. That’s also much better for the quality of our products.”
The same applies to changes driven by new laws and regulations.
“On paper, you really have to leaf through everything. In Remindo, a search term brings up every item related to it in one go. So you can surface the changes that new laws and regulations require much faster.”
Because ESS can now look at anonymised responses behind the scenes, the organisation depends less on feedback from schools and has a better understanding of how students reason.
“Sometimes things seem completely logical to us, but we are looking at them from an adult’s point of view. Then you see the students’ answers and think: they looked at this in a completely different way. We just didn’t pick that up before. Now we can trace those things back and adjust them much more easily. That is a really big gain in the quality of our own products.”
For practical exams, the gain is mainly in visibility and time. Everyone can follow a student’s status at any moment, so schools can step in earlier.
“In the past, it sometimes happened that a school visited a placement company and nothing had been submitted and no assessment forms had been completed. You’ve lost your travel time and planned your schedule around it. Because you can now see it digitally in advance, you can say: I’m coming by tomorrow, please upload it. As a school, you simply have a better view of what’s going on at your placement companies.”
Paper is also disappearing from the process. Some institutions now refer to the system in the student’s exam logbook instead of printing everything, which saves on printing, storage space and managing retention periods.
Work is under way on an OKE integration for Skillsly, which will allow data to be exchanged with student information systems such as Eduarte and Magister.
“Let’s be completely honest: the human factor is where errors can creep in. Being able to digitise this part of the process soon means that will happen less often.”
According to Sylvia, digitisation also makes schools’ own exam processes more consistent. Where paper left more room for individual interpretation, a digital process gives firmer direction.
“For examination committees, that’s really good. They often had that standardisation well documented already and assumed people were following it. But now there is genuinely no other way to do it. For the value of the diploma, it is much better that it works like this, because those processes really are more rigorous.”
For the future, there is one clear wish: an integration between Skillsly and Remindo, so schools only need to enter data in one system. That development runs in parallel with a broader shift at institutions, which are consolidating their administration from many small environments into a few large ones.
What she values most about working with Paragin Group is the room to contribute.
“You’re not just standing on the sidelines. You also have room to think along actively and make a contribution. The fact that we are still approached to share our views on certain matters, that we are taken seriously and really listened to, is something I find very valuable as a customer.”