Conducting proficiency tests
Guideline 5.0 (GMM Resolution, Königswinter 2012)
The following statements will serve as guidelines for the uniform handling of the planning, preparation, execution, evaluation, and reporting on proficiency tests within the GFS.
1.2 Scope of validity
The guidelines apply to GFS members and to external third parties who have been commissioned with the organisation of proficiency tests.
1.3 The foundations or starting points are described in:
- “Guidance on the Conduct of Proficiency Tests and Collaborative Exercises within ENFSI”:
www.enfsi.eu / Standing Committees / QCC / Approved Documents
- ILAC-G13:08/2007 'Guidelines for the Requirements for the Competence of Providers of Proficiency Testing Schemes':
- EA-03/04 'Use of Proficiency Testing as a Tool for Accreditation in Testing':
- ISO/IEC 17043-2010 ‘Conformity Assessment – General Requirements for Proficiency Testing’
The anonymisation of the participants must be ensured. Only unanimous consent by the participants will permit the revocation of this anonymity. Numbers will be assigned to the individual participants, and their answers will be returned through an appointed representative.
2.2 Types of qualification tests
Qualification tests (proficiency tests) may include qualitative identifications (comparisons), quantitative measurements, and/or interpretations.
There are two types:
In the first type, the test material (e.g. original document) is to be passed on to the next participant in line with the schedule provided. Also to be observed:
- Identical conditions must be assured, so that all participants receive identical test materials (apart from changes or damage that may result from improper handling).
- Only non-destructive test methods are permitted.
- Sufficient time must be calculated for passing on the test material within the group of participants (provide schedule).
A second type involves providing each participant with a set of test materials. To be observed here:
- Differences in the individual sets must be taken into consideration in the test evaluation.
- Non-destructive examinations are possible.
A combination of the two test types, with one set of test materials per group of participants, is also possible.
2.3 Expected result
Before the proficiency test is released, a ‘competent body’ (e.g. testing commission) should examine it in regard to its conclusiveness and its expected results. The determination of the ‘competent body’ can be communicated to the organiser.
3.1 Test area/scope
Proficiency tests should correspond to day-to-day casework and should be based on available test methods and tools, as described in Guideline 1.01. Depending on the content of the test, a brief description of the case is possible. Complex details regarding the case, as well as those which might be misleading, should be avoided. It should not take more than two work days to complete the test.
3.2 Test material
To enable the comparison of the individual results (evaluation, representativity), all of the participants are to be provided with identical or uniform test material, if possible. (This applies to tests using various sets of test material.) The test material must be clearly marked (incriminating material, comparative material). The provenance of the comparative material (unbiased or ad hoc) should be named.
Preparation of the test material should be documented to make it possible to trace its condition of origin.
Participation is obligatory for members. For candidate members, participation is voluntary.
Notification of the test will be made in advance. This notification will contain the following information:
- A short description of the proficiency test and its purpose
- Criteria for participants who are not also members of the GFS
- Scheduling: Registration deadline, filing deadline
- Possible costs and payment terms for non-members
- Name/contact address of the test organiser.
The packaging must ensure that the test material will not be damaged or altered. For proficiency tests in which the test material is forwarded to the next participant, it must be ensured that this requirement can be fulfilled by all those involved.
In this case, it must be very clear to whom the test material is to be forwarded by each individual participant. The organiser is to be informed when the material is forwarded, so that the schedule can be monitored.
4.3 Answer sheet
The results are to be recorded by the participants on an answer sheet that is provided and that is clearly structured and easy to understand. The test is to be based on a fair, practice-related approach.
Multiple choice questions will simplify the evaluation (uniform assessment scale), but are not appropriate for every case.
The participants are to indicate which test methods and equipment are used.
To ensure a meaningful statistical evaluation of the test, it is permissible to request additional information about the participants (training, experience in the field, expert status, quality control, etc.), as long as their anonymity is ensured.
It must be possible to return the result in anonymised form for the evaluation.
The correct test result is not to be communicated by the organiser before the closing date.
If this is statistically meaningful and possible on the basis of the test arrangement, the results are to be evaluated and documented, including the statistical methods, standard deviation, and measurement uncertainty. This is the only way to guarantee a uniform and objective evaluation of the test results.
The evaluation results are to be compiled in a suitable form (e.g. chart, table) and presented at one of the following membership meetings. The overall results will be summarised in anonymised form. Individual results will be presented so that each of the participants will be able to identify him/herself, but will not be identified by others. The organiser will comment on the results.
The result of a proficiency test is to be discussed in the appropriate setting. If this should be necessary (e.g. in the case of a false expertise, but also failed test preparation or erroneous/insufficient evaluation), appropriate measures should be recommended for all participants (e.g. optimisation of the test methods, changes in training, help in the preparation and evaluation of a proficiency test by scientifically experienced experts). If an expertise is incorrect, assuming the test has been correctly prepared and implemented, responsibility for measures to be taken lies with the test participants. The test organiser does not have the appropriate competence. However, he does bear responsibility for the scientifically correct development, implementation and evaluation of the test.
All documents connected with the test that has been carried out are to be stored for five years after the results have been presented to the members. Anonymisation documents are to be stored at the GFS office, together with the rest of the documents relating to this test, in a separate, closed envelope, signed by the organiser,. After the prescribed period of time has elapsed, the documents are to be destroyed without being opened. A record should be made of the fact that they have been destroyed.
5.4 Confirmation of participation
The participants should be given a confirmation of their participation (without an evaluation of the results).