Exam Development Policy

The exam development methodology used is key to build reliable and valid testing methods. At SISA, we ensure that our exams are created using best standard practices. All our exams are build keeping in mind that they are valid, reliable and error free.

SISA Exam Development Methodology:

Phase 1: Defining the Test Objective and Job Functions

SISA engages industry experts at the beginning of the exam development process to establish a clear, focused test purpose which outlines the significant job functions relevant to the title the exam is built to test. A group of experienced professionals having equal representation of the relevant industry agree to the job functions that are key to the exam title.

Phase 2: Market Research and Job Analysis

The job functions suggested by the experts is further rolled out as a survey, encouraging more industry professionals to validate the job functions as well as suggest the relevance of each of the job tasks over the other based on their experience and expertise. This goal of the survey is to outline what is important for that specific professional to know and be able to do relevant to the exam title. The data from the survey is then presented to the same group of professionals for their consideration. The final job functions are concluded based on this discussion with a clear weightage per job function in the form of an exam blueprint.

Phase 3: Test Structure

Using the exam blueprint as the skeleton, a test structure is created that includes details such as length of the test, time duration, question format, exam administration model, scoring pattern.

Phase 4: Item building

Each item (question) is designed with the goal for it to be able to accurately measure the objective of the test, is psychometrically sound and legally defensible. SISA encourages SME groups and Industry experts to contribute items providing them with an item writing guide. Upon collection of a pool of items, a panel of experts carefully reviews the items and selects the items based on the defined exam and test structure. The final exam pool only includes items that are technically, grammatically, and semantically clear, unbiased, and relevant.

Phase 5: Pilot Testing

This phase is focused on ensuring that all exams are pre-tested and evaluated. The exam is administered among the beta testers and focus groups where candidates are also asked to mark exam items that may be misinterpreted, have poor wording, or have either no or more than the expected correct answers.
The date collected from this exercise is then reviewed by a qualified psychometric professional who performs a a detailed study to assess fairness, reliability, and validity of each item; and examine performance discrepancies.

Phase 6: Standard Setting

The Standard setting phase includes decision on the passing score (also known as the passing point, the cutoff score, or the cut-score, an examinee’s score must be equal to or greater than the passing score in order for that examinee to be pass the test. Standard setting is the process SISA uses to select a passing score for an exam. A panel of experts overlooking the exam building process are once again engaged to look at the overall pilot testing results, reports and recommendations from the psychometric expert to decide on the passing score. This panel of experts work together and based on the data available set the passing score at a score point on the exam that reflects the minimum level of competency required to perform the job role defined by the exam title.

Phase 7: Exam Deployment

The exam set is then deployed for those who wish to challenge the exam in line with the exam application policy.

Note: All SISA exams are evaluated from time to time to ensure the exam quality is intact and the content is relevant.