The Electronic Marking Solution
Score objective and subjective items using a web based system from wherever you are. ScoreOnWeb is designed to fulfill the MOE needs in large scale scoring operatives. Totally configurable and adaptable to your process.
ScoreOnWeb is a software solution specially developed to allow correction of large volumes of tests with closed (multiple choice) and open handwritten responses under a safe, controlled and user-friendly environment.
From its first version dated on 2001, used to successfully score 11,000 tests, with more than 33,000 open handwritten responses, in the Argentine province of La Pampa; ScoreOnWeb Constantly evolved allowing testing organizations solve all their scoring needs.
On 2006, ScoreOnWeb was first used nationally in Argentina. The ScoreOnWeb team worked closely with the Ministry of Education of the country to define the entire process, providing consultancy and advice for the design and printing of examination forms, adjusting the parameters of the system to operational needs, developing material for training of all users, scanning all the exams, providing the web server and Internet access during scoring, and delivering final reports according to the needs of the MOE. The system was used to process 1.6 million tests that included 4 million open response and 32 million multiple-choice questions; more than 6.4 million sheets of paper. This experience involved more than 600 qualifiers distributed throughout the country, connected over the Internet from remote locations over 1,500 km of the system server. The process was carried out on time and on budget, and the final scoring result was given 25 days from the scan the first test. As a result of these efforts, the Ministry was able to examine millions of students, using open-response questions manuscript in record time and at a fraction of the cost that would have meant the use of traditional methods of classification.
It has been several years since the first version, and today ScoreOnWeb is a system with a multilingual graphical interface, supporting any language in the world, providing information compatible with any statistical software, while maintaining the simplicity of use.
ScoreOnWeb became an indispensable tool for all institutions wishing to automate the process of scoring exams.
ScoreOnWeb's Benefits
With over 9 years helping a variety of organizations process open-ended items, 4s Software and Services project managers have noticed several challenges that hamper many organizations’ efforts to complete their scoring projects in an accurate, timely and cost-efficient manner. These include issues like the following
Limitation of OMR Technology
The heart of the ScoreOnWeb solution is a highly customizable software application, accessed through Internet browsers. This program was created specifically for testing organizations, to automate the classification of handwritten open response questions, maintaining the accuracy of OMR optical mark rating for closed-response questions. Except the scanning of the documents, all aspects of the system are controlled using standard Internet browsers.
Accurate scoring is of paramount concern to any testing body
To ensure that each response is accurately scored, some organizations will have two or more scorers grade the same item and then compare the results. In a manual environment, this posses logistical problems not only with paper distribution, but also in ensuring that one scorer doesn’t see a previous scorer’s evaluation. ScoreOnWeb eliminates this problem by facilitating true “double-blind” scoring that is controlled automatically through its unique workflow system. ScoreOnWeb employs technology including double-blind scoring and randomly embedded pre-scored items in the processing queue to monitor the accuracy of individual scorers. These processes can be easily customized within ScoreOnWeb to suit the needs of any testing body.
Timeliness
Many testing services miss their targets for reporting on their subjective items. ScoreOnWeb can help reduce turn-around time by automating the scoring process and allowing administrators to see the overall progress of scoring in real-time. This allows administrators to identify bottlenecks and address concerns in a timely manner that isn’t possible with a manual process.
Answer Sheet Security
The ScoreOnWeb server in a web server running Windows™ Server operating system, with Internet Information Services (IIS) installed. Al system data are stored in a MS SQL Server database.
Exams and Users language
There are more than 6,000 different languages in the world. ScoreOnWeb is a multilingual system, with real time on-screen language change. Its unique technology allows that different people working over the same test, seeing their own screens in their chosen language. Being a highly configurable system, each organization can set all the texts of the screens to the languages and dialects of the area, greatly facilitating the understanding of the notes or messages delivered by the system.
Process Anonymity
Typically, a testing organization wants to ensure that scorers cannot identify individual students during the grading process. This is a challenge in a manual environment as it is common to have the student name appear on the test form. ScoreOnWeb eliminates this issue by temporarily separating any student identification from their responses. The scorers see only “clips” of the written response areas for evaluation. When the score has been given, the system automatically records the results in the appropriate student record.
Examination Control
The solution uses high quality image “clips” of student writing and routes them to Scorers using standard Internet technology. It allows the examinations to be under the direct control of the testing organization at all times.
Eliminates time in moving the papers
The application accessible from any browser eliminates the time and expense of moving paper forms to individual Scorers.
Eliminates expenses and logistical problems descentralizing people
Because the system is Web-based, Scorers, Supervisors, and Administrators can be physically located in any place that has Web access. This eliminates the expense and logistical problems with bringing Scorers to a centralized location.
ScoreOnWeb's Process
The heart of the ScoreOnWeb solution is a highly customizable software application, accessed through Internet browsers. This program was created specifically for testing organizations, to automate the classification of handwritten open response questions, maintaining the accuracy of OMR optical mark rating for closed-response questions. Except the scanning of the documents, all aspects of the system are controlled using standard Internet browsers.
Planning Consultation
A successful ScoreOnWeb implementation begins with pre-deployment consultations. During these on-site meetings, our development team will work with the testing agency to design an appropriate workflow process that will be implemented with the ScoreOnWeb solution. We will also make recommendations on appropriate test forms, discuss throughput requirements, recommend appropriate scanning equipment, etc.
Forms Design
The appropriated design of the forms in the test booklet ensures that both the OMR bubbles as the images of handwritten responses can be properly scanned by the scanning software. The adjusting and redesigning exam forms is part of the advisory work carried out as part of the operation.
Forms Printing
The paper quality, sheet cutting and the inks used in exams' printing ensures that the scanning operation is conducted safely and smoothly. Our print partners produce the forms with the highest levels of quality, using only approved materials, and meeting all the requirements for a successful operation.
Packing and Delivery
The distribution of exams is a major logistical problem for testing organizations. The design, scanning and document loading used by ScoreOnWeb allows capturing personal information entered by students at the time of examination, or customize each booklet with bar codes and pre-loaded data, simplifying the process and eliminating the possibility of errors. The consulting services provided by our team to improve the distribution of booklets allows efficient and economical management of resources, facilitating the operation.
Tests Application
When the time of the examination, an official of the institution should take care to keep the process orderly. As part of the service, our team provides advice on how to prepare batches of tests for each course in order to send them to the scanning facility. It is important that each lot is clearly identified and packaged for shipment, and properly documenting the contents and destination of each packet.
Booklet Scanning
The scanning of exam booklets can be performed centralized in only one site, or distributed in several capture and digitalization centers. The digitized data are entered into ScoreOnWeb maintaining the relationship between the student data and the exam, allowing the subsequent preparation of segmented results reports. ScoreOnWeb technology allows the scorers can begin to grade answers as soon as the images begin to be loaded into the system, without waiting the finalization of loading process:
Objective items scoring
Closed responses items, with multiple-choice bubbles, are scored durinf the scanning process. This task is accomplished by optical mark recognition (OMR), using the true OMR readers of Scantron scanners. As soon as each test is scanned and entered into the system, the results of closed answers are already available in the reports.
Subjective items scoring
The scoring of open-response items is done by scorers, accesing the students' handwritten responses on the screen of their computers, either connected to the institutiion LAN or accessing the Internet. Each scorer, together with response image, has on its screen the possible scores for each answer, aids to the interpretation of responses, guidance on the question and the possibility to check online with his supervisor ScoreOnWeb has many tools that facilitate the scoring task, and strict controls that ensure that the grade given is secure.
Eliminates expenses and logistical problems descentralizing people
Because the system is Web-based, Scorers, Supervisors, and Administrators can be physically located in any place that has Web access. This eliminates the expense and logistical problems with bringing Scorers to a centralized location.
Reporting
ScoreOnWeb delivers real-time reports on the result of each exam, and management reports on the progress of the operation. Reports of exams include the customized for each student, and statistical reports on the total tests, segmented in the way the client requires. The management reports show the development of tasks, and allow identifying bottlenecks in the development of the operation.
Technical aspects
ScoreOnWeb System Diagram
The heart of the ScoreOnWeb solution is a highly customizable software application, accessed through Internet browsers. This program was created specifically for testing organizations, to automate the classification of handwritten open response questions, maintaining the accuracy of OMR optical mark rating for closed-response questions. Except the scanning of the documents, all aspects of the system are controlled using standard Internet browsers.
ScoreOnWeb Roles
ScoreOnWeb operation is made by different users, according to associated roles. Administrator Has all needed attributes to manage the system, defines users and roles, has access to productivity and final result reports. Pedagogic Director Is the pedagogic authority. Defines scoring rubrics, creates validation (anchor) responses and supervises scorers work. Supervisor Supervises scorers work related with one subject and level, answer scorer's questions and gives pedagogic support. Grader or Scorer Scores opend-ended items related with assigned subject and level. Configurer makes initial system configuration, adjusts parameter according to other users needs, scan and load exam images into the system.
Hardware Components
The Scan Stations are P.C.’s connected to Scantron™ OMR / Imaging scanners. These stations run our ScanTools™ software that defines how data is to be collected from the forms. Depending on the customer requirements, scanning operations can be centralized in one location or distributed over an unlimited number of locations.
ScoreOnWeb Server
The ScoreOnWeb server in a web server running Windows™ Server operating system, with Internet Information Services (IIS) installed. Al system data are stored in a MS SQL Server database.
Administrators, Supervisors and Scorers workstations
The workstation in any PC, only requirements are LAN connection and any internet browser (Firefox, Internet Explorer, Chrome, etc)