Assessment
PowerPoint
Presentation
Sunil
Hazari, Ed.D
Associate Professor
Department of Management
Richards College of Business
Carrollton, GA 30118
Introduction
Testing and assessment remain an integral part of instructional systems design for traditional classroom based courses as well as web based training courses. The goal of testing is to determine if learning objectives have been accomplished. Formative evaluation using online testing can be used to help students assess their level of knowledge of course material. In addition it gives the instructor a better idea of what students are understanding as well as the concepts that still need clarification.
Although there are commercial web authoring tools available that integrate testing, many are proprietary in nature and not available to instructors and trainers who would like to integrate only the testing component without using entire suite of embedded tools of commercial programs. Two methods that can be used for online testing will be the focus of this chapter.
Online testing can be broadly categorized according to their use of either client side or server side processing. For client side method, JavaScript is a standard scripting language commonly used today that continues to be very popular since it works with all browsers with a graphical user interface. As a result it is attractive to most trainers and instructors who are interested in integrating testing in training courses delivered on the Web. Server side method on the other hand relies on a back end machine to process test results and return the output in HTML format to the user. A non-proprietary technology called Common Gateway Interface (CGI) is used to implement server based assessment.
A comparison of these two methods will be conducted giving advantages, disadvantages, and examples. While this manuscript will not teach programming language, it will give readers an insight into the work that needs to be done to incorporate interactivity and online testing for web based training material. Supplementary materials including an extensive annotated resource list with demos and working models is made available to readers from the author's web site located at http://sunil.umd.edu
With exponential growth of Internet, Intranet, and Extranet, Web Based Training has offered tremendous opportunities for training of employees and students in corporations and educational institutions by using the Web as a training medium. Advantages such as flexibility, convenience, updated material delivery, cross-platform compatibility, world wide distribution has led to many new courses being designed for the web, or existing courses being modified for Web based delivery. The first generation of Web based courses were 'static' in nature. Most were a result of paper based material scanned into electronic format and uploaded to the Web server. Information in these courses was only presented to be browsed without the user being actively involved with changing and manipulating information online. Although effective web-based educational systems must provide basic instructional functionalities (Overbaugh, 1994), one of the challenges for Web Based Training courses remains the use of Interactivity within courses. Interactivity defined here within the context of Web Based Training can be said to be a process where the learner is part of an active system that supports learning by encouraging user participation. Interactivity can be built in Web based training by the use of discussion groups, bulletin boards, synchronous communication tools such as chat forums, quiz, and evaluation/feedback instruments.
In traditional courses, once learning objective to be accomplished have been identified, there usually is an assessment process to measure the degree to which learners have acquired knowledge and can perform skills as required by the objectives (Kemp, 1985). Gagné & Briggs (1979) stress the need for evaluating student's understanding, providing feedback during evaluation, and assessing complete understanding of each concept as part of effective learning process. Web based training courses that follow models of traditional course instructional design should also have a component for evaluation, assessment, or testing. The purpose of this component is to determine if learning objectives have been met from Web based learning. Most cognitive-domain objective in Web based training can be evaluated by tests that can be delivered on-line. Objective type tests such as multiple choice, true or false, fill-in-the-blanks can be implemented on the Web. Feedback can also be built in so users can check answers. The role of these tests can not only be for formal assessment but also serve an important part if used for student self-evaluation. Depending on the role, this self-evaluation can help the student (or instructor) monitor progress, recognize difficulties, or serve as review material to better ensure learner preparation for alternative formal assessment instrument. In addition, online tests also offer an option of delivering multimedia content such as audio and video clips that can be directly tied to learning outcomes. This may not be possible when paper based traditional testing methods are used. Examples of some subject areas that can benefit from multimedia enabled online tests are Language Learning and Music Education where audio clips can be used to test pronunciation of foreign languages words or music notes and pitch. Similarly video clips can be used to provide instruction as well as assessment of topics such as non verbal communication, posture, hand position, slide position, and proper breathing.
Ideally, commercial full fledged Web course development programs include course administration and assessment tools that integrate student registration with password authentication, question delivery in various formats, randomization of question items, real time grading, statistical analysis of responses, and storage of results in a standard database format that can be accessed by the instructor at a later date. User centered learning systems use some or all of these components within web based courses to address one of the needs (i.e. testing) for learner engagement. But issues such as viability of applying these tools also have to be considered (Norman & Spohrer, 1996). To trainers and instructors not using proprietary commercial tools, assessment can be built in to web based training courses by using two methods: either a scripting language such as JavaScript in which processing is done on the client side, or another technique that uses Common Gateway Interface (CGI) technology on the server side.
JavaScript is an interpreted scripting language (similar to dBase or HyperCard program languages) that is embedded with HTML code to develop interactive pages. It is supported by popular browsers such as Netscape 2.0 and higher or Microsoft Internet Explorer 3.0 and higher. JavaScript is completely different from 'Java' which is an object oriented programming language that is used to create applets. JavaScript is also different from Java since it works with HTML without having to open other helper applications. Once the HTML/JavaScript code is downloaded from the server, all processing is done on the client side. This saves bandwidth since all work is done locally. A typical example where this could be useful is when many students are required to access the page and be assessed individually at the same time.
General format for embedding JavaScript code with HTML is shown in Figure 1.
| <html> <body> <br> Sample HTML document <br> <script language="JavaScript"> document.write("Web Based Training Assessment") </script> <br> Back to HTML text here. </body> </html> |
Figure 1: JavaScript coding style
Like other programming languages,
JavaScript uses identifiers, literals, variables, functions, objects, properties, and
arrays. Although detailed explanation of each feature is beyond the scope of this chapter,
the reader is referred to online documentation and frequently asked questions on
JavaScript available on the Internet. Figure 2 shows a brief example of JavaScript syntax
code that is used to check for a correct answer.
| function checkAnswer () { var answer = document.forms.field1.value; if (answer == "Correct Answer") { alert ("Well done!"); } else |
Figure 2: Sample JavaScript code for checking answers.
Multiple choice, Fill-in-the-blank, True/False type questions can be developed using JavaScript functions. Another advantage of JavaScript is that form validation can be built in which allows immediate feedback to be given to the students based on their response. Aronson and Briggs (1983) identify feedback as a vital and indispensable instructional activity in the learning process. Using the sample code shown above, incorrect answer would pop up another small browser window informing the student that the response was incorrect and providing hints to guide student to the correct response. It is also possible to integrate features that limit the number of attempts on each question, keep track of elapsed time, and calculate total score for the entire test. One main disadvantage of using JavaScript is that the code cannot be hidden completely. A user could view source from one of the browser menus and find answers in the source code. However this can be overcome by using advanced coding techniques such as 'cookies' that hide answers from users.
Although JavaScript processes information on the client side, most of the processing (such as checking correct answers) takes place on the users browser. For self assessment, the student gets to see what questions were marked correct or incorrect. Depending on the purpose of assessment, there may have to be grades that are recorded by the instructor or administrator software. This can be achieved by using the 'Forms' feature of HTML. After the test has been completed, the user clicks a 'Submit' button that sends results of the quiz to an e-mail address. (For those familiar with HTML, this action is achieved by using the HTML code similar to:
<form METHOD="POST"
ACTION="mailto:shazari@glue.umd.edu">
placed at the top of the form. In this case, when the 'Submit' button is pressed, the form
is mailed to shazari@glue.umd.edu e-mail
address. Using this data and a form processing script, standard reports can be generated
by database software such as Access or Oracle. Items contained in reports may include
student data, date and time test taken and/or completed, and scored results. These reports
can also be used to perform item analysis and create charts or graphics showing student
performance and comparing results between groups based on demographic data.
Picking up a new programming language is difficult even for experienced programmers. Fortunately, there are many pre-designed JavaScript templates that are made freely available for instructors/trainers to use and customize. A list of these programs is available from the author's web site (http://sunil.umd.edu). For more sophisticated use of assessment techniques that rival those built into high-end web authoring programs, a technique called CGI is used.
Common Gateway Interface (CGI)
CGI is a server-based method that can be used to implement online testing. A student would connect to the Quiz page and fill out a HTML form that may ask for student name and identification number. A series of questions (multiple choice, true or false, fill in the blanks) are presented to the student by the browser and using mouse clicks or keystrokes the student enters answers to questions displayed on the screen. When all questions have been answered the test is submitted for 'grading'. Results of the quiz are then displayed on student screen, mailed to the instructor and/or stored in a file on the server for instructor records. These apparently simple procedures on the user's side hide the complex sequence of database queries on the server side to generate a complete graded test. The sequence is shown in Figure 3:
Figure 3: CGI script execution sequence
- Student connects to the web server using browser such as Netscape Navigator or Microsoft Internet Explorer
- Server presents form data to student’s browser
- User selects answers from form items and submits data to server
- Server performs the following actions:
- Data is parsed into variables
- Query is generated by gateway script
- Query is submitted to database
- Database returns dataset items that match query
- Dataset is formatted to HyperText Markup Language (HTML)
- Server sends formatted output to client
- Client displays result page on buyer’s browser.
In most cases CGI scripts are written using the Perl scripting language because Perl is a common programming language, fairly easy to learn, modify, and is portable across operating systems. Figure 4 shows sample HTML form script with built-in directives for Perl interpreter. Figure 5 demonstrates output of sample form that can be completed by a user and submitted for processing. If setup correctly, almost all Web servers in the market today understand CGI interface. For instructors and trainers interested in implementing testing using CGI, a freeware Multiple Choice Grader is available that can be downloaded and installed without having to write any code (see http://www.extropia.com). Only the questions and answers need to be changed by using a text editor. Installation requires access to a cgi-bin directory that is setup by a system administrator specifically to run CGI code.
| #!/usr/local/bin/perl # This is a cgi generated HTML submission form print "Content-type: text/html\n\n"; Your Fullname<BR> <DD><INPUT TYPE = "radio" NAME =
"1" VALUE = "Fred">Fred<BR> <DL> <DL> <CENTER><INPUT TYPE = "submit" VALUE
= "Submit your answers"></CENTER> |
Figure 4: Sample HTML/Perl Script
Figure 5: Output of
HTML/Perl Script to be submitted to server for processing
Although CGI is very powerful and robust means of implementing online evaluation, there are some disadvantages to using CGI. Many system administrators do not permit use of CGI scripts on their web servers because it means allowing users with Web browsers to submit queries to a program that actually executes on the server (Dwight & Erwin, 1996). If not setup correctly this has the potential of allowing malicious code to be run in server space. It is imperative that security on files be checked and re-checked before allowing CGI access. Since all processing takes place on the server side, high volume of queries (example, an entire class of students taking the test at one time) puts an extra load on the server that may slow other processes running on the Web server.
Use of both these testing methods can be seen in a sample course EDIT 530: Scripting Languages in Authoring Educational Material (available from http://sunil.umd.edu). In this course, students are required to develop computer-based educational materials using widely known educational scripting languages. Course requirements also called for students to explore basic authoring capabilities and learn and apply those capabilities by designing and producing material using commands, procedures, and functions of scripting language.
Since this was primarily a skills based
course, majority of course grade was assigned to projects and products created by students
individually or in groups. However, survey of authoring tools, programs, scripts, systems
and topics such as multimedia video, sound formats, graphical user interface design,
flowcharting, storyboards, instructional design process were also studied. Students were
tested on these topics by using online assessment instead of traditional paper based
assessment. There were two types of testing used: one for student progress intended solely
for reinforcing student concepts (no grade assigned), and the other where grade assigned
on online tests became part of the course grade. In this case because of better reporting
capabilities, the CGI script was used for recording grades and the JavaScript program for
self-monitoring by students.
As previously shown in Figure 5, tests could be presented to the student who would select answers and submit the form to the server for processing. For the instructor there are various options available to collect data of for scored tests. This is shown in Figure 6.
Figure 6: Reporting capabilities available to instructor
With data collected and stored on the server, further analysis can be done on the tests by downloading it to a spreadsheet using tab delimited or comma separated values that can be imported in any desktop spreadsheet or database application for viewing individual data online as shown in Figure 7.
Figure 7: Individual tabulated results for item analysis
The table in Figure 7 shows name of the student taking the test, date and time the test was taken, the IP number from which the tests was taken (some programs allow setting restrictions based on time, date, and IP# to limit test taking), time taken by the student to complete the test (in seconds) and how each question was answered. In addition, the program also calculates the score and percentage of correct answers. This type of reported data provides the instructor additional options such as calculating mean scores, standard deviation, determining internal consistency, distribution, and item analysis for each question if required.
Web Based Training when used as an instructional tool must have some means of assessing the fact that learning is taking place. For interactive sites there are many different methods of integrating web based testing and evaluation as part of the online learning process. JavaScript and CGI are two of the most effective and easily implemented options available. For the instructional designer or trainer, final decision on which tool to use will depend on variables such as learning outcomes to be measured, type of server available and its capabilities, degree of complexity required, and instructor, or design team programming expertise. Since CGI relies on server processing it is more secure but at the same time large number of hits may cause the server to slow down. Implementing CGI requires knowledge of setting server directory permissions and user rights. If not setup correctly, there is potential for damage to files since programs actually execute on the server. JavaScript compared to CGI places a much lighter load on the server since all code is downloaded to the client machine and no other server process is required to actually run the program. This type of processing exemplifies the client-server model where the client has the intelligence to process tasks. However if client side scripts need to be submitted for recording results, a link between the client and server has to be re-established for student data to be entered in the database and another script written to process information stored on the server. In general, trainers and instructors should consider client side JavaScript based assessment programs when local data validation, individual browser control (such as hiding browser menus), advanced functionality on client side, and high degree of interactivity is required. For applications where source code needs to be hidden, user navigation closely tracked, server side database updated with user data, use of CGI server based programs would be more appropriate.
Both these methods require basic to
intermediate knowledge of some scripting language.
For trainers who are just starting to implement interactivity and on-line assessment in
Web Based Training, use of existing scripts that are freely available in the public domain
should not be overlooked.
Aronson, D. T. & Briggs, L. J. (1983). Contributions of Gagné and Briggs to a prescriptive model of instruction. In C. Reigeluth (ed.), Instructional design theories and models: An overview of their current status. NJ: Prentice-Hall.
Dwight, J. & Erwin, M (1996). Special edition using CGI. Indiana: Que.
Gagné, R. M. & Briggs, L. J. (1979). Principles of instructional design, (2nd ed.). New York: Holt, Rinehart and Winston.
Kemp, J. E. (1985). The instructional design process. NY: Harper & Row.
Norman, D. A. & Spohrer, J. C. (1996). Learner centered education. Communications of ACM. 39 (4), 24-27.
Overbaugh, R. C. (1994). Research-Based guidelines for computer-based instruction development. Journal of Research on Computing in Education, 27(1), 29-47.
Selena Sol Multiple Choice Grader (http://www.extropia.com).
Dr. Sunil Hazari is Campus Computing Faculty Research Associate in the Robert H. Smith School of Business at University of Maryland. He was formerly Director of Technology Training in the Graduate School of Education at George Mason University. His research interests are in the areas of Technology Training, Web Based Instruction, Local Area Networking, and Multimedia Applications.
E-mail: shazari@glue.umd.edu