The JTAP Virtual Seminars Project:
User Evaluation Report
Background to the Survey
In the academic year 1997/8, Sarah Porter, of the Humanities Computing Unit at Oxford University, conducted a survey into the usage of the JTAP Virtual Seminars. (For the case study on that resource, click here), The survey asked detailed questions of seven teachers and lecturers who had utilised the Virtual Seminars in courses they had taught, and it also asked for some general observations on the use of IT in teaching. The purpose of the survey was not to find if the technology worked per se but how the use of IT affected the process of teaching and learning. The results of Ms. Porter's survey have been collated here, and comments from the completed questionnaires are included. In particular, this includes the comments of Colleen McKenna, who explained at length how she incorporated the resource into the module she taught to students at the University of Kingston.
The survey featured eight separate classes that had used the Virtual Seminars. Teachers who employed the resource came from a variety of educational environments - some were teaching first year undergraduates at British universities, others were teaching final year students in American high schools. While the majority of these lecturers were teaching literature, the resource was also applied to history classes, as well as to a Danish class who were studying English as a foreign language. Ms. McKenna notes that within her own group, the make-up of students was very varied - there was a balance between those who primary subject was English Literature, and those who were doing the module as an outside course; some were new to IT, some were experienced. There was also a wide age span - the class incorporated mature as well as younger students. The eight classes had a combined total of almost 500 students, although of course, as informal feedback to the site suggests, the total number of students who have utilised the site is much higher.
Integrating the Virtual Seminars into Coursework
Each of the lecturers who responded had made some prior use of IT resources, and generally had very favourable opinions on the integration of digital technology into courses they taught. All interviewees believed that using electronic teaching facilities had a positive effect on the quality of teaching and on the breadth and depth of material (whether primary or secondary) available to students. "In particular," one lecturer noted, "the audio and visual opportunities of the medium have been very valuable."
These general trends were mirrored when the lecturerers responded to questions on the utilisation of the Virtual Seminars. The consensus was that using the resource offered more fruitful methods of teaching World War One poetry. 
Lecturers did this in a number of ways. For example, some lecturers employed the resource in an occasional group tutorial held in the computing laboratory. Ms. McKenna's seminar group, taking part in a term-long module entitled Understanding Poetry, consisted of 25 students. (This was out of a total of 140 students taking the course as a whole.)Presenting the students with text worksheets at the beginning of the tutorial, Ms. McKenna helped the class while they worked through Tutorial III of the Virtual Seminars (this is the seminar analysing different manuscript versions of Wilfred Owen's Dulce et Decorum Est). Ms. McKenna was quick to point out that the rest of her seminars took place outside the computing lab; she wanted to maintain the traditional round-table seminars along with the one aided by computer technology.
All the lecturers recommended the Virtual Seminars as part of the course bibliography. The resource was not viewed as a stand-alone device, but a source to be used in conjunction with more traditional primary and secondary evidence - monographs, journals, works of fiction, etc. Ms. McKenna discovered that many of her students used the Virtual Seminars for background historical and bibliographical information. Another lecturer, Duco van Oostrum, teaching a course entitled Modern Literature at the University of Sheffield, found that "it was mostly used as a self-study device … About half the students had used the resources as a way to prepare for the next [week's seminar]." A small proportion of those interviewed also gave students the opportunities to take assessments based on the Seminars. In the case of Laura Bedingfield, teaching in Atlanta, Georgia, she gave her students assignments that revolved around the evidence they discovered on the website. Ms. McKenna gave students the option of working through Tutorial III, creating their own version of Dulce et Decorum Est. She then asked them to justify their decisions, comparing their copy to the available printed version.
Thematic Integration
The variety of students and courses that responded to the survey gives a good demonstration of how the flexible the Virtual Seminars are. The four different tutorials and the path creation scheme allow the teacher to tailor the resource so it can work in connection with other aspects of her course. For example, Ms. McKenna's Understanding Poetry module was intended not as an introduction to the poetry of Wilfred Owen, but as a general course sharpening the student's critical faculties for the appreciation and understanding of poetry. Tutorial III slotted well into this, precisely because it encourages the student to elaborate upon why one version of the poem may be more 'aesthetically effective' than another. Another lecturer, from Austin College in Texas, merged the Virtual Seminars into her War Literature course; for her the most important tutorials were the first and second, giving an introduction to World War One poetry, and also giving an example for study in Rosenberg's Break of Day in the Trenches. For Stephen Catchpole, teaching students at a Danish college, the Virtual Seminars could help an understanding of contemporary Europe. He employed the resource "to try and ignite an interest in reading English and an historical understanding of European history up to our present European Union."
Along with its flexibility, lecturers commented that the success of the Virtual Seminars depended upon its "clarity, ease-of-use and intellectual content", to cite one source. If such digital resources were to be successful as long-term teaching tools, the general feeling was, they had to have a clear interface, and have high-quality content, equalling the rigourous standards of traditional academic resources.
Improving the Learning Process
Ms. McKenna's belief that the Virtual Seminars provided students with a more complete learning experience was a view shared by others that took part in the survey. The width and depth of information provided a resource which could often be just as, or even more, effective than traditional resources.Laura Bedingfield commented that "the students were able to dig through the information that interested them in particular - I think they appreciated the aspect of taking control of their learning more than anything else." 
Ms. McKenna was especially pleased with the revised conception of poetry that students took away after having used the Virtual Seminars. Analysing the different manuscripts of Dulce et Decorum Est changed the poem from being an abstract entity and an object of study to being a powerful mode of expression with an obvious human dimension. One student noted that it was "fascinating to see Owen's handwriting, [it] made me feel closer to the poet to see what he had been through in writing the poem." This was, to quote Ms. McKenna, a "much more real experience" of studying poetry. Ms. McKenna added that many in her seminar group had "no concept of the editing process" or even the notion that more than one version of a poem could exist. The tutorial facilitated her teaching task in breaking down traditional stereotypes about artistic creation.
Many lecturers found that incorporating IT into their courses facilitated a more democratic process of teaching. Ms. McKenna established a website and an electronic mailing list for her module. On the website she placed some guidelines for the course, lecture notes and a few links to useful resources. Another lecturer noted that the general push of using IT inspired students to find additional material via the web: "Students sometimes find material that is stunning and original. There can be more creative types of learning experiences." Ms. McKenna found that even though she had prescribed Tutorial III for her seminar, students from the module used other parts of the resource to discover contextual material for their work. Others that took part in the survey noted this trend. Students repeatedly used the biographical sections to secure their background knowledge - perhaps because they could not access printed books with such information. Ms. McKenna also found that, as a general pattern, students were increasingly citing other Internet resources in their bibliographies.
Many of those who responded in the survey were appreciative of the extra material that the Virtual Seminars provided, material that was not typically available to the undergraduate student. 
One lecturer mentioned that "in cases where real archival material is present and where it's presented in an interactive manner, students are able to do research at a far higher level than they would in our library." Also noted was the increased enthusiasm many students took from an interactive resource. Without having to sacrifice intellectual standards, lecturers found it easier to engage students in the subject they were teaching.
As with the lecturers, feedback from the students was positive. Every student interviewed either found the use of the resource to be "very positive" or "quite positive." Many students thought that learning online was a more enjoyable experience than traditional learning. The bulletin board was particularly well received, allowing students to continue their ideas after classes had ended, to gain help from outside their own institutions, and also to explore themes related to the academic questions in the seminar room. "[I used the bulletin board] out of interest in less formal reactions to World War One experiences," one commented. "It is wonderful," another surmised, "to have so much information at your fingertips."
The Problem of Preparation
The major drawback identified in the survey was the additional time necessary to incorporate this new style of resource into the classroom. One lecturer from the United States noted that "it takes a long time to locate sites, check them through for content (appropriate or not), work up an assignment etc." However, many teachers reckoned that this problem would diminish in subsequent years. Once the IT had been incorporated into a course in the first instance there would be fewer difficulties in repeating the process the following year. Others felt that digital resources proved to be more time-effective even when preparing courses for the first time. Dr. van Oostrum commented "Whenever I need a picture to enliven my lectures, I do search on the net, and I invariably find an acceptable picture within 20 minutes." The odd worry concerning focussing student interest was also raised - there is the awareness that there can be an excess of information on the World Wide Web. Ms. McKenna noted that she had to arrange particular goals for the students so they knew where to extract the most relevant information from the Virtual Seminars site.
Training of Students in IT
Ms. McKenna organised two IT training sessions at the beginning of her module. She felt this was a necessary task, although it is becoming less important as more students arrive at class equipped with IT skills. The training sessions guided the students through the basic operations of handling a computer and exploring the World Wide Web - a text worksheet was handed out to give added guidance. There were a few worries that the use of new technology would prejudice older students, but these were quickly dispelled. In fact, Ms. McKenna commented that "mature students were by far the keenest." When students did have difficulties in the computing laboratory, others were quick to provide help. A few others identified teething troubles, predominantly with students that were "a bit hesitant because they feel they do not have the necessary computer skills." These, however, were in a minority. Dr. van Oostrum's students were keen to use the Virtual Seminars precisely because it did teach them IT skills: "Most students ask for IT skills that could be applied to their area of study. They find it makes them more suitable for the current job market."
Organising computer terminals for the students proffered only minor problems. An entire lab of 25 computers was failing to work when Ms. McKenna first tried to run the seminar, but her university was well stocked with other computer rooms they could relocate to. On subsequent occasions labs for teaching small groups were readily available. In general, Ms. McKenna said "computer facilities and provision are improving all the time," and any difficulty that lecturers are sustaining at the moment should prove to be only a "short-term obstacle." This final comment is perhaps representative of the use of digital resources in teaching and learning as a whole - the minor problems are outweighed by the wider scope that resources such as the Virtual Seminars introduce.
It is also possible to read a case study on the development of this resource.
This evaluation was based on the survey conducted by Sarah Porter,
the detailed comments of Colleen McKenna, and was written by Alastair Duning.
Additional information was given by Stuart Lee.
