OWL at Purdue Logo

This page is brought to you by the OWL at Purdue (https://owl.english.purdue.edu/). When printing this page, you must include the entire legal notice at bottom.

Contributors:Jingfang Ren, Dana Lynn Driscoll, Elizabeth Angeli.
Summary:

This is a teaching guide for professional writing instructors who are teaching their students to write usability reports. It includes teaching tips, suggested deliverables and class activities as well as a list of resources. Although this guide is created primarily for English 420 (Business Writing) and English 421 (Technical Writing) instructors at Purdue University who assign usability report as part of the OSDDP (Open Source Development and Documentation Project) initiated at Purdue, you may find many of the tips and resources useful and ready to be adapted to your own classroom context.

Summary and Description of the Project

What Is OSDDP?

Implemented by the professional writing program at Purdue in the fall of 2004 as a new approach to networked learning, collaborative education, and public writing, OSDDP (the Open Source Development and Documentation Project) has produced many success stories among students and instructors in various business and technical writing classes at Purdue. However, teaching the OSDDP project also presents many challenges to instructors of professional writing, especially new instructors, as it requires a thorough understanding of Open Source products and the concepts and philosophy underlying these products as well as genre knowledge about the deliverables that students are expected to produce throughout the project. This guide introduces new PW instructors to one particular OSDDP project, the OSDDP usability project and provides tips and resources that aid teaching.

What Does the OSDDP Usability Project Entail?

For the OSDDP Usability Project, students are expected to complete two main tasks: 1) conduct usability testing of an open source application (software, content management system etc.) or documentation (tutorial or user manual); 2) write up the test results in the form of a usability report and present it to the open source community by posting it to Purdue's OSDDP site under creative commons.

Why Assign the OSDDP Usability Project?

This project provides students with an opportunity to 1) learn more about how users/customers in real-world situations interact with commercial and non-commercial products (in this case, software applications and manuals); 2) use usability testing as a research method to generate raw data about such interactions; 3) practice public writing or writing that is subject to public reviews and revisions.

Although this project involves some individual elements, the majority of the project is the result of collaboration. This project involves approximately 6-7 weeks of work with each week typically consisting of two 75-minute class meetings or three 50-minute class meetings.

What Can This Teaching Guide Do for Me?

This teaching guide includes a brief summary of the project, suggested group and individual deliverables as well as evaluation criteria, week-by-week suggested activities with annotations, resources for instructors, and sample deliverables produced by business writing students at Purdue.

Contributors:Jingfang Ren, Dana Lynn Driscoll, Elizabeth Angeli.
Summary:

This is a teaching guide for professional writing instructors who are teaching their students to write usability reports. It includes teaching tips, suggested deliverables and class activities as well as a list of resources. Although this guide is created primarily for English 420 (Business Writing) and English 421 (Technical Writing) instructors at Purdue University who assign usability report as part of the OSDDP (Open Source Development and Documentation Project) initiated at Purdue, you may find many of the tips and resources useful and ready to be adapted to your own classroom context.

Suggested Deliverables and Evaluation Criteria

For this project, students are expected to produce 7 group deliverables and 4 individual deliverables.

Group Deliverables

Project proposal

Description: This one-to-two page single-spaced memo proposes an OSDDP project that fits with the level of expertise of the usability research team in doing primary research as well as their interest in a particular open source product (software application or user manual/tutorial). The proposal should include, but not be limited, to the following elements: description of the product to be tested by the student researchers, rationale underlying their choice of this particular product, qualifications of the students as usability researchers, as well as potential benefits of the project for the developers of the product, target users of the product, and the team of student researchers themselves.

Evaluation:The proposal needs to be both informative and persuasive. It should convince the instructor that what is being proposed is both worthwhile and manageable in a 6 or 7-week time frame.

Example: See page 1 of the pdf document attached in the "media" box.

Team project plan with Gantt chart

Description: The team plan should answer several important questions that the team needs to consider at early stages of the project, such as who is the typical user of the product to be tested, what are some of the characteristics of the "typical" or "average" user, what does the typical user use the product for, what methods will be used to recruit and select users, what problems the student researchers anticipate to occur before, during, and/or after the test sessions, what role(s) each team member plays and so forth. The Gantt chart is a visual way of presenting the timeline by mapping out major tasks to be completed vertically and the expected length of time it takes to complete each task horizontally.

Evaluation: The group plan should demonstrate signs of careful thinking into the whole process that the team will go through from selecting the product to be tested to submitting final draft of the usability report by thoroughly answering questions mentioned above. The Gantt chart should represent the timeline clearly and accurately.

Example: See pages 2-3 of the pdf document attached in the "media" box.

Periodic oral progress report

Description: This is an informal oral report on what the team has achieved in a particular week and what needs to be done in the following week. It's an opportunity for each team to share their work-in-progress with other teams and get feedback and suggestions.

Evaluation: The report should be informative and concise. It should update other teams on both the products/deliverables that the team has produced AND the process the team is going through.

Usability testing materials

Description: The usability testing materials package should include pre-test background survey, observational data sheet to record user reactions to and interactions with the product, and follow-up survey or interview questions.

Evaluation: While testing materials differ significantly depending on what is to be tested, well-developed testing materials packages share several things in common. They should reflect clear and well-articulated testing objectives as well as maintain a good balance between good planning and room for flexibility. They maximize both validity and reliability of the tests.

Example: See pages 14-17 of the pdf document attached in the "media" box.

Usability testing

Description: Each team should select a reasonable number of users as participants in the tests. (For a 3-person team, 6-9 users would be good enough. More than 9 may not be manageable considering the time constraints.) Each team should try to select users from a variety of backgrounds, taking into consideration such factors as gender, race, education, prior knowledge about the product, level of expertise in the product and so forth. In addition, to help enhance the reliability of test results, users should not have a previous connection with any of the team members. Each test session should last approximately 45-60 minutes.

Evaluation: The test sessions will not be evaluated on their own as quality of the tests will be reflected in the report itself, especially in the analysis, findings, and discussion sections of the report.

Usability report

Description: The report should provide adequate information on the background of the product tested, the methods used (user profiles, researcher roles, testing facility, data collection and analysis methods and procedures) as well as thorough discussion of test results (key findings and recommendations). Encourage your students to follow the CIF (Common Industry Format) guidelines developed by NIST (National Institute of Standards and Testing) and adapt these guidelines for their own purposes. Click here for a sample handout on the structure of a usability report based on the CIF guidelines.

Evaluation: The report should allow the readers to see not only the results of the test sessions but also the process that the usability researchers have gone through. The researchers should be able to identify patterns in the raw data they've collected by using appropriate analytic tools/frameworks and justifying their choice. They should interpret their findings in a fair manner, and the recommendations they make should match and be based on their findings.

Example: See pages 4-17 of the pdf document attached in the "media" box.

Oral presentation

Description: Each presentation should run 15-20 minutes. In addition, presenters should be prepared to answer questions from the audience. Each Q-A session lasts about 5-10 minutes. All presentations must include multimedia elements and be accompanied by a handout. Each team member will be expected to take part in the presentation.

Evaluation: Use the following categories to evaluate the oral presentations: content and organization, audience awareness, slide design and use of visuals, and delivery.

Individual Deliverables

Professional profile of skills and interests

Description: The profile should include the following information: name, major and minor, expertise/specialties (tie this to the tasks to be accomplished), products wanting to test, and contact information. The purpose of this assignment is to help students find out whom they are likely to work well with. This information can also be used later by the teams to assign specific roles to each individual member.

Evaluation: This step itself won't be graded but will be used to help form teams.

Project log

Description:It details each individual's contributions to the group project by describing what each individual has done in a particular week and the amount of time spent.

Evaluation: This step itself will not be graded, but will be used as one factor in your determination of each individual's participation grade for this project.

Project assessment

Description: This one-to-two page single-spaced reflective piece describes each individual's experience of doing primary research in the form of usability testing, analyzing data and reporting on key findings in a clear and logical manner.

Evaluation: While description of certain aspects of the project is necessary, the bulk of this document should go beyond description and should answer such questions as "what worked? what didn't work and why? what would I do differently next time, knowing what I know now?" and so forth.

Peer collaborative project evaluation document

Description: This document, accessible to the instructor only, describes each individual's contribution to the project in detail. It tells you who deserves more or less credit than others based on such descriptions. This document can be in paragraph or memo format; or you can use a form similar to the one being used by English 420 (Business Writing) instructors at Purdue.

Evaluation: This document itself won't be graded, but will be used as one factor in your determination of each individual student's participation grade for this project.

Contributors:Jingfang Ren, Dana Lynn Driscoll, Elizabeth Angeli.
Summary:

This is a teaching guide for professional writing instructors who are teaching their students to write usability reports. It includes teaching tips, suggested deliverables and class activities as well as a list of resources. Although this guide is created primarily for English 420 (Business Writing) and English 421 (Technical Writing) instructors at Purdue University who assign usability report as part of the OSDDP (Open Source Development and Documentation Project) initiated at Purdue, you may find many of the tips and resources useful and ready to be adapted to your own classroom context.

Week-by-Week Suggested Activities

A 7-week time frame is used here, with each week typically consisting of two 75-minute class meetings or three 50-minute class meetings. Feel free to adapt it to your own class schedule in any way you deem necessary.

Week 1: Introduce Project and Form Usability Teams

If you happen to have someone in your class with a programming background and knows open source from the developer's view point, invite him or her to give a mini-speech on the developer's side of the story.

Week 2: Select and Propose Product to Be Tested, Set up and Conduct Mock Tests

A good product to use for the mock test would be SmartDraw, which is a software program that helps users to create Gantt chart and other types of charts. It's simple enough to be mastered in a short period of time. Further, since one of the deliverables students are expected to produce for this project is a Gantt chart, having students draw up a Gantt chart using this particular software allows students to accomplish two tasks at once.

Week 3: Analyze Mock Tests and Develop Testing Materials

Week 4: Conduct Usability Tests and Analyze Data

Week 5: Draft Usability Reports and Present Data Visually

Week 6: Peer Review Draft, Conference, and Revise Draft

Week 7: Present, Reflect, and Evaluate

 

Contributors:Jingfang Ren, Dana Lynn Driscoll, Elizabeth Angeli.
Summary:

This is a teaching guide for professional writing instructors who are teaching their students to write usability reports. It includes teaching tips, suggested deliverables and class activities as well as a list of resources. Although this guide is created primarily for English 420 (Business Writing) and English 421 (Technical Writing) instructors at Purdue University who assign usability report as part of the OSDDP (Open Source Development and Documentation Project) initiated at Purdue, you may find many of the tips and resources useful and ready to be adapted to your own classroom context.

Resources for Instructors

Open Source and OSDDP at Purdue

Open Source Initiative: Open source definition, products, philosophy, and more.

Drupal: An open source content management platform, which can support a variety of websites ranging from personal weblogs to large community-driven websites.

Moodle: An open source course management system, designed to help educators create effective online learning communities.

GNU: An open source operating system, alternative to UNIX.

Wikipedia: An open source encyclopedia available in 10 different languages.

Open Office Suite: An open source multilingual and multi-platform suite, which is compatible with and can be used as an alternative to the Microsoft Office suite.

Mozilla Firefox and Mozilla Thunderbird: Mozilla Firefox is an open source alternative to IE and other commercial browsers. Mozilla Thunderbird is an open source alternative to Outlook and other commercial inbox organizers.

Open Source Development and Documentation Project at Purdue: OSDDP mission statement, press release, pedagogical rationale, sample projects, teaching tips, discussion forums.

Project Management Tips and Free Software

Project KickStart Website: Project management tips provided by consultants and trainers who helped develop Project KickStart, a project management software package.

All PM (Project Managers) Website: Specifically created for project managers in the workplace, has a large collection of materials ranging from glossary, articles and tips, white papers, to document templates, many of which can be adapted for classroom use. Also has a discussion forum where you can post questions about project management and get answers from project managers all over the country. Membership free.

MindTools' Guide to Gantt Chart: Offers a good conceptual explanation of Gantt chart as a project management tool.

SmartDraw Website: Free trial version of SmartDraw, a software program for creating Gantt chart and other charts, as well as Gantt chart definition/explanation and examples.

Usability Testing and Usability Report: Principles, Techniques, and Examples

Usable Information Technology Website: Primarily column articles, reports, and newsletters on usability by the website's creator, Jakob Nielsen, a former Sun Microsystems engineer and a well published usability researcher with expertise in website usability research, especially the heuristic evaluation method.

Information & Design Website: Tips on the design, analysis, evaluation stages of usability testing provided by Information & Design, a usability consultancy based in Melbourne, Australia. Sample testing materials also included.

CIF Guidelines — Common Industry Format for Usability Test: Guidelines for creating a usability report using the format, recommended sections in a usability report, checklist etc.

Design Technologies Website: A large collection of usability resources, including heuristics, methods, style guides, and links to other resources.

Microsoft Usability Website: Microsoft Usability Publications, including guidelines, techniques, and sample reports.

Dialog Design Usability Website: Standard report format and sample report, good in informal testing situations when testing needs to be done in a fast and easy way.

Gnome Usability: A good example of a formal report, when thoroughness is more important than efficiency.

Contributors:Jingfang Ren, Dana Lynn Driscoll, Elizabeth Angeli.
Summary:

This is a teaching guide for professional writing instructors who are teaching their students to write usability reports. It includes teaching tips, suggested deliverables and class activities as well as a list of resources. Although this guide is created primarily for English 420 (Business Writing) and English 421 (Technical Writing) instructors at Purdue University who assign usability report as part of the OSDDP (Open Source Development and Documentation Project) initiated at Purdue, you may find many of the tips and resources useful and ready to be adapted to your own classroom context.

Sample Project Deliverables

In this section, you will find some deliverables produced for the OSDDP usability project by a group of students enrolled in English 420 (Business Writing), with all the names taken out. You may use these as examples or a discussion-starter in your own classroom if usability report writing is a component in your curriculum.

These deliverables are included in one PDF document attached in the "media" box, and they are separated by page numbers.