HMI 503. Human Factors Research Methodology and Field Experimentation. 5 points.

Kip Smith
kipsm@ikp.liu.se
013-28 27 64

Human Factors is the science of understanding how and why we use artifacts the way we do. Artifacts are things that are designed and created by people for other people to use. In this course you will (1) learn methods that you can use to inform the design process and improve the likelihood that artifacts fit their users, (2) apply those methods, and (3) document your use of those methods.

The focus this term is on teams and on why and how we work with artifact the way we do when we are members of a team.

Methods under consideration will include:

Dual tasks
Microworlds
Naturalistic observation
Protocol analysis
Psychophysiologic monitoring
Simulation
Task analysis
Quasi-experimentation
Usability testing

Discussion

In this course, we focus on the situated behavior of teams that involves the use of artifacts. The term situated means that the behavior occurs within and is largely shaped by its environment, that is, by the ambient array of stimuli. Salient among stimuli are artifacts . Artifacts are the products of a design process. Artifacts are often tangible objects. Familiar intangible artifacts include software and web pages.

The degree to which behavior is shaped by its context, rather than by, e.g., preconception or by obedience to arbitrary authority, is called fit . Accordingly, in this course you will learn, use, and document your use of methods for understanding the fit between (1) teams performing tasks in environments filled with artifacts, (2) the tasks they perform, and (3) their artifacts.

There are at least three reasons why you should want to understand the fit between behavior, tasks, and artifacts. First, when the fit is poor, error is common and behavior can become pathologic (e.g. “fixing” a VCR with a sledge hammer). There are many factors governing the quality of the fit between an artifact and its user. This course is designed to help you know how to investigate the multitude of interacting factors shaping the quality of the fit.

Second, knowledge of when where why and how users actually use an artifact is critical to successful design. Understanding the interaction between an artifact and its user prior to reifying its design invariably improves the fit.

Objectives

When you complete this course, you should be able to:

• Apply a variety of methods for observing keenly and analyzing succinctly situated human behavior.
• Apply a variety of methods for testing hypotheses about situated human behavior.

Demonstrations of competence (grading)

To earn a passing grade in this course you will:

• Conduct a observational field study of a real work team of your choosing. The study will (a) describe the interaction of an environment (workspace), a (group of) user(s) (worker, agent, expert), and a task, (b) extract from that interaction an exemplar of (poor) fit between an artifact and its user(s) that demands explanation, and (c) generate multiple testable hypotheses for that explanation.
• Write a report in standard form about that study.
• Design a quasi-experimental study to test the hypotheses about the exemplar of fit.
• Make a 30 minute oral presentation covering the field study and the design of an appropriate quasi-experiment. The presentation should take advantage of presentation software, document the variety of methods used, make evident the utility of the results, and sustain audience interest.

Organization of the course

This course has four blocks of activity.

Block 1

Class sessions.

Weeks 9, 10, 12, 13:

March 3, 7, 10 (proposal due), 21, 24, 31.

Models of and methods for naturalistic observation and description of the situated behavior of teams. Introductions to protocol analysis, task analysis, and content analysis. QuALMRI rubric for the standard form of papers.

On March 10, students will hand in a written proposal and present a brief oral proposal covering the team, the work context, and the artifacts they propose to study in Block 2. This is an individual project.

Block 2

Observation project and paper. Weeks 11 & 16.

Observe, analyze, report. This is an individual project, not a group project. Written report in standard form due May 2.

Block 3

Weeks 14, 15, 17, 18, 19:

April 4, 7, 11, 14, 25, 28; May 2 (report due), 9, 12.

Block 4

Week 20: May 19.

Every student will make an oral presentation covering the observational study and the design of a quasi-experiment to test questions and hypotheses raised by the observational study. All students must attend all presentations and are expected to provide feedback to the presenters

http://www.ikp.liu.se/iav/Education/HMI503/index.asp

Materials

Two books to be purchased by the student

Simon, H. A. (1996). Sciences of the artificial , 3rd Edition. Cambridge , MA : MIT Press. ISBN 0- 262-69191-4

Dumas, J. and Redish, J. A (1999). Practical guide to usability testing , Revised edition. Where? : Intellect. ISBN: 1-84150-020-8.

Readings packet to be purchased by the student

The packet will include but will not be limited to:

Book chapters

Alderfer, C. P., & Sims, A. D. (2003). Diversity in organizations. In W. C. Borman, et al. (Eds.), Handbook of psychology: Vol. 12. Industrial and Organizational psychology . Hoboken , NJ : John Wiley & Sons.

Brehmer, B. (1991). Distributed decision making: Some notes on the literature. In J. Rasmussen & B. Brehmer & J. Leplat (Eds.), Distributed decision making: Cognitive models for cooperative work (pp. 3-14). Chichester : John Wiley & Sons Ltd.

Chrobot-Mason, D., & Quiñones, M. A. (2002). Training for a diverse workplace. In K. Kraiger (Ed.), Creating, implementing, and managing effective training and development . San Francisco , CA : Jossey-Bass.

Church, A. H., & Waclawski, J. (2001). Designing and using organizational surveys: A seven-step process . San Francisco , CA : Jossey-Bass.

Funke, J. (1993). Microworlds based on linear equation systems: A new approach to complex problem solving and experimental results. In G. Strube & K. F. Wender (Eds.), The Cognitive Psychology of Knowledge . Amsterdam : Elsevier.

Hutchins, E. (1995). Cognition in the wild . Chapter 1. Cambridge , MA : MIT Press.

Jackson, S. E., & Joshi, A. (2002). Research on domestic and international diversity in organizations: A merger the works? In Anderson, N. et al. (Eds.). Handbook of industrial, work & organizational psychology: Vol. 2. Organizational psychology . Thousand Oaks , CA : Sage Publications.

Klein, G. A., Orasanu, J., Calderwood, R., & Zsambok, C. E. (Eds.). (1993). Decision making in action: Models and methods . Selected chapters. Norwood , NJ : Ablex Publishing Corporation.

Langan-Fox, J. (2002). Communication in organizations: Speed, diversity, Networks, and influences on organizational effectiveness, human health, and relationships. In Anderson, N. et al. (Eds.). Handbook of industrial, work & organizational psychology: Vol. 2. Organizational psychology . Thousand Oaks , CA : Sage Publications.

Lewicki, R, J. & Bunker, B. B. (1996). Developing and maintaining trust in work relationships. In R. M. Kramer & T. R. Tyler (Eds.), Trust in organizations: Frontiers of theory and research (pp. 114-139). Thousand Oaks , CA : Sage.

Meyerson, D., Weick, K. E., & Kramer, R. M. (1996). Swift trust and temporary groups. In R. M. Kramer &, T. R. Tyler (Eds.), Trust in organizations: Frontiers of theory and research (pp. 166-195). Thousand Oaks , CA : Sage

Neisser, U. (1976). Cognition and Reality. Chapters 1-4. San Francisco , CA : Freeman.

Roth, E. M., Malin, J. T., & Schreckenghost, D. L. (1997). Paradigms for intelligent interface design. In M. Helander & T. K. Landauer & P. Prabhu (Eds.), Handbook of Human-Computer Interaction (2nd ed., pp. 1177-1201). Amsterdam : Elsevier Science.

Senge, P. M. (1990). System archetypes. The fifth discipline: The art and practice of the learning organization, Appendix 2 . New York : Doubleday.

Shneiderman, B. (1998). Designing the user interface : strategies for effective human-computer-interaction (3rd ed.). Selected chapters. Reading , Mass: Addison Wesley Longman.

Woods, D. D. (1995). Toward a theoretical base for representation design in the computer medium: Ecological perception and aiding human cognition. In J. Flach & P. Hancock & J. Caird & K. Vicente (Eds.), Global Perspectives on the Ecology of Human-Machine Systems (pp. 157-188). Hillsdale , NJ : Lawrence Erlbaum Associates, Publishers.

Journal articles

Adelson, M. (1961). Human decisions in command and control centers. Annals of the New York Academy of Science, 89, 726-731.

Brehmer, B. (1992). Dynamic decision making: Human control of complex systems. Acta Psychologica, 81 , 211-241.

Brehmer, B., & Dörner, D. (1993). Experiments with computer-simulated microworlds: Escaping both the narrow straits of the laboratory and the deep blue sea of the field study. Computers in Human Behavior, 9 (2-3), 171.

Eden, D. (1986). Team development: Quasi-experimental confirmation among combat companies. Group and organization studies, 11 (3), 133-146.

Funke, J. (2001). Dynamic systems as tools for analysing human judgement. Thinking and Reasoning, 7 (1), 69-89.

Gray, W. D. (2002). Simulated task environments: The role of high-fidelity simulations, scaled worlds, synthetic environments, and laboratory tasks in basic and applied cognitive research. Cognitive Science Quarterly, 2 , 205-227.

Hutchins, E. (1991). Organizing work by adaptation. Organization Science, 2 (1), 14-39.

Jarvenpaa, S. L. & Leidner, D. E. (1999). Communication and trust in global virtual teams. Organization Science, 10 (6), 791-815.

Kaempf, G. L., Klein, G., Throdsen, M., & Wolf, S. (1996). Decision making in complex command and control environments. Human Factors, 38 (2), 220-231.

McAllister, D. (1995). Affect- and cognition-based trust as foundations for interpersonal cooperation in organizations. Academy of Management Journal, 38 (1), 24-59.

Olson, G. & Olson, J. (1999). Distance matters. Human-Computer Interaction, 15 , 139-178.

Phatak, A. V., & Bekey, G. A. (1969). Decision processes in the adaptive behavior of human controllers. IEEE Transactions of Systems, Science and Cybernetics , SSC-5 , 339-352.

Smith, K., & Hancock, P. A. (1995). Situation awareness is adaptive, externally-directed consciousness. Human Factors, 27 (1), 137-148.

Weick, K. E., & Roberts, K. H. (1993). Collective mind in organizations: Heedful interrelating on flight decks. Administrative Science Quarterly, 38 , 357-381.

van Knippenberg, D. et al. (2004). Work group diversity and group performance: An integrative model and research agenda. Journal of Applied Psychology, 89 (6), 1008-1022.

Vicente, K. J., & Rasmussen, J. (1992). Ecological interface design: Theoretical foundations. IEEE Transactions on Systems, Man and Cybernetics, 22 (4), 589-606.

Wageman, R. (1995). Interdependence and group effectiveness. Administrative Science Quarterly, 40 , 145-180.