Personal tools
You are here: Home 2005 Vol.37 No.2, May 2005 Review of Mixing Methods in Psychology
Document Actions

Review of Mixing Methods in Psychology

This collection of 12 essays attempts to justify the relevance of qualitative methods in psychological research, and to give theoretical arguments and practical advice on the integration of qualitative and quantitative methods. As with most edited books, some chapters are more illuminating and helpful than others. We highly recommend this book as an important step in the advancement of psychological methodology; we also believe this book to be of great import to the HCI community, and will focus on those chapters we believe to be of high quality and particular relevance to HCI.

Mixing Methods in Psychology:
The Integration of Qualitative and Quantitative Methods in Theory and Practice

Edited by Zazie Todd, Brigitte Berlich, Suzanne McKeown and David D. Clarke (2004) Psychology Press
(Softcover: $31.95, Hardcover: $80)


Abstract

This collection of 12 essays attempts to justify the relevance of qualitative methods in psychological research, and to give theoretical arguments and practical advice on the integration of qualitative and quantitative methods. As with most edited books, some chapters are more illuminating and helpful than others. We highly recommend this book as an important step in the advancement of psychological methodology; we also believe this book to be of great import to the HCI community, and will focus on those chapters we believe to be of high quality and particular relevance to HCI.

Introduction

The subjective nature of qualitative methods has often been a source of criticism, but quantitative methods are problematic as well; Benjamin Disraeli once commented “There are lies, damn lies, and statistics.” In this collection of essays, several experimental psychologists attempt to bridge the divide between qualitative and quantitative methods, both by expressing theoretical arguments for their integration (such as triangulation of results), and by giving examples of research methods and experiments that successfully integrate the two.

Noteworthy Chapters

The first section of the book outlines theoretical and historical arguments for mixing methods. In the first chapter, Todd, Nerlich, and McKeown discuss the importance of integrating qualitative and quantitative methods. The chapter discusses a variety of reasons for using multiple methods, which can be categorized as supporting additional validity, properly staging experiments, and bridging the methodological divides.

In the second chapter, “Coming full (hermeneutic) circle”, Nerlich defends qualitative methods by providing a comparative review of the history and philosophy of qualitative and quantitative method use in psychology. Although compelling and well written, this chapter will mainly be of interest to the more theoretically-grounded members of the HCI community. The third chapter, Henwood’s “Reinventing validity”, also provides a justification for mixing methods, but one heavily biased by perceptions of clinical psychologists, and as such is not a critical read for the HCI community.

The second section deals with actual practice of experiments involving mixed quantitative and qualitative methods. Harré and Crystal’s chapter, “Discursive analysis and the interpretation of statistics”, provides a model for much of the practices described in the book; a qualitative method is used initially to determine components for a later quantitative analysis.

David D. Clarke’s essay, “’Structured judgement methods’ – the best of both worlds?” is perhaps the most relevant to HCI theory and practice. His arguments for integrating methods, along with his own integrative methodology, line up nicely with the general trend in HCI research to resolve the separation of heuristic and formal approaches.

Clarke’s insight into the problem with single-method approaches is perhaps not novel, but well-constructed nonetheless. He notes that purely quantitative models give rise to repeatable, verifiable research with (potentially) limited practical value, while purely qualitative methods are suspect for their reliance on the bias of the observer. He also notes that quantitative methods give more readily comparable results, but results that may have been stripped of critical context and meaning.

Clarke’s proposed integrated method, named “structured judgement methods” (p. 86) comes from research in forecasting. The method integrates a number of participants actively (and iteratively) making forecasts, assessing those against real outcomes, and eventually producing a quantitatively valid set of predictive heuristics. He shows that this knowledge-based approach produces much more reliable forecast models than purely quantitative (i.e. model-based) approaches.

In “Q method and qualiquantology”, Stenner and Stainton Rogers present an engaging and well-constructed argument for including qualitative methods to bolster the validity of factor analysis. The authors do warn, however, that Q methodology itself lacks mainstream acceptance as a method, although it might be relevant for analyzing technology adoption, and we believe is flexible enough to allow for an adoption analysis to comprehend technology appropriation, should it be encountered.

In “Valuing the ‘value of life’”, Chilton et al., present a case for the use of qualitative methods alongside contingent valuation (CV) as a mechanism to analyze costs and risks. This research could be readily applied to HCI, despite the authors’ primary concern with physical safety (a factor only occasionally relevant in HCI). Primarily, the method involves providing interview-based context to informed decisions about tradeoff analysis; we believe this technique could be used to provided quantitative validation of claims analysis.

In “Method and methodology on interpretive studies of cognitive life”, Vann and Cole engage in aggressively Vygotskian research of two cognitive tasks (rebus interpretation and order filling). Researchers engaged in Cognitive Work Analysis may find the combination of methods in this chapter particularly relevant.

In “Integrating survey and focus group research”, Todd and Lobeck discuss a key problem of mixed methods: conflicting results. Like all problematic research, this generates more questions than answers, but the authors end the chapter with a thorough analysis of the potential causes for the conflicts. This analysis can provide both comfort and useful direction to any researcher facing conflicting data.

The final section of the book deals with larger scale questions of when and how to mix methods. Miller’s chapter, “Educational psychology and difficult behavior”, deals with an historical analysis of the methods used to study student behavior in public schools in both Britain and the US. This chapter would provide a good foundation for a methodologically-aware literature review in HCI. Nicolson’s chapter, “Taking quality seriously: The case for qualitative feminist psychology in the context of quantitative clinical research on postnatal depression”, is far too grounded is psychology to be readily taken into another domain. In addition, we are not sure if the community is quite ready for a feminist theory of HCI.

The book finishes with a chapter on “Future directions”, in which Todd and Nerlich make several practical recommendations for integrating qualitative and quantitative methods in both training and research, with particular respect to the analysis of existing data sets. The authors conclude by asserting that this avenue of research will increase in relevance, an assertion that we believe is relevant for HCI, as well. Any such approach must take seriously the complexity of mixed-method research, and this book provides useful guidance.

Conclusion

Much HCI research and practice is Vygotskian in nature; we actively modify the environment (system) as we research, evolving both our product and our process throughout. As a result, practices that can increase validity are of particular value for our practical domain. Merging qualitative and quantitative methods must be done carefully; otherwise, considerable time can be spent adding little or no additional validation. This book provides excellent theoretical and practical guidance on why, when, and how multiple methods can be integrated; while written for a more traditional audience of experimental psychologists, HCI researchers and practitioners can find much of value in this book.

Reviewed by

Joshua B. Gross and Scott M. Robertson
School of Information Sciences and Technology
The Pennsylvania State University
{jgross, srobertson}@ist.psu.edu
311B IST Building The Pennsylvania State University
University Park, PA 16802 USA

 

Powered by Plone CMS, the Open Source Content Management System

This site conforms to the following standards: