Appropriate validity and its attainment
within action research: an illustration
using soft systems methodology 1Bob Dick and Pam Swepson
This is a resource file which supports the regular public program "areol" (action research and evaluation on line) offered twice a year beginning in mid-February and mid-July. For details email Bob Dick bdick@scu.edu.au or bd@uq.net.au
... in which some issues of validity in action research are considered, and soft systems methodology is analysed to identify the source of some of its validity
Contents
- Summary
- Keywords
- Too
narrow a definition of
validity
- Different
paradigms, different aims, different validity
issues
- Achieving
validity in action research
- Soft
systems methodology
- Soft
systems methodology as an inquiry
process
- Sources
of validity in soft systems
methodology
- Conclusions
- Notes
- References
Summary
We wish to open discussion on the meaning of validity for all paradigms of inquiry, but particularly for the paradigm of action research. We also wish to describe a methodology that will help action research with its particular validity issues.
Keywords
Qualitative research; validity; methodology; paradigms of inquiry; soft systems methodology
Too narrow a definition of validity
We take our definition of validity from John Dewey (1966). He said that scientists, like workers in other areas, were in the business of providing "warranted assertion": being able to mount evidence to support their convictions, at least for the moment.
It is our belief that validity has taken on a meaning that is more relevant to one paradigm of inquiry -- that of experimentalist science -- than to others. "Validity" has come to connote precise quantitative measures arising from tight experimental control. This is an appropriate definition for a paradigm which aims to inquire into the world to arrive at universal principles which can be applied to a variety of situations.
We would argue that other paradigms of inquiry have different aims and require different sorts of evidence to support their assertions. Achieving such aims requires different methods to collect the relevant evidence.
Different paradigms,
different
aims, different validity issues
We believe that the paradigms of inquiry of experimental research, ethnography and action research have two things in common:
- a wish for
understand "the world";
- a wish to improve the human condition.
Within this common framework their aims differ in regard to:
- the relationship
between understanding and improving the human condition;
- the size of the world they wish to understand.
The differences in the three paradigms are choices between the primacy of universal knowledge, or that of specific knowledge. The methodologies of the three paradigms as evident in data collection and analysis will focus on one, at the expense, if necessary, of the second. In methodological terms, this is usually expressed in terms of generalisability. It may more usefully be seen as a trade-off between local relevance and global relevance.
As we have mentioned, experimental science's primary focus is on principles about the whole world for use in specific situations. There is no concern for specific knowledge at all. This is the concern of practitioners who apply the knowledge.
The validity of any assertions that experimental science makes depends on evidence and arguments that support the universality of its results. This is usually done by reducing the variables which they study to those that they can manipulate or measure, preferably under controlled conditions, to establish evidence of strong causal links. The variables that they can treat this way may or may not be the important ones in a specific situation. It is left to those applying the knowledge to adjust it to local circumstances. Experimental science chooses to mount arguments about universal principles at the expense, if necessary, of arguments for specific usefulness.
Ethnography, on the other hand, wishes primarily to explain only one complex social situation, usually without any intention of changing it. There is a secondary interest in gaining knowledge of human nature in general and methods for exploring it.
The validity of any assertions that ethnography makes depends on evidence and arguments that support the accuracy of the explanation of a complex social situation. This is usually done by interviewing a wide range of people in that situation and attempting not to interpret the data but to allow patterns of behaviour to emerge from the interview data. The trade off can be at the expense of comparing between groups or comparing the data with some existing theory to add to the body of knowledge about human behaviour in general. Action research aims primarily to achieve action and understanding in one complex social situation at the same time -- action to inform the understanding which informs the action. There is a secondary interest in gaining knowledge about human nature in general and methods for exploring it.
The validity of any assertion that action research makes depends on evidence to support its claims of change in, and an explanation of, a complex social situation.
Achieving validity in
action research
We have written elsewhere about the strategies that can be used to achieve validity in action research. Important amongst these strategies are
- a cyclic process, so
that early interpretations can be challenged and refined
- the creation of a dialectic between two different sources of information or perspectives on them.
In what follows, we provide a description of soft systems methodology which makes apparent these strategies.
Soft systems methodology
In current use, soft systems methodology has come to refer to a particular system of inquiry developed at Lancaster University by Peter Checkland and his colleagues. In the earlier descriptions, such as Checkland (1981), it is a seven-step process. It may be described as follows.
- the researcher is
immersed in the problem situation;
- the problem systems
and their immediate context are defined;
- root definitions of
the relevant systems (comprising the essence of the systems)
are defined;
- conceptual models of
the systems, intended as improvements, are developed;
- the conceptual
models are compared to reality;
- feasible and
desirable changes are identified;
- action is taken to improve the situation.
In later descriptions such as that by Checkland (1990) the focus is more upon comparing reality to a set of conceptual models, and less upon a step-by-step process for doing this.
Action research methodologies, as we said above, are intended to produce both understanding and change. The methodologies can therefore often be divided conceptually into two parts, addressing these two goals. The first is a process or methodology to achieve change; the second to achieve understanding. In addition, there is often a set of concepts or a model which provides a means for describing what is being examined. In soft systems methodology, systems concepts form the third of these.
In most of the action research literature, the process for achieving change has in general been described in less detail than that for understanding. This is true also of the soft systems methodology literature, more so in the earlier literature. We have no wish to underestimate the importance of change processes in action research. In particular, we think that involving participants and negotiating a relationship with them are often crucial to the success of the endeavour.
However, our present interests are in the validity of the understanding achieved. We therefore focus primarily on the process for understanding. It might be described as an inquiry process.
Soft systems
methodology
as an inquiry process
Viewed as an inquiry process, soft systems methodology illustrates well the strategies of multiple cycles, and a dialectic between different information sources or perspectives. It can be described as four sets of cycles, each involving a dialectic. In this instance, each dialectic is between two different perspectives.
The first cycle comprises a dialectic between the situation, and a description of the essence of that situation. In Checkland's writing, the essence is described in the language of systems theory, in what Checkland calls "root definitions".
In this and subsequent cycles, the researchers (and others involved, if any) alternate between the two perspectives until satisfied with the outcome.
The second dialectic is between the essence, and the conceptual models of the situation which the researchers develop. These models, too, are framed in systems terms.
Within the third cycle, the dialectic is between the conceptual models and the original situation. Out of this comparison, there arise possible improvements to the situation. From these, a plan of action can be developed.
Notice, here, the purpose which is served by the descriptions of the essence. They allow the essential features of the situation to be abstracted from the complicated situation. The second cycle, therefore, can ignore reality for the moment. The conceptual models are thus less influenced by the idiosyncrasies of the particular situation.
The fourth dialectic is that of implementation, and takes place between the intentions of the plans and the requirements of reality. This, though not as central a part of the inquiry process, nevertheless subjects the conclusions to a further test.
The process we have described here need not use systems concepts as the basis for its conceptual models. Other concepts, perhaps drawn from more than one area or discipline, might be used. In effect, Checkland has developed a process which can be regarded as a general inquiry process for action research.
It is important, we believe, to recognise this as a simplified description. As Checkland makes apparent, it is often necessary to detour from the later stages back to earlier stages. The cycles described above are therefore often embedded within larger cycles.
Sources of validity in
soft
systems methodology
At this point we return to the points we raised earlier in this paper. We argued there, following Dewey, that validity is a function of situation and intended outcomes. As an action research methodology, soft systems methodology pursues understanding which can inform action and be informed by it. We offered, as strategies for achieving validity, multiple cycles and dialectic. We believe it is apparent in our description of soft systems methodology that it illustrates both of these.
The use of multiple cycles allows the early conclusions of the researcher(s) to be scrutinised and refined in the later stages. As each cycle examines the topic of study from two different perspectives, the biases of any one of them are more likely to be identified. Finally, implementation determines in a compelling way if the understanding can be used to improve the situation.
Conclusions
To summarise, we have argued that appropriate concepts of validity depend upon situation and intended outcomes. Validity has come to have meanings which are appropriate for experimental paradigms. For action research, with its need for responsiveness and change, different concepts of validity are required. All paradigms seek to understand the world. Action research wishes to use this understanding to inform simultaneous action.
We have offered here two strategies for achieving valid understanding: the use of a cyclic procedure; and, within each cycle, the use of multiple sources of information or different perspectives on what is being studied. We illustrate them by describing soft systems methodology in a way which reveals more clearly the use of these strategies.
Finally, we suggest that the resulting description provides a general inquiry process which need not depend upon systems concepts as the language for describing the situation studied.
Notes
- This document, with only minor editing, is a copy of a paper given at the Third Action Research Congress, Bath, 1994.
References
Checkland, P. (1981) Systems thinking, systems practice. Chichester: Wiley.
Checkland, P. and Scholes, J. (1990) Soft systems methodology in action. Chichester: Wiley
Dewey, J. (1916) Democracy and education : an introduction to the philosophy of education. New York: Macmillan.
Zuber-Skerritt, O., ed. (1991) Action research for change and development. Aldershot: Gower.
_____
Copyright (c) Bob Dick and Pam Swepson 1994-1999. This document may be copied if it is not included in documents sold at a profit, and this and the following notice are included.
This document can be cited as follows:
Dick, B. and
Swepson, P. (1994) Appropriate validity and its
attainment within action research: an illustration using soft
systems methodology [On line].
Available at
http://www.uq.net.au/action_research/arp/sofsys2.html
