Concept mapping is a type of structured conceptualization which can be used by groups to develop a conceptual framework which can guide evaluation or planning. In the typical case, six steps are involved: 1) Preparation (including selection of participants and development of focus for the conceptualization); 2) the Generation of statements; 3) the Structuring of statements; 4) the Representation of Statements in the form of a concept map (using multidimensional scaling and cluster analysis); 5) the Interpretation of maps; and, 6) the Utilization of Maps. Concept mapping encourages the group to stay on task; results relatively quickly in an interpretable conceptual framework; expresses this framework entirely in the language of the participants; yields a graphic or pictorial product which simultaneously shows all major ideas and their interrelationships; often improves group or organizational cohesiveness and morale. This paper describes each step in the process, considers major methodological issues and problems, and discusses computer programs which can be used to accomplish the process.
Probably the most difficult step in a planning or evaluation project is the first one -- everything which follows depends on how well the project is initially conceptualized. Conceptualization in this sense refers to the articulation of thoughts, ideas, or hunches and the representation of these in some objective form. In a planning process, we typically wish to conceptualize the major goals and objectives, needs, resources and capabilities or other dimensions which eventually constitute the elements of a plan. In evaluation, we may want to conceptualize the programs or treatments, samples, settings, measures and outcomes which we believe are relevant.
This special section of Evaluation and Program Planning extends earlier work by Trochim and Linton (1986) who proposed a general framework for structured conceptualization and showed how specific conceptualization processes can be devised to assist groups in the theory and concept formation stages of planning and evaluation. The papers presented here focus on one specific type of structured conceptualization process which we term "concept mapping". In concept mapping, ideas are represented in the form of a picture or map. To construct the map, ideas first have to be described or generated, and the interrelationships between them articulated. Multivariate statistical techniques -- multidimensional scaling and cluster analysis -- are then applied to this information and the results are depicted in map form. The content of the map is entirely determined by the group. They brainstorm the initial ideas, provide information about how these ideas are related, interpret the results of the analyses, and decide how the map is to be utilized.
The process described here is not the only way to accomplish concept mapping. For instance, Novak and Gowin (1984) suggest that concept maps be drawn "free-hand" after an initial articulation of the major ideas and classification of them into hierarchical concepts. In a similar manner, Rico (1983) has advocated "free-hand" concept mapping or drawing as a useful method for developing a conceptual framework for writing. These and other approaches have value for planning and evaluation, but fall outside of the scope of this paper. The major differences between the method described here and other concept mapping processes are: this method is particularly appropriate for group use -- the method generates a group aggregate map; it utilizes multivariate data analyses to construct the maps; and it generates interval-level maps which have some advantages for planning and evaluation, especially through pattern matching as described later. Despite these differences, this paper should be viewed as a clear call for the importance of further exploration of any processes which improve conceptualization in planning and evaluation. Throughout the papers in this volume, however, the term "concept mapping" should be understood to refer only to the process described here, and its variations.
Group concept mapping is consistent with the growing interest in the role of theory in planning and evaluation. In evaluation, for instance, this interest is evidenced in writings on the importance of program theory (Bickman, 1986; Chen and Rossi, 1983, 1987; Rossi and Chen, in press); in the increased emphasis on the importance of studying causal process (Mark, 1986); in the recognition of the central role of judgment -- especially theory-based judgment -- in research (Cordray, 1986; Einhorn and Hogarth, 1986); and, in the thinking of critical multiplism (Shadish et al, 1986) which emphasizes the role of theory in selecting and guiding the analysis of multiple operationalizations. Concept mapping can be viewed as one way to articulate theory in these contexts. In planning, conceptualization has had somewhat more attention and is evidenced in the sometimes daunting proliferation of different planning models and methods of conceptualizing (Dunn, 1981).
This paper introduces the concept mapping process, suggests some of the major technical or methodological issues which are involved, and offers some suggestions about computer programs which can be used to accomplish concept mapping. The remaining papers provide numerous examples of the use of this process and its variations in a wide variety of contexts and consider in greater detail some of the methodological issues which face researchers in this area.
The term "structured conceptualization" refers to any process which can be described as a sequence of concrete operationally-defined steps and which yields a conceptual representation (Trochim and Linton, 1986). The specific concept mapping process described here and discussed throughout this volume is considered only one of many possible structured conceptualization processes. This process can be used whenever there is a group of people who wish to develop a conceptual framework for evaluation or planning, where the framework is displayed in the form of a concept map. A concept map is a pictorial representation of the group's thinking which displays all of the ideas of the group relative to the topic at hand, shows how these ideas are related to each other and, optionally, shows which ideas are more relevant, important, or appropriate.
The scenario within which concept mapping is applied assumes that there is an identifiable group responsible for guiding the evaluation or planning effort. Depending on the situation, this group might consist of the administrators, staff or members of the board of an organization; community leaders or representatives of relevant constituency groups; academicians or members of the policy making community; funding agents or representatives of groups with oversight responsibility; representatives of relevant client populations; or combinations of these. The concept mapping process is guided by a facilitator who could be an outside consultant or an internal member of the group responsible for the planning or evaluation effort. The facilitator's role is only to manage the process -- the content, interpretation and utilization of the concept map are determined entirely by the group.
An overview of the concept mapping process is provided in Figure 1.
The figure shows six steps which are followed in developing a useful group concept map. Each of these steps will be discussed in some detail and illustrated with data from a concept mapping process which was conducted in York County, Maine, to assist representatives of a number of county human service agencies to develop a conceptual framework for planning services for the elderly. While the major focus of this project was on planning, some comments will be offered regarding the potential use of this conceptualization for evaluation purposes.
There are two major tasks which must be undertaken prior to commencement of the actual group process. First, the facilitator must work with the parties involved to decide on who will participate in the process. Second, the facilitator must then work with the participants or a subgroup to decide on the specific focus for the conceptualization.
Selecting the Participants. One of the most important tasks which the facilitator addresses is who will participate in the concept mapping process. Our experience has been that a conceptualization is best when it includes a wide variety of relevant people. If we are conducting strategic planning for a human service organization, we might include administrative staff, service staff, board members, clients, and relevant members of community groups. In a program evaluation context, we might similarly include administrators, program staff, clients, social science theorists, community members, and relevant funding agent representatives. Broad heterogenous participation helps to insure that a wide variety of viewpoints will be considered and encourages a broader range of people to "buy into" the conceptual framework which results.
In some situations, however, we have used relatively small homogenous groups for the conceptualization process. For instance, if an organization is beginning a strategic planning effort and would like to lay out quickly some of the major concepts around which the planning will be based, they might prefer to use a relatively small group of administrators and organizational staff members. The obvious advantage of doing this is that it is logistically simpler to get people together for meetings if they are all on the staff of the organization. This type of group works well when a quick conceptualization framework is desired, but in general we would recommend a broader sampling of opinion.
In some contexts it might be reasonable to use some random sampling scheme to select participants from a larger defined population. This is most useful when one wishes to argue that the resulting concept map is generalizable to some larger population of interest. Simple random sampling schemes, of course, run the risk of underrepresenting minority groups from the population and so, if sampling is used, it will typically be best to attempt either some form of stratified random sampling or purposive sampling for heterogeneity.
There is no strict limit on the number of people who can be involved in concept mapping. It is feasible, with some process modifications, for an individual to conduct a conceptualization alone (see Dumont, this volume, for an example). At the other extreme, we have worked with groups as large as 75-80 people in this process. Typically, we have had between 10 and 20 people in most of our studies and this seems to be a workable number. Groups of that size insure a variety of opinions and still enable good group discussion and interpretation.
It is also not necessary that all participants take part in every step of the process. One might, for instance, have a relatively small group do the generation (e.g., brainstorming) step, a much larger group perform the structuring (i.e., sorting and rating) and a small group for interpretation and utilization. In general, however, we have found that concept maps are better understood by people who have participated in all phases of the process than by those who have only taken part in one or two steps.
In the York County study, the purpose of the conceptualization was to bring together a small group of representatives from a number of local agencies which provide services to the elderly in order to develop a framework for planning. There was also a strong interest in piloting the concept mapping process with the idea that it might possibly be applied later with a broader constituency group which included elderly persons from the county. Between 10 and 15 people participated in the two meetings including representatives of the United Way and several health and mental health organizations.
Developing the Focus. The second major preparatory step involves developing the focus or domain of the conceptualization. There are two separate products which are desired here. First, the participants must define the focus for the brainstorming session. Second, the focus for ratings which are performed during the structuring step of the process needs to be developed. This essentially involves defining the dimension(s) on which each of the brainstormed statements will be rated.
It is essential that the focus for both the brainstorming and the ratings be worded as statements which give the specific instruction intended so that all of the participants can agree in advance. In developing both the brainstorming and rating focus statements, the facilitator usually has a meeting with the participants or some representative subgroup. In this meeting the facilitator discusses various alternatives for wording each focus and attempts to achieve a group consensus on final choices. For example, the brainstorming focus in a strategic planning process might be worded: "Generate short phrases or sentences which describe specific services which your organization might provide." Similarly, a rating focus for a program evaluation might be worded: "Rate each potential outcome on a seven point scale in terms of how strongly you think it will be affected by the program, where '1' means 'Not at all affected', '4' means 'Moderately affected' and '7' means 'Extremely affected'." The group should agree on the specific wording for each of these focus statements.
The Brainstorming Focus. For any brainstorming session, there are a variety of ways in which the focus can be stated. For instance, if we are interested in strategic planning, participants might focus on the goals of the organization, the mission of the organization, or the activities or services which the organization might provide. Similarly, in program evaluations they might focus on the nature of the program, the outcomes they would like to measure, or the types of people they would like to participate in the evaluation.
In defining the brainstorming focus, it is important to try to anticipate the types of statements which will result. For instance, one usually wants to avoid giving double-barrelled focus statements like: "Generate short statements or sentences which describe the goals of our organization and the needs of our clients" because, when sorting, participants are likely to perceive these two categories as particularly distinct and consequently, they are likely to show up as two major clusters on the final concept map and obscure some of the finer relationships which might be of interest (see Keith, this volume, for a discussion of this issue). If both emphases are desired, it would probably be better to conduct two separate conceptualizations to address each issue. In general, it is best if the final set of brainstormed statements are "of a kind", that is, share the same level of conceptual generality and grammatic structure. For example, a brainstormed set which consists entirely of statements which describe specific services which an organization might provide and which are all worded in a similar manner (e.g., some aren't in question form while others are worded as statements) would be best if the interest is in conceptualizing a taxonomy of potential services.
In the York County study, the brainstorming focus statement was a broad one: "Generate statements which describe the issues, problems, concerns or needs which the elderly have in York County." Although this may seem to violate the advice above on avoiding double-barrelled statements, the participants felt that "issues, problems, concerns or needs" were essentially indistinguishable and that the brainstormed statements which resulted would constitute an interpretable, relatively homogeneous set.
The Rating Focus. In developing the focus for the ratings, one needs to consider what kind of information will be most useful. In planning, it is often useful to ask the participants to rate how important each brainstormed item is, or how much emphasis should be placed upon it in the planning process. In evaluation, it might be useful to ask them to rate how much effort should be given to various program components, or how much they believe each outcome is likely to be affected by the program.
In the York County study, the rating focus statement was: "Rate each statement on a 1 to 5 scale for how much priority it should be given in the planning process, where '1' equals the lowest priority and '5' equals the highest priority."
Once the participants and focus statements have been defined, the actual concept mapping process begins with the generation of a set of statements which ideally should represent the entire conceptual domain for the topic of interest. In the typical case, brainstorming is used and the focus statement constitutes the prompt for the brainstorming session. The usual rules for brainstorming apply (Osborn, 1948; Dunn, 1981). That is, people are encouraged to generate lots of statements and are told that there should be no criticism or discussion regarding the legitimacy of statements which are generated during the session. Participants are encouraged to ask for clarification of any unfamiliar terms or jargon so that all who participate may understand what was intended by a given statement. Usually, the facilitator records the statements as they are generated so that all members of the group can see the set of statements as they evolve. We have done this by writing the statements on a blackboard, on sheets of newsprint , or by directly entering them into a computer program which is displayed on a large screen so that everyone can see the statements. If the facilitator believes that there may be some participants who would be reluctant to state publicly some idea because of its controversial or potentially embarrasing nature, it may be desirable to also allow each participant to submit several statements anonymously on paper so that confidentiality will be preserved.
Theoretically, there is no limit to the number of statements which can be generated. However, large numbers of statements impose serious practical constraints. Based on our experience, we now limit the number of statements to one hundred or less. If the brainstorming session generates more than a hundred statements, we reduce the set. There are a number of ways in which this can be accomplished. The group as a whole or some subgroup can examine the set of statements for redundancies or ones which can be chosen to represent a set of others. In some instances, we have taken a simple random sample of statements from the larger set and had participants examine the selected set to be sure that no key ideas were being omitted. For instance, Linton (this volume) randomly selected 150 statements out of a larger set of 710 statements that were generated. On several occasions we have tried more formal thematic analysis of the text statements using a "key words in context" approach (Stone et al, 1966; Krippendorf, 1980) which seems promising.
Once a final set of statements has been generated, it is valuable for the group to examine the statements for editing considerations. Sometimes the wording of statements generated in a brainstorming session is awkward or technical jargon is not clear. In general, each statement should be consistent with what was called for in the brainstorming prompt and should be detailed enough so that every member of the group can understand the essential meaning of the statement.
There are many other ways to generate the conceptual domain than brainstorming. Sometimes a set of statements can be abstracted from existing text documents such as annual reports, internal organizational memos, interviews or field notes. For instance, Dumont (this volume) utilized the "documentary coding method" described by Wrightson (1976) to abstract statements or entities from interview records. In some cases there was no need to generate statements at all because the nature of the conceptualization dictates the elements of the conceptual domain. Thus, if the goal is to conceptualize the interrelationships between the set of 10 departments in an organization, we might simply use the 10 department names as the set of statements. Marquart (this volume) used a set of concepts derived from the literature on employer-sponsored child care as the set of statements for mapping. Caracelli (this volume) used the 100 items from the California Q-sort as the set of statements. We have also done some preliminary work to examine the use of outlines for generation where each entry in the outline would be considered a separate statement. One potential advantage of doing this is that we may be able to use the implicit structure of the outline headings and subheadings to structure the conceptual domain directly without asking people to sort the statements. Cooksy (this volume) describes some preliminary attempts to develop models which take outlines as input and estimate the similarity between statements (based on outline structure) as output.
In the York County study, a set of 95 statements was generated in the brainstorming session and are shown in Table 1. The statements describe a broad range of issues which were thought by participants to be salient to the elderly in York County.
Once we have a set of statements which describes the conceptual domain for a given focus, we minimally need to provide information about how the statements are related to each other. In addition, we often want to rate each statement on some dimension which is defined by the rating focus statement. Both of these tasks constitute the structuring of the conceptual domain.
Typically, we obtain information about interrelationships using an unstructured card sorting procedure (Rosenberg and Kim, 1975). Each of the brainstormed statements is printed on a separate 3x5 index card and the complete set of cards is given to each participant. Each person is then instructed to sort the cards into piles "in a way that makes sense to you." There are several restrictions placed on this procedure: each statement can only be placed in one pile (i.e., an item can't be placed in two piles simultaneously); all statements cannot be put into a single pile; and, all statements cannot be put into their own pile (although some items may be sorted by themselves). Except for these conditions, people may pile the cards in any way that makes sense to them. Often the participants perceive that there may be several different ways to sort the cards, all of which make sense. To address this, we have either instructed participants to select the most sensible arrangement or, in some studies, have had each participant sort the cards several times.
When each person has completed the sorting task, the results must be combined across people. This is accomplished in two steps. First, the results of the sort for each person are put into a square table or matrix which has as many rows and columns as there are statements. All of the values of this matrix are either zero or one. A '1' indicates that the statements for that row and column were placed by that person together in a pile while a '0' indicates that they were not. This is illustrated in Figure 2 for a hypothetical person who sorted ten statements into 4 piles.
We can see in the figure that statements 5 and 8 were sorted together in a pile. Therefore, in the table the row 5 - column 8 and row 8 - column 5 entries are '1'. Because statement 5 was not sorted with statement 6, the row 5 - column 6 and row 6 - column 5 entries are '0'. This individual matrix is termed a binary symmetric similarity matrix. Notice that all of the diagonal values are equal to '1' because a statement is always considered to be sorted into the same pile as itself.
Second, the individual sort matrices are added together to obtain a combined group similarity matrix. This matrix also has as many rows and columns as there are statements. Here, however, the value in the matrix for any pair of statements indicates how many people placed that pair of statements together in a pile regardless of what the pile meant to each person or what other statements were or were not in that pile. Values along the diagonal are equal to the number of people who sorted. Thus, in this square group similarity matrix, values can range from zero to the number of people who sorted. This final similarity matrix is considered the relational structure of the conceptual domain because it provides information about how the participants grouped the statements. A high value in this matrix indicates that many of the participants put that pair of statements together in a pile and implies that the statements are conceptually similar in some way. A low value indicates that the statement pair was seldom put together in the same pile and implies that they are conceptually more distinct. There are many others ways than sorting to structure the conceptual domain, some of which are briefly described in Trochim and Linton (1986). The major advantages of the sorting procedure are that it is easily understandable by participants and that it takes little time to accomplish.
The second task in structuring the conceptual domain involves rating each statement on some dimension as described in the rating focus statement. Usually this rating is accomplished using a Likert-type response scale (e.g., 1-to-5 or 1-to-7 rating) to indicate how much importance, priority, effort or expected outcome is associated with each statement. For each statement one then obtains at least the arithmetic mean of the ratings and sometimes other descriptive statistical information.
There are three steps involved in the way in which we typically represent the conceptual domain. First, we conduct an analysis which locates each statement as a separate point on a map (i.e., the point map). Statements which are closer to each other on this map were likely to have been sorted together more frequently; more distant statements on the map were in general sorted together less frequently. Second, we group or partition the statements on this map into clusters (i.e., the cluster map) which represent higher order conceptual groupings of the original set of statements. Finally, we can construct maps which overlay the averaged ratings either by point (i.e., the point rating map) or by cluster (i.e., the cluster rating map).
To accomplish the first step, the mapping process, we typically conduct a two-dimensional nonmetric multidimensional scaling of the similarity matrix obtained from Step 3. Nonmetric multidimensional scaling is a technique which takes a proximity matrix and represents it in any number of dimensions as distances between the original items in the matrix. A good introductory discussion of multidimensional scaling can be found in Kruskal and Wish (1978) and a more technical description of the algorithm which is used is given in Davison (1983).
A simple example of the principle which underlies multidimensional scaling can be given. If you were given a geographical map of the United States and asked to construct a table of distances between three major cities, say New York, Chicago, and Los Angeles, you could accomplish this fairly easily. You might take a ruler and measure the distances between each pair of cities and enter them into a 3 x 3 table of ruler-scale relative distances. However, if you were given only a table of distances between the three cities and were asked to draw a map which located the three cities on it as points in a way that fairly represented the relative distances in the table, the task would be slightly more difficult. You might begin by arbitrarily placing two points on a page to represent two of the cities and then try to draw in a third point so that its distances to the first two cities was proportionate to the distances given in the table. You would be able to accomplish this if the table consisted of three cities, but for more this task would become extremely complex. Multidimensional scaling is a multivariate analysis which accomplishes this task. It takes a table of similarities or distances and iteratively places points on a map so that the original table is as fairly represented as possible. In concept mapping, the multidimensional scaling analysis creates a map of points which represent the set of statements which were brainstormed based on the similarity matrix which resulted from the sorting task.
Typically, when multidimensional scaling analysis conducted the analyst has to specify how many dimensions the set of points is to be fit into. If a one-dimensional solution is requested, all of the points will be arrayed along a single line. A two-dimensional solution places the set of points into a bivariate distribution which is suitable for plotting on an X-Y graph. The analyst could ask for any number of solutions from 1 to N-1 dimensions. However, it is difficult to graph and interpret solutions which are higher than three-dimensional easily. The literature on multidimensional scaling discusses this dimensionality issue extensively. One view is that the analyst should fit a number of solutions (e.g., one- to five-dimensional solutions) and examine diagnostic statistics to see whether a particular dimensional solution is compelling. This is analogous to examining J-plots of eigenvalues in factor analysis in order to decide on the number of factors. Another view suggests that in certain contexts automatic use of two-dimensional configurations might make sense. For instance, Kruskal and Wish (1978) state that
Since it is generally easier to work with two-dimensional configurations than with those involving more dimensions, ease of use considerations are also important for decisions about dimensionality. For example when an MDS configuration is desired primarily as the foundation on which to display clustering results, then a two-dimensional configuration is far more useful than one involving 3 or more dimensions" (p. 58).
In studies where we have examined other than two-dimensional solutions, we have almost universally found the two-dimensional solution to be acceptable, especially when coupled with cluster analysis as Kruskal and Wish suggest. Therefore, in concept mapping we usually use a two-dimensional multidimensional scaling analysis to map the brainstormed statements into a two-dimensional plot.
The second analysis which is conducted to represent the conceptual domain is called hierarchical cluster analysis (Anderberg, 1973; Everitt, 1980). This analysis is used to group individual statements on the map into clusters of statements which presumably reflect similar concepts. There are a wide variety of ways to conduct cluster analysis and there is considerable debate in the literature about the relative advantages of different methods. The discussion centers around ambiguity in the definition of the term "cluster." Everitt (1980) and Anderberg (1973) present more extensive discussions of this issue. We have tried a number of different cluster analysis approaches. Originally, we used an oblique principle components factor analysis approach to hierarchical cluster analysis where the input for the analysis consisted of the similarity matrix. The problem with this approach was that it often led to results which did not visually correspond with the way in which multidimensional scaling mapped the points. This is because differences in the two algorithms (i.e., multidimensional scaling and cluster analysis) when applied to the same similarity matrix, sometimes meant that points which were close to each other on the map were placed in separate clusters by the cluster analysis. These results were hard to interpret and seemed to give equal weight to multidimensional scaling and cluster analysis. Instead, it makes sense to view the mathematical basis for multidimensional scaling as stronger than basis for cluster analysis and, accordingly, to rely on the multidimensional scaling rather than cluster analysis to depict the basic inter-statement conceptual similarities. What we wanted was a cluster analysis which grouped or partitioned the statements on the map as they were placed by multidimensional scaling. We found that this could be accomplished by using the X-Y multidimensional scaling coordinate values for each point (rather than the original similarity matrix) as input to the cluster analysis. In addition, we also found that Ward's algorithm for cluster analysis generally gave more sensible and interpretable solutions than other approaches (e.g., single linkage, centroid). Therefore we have moved to an approach which uses Ward's hierarchical cluster analysis on the X-Y coordinate data obtained from multidimensional scaling as the standard procedure. This in effect partitions the multidimensional scaling map into any number of clusters.
Just as deciding on the number of dimensions is an essential issue for multidimensional scaling analysis, deciding on the number of clusters is essential for cluster analysis. All hierarchical cluster analysis procedures give as many possible cluster solutions as there are statements. In principle, these clustering methods begin by considering each statement to be its own cluster (i.e., an N-cluster solution). At each stage in the analysis, the algorithm combines two clusters until, at the end, all of the statements are in a single cluster. The task for the analyst is to decide how many clusters the statements should be grouped into for the final solution. There is no simple way to accomplish this task. Essentially, the analyst must use discretion in examining different cluster solutions to decide on which makes sense for the case at hand. Usually, assuming a set of a hundred or fewer statements, we begin by looking at all cluster solutions from about 20 to 3 clusters. Each time the analysis moves from one cluster level to the next lowest (e.g., from 13 to 12 clusters) we examine which statements were grouped together at that step and attempt to decide whether that grouping makes sense for the statements in the conceptualization. In examining different cluster solutions we have found it useful to use a cluster tree which shows pictorially all possible cluster solutions and mergers. In general, we attempt to decide on a cluster solution which, if anything, errs on the side of more clusters than fewer. Clearly this is a task which requires discretion on the part of the analyst and it would be ideal if we could involve the participants directly in this decision making process, but as of yet we have not determined an easy way to accomplish this kind of task within a group process.
Our experience shows that, in general, the cluster analysis results are less interpretable than the results from multidimensional scaling. The cluster analysis is viewed as suggestive and, in some cases, one may want to "visually adjust" the clusters into more sensibly interpretable partitions of the multidimensional space. The key operative rule here would be to maintain the integrity of the multidimensional scaling results, that is, try to achieve a clustering solution which does not allow any overlapping clusters (e.g., a true partitioning of the space).
Once we have conducted the multidimensional scaling and cluster analysis, we are able to generate a point and a cluster map. The final analysis involves obtaining average ratings across participants for each statement and for each cluster. These can then be overlayed graphically on the maps to produce the point rating map and the cluster rating map as will be shown later.
We have several products at the end of the representation step. First, we have the two-dimensional point or statement map which locates each of the brainstormed statements as a point. Next to each point we place the number of the statement so that participants can identify each point as a statement. Second, we have a cluster map which shows how the cluster analysis grouped the points. Third, we have the point rating map which shows the average ratings for each statement on the point map. Finally, we also have the cluster rating map which shows the average rating for each cluster on the cluster map. This information forms the basis of the interpretation in the next step.
To interpret the conceptualization, we usually assemble a specific set of materials and follow a specific sequence of steps -- a process which has been worked out largely on the basis of our experiences on many different projects. The materials consist of:
Notice that there are four different types of maps here. Which of them is the concept map? In fact, they are all concept maps. Each of these maps tells us something about the major ideas and how they are interrelated. Each of them emphasizes a different part of the conceptual information. While the maps are distinctly different ways of portraying or representing the conceptual structure (and consequently, different names are used to distinguish them), it is important to remember that they are all related to each other and are simply reflecting different sides of the same underlying conceptual phenomenon. In the remainder of this paper, if the type of concept map is not specified, it is usually fair to assume that the discussion pertains to the cluster map because that is usually the most directly interpretable map.
The facilitator begins by giving the group the original set of brainstormed statements (for the York County example, the statement list is shown in Table 1 above) and recalling that the statements were generated by the them in the brainstorming session. Participants are then reminded that they grouped these statements into piles and told that the individual sortings were combined for the entire group. The statements as they were grouped by the cluster analysis (the cluster list) are then presented. For the York County study, the cluster listing is given in Table 2.
Each participant is asked to read through the set of statements for each cluster and come up with a short phrase or word which seems to describe or name the set of statements as a cluster. This is analogous to naming factors in factor analysis. When each person has a tentative name for each cluster, the group works cluster-by-cluster in an attempt to achieve group agreement on an acceptable name for each cluster. This is an often interesting negotiating task. When each person in turn gives their name for a certain cluster, the group can often readily see a consensus which exists. For some clusters, the group may have difficulty in arriving at a single name. This is because the statements in that cluster might actually contain several different ideas and, had a higher cluster solution been selected, the statements would have been subdivided into subclusters. In these cases, the facilitator might suggest that the group use a hybrid name, perhaps by combining titles from several individuals. In any event, the group is told that these names are tentative and may be revised later.
When the group has reached consensus on the names for each cluster, they are presented with the numbered point map. The York County map is shown in Figure 3.
They are told that the analysis placed all of the statements on the map in such a way that statements which were piled together frequently should be closer to each other on the map than statements which were not piled together frequently. Usually it is a good idea to give them a few minutes to identify a few statements on the map which are close together and examine the wording of those statements on the original brainstormed statement list as a way to reinforce the notion that the analysis is placing the statements sensibly. When they have become familiar with this numbered point map, they are told that the analysis also organized the points into groups as shown on the list of clustered statements which they just named. The cluster map is presented and participants are shown that the map portrays visually the exact same clustering which they just looked at on the cluster list. They are then asked to write the cluster names which the group arrived at next to the appropriate cluster on the cluster map. They are then asked to examine this named cluster map to see whether it makes any sense. The facilitator should remind participants that in general, clusters which are closer together on the cluster map should be more similar conceptually than clusters which are farther apart and ask them to assess whether this seems to be true or not. Participants might even begin at some point on the map and, thinking of a geographic map, "take a trip" across the map reading each cluster in turn to see whether or not the visual structure makes any sense. For the York County example, the named cluster map is given in Figure 4.
The participants are then asked to see whether there are any sensible groups or clusters of clusters. Usually, the group is able to perceive several major regions. These are discussed and partitions are drawn on the map to indicate the different regions. Just as in naming the clusters, the group then attempts to arrive at a consensus concerning names for these regions. In the York County study, people were satisfied with the clustering arrangement and did not wish to define any regions.
This final named cluster map constitutes the conceptual framework and the basic result of the concept mapping process. The facilitator should remind the participants that this final map is their own product. It was entirely based on statements which they generated in their own words and which they grouped. The labels on the map represent categories which they named. While in general the computer analysis will yield sensible final maps, the group should feel free to change or rearrange the final map until it makes sense for them and for the conceptualization task at hand. At this point it is useful for the facilitator to engage the participants in a general discussion about what the map tells them about their ideas for evaluation or planning.
If ratings were done in the structuring step, the facilitator then presents the point rating and cluster rating maps. The point rating map for the York County Study is given in Figure 5
while the cluster rating map is shown in Figure 6.
Participants examine these and attempt to determine whether they make sense and what they imply about the ideas which underlie their evaluation or planning task.
In the York County study some interesting insights arose from the interpretation process. If we look at either of the cluster maps (Figures 4 or 6) we can see that the clusters on the right side of the map -- Personal Growth and Education, Stereotyping, Socialization Needs, and Political Strength and Advocacy -- tend to be the types of issues of most concern to the "well elderly", people who are not ill or institutionalized. As we move counter-clockwise on the map, the clusters on the top represent concerns of those who are becoming ill or are need of service, those on the left are most relevant for people who are already ill and/or in need of intensive home or center-based care, and those on the bottom pertain most to persons who are severely ill or dying. The group perceived this counter-clockwise cycle as a good description of their implicit theory of the aging process. Furthermore, they believed that most of the political strength and advocacy work that is exists around aging issues tends to be done by the "well elderly" as the map implies. This led them to discuss the desirability of working within the community to encourage the well elderly to perceive their position within the entire aging cycle and begin to engage in more active advocacy for the full range of concerns which the map describes. Thus, the map provided a foundation for an approach to the elderly which addresses major concerns and emphasizes the more active involvement especially of the well elderly.
At this point in the process we turn our attention back to the original reason for conducting the structured conceptualization. The group discusses how the final concept map might be used to enhance either the planning or evaluation effort. The uses of the map are limited only by the creativity and motivation of the group. A number of straightforward applications suggest themselves. For instance, if the conceptualization was done as the basis for planning, the final map might be used for structuring the subsequent planning effort. The planning group might use it for dividing up into subgroups or task forces, each of which is assigned a specific cluster or region. Each task group could then examine issues like: the organizational budget allocation for each cluster, how organizational personnel are distributed within each cluster, how important each cluster is relative to the others, what resources might be brought to bear in addressing each cluster, what level of competition exists from other organizations providing services in each cluster, and so on. The task forces can use the individual statements within a cluster as cues or prompts concerning what they should consider specifically within each cluster. One major advantage to having the concept map is that the results of these task force investigations can often be usefully displayed directly on the concept map as were the priority ratings for the York County example in Figures 5 or 6. Thus, any number of maps can be created showing such variables as budget allocations, staff effort, or degree of need, and displayed by individual statement and/or by cluster.
For planning purposes, the concept map can also be used as the framework for an outline of a planning report. Regional headings would constitute the highest level of indentation for the outline, clusters would be subheaded within their appropriate regions, individual statements could be subheadings within clusters, and any statement-level information relevant to planning could be subheaded within this structure. Thus for planning, the concept map provides a framework for understanding important issues in a way which enables sensible pictorial and outline representations.
The concept map is also extremely useful in evaluation contexts. Here, its utilization depends on what the focus was for the conceptualization. If the focus was on planning a program or service which would then be evaluated, the concept map can act as an organizing device for operationalizing and implementing the program. For instance, if the program is a training program in a human service agency, the training can be constructed based on the concept map with different training sessions designed to address each cluster and the individual brainstormed statements acting as cues for what kinds of information should be covered in each session. The concept map is the framework for the program construct and can form the basis of a process evaluation of the program. In this case or in the case where the focus of the conceptualization was on the outcomes of some program, the concept map can guide measurement development. Each cluster can be viewed as a measurement construct and the individual statements can suggest specific operationalizations of measures within constructs. For instance, if the group wished to develop a questionnaire, they could use the concept map by having each cluster represented with questions on the questionnaire. Furthermore, the original brainstormed statements might provide question prompts which either directly or with some revision could be included on the questionnaire along with some rating response format. Alternatively, if a more multimethod approach to measurement was desired, the group could make sure that within each cluster several different types of measures were constructed to reflect the cluster. The exciting prospect here is that the concept map provides a useful way to operationalize the multitrait-multimethod approach to measurement which was outlined by Campbell and Fiske (1959) and is described in greater detail in the paper by Davis (this volume). In this example, the concept map represents the group's theoretical expectations about how the major measurement constructs are conceptually interrelated. From the concept map we can predict the rank order which we expect in the correlations between measures. These expectations (the theoretical pattern) could then be directly compared with a matrix of correlations as obtained in the study (the observed pattern) and the degree to which the two match can constitute evidence for the construct validity of the measures. This "pattern matching" approach to construct validity is discussed in detail in Trochim (1985; in press).
A number of papers in this volume illustrate the use of structured conceptualization for evaluation. For instance, Valentine (this volume) used concept mapping to construct an instrument which could be used to assess caring in a nursing context. Galvin (this volume) used concept mapping to describe the central concepts in a Big Brother/ Big Sister program, constructed an instrument from the map, and used the instrument to explore how well the program appeared to be performing.
An introduction to concept mapping would be incomplete without some consideration of the computer programs which can be used to accomplish this process. Essentially, there are two options: use some combination of standard general-purpose word processing and statistics packages; or, use the computer package which was designed by the author specifically for accomplishing concept mapping. Each is discussed in turn.
When using available general-purpose software packages, the analyst will have to be prepared to experiment with different processing options until a suitable procedure can be constructed. Minimally, it is desirable to have a good word processing program; a statistics package which has routines for multidimensional scaling and cluster analysis, and which has fairly flexible data manipulation capabilities; and, a graphics program to plot the final maps.
On a mainframe computer, brainstormed statements can be entered into any standard editor. They would then need to be formatted and printed onto cards, labels, or sheets in a manner which allows the analyst to assemble them into sorting decks. In addition, the statements would need some minor formatting in order to produce an instrument for the rating task. Multidimensional scaling and cluster analysis are available in most large scale statistical systems including SAS and SPSSx. The major difficulty with this approach will involve the entry and manipulation of the sort data. The sort results could be entered into a data matrix using a standard editor where each participant constitutes a single record (row) and there are as many variables (columns) as there are statements. The analyst would arbitraily assign a unique number to each pile of sorted statements and enter this number in the appropriate row and column. There are two main difficulties with this procedure, both of which are resolvable. First, since the data for any participant will be entered in the order of the statements, the sort data will have to be coded. That is, the analyst will have to set up a coding sheet with the statements numbered sequentially and enter the pile number next to each statement before entering the data. Second, and perhaps more troublesome, the analyst will have to write a computer program which takes the sort data matrix as input, constructs the binary square similarity matrix for each person and then adds these to get the group similarity matrix. This matrix can then be ported to the statistics package for input to the analysis. On the graphics side, it is possible to use the plotting routines available with statistical packages to obtain the point map. However, it may be difficult to obtain automatically the cluster drawings and any 3-D or pseudo-3-D plots which overlay ratings as shown in Figures 5 and 6. Listings of statements by cluster will almost certainly have to be accomplished either by re-entering the statements in cluster order or by editing the original file.
The concept mapping analyses can also be accomplished on microcomputers, although there are some trade-offs involved. For instance, there are few microcomputer implementations of multidimensional scaling available, although there are many graphics packages which might be easier to use than mainframe counterparts to produce the maps. Again, one could use virtually any word processing program for entering the statements (e.g., Microsoft WORD, WordPerfect). As with the mainframe option, the statements would need to be formatted for printing as sorting decks and as an instrument for the rating task. The SYSTAT program, available for both MS/DOS and Macintosh environments is one of the few microcomputer programs at this time which includes both multidimensional scaling and cluster analysis (although the cluster analysis package does not include Ward's algorithm). As with the mainframe option, the analyst will have to write a program to construct the group similarity matrix. The maps can be produced using a combination of plotting and painting programs. For instance, one might use CricketGraph on the MAC (or even the SYSTAT Graph option) to generate the x-y point plot and then load that picture into a painting program (e.g., SuperPaint, FullPaint, CANVAS) to manually draw cluster boundaries and construct pseudo-3-D rating plots like those in Figures 5 and 6. Again, lists of statements by cluster will have to be constructed by editing the original brainstormed statement file.
Because of the inconvenience of using general-purpose programs for accomplishing concept mapping, the author has written a computer program, called The Concept System specifically to accomplish this task. The program is available for both the MS/DOS and Macintosh microcomputer environments. The program is interactive and has separate menu-selected options for: entering the brainstormed statements; printing decks of cards or pre-formatted rating sheets; entering sort data (the data can be entered by cluster simply by typing in the ID numbers of the statements in any given cluster); entering the rating data; conducting the analysis (including construction of the similarity matrix, multidimensional scaling, cluster analysis, and averaging of ratings); and, interactive graphing of the results in a wide variety of ways to produce any of the maps discussed above. The Concept System allows the user to examine interactively any possible clustering solution, print a cluster tree and lists of statements by any selected number of clusters. The program limits the user to no more than 100 statements in any concept mapping project. Anyone interested in obtaining information about The Concept System program may do so by contacting the author directly. Virtually all of the projects reported in this volume utilized The Concept System to compute maps. While this is undoubtedly the easier way to accomplish concept mapping because the program was written specifically for this purpose, it is certainly also feasible to do so using available general-purpose programs as well.
Concept mapping of the type described here is designed to bring order to a task which is often extremely difficult for groups or organizations to accomplish. The process has several distinct advantages (many of which are discussed in greater detail in the other papers in this volume). First, it encourages the participant group to stay on task and to lay out relatively quickly a framework for a planning or evaluation study. Second, it expresses the conceptual framework in the language of the participants rather than in terms of the evaluator or planner's language or the language of social science theorizing. Third, it results in a graphic respresentation which at a glance shows all of the major ideas and their interrelationships. Fourth, this graphic product is comprehensible to all of the participants and can be presented to other audiences relatively easily. Finally, we have observed over many concept mapping projects that one of the major effects of the process is that it appears to increase group cohesiveness and morale. Especially in groups which have previously tried to accomplish conceptualizing through committee discussions, we have found that they readily appreciate the structure of the process and the ease with which it produces an interpretable starting point for subsequent evaluation or planning work.
This concept mapping process is by no means the only way in which group conceptualization can be accomplished nor is it necessarily the best way for any given situation. In situations where a group can achieve consensus relatively easily on their own or where a pictorial representation of their thinking is not desired or deemed useful, this approach would not be recommended.
As the other papers in this volume will show, we are only just beginning our efforts to devise sensible group conceptualization processes. There are still many methodological issues which need to be explored in order to improve the process described here, and that work is continuing. However, as it now stands, the concept mapping process is a useful procedure which helps a group to focus on the conceptualization task, and results in an easily understandable pictorial representation of their thinking which emerges from a group process which insures input from all participants.
Bickman, L. (Ed.). (1986). Using program theory in evaluation. New Directions for Program Evaluation. San Francisco, CA: Jossey-Bass.
Chen, H.T. and Rossi, P.H. (1983). Evaluating with sense: The theory-driven approach. Evaluation Review. 7, 283-302.
Chen, H.T. and Rossi, P.H. (1987). The theory-driven approach to validity. Evaluation and Program Planning. 10, 95-103.
Cordray, D.S. (1986). Quasi-experimental analysis: A mixture of methods and judgment. In W. Trochim. (Ed.). Advances in quasi-experimental design and analysis. New Directions in Program Evaluation. San Francisco, CA: Jossey-Bass.
Davison, M.L. (1983). Multidimensional Scaling. New York, NY: John Wiley and Sons.
Dunn, W. (1981). Public policy analysis: An introduction. Englewood Cliffs, NJ: Prentice Hall.
Einhorn, H.J. and Hogarth, R.M. (1986). Judging probable cause. Psychological Bulletin. 99, 1, 3-19.
Everitt, B. (1980). Cluster Analysis (2nd Edition). New York, NY: Halsted Press, A Division of John Wiley and Sons.
Krippendorf, K. (1980). Content analysis: An introduction to its methodology. Beverly Hills, CA: Sage Publications.
Kruskal, J.B. and Wish, M. (1978). Multidimensional scaling. Beverly Hills, CA: Sage Publications.
Mark, M.M. (1986). Validity typologies and the logic and practice of quasi-experimentation. In W. Trochim. (Ed.). Advances in quasi-experimental design and analysis. New Directions in Program Evaluation. San Francisco, CA: Jossey-Bass.
Novak, J.D. and Gowin, D.B. (1984). Learning how to learn. Cambridge, England: Cambridge University Press.
Osborn, A.F. (1948). Your creative power. Hew York, NY: Charles Scribner.
Rico, G.L. (1983). Writing the natural way: Using right-brain techniques to release your expressive powers. Los Angeles, CA: J.P. Tarcher.
Rosenberg, S. and Kim, M.P. (1975). The method of sorting as a data-gathering procedure in multivariate research. Multivariate Behavioral Research, 10, 489-502. Rossi, P.H. and Chen, H. (in press). Issues and overview of the theory-driven approach. Evaluation and Program Planning.
Shadish, W.R. Cook, T.D. and Houts, A.C. (1986). Quasi-experimentation in a critical multiplist mode. In W. Trochim. (Ed.). Advances in quasi-experimental design and analysis. New Directions in Program Evaluation. San Francisco, CA: Jossey-Bass.
Stone, P.J., Dunphy, D.C., Smith, M.S. and Ogilvie, D.M. (1966). The general inquirer: A computer approach to content analysis. Cambridge MA: The Massachusetts Institute of Technology.
Trochim, W. (1985). Pattern matching, validity, and conceptualization in program evaluation. Evaluation Review, 9, 5, 575-604.
Trochim, W. and Linton, R. (1986). Conceptualization for evaluation and planning. Evaluation and Program Planning, 9, 289-308.
Trochim, W. (in press) Pattern matching and program theory. Evaluation and Program Planning.
Wrightson, M. (1976). The documentary coding method. In R. Axelrod (Ed.). The structure of decision: The cognitive maps of political elites. Princeton, NJ: The Princeton University Press.
1) lack of public transportation
13) inadequate home care services
2) lack of money
75) in-home visits by physicians
78) clergy who can make home visits
Cluster 2
3) high cost of medication
4) high cost of health care
51) long-term care insurance
52) Medicare coverage being gradually "squeezed down"
91) high cost of home nursing care
53) lack of state and federal funding commitment to elderly
Cluster 3
24) lack of funding for community care
32) Medicare limitations
36) housing costs
70) access issues for disabled elderly
76) clergy that are aware of community resources
Cluster 4
26) confusing social service system
29) duplication of services
89) better understanding of elderly by legal profession
50) awareness of community resources
Cluster 5
14) lack of nursing homes
49) need for respite for caregivers
94) lack of support for family caregivers
18) emergency supervised housing
37) different housing alternatives
83) infantilization of elderly by providers
Cluster 6
27) exploitation
28) physical abuse and neglect by caretakers
Cluster 7
30) lack of outreach
33) nutrition services
34) expenses of maintaining family contacts
Cluster 8
25) lack of expertise to treat mentally ill elderly
95) law enforcement intervention when elderly refuse to be hospitalized
73) placement outside of home
39) over-medication by physicians
60) assistance with managing finances
Cluster 9
48) deterioration of family life
90) distress about the family issues of their children
67) care for elderly persons with dementia
93) depletion of resources by children
Cluster 10
5) stigmatization of elderly persons
61) job descrimination
66) premature categorization as "old" or elderly
Cluster 11
44) continuation of vocational satisfaction
69) improved use of educational opportunities
62) lack of education on life changes
74) means to maintain independence
65) how do some elderly function so well? (exceptional elderly)
92) role models for the elderly
64) desire to pass on traditions and knowledge/experiences
72) more diverse social opportunities
Cluster 12
35) need for political advocacy
58) desire for access to educational/recreational programs
63) exclusion of elderly as an educational resource
71) accessible education through creative programming
77) improved public awareness and support of elderly needs
43) expanding (exploding) elderly population
Cluster 13
11) late life role changes
45) lack of preparation for aging (including retirement)
57) feeling of being excluded by other generations or groups
21) inability to cope with change
85) giving elderly persons the right to choose to die
46) issues around self-worth and self-esteem
Cluster 14
16) loss of recreation
19) lack of socialization
59) desire to be treated like everybody else
79) understanding and respect for sexuality
41) need for independence
47) changing role in the community
Cluster 15
6) isolation
23) death of spouse
68) confusion between aging and illness
17) concern for mental wellness
54) fear of prolonged dying
86) fear of being abandoned
20) fears about not being able to protect themselves
Cluster 16
9) deterioration of mental capacity
38) over self-medication
12) depression
80) fear of being crazy
81) suicide among the elderly
55) concern over being a burden
56) feeling of uselessness
Cluster 17
7) dependency -- interpersonal
8) loss of close support
87) loss of legal control
40) inability to understand or read medication instructions
42) lack of awareness in mixing drugs and alcohol
82) alcoholism among the elderly
Cluster 18
10) physical disabilities
88) loss of financial control
15) decreased functional status
31) losing your driver's license
22) effects of the weather on the elderly
84) an institutional end (decline and death)