Back to Contents
In the representation step, the sorting and rating data were entered
into the computer, the MDS and cluster analysis were conducted,
and materials were produced for the interpretation step.
The concept mapping analysis begins with construction from the sort information of an NxN binary, symmetric matrix of similarities, Xij. For any two items i and j, a 1 was placed in Xij if the two items were placed in the same pile by the participant, otherwise a 0 was entered (Weller and Romney, 1988, p. 22). The total NxN similarity matrix, Tij was obtained by summing across the individual Xij matrices. Thus, any cell in this matrix could take integer values between 0 and 11 (i.e., the 11 people who sorted the statements); the value indicates the number of people who placed the i,j pair in the same pile.
The total similarity matrix Tij
was analyzed using nonmetric multidimensional scaling (MDS) analysis
with a two-dimensional solution. The solution was limited to two
dimensions because, as Kruskal and Wish (1978) point out:
Since it is generally easier to work with two-dimensional
configurations than with those involving more dimensions, ease
of use considerations are also important for decisions about dimensionality.
For example, when an MDS configuration is desired primarily as
the foundation on which to display clustering results, then a
two-dimensional configuration is far more useful than one involving
three or more dimensions (p. 58).
The analysis yielded a two-dimensional (x,y) configuration of
the set of statements based on the criterion that statements piled
together most often are located more proximately in two-dimensional
space while those piled together less frequently are further apart.
This configuration was the input for the hierarchical cluster
analysis utilizing Ward's algorithm (Everitt, 1980) as the basis
for defining a cluster. Using the MDS configuration as input to
the cluster analysis in effect forces the cluster analysis to
partition the MDS configuration into non-overlapping clusters
in two-dimensional space. There is no simple mathematical criterion
by which a final number of clusters can be selected. The procedure
followed here was to examine an initial cluster solution that
on average placed five statements in each cluster. Then, successively
lower and higher cluster solutions were examined, with a judgment
made at each level about whether the merger/split seemed substantively
reasonable. The pattern of judgments of the suitability of different
cluster solutions was examined and resulted in acceptance of the
fifteen cluster solution as the one that preserved the most detail
and yielded substantively interpretable clusters of statements.
The MDS configuration of the ninety-six points was graphed in
two dimensions and is shown in Figure 1. This "point map"
displayed the location of all the brainstormed statements with
statements closer to each other generally expected to be more
similar in meaning. A "cluster map" was also generated
and is shown in Figure 2. It displayed the original ninety-six
points enclosed by boundaries for the eighteen clusters.
The 1-to-5 rating data was averaged across persons for each item
and each cluster. This rating information was depicted graphically
in a "point rating map" (Figure 3) showing the original
point map with average rating per item displayed as vertical columns
in the third dimension, and in a "cluster rating map"
which showed the cluster average rating using the third dimension.
The following materials were prepared for use in the second session:
(1) the list of the brainstormed statements grouped by cluster
(2) the point map showing the MDS placement of the brainstormed statements and their identifying numbers (Figure 1)
(3) the cluster map showing the eighteen cluster solution (Figure 2)
(4) the point rating map showing the MDS placement of the brainstormed statements and their identifying numbers, with average statement ratings overlaid (Figure 3)
(5) the cluster rating map showing the eighteen cluster solution,
with average cluster ratings overlaid
The final stress value for the multidimensional scaling analysis
was .2980101.
Methods for estimating the reliability of concept maps are described
in detail in Trochim (1993). Here, six reliability coefficients
were estimated. The first is analogous to an average item-to-item
reliability. The second and third are analogous to the average
item-to-total reliability (correlation between each participant's
sort and the total matrix and map distances respectively). The
fourth and fifth are analogous to the traditional split-half reliability.
The sixth is the only reliability that examines the ratings, and
is analogous to an inter-rater reliability. All average correlations
were corrected using the Spearman-Brown Prophesy Formula (Weller
and Romney, 1988) to yield final reliability estimates. The results
are given in Table 2.
Back to Contents