This research was supported in part through NIMH Grant R01MH46712-01A1, William M.K. Trochim, Principal Investigator; and by the Center for Mental Health Services, Substance Abuse and Mental Health Services Administration and the US Dept. of Education, National Institute on Disability and Rehabilitation Research (Cooperative Agreement H133B00011), Judith A. Cook, Principal Investigator. The opinions expressed herein do not reflect the position or policy of any federal agency and no official endorsement should be inferred.
This paper describes the use of concept mapping to develop a pictorial multivariate conceptual frameworkof staff views of a program of supported employment(SE) for persons with severe mental illness. The SE program involves extended individualized supported employment for clients through a Mobile Job Support Worker (MJSW) who maintains contact with the client after job placement and supports the client in a variety of ways. All fourteen staff members of a psychiatric rehabilitation agency with assignments associated with the SE program took part in the process. They brainstormed a large number of specific program activity statements(N=96), sorted and rated the statements, and interpreted the map that was produced through multidimensional scaling and hierarchical cluster analysis . The resulting map enabled identification of four issues that should be included in any theory of SE programs -- the specific activity sequences that characterize the program itself; the pattern of local program evolution, the definition of program staff roles ; and the influence of key contextual factors such as the client's family or the program's administrative structure. The implications of concept mapping methodology for theory development and program evaluation are considered.
Over the past quarter century a shift has occurred from traditional institution-based models of care for persons with severe mental illness to more individualized community-based treatments. Along with this, the theories and methods that guide psychiatric care necessarily change to address these shifting realities. This is perhaps most apparent in the field of psychiatric rehabilitation with its emphases on individual consumer goal setting, skills training, job preparation and employment support (Cook, Solomon and Mock, 1989; Cook, Jonikas and Solomon, 1992). The theoretical models available to guide psychiatric rehabilitation programs are relatively new and theory-driven field evaluations of these models are rare or have only recently been initiated (Cook and Razzano, 1992; Cook, 1992). A certain amount of conceptual "looseness" exists in the psychiatric rehabilitation field, especially in the area of vocational rehabilitation, regarding what constitutes services under various service delivery models. To date there has been no significant systematic theoretical attempt to unify the disparate concepts involved in vocational rehabilitation services in order to develop a comprehensive perspective that might guide further empirical work.
This study demonstrates how psychiatric rehabilitation programs as they exist in practice might be represented theoretically. A relatively new methodology -- termed concept mapping -- is used to represent the implicit constructs and theories of agency staff members directly involved in service delivery. The concept maps that result can help inform theoreticians about the thinking of practicing professionals and can act as a framework for more rigorous evaluation of programs that are being implemented.
Over the past several decades, the theory of vocational rehabilitation has experienced several stages of evolution. Original models of vocational rehabilitation were based on the idea of sheltered workshop employment. Clients were paid a piece rate and worked only with other individuals who were disabled. Sheltered workshops tended to be "end points" for persons with severe and profound mental retardation since few ever moved from sheltered to competitive employment (Woest, Klein & Atkins, 1986). Controlled studies of sheltered workshop performance of persons with mental illness suggested only minimal success (Griffiths, 1974 ; Weinberg & Lustig, 1968) and other research indicated that persons with mental illness earned lower wages, presented more behavior problems, and showed poorer workshop attendance than workers with other disabilities (Whitehead, 1977; Ciardiello, 1981; Olshansky, 1973; Olshansky & Beach, 1974; 1975).
Partly in reaction to sheltered workshops, the field of psychiatric rehabilitation services developed a model called transitional employment (TE). TE uses a "train-then-place" approach with initial training on prevocational work crews, followed by a series of temporary community job placements at minimum wage or above, with support that gradually tapers as the client moves through jobs with increasing independence, pay, hours, and responsibilities (Dincin, 1975, Robinault & Weidinger, 1978).
In the 1980s, a new model of services called Supported Employment (SE) was proposed as less expensive and more normalizing for persons undergoing rehabilitation (Wehman, 1985). Using a "place-then-train" approach, the SE model emphasizes first locating a job in an integrated setting for minimum wage or above, and then placing the person on the job and providing the training and support services needed to remain employed (Wehman, 1985). Services such as individualized job development, one-on-one job coaching, advocacy with co-workers and employers, and "fading" support were found to be effective in maintaining employment for individuals with severe and profound mental retardation (Revell, Wehman & Arnold, 1984). The idea that this model could be generalized to persons with all types of severe disabilities, including severe mental illness, became commonly accepted (Wehman, 1982; Chadsey-Rusch & Rusch, 1986; Roessler, 1980).
Currently, the most popular vocational rehabilitation models for persons with severe mental illness are TE, SE, and variations on these. The major differences between the TE and SE models are: 1) TE is a train-then-place model whereas SE is a place-then-train model; 2) TE generally has a limit on the total amount of time clients can hold a particular position while SE has no limit; and, 3) TE jobs are filled by the agency as it sees fit and do not technically belong to the client while SE positions are held directly by the client.
Over time, some TE programs have moved to a more place-then-train SE approach, eliminating their prevocational crews and dropping transitional positions. Others have rejected SE, arguing for the necessity of transitional steps, for assessing clients' vocational strengths and weaknesses, and for creating a sense of immediate participation in productive, if unpaid, work. Still others have integrated principles of SE into ongoing TE programs, creating "hybrid" models that consist of an "array of services" (Cook, Jonikas & Solomon, 1992).
Brief description of a Mobile Job Support Worker
One such hybrid approach was developed at Thresholds, the site for the present study, which created a new staff position called the mobile job support worker (MJSW) and removed the existing six month time limit for many placements. MJSWs provide ongoing, mobile support and intervention at or near the work site, even for jobs with high degrees of independence (Cook & Hoffschmidt, 1993). Time limits for many placements were removed so that clients could stay on as permanent employees if they and their employers wished. The suspension of time limits on job placements, along with MJSW support, became the basis of SE services delivered at Thresholds.
Vocational rehabilitation programs vary in how they operationalize services or programs -- different programs often handle vocational, clinical, programmatic and administrative aspects differently. There is no comprehensive theory that considers the definitions of and relationships among these elements as they are implemented in practice, thus hampering efforts to evaluate program effects. One way to develop a more comprehensive theory would begin with observations of programs as implemented in the field and work inductively toward clearer delineation of the implicit theories that currently guide them. This inductive strategy requires a methodology that can elicit the central constructs of program staff members in a way that is both theoretically and operationally sensible.
Brief description of Concept Mapping
A methodology developed by Trochim (1989a) , called "concept mapping," seems well-suited for an inductive approach to developing such a theory, and is employed in this study. Concept mapping combines a group process (brainstorming, unstructured sorting and rating of the brainstormed items) with several multivariate statistical analyses (multidimensional scaling and hierarchical cluster analysis) and concludes with a group interpretation of the conceptual maps that result. This paper illustrates the use of concept mapping for developing a graphic conceptual framework of the views of the staff of a psychiatric rehabilitation agency that provides a vocational rehabilitation program of SE for persons with severe mental illness.
All participants were staff members at the Thresholds in Chicago, Illinois. Thresholds is a mental health service provider and its Research Institute serves as the site for a National Research and Training Center on Rehabilitation and Mental Illness, with several programs of ongoing mental health research and training. Thresholds has two urban rehabilitation service branches, one on the north side and one on the south side of Chicago. The focus for this study was on the agency's specific SE Program. This program is designed to provide extended individualized SE for clients through a Mobile Job Support Worker (MJSW) who maintains contact with the client after job placement and supports the client in a variety of ways, including going to the job site if necessary to work with the client, employer, or fellow employees. Along with the longer-term post-placement provision of SE through the MJSW once placed in a job, all program participants receive in-agency life skills and employment training and assistance in job development and placement. The program operates at both branches of the agency.
Participants consisted of all agency staff members identified with assignments relevant to the SE program. This included the directors of both branches, vocational staff who provide in-agency job skills training, job coaches who support clients at community work sites, and the MJSWs who provide post-placement employment support. Fourteen agency staff members met these criteria and attended the two concept mapping sessions; two were persons with disabilities.
The general procedure for concept mapping is described in detail in Trochim (1989a) . Examples of results of numerous concept mapping projects are given in Trochim (1989b) . The process implemented here was accomplished in two successive evening sessions in January 1992, with each session lasting about three hours. A portable IBM computer was used throughout the sessions along with an overhead projector panel so that all participants could observe the computer operations. All analyses were conducted and maps produced using the Concept System© computer software1 that was designed for this process.
Brief description of Specific Program Activity Statements.
Session 1: Generation and Structuring of Conceptual Domain. At the first session, participants generated statements using a structured brainstorming process (Osborn, 1948) guided by a specific focus prompt that limits the types of statements that are acceptable. The focus statement or criterion for generating statements was operationalized in the form of the instruction to the participants:
Generate statements (short phases or sentences) which describe specific
activities that are part of the supported employment program at Thresholds.
The general rules of brainstorming applied. Participants were encouraged to generate as many statements as possible (with an upper limit of 100); no criticism or discussion of other's statements was allowed (except for purposes of clarification); and all participants were encouraged to take part. The group brainstormed ninety-six statements in approximately a half-hour.
Participants were given a short break while the statements were printed and duplicated for use in the structuring stage. Structuring involved two distinct tasks, the sorting and rating of the brainstormed statements. For the sorting (Rosenberg and Kim, 1975; Weller and Romney, 1988), each participant was given a listing of the statements laid out in mailing label format with twelve to a page and asked to cut the listing into slips with one statement (and its identifying number) on each slip. They were instructed to group the ninety-six statement slips into piles "in a way that makes sense to you." The only restrictions in this sorting task were that there could not be: (a) N piles (in this case 96 piles of one item each); (b) one pile consisting of all 96 items; or (c) a "miscellaneous" pile (any item thought to be unique was to be put in its own separate pile). Weller and Romney (1988) point out why unstructured sorting (in their terms, the pile sort method) is appropriate in this context:
The outstanding strength of the pile sort task is the fact that it can accommodate a large number of items. We know of no other data collection method that will allow the collection of judged similarity data among over 100 items. This makes it the method of choice when large numbers are necessary. Other methods that might be used to collect similarity data, such as triads and paired comparison ratings, become impractical with a large number of items (p. 25
After sorting the statements, each participant recorded the contents of each pile by listing the statement identifying numbers on the back of the rating sheet. For the rating task, the brainstormed statements were listed in questionnaire form and each participant was asked to rate each statement on a 5-point Likert-type response scale in terms of how important the statement is to their idea of supported employment where 1=relatively unimportant (compared with the rest of the statements); 2=somewhat important; 3=moderately important; 4=very important, and, 5=extremely important. Because participants were unlikely to brainstorm statements that were totally unimportant with respect to SE, it was stressed that the rating should be considered a relative judgment of the importance of each item to all the other items brainstormed.
This concluded the first session. Between the two sessions, the sorting and rating data were entered into the computer, the MDS and cluster analysis were conducted, and materials were produced for the second session.
Brief description of Multidimensional Scaling.
Data Analysis. Examination of the data and preliminary results indicated problems in the sorts of several participants. For one of the fourteen participants, the sort was not completed. For two others, the sorts had a single pile with an unusually large number of statements in it (i.e., more than one-third of all brainstormed statements). This might have been due to participant misunderstanding of the sorting task, resistance to doing this task conscientiously, or it may legitimately represent the way they saw the categories. Weller and Romney (1988) point out that problems that can affect the final MDS configuration can arise when including data from persons who create larger, more generic categories ("lumpers") with data from those who create smaller categories with finer distinctions ("splitters"). With a small overall sample size, the inclusion of sort data from extreme lumpers could easily lessen the interpretability of the maps. Consequently, a decision rule was used where any sort that had a single pile including more than one-third of the brainstormed statements would be eliminated from the final analysis. According to this rule, two participant sorts were eliminated. In addition, the incomplete sort was also excluded, yielding a final sorting sample size of eleven participants. All fourteen participants completed the ratings and their ratings were included in the analysis.
The concept mapping analysis begins with construction from the sort information of an NxN binary, symmetric matrix of similarities, Xij. For any two items i and j, a 1 was placed in Xij if the two items were placed in the same pile by the participant, otherwise a 0 was entered (Weller and Romney, 1988, p. 22). The total NxN similarity matrix, Tij was obtained by summing across the individual Xij matrices. Thus, any cell in this matrix could take integer values between 0 and 11 (i.e., the 11 people who sorted the statements); the value indicates the number of people who placed the i,j pair in the same pile.
The total similarity matrix Tij was analyzed using nonmetric multidimensional scaling (MDS) analysis with a two-dimensional solution. The solution was limited to two dimensions because, as Kruskal and Wish (1978) point out:
Since it is generally easier to work with two-dimensional configurations than with those involving more dimensions, ease of use considerations are also important for decisions about dimensionality. For example, when an MDS configuration is desired primarily as the foundation on which to display clustering results, then a two-dimensional configuration is far more useful than one involving three or more dimensions (p. 58).
The analysis yielded a two-dimensional (x,y) configuration of the set of statements based on the criterion that statements piled together most often are located more proximately in two-dimensional space while those piled together less frequently are further apart.
Brief description of Hierarchical Cluster Analysis.
This configuration was the input for the hierarchical cluster analysis utilizing Ward's algorithm (Everitt, 1980) as the basis for defining a cluster. Using the MDS configuration as input to the cluster analysis in effect forces the cluster analysis to partition the MDS configuration into non-overlapping clusters in two-dimensional space. There is no simple mathematical criterion by which a final number of clusters can be selected. The procedure followed here was to examine an initial cluster solution that on average placed five statements in each cluster. Then, successively lower and higher cluster solutions were examined, with a judgment made at each level about whether the merger/split seemed substantively reasonable. The pattern of judgments of the suitability of different cluster solutions was examined and resulted in acceptance of the eighteen cluster solution as the one that preserved the most detail and yielded substantively interpretable clusters of statements.
The MDS configuration of the ninety-six points was graphed in two dimensions. This "point map" displayed the location of all the brainstormed statements with statements closer to each other generally expected to be more similar in meaning. A "cluster map" was also generated. It displayed the original ninety-six points enclosed by boundaries for the eighteen clusters.
The 1-to-5 rating data was averaged across persons for each item and each cluster. This rating information was depicted graphically in a "point rating map" showing the original point map with average rating per item displayed as vertical columns in the third dimension, and in a "cluster rating map" which showed the cluster average rating using the third dimension. The following materials were prepared for use in the second session:
Session 2: Interpretation of the Concept Maps. The second session convened to interpret the results of the concept mapping analysis. This session followed a structured process described in detail in Trochim (1989a) . The facilitator began the session by giving the participants the listing of clustered statements and reminding them of the brainstorming, sorting and rating tasks performed the previous evening. Each participant was asked to read silently through the set of statements in each cluster and generate a short phrase or word to describe or label the set of statements as a cluster. The facilitator then led the group in a discussion where they worked cluster-by-cluster to achieve group consensus on an acceptable label for each cluster. In most cases, when persons suggested labels for a specific cluster, the group readily came to a consensus. Where the group had difficulty achieving a consensus, the facilitator suggested they use a hybrid name, combining key terms or phrases from several individuals' labels.
Once the clusters were labeled, the group was given the point map and told that the analysis placed the statements on the map so that statements frequently piled together are generally closer to each other on the map than statements infrequently piled together. To reinforce the notion that the analysis placed the statements sensibly, participants were given a few minutes to identify statements close together on the map and examine the contents of those statements. After becoming familiar with the numbered point map, they were told that the analysis also organized the points (i.e., statements) into groups as shown on the list of clustered statements they had already labeled. The cluster map was presented and participants were told that it was simply a visual portrayal of the cluster list. Each participant wrote the cluster labels next to the appropriate cluster on their cluster map.
Participants then examined the labeled cluster map to see whether it made sense to them. The facilitator reminded participants that in general, clusters closer together on the map should be conceptually more similar than clusters farther apart and asked them to assess whether this seemed to be true or not. Participants were asked to think of a geographic map, and "take a trip" across the map reading each cluster in turn to see whether or not the visual structure seemed sensible. They were then asked to identify any interpretable groups of clusters or "regions." These were discussed and partitions drawn on the map to indicate the different regions. Just as in labeling the clusters, the group then arrived at a consensus label for each of the identified regions.
The facilitator noted that all of the material presented to this point used only the sorting data. The results of the rating task were then presented through the point rating and cluster rating maps. It was explained that the height of a point or cluster represented the average importance rating for that statement or cluster of statements. Again, participants were encouraged to examine these maps to determine whether they made intuitive sense and to discuss what the maps might imply about the ideas that underlie their conceptualization. Given this new context for meaning, cluster labels, regions and region labels were open for revision, although none were suggested. The remainder of the session was devoted to summarizing the process.
To illustrate the types of statements that were brainstormed and how these were clustered in the analysis, the statements comprising two of the final eighteen clusters are shown in Table 1 along with their identifying numbers and average importance ratings. 2
For the two-dimensional solution of the MDS analysis, the final Stress value was .31. The two-dimensional configuration of the ninety-six brainstormed statements is graphed in Figure 1.
In the figure, each statement is indicated by a dot with the statement identifying number next to it. Clusters of statements are enclosed in polygons and numbered. The statements given in Table 1 can be found in cluster 3 on the lower right and cluster 9 on the bottom of the map. There are several pairs of statements that fall in virtually the same place on the maps. If the assumptions underlying the analysis are correct, these should be very similar statements. For instance, on the left of the map, statements 77 (helping families adjust to member independence) and 78 (helping family members accept realistic placement of member) are located in the same place, as expected given their similar content. On the bottom of the map, statements 33 (providing vocational support groups) and 96 (having an employment group) overlap, while in the lower right, statements 7 (modeling job activities) and 13 (modeling appropriate behavior for the work environment) fall together. In all of these, the proximity of the statement pairs makes sense given the high degree of similarity in meaning.
The final concept map that resulted from the group interpretation is shown in Figure 2.
Here, each cluster is shown with its label. Clusters have been divided into four general regions that were also labeled by the participants. The number of layers used to depict the cluster borders indicates the average importance rating across all items in the cluster. A cluster border is drawn only to show the statement points that fall within that cluster -- there is no meaningful interpretation to the size or specific shape of a cluster. Although the distances among points and clusters are fixed in MDS, the directionality of the map is entirely arbitrary. The map could be rotated in any direction or flipped horizontally or vertically without changing the distances among items or clusters. In other words, there is no substantive meaning to the fact that the Administration region is located at the top while the Prevocational region is on the bottom of the page.
It is important to consider the reliability or consistency of the results of the concept mapping process. No single estimate of the reliability of this complex multi-step process is possible. However, the reliability or consistency of the key part of the concept mapping analysis -- from sorting through MDS -- can be examined in several ways.
For instance, an estimator analogous to an average inter-item correlation for the sort data can be computed by looking at the average interrelationships among the sorts of the participants. This can be accomplished by computing the contingency coefficient (McNemar, 1955) for all pairs of sorts. For the thirteen participants who had complete sort data (two of these were subsequently excluded from the final map analysis as described earlier), there are 78 pairs ((13x12)/2) for which the contingency coefficient can be estimated. The resulting coefficients ranged from .62 to .95, with an average of .85. All coefficients were statistically significant with p<.01 as determined by a chi-square test. The contingency coefficient is not without interpretative complication (Siegel, 1956), but nevertheless indicates that there is considerable consistency among the sorters.
Perhaps the best way to assess the reliability of the analysis is to estimate a coefficient analogous to a split-half reliability. Here, the eleven participants were randomly divided into two groups (five in one and six in the other). Similarity matrices were constructed separately for each random group, and separate MDS configurations were computed. The correlation between the two halves was estimated and the Spearman-Brown correction was applied to estimate reliability. The reliability estimate for the similarity matrices was .79 (df=4,559, p<.001) and the reliability estimate for the final MDS configuration was .56 (df="4,559," p<.001). These significant positive correlations imply that even with the extremely small number of participant sorts aggregated, there is a clear discernible statistical consistency in the results.
Two cases were excluded because their sorts contained at least one pile with over one-third of all the brainstormed statements. The effects of these exclusions can be examined in two ways. First, the correlation of the similarity matrices with and without exclusions was estimated. This correlation was .95 (n=4,560 pairs of 96 similarities, p<.001). Second, as one would expect, the relationship between the final MDS configurations with and without the exclusions was lower (r=".56," df="4,559," p<.001), although still highly significant. While excluding the two "lumpers" from the analysis may make the result more interpretable, the final maps with and without exclusions are highly and significantly correlated, suggesting that no substantial meaning was lost.
All three analyses taken together suggest that there is an internal consistency to the final map -- the results cannot plausibly be attributed to chance or systematic error.
The final map is complex and could be interpreted in a wide variety of ways. This discussion focuses on four salient aspects of the map and their implications for improving our understanding of the theory of SE and subsequent evaluations of SE programs.
Brief description of Specific Activity Sequences.
The Activity Sequence. The final map shows a progression of services moving from the bottom left quadrant counter-clockwise along the outside toward the right side. In this "activity sequence", SE has a prevocational phase (i.e., training) involving goal setting, money management, job preparation skills, and self-reliance training. Next, work logistics appear (i.e., placement), followed by the clusters of job coaching, job site liaison services, mobile job support, assessing/processing work experiences, and caseworker responsibilities (i.e., training). In the terms presented earlier, this sequence does not perfectly fit either of the two dominant psychiatric vocational rehabilitation models (i.e., the transitional train-then-place or the supported place-then-train). The model reflected in the map is more a hybrid of the two idealized models, probably best described as a "train-then-place-then-train" variation.
The map also describes the major concepts that make up the various training and placement activities in the sequence. For instance, the initial prevocational training includes the clusters "Goal Setting", "Money Management", "Self Reliance", and "Job Preparation and Skills." The specific brainstormed statements within each of these clusters describes operationalized activities involved in implementing those aspects of the program. For example, the Goal Setting cluster includes brainstormed statements 14 (recruiting members for job site) and 30 (discussing appropriate long and short term goals). The cluster Job Preparation and Skills, includes statements 11 (teaching a member to look for a job) and 35 (assisting with resume writing).
Thus, the map shows that a unique hybrid sequence of activities (train-then-place-then-train) represents the model followed in the program at Thresholds. In addition, the map suggests some of the major theoretical concepts (i.e., the clusters) involved in each of these three activity stages. Finally, the map provides considerable detail about how these concepts are operationalized in the program from the perspective of the staff members who provide the services.
Brief description of the Pattern of Local Program Evolution.
Local Program Evolution. The evidence in the map of a train-then-place-then-train hybrid model reflects the evolution of the SE program at this agency. As described in Cook and Razzano (1992) , Thresholds originally used a traditional train-then-place transitional employment (TE) model. Clients started out in a prevocational crew where they learned generic work skills such as punctuality and productivity while undergoing a series of vocational assessments designed to target work-related difficulties. At the same time, clients began participating in a program offering an extensive set of services, including social skills, housing and independent living training, leisure and recreation activities, medication management, and education services. Services were meant to complement each other and to prepare clients for employment in the community. This traditional train-then-place model of TE is described on the map by the four Prevocational clusters (train) and the cluster Work Logistics (place).
With the advent of supported employment, Thresholds produced its hybrid model by combining several elements of SE models with its ongoing TE program to produce the hybrid model. The two most important features added to the TE program were: 1) the suspension of time limits on job placements, and 2) the development of the Mobile Job Support Worker (MJSW) position to provide ongoing training and support after placement. A different map would be expected from an agency that introduced an SE program without a TE program already in place. In this case, one might expect that staff would abbreviate or omit the Prevocational region shown in the lower left of the map and begin with immediate job placement followed by specific skills training and workplace-based support, etc.
This is an important issue for both theory development and evaluation of SE programs. Simple distinctions between place-then-train and train-then-place models may not describe program practice well. In methodological terms, such simple theoretical distinctions do not ensure the construct validity of the treatment. Generalizing from hybrid programs grossly classified as either TE or SE may obscure theoretical implications and confuse the interpretation of observed results. The quality of cross-study meta-evaluations will rest in part on our ability to describe the central program activity sequences accurately.
In a larger sense, it is also worth noting that one can read the map from left to right as a description of the transition or evolution from facility-based care to community-based job placement. On the facility side are familiar traditional issues of family involvement, clinical feedback, and even staff training and development. On the far right are the newer community-based arenas related to the MJSW role, job coach and job site liaison issues, and employer concerns. The map depicts this facility-based to community-based dimensionality and suggests some of the key constructs that bridge between the two (e.g., work logistics).
Brief description of Staff Role.
Staff Roles. The map provides evidence that the staff perceives a strong relationship between case management (the Case Worker Responsibility cluster) and vocational services (the MJSW cluster). Traditionally, case management and vocational services are considered to be distinct and are delivered by separate staff members. The proximity of these clusters on the map suggests that staff do not perceive this role distinction. This finding is corroborated by Cook and Razzano's (1992, p. 37) analysis of three years of logged descriptions of MJSW support where the SE services included many of the services on the right-hand, outside quadrant of the map (e.g., casework, ,job site liaison, and providing verbal support as well as resolving vocational problems).
The close relationship of these two staff roles is important for the theory of SE programs. In the traditional SE model they are viewed as distinct, but there may be compelling reasons for combining them. As Cook and Razzano (1992) note, as clients with mental illness succeed at their jobs they may confront a set of social and interpersonal demands that require case management support. As a result, SE for persons with psychiatric disabilities may need to extend beyond skills training and workplace support (the vocational component) to other life areas reflecting the nature of rehabilitation as a multi-dimensional process. During the implementation phase of SE services at Thresholds, considerable time was spent discussing caseworkers' concerns that MJSWs would encroach on their turf, while MJSWs reported that clients were calling upon them for "non-vocational" needs (i.e., case management and clinical assistance). It does appear that these services are closely related both in the ways staff think about their jobs (as reflected in the maps) and in the amount of time they actually devoted to these services (as reflected in the logged data reported by Cook and Razzano, 1992) . Theories of SE need to address how these roles are defined now and could be redefined in the future.
Brief description of Contextual Factors.
Contextual Factors. The concept map shows two major areas that consist primarily of what might best be termed contextual factors -- psychosocial issues on the left side and administrative ones on the top. Both have important roles to play in any theory of SE.
The distance of the family and psychiatric issue clusters from both the Prevocational and the Vocational Service regions suggests that staff do not view them as intimately involved with SE services. The relatively strong importance rating given to these clusters suggests that staff view family and psychiatric issues as important in this context. Several investigators have commented on the exclusion of families in the process of psychiatric rehabilitation (Cook, 1988; Spaniol, Zipple and Lockwood, 1992). The Thresholds agency whose staff participated in the mapping has no formal mechanism for family involvement in vocational service delivery. This may explain why family issues were located in the extreme left-hand section of the map away from all other service clusters. The location of the family and psychiatric clusters near the Prevocational region suggests that if families and the traditional psychiatric treatment community do have a role to play in SE, it may be most important and relevant in the early formative stages of the transitional process. Or, if their involvement in later phases is desired, it must be explicitly structured into the community-based component because staff may not naturally see them as part of SE.
The other notable contextual area of the map concerns administrative issues. It comes as no surprise that program staff members would perceive administrative matters as integral to their idea of SE programs. What is perhaps surprising is that evaluations of social programs so seldom incorporate administrative matters into their theoretical frameworks. Chen (1990) emphasizes the need to do so, describing the implementation environment as one of six important domains that need to be included in program theory. The concept map identifies some of the major administrative areas of relevance. The specific brainstormed statements in the clusters begin to delineate how these areas are operationalized in the management of the program. Information of this nature is essential for the development of implementation theory for SE programs.
The concept map that resulted from this process suggests a number of central constructs that ought to be included in any theory about SE programs. These fall into four major areas. First, one must address the variety of activity sequences that characterize SE and related programs. In addition to the train-then-place (TE) and place-then-train (SE) models, it is especially important to investigate whether programs as implemented follow these idealized models or manifest more complex variations, as was evident in the program studied here. The theory should spell out the different possible sequences and delineate the specific types of activities that are implied by "train" and "place." Second, SE theory should consider the local program evolution that provider agencies follow in developing SE programs out of existing ones. Providers seldom start new programs in a vacuum. Those who develop SE programs will often continue to implement some current program efforts while phasing out others. The transition itself is likely to affect the nature of the resulting SE program and contribute to the development of various hybrid models, as was evident in the concept map developed here. Third, theory development must consider the implications of SE programs with respect to staff roles. The potential for role confusion or conflict is especially great between staff who represent newer community-based SE models and those who were engaged in more agency-based vocational programs. Role issues are also more likely to arise in agencies that are transitioning from other models to SE programs than in agencies that are developing totally new programs. Fourth, a theory of SE must include consideration of contextual factors that affect program development and implementation. Important among these factors would be the role of the family in SE, the influence of the broader health and mental health system, the influence of local employment factors, and the administrative structures and processes of the provider agencies.
Of course, there are likely to be other important elements for a theory of SE programs than the ones identified through this concept mapping process alone, although the four outlined above would almost certainly need to be included in any credible theory. This study was conducted at only one agency with a small group of staff members. Results may not be generalizable beyond that immediate context. Replication of the concept mapping process described here with the staffs of other SE providers would permit a better assessment of the validity of these findings. Replication with different relevant populations, especially other mental health and rehabilitation professionals, social scientists, family and community members, and the consumers of the services, would likely help identify additional key constructs for a comprehensive theory of SE programs.
Brief description of the implications of Concept Mapping Methodology.
The concept mapping process offers the consulting psychologist and other professional facilitators a practical tool that has broad applicability. In addition to its value for theory development as described in this study, concept maps can be used in planning the future implementation of a program. Participants might divide into small task groups to consider how to improve different aspects of a program. Each task group could be given one or two cluster areas from the map (concentrating first on those clusters that were rated most important). They might read the statements that fell into their assigned clusters and generate a few specific action statements that could help improve the quality of the program in that area. These can then be brought back to the group as a whole for consideration at a single meeting with a concrete action being taken on each recommended action statement (e.g., reject the action statement with reasons detailed; accept a modified form of the action statement with rationale for the changes; or accept the action as suggested). This type of process, with specific feedback, helps to reinforce the importance of staff input in assuring the ongoing quality of the program.
Concept mapping is also useful in periodic implementation assessment. On a regular basis (quarterly, semi-annually) program staff can meet to go over each of the major clusters on the map and discuss how well the program is being implemented. The map essentially acts as an organizing device or agenda for the discussion. It is used to review progress and spot implementation problems before they become serious. The maps might also be used directly with clients to review the program.
Concept mapping can be used in reporting and describing the program. Sometimes, it is useful to organize records (manual files, databases) based on the taxonomy of issues that appear on the map. Cluster topics can be major headings, with the statements in each cluster serving as potential folders or sub-files. Or, the structure of the map can be used when summarizing program activities. This can be done in graphic form on the map itself, or the cluster labels can be used as the elements in an outline or text presentation.
Finally, concept mapping can be used to enhance the evaluation of programs. The map provides a multivariate framework that describes the central constructs of the program and, through the specific brainstormed statements, suggests operationalizations that can be implemented or observed. Clusters on the map can be used as a guide for setting up program implementation checks that assess the degree to which the various activities are carried out in each area. Results from such checks can be displayed on the original map and compared with the importance ratings of staff to determine whether the activities deemed most important are most salient in actual implementation. Thus, the map can be viewed as an operationalization of program theory, essential for examining the construct validity of the cause (Cook and Campbell, 1979). An analogous process directed toward mapping program outcomes would improve our ability to assess the construct validity of the effect and make possible a much richer multivariate pattern matching approach (Trochim, 1989c; Trochim and Cook, 1992) to evaluating social programs.
Footnotes
1 The Concept System © computer software is available for IBM-PC and compatabile computers. The program is a complete user-friendly package for implementing the concept mapping process. It is used to enter brainstormed statements, print these for sorting and rating, enter sorting and rating data, conduct the statistical analysis (including multidimensional scaling and hierarchical cluster analysis) and display a wide variety of map results. The user can interact directly with the program when creating and examining maps. Information about the software may be obtained by writing to Concept Systems, P.O. Box 4721, Ithaca NY 14853 or calling (607) 257-2375.
2 This abbreviated listing is presented for illustrative purposes only. The complete list of all ninety-six brainstormed statatements and eighteen clusters can be obtained from the first author: William M.K. Trochim, Department of Human Service Studies, MVR Hall, Cornell University, Ithaca NY 14853.