Week 1 Discussion 2

Introduction to Assessment Instruments

After reading the Patton and McIlveen (2009) article and paying particular attention to the Assessment section, critique at least one of the career assessments currently being utilized in the career counseling field. Address the following questions in your response: What specifically is the instrument used for? In what situations might this instrument be most appropriate to use? What are the strengths of the instrument? What, if any, weaknesses have been identified? You may need to conduct follow-up research to adequately address these questions. Post should be at least 300 words. Respond to at least two of your classmates’ postings by Day 7.

Career Assessment Interests As previous annual reviewers have found (see Chope), the study of interests remains one of the most researched areas in the field, with many studies over the year advancing and refining assessment instruments or creating modifications for particular cohorts. Leierer, Strohmer, Blackwell, Thompson, and Donnay developed an occupational scale for the Strong Interest Inventory (SII) using data from rehabilitation counselors. The newly developed Rehabilitation Counselor Scale (included as part of the newly revised SII) proved to be successful in discriminating the pattern of interests of rehabilitation counselors from interests of people in general. The authors reported appropriate psychometric data for the new scale and suggested implications for its use in confirming rehabilitation counseling as a distinct profession and, therefore, in identifying and recruiting new people into the profession. The distinctiveness of the interest patterns should also assist people to identify their own interest fit before committing to educational programs for entry to the profession. In a complementary article, Leierer, Blackwell, Strohmer, Thompson, 124 The Career Development Quarterly December 2009 • Volume 58 and Donnay developed prototypical SII profiles for male and female rehabilitation counselors. As would be expected, these profiles drew on people skills, verbal abilities, listening and understanding capacities, and a preference for settings that foster creativity. The SII also featured in a study by Bailey, Larson, Borgen, and Gasser, who conducted the first reported study to examine the equivalence of the 1994 and 2005 versions of the SII using the parallel content scales. These scales included the six General Occupational Themes (GOTs), 22 ofthe 25 Basic Interest Scales (BISs), and four of the Personal Style Scales (PSSs). The scale correlations between the two versions were equal to or greater than .85 for the GOTs and all but one ofthe PSSs. The BIS correlations ranged from .64 to .97. The Personal Globe Inventory (PGI; Tracey, 2002) was the subject of two articles that evaluated the Spherical Model of Interests in both Serbia (Hedrih) and Croatia (I. Sverko). Hedrih’s data were collected from participants of different ages, educational levels, and genders and demonstrated good fit to Tracey’s model as well as Holland’s hexagonal model. I. Sverko also gathered data on the PGI from late primary school, late secondary school, and university students. The data from this study demonstrated several key findings that not only contribute to the assessment literature and data on the PGI but also add to the body of knowledge on the structure of interests across cultures. I. Sverko reported that the Croatian data confirmed three factors that were underlying interest items. In addition, the spherical representation and structural stability of interests across age and gender groups in Croatian adolescents and young adults was supported. Sodano and Tracey also furthered the work of the PGI in exploring the place of prestige in interest activity assessment. Two samples of college students rated each activity item from the PGI for prestige, effort required, skill required, competition involved, and female and male sex typing. The analyses of the data included matching of ratings with the theoretical structure for the first sample and the empirical structure of PGI items for the second sample. In both samples, following both sets of analyses, “the PGI prestige dimension was highly related to ratings for prestige, effort, skill, and competition, but unrelated to sex-typing” (Sodano & Tracey, p. 310). Complementary studies by Liao, Armstrong, and Rounds and by Armstrong, Allison, and Rounds addressed the limitations in the use of interest measures based on Holland’s RIASEC (i.e.. Realistic, Investigative, Artistic, Social, Enterprising, and Conventional) types that have been commercially developed and that are often lengthy and subject to copyright restrictions. Both studies report on research that aimed to develop and validate publicly available short versions of the measures to facilitate their greater usefulness in research settings. Liao et al. developed public domain Basic Interest Markers (BIMs), which are freely available on the U.S. Department of Labor’s Occupational Information Network (see O*NET Interest Profiler; http://www.onetcenter. org/IP.html). These items were reviewed through numerous phases of test development (e.g., review by small numbers of students, completed by several samples of college students). Reliability and several forms of validity data are reported, with the authors concluding that the freely available BIMs are psychometrically sound. Implications for research The Career Development Quarterly December 2009 • Volume 58 125 and for applied settings are provided. In recognition of the prominence of the Holland RIASEC model in vocational psychology, the authors emphasized the need for a follow-up study to develop a set of RIASEC markers to complement the BIMs. This follow-up work is reported by Armstrong, Allison, and Rounds, who discuss the development of short-form RIASEC scales for the public domain. Two sets of eight item scales to examine activity-based RIASEC assessment and two sets of eight-item scales to examine occupation-based RIASEC assessment were developed. The preliminary data support the utility of the short form scales, indicating acceptable levels of reliability and convergent validity with other interest measures. Additional work on Holland’s model of vocational interests was conducted by Gupta, Tracey, and Gore. These authors used the Unisex Edition of the ACT Interest Inventory to test the structural validity of Holland’s model across racial/ethnic groups, using data from high school juniors in two states of the United States. The fit of the Circumplex Model was evaluated for the general sample and five subgroups: Caucasians/Euro-Americans, African Americans, Asian Americans, Latinos, and Native Americans. The article provides data on four methods of analysis to test circumplex structure: randomization test of hypothesized order relations, constrained multidimensional scaling, circular unidimensional scaling, and structural equation modeling. In general, although these different analyses provided various levels of support for the model fit, no differences in fit of the Circumplex Model were found across the ethnic groups in the sample. Hirschi and Läge (b) explored the empirical relationship between the accuracy of adolescents’ self-estimates of their interest test scores and career choice readiness. Accuracy of students’ self-estimation was moderate to high, and female students were better able to predict their interest type. However, overall, the results failed to demonstrate a relationship between accuracy of self-estimation and career choice readiness.

Other Assessment Instruments A range of other assessment instruments was featured in research conducted during 2008. Zytowski’s revised version of Super’s Work Values Inventory was examined in a study reported by Robinson and Betz with an aim to extend the very early psychometric data on the revised scale. Data obtained from a mixed sample of college students provided support for the revised scale’s internal consistency, construct validity, and discriminant validity. The authors emphasize that further work is necessary with broader and larger samples as well as further examination of the stability of the measure and its concurrent and predictive validity. Amir, Gati, and Kleiman developed and tested a procedure for interpreting individuals’ responses in assessments with multiple scales. The authors used the Career Decision-Making Difficulties Questionnaire as an example to attempt to clarify the process for Internet and paper-andpencil administration to enable clearer understanding and interpretation of survey responses. Expert judges were used in a number of studies to develop and apply the process. Implications for both career counseling and research are proffered. In line with what has appeared as a common theme in the studies reviewed for this section—developing short forms of assessment measures—Rigotti, Schyns, and Mohr developed a short six-item version of the Occupational 126 The Career Development Quarterly December 2009 • Volume 58 Self-Efficacy Scale. The six-item version showed sound psychometric properties. Data were gathered fi-om employees in five countries to examine construct validity. In addition, hypotheses exploring the relationship between occupational self-efficacy and performance, job satisfaction, commitment, and job insecurity were supported in the expected direction. Hofmans, Dries, and Paperman’s gathered data from Dutch managers to examine gender differences in response to the Career Satisfaction Scale. The authors reported that gender invariance does not hold and provided considerable detail about measurement invariance in the scale and suggestions for operationalizing and measuring subjective career success. With a similar focus. Bowling and Hammond conducted a meta-analysis to examine the construct validity of the Job Satisfaction subscale (JSS) of the Michigan Organizational Assessment Questionnaire (MOAQ). The authors reported acceptable reliability, construct validity, and antecedents of and correlations with a number of constructs as per their hypotheses (e.g., “Job characteristics, social and organizational support, and person environment fit were positively related to and job Stressors were negatively related to the MOAQ-JSS,” Bowling & Hammond, p. 72). The following article has been included here, although it does not focus on a particular assessment scale. Rather the article by Ganster provides a comprehensive review of work stress measurement, with a focus on mental and physical well-being outcomes. Ganster discusses the theoretical and substantive literature related to work stress and provides a summary of the measurement literature, including a discussion on issues regarding self-report versus observational ratings. He offers suggestions on ways to proceed with job stress research. Two other articles focus on sense of coherence and career thoughts for individuals with disabilities (Lustig & Strauser) and a psychometric evaluation of a French version of the Career Transition Inventory (CTI-F; Fernandez, Fouquereau, & Heppner). Lustig and Strauser reported that a strong sense of coherence correlates with lower levels of dysfunctional career thoughts for individuals with disabilities, although they noted that these individuals had lower levels of sense of coherence and higher levels of dysfunctional career thoughts than did a sample of college students without disabilities. Suggestions for rehabilitation counselors were proffered. Fernandez et al. reported sound psychometric properties for the CTI-F and noted that the development of” this measure affords opportunities for cross-cultural research in this area. Finally, in this section, we review the article by Zytowski, who provides a comprehensive history of test scoring. He discusses such topics as the pragmatics of test and score sheet provision; scoring by the test taker for interpretation by the independent expert and the inherent issue of error; and forms of machine scoring and, ultimately, paperless testing through the use of the Internet. With the latter approach, tests can be completed and outcomes provided almost immediately. Zytowski concludes that “the highly developed contemporary status of psychological and educational assessment now rests as much on the technology of test scoring as it does on the development of psychometric science” (p. 509).

Career Assessment: Summary and Conclusion Numerous themes are evident in the review of the assessment literature. Many instruments are being developed in different languages for more The Career Development Quarterly December 2009 • Volume 58 127 effective cross-cultural evaluation as well as a more global understanding of career constructs and their application. The need for more efficient and cost-effective instruments for widespread use has led researchers to find alternatives to long and expensive commercially available tests. To this end, short forms of instruments are being developed and tested for psychometric appropriateness. Short forms are also necessary because researchers and counselors want to gather data on multiple constructs in one sitting. The increase in the use of the Internet and the provision of instruments for free public access are also driving this work. Finally, interests remain a primary focus in the assessment literature