Albright & Thyer (2010) A Test of Validity of the LCSW Examination: Quis Custodiet Ipsos Custodes

 

Albright & Thyer (2010) A Test of Validity of the LCSW Examination: Quis Custodiet Ipsos Custodes

            This article describes a study conducted to evaluate the validity of the Licensed Clinical Social Worker Exam (LCSW), which is considered to be a reliable measure of social workers’ professional competency. The LCSW “pass/fail” exam is administered to social workers with varied levels of education and work experience (Associate, BSW, MSW, Advanced Generalist, and Clinical)[i] and is mandatory in most jurisdictions in the USA.  Administered since 1983, it is “… prepared, marketed and administered…” by the Association of Social Work Boards (ASWB), with the goal to “promote consumer protection” and it is seen as, “… one of the most important assurances that a social worker possesses the competence to practice responsibly”. In their study, Albright & Thyer (2010) administered a modified version of the clinical-level practice exam to 59 MSW students who were in their first year of studies. The exam was modified so that none of the 50 questions could be seen, leaving only the 4 multiple-choice answers related to the deleted questions, visible. Students were asked to guess the right answers. Whereas, “on the basis of chance alone” 25% of the answers should have been guessed correctly”, surprisingly, the students correctly guessed 52% of the answers, raising serious doubt about the ASWB’s claim that it is “a valid assessment of competence to practice social work”.  The results of their study highlight several important points about the importance of using measurements that are valid and psychometrically sound.

– Current literature is lacking in the area of licensing for social workers and social work examinations.  Publishing the psychometric properties behind this measure in professional journals would generate a clearer understanding of the process undertaken and help reassure “…members of the profession who have historically been suspicious of standardized licensing tests.”

– The ASWB asserts that the LCSW is not “… created in a vacuum”, and that the individuals items included in the 11 question categories have face and content validity. However, this is not enough the ensure validity as face and content validity are subjective in nature and are not generally considered to be persuasive by social scientists (Singleton & Straits, 2010). Criterion validity would be more appropriate.[ii]

– The items on the LCSW exam stem from the responses of licensed social workers who are surveyed about the types of tasks they perform, the importance they attribute to these tasks, and the extent to which they consider the task to be a necessary skill. The ASWB ensures that each “…item undergoes a statistical and sensitivity analysis by a group of expert social work professionals”, however no mention is made about the type of analysis and no details are available publically. In addition, the assurances made by the ASWB are not questioned by members of the social work profession. Rather, they are accepted “…as a matter of faith”.  Given the enormous weight attributed to the results of this test and its impact on a social worker’s professional trajectory, and considering the consumer protection objective of the exam, “… anything less than the scientific transparency provided by both internal and independently conducted and published psychometric evaluations…” is not enough.

Other concerns about the LCSW exam:

– As aforementioned, the ASWB is involved in all fundamental and implemental aspects of the LCSW exam, raising the question of Cuis custodiet ipsos custodes? “But who will guard the custodians themselves”?[iii]

– There are tremendous financial gains associated to the ASWB exam yet no financial disclosure on the part of the ASWB and this raises questions. As indicated on the Association of Social Work Boards official website, the current rate for the Clinical exam (used in this study) is $260.00 per exam. A printed study guide can be purchased for $38.00 and a practice exam is available at the cost of $85.00.  In 2013, ten thousand, eight hundred and three (10,803) social workers took the exam, a total cost of $4,137,549.00.  Of this group, approximately 2377 social workers (78%) did not pass the exam and had to retake it, at the full cost (approximately $618,020).  The potential earnings from this clinical cohort alone were $4,755,569.00.[iv]

For the curious and daring, here are 2 sample questions from the clinical-level exam: [v]

1. A six-year-old child lives with a foster family. His father is in prison and his mother is in residential treatment for alcohol dependence. The child is small for his age, often has temper outbursts, and has difficulty completing schoolwork. The social worker notes that his speech is immature. What should the social worker do FIRST?

(A) Work with the foster parents on a behavior modification plan

(B) Suggest that the child’s teacher refer him for special education placement

(C) Refer the child for assessment for fetal alcohol syndrome

(D) Work with the child’s biological mother toward reunification

 

2. A social worker is conducting a first interview with a client who attempts to dominate the interview from the beginning. The client complains that his telephone is tapped, and says that his house is watched by the police. How can the social worker BEST establish a beginning level of rapport with the client?

(A) Interrupt the client to ask factual questions about his background

(B) Ask the client about the ways in which the social worker can be helpful with these problems

(C) Question the client about when he first believed that his house was being watched

(D) Ask the client to describe the evidence he has that his phone is being tapped

 

Answers:  1, C; 2, B

How did you do? (Phew, I passed)

 

 

 


[i] Association of Social Work Boards official website.  Accessed on May 12, 2014 from:

http://www.aswb.org/exam-candidates/

[ii] Singleton Jr, R. A., & Straits, B. C. (2010). Measurement. In Approaches to social research. New York: Oxford University Press.

[iii] Yahoo Voices (2014). Ten Latin Proverbs Everyone Should Know.  Accessed on May 12, 2014 from:

http://voices.yahoo.com/10-latin-proverbs-everyone-know-6872742.html?cat=37

[iv] Association of Social Work Boards official website.  Accessed on May 12, 2014 from:

http://www.aswb.org/exam-candidates/

[v] See iv

 

Summer Camp for Social Scientists! Reflections Edition

Tonight I was lucky enough to attend a lecture titled “Studying Racial Discrimination and its Consequences: Where We Started, Where We Are, and Where We Should Go” by sociology Professor Tony Brown, who researches racism and health. It was deeply thought provoking in many ways, raising questions and responses in me regarding not only the subject matter, but also measurement theory, study design, and more generally how advanced social science contributes to wider contemporary public debate.

I am approaching the end of the first session here at the ICPSR Summer Program in Quantitative Methods of Social Research, and I’ve been spending a whole lot of time cramming matrix algebra and calculus, and working on advanced methods of assessing, fixing, and presenting linear models. Although I’ve had a few good discussions with fellow participants on ontology and epistemology, we’ve generally not had opportunities to discuss the real-world implications of the methods we’re studying.

This talk was a breath of fresh air. Professor Brown started by giving us a brief survey of the state of racial discrimination research, and highlighted the problems with construct and measurement development in the field. He also highlighted a huge gap in theoretically driven research outside the commonly used stress process model. He then shared with us some very exciting preliminary work he had done to move the study of racial discrimination forward, including: a move away from measuring scales of ‘unfair treatment’ without immediate attribution to the cause of the unfair treatment, to actually measuring perceived acts of racial discrimination. He also identified a sub-study he had done that measured racial trauma similarly to how PTSD is measured in the DSM-IV. Most interestingly to me, he identified a sub-study with research questions regarding discriminators (e.g. do white self-identified discriminators have higher levels of health and life satisfaction?).

His talk ended by asking us what we thought. This was preliminary work, and he was more than generous to not only share it with us, but ask our opinions on it. He wanted our answers. But for me, I came away with many questions. Namely, why do we accept poor measurement construction without pushing and developing better measures? And, why, after so many years of study, are we still stuck with poor explanations of the causes and effects of racism?

Overall, I was deeply inspired by this lecture, and Professor Brown’s work in general. It was so heartening to reaffirm that quantitative methods in social science can not only be tools to further general scientific knowledge but also be tools of advancing critical debate. In the words of Brown et al. (2009):

In conclusion, investigations of racial attitudes address more than an academic issue. At stake are contentious claims such as whether U.S. society is approaching racial parity, harmony, and integration; whether ameliorative policies are necessary; and whether government is obligated to engineer anti-racism. In the past, present, and probably well into the foreseeable future, these claims have pitted citizens against one another. Duration of such antagonisms proves persistence of the color line as our nation’s longest standing social problem and mandates continued study of racial attitudes, among whites and blacks and others.

Blog authors are solely responsible for the content of the blogs listed in the directory. Neither the content of these blogs, nor the links to other web sites, are screened, approved, reviewed or endorsed by McGill University. The text and other material on these blogs are the opinion of the specific author and are not statements of advice, opinion, or information of McGill.