The Development of a Validity Scale for the Dissociative Experience Scale (DES)

Malingering is an issue that most, if not all, professionals in the mental health field will encounter. For various reasons, clients present with symptoms that may be overstated, exaggerating symptom complexity or severity, just as an individual wishing stronger dosages of a medication might exaggerate physical or mental pain. Conversely, they may try to minimize symptoms, understating their level of distress to protect their own or another’s self-image, as an individual interested in gaining custody might downplay their own or their children’s pathology. Individuals who complete trauma self-reports, a typical method for assessing trauma, may distort their reports in a way that casts them in a more or a less favorable light, depending on the desired outcome (Berry, 2000). It is the responsibility of the mental health professional to review the information provided and utilize validity scales when available to ensure that the data they collect are reliable. In addition, it is important to understand how a trauma history may increase the likelihood of some validity scales being elevated. Psychologists and psychiatrists agree that assessing malingering is a critical aspect of clinical evaluation, particularly in forensic evaluation (Paulson, Straus, Bull, MacArthur, DeLorme, & Dalenberg, 2018).

Malingering consists of a purposeful attempt by an individual to exaggerate symptoms, or fake diagnoses in order to obtain personal gain (American Psychiatric Association, 2013). There are various reasons that individuals may malinger, including attempts to win civil compensation or disability, or efforts to avoid criminal responsibility (Weiss & Van Dell, 2017). Validity scales can provide some data about the possibility of malingering stemming from any of these sources; however, it is critical to note that one scale cannot be considered sufficient to determine someone is engaging in malingering.

Although “malingering” is the more common term for intentionally inaccurate responding (used in more than 1600 publications in PsycINFO in the last decade), some argue that “feigning” would be a more accurate term.  In many cases, there is no way to determine whether external incentive is the reason behind a given inaccurate response, which Rogers (2018) argues should be a prerequisite for the use of the term malingering. Although early uses of the word malinger did refer to soldiers who were feigning illness to shirk duty, the term is now used more broadly. Malingering is part of the nomenclature of many of the more common scales and tests designed to detect purposeful inaccuracy, such as the Test of Memory Malingering, the Malingering Probability Scale, or the Structured Inventory of Malingered Symptomatology. Although there are meaningful differences between individuals who are feigning symptoms as a cry for help and those exaggerating for external incentive, we will use the term malingering in order to match the most common usage.

Malingering and Dissociation

Malingering is often a reported source of concern to those administering the Dissociative Experiences Scale (DES; Bernstein & Putnam, 1993). The DES and its variants (e.g., DES-II, DES-C) are among the more commonly administered screeners for pathological dissociation. Research utilizing the DES is widespread, with over 2,200 hits on PsycINFO alone, thus lending credence to the notion that dissociation in both psychopathological and normative populations is predictive of a variety of important outcomes. Dissociation as a symptom of exposure to trauma (as underlined in the development of the Dissociative subtype of posttraumatic stress disorder [PTSD]), and as a central symptom in the more controversial trauma-related syndromes and disorders (e.g., Dissociative Identity Disorder) has led to the use of the DES in forensic settings, where those being tested may have incentive to over-report dissociative symptoms. Thus, the lack of a validity scale in the DES is a problem for those attempting to make use of the scale in research, forensic, and clinical settings.

In research, use of the DES in online surveys may be especially problematic in the absence of a reliable validity scale. One of the major challenges in administering the DES (or any other self-report measure) online is that respondents, in order to maximize payments for the number of surveys completed, try to complete online forms quickly, use “bots” to answer the questions, or lower the standards for accurate reporting, thus reducing the fidelity of responses. In addition, individuals who do not have an adequate level of English proficiency can complete online assessments without having a clear understanding of the questions (a problem not specific to the DES, but relevant to online survey use in general). These factors, among others, have enhanced the need for a DES validity scale in recent years, as “big data” questions are posed online by trauma researchers.

In creating a validity scale for the DES, we incorporated several approaches frequently used when evaluating validity:

Inconsistency. This method requires a specific construct to be assessed several times during the assessment to determine agreement. Six items were developed for the DES that directly contradicted statement posed in the original test.  For instance, the item “Even if I am watching television, it is very easy to get my attention” was designed to be negatively correlated with affirming a frequent experience of watching television or a movie and becoming so absorbed in the story that they were unaware of other events happening around them. A score from 0-6 was calculated for number of paired strong agreements.

Atypicality. Atypical questions are items that describe extremely rare symptoms that are known to be uncommon to the target population. These questions appear plausible to those attempting to malingering these symptoms. Six atypical items were developed by the Trauma Research Institute team for the DES and verified by three highly published authors on dissociation (e.g., some people find that when they think about the trauma they faint or have a headache).

Distorted Structure. This method involves observing a pattern in an individual’s full set of DES responses that is unusual or uncommon in dissociative individuals. Here we took advantage of the relationship between the known high base rate items of the DES (absorption items) and the known low base rate items (taxon items) indicating more serious or pathological forms of dissociation. Taxon scores at or above the level of absorption score, indicating elevation in more serious symptoms without an elevation in more common symptoms, thus are suspect

Unlikely Completion Time. By asking a group of PhD students to complete our survey and recording their completion times, we established a cutoff for time to completion that may indicate insufficient care in answering the questionnaire. To measure the validity of responses, participants who completed the survey one standard deviation faster than our PhD group were considered to have put forth less effort than required for this assessment.

English Fluency. We tested for English proficiency in these online surveys to evaluate likelihood that participants were able to understand the instructions and answer the survey accordingly. An 8^thgrade reading level vocabulary test was used according to the Spache Readability Formula. Respondents who received scores under 60% on the vocabulary test were considered potentially invalid.  Those who had internet protocol (IP) addresses that indicated residency in non-English speaking countries, were eliminated from the respondent pool.

In our initial study, we examined participants within four groups. A PTSD group (n= 30) was asked to complete the survey and answer the questions honestly. A second online community sample, recruited through Amazon MTurk, also was asked to answer the survey honestly (n= 66). An additional press for accuracy was included by stating that “giving dishonest answers on a medical survey is like giving contaminated blood in a blood donation. It can be extremely harmful for the scientific project.” In prior trauma work in our lab, when trauma status was known, this statement has been shown to enhance accuracy in our community samples. Participants in the malingering group (n= 59) were asked to pretend to be someone with dissociative symptoms and malinger on this survey. They were provided a brief description of dissociation. An additional monetary incentive of five dollars was offered to the best malingerers. Lastly, participants in the “careless” group (n= 93) were asked to treat this task as they would if they were making a living from answering these surveys, completing the survey as fast as they could while maintaining accuracy. After all participants were given their respective instructions, a multiple choice question assessed whether the participant was able to identify their instruction set.

In our first draft of this validity scale, preliminary findings revealed that differentiation between the groups was possible using Atypicality, Inconsistency, Distorted Structure, and Unlikely Completion Time. The four groups significantly differed on endorsement of atypical items, with the malingering group differing from the other groups. The forced malingering group had a higher inconsistency mean and a smaller difference between the taxon and absorption items than did the two honest groups, with the careless group scoring between the honest and malingering samples. Importantly, the PTSD group, who received a score on the DES well above the normative mean, had a false positive rate (falsely indicating that the individual was malingering) under 10%. A logistic regression was able to correctly classify 84.8% of the honest participants and 62.7% of the known malingerers. In the final publication version, we intend to double the sample size and refine the atypical and inconsistency items.  We are pleased with the initial success of the scale, and hope that it will be of value to other researchers and clinical evaluators.  We would also like to note, however, that the DES is a screening tool for dissociative symptoms, and would not be a definitive diagnostic tool for identifying dissociative disorders with or without our added validity indicators.  Ideally, if malingering is suspected after use of our methods, further malingering assessment would investigate varying reasons for the likely inaccurate responding (e.g., the Test of Memory Malingering, the Structured Inventory of Malingered Symptomatology).

Summary and Conclusions

Malingering is an issue across many fields of psychological research, yet it is unclear how often malingering is actually assessed when conducting online research. A recent search of PsycINFO yielded 1,278 hits for the combined search terms of “MTurk, Mechanical Turk, or Qualtrics,” most in the last 5 years, suggesting the importance of question. In 50 randomly chosen MTurk survey studies, 28 included no malingering or accuracy assessment at all, with 4 more using only checks for repeated IP addresses.  Most of the remaining studies either used at least one test with an existing validity scale, or embedded consistency or attention measures. We suggest that our general method might be used to address inaccuracy issues in many online survey areas.

An additional consideration that emerged in the course of this study was the prevalence of suspect IP addresses. For researchers who wish to limit their sample to U.S. citizens or residents and/or to native English speakers, the presence of non-U.S. IP addresses among survey respondents should be concerning. A check of IP addresses should be a standard practice during data cleaning. In our initial MTurk sample, 14% of the respondents had IP addresses from non-English speakers countries, with over 50% of this subgroup failing one or more of the malingering or inaccuracy screens. An additional 8% did not pass the English vocabulary test.

We recommend that researchers review the assessments they use and ensure that a validity/malingering check is a part of their procedures or, in the absence of a developed scale, include one or more of the procedures noted here. Appending such a subscale to online assessment batteries is easy and efficient (that is, adds very little time to the assessment) and may partially protect the research effort from careless respondents, purposely inaccurate respondents, and bots. Additionally, a standard component of data cleaning should include an assessment of the IP addresses of all participants, potentially eliminating subjects if their address raises concerns over citizenship, residency, language ability, or other related issues. The DES would benefit from an embedded validity scale given its rate of use and clinical utility when assessing dissociative symptoms. As it stands, there is no scale within the DES that would reliably indicate instances of overreporting dissociative symptoms, leaving it vulnerable to symptom exaggeration, especially with the advancement of administering assessment measures online. Inclusion of an embedded validity scale would be an asset, and could potentially provide for greater clinical utility of the DES.

References

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.

Berry, D. T. R. (2000). Malingering. In Encyclopedia of psychology, Vol. 5.(pp. 97–99). Washington, DC; New York, NY: American Psychological Association. doi:10.1037/10520-044

Olsen, S. A., Clapp, J. D., Parra, G. R., & Beck, J. G. (2013). Factor structure of the Dissociative Experiences Scale: An examination across sexual assault status. Journal of psychopathology and behavioral assessment, 35(3), 394-403.

Paulson, K. L., Straus, E., Bull, D. M., MacArthur, S. K., DeLorme, J., & Dalenberg, C. J. (2018). Knowledge and views of psychological tests among psychiatrists and psychologists. Journal of Forensic Psychology Research and Practicedoi:10.1080/24732850.2018.1546071

Rogers, R., & Bender, S. D. (Eds.). (2018). Clinical assessment of malingering and deception. Guilford Publications.

Weiss, K.J. & Van Dell, L. (2017). Liability for diagnosing malingering. Journal of the American Academy of Psychiatry and the Law Online, 45(3), 339-347.

Ana Abu-Rus, M.A. is a Clinical Psychology Doctoral Student at the California School ofProfessional Psychology, San Diego and a member of the Trauma Research Institute headed by Dr. Constance Dalenberg. During her seven years as an Air Traffic Controller in the Air Force, Ms. Abu-Rus wasan instructor charged with teaching fellow militarymember’s skills to increase psychological resilience. Hercurrent research is focused on psychological resilience post-trauma and its relationship to physiologicalresilience such as heart rate variability. Her clinicalinterests are centered around trauma treatment in the veteran population and neuropsychological functioning associated with trauma.

Kenneth J. Thompson, M.L.A., is a clinical psychology doctoral student at the California School of Professional Psychology, San Diego, and a member of the Trauma Research Institute headed by Dr. Constance Dalenberg. Previously, he served in the Marine Corps, where he held a variety of senior leadership and management positions, and retired as a lieutenant colonel. His current research concerns thepsychosocial underpinnings of alexithymia. His clinicalinterests include trauma treatment and veteran mental health care.

Cassie Brown is a current Ph.D. student at the California School of Professional Psychology atAlliant International University in San Diego, CA.She is studying clinical psychology with an emphasisin trauma focused forensic psychology. Originally from the San Francisco Bay Area, she completed her undergraduate education at San Francisco State University, where she received a Bachelor of Arts in psychology. Cassie’s research interests include neuropsychology, forensic mental health, trauma, malingering, and memory, and her clinical interests include forensic evaluations and trauma-informed intervention with forensic populations.

Monica Ortiz, M.A. is a Clinical PsychologyDoctoral student at Alliant International University,California School of Professional Psychology. She is amember of the Trauma Research Institute, which is led by Dr. Constance Dalenberg Ph.D. Mrs. Ortiz’ currentresearch interest is on Post-Traumatic Stress Disorderand dissociation in military members and veterans. As a Health Emphasis student, her goals are to collaborate with members of the medical field, and researchers, to improve the psychological well-being of current and prior military members.

Dr. Constance Dalenberg is a clinical and forensic psychologist. She is a Distinguished Professor ofPsychology at Alliant International University, where she proudly directs the Trauma Research Institute. Sheis former President of Division 56 and Associate Editorfor Psychological Trauma. Her work has focused ontreatment, assessment and consequences of trauma, and her seminars on countertransference and treatmentof trauma have been presented nationally andinternationally. Professional recognition has includedthe Morton Prince Award for Scientific Achievement from the International Society for the Study of Trauma and Dissociation, and the Lifetime Achievement Awardfrom Division 56, American Psychological Association.