Skip to main content
SpringerLink
Log in

Advantages and psychometric validation of proximal intensive assessments of patient-reported outcomes collected in daily life

  • Special Section: PROs in Non-Standard Settings (by invitation only)
  • Published:
Quality of Life Research Aims and scope Submit manuscript

An Erratum to this article was published on 08 April 2016

Abstract

Objectives

Ambulatory assessment data collection methods are increasingly used to study behavior, experiences, and patient-reported outcomes (PROs), such as emotions, cognitions, and symptoms in clinical samples. Data collected close in time at frequent and fixed intervals can assess PROs that are discrete or changing rapidly and provide information about temporal dynamics or mechanisms of change in clinical samples and individuals, but clinical researchers have not yet routinely and systematically investigated the reliability and validity of such measures or their potential added value over conventional measures. The present study provides a comprehensive, systematic evaluation of the psychometrics of several proximal intensive assessment (PIA) measures in a clinical sample and investigates whether PIA appears to assess meaningful differences in phenomena over time.

Methods

Data were collected on a variety of psychopathology constructs on handheld devices every 4 h for 7 days from 62 adults recently exposed to traumatic injury of themselves or a family member. Data were also collected on standard self-report measures of the same constructs at the time of enrollment, 1 week after enrollment, and 2 months after injury.

Results

For all measure scores, results showed good internal consistency across items and within persons over time, provided evidence of convergent, divergent, and construct validity, and showed significant between- and within-subject variability.

Conclusions

Results indicate that PIA measures can provide valid measurement of psychopathology in a clinical sample. PIA may be useful to study mechanisms of change in clinical contexts, identify targets for change, and gauge treatment progress.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Schwarz, N. (2012). Why researchers should think “real-time”: A cognitive rationale. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 22–42). New York: Guilford Press.

    Google Scholar 

  2. Reis, H. T., & Gable, S. L. (2000). Event-sampling and other methods for studying everyday experience. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods for social and personality psychology (pp. 190–222). New York: Wiley.

    Google Scholar 

  3. Nezlek, J. B. (2012). Diary methods for personality and social psychology. Los Angeles: Sage.

    Google Scholar 

  4. Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespaul, P., & van Os, J. (2009). Experience sampling research in psychopathology: Opening the black box of daily life. Psychological Medicine, 39(9), 1533–1547.

    Article  CAS  PubMed  Google Scholar 

  5. Shrout, P. E., & Lane, S. P. (2012). Psychometrics. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 302–320). New York: Guilford.

    Google Scholar 

  6. Kashdan, T. B., & Nezlek, J. B. (2012). Whether, when, and how is spirituality related to well-being? Moving beyond single occasion questionnaires to understanding daily process. Personality and Social Psychology Bulletin, 38, 1523–1535.

    Article  PubMed  Google Scholar 

  7. Nezlek, J. B., Sorrentino, R. M., Yasunaga, S., Otsubo, Y., Allen, M., Kouhara, S., & Shuper, P. (2008). Cross-cultural differences in reactions to daily events as indicators of cross-cultural differences in self-construction and affect. Journal of Cross-Cultural Psychology, 39, 685–702.

    Article  Google Scholar 

  8. Nezlek, J. B., & Gable, S. L. (2001). Depression as a moderator of relationships between positive daily events and day-to-day psychological adjustment. Personality and Social Psychology Bulletin, 27, 1692–1704.

    Article  Google Scholar 

  9. Wilhelm, P., & Schoebi, D. (2007). Assessing mood in daily life: Structural validity, sensitivity to change, and reliability of a short-scale to measure three basic dimensions of mood. European Journal of Psychological Assessment, 23, 258–267.

    Article  Google Scholar 

  10. Kimhy, D., Delespaul, P., Corcoran, C., Ahn, H., Yale, S., & Malaspina, D. (2006). Computerized experience sampling method (ESMc): Assessing feasibility and validity among individuals with schizophrenia. Journal of Psychiatric Research, 40, 221–230.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Granholm, E., Loh, C., & Swendsen, J. (2008). Feasibility and validity of computerized ecological momentary assessment in schizophrenia. Schizophrenia Bulletin, 34, 507–514.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Kimhy, D., Delespaul, P., Ahn, H., Cai, S., Shikhman, M., Lieberman, J. A., et al. (2010). Concurrent measurement of “real-world” stress and arousal in individuals with psychosis: Assessing the feasibility and validity of a novel methodology. Schizophrenia Bulletin, 36, 1131–1139.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Wheeler, L., & Reis, H. T. (1991). Self-recording of everyday life events: Origins, types, and uses. Journal of Personality, 59, 339–354.

    Article  Google Scholar 

  14. Csikszentmihalyi, M., & Larson, R. (1987). Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Disease, 175(9), 526–536.

    Article  CAS  PubMed  Google Scholar 

  15. Stone, A. A., & Shiffman, S. (1994). Ecological momentary assessment in behavioral medicine. Annals of Behavioral Medicine, 16, 199–202.

    Google Scholar 

  16. Ariely, D., & Carmon, Z. (2000). Gestalt characteristics of experiences: The defining features of summarized events. Journal of Behavioral Decision Making, 13, 191–201.

    Article  Google Scholar 

  17. Redelmeier, D. A., & Kahneman, D. (1996). Patients’ memories of painful medical treatments: Real-time and retrospective evaluations of two minimally invasive procedures. Pain, 66, 3–8.

    Article  CAS  PubMed  Google Scholar 

  18. Redelmeier, D. A., Katz, J., & Kahneman, D. (2003). Memories of colonoscopy: A randomized trial. Pain, 104, 187–194.

    Article  PubMed  Google Scholar 

  19. Kamarck, T. W., Shiffman, S., Muldoon, M. F., Sutton-Tyrrell, K., Gwaltney, C. J., Janicki, D. L., & Schwartz, J. E. (2007). Ecological momentary assessment as a resource for social epidemiology. In A. A. Stone, S. Shiffman, A. Atienza, & L. Nebeling (Eds.), The science of real-time data capture: Self-reports in health research (pp. 268–285). New York: Oxford University Press.

    Google Scholar 

  20. Carlson, E. B. (2001). Psychometric study of a brief screen for PTSD: Assessing the impact of multiple traumatic events. Assessment, 8, 431–441.

    Article  CAS  PubMed  Google Scholar 

  21. Caspi, Y., Carlson, E. B., & Klein, E. (2007). Validation of a screening instrument for posttraumatic stress disorder in a community sample of Bedouin men serving in the Israeli Defense Forces. Journal of Traumatic Stress, 20, 529–539.

    Article  Google Scholar 

  22. Carlson, E. B., Waelde, L., Palmieri, P. A., Smith, S. R., McDade-Montez, E., & Macia, K. (in press). Development and validation of the Dissociative Symptoms Scale. Assessment.

  23. Ehlers, A., & Clark, D. M. (2000). A cognitive model of posttraumatic stress disorder. Behaviour Research and Therapy, 38, 319–345.

    Article  CAS  PubMed  Google Scholar 

  24. Stanton, A. L., Kirk, S. B., Cameron, C. L., & Danoff-Burg, S. (2000). Coping through emotional approach: Scale construction and validation. Journal of Personality and Social Psychology, 78, 1150–1169.

    Article  CAS  PubMed  Google Scholar 

  25. Foa, E. B., Ehlers, A., Clark, D. M., Tolin, D. F., & Orsillo, S. M. (1999). The posttraumatic cognitions inventory (PTCI): Development and validation. Psychological Assessment, 11, 303–314.

    Article  Google Scholar 

  26. Bolton, E. E., Glenn, D. M., Orsillo, S., Roemer, L., & Litz, B. T. (2003). The relationship between self-disclosure and symptoms of posttraumatic stress disorder in peacekeepers deployed to Somalia. Journal of Traumatic Stress, 16, 203–210.

    Article  PubMed  Google Scholar 

  27. Cordova, M. J., Ruzek, J. I., Benoit, M., & Brunet, A. (2003). Promotion of emotional disclosure following illness and injury: A brief intervention for medical patients and their families. Cognitive and Behavioral Practice, 10, 358–371.

    Article  Google Scholar 

  28. Emmons, R. A., McCullough, M. E., & Tsang, J.-A. (2003). The assessment of gratitude. In S. J. Lopez & C. R. Snyder (Eds.), Positive psychological assessment: A handbook of models and measures (pp. 327–341). Washington, DC: American Psychological Association.

    Chapter  Google Scholar 

  29. Feldman Barrett, E., & Barrett, D. J. (1999). Experience sampling program. Boston: Boston College.

    Google Scholar 

  30. Jacobs, N., Nicolson, N., Derom, C., Delespaul, P., van Os, J., & Myin-Germeys, I. (2005). Electronic monitoring of salivary cortisol sampling compliance in daily life. Life Sciences, 76, 2431–2443.

    Article  CAS  PubMed  Google Scholar 

  31. Wichers, M. C., Myin-Germeys, I., Jacobs, N., Peeters, F., Kenis, G., Derom, C., et al. (2007). Evidence that moment-to-moment variation in positive emotions buffer genetic risk for depression: A momentary assessment twin study. Acta Psychiatrica Scandinavica, 115, 451–457.

    Article  PubMed  Google Scholar 

  32. Delespaul, P., deVries, M., & Van Os, J. (2002). Determinants of occurrence and recovery from hallucinations in daily life. Social Psychiatry and Psychiatric Epidemiology, 37, 97–104.

    Article  PubMed  Google Scholar 

  33. Myin-Germeys, I., van Os, J., Schwartz, J. E., Stone, A. A., & Delespaul, P. A. (2001). Emotional reactivity to daily life stress in psychosis. Archives of General Psychiatry, 58(12), 1137–1144.

    Article  CAS  PubMed  Google Scholar 

  34. Geschwind, N., Peeters, F., Jacobs, N., Delespaul, P., Derom, C., Thiery, E., et al. (2010). Meeting risk with resilience: High daily life reward experience preserves mental health. Acta Psychiatra Scandinavica, 122, 129–138.

    Article  CAS  Google Scholar 

  35. Nezlek, J. B. (2011). Mutilevel modeling for social and personality psychology. Los Angeles: Sage.

    Google Scholar 

  36. Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press.

    Book  Google Scholar 

  37. Dalenberg, C. J., & Carlson, E. B. (2012). Dissociation in posttraumatic stress disorder part II: How theoretical models fit the empirical evidence and recommendations for modifying the diagnostic criteria for PTSD. Psychological Trauma, 4, 551–559.

    Article  Google Scholar 

  38. Breslau, N., Kessler, R. C., Chilcoat, H. D., Schultz, L. R., Davis, G. C., & Andreski, P. (1998). Trauma and posttraumatic stress disorder in the community: The 1996 Detroit Area Survey of Trauma. Archives of General Psychiatry, 55, 626–631.

    Article  CAS  PubMed  Google Scholar 

  39. Carlson, E. B., Dalenberg, C. J., & McDade-Montez, E. (2012). Dissociation in posttraumatic stress disorder part I: Definitions and review of research. Psychological Trauma, 4, 479–489.

    Article  Google Scholar 

  40. Kaniasty, K. (2005). Social support and traumatic stress. PTSD Research Quarterly, 16(2), 1–7.

    Google Scholar 

  41. Cordova, M. J., Cunningham, L. L. C., Carlson, C. R., & Andrykowski, M. A. (2001). Social constraints, cognitive processing, and adjustment to breast cancer. Journal of Consulting and Clinical Psychology, 69, 706–711.

    Article  CAS  PubMed  Google Scholar 

  42. Christianson, S. (Ed.). (1992). Handbook of emotion and memory: Research and theory. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  43. Myin-Germeys, I. (2012). Psychiatry. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 636–650). New York: Guilford Press.

    Google Scholar 

  44. Stone, A. A. (2007). Thoughts on the present state of real-time data capture. In A. A. Stone, S. Shiffman, A. Atienza, & L. Nebeling (Eds.), The science of real-time data capture: Self-reports in health research (pp. 361–370). New York: Oxford University Press.

    Google Scholar 

Download references

Acknowledgments

We wish to thank Abbey Tillery, Marianne Kabour, Rob Wheeler, Donn Garvert, Luma Muhtadie, and Janet Neff for their work on the research and all of the research participants who generously contributed their time and effort to benefit others.  We thank John Nezlek for his comments and suggestions for statistical analyses. The contents do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.

Funding

This study was funded by the National Institute of Mental Health (MH69876).

Author information

Authors and Affiliations

  1. National Center for PTSD and VA Palo Alto Health Care System, Department of Veterans Affairs, Menlo Park, CA, USA

    Eve B. Carlson & Josef I. Ruzek

  2. Pacific Graduate School of Psychology, Palo Alto University, Palo Alto, CA, USA

    Nigel P. Field

  3. School of Psychology, University of New South Wales, Sydney, NSW, Australia

    Richard A. Bryant

  4. Alliant International University, San Diego, CA, USA

    Constance J. Dalenberg

  5. Departments of Psychiatry and Psychology, Boston University School of Medicine, Boston, MA, USA

    Terrence M. Keane

  6. National Center for PTSD and VA Boston Health Care System, Department of Veterans Affairs, Boston, MA, USA

    Terrence M. Keane

  7. Department of Surgery, Stanford University School of Medicine, Stanford, CA, USA

    David A. Spain

Authors
  1. Eve B. Carlson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nigel P. Field
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Josef I. Ruzek
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Richard A. Bryant
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Constance J. Dalenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Terrence M. Keane
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. David A. Spain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eve B. Carlson.

Ethics declarations

Conflict of interest

All authors declare they have no conflict of interest. There are no commercial products associated with this research.

Ethical approval

All aspects of the conduct of this study were in accordance with the ethical standards of the VA Palo Alto Healthcare System and the Stanford University Institutional Review Board and with the 1964 Declaration of Helsinki and its later amendments.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Carlson, E.B., Field, N.P., Ruzek, J.I. et al. Advantages and psychometric validation of proximal intensive assessments of patient-reported outcomes collected in daily life. Qual Life Res 25, 507–516 (2016). https://doi.org/10.1007/s11136-015-1170-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-015-1170-9

Keywords

Search

Navigation