File(s) not publicly available
Using dynamic factor analysis to provide insights into data reliability in experience sampling studies
journal contribution
posted on 2017-09-01, 00:00 authored by Matthew Fuller-TyszkiewiczMatthew Fuller-Tyszkiewicz, Linda Hartley-Clark, Robert CumminsRobert Cummins, Adrian TomynAdrian Tomyn, Melissa Weinberg, Ben RichardsonThe past 2 decades have seen increasing use of experience sampling methods (ESMs) to gain insights into the daily experience of affective states (e.g., its variability, as well as antecedents and consequences of temporary shifts in affect). Much less attention has been given to methodological challenges, such as how to ensure reliability of test scores obtained using ESM. The present study demonstrates the use of dynamic factor analysis (DFA) to quantify reliability of test scores in ESM contexts, evaluates the potential impact of unreliable test scores, and seeks to identify characteristics of individuals that may account for their unreliable test scores. One hundred twenty-seven participants completed baseline measures (demographics and personality traits), followed by a 7-day ESM phase in which positive and negative state affect were measured up to 6 times per day. Analyses showed that although at the sample level, scores on these affect measures exhibited adequate levels of reliability, up to one third of participants failed to meet conventional standards of reliability. Where these low reliability estimates were not significantly associated with personality factors, they could-in some cases-be explained by model misspecification where a meaningful alternative structure was available. Despite these potential differences in factor structure across participants, subsequent modeling with and without these "unreliable" cases showed similar substantive results. Hence, the present findings suggest typical analyses based on ESM data may be robust to individual differences in data structure and/or quality. Ways to augment the DFA approach to better understand unreliable cases are discussed.
History
Journal
Psychological assessmentVolume
29Issue
9Pagination
1120 - 1128Publisher
American Psychological AssociationLocation
Washington, D.C.Publisher DOI
eISSN
1939-134XLanguage
engPublication classification
C Journal article; C1.1 Refereed article in a scholarly journalCopyright notice
[2017, American Psychological Association]Usage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC