RTG Student Work Group

Sample Designs for Evaluating the Quality of Data Collection

Cherié J. Alf, Sarah M. Nusser, Veronica Lessard
Abstract • July 22, 2008

A vital component in any survey is the quality assessment of data collection. Although it is standard practice for a subset of data collectors’ cases to be monitored, sample designs are not always used to select cases for review. To be most effective, the quality review sample is chosen as the data collection is taking place, which calls for in-stream sampling methods. Several considerations go into designing an in-stream quality review sample, including which sample units to target, what information is available for targeting the selection process (e.g., sample frame information, survey responses, paradata from the interview process), temporal aspects of the sampling rate, and how to design the sample so that the data collector cannot predict when review will occur. We will consider this problem generally and provide an example from the National Resources Inventory.