Surveys can improve measurement of public program access burdens

Better measures can capture psychological burdens like stress and stigma and support agencies' burden reduction efforts

Image generated by Leonardo.ai

Researchers have increasingly documented the importance of administrative burdens in access to public benefits and services. Drawing from agenda-setting work from public administration scholars Pamela Herd and Donald Moynihan, administrative burdens refer to the costs that individuals face when attempting to interact with government. Responding to the growing academic literature, the Biden-Harris Office of Management and Budget (OMB) recently launched a new initiative to help federal agencies better gauge the burdens that might be present in accessing public benefits and services, and to reduce those burdens. The initiative has already had important successes, but one obstacle to its implementation is the lack of quantitative measures of burden across programs, especially psychological burdens. It is important for federal agencies to have access to systematic, quantitative measures of burden, and surveys can be an important tool for producing them, especially for psychological burdens. Research on applicants to the Supplemental Nutrition Assistance Program (SNAP, or food stamps) illustrates how surveys can measure psychological burdens related to stress and stigma.

Administrative burdens, why they matter, and why they can be difficult to tackle

Administrative burdens can include the time it takes to learn about a government program and assess whether or not you might be eligible (learning costs), stress or stigma you might encounter in the application process (psychological costs), the cost of transportation and child care necessary to travel to government offices (compliance costs); and the hassle involved in using a government program (redemption costs). Researchers have found that these burdens can create significant obstacles to accessing government programs and impose heavy costs even when individuals do access programs. For example, when Social Security Administration field offices close, making it harder for individuals with disabilities to get assistance with paperwork, receipt of disability benefits falls–an example of compliance costs. Similarly, when applications for unemployment benefits are more complicated, otherwise-eligible workers may not apply–an example of learning costs.

Burdens are often associated with a program’s eligibility screens, or the process by which administrators try to target a benefit or service to its intended recipients. As a result, policymakers in government often feel they face a trade-off when considering reducing burdens. By reducing burdens and making it easier for eligible individuals to qualify for benefits or services, some policymakers might be concerned about ineligible individuals accessing that benefit or service. 

Ideally, policymakers would be able to bring data to bear on this trade-off to understand exactly how many eligible people might be screened out and how many ineligible people might be screened in for any given level of burden in a particular program. But, as it stands now, government agencies tend to focus overwhelmingly on one side of “program integrity”–ineligible people receiving benefits–and not so much on eligible people who aren’t accessing benefits, or are experiencing high burdens when applying. That is because government agencies often do not have good data on who experiences burden, what those burdens are, and which steps in the application or recertification process for benefits and services create those burdens. This is especially true for psychological burdens, such as stress, stigma, or disrespect, despite research suggesting they can be just as important as other forms of burden.

Well-targeted surveys can be helpful tools for agencies to track psychological burden

One important tool for measuring burden in public benefit and service programs are surveys. Surveys permit agencies and researchers to understand the experiences held by individuals who applied for government programs, as well as individuals who might have been eligible but who did not end up applying–individuals who would not otherwise show up in administrative data. Surveys are especially helpful for measuring psychological burden because they can ask about the feelings, attitudes, and views of individuals not otherwise captured in other data, like administrative records on how long it took an individual to complete a form. 

Surveys have limits that are worth recognizing as well. It can be difficult to reach some individuals, particularly very vulnerable or underserved communities, with standard survey methods. If agencies or researchers want to generalize from a survey to a broader population, they need to pay close attention to the representativeness of their survey methods and how respondents are recruited. And how surveys are designed–such as the phrasing or order of questions–can affect the interpretation of results. Even with these general limits, however, surveys can be a useful strategy for better understanding the burdens that individuals face in accessing public benefits and programs.

In a recent article, Sebastian Jilke, Martin Bækgaard, Pamela Herd, and Donald Moynihan took an important step forward in testing survey measures of psychological burden using a battery of new questions, finding items that correlated well with one another and were correlated with demographic characteristics we might expect to be related to burden–such as poorer health, financial scarcity, and lower levels of levels of formal education. Their preferred battery of psychological burden included the following items:

• How difficult was the process of finding information about the program, such as how to apply or what you needed to do to renew your benefit?

• How was the process of filling out the paperwork, providing proof of eligibility (such as pay stubs, proof of residence, birth certificates, etc.), and/or attending interviews?

• Please describe how you felt during these experiences: Frustrated?

Employing a similar but different approach, I have also tested the validity of survey items that could be used to gauge two specific aspects of psychological burden: stress and dignity (or stigma). Focusing on the Supplemental Nutrition Assistance Program (SNAP), the primary food assistance program for low-income households, my goal was to build a replicable approach that agencies and researchers could use to reach a broad swath of potential program beneficiaries. A sample of users of Meta social media products who had experience applying for, or receiving SNAP were asked three questions intended to tap into stress and stigma that individuals might have felt when applying for benefits:

• How much do you agree or disagree with the following statement: "Applying for the Supplemental Nutrition Assistance Program (SNAP), or food stamps, was stressful."

• How much do you agree or disagree with the following statement: "Supplemental Nutrition Assistance Program (SNAP), or food stamps, staff treated me with respect when I was applying for benefits."

• How much do you agree or disagree with the following statement: "The whole process of applying for Supplemental Nutrition Assistance Program (SNAP) benefits, or food stamps, made me feel respected."

Overall, respondents were able to give meaningful responses to all three items, and in the most recent survey, fielded in March 2024 (n=479), less than 1 percent of respondents said that they were not sure of their stress during their most recent SNAP application or how their most recent application made them feel. Just 2% said the same about not being sure about the respect they received from SNAP staff. 

I also fielded the same battery of items that Jilke et al. proposed in their recent paper to compare how correlated they were with my three measures. Three primary conclusions emerge from this analysis.

First, and most importantly, stress and respect (or stigma) are only modestly related to one another, indicating that they are tapping into different aspects of psychological burden. This strongly suggests the need to distinguish between the two when measuring burdens in applications.

Second, perceptions of interactions with SNAP staff and the respect those interactions conferred were strongly related to perceptions of respect in the overall process. This suggests that staff interactions are very important to individuals’ overall assessment of respect in the whole SNAP application process.

Last, and most importantly, my measure of stress was strongly related to Jilke et al.’s scale, but the measures of respect were less correlated. These findings again suggest that to measure respect and dignity, agencies and researchers may need separate items, like the ones I have explored here, that are distinct from items related to stress, frustration, or learning costs.

Next steps: better engagement with the public to understand and tackle psychological burdens

In April 2022, the Biden-Harris OMB issued an important new policy document for federal agencies. OMB called on agencies to more comprehensively assess the burdens that agencies place on the public as part of accessing public benefits and services and then charged agencies with lowering those burdens. A report issued in the summer of 2023 summarizes some of the high-profile efforts the Administration has succeeded in implementing to reduce burdens, and implementation of the initiative is ongoing. That work has included simplifying applications for farm loans; improving plain language communication in the unemployment insurance program; and expanding access to free school meals without additional paperwork for eligible households.

Despite this important progress, agencies have been slower to begin reporting more comprehensive burden measures, especially related to psychological burdens. Reviewing agencies’ commitments to burden reduction reported to OMB last year, for instance, the vast majority of agencies still focused on paperwork time as the primary measure of burden. And when they did mention psychological burdens, they tended to do so in qualitative and broad terms, rather than offering specific, quantitative estimates. 

Survey-based approaches to measuring psychological burden can be an important tool for agencies to begin meeting the broader charge OMB gave them to more comprehensively estimate the “beginning to end experience” of applying for, and using, public benefits and services. In particular, well-designed surveys could tap into different measures of psychological burden–for instance, separating the measurement of stress and frustration from respect and stigma. They permit a better understanding of the distribution of burden across different groups in the population, and tracking impacts of program interventions. 

For example, in a soon-to-be-released issue brief with the Washington Center for Equitable Growth, I use these measures to assess which socioeconomic groups experience the most stress and least respect in the SNAP application process as well as the impact of state-level changes in SNAP policies for psychological burdens. Those results show that not all prospective SNAP applicants face the same barriers when applying for benefits, and burden is distributed unequally across geography, age, race and ethnicity, and disability status. For example, I find that applicants of color are substantially less likely to report feeling respect when applying for SNAP. By comparison, parents and older applicants report receiving great respect. 

As OMB’s guidance to agencies makes clear, public engagement—through surveys, focus groups, participant-observation, and other consultations–is critical to better measuring and understanding the prevalence and impact of burden on members of the public. To meet the goals of the Administration's burden reduction initiative and sustain that work in coming years, we need to build the capacity for federal agencies and researchers to more systematically engage the public about their experiences with burden in specific government programs.