In a new white paper, “An Overview of the Analytical Flaws and Methodological Shortcomings of the CFPB’s Survey of Consumer Experiences with Debt Collection,” ACA International takes aim at the report released by the CFPB in January 2017 that presented the findings of the CFPB’s national debt collection consumer survey.
In our blog post about the survey, we commented that the CFPB’s current leadership would likely attempt to use the survey to justify the CFPB’s current regulatory focus and lay the groundwork for future enforcement and rulemaking priorities. In its white paper, ACA charges the CFPB with “potentially manipulating inconclusive results to promote the incorrect perception of debt collectors as predatory.” ACA concludes that because “the data obtained by the CFPB through the consumer survey is insufficient at best and fundamentally flawed at worst,” it “cannot be used as the basis to properly inform the Bureau’s debt collection rulemaking efforts” and the CFPB “must conduct further study and analysis of the debt collection market before it will be positioned to issue evidence-based, comprehensive rules to regulate this complex industry.”
The analytical flaws and methodological shortcomings of the survey described by ACA include the following:
- The survey was touted by the CFPB as representing “the first comprehensive and nationally representative data on consumers’ experiences and preferences related to debt collection.” ACA calls the CFPB’s overall sample of individuals with experience with the debt collection industry “remarkably small,” with only 682 (32%) of the 2,132 survey respondents reporting that he or she was contacted by a debt collector. ACA states that the CFPB’s claims about the representativeness and overall quality of the data are “undermined by an array of caveats found throughout the report.” For example, the CFPB minimizes the survey data by describing the report as a “descriptive” exercise to “highlight patterns that may be of policy interest” and “to sketch, from consumers’ perspectives, the broad experience of debt collection.” In addition, ACA observes that the CFPB leaves the reader without any basis for determining the degree to which the findings are representative of the population as a whole by acknowledging that the report “does not present standard errors or statements about the statistical significance of the differences [across groups of consumers.]”
- The CFPB’s focus on percentages, coupled with a near-total absence of raw numbers or sample sizes for individual questions provides a limited context for interpreting responses or situating them within the larger sample. For example, the CFPB reported that “almost one-third of consumers (32 percent) reported being contacted over the past year by a creditor or debt collector about a debt.” ACA points out that “the inclusion of raw numbers enables a reader to clearly see that the percentage represents roughly 682 consumers out of the 2,132 surveyed.”
- Many of the findings highlighted in the CFPB’s press release about the survey and Director Cordray’s related remarks relied “on the presentation of a percentage that obscures the total number of responses for a given question.” For example, the CFPB’s press release highlighted the survey’s finding that “three-in-four consumers report that debt collectors did not honor a request to cease contact.” According to ACA, “a more accurate description of this finding would note that the 75% of consumers who reported repeated contact after a request to cease communication are a subset of the 42% who requested contact cease [(about 215 consumers)]; this 42% is itself a subset of the 32% of the total sample that have been contacted about a debt in collection [(about 286 consumers)].” Accordingly, ACA calls the “CFPB’s public statement that ‘three-in-four consumers’ were continuously contacted by debt collectors after requesting contact be ceased… an overt exaggeration.” In ACA’s view, the CFPB has “implied harassing behavior on the part of debt collectors” by using “a tactic that serves to characterize the debt collection industry as problematic.”
In July 2016, in anticipation of convening a SBREFA panel, the CFPB issued an outline of the proposals it was considering for a rule covering “debt collectors” subject to the FDCPA. At that time, the CFPB stated that it expected to conduct a separate SBREFA proceeding for a rule covering first-party creditors collecting their own debts and others engaged in debt collection not covered by the proposals. The CFPB’s Fall 2016 rulemaking agenda indicated that the CFPB expected to convene a second SBREFA proceeding in 2017 and gave a February 2017 estimated date for further prerule activities (which would likely include testing of model validation notices and other disclosures.)