A Key to Obtaining The Truth, The Whole Truth and Nothing But The Truth (Or As Close As You Can Get To It) In Your Member Surveys

By Jon Haller
Director of Market Research
Credit Union National Association

Question: Which one of the following best describes your credit union’s approach to conducting member surveys?

A) We are willing to "cut a corner or two," even if we run the risk of obtaining misleadingly positive survey results and missing out on identifying areas within our credit union that actually need attention, adjustments and/or improvement.

B) We want our survey results to truly represent the behaviors, financial needs and perceptions of our members--positive or negative--so that we can be confident that we are identifying and addressing our actual product- and service-quality challenges and opportunities.

If you answered "B", please read on.

When it comes to doing member surveys, credit union CEOs and marketers want to know, above all else, that their survey findings are reliable and accurately reflect the needs of the membership.

The truth is, though, that many credit unions conducting surveys--whether themselves or through an outside provider--may be making (or not making) adjustments to their policies, procedures, marketing strategies, and new- service offerings based on erroneous or misleading member survey information.

Conducting "two-wave mailings"--sending a second (identical) questionnaire to members who did not return the original questionnaire you sent them--is an important key to obtaining accurate and reliable member feedback, and ensuring that your credit union is making adjustments based on actual member needs. Yet only a handful of providers (CUNA Research being among them) employ this important feature in the studies they do for credit unions.

Simply put: The two-wave mailing yields more-accurate results than does the traditional one-wave mailing. The importance of the two-wave mailing centers on the tendency for noticeably more of your "high-involvement/loyal" members, rather than your "lower-involvement" members, to respond to a one-wave mailing.

Your high-involvement members are your PFI (primary financial institution) members. As you might suspect, compared to lower-involvement members, your PFI members tend to be much more satisfied with the various aspects of your service, convenience, etc., typically use many more of your services and use far fewer services with your competitors.

Many lower-involvement members need that extra "nudge" provided by the second mailing in order to respond to the survey. (And assuming you'd like to begin attracting more of your low-involvement members’ business, finding out why they aren't currently bringing it to you could go a long way in helping you turn things around.)

However, if you conduct only a one-wave mailing, your feedback/survey results will quite likely be inaccurate, as they will more heavily represent only the views and needs of your most loyal supporters and heaviest product users.

As a result, your survey findings could reveal any or all of the following "untruths" for your credit union: higher market shares of your members' loan and other financial-service business, inflated PFI levels, and overly- positive evaluations of the credit union and its service quality. These could lead to taking the wrong actions and/or not taking actions at all when, in fact, they would be warranted, with regard to various issues within your credit union.

Here’s a summary of discrepancies from an actual survey CUNA Research has conducted. It shows the results from members responding to the first wave, the results from those responding to the second wave, and, most importantly, the type of potential problem areas that would have gone undetected had the two-wave process not been used. Note, in particular, the impact on the credit union’s stated market shares for services.

Without the two-wave mailing, this credit union would have--among other things--mistakenly concluded they were a "high performer" with respect to attracting their members’ loan business and would find no need to make any adjustments in their loan policies, procedures or rates. Potential loan business, however, would have continued to walk out the door (to their competitors), negatively impacting both the credit union’s objective of meeting their members’ needs and, just as important, their bottom line.

You no doubt choose to conduct surveys to ensure your credit union is meeting your members’ needs as best it can. But by using only a one-wave mailing, you - and ultimately, your members - won't be receiving all the survey-related benefits you should from your financial- and time investments in the project.

An additional, important advantage of two-wave mailings is that they yield higher response rates (i.e., the number of returned questionnaires divided by the number of members in your sample) than do one-wave mailings, and higher response rates yield more-reliable results.

Member surveys are an extremely valuable tool in helping you serve your members. Regardless of whether you do your survey in-house or turn to an outside provider, insist on a two-wave mailing. It will better help you both find and act on "the truth."

Other Issues of Research Review

Previous Issue:
Number of Prime Borrowers Among CU Members Continues to Drop