Authors: Robert Bradley, Scott Bruce

Corresponding Author:
Robert Bradley, EdD, LAT, ATC
PO Box 910
State University, AR. 72467
rbradley@astate.edu
870-972-3766

Robert Bradley is the program director of the master of athletic training program at Arkansas State University. He is an assistant professor in the College of Nursing and Health Professions and the curriculum coordinator for the Arkansas Athletic Trainers Association.

Scott Bruce is a research faculty member for the master of athletic training program at Arkansas State University.  He is an associate professor in the College of Nursing and Health Professions.

Survey Return Rates for Athletic Trainers

ABSTRACT

Purpose: To determine which method of survey distribution produces higher return rates. Researchers ask athletic trainers to complete surveys to gather data using electronic and non-electronic means.  There are no publications looking at the return rates and the method of distribution of survey instruments. This article seeks to determine which method of survey distribution produces higher return rates

Methods:  The writer searched research articles published between January 2008 and December 2017 within the Journal of Athletic Training and the Athletic Training Education Journal with the term “survey”. Eligible studies included only those surveys where the intended audience were certified athletic trainers and found within the natajouranls.org website. The writer excluded articles that did not indicate how they distributed the survey, did not report their return rates or failed to provide the number of participants in their sample.

Results: Between 2008 and 2017, 81 publications included data obtained using a survey. 87.65% of surveys were sent via email or electronic form and 13.2% with a mailed survey. Electronically send surveys (e-mails) were exceedingly popular, the return rates for electronic surveys was 34.21% while non-electronically (mailed) surveys had a return rate of 66.9%.

Conclusions: When researchers send surveys to certified athletic trainers, those athletic trainers tend to respond in larger numbers (increased return rate) if those surveys were sent through the mail, than with emails.

Application in sport: Any attempt to garner information from athletic trainers either for research or marketing purposes will find that athletic trainers respond at higher rates with mailed surveys, than with electronically sent surveys.

Key Words: survey, return rates, email, mail

INTRODUCTION

When a researcher makes the decision to distribute a survey to potential subjects, there are usually two means of distribution, either by mail or by phone.  The earliest researched use of a telephone used for survey distribution started in 1954 (4).   Since 1986 with the introduction of the fax then eventually the internet, there has been growth in the use of electronic surveys in the forms of internet sites, e-mails, and faxes (5,8,12,18). Researchers started using e-mail as a means for the distribution because of the decreased costs and hopeful increased return rates (12).  Even though e-mails are faster, some researchers liked the availability of the fax as a means of survey distribution (5). Despite the introduction of the fax and e-mails, researchers have not yet abandoned telephone and letter surveys. Over the decades the return rate for health care professionals has slowly been in decline to less than 50% (5).

One question the researcher needs to consider is the method of survey distribution. The decision as to whether or not to send the surveys via an electronic method or postal can be attributed to the cost and turnaround time. Electronically sent surveys did show to have faster turnaround times (6,7). However, surveys sent mailed shown better return rates (1,2,3,6,15,19,22).

In Sheehan’s article the authors made an effort to understand the return rates of physicians for the specific purpose of determining what were best methods of increasing return rates (20).  Markwell and Wainer, Raheel and Kujan found that the best incentives to encourage response rates for physicians included one or more of the following; money, surveys with short questions, or mailing the survey with a self-addressed stamped envelope (SASE).  Even with these incentives several authors found that the response rates for physicians was below 40% (14,17).

To date, there have been no publications showing the methods used for the distribution of surveys and the return rates of survey instrument distributed to subjects identified as certified athletic trainers. The purpose of this literature review study was to examine, the method of survey distribution, the response rates, survey topic, and the number of surveys per year between the years of 2008 and 2017. 

METHOD

Articles between January 2008 and December 2017 were searched within the natajournals.org website for the term “survey” and then narrowed to include publications listed as ‘research articles’.  This yielded 244 articles.  The search was further narrowed to include only those where the intended sample were certified athletic trainers. Articles were excluded if they did not indicate how they distributed the survey, did not report their return rates or failed to provide the number of participants in their sample. Data gathered from each article included the number of participants in the sample, the number of surveys returned, and return rates.  Data were analyzed using IBM SPSS Statistics, Version 25 (IBM Corp. Armonk, NY). An independent t-test was performed to compare return rates between mailed and electronically sent surveys. The alpha level was set at (p ≤ = 0.05).

RESULTS AND DISCUSSION

From 2008 through 2017, 81 survey-related article were published.  Ten of the 81 (13.2%) published articles used a mailed or non-electric form for survey distribution and collected with 71 of the 81 (87.65%) distributed and collected via electronic means (i.e., e-mail). Sixty-seventy of the 81 articles (82.7%) were published in the Journal of Athletic Training while 14 of the 81 (17.3%) were published in the Athletic Training Education Journal.

The sample size varied greatly, the largest sample was 20,365(16) while the smallest was 17(10) and a mean subject size of 2068.2 (± 3505.9) and a median of 1000 subjects.  For electronic/emailed surveys the average sample size was 2293.94 subjects.  For non-electronic/mailed surveys the average size of subjects was 465.6.

Response rates for all surveys ranged from 2.2%(9) to 82.6%(13) with a mean of 37.99% for all surveys and a median of 33.9%.  Electronic surveys reported an average return rate of 34.21% while non-electronic/mailed surveys averaged 66.9% for an overall average return rate of 37.99% ±19.73 (Table 1,2,3,4).

Table 1

Method of Distribution Return Rate mean/SD
Non-electronic/mailed 64.8%/±16.26%
Electronic/emailed 34.21%/±17.13%
Return Mean 37.99%/±19.73%

Table 2

n = 81 Mean (±sd) Median
Number Distributed 2068.22 (±3505.9) 1000
Number Returned 582.04 (±1276.9) 232
Return Rate 37.99% (±19.73) 33.92

Table 3

  Levene’s Test for Equality of Variances Independent t-test
  F Sig. t df Sig. (2-tailed)
Equal variances assumed 0.211 0.647 5.325 79 0.000
Equal variances not assumed     5.542 11.997 0.000

Table 4

Statistics Number Distributed Number Returned Return Rate Return Rate %
N 81 81 81 81
Mean 2068.22 596.04 .380 38%
Median 1000 249 .339 33.90%
Std. Deviation 3505.965 1298.41 .1973231 19.73%

These return rates almost mirror a similar study performed with physicians as the target responder (21).  In this study, physicians responded to online surveys 33.7% of the time while 66.3% responded to mailed surveys.  From the different waves of surveys sent, the response rate for mailed reached as high as 75.9%

The mean for non-electronic surveys was 64.8% with electronic surveys mean at 34.2%. An independent t-test was statistically significant (p< 0.001). Standard deviation for the non-electronic was 16.26 and electronic at 17.13.

While electronic surveys were seven times more popular than non-electronic surveys, there did not appear to be a tendency toward higher return rate. Not included in this research were attempts by the authors to use pre-notification or follow up communications with their method of survey distribution, as technique to increases return rates (11).  It is possible that the lower return rates for electronic surveys was due to the influences that affects return rates such as length of survey, lack of pre-notification, lack of follow up or lack of interest in the topic (18).  The data show that the subject size for mailed surveys was smaller than the subject size for surveys distributed via electronic means.  It is possible that the decision for the method of the delivery of the survey depended upon the desired sample size.  Additionally this decision may have occurred because of the potential cost of mailing surveys through the postal service.

The average sample size for electronic surveys was 2293 while mailed subject pools was much lower at 466. The financial cost of the mailing surveys may have been the cause of the discrepancy between these two variables.  It is quite possible that each author looked at their specific populations and their budget and made their decision whether to mail or e-mail base on those concerns.

CONCLUSION

Athletic trainers are sent many surveys from fellow athletic trainers or researchers hoping to gain insight into the daily practices, wisdom, feelings and observation of practicing certified athletic trainers.  To date no one has examined the return rates of surveys based on the method of survey contact.  Data from this study demonstrates that electronic surveys are twice as popular as mailed surveys but produce a lower return rate (34.21% electronic, 64.8% mailed).  The lower return rates from electronic surveys may be contributed to a lack of pre-notification, lack of follow up e-mails, lack of interest or lack of personal attention.  Further longitudinal research needs to be done to examine how each of these factors affects the rate of return on electronic surveys.  If the goal is to achieve a higher return rate, these data suggests distributing a survey through the mail, rather than an electronically distributed method would be the preferred method.

APPLICATIONS IN SPORT

People wanting to communicate with athletic trainers must consider the athletic trainers preferred method of communication.   This data can indicate that athletic trainers prefer the kinesthetic activity and tactile feel of paper in their hands to that of a mass email when responding to attempts of research or marketing. 

ACKNOWLEDGEMENTS

There are no financial conflicts of interest.

REFERENCES

  1. Aitken C., Power R., Dwyer R. (2008). A very low response rate in an online survey of medical practitioners. Australian & New Zealand Journal of Public Health, 32(3), 288–289.
  2. Cho Y. I., Johnson T. P., VanGeest J. B. (2013). Enhancing surveys of health care professionals: A meta-analysis of techniques to improve response. Evaluation & the Health Professions, 36(3), 382–407. doi:10.1177/0163278713496425
  3. Couper M. P. (2011). The future of modes of data collection. Public Opinion Quarterly, 75(5), 889–908. doi:10.1093/poq/nfr046
  4. Curtin, R., Presser, S., & Singer, E. (2005). Changes in Telephone Survey Nonresponse over the Past Quarter Century. Public Opinion Quarterly, 69(1), 87-98. doi:10.1093/poq/nfi002
  5. Dickson, J. P., & Maclachlan, D. L. (1996). Fax Surveys: Return Patterns and Comparison with Mail Surveys. Journal of Marketing Research, 33(1), 108. doi:10.2307/3152017
  6. Dykema J., Jones N. R., Piche T., Stevenson J. (2013). Surveying clinicians by web: Current issues in design and administration. Evaluation & the Health Professions, 36(3), 352–381. doi:10.1177/0163278713496630
  7. Fan W. M., Yan Z. (2010). Factors affecting response rates of the web survey: A systematic review. Computers in Human Behavior, 26, 132–139. doi:10.1016/j.chb.2009.10.015
  8. Fricker, S. (2005). An Experimental Comparison of Web and Telephone Surveys. Public Opinion Quarterly, 69(3), 370-392. doi:10.1093/poq/nfi027
  9. Hamson-Utley, J. J., Martin, S., & Walters, J. (2008). Athletic Trainers and Physical Therapists Perceptions of the Effectiveness of Psychological Skills Within Sport Injury Rehabilitation Programs. Journal of Athletic Training, 43(3), 258-264. doi:10.4085/1062-6050-43.3.258
  10. Hazelbaker, C. B. (2013). Perceived Skills and Abilities Required by Athletic Trainers in Hospital and Clinical Management Positions: A Delphi Study. Journal of Athletic Training, 48(1), 87-91. doi:10.4085/1062-6050-47.6.2
  11. Hembroff, L. A., Rusz, D., Rafferty, A., Mcgee, H., & Ehrlich, N. (2005). The Cost-Effectiveness of Alternative Advance Mailings in a Telephone Survey. Public Opinion Quarterly, 69(2), 232-245. doi:10.1093/poq/nfi021
  12. Kellerman, S. (2001). Physician response to surveys A review of the literature. American Journal of Preventive Medicine, 20(1), 61-67. doi:10.1016/s0749-3797(00)00258-0
  13. Manspeaker, S. A., & Hankemeier, D. A. (2017). Athletic Trainers Current Knowledge and Envisioned Use of Foundational Evidence-Based Practice Concepts. Athletic Training Education Journal, 12(2), 121-133. doi:10.4085/1202121
  14. Markwell A. L., Wainer Z. (2009). The health and wellbeing of junior doctors: Insights from a national survey. Medical Journal of Australia, 19(18), 441–444.
  15. McLeod C. C., Klabunde C. N., Willis G. B., Stark D. (2013). Health care provider surveys in the United States, 2000-2010: A review. Evaluation & the Health Professions, 36(1), 106–126. doi:10.1177/0163278712474001
  16. Pike, A. M., Pryor, R. R., Vandermark, L. W., Mazerolle, S. M., & Casa, D. J. (2017). Athletic Trainer Services in Public and Private Secondary Schools. Journal of Athletic Training, 52(1), 5-11. doi:10.4085/1062-6050-51.11.15
  17. Raheel S. A., Kujan O. B. (2016). The opinion and response of health professionals associated with academics about the research design and methods: A study. Journal of International Society of Preventive & Community Dentistry, 6(2), 154–160. doi:10.4103/2231-0762.178745
  18. Schaefer, D.R., & Dillman, D.A. (1998).  Development of a Standard E-mail Methodology.   Revision of a paper presented at the 53rd Annual Conference of the American Association of Public Opinion Research.  St. Louis, Mo.
  19. Scott, A., Jeon, S., Joyce, C. M., Humphreys, J. S., Kalb, G., Witt, J., & Leahy, A. (2011). A randomised trial and economic evaluation of the effect of response mode on response rate, response bias, and item non-response in a survey of doctors. BMC Medical Research Methodology, 11(1). doi:10.1186/1471-2288-11-126
  20. Sheehan, K. (2001).  E-mail Survey Response Rates: A Review.  Retrieved from University of Oregon School of Journalism and Communications http://jcmc.indiana.edu/vol6/issue2/sheehan.html
  21. Taylor, T., & Scott, A. (2018). Do Physicians Prefer to Complete Online or Mail Surveys? Findings From a National Longitudinal Survey. Evaluation & the Health Professions, 42(1), 41-70. doi:10.1177/0163278718807744
  22. Yan W., Cheng T., Scott A., Joyce C., Humphreys J., Kalb G. (2011). Data survey. Medicine in Australia: Balancing employment and life. Australian Economic Review, 44, 102–112.

Print Friendly, PDF & Email