- Member Services
- Patient Safety
|Outcome Measures in Interventional Spine – A Focused Review|
Brett J. Teran MD1; Denise L Norton MD1; Clifford R Everett, MD, MPH1
1University of Rochester, Department of Orthopaedics and Rehabilitation, 601 Elmwood Ave. Box 665, Rochester NY 14642
Health care reform will change the practice of medicine in the United States. How much change is currently unclear, but the future appears to include accountable care organizations and improved communication on outcomes of care. The selection of outcome instruments does appear to be open to the health care system. This article aims to include the most utilized questionnaires in the field of interventional spine. The focus of this article will be on attributes and distinctions of each questionnaire. There will be an effort to compare each subjective measurement tool and emphasize the unique characteristics that each has in the evaluation of low back pain and its effect on physical functioning. The article concludes with the future of standardized questionnaires for clinical and research purposes. This includes the Patient Reported Outcome Measurement System (PROMIS) and computerized adaptive testing (CAT) with a comparison relating to the currently used questionnaires.
Within interventional spine the outcome measures are multiple especially for lumbar disorders due to the commonality of this issue and the financial impact. Low back pain is the most common spinal disorder, affecting over 80% of people at some point in their life, and from 4–33% of a population at any one time.1 Low back pain is the fifth leading cause of all physician visits in the United States accounting for over $100 billion a year spent on management and treatment.2 A majority of patients with low back pain have significant symptom improvement within 4 weeks of onset, but about one third of patients experience long-term sequelae.3
Physicians practicing in the field of interventional spine focus on the diagnosis and treatment of spinal disorders. During the initial encounter, a critical component is gathering symptom characteristics. This process happens, in most practices, through the use of questionnaires completed prior to arrival. Completing a questionnaire is one method of data collection. It relies heavily on subjective measures of pain and function. Pain is most often the reason for a clinical evaluation, but it remains a subjective variable. Pain is assessed using well-substantiated tools such as the Visual Analog Scale (VAS) or the Numerical Pain Rating Scale (NPRS).4 Both of these tools utilize similar concepts in quantifying patients’ experience of pain. The assessment of pain plays a key role in establishing a baseline that is important in demonstrating subjective improvement over time.
A full evaluation must encompass more than just the quantification of pain. The evaluation should encompass a spectrum of complaints related to how pain affects physical functioning and overall quality of life. A survey commissioned by the American Academy of Orthopedic Surgeons found that 72 percent of those who sought treatment for back pain gave up on exercising or sports-related activities; in addition, 60 percent said they were unable to perform ‘some’ activities of daily living.6
Measuring functional independence in the setting of back pain is complex, yet a vital component of gathering data. Compared to pain, a patient’s function can be measured objectively. This has been accomplished recently through the use of activity monitors i.e. accelerometers to capture walking performance. Activity monitors, though promising, have not been extensively studied and validated in Spine research to this point. Until further research is completed, a subjective assessment of function must be collected and utilized in concert to pain.
Today, physicians have their choice of a myriad of questionnaires that assess low back pain, comprising the domains of pain severity, quality of life and functional status. These questionnaires are divided into two categories; general and condition-specific. General surveys focus on the overall health, whereas condition-specific encompass the functional impact of a specific condition. Existing studies show inconsistency regarding the measure of responsiveness with general versus condition-specific questionnaires.7
Most Commonly Utilized Questionnaires
The Oswestry Disability Index (ODI) and the Roland Morris disability questionnaire (RMDQ) are the most commonly recommended condition-specific outcome measures for spinal disorders and are mentioned first.8
Oswestry Disability Index
The Oswestry Disability Index (ODI) is the most commonly used low back pain questionnaire.9 John O’Brien initiated this questionnaire in 1976 to evaluate how back or leg pain affects the patient's ability to manage in everyday life. It was developed in a specialist referral clinic, where large numbers of patients with chronic low back pain were evaluated for functional disturbances in their activities of daily living (ADL). Version 1 of the questionnaire was published in 1980 by Fairbank and colleagues and widely distributed following the 1981 International Society for the Study of the Lumbar Spine (ISSLS) meeting in Paris, France.8
The ODI has been used in over 200 published articles since its creation.8 The ODI is a reliable and responsive condition-specific assessment tool suited for use in clinical practice. It has been credited with acceptable internal consistency, has high test-retest reliability and responsiveness to changes in condition along with a low burden of administration. It can be used to objectify patient’s complaints and monitor therapy as well.8 The National Spine Network (NSN) has deemed the ODI as the most sensitive measure of change in patients with low function.10
There are 10 questions (items) associated with the ODI. Each question has 6 possible responses ranging from 0 to 5. The ODI is a quick and easy questionnaire that can be completed in less than 5 minutes and scored in under 1 minute. The score is summed and divided by the highest possible score of 50, which produces a percentage from 0 to 100% with the specific grading below:
The minimal detectable change (at 90% confidence) is 10 percentage points. This means that a minimum of 10%change is required to be clinically meaningful. Thus, a change of less than this may be attributable to error in the measurement.
The use of the ODI should be implemented to measure functional advancement, initially at baseline, followed by every 2 weeks for the duration of the treatment program. An important consideration when reviewing the questionnaire is the documentation of "zero.” Few patients will document a zero once in treatment and will continue to score between 10- 30%, despite having made excellent recovery (i.e., returned to work). It is imperative to use clinical judgment when assessing the score and avoid the trap of "treating till zero,” as this is not supportable based on current evidence.
Roland Morris Disability Questionnaire
Roland Morris Disability Questionnaire (RMDQ) was created to evaluate disability related to low back pain. It is a widely used measure of health that can be employed in research or clinical practice.8 In 1983 Roland and Morris created this survey, derived from the Sickness Impact Profile. RMDQ was designed for use in research as an outcome measure, but has been useful in clinical practice monitoring patient’s physical and mental functioning. It has been shown to be reliable and beneficial in assessing the level of disability.13 It is sensitive to change over time in those with low back pain. The RMDQ is proven to be valid, reliable and responsive.13
The RMDQ consists of 24 items covering a variety of activities of daily living. The questions stem from physical and mental functions affected by low back pain. Patients are questioned about their abilities and limitations to perform 24 different activities. Each item is qualified with the phrase "because of my back pain” to distinguish back pain disability from disability related to other causes.
Patients completing the RMDQ are asked to place a check mark beside the statement if it applies to them that day. This approach is ideal for observing acute changes in back pain (e.g., the relatively rapid resolution of symptoms of most patients seen in primary care) or responses to treatment in the short-term.
The RMDQ consists of24 items that measures daily functioning affected by low back pain. To calculate a score, the items are added together. The RMDQ is not weighted. Scores range from 0 to 24, with 24 equaling maximum disability. Scores under 4 may not show significant change over time and deterioration may not show in patients who have scores greater than 20.Depending on the author and level of disability, the smallest change necessary to be clinically significant lies between 2.5 and 5 points.
In comparison to the ODI, the RMDQ is short, simple to complete, readily understood with less ambiguity.13 These characteristics, along with evidence of scientiﬁc validity, have led to its widespread use. The RMDQ scores correlate well with other measures of physical function, including the physical subscales of SF-36, the Quebec Back Scale, and the ODI.8
Short Form 36
The Medical Outcomes Study (MOS) short form (SF-36) is a generic health survey, widely used as a tool to measure health-related quality of life. It is judged to be the most universally used generic instrument for measuring quality of life in the AHA Journal15 and in the British Medical Journal.16
The SF-36 has been found to have excellent validity and reliability when applied to community and clinic populations. Psychometric evaluation of the SF-36 has produced two summary scores: the Mental Health Component Score and the Physical Health Component Score. It yields an 8-scale profile of functional health and well-being, as well as, psychometrically-based physical and mental health summary measures and a preference-based health utility index.17,18 The eight health concepts that were chosen represent the most frequently measured concepts in widely used health surveys and those most affected by disease and treatment.15,18 The items selected also represent multiple operational indicators of health including behavioral function and dysfunction, distress and well being, objective reports and subjective ratings, and both favorable and unfavorable self-evaluations of general health.18
The original SF-36 was derived by the RAND Corporation in the 1980’s in the Medical Outcome Study; a multi-year, multi-site study in order to explain variations in quality-of-life measures regarding physical limitation and emotional troubles as they relate to pain. The SF-36 was made available in a "developmental” form in 1988, then in "standard” form in 1990.15 Most SF-36 items originate from instruments that have been in use since the 1970’s and 1980’s, including items from the General Psychological Well-Being Inventory (GPWBI), various physical and role functioning measures; the Health Perceptions Questionnaire (HPQ); and other measures that proved to be useful during the Health Insurance Experiment (HIE).15,18 Shorter versions of the scale have also been developed, including the SF-12.15,18
Translations of the SF-36 have been the subject of more than 500 publications involving investigators in 22 countries. Ten or more studies have been published from 13 countries.
There are eight health concepts measured including: limitations in physical activities because of health problems; limitations in social activities because of physical or emotional problems; limitations in usual role activities because of physical health problems); bodily pain; general mental health (psychological distress and well-being); limitations in usual role activities because of emotional problems; vitality (energy and fatigue); and general health perceptions. The last item measures health transition but does not contribute to any of the scale scores. The standard form of the instruments asks for participants to reply to questions according to how they have felt over the previous week. The items use Likert-type scales, some with 2 to 3 points and others with 5 or 6 points.
The scales were scored and linearly transformed according to the original algorithms; in which higher scores represent better health status.
Like the ODI, the SF-36 takes about 5-10 minutes to complete. Items on each scale are coded, summed, and given final percentage values, ranging from 0 (worst health) to 100 (best health). Numbers are transformed to give a mean of 50 and a standard deviation for each scale is equalized at 10. Two summary scores reflecting physical or mental health can be derived from the SF-36. The second question on this test addresses change in health status over the past year, and is not counted in scoring the eight dimensions but is used to estimate change in health form across-sectional administration of the SF-36.
Quebec Back Pain Disability Questionnaire
The Quebec back pain disability scale (QBPDS) is a 20-item, condition-specific questionnaire that assesses the degree of disability in patients with back pain by evaluating daily tasks. The scale is a reliable and valid measure used for monitoring the progress of individual patients participating in treatment or rehabilitation programs.19
The questionnaire assesses self-care, sleeping, walking, climbing stairs, sitting, standing, lifting large or heavy objects, bending and stooping, physical activities and house work but social life, sex life and pain intensity are omitted. This being said, it is recommended that another survey/questionnaire be employed to assess pain. Each one of 20 daily activities is scored with a six-point scale ranging from 0 to 5, the latter signifying ‘unable to do’. The item scores are added up in order to obtain the disability score, which ranges between 0 and 100. The higher values represent greater disability.19
The questionnaire is self-administered by the patient; easily accomplished in 5–10 minutes and scored in less than 2 minutes.
McGill Pain Questionnaire
The McGill Pain Questionnaire, also known as McGill pain index, is a scale rating pain. In 1971 at McGill University in Canada, Melzack and Torgerson developed the questionnaire.51
The questionnaire provides the physician with 7 descriptive expressions used to describe the quality and intensity of the pain experienced by the patient, in addition to, an intensity scale. The character and quality of pain stems from 3 classes: affective, sensory and evaluative descriptors. The questionnaire is used to evaluate, monitor and determine the efficacy of interventions over time, employed to treat pain. The questionnaire provides quantitative measures of clinical pain that can be treated statistically.
The Waddell disability index (WDI) is a nine-item scale that evaluates disability as a result of low back pain, gauged by impairments in activities of daily living. The questionnaire is composed of yes/no questions regarding lifting, sitting, standing, traveling, walking, sleeping, social life, sex life and putting on footwear; work, self-care and sports are not included. Questions are not related to a specific time period. The final score is calculated by adding up positive items, and ranges from 0 to 9.21
The questionnaire is easy to administer; it can be filled out in about 5 minutes and scored in less than 1 minute.
Pain Disability Index
The pain disability index is a tool that is used to capture the impact of pain. It was developed by the Saint Louis University Medical Center with the first study evaluating its validity in 1984. The Pain Disability index is used to evaluate the impact of pain on a person’s ability to function in everyday life.22
Comparison of Questionnaires
The most common questionnaires have been discussed above. These surveys are not without deficiencies. By correcting these deficiencies, improvements in data collection can be made. This can lead to better utilization of questionnaires, improvement in management strategies and research in the field of interventional spine.
Some of the questionnaires discussed are long and arduous for the patients. Length of time required to complete the questionnaire places a burden on the patient prior to the clinical encounter. Equally so, if the questionnaire is cumbersome, it will have a limited role in a busy clinical environment. Short forms like the SF-36 were created to reduce respondent burden, but these short forms have limited value in comparison to the long form versions. Large-scale data collection through means of a questionnaire is not always feasible in a clinical practice, even with the short form surveys.
Some of the questionnaires have significant content redundancy. For example, the SF-36 vitality scale asks, "Do you have a lot of energy, did you feel worn out, in addition, and did you feel tired?” This repetition is incommodious when time is limited. Patients already spend a significant portion of their day at the doctor’s office: time off from work, driving to the clinic, waiting to be seen, not to mention the paperwork and questionnaires that take time prior to arrival.
Another deficiency in the questionnaires relates to the evolving societal standards through the years. Current scoring allows for a ‘ceiling effect.’ This is demonstrated in surveys through the author’s definition of the upper most capability, making variations difficult to assess. For example, 25 years ago, the ‘ceiling’ was the ability to climb several flights of stairs without limitations and now the ‘ceiling’ has been raised to vigorous activities without limitations. In addition, a ‘floor’ effect exists. This effect is the other end of the scale, accounting for the minimal amount of activity assessed.23
The ODI shows moderate correlation with pain measures such as the visual analogue scale and the McGill Pain Questionnaire.24 The ODI has been used to validate the Pain Disability Index and a functional capacity evaluation.26 The ODI correlates with the SF-36.20 The ODI is also a better predictor of returning to work than two different mechanical methods of lumbar spine assessment.25 Some physical tests correlate with the ODI as well.26 The ODI has also been used as a source of questions for other questionnaires.8 The Quebec and Waddell questionnaires have not been extensively tested with regards to their reliability.50 Finally, the RMDQ has been proven to be more beneficial in patients with mild to moderate disability, while the ODI appeared more sensitive to persistent severe disability.27 The RMDQ contains no direct psychological or social measure associated with back pain and therefore must be combined with another survey of function. The overarching conclusion is that the ODI is more favored than the other questionnaires through extensive validation leading to its increased use in the field of interventional spine. But even the ODI contains insufficiencies in its use.24,28
Each survey must cover a wide array of problems, through a limited number of questions that a patient with back pain may face. No questionnaire has demonstrated the ability to be inclusive of all domains: psychological, social and functional. In fact, recent studies recommend separate questionnaires for each domain which is simply not a feasible option considering participant burden and busy clinical environments. This lends to inherent weakness to the questionnaire. Emotional and social functioning is modestly assessed in self-reported questionnaires.
Diversity exists in health literacy, language, culture and ethnicity that effect completion of the questionnaires, and ultimately, the utility of the forms. Questionnaires should be applicable to the myriad of populations measured. There is no one questionnaire that encompasses the effect of low back pain on functionality in a diverse population; variations in age, race and sex.
The Future of Data Gathering: Patient Reported Outcome Measurement System (PROMIS)
The Patient Reported Outcomes Measurement Information System (PROMIS) utilizes patients’ reported symptoms and functional conditions through specific organized questions. Data from PROMIS has been collected and studied within various clinical settings, measuring the effectiveness of treatments in a multitude of chronic diseases.
PROMIS started with a series of meetings in 2002 through the NIH common fund. The NIH common fund was created to address significant public health problems needing further medical research too large for any single institution. One of the programs through the NIH common fund was the Re-engineering Clinical Research Enterprise initiative, which was involved in the patient-reported chronic disease outcome assessment. In 2004, PROMIS was initiated as part of a multi-center cooperative group and 6 primary research sites were subsequently funded with PROMIS grants in 2004-2009.
PROMIS is touted as effective in assisting with treatment plans, improving communication and management of chronic disease. It illustrates four unique domains: comparability with standardized domains, reliability and validity (undergone rigorous review and testing), flexibility with administration and inclusiveness; from low functioning persons, low IQs and starting at a young age, age 8 and up.
Patient reported outcomes include quality of life surveys, pain questionnaires, functional surveys, and satisfaction with care surveys. In 2006-2007, the PROMIS research sites gathered data on both the general and diseased populations: normal subjects ~ 7,500, cancer ~ 1,000, heart disease ~ 500, rheumatoid arthritis ~ 500, osteoarthritis ~ 500, etc. The general population was constructed to ensure adequate representation with respect to: sex, age, ethnicity and education (for a US population).29
The Assessment Center website (www.assessmentcenter.net) demonstrates adult testing in over 20,000 individuals and child and youth testing in over 4,000 individuals in the US. Both traditional (classical test theory) and modern (item response theory) analytic methods were applied to the data. The analysis allowed the validation of surveys, development and selection of appropriate rating scales and of individual items. The Assessment Center that is part of PROMIS allows a clinician to create a study, administer the questionnaire, and then download the data for analysis. The clinician can compare the results of the study created to the general population data from PROMIS, or to disease specific populations.
In order to utilize these data banks in clinical practice, a computerized adaptive test (CAT) must be instituted. An adaptive test is a tailored, individualized measure that involves selecting a set of test items for each individual that best measures the psychological characteristics.31 CAT is a test that adapts to the examinee's ability level and selects questions based on previous responses. It plays an integral part in the completion of questionnaires. CAT eliminates the floor and ceiling effects, as described earlier. It requires fewer test questions to arrive at equally accurate scores, as compared to standard testing. It also allows for automatic administration, data recording, scoring and reporting, as well as, immediate feedback. CAT is convenient and effective for patients, not to mention, for the doctors. CAT can be integrated into an existing electronic medical record.
The considerations for CAT usage in clinical practice include: Integration into the flow of research/clinical practice, integration into existing systems, usability for the patient, visual appearance, ease of use, data security and back-up, quality control and feedback (which can be immediate) to respondents and clinicians.
PROMIS utilizes the technology of CAT and is able to measure emotional and physical functioning. Question focus is broad including mental health topics: fatigue, anxiety, and/or physical health topics: pain, sleep impairment, and/or social health: ability to participate in roles and activities, or a mixture of these. The answers to a set of questions are calculated, scored and used to improve communication, manage health conditions and design treatment plans.
It is documented that as few as 4-items may be needed to estimate a score for a participant. This means that multiple measures can be gathered without fatiguing the participants. The short-forms are also available, which include a scoring format so that each clinician can total a participant’s form and arrive at a stable measure that compares the patient to a normal population. The short forms provide a great opportunity for those who might wish to compare their patients to the larger population. New patients could, for example, fill out an 8- to 10-item form on pain, disability, or fatigue and the researcher would have some evidence as to how they compare to the standard. This demonstrates the usefulness of the PROMIS and CAT method.
A recent study evaluated the lumbar CAT and Modified Oswestry Disability Low Back Pain Questionnaire (MODLBPQ). It examined the discriminating ability and minimal clinical improvement calculated using functional status. Results favored the Lumbar CAT over the MODLBPQ. The Lumbar CAT demonstrated better discrimination, precision and efficiency for a busy, automated outpatient therapy clinic.30
The surveys discussed in this article represent the most frequently utilized for low back pain in an interventional spine practice. Each of the questionnaires discussed above have been evaluated to determine their beneficial aspects as well as their shortcomings. In some instances the inadequacies of one questionnaire necessitates a supplementary survey for clinical completeness. With the addition of each questionnaire, there is increased time required with resulting decrease in clinical efficiency.
PROMIS eliminates the majority of insufficiencies in the questionnaires and offers a streamlined approach to management of low back pain. The utilization of CAT simplifies the process from the perspective of both patient and clinician. A complete patient evaluation can be achieved more effectively and efficiently with the new practice using PROMIS and CAT in the field of interventional spine, with the goal of improved data capturing for research and patient outcomes.
1. World Health Organization. Chronic disease and Health Promotion. http://www.who.int/chp/topics/rheumatic/en/
2. Katz JN. Lumbar disc disorders and low-back pain: socioeconomic factors and consequences. J Bone Joint Surg Am. 2006;88 Suppl 2:21.
3. PengelLH, HerbertRD, MaherCG, RefshaugeKM.Acute low back pain: systematic review of its prognosis.BMJ.2003;327(7410):323.
4. Hartrick CT, Kovan JP, Shapiro S (December 2003). "The numeric rating scale for clinical pain measurement: a ratio measure?".Pain Pract3(4): 310–6.
5. Commentary: One small step. Matthew Smuck, MD*, Eugene J. Carragee, MD Department of Orthopaedic Surgery, Stanford University School of Medicine, 450 Broadway St, Redwood City, CA 94063, USA Received 14 June 2011; accepted 30 August 2011
6. Henschke N, et al. Prognosis in patients with recent onset low back pain in Australian primary care: inception cohort study. BMJ 2009;339:b3829.
7. Resnik L., Dobrzykowski E.. Guide to Outcomes Measurement for Patients With Low Back Pain Syndromes. Journal of Orthopedic & Sports Physical Therapy. Volume 33, Number 6, June 2003.
8. Fairbank J, Pynsent P. The Oswestry Disability Index. Spine 2000; 25:2940-52.
9. Vianin M. J Chiropr Med. Psychometric properties and clinical usefulness of the Oswestry Disability Index. Board member, Swiss Academy for Chiropractic, Department of Postgraduate Studies, Fribourg, Switzerland; staff physician, Clinque Générale, Fribourg, Switzerland; private practice, rue Georges-Jordil 4, 1700 Fribourg, Switzerland. 2008 Dec;7(4):161-3. doi: 10.1016/j.jcm.2008.07.001
10. Walsh T.L., et al. Is a Condition-Specific Instrument for Patients with Low Back Pain/Leg Symptoms Really Necessary? The responsiveness of the Oswestry Disability Index, MODEMS, and the SF-36. SPINE 2003. Volume 28, Number 6, pp. 607-615.
11. Deyo RA, Battie M, Beurskens A, et al. Outcome measures for low back pain research: a proposal for standardized use. Spine 1998; 23: 2003–13.
12. Roland MO, Morris RW. A study of the natural history of back pain. Part 1: Development of a reliable and sensitive measure of disability in low back pain. Spine 1983; 8: 141-144.
13. Roland M, Fairbank J. The Roland-Morris Disability Questionnaire and the Oswestry Disability Questionnaire. Spine 2000; 25: 3115-3124.
14. Roland M, Morris R. A study of the natural history of back pain: part I: development of a reliable and sensitive measure of disability in low-back pain. Spine 1983;8:141-144.
15. Ware, J.E. SF-36 Health Survey Update. 1992.
16. Garratt et al. Quality of life measurement: bibliographic study of patient assessed health outcome. 2002.
17. McHorney, C.A., Ware, J.E. Raczek, A.E. (1993). The MOS 36-item Short Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Medical Care, 31, 247-263.
18. Ware, J.E., Kosinski, M., Keller, S.D. (1994). SF-36 Physical and Mental Health Summary Scales: A Users' Manual. Boston: The Health Institute.
19. Kopec JA, et al. The Quebec Back Pain Disability Scale: conceptualization and development. J Clin Epidemiol.1996 Feb;49(2):151-61.
20. Haas M, Nyiendo J. Diagnostic utility of the McGill Pain Questionnaire and the Oswestry Disability Questionnaire for classification of low back syndromes. J Manipulative Physiol Ther 1992; 15: 90–8.
21. Waddell G. The Back Pain Revolution. Edinburgh: Churchill Livingstone, 1998.
22. Gronblad M, Jarvinen E, Hurri H, et al. Relationship of the Pain Disability Index (Pdi) and the Oswestry Disability Questionnaire (Odq) with 3 dynamic physical tests in a group of patients with chronic low-back and leg pain. Clin J Pain 1994; 10: 197–203.
23. Bjorner, J. Developing Tailored Instruments: Item Banking and Computerized Adaptive Assessment. QualityMetric Incorporated, Lincoln, RI and Health Assessment Lab, Waltham, MA. Presented at the Advances in Health Outcomes Measurement Conference, Bethesda, MD (2004).
24. Leclaire R, Blier F, Fortin L, et al. A cross-sectional study comparing the Oswestry and Roland-Morris functional disability scales in two populations of patients with low back pain of different levels of severity. Spine 1997;22:68 –71.
25. Salen BA, Spangfort EV, Nygren AL, et al. The Disability Rating Index: an instrument for the assessment of disability in clinical settings. J Clin Epidemiol 1994; 47: 1423–34.
26. Gronblad M, Hupli M, Wennerstrand P, et al. Intercorrelation and test-retest reliability of the Pain Disability Index (PDI) and the Oswestry Disability Questionnaire (ODQ) and their correlation with pain intensity in low back pain patients. Clin J Pain 1993; 9: 189–95.
27. Manniche C, Asmussen K, Lauritsen B, et al. Low-back-pain rating-scale: validation of a tool for assessment of low-back-pain. pain 1994; 57: 317–26.
28. Baker CD, Pynsent PB, Fairbank JCT. The Oswestry Disability Index revisited: its reliability, repeatability and validity, and a comparison with the St. Thomas's Disability Index. In: Roland MO, Jenner JR, eds. Back Pain: New Approaches to Education and Rehabilitation. Manchester University Press, 1989:174-86.
30. Hart DL, Stratford PW, Werneke MW, Deutscher D, Wang YC. Lumbar computerized adaptive test and Modified Oswestry Low Back Pain Disability Questionnaire: relative validity and important change. J Orthop Sports Phys Ther. 2012 Jun;42(6):541-51. doi: 10.2519/jospt.2012.3942. Epub 2012 Apr 19. Focus On Therapeutic Outcomes, Inc, White Stone, VA, USA.
31. Weiss DJ. Adaptive testing by computer. J Consult Clin Psychol. Dec 1985;53(6):774-789.
32. Deyo RA. Comparative validity of the sickness impact profile and shorter scales for functional assessment in low back pain. Spine 1986:11;951-0954.
33. Jensen MP, Strom SE, Turner JA, Romano JM. Validity of the Sickness Impact Profile Roland scale as a measure of dysfunction in chronic pain patients. Pain 1992:50;157-162.
34. Patrick DL, Deyo RA, Atlas SJ, Singer DE, Chapin A, Keller RB. Assessing health related quality of life in patients with sciatica. Spine 1995;20:1899-909.
35. Roberts A. The conservative treatment of low back pain. MD thesis, University of Nottingham, 1991.
36. Stratford PW, Binkley JM. Measurement properties of the RM 18: a modified version of the Roland-Morris disability scale. Spine 1997;22:2416-2421.
37. CareTrak outcomes software. Grand Rapids, MN; (800) 393-7255, www.caretrak-outcomes.com.
38. Ware. JE. Future of Outcomes Measurement: Item banking, Short forms and CAT. QualityMetric Incorporated, Lincoln, RI and Health Assessment Lab, Waltham, MA. Presented at the Advances in Health Outcomes Measurement Conference, Bethesda, MD (2004). Quality of life measurement: bibliographic study of patient assessed health outcome measures. BMJ 2002; 324.
39. Stratford PW, Binkley J, Solomon P, et al. Assessing change over time in patients with low-back-pain. Phys Ther 1994;74:528 –33.
40. Grevitt M, Khazim R, Webb J, et al. The short form-36 health survey questionnaire in spine surgery. J Bone Joint Surg [Br] 1997; 79: 48–52.
41. Kaplan GM, Wurtele SK, Gillis D. Maximal effort during functional capacity evaluations: an examination of psychological factors. Arch Phys Med Rehabil 1996; 77: 161–4.
42. Loisel P, Poitras S, Lemaire J, et al. Is work status of low back pain patients best described by an automated device or by a questionnaire? Spine 1998; 23: 1588–94.
43. De Haan. Measuring Quality of Life After Stroke Using the SF-3. AHA journal. 2002.
44. Patrick, D.L., Bush, J.W., & Chen, M.M. (1973). Toward an operational definition of health. Journal of Health and Social Behavior, 14, 6-21.
45. Hulka, B.S., & Cassel, J.C. (1973). The AAFP-UNC study of the organization, 46 utilization and assessment of primary medical care. American Journal of Public Health. 63(6), 494-501.
46. Reynolds, W.J., Rushing, W.A., & Miles, D.L. (1974). The validation of a functional status index. Journal of Health and Social Behavior, 15, 271-89.
47. Stewart, A.L., Ware, J.E., & Brook, R.H. (1981). Advances in the measurement of functional status: Construction of aggregate indexes. Medical Care, 19(5), 473-88.Brook, Ware, Davies-Avery, Stewart, Donald, Rogers, Williams, & Johnston, 1979.
48. Chapman JR et al. Evaluating Common Outcomes for measuring treatment success for chronic low back pain. Spine (Phila Pa 1976) 2011: 36(21 Suppl):S54.
49. Resnik L., Dobrzykowski E.. Guide to Outcomes Measurement for Patients With Low Back Pain Syndromes. Journal of Orthopedic & Sports Physical Therapy. Volume 33, Number 6, June 2003.
50. Davidson, M. and Keating, J. A comparison of Five low back disability questionnaires: Reliability and responsiveness. Journal of the American Physical Therapy Association. Volume 82, number 1, January 2002.
51. Melzack, R. The McGill pain questionnaire: from description to measurement. Anesthesiology. 2005 Jul;103(1):199-202.
10/21/2016 » 10/23/2016
European Congress - Lisbon, Portugal
11/11/2016 » 11/13/2016
Advanced Cervical Bio-Skills Lab - Long Beach, CA
1/13/2017 » 1/15/2017
Radiofrequency Bio-Skills Lab - Long Beach, CA
1/27/2017 » 1/29/2017
Evidence-Based Spine Interventions Seminar - Palm Springs, CA
Inaugural SIS Golf Tournament
2/10/2017 » 2/12/2017
Lumbar Bio-Skills Lab - Phoenix, AZ