- Email: [email protected]
Clinical Audit Research
~~
Examples of Iniquity MADAM - I am perplexed; can anyone explain to me why it seems perfectly acceptable t o publish inaccurate, incorrect or incomplete statistics in research papers in Physiotherapy? An example of this was t h e recent paper by Kilby e t a1 (Physiotherapy, September), who concluded from inaccurate statistics that the algorithm they describe was reliable and could be used as a research tool. Unfortunately, the information they present does not allow this conclusion. As with previously published papers demonstrating the intra-rater or inter-rater reliability of assessments (eg R Hallet et al, Physiotherapy 1987, 73, 220-225, V Pomeroy, Physiotherapy 1990, 76, 446-448) p values, the probability of an event occurring by chance, are not included. Without knowledge of t h e level of significance the reader cannot accept the null hypothesis. I am also confused by the statistical reporting of the Kappa coefficient and would have found it useful if the authors had referenced their source. If the Kappa coefficient is equal to 0 it means that there is no agreement (except that expected by chance) between the raters (Siegel and Castellan, 1988). They report a 95.2% agreement and a Kappa coefficient of 0, inferring that 95.2% agreement can be expected by chance and shows no agreement between the raters. Yet the authors base their argument on one result, a 90% agreement with a Kappa coefficient of 0.51 and do not quote the probability of this happening by chance. As it stands, based on these results, the paper is inaccurate and incomplete and the algorithm is not yet ready to be used as a research tool. Most research papers contain grey areas and thus some degree of personal opinion must be allowed, which results in healthy discussion and argument between professionals. Sadly, this is not the case with this paper. The inaccuracy in the statistical reporting leaves no room for argument and undermines the authors' ideas. How can this happen? How can it be published in this form? To some extent I can excuse the mistakes made by the authors as everyone makes errors and sometimes it is difficult to proof-read your own work. What is not excusable is that the reviewing process did not correct these errors, which are then compounded and ultimately published for all to see. This is not a new problem and this is not the first letter criticising research papers published in Physiotherapy. Ironically, the same issue published an excellent letter by M Hurley addressing these problems. The point I am making is that the Journal continues in making the same mistakes, publishing inaccurate research papers which are then criticised. Why not correct the errors before publication? Our profession does possess the research expertise, why isn't it used? Kilby et a1 conclude by saying that the algorithm can be used as an educational tool. Unfortunately, the most obvious educational use in its published form, is as an example of the misinterpretation of medical statistics. Can I make a plea that the editor and the Journal Committee actually respond to these
letters of criticism by changing their policy and developing a method of processing articles whereby only relevant, logical and accurate research papers are published. We should try t o match the professional standards demonstrated by other physiotherapy journals. CATH SACKLEY MSc MCSP Nottingham REFERENCE Siegel, S and Castellan, N J (1988). 'Nonparametric Statistics for the Behavioural Sciences, McGraw Hill, New York.
for patients with low back pain (Meade et al, British Medical Journal, 737, 1431-37) and subsequent discussions, I am pleased to welcome a positive step towards informing physiotherapists about the complex issues involved in collaboration in research. A study day is to be held at the CSP on November 27, organised by the Physiotherapy Research Foundation. Dr Cecily Partridge willgive an introductory talk on the subject, and lead discussions and debate. Details of this and other research initiatives are available from the Research and Information Unit at the CSR
Standards of Research MADAM - For the last six months the Journal has been publishing comments and correspondence relating to appropriate methods of research in physiotherapy and to the standard of research papers. This is a healthy debate, and is raising important issues for the future of the profession and the Journal. Is the discussion to be allowed to run and run without editorial comment or a statement of policy and intent by the Journal Committee or the research and information officer? It is time that the readership knew what was being done about the questions that have been raised so clearly by Dr Parry and Mr Hurley. MARGARET SCHOLEY MSc MCSP DipTP London WCl Sue Nayee, CSP research and information officer replies: I too have followed with interest the debate regarding appropriate methods of research in physiotherapy, which has recently been a feature of the Journal, and have welcomed it as an indication of a lively interest and awareness. The concern of correspondents regarding the standards of published research in Physiotherapy also indicates the high standard members expect of their Journal. The following points may be helpful: 1. Concern about the standard of research articles is noted and shared by the Journal Committee, and efforts are being made to widen the circle of reviewers who are called upon to review research articles. 2. A hard-pressed specially selected panel with research knowledge reviews or refers such articles for review and recommends accordingly. The appointment of a 'Scientific Editor' w h o would take specific responsibility for the scientific and research content of the Journal is under review. 3. Pressure for inclusion in Index Medicus is now so great that no matter how 'improved' the standard of research articles in Physiotherapy might be, its re-admittance is extremely unlikely. Even discounting the American and medical bias of the Index, between 15,000-20,000 biomedical journals of recognised stature are now published each year of which only 3,000 are included. I believe a far greater problem is the reliance of many libraries on Index Medicus lor Medline, the on-line version) as the only source of information. Physiotherapists at least should be aware of the deficiencies of Medline in this respect, and should expect to use other sources also, such as Cinahl and Physiotherapy Index. Following the debate and concern about the report on chiropractic and physiotherapy
Friends in the Gulf? Several members of the Chartered Society of Physiotherapy are being detained in Iraq and Kuwait. It is possible to write to them using aerogrammes only. It is advisable for the sake of the hostages to avoid criticism of political regimes or any other controversial topic as letters may be inspected. The Post Office dispatches letters five times a week to Iraq and they take t w o weeks to reach that country. Transit times after that depend on the situation of the recipient, but the Gulf Support Group has evidence that some letters have been received. Aerogrammes should be address to: (Name of recipient) British Guest in Iraq PO Box 39090 or 50300 or 50285 Baghdad Iraq It does not matter which box number is used.
Clinical Audit Research MADAM - The Department of Health has just given the go-ahead for a six-month pilot project of clinical audit work in professions allied to medicine and related therapy professions under the direction of Dr Charles Normand, director of the Health and Health Care Research Unit at the Queen's University of Belfast. The project team would welcome information and contributions from professionals, researchers and managers who have been developing work in the field of clinical audit and standards. Please write to me at the address below. GLORIA HENDERSON PhD Research Consultant Health and Health Care Research Unit Queen's University of Belfast Mulhouse Building Institute of Clinical Science Grosvenor Road, Belfast BT12 6BJ
Restlexercise for MS MADAM - I am interested to know whether any physiotherapists are involved with using rest-exercisetherapy, as described by Ritchie Russell, in the treatment of multiple sclerosis. I would also be interested if therapists have stopped using this form of treatment and the reasons why.
S J ROUSE (Mr) MCSP DipTP School of Physiotherapy The General Infirmary at Leeds Great George Street, Leeds LS1 3EX
~~
692
Physiotherapy, November 1990, vol76, no 11
Examples of Iniquity MADAM - I am perplexed; can anyone explain to me why it seems perfectly acceptable t o publish inaccurate, incorrect or incomplete statistics in research papers in Physiotherapy? An example of this was t h e recent paper by Kilby e t a1 (Physiotherapy, September), who concluded from inaccurate statistics that the algorithm they describe was reliable and could be used as a research tool. Unfortunately, the information they present does not allow this conclusion. As with previously published papers demonstrating the intra-rater or inter-rater reliability of assessments (eg R Hallet et al, Physiotherapy 1987, 73, 220-225, V Pomeroy, Physiotherapy 1990, 76, 446-448) p values, the probability of an event occurring by chance, are not included. Without knowledge of t h e level of significance the reader cannot accept the null hypothesis. I am also confused by the statistical reporting of the Kappa coefficient and would have found it useful if the authors had referenced their source. If the Kappa coefficient is equal to 0 it means that there is no agreement (except that expected by chance) between the raters (Siegel and Castellan, 1988). They report a 95.2% agreement and a Kappa coefficient of 0, inferring that 95.2% agreement can be expected by chance and shows no agreement between the raters. Yet the authors base their argument on one result, a 90% agreement with a Kappa coefficient of 0.51 and do not quote the probability of this happening by chance. As it stands, based on these results, the paper is inaccurate and incomplete and the algorithm is not yet ready to be used as a research tool. Most research papers contain grey areas and thus some degree of personal opinion must be allowed, which results in healthy discussion and argument between professionals. Sadly, this is not the case with this paper. The inaccuracy in the statistical reporting leaves no room for argument and undermines the authors' ideas. How can this happen? How can it be published in this form? To some extent I can excuse the mistakes made by the authors as everyone makes errors and sometimes it is difficult to proof-read your own work. What is not excusable is that the reviewing process did not correct these errors, which are then compounded and ultimately published for all to see. This is not a new problem and this is not the first letter criticising research papers published in Physiotherapy. Ironically, the same issue published an excellent letter by M Hurley addressing these problems. The point I am making is that the Journal continues in making the same mistakes, publishing inaccurate research papers which are then criticised. Why not correct the errors before publication? Our profession does possess the research expertise, why isn't it used? Kilby et a1 conclude by saying that the algorithm can be used as an educational tool. Unfortunately, the most obvious educational use in its published form, is as an example of the misinterpretation of medical statistics. Can I make a plea that the editor and the Journal Committee actually respond to these
letters of criticism by changing their policy and developing a method of processing articles whereby only relevant, logical and accurate research papers are published. We should try t o match the professional standards demonstrated by other physiotherapy journals. CATH SACKLEY MSc MCSP Nottingham REFERENCE Siegel, S and Castellan, N J (1988). 'Nonparametric Statistics for the Behavioural Sciences, McGraw Hill, New York.
for patients with low back pain (Meade et al, British Medical Journal, 737, 1431-37) and subsequent discussions, I am pleased to welcome a positive step towards informing physiotherapists about the complex issues involved in collaboration in research. A study day is to be held at the CSP on November 27, organised by the Physiotherapy Research Foundation. Dr Cecily Partridge willgive an introductory talk on the subject, and lead discussions and debate. Details of this and other research initiatives are available from the Research and Information Unit at the CSR
Standards of Research MADAM - For the last six months the Journal has been publishing comments and correspondence relating to appropriate methods of research in physiotherapy and to the standard of research papers. This is a healthy debate, and is raising important issues for the future of the profession and the Journal. Is the discussion to be allowed to run and run without editorial comment or a statement of policy and intent by the Journal Committee or the research and information officer? It is time that the readership knew what was being done about the questions that have been raised so clearly by Dr Parry and Mr Hurley. MARGARET SCHOLEY MSc MCSP DipTP London WCl Sue Nayee, CSP research and information officer replies: I too have followed with interest the debate regarding appropriate methods of research in physiotherapy, which has recently been a feature of the Journal, and have welcomed it as an indication of a lively interest and awareness. The concern of correspondents regarding the standards of published research in Physiotherapy also indicates the high standard members expect of their Journal. The following points may be helpful: 1. Concern about the standard of research articles is noted and shared by the Journal Committee, and efforts are being made to widen the circle of reviewers who are called upon to review research articles. 2. A hard-pressed specially selected panel with research knowledge reviews or refers such articles for review and recommends accordingly. The appointment of a 'Scientific Editor' w h o would take specific responsibility for the scientific and research content of the Journal is under review. 3. Pressure for inclusion in Index Medicus is now so great that no matter how 'improved' the standard of research articles in Physiotherapy might be, its re-admittance is extremely unlikely. Even discounting the American and medical bias of the Index, between 15,000-20,000 biomedical journals of recognised stature are now published each year of which only 3,000 are included. I believe a far greater problem is the reliance of many libraries on Index Medicus lor Medline, the on-line version) as the only source of information. Physiotherapists at least should be aware of the deficiencies of Medline in this respect, and should expect to use other sources also, such as Cinahl and Physiotherapy Index. Following the debate and concern about the report on chiropractic and physiotherapy
Friends in the Gulf? Several members of the Chartered Society of Physiotherapy are being detained in Iraq and Kuwait. It is possible to write to them using aerogrammes only. It is advisable for the sake of the hostages to avoid criticism of political regimes or any other controversial topic as letters may be inspected. The Post Office dispatches letters five times a week to Iraq and they take t w o weeks to reach that country. Transit times after that depend on the situation of the recipient, but the Gulf Support Group has evidence that some letters have been received. Aerogrammes should be address to: (Name of recipient) British Guest in Iraq PO Box 39090 or 50300 or 50285 Baghdad Iraq It does not matter which box number is used.
Clinical Audit Research MADAM - The Department of Health has just given the go-ahead for a six-month pilot project of clinical audit work in professions allied to medicine and related therapy professions under the direction of Dr Charles Normand, director of the Health and Health Care Research Unit at the Queen's University of Belfast. The project team would welcome information and contributions from professionals, researchers and managers who have been developing work in the field of clinical audit and standards. Please write to me at the address below. GLORIA HENDERSON PhD Research Consultant Health and Health Care Research Unit Queen's University of Belfast Mulhouse Building Institute of Clinical Science Grosvenor Road, Belfast BT12 6BJ
Restlexercise for MS MADAM - I am interested to know whether any physiotherapists are involved with using rest-exercisetherapy, as described by Ritchie Russell, in the treatment of multiple sclerosis. I would also be interested if therapists have stopped using this form of treatment and the reasons why.
S J ROUSE (Mr) MCSP DipTP School of Physiotherapy The General Infirmary at Leeds Great George Street, Leeds LS1 3EX
~~
692
Physiotherapy, November 1990, vol76, no 11