- Email: [email protected]
Insurance Inaccuracy
~~
Kappa Confusion MADAM - We were sorry to read that the reporting of the kappa statistic in our article (Kilby et a/, Physiotherapy, September) confused Mrs Sackley (Physiotherapy, November). We are also disappointed that we were not able to answer her comments directly since we feel that readers would be able t o make more sense of such remarks if they can read the authors' reply. We are in error for not referencing the kappa statistic but had hoped that the explanatory paragraph would be sufficient. Hartman (1977)presents a comprehensive report on the use of statistical tests of reliability, including kappa. The crux of the matter is that agreement is not the same as reliability. This leads to the paradox of 90% agreement producing a kappa value of 0.51 while a 95.2% agreement produces a kappa value of 0, indicating no reliability. How can this be? Tables 1 and 2 of our article show what is happening. The question 'Is the pain constant?' was asked in 21 cases giving 2 x 21 answers. Agreement occurred in 20 cases where the answer was 'no'. In the remaining case one physiotherapist answered 'no' and the other answered 'yes'. In total there were 41 'nos' and 1 'yes'. The expected agreement was thus f8 x 100%. This accords with Hartman's method of calculating expected agreement. In this case, because of the distribution of answers in the 2 x 2 table, expected agreement equalled actual agreement and kappa is thus 0. This is, perhaps, not a fair test of a physiotherapist's ability to distinguish patients with constant pain from the rest in
Costing of Electrotherapy MADAM - We should like to thank Mrs Williams and Mr Jones for their interest in the cosfing of electrotherapy (Physiotherapy, January), but were disturbed to find that they had both misinterpreted the paper (Physiotherapy, November, 1990). Mrs Williams' letter assumes that the objective of our article was to calculate a price at which electrotherapy treatment time should be sold by a physiotherapy department. This assumption runs throughout the letter, in particular when she emphasises that private practices or independent hospitals would not charge a price as low as that we have calculated in order to cover their costs. However, such an assumption was not made at any point in our article, still less did we have the intention of including a risk or profit element as contractors are likely to do. The costs calculated should not be used for purposes for which they are not intended. Our intention was to calculate costs to the Health Service. We made it clear in the article that labour and overheads were to be treated as fixed cost elements of the costs of electrotherapy. These costs do not depend upon the use of the electrotherapy equipment. We repeat the assertion from our article that the hospital stays standing and functioning whether or not electrotherapy treatment takes place. A physiotherapist is not redundant if she or he is not carrying out electrotherapy; she or he carries out other activities. Fixed costs do not depend upon how much electrotherapy is used. Because these cost elements are fixed, in order to obtain cost per hour, we should divide them by the
1 02
~
view of the fact that there appeared to be no patients with constant pain to examine. We felt that this question had to be placed in the algorithm to act as a check to prevent those patients with a pathological cause for their backache from being treated. It is possible to perform weightings to allow for population distributions of features but we did not know the incidence of pathological causes of low back pain in the patients seen in the physiotherapy department where the study took place and so used the standard weightings of 1 for total agreement and 0 for total disagreement. Mrs Sackley's point on the reporting of probability in reliability studies is well taken, but a moment's thought will show why we did not do this. In testing the algorithm we could have either examined the whole alaorithm. observina the reliability of a&eement in the eventual 'diagnoses', or examined each point on the algorithm. In the first case we would have ended up w,th an 11 11 table which would have had insufficient numbers in each cell to offset the 100 degrees of freedom produced by such a large table. We estimate that we would have needed to have examined approximately 250 patients to produce worth-while statistics. We took the second course examining the individual elements of the algorithm. Here again there were difficulties as with each branch in the decision tree the numbers dropped until at the final diagnosis an average of just under t w o patients were agreed on in each diagnostic category. It is clearly illogical to ask all points in every case as some questions were irrelevant in some cases.
~~~
~
~
Accordingly we do not present p values, as without sufficient power they are worse than useless. Had we more time and resources then we would have been happy to have soldiered on to gain sufficient numbers to answer these matters in a more definitive fashion. What we have been able to do is show that the detection of relevant lateral shift and loss of lumbar lordosis are weak points in the McKenzie assessment scheme which require further work to refine techniques of detection. We make no apologies for presenting a paper on a complex issue and hope that it will lead others to improve on our efforts and attempt similar Droiects. . .
number of hours paid for by the Health Service and not, as Mrs Williams and Mr Jones suggest, by the number of hours spent on treatment itself. Most of the comments made by Mrs Williams and Mr Jones relate to the number of hours between which annual costs should be divided in order to arrive at costs per hour. We accept that we have not taken into account statutory and bank holidays. However, we do not accept the arguments that the number of hours spent on sick leave, study leave, authorised absence, compassionate leave and activities other than treatment should also be subtracted. The Health Service pays physiotherapists to work a contracted number of hours and it is appropriate to divide costs by the number of hours paid for. If we recalculated costs per hour subtracting a further two weeks to account for statutory and bank holidays, the pay cost price per hour goes up from €9.47 to f9.89 rather than the €15.14 suggested. We agree that the cost of physiotherapy helpers should be added, but again, these should be divided by 1,620 (45weeks x 36 hours), which is the length of the working year taking account of statutory and bank as well as other holidays, multiplied by the offical working week, rather than the 1,058 hours suggested. Turning to the arguments on overhead costs, Mrs Williams suggests that these too should be divided by the number of hours of treatment provided. However, we worked on the assumption that 'heat, light, laundry, postage, telephones, cleaning, personnel, administrative costs and so on' are not provided merely for physiotherapy treat-
ANDREW ROBERTS FRCS KILBYMCSP MARK Bsc MCSP Universitv Hosoital, Nottinaham
,
Hartman,'D (;977). ,Cons~derationsin the choice of inter-observer reliability estimates', Journal of Applied Behaviour Analysis, 10, 103, 106.
Insurance Inaccuracy MADAM - Because minor inaccuracies, if uncorrected, enter the realm of folklore, I must correct the statement in Physiotherapy (December 1990, page 783 'Resources') that the subscription to the Society of Chiropodists is €128,and that this does not include insurance. Insurance is included in our annual subscription (from January 1, 1991)of €125. J G C TROUNCER General Secretary The Society of Chiropodists ment, but for the hospital as a whole which is open for the whole year. It therefore does not make sense to divide these costs only by the number of hours of physiotherapy treatment. Only if, as Mr Jones suggests, administrative support is supplied specifically to the physiotherapy department which does not appear in the Regional costs statements, should we add these costs and could we avoid double-counting. Then we would have to be sure that such support was uniquely provided by the physiotherapy department in order legitimately to divide them by the number of paid hours. This debate highlights the dangers of using cost calculations to serve objectives for which they were not intended. Our intention has not been to present prices at which physiotherapy department time should be sold, and our article gave no indication that it was. Had this been our intention, then all elements of cost would have been treated as marginal; we would have considered what would be the necessary payment on the open market to induce an extra hour's supply of each element. We acknowledge that with the current developments towards a 'contract culture' there is increasing emphasis on selling services, hence there is a demand for more information on how to cost services in order to sell them. We look forward to further developments in this direction. CAROLINE ALLEN BA Canterbury SHEILA KITCHEN MSc MCSP DipTP London WC2
Physiotherapy, February 1991,vol77, no 2
Kappa Confusion MADAM - We were sorry to read that the reporting of the kappa statistic in our article (Kilby et a/, Physiotherapy, September) confused Mrs Sackley (Physiotherapy, November). We are also disappointed that we were not able to answer her comments directly since we feel that readers would be able t o make more sense of such remarks if they can read the authors' reply. We are in error for not referencing the kappa statistic but had hoped that the explanatory paragraph would be sufficient. Hartman (1977)presents a comprehensive report on the use of statistical tests of reliability, including kappa. The crux of the matter is that agreement is not the same as reliability. This leads to the paradox of 90% agreement producing a kappa value of 0.51 while a 95.2% agreement produces a kappa value of 0, indicating no reliability. How can this be? Tables 1 and 2 of our article show what is happening. The question 'Is the pain constant?' was asked in 21 cases giving 2 x 21 answers. Agreement occurred in 20 cases where the answer was 'no'. In the remaining case one physiotherapist answered 'no' and the other answered 'yes'. In total there were 41 'nos' and 1 'yes'. The expected agreement was thus f8 x 100%. This accords with Hartman's method of calculating expected agreement. In this case, because of the distribution of answers in the 2 x 2 table, expected agreement equalled actual agreement and kappa is thus 0. This is, perhaps, not a fair test of a physiotherapist's ability to distinguish patients with constant pain from the rest in
Costing of Electrotherapy MADAM - We should like to thank Mrs Williams and Mr Jones for their interest in the cosfing of electrotherapy (Physiotherapy, January), but were disturbed to find that they had both misinterpreted the paper (Physiotherapy, November, 1990). Mrs Williams' letter assumes that the objective of our article was to calculate a price at which electrotherapy treatment time should be sold by a physiotherapy department. This assumption runs throughout the letter, in particular when she emphasises that private practices or independent hospitals would not charge a price as low as that we have calculated in order to cover their costs. However, such an assumption was not made at any point in our article, still less did we have the intention of including a risk or profit element as contractors are likely to do. The costs calculated should not be used for purposes for which they are not intended. Our intention was to calculate costs to the Health Service. We made it clear in the article that labour and overheads were to be treated as fixed cost elements of the costs of electrotherapy. These costs do not depend upon the use of the electrotherapy equipment. We repeat the assertion from our article that the hospital stays standing and functioning whether or not electrotherapy treatment takes place. A physiotherapist is not redundant if she or he is not carrying out electrotherapy; she or he carries out other activities. Fixed costs do not depend upon how much electrotherapy is used. Because these cost elements are fixed, in order to obtain cost per hour, we should divide them by the
1 02
~
view of the fact that there appeared to be no patients with constant pain to examine. We felt that this question had to be placed in the algorithm to act as a check to prevent those patients with a pathological cause for their backache from being treated. It is possible to perform weightings to allow for population distributions of features but we did not know the incidence of pathological causes of low back pain in the patients seen in the physiotherapy department where the study took place and so used the standard weightings of 1 for total agreement and 0 for total disagreement. Mrs Sackley's point on the reporting of probability in reliability studies is well taken, but a moment's thought will show why we did not do this. In testing the algorithm we could have either examined the whole alaorithm. observina the reliability of a&eement in the eventual 'diagnoses', or examined each point on the algorithm. In the first case we would have ended up w,th an 11 11 table which would have had insufficient numbers in each cell to offset the 100 degrees of freedom produced by such a large table. We estimate that we would have needed to have examined approximately 250 patients to produce worth-while statistics. We took the second course examining the individual elements of the algorithm. Here again there were difficulties as with each branch in the decision tree the numbers dropped until at the final diagnosis an average of just under t w o patients were agreed on in each diagnostic category. It is clearly illogical to ask all points in every case as some questions were irrelevant in some cases.
~~~
~
~
Accordingly we do not present p values, as without sufficient power they are worse than useless. Had we more time and resources then we would have been happy to have soldiered on to gain sufficient numbers to answer these matters in a more definitive fashion. What we have been able to do is show that the detection of relevant lateral shift and loss of lumbar lordosis are weak points in the McKenzie assessment scheme which require further work to refine techniques of detection. We make no apologies for presenting a paper on a complex issue and hope that it will lead others to improve on our efforts and attempt similar Droiects. . .
number of hours paid for by the Health Service and not, as Mrs Williams and Mr Jones suggest, by the number of hours spent on treatment itself. Most of the comments made by Mrs Williams and Mr Jones relate to the number of hours between which annual costs should be divided in order to arrive at costs per hour. We accept that we have not taken into account statutory and bank holidays. However, we do not accept the arguments that the number of hours spent on sick leave, study leave, authorised absence, compassionate leave and activities other than treatment should also be subtracted. The Health Service pays physiotherapists to work a contracted number of hours and it is appropriate to divide costs by the number of hours paid for. If we recalculated costs per hour subtracting a further two weeks to account for statutory and bank holidays, the pay cost price per hour goes up from €9.47 to f9.89 rather than the €15.14 suggested. We agree that the cost of physiotherapy helpers should be added, but again, these should be divided by 1,620 (45weeks x 36 hours), which is the length of the working year taking account of statutory and bank as well as other holidays, multiplied by the offical working week, rather than the 1,058 hours suggested. Turning to the arguments on overhead costs, Mrs Williams suggests that these too should be divided by the number of hours of treatment provided. However, we worked on the assumption that 'heat, light, laundry, postage, telephones, cleaning, personnel, administrative costs and so on' are not provided merely for physiotherapy treat-
ANDREW ROBERTS FRCS KILBYMCSP MARK Bsc MCSP Universitv Hosoital, Nottinaham
,
Hartman,'D (;977). ,Cons~derationsin the choice of inter-observer reliability estimates', Journal of Applied Behaviour Analysis, 10, 103, 106.
Insurance Inaccuracy MADAM - Because minor inaccuracies, if uncorrected, enter the realm of folklore, I must correct the statement in Physiotherapy (December 1990, page 783 'Resources') that the subscription to the Society of Chiropodists is €128,and that this does not include insurance. Insurance is included in our annual subscription (from January 1, 1991)of €125. J G C TROUNCER General Secretary The Society of Chiropodists ment, but for the hospital as a whole which is open for the whole year. It therefore does not make sense to divide these costs only by the number of hours of physiotherapy treatment. Only if, as Mr Jones suggests, administrative support is supplied specifically to the physiotherapy department which does not appear in the Regional costs statements, should we add these costs and could we avoid double-counting. Then we would have to be sure that such support was uniquely provided by the physiotherapy department in order legitimately to divide them by the number of paid hours. This debate highlights the dangers of using cost calculations to serve objectives for which they were not intended. Our intention has not been to present prices at which physiotherapy department time should be sold, and our article gave no indication that it was. Had this been our intention, then all elements of cost would have been treated as marginal; we would have considered what would be the necessary payment on the open market to induce an extra hour's supply of each element. We acknowledge that with the current developments towards a 'contract culture' there is increasing emphasis on selling services, hence there is a demand for more information on how to cost services in order to sell them. We look forward to further developments in this direction. CAROLINE ALLEN BA Canterbury SHEILA KITCHEN MSc MCSP DipTP London WC2
Physiotherapy, February 1991,vol77, no 2