- Email: [email protected]
MAGIC OF p VALUES
1398 MAGIC OF p VALUES
SIR,-Does it help the reader to know whether p is below or above 5%? Imagine two scientists (N and R), each of whom did 1000 experiments. N’s ideas were highly original, at times far-fetched. In 900 of his experiments the null hypothesis (Ho) was true. R only repeated the experiments of others; Ho was true in 100 of his experiments (table). In the long run both N and R will obtain a significant result, by chance, in 5 % of cases when Ho is true. In 10% of cases both will sometimes obtain by chance a non-significant result when in fact Ho is false. The frequency of this happening cannot be calculated exactly, but 10% is not unreasonable if the sample size is high. RESULTS OF SIGNIFICANCE TES’I’S ON
1000 EXPERIMENTS BY TWO
HYPOTHETICAL RESEARCHERS
often beyond the scope of their competence,’ on the basis of limited and usually second-hand information. Social workers and nonmedical disciplines have no scientific background, and the prevailing assumption is that hypotheses are valid unless proven otherwise. There is no requirement that medical personnel be in attendances and the only "experts" present tend to be selfappointed ones. Crucial decisions about the future of children (and even medical diagnosis and treatment) may be taken as part of a debate based on emotional prejudice rather than professional objectivity. Yet case conferences are answerable to no-one because they have no continued existence, and no individual can be held personally accountable for their decisions. Gross inaccuracies about individuals may be spread without any fear of legal redress,3 and the disparate nature of the group ensures that standards of confidentiality are low. The most frequent outcome of the case conference is that the child’s name is placed upon the at-risk
register.
So we can rely on R’s but not N’s significant results. R was right in 810 of 815 cases (99%) when he rejected Ho, whereas N was right in 90/135 (67%). N, however, did much better than R when he accepted Hg on the basis of a non-significant p value. He was right in 855 of 865 cases (99%) whereas R was right in 95/185 (51 %). When we see a p value, we must consider whether the experiment belongs to N’s or to R’s universe (or perhaps to one more extreme or in between). If the idea is far-fetched it would be unreasonable to rely on a p value of, say, 4 or 5%, whereas it would be equally unreasonabale not to rely on such a p value if there is prior evidence that Ho is false. If the sample size is very small, we must also take into account that there will be a bigger risk of overlooking a true difference: "c" will be larger at the expense of "a" and, consequently, "a - (a + b)" will be smaller. In other words, we should pay less attention to a significant p value when the sample is small. Admittedly, this presentation oversimplifies matters. The calculations are based on the dichotomy ofH true/Ho false. But in reality we must distinguish between no difference at all, small unimportant differences, and larger important differences. Bayesian statisticians have tried to provide the mathematical answers to this complexity, but the principles remain the same. However, I wish to point out that the conventional distinction between signficant and non-significant p values is misleading. It would be a step in the right direction if journals decided not to accept "not significant" and "p < 0-05" but required the actual p value or a narrow range (eg, 0-025 < p < 0-05). Medical Department C, Herlev University Hospital, 2730 Herlev, Denmark
HENRIK R. WULFF
CHILD ABUSE REGISTERS
SiR,—The increase in the number of children placed upon local authority child abuse registers has been cited by the National Society for the Prevention of Cruelty to Children as evidence of an increase in child abuse This argument has been challenged on epidemiological grounds.2 A conflicting view is that the increase in registration shows that the child abuse register as an instrument has failed, and that its failure is a symptom of a deeper malaise in the system for dealing with suspected cases of child abuse. Placement on the register generally follows a case conference. It has been suggested that "the function of each of these two procedures seems to be to justify the other". A case conference is an ad-hoc gathering of workers from widely differing disciplines who meet in an atmosphere of anxiety and suspicion to make decisions,
The at-risk register is a defensive instrument whose main function is to allay the anxieties of the workers involved and protect them against possible future charges of inaction. Registration of a child is often accompanied by "family work", the social worker’s panacaea for all ills. Whilst this can be a stimulating experience for the social worker, it may be of little benefit to the childRegistration may in itself be dangerous, in that it can encourage a false sense of security. Registering a child’s name is no substitute for taking practical steps,’ or pursuing these to their logical conclusion. Registration before this is done is irrelevant and afterwards inappropriate. As a reference document, the register is of doubtful value, anyone consulting it must already have considered the possibility of abuse and should act accordingly, whether or not the name is there. The at-risk register has become a device for those who wish to "play safe" for themselves rather than for the child, and the increase in its use is symptomatic of a paralysis in decision-making. Social workers are in an invidious position in trying to satisfy the often contradictory demands of the public regarding child protection. By inclination and by training, they are ill-equipped to deal with the logical evaluation of risk. The management of suspected child abuse, in particular sexual abuse, should be drawn more directly into the medical domain. Guidelines for good practice are available,* and more are awaited. The at-risk register in its current form has outlived its usefulness. We thank Mr Chris Davies and Mr Harvey Ratner for useful insights.
Departments of Psychiatry, and Paediatrics, St Mary’s Hospital, London W2 1NY
DAVID V. JAMES KATHRYN F. WARD
Creighton SJ. Child abuse in 1986. Initial findings from NSPCC’s register research (NSPCC Res Briefing no 8). London: NSPCC, 1987. 2. Markowe HLJ. The frequency of childhood sexual abuse in the UK. Health Trends 1.
1988; 20: 2-6.
Chapman MGT, Woodmansey AC, Garwood A. Policy on child abuse Soc Clin Psychiat Rep 1985; no 13. 4. Zeitlin H. Investigation of the sexually abused child. Lancet 1987; ii: 842-45. 5. Child abuse. Working together: a draft guide to arrangements for inter-agency co-operation for the protection of children. London: DHSS, 1986. 6. A child in trust. The report of the panel of inquiry into the circumstances surrounding the death of Jasmine Beckford. London Borough of Brent, 1985. 7. Lynch MA, Roberts J. Predicting child abuse. Br Med J 1977; i: 624-26. 8. Independent Second Opinion Panel. Child sexual abuse: Principles of good practice. Br J Hosp Med 1988; 39: 54-62. 3.
FUNDING THE NATIONAL HEALTH SERVICE
SiR,—The press, including even some medical journals, has contained in the past year a great deal of nonsense about the funding of the National Health Service. Irrelevant arithmetic has been paraded by the small group of detractors as if it could be treated as statistical analysis. The proponents of greater use of the private sector seem concerned only to reduce taxes, even though the total cost to the nation may thereby be increased. The least attractive of all the suggestions was recently attributed to managers-namely, that the NHS should concede priority for elective surgery to those willing to pay for that advantage.
SIR,-Does it help the reader to know whether p is below or above 5%? Imagine two scientists (N and R), each of whom did 1000 experiments. N’s ideas were highly original, at times far-fetched. In 900 of his experiments the null hypothesis (Ho) was true. R only repeated the experiments of others; Ho was true in 100 of his experiments (table). In the long run both N and R will obtain a significant result, by chance, in 5 % of cases when Ho is true. In 10% of cases both will sometimes obtain by chance a non-significant result when in fact Ho is false. The frequency of this happening cannot be calculated exactly, but 10% is not unreasonable if the sample size is high. RESULTS OF SIGNIFICANCE TES’I’S ON
1000 EXPERIMENTS BY TWO
HYPOTHETICAL RESEARCHERS
often beyond the scope of their competence,’ on the basis of limited and usually second-hand information. Social workers and nonmedical disciplines have no scientific background, and the prevailing assumption is that hypotheses are valid unless proven otherwise. There is no requirement that medical personnel be in attendances and the only "experts" present tend to be selfappointed ones. Crucial decisions about the future of children (and even medical diagnosis and treatment) may be taken as part of a debate based on emotional prejudice rather than professional objectivity. Yet case conferences are answerable to no-one because they have no continued existence, and no individual can be held personally accountable for their decisions. Gross inaccuracies about individuals may be spread without any fear of legal redress,3 and the disparate nature of the group ensures that standards of confidentiality are low. The most frequent outcome of the case conference is that the child’s name is placed upon the at-risk
register.
So we can rely on R’s but not N’s significant results. R was right in 810 of 815 cases (99%) when he rejected Ho, whereas N was right in 90/135 (67%). N, however, did much better than R when he accepted Hg on the basis of a non-significant p value. He was right in 855 of 865 cases (99%) whereas R was right in 95/185 (51 %). When we see a p value, we must consider whether the experiment belongs to N’s or to R’s universe (or perhaps to one more extreme or in between). If the idea is far-fetched it would be unreasonable to rely on a p value of, say, 4 or 5%, whereas it would be equally unreasonabale not to rely on such a p value if there is prior evidence that Ho is false. If the sample size is very small, we must also take into account that there will be a bigger risk of overlooking a true difference: "c" will be larger at the expense of "a" and, consequently, "a - (a + b)" will be smaller. In other words, we should pay less attention to a significant p value when the sample is small. Admittedly, this presentation oversimplifies matters. The calculations are based on the dichotomy ofH true/Ho false. But in reality we must distinguish between no difference at all, small unimportant differences, and larger important differences. Bayesian statisticians have tried to provide the mathematical answers to this complexity, but the principles remain the same. However, I wish to point out that the conventional distinction between signficant and non-significant p values is misleading. It would be a step in the right direction if journals decided not to accept "not significant" and "p < 0-05" but required the actual p value or a narrow range (eg, 0-025 < p < 0-05). Medical Department C, Herlev University Hospital, 2730 Herlev, Denmark
HENRIK R. WULFF
CHILD ABUSE REGISTERS
SiR,—The increase in the number of children placed upon local authority child abuse registers has been cited by the National Society for the Prevention of Cruelty to Children as evidence of an increase in child abuse This argument has been challenged on epidemiological grounds.2 A conflicting view is that the increase in registration shows that the child abuse register as an instrument has failed, and that its failure is a symptom of a deeper malaise in the system for dealing with suspected cases of child abuse. Placement on the register generally follows a case conference. It has been suggested that "the function of each of these two procedures seems to be to justify the other". A case conference is an ad-hoc gathering of workers from widely differing disciplines who meet in an atmosphere of anxiety and suspicion to make decisions,
The at-risk register is a defensive instrument whose main function is to allay the anxieties of the workers involved and protect them against possible future charges of inaction. Registration of a child is often accompanied by "family work", the social worker’s panacaea for all ills. Whilst this can be a stimulating experience for the social worker, it may be of little benefit to the childRegistration may in itself be dangerous, in that it can encourage a false sense of security. Registering a child’s name is no substitute for taking practical steps,’ or pursuing these to their logical conclusion. Registration before this is done is irrelevant and afterwards inappropriate. As a reference document, the register is of doubtful value, anyone consulting it must already have considered the possibility of abuse and should act accordingly, whether or not the name is there. The at-risk register has become a device for those who wish to "play safe" for themselves rather than for the child, and the increase in its use is symptomatic of a paralysis in decision-making. Social workers are in an invidious position in trying to satisfy the often contradictory demands of the public regarding child protection. By inclination and by training, they are ill-equipped to deal with the logical evaluation of risk. The management of suspected child abuse, in particular sexual abuse, should be drawn more directly into the medical domain. Guidelines for good practice are available,* and more are awaited. The at-risk register in its current form has outlived its usefulness. We thank Mr Chris Davies and Mr Harvey Ratner for useful insights.
Departments of Psychiatry, and Paediatrics, St Mary’s Hospital, London W2 1NY
DAVID V. JAMES KATHRYN F. WARD
Creighton SJ. Child abuse in 1986. Initial findings from NSPCC’s register research (NSPCC Res Briefing no 8). London: NSPCC, 1987. 2. Markowe HLJ. The frequency of childhood sexual abuse in the UK. Health Trends 1.
1988; 20: 2-6.
Chapman MGT, Woodmansey AC, Garwood A. Policy on child abuse Soc Clin Psychiat Rep 1985; no 13. 4. Zeitlin H. Investigation of the sexually abused child. Lancet 1987; ii: 842-45. 5. Child abuse. Working together: a draft guide to arrangements for inter-agency co-operation for the protection of children. London: DHSS, 1986. 6. A child in trust. The report of the panel of inquiry into the circumstances surrounding the death of Jasmine Beckford. London Borough of Brent, 1985. 7. Lynch MA, Roberts J. Predicting child abuse. Br Med J 1977; i: 624-26. 8. Independent Second Opinion Panel. Child sexual abuse: Principles of good practice. Br J Hosp Med 1988; 39: 54-62. 3.
FUNDING THE NATIONAL HEALTH SERVICE
SiR,—The press, including even some medical journals, has contained in the past year a great deal of nonsense about the funding of the National Health Service. Irrelevant arithmetic has been paraded by the small group of detractors as if it could be treated as statistical analysis. The proponents of greater use of the private sector seem concerned only to reduce taxes, even though the total cost to the nation may thereby be increased. The least attractive of all the suggestions was recently attributed to managers-namely, that the NHS should concede priority for elective surgery to those willing to pay for that advantage.