- Email: [email protected]
European vascular registries: Vascunet and Eurovasc
International Congress Series 1272 (2004) 130 – 136
www.ics-elsevier.com
European vascular registries: Vascunet and Eurovasc Leif Panduro Jensen a,*, Jesper Laustsen b b
a Department for Vascular Surgery B, Gentofte University Hospital, Hellerup DK-2900, Denmark Department of Cardiothoracic & Vascular Surgery T, Aarhus University Hospital, Skejby Sygehus, Denmark
Keywords: Vascular surgery
1. Introduction Vascunet is a forum for vascular registries, mainly European. It has until now been an informal forum for exchange of experiences between the existing vascular registries, and has served as inspiration for colleagues involved in the creation of new registries. Recently, Vascunet became a more formal and permanent body under the Vascular Society. The concept of vascular registries began in Europe in the mid-1980s. The purpose of a vascular registry is provision of data on everyday performance, providing the basis for promotion of advances in the treatment of vascular diseases, research, statistical purposes and surgical audit. The strength of a vascular registry is that data reflect results and complications of everyday practice more realistically than a series published in the medical literature. At the same time, this is the weakness of a registry that data collection and validity are less controlled and the case mix of patients often leads to difficulties in interpretation of data. If used the right way and with proper caution to these aspects, registries have proven to be valuable sources of information over the last decade. 2. The history and organisation of Vascunet Vascunet was founded in 1996, at an informal meeting between a few European registries, at the ESVS meeting in Lisbon. It has since then grown in size; in 2003 more than 10 countries contributed actively to the content of the meeting. The Vascunet meetings have always been in connection with the yearly meeting of the European Society for Vascular Surgery. At first, the Vascunet meetings only took place as small lunch meetings, but in recent years with more extensive meetings and even scientific symposiums on matters related to vascular registries. This development will probably continue to grow and extend in the future. Up till 2003, there was no formal leadership of Vascunet. Thomas Troe¨ng from Sweedvasc chaired the first meetings, which later on continued with Leif Panduro * Corresponding author. Tel.: +45-3977-3403; fax: +45-3977-7674. E-mail address: [email protected] (L.P. Jensen). 0531-5131/ D 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.ics.2004.05.079
L.P. Jensen, J. Laustsen / International Congress Series 1272 (2004) 130–136
131
Jensen from the Danish Vascular Registry (Karbase), who also arranged the first symposiums. In 2003, at the meeting in Dublin, it was decided to create a more formal organisation with a steering group and a spokesman. Martin Bjo¨rck (spokesman, Sweden), Leif Panduro Jensen (Denmark), Jesper Laustsen (Denmark), Rosa M. Moreno Carriles (Spain) and Simon Ashley (UK) were elected. The purpose of this group was to further develop the concept of Vascunet into a more permanent group or committee under the ESVS, and to further strengthen the cooperation between the existing and developing vascular registries. 3. Eurovasc Along with the creation of Vascunet, Prof. W. Paaske did some work on collecting information regarding vascular surgery in Europe from volunteering registries. These data were published in the Journal of Vascular and Endovascular Surgery as the Eurovasc Report. The Eurovasc Report only contained figures on number of procedures for each of the defined index procedures [1] (Table 1), and not any other parameters. Even with this limited information, the figures showed remarkable differences in practice among the different countries, thus raising interesting questions on the basis of differences in how often various procedures were performed and the proportion of endovascular to surgical procedures, etc. There has been a pause in these publications since 2001, where the figures from 1999 were published. Their continuation will be a task for the reorganised Vascunet in the future. 4. Why is Vascunet needed? The world is getting smaller, Europe as well, and also in the field of Vascular Surgery. At the same time, as the professional organisations are becoming increasingly important, the patients are becoming increasingly alert to the conditions and possibilities Table 1 Eurovasc index procedures Open surgical procedures A B C D E
Carotid Artery disease Abdominal Aortic Aneurysm Central Arterial Occlusive disease Peripheral Arterial Occlusive disease above knee Peripheral Arterial Occlusive disease below knee
Endovascular procedures F G H I J
Central PTA F stent Peripheral PTA F stent Stent graft for Abdominal Aortic Aneurysm Stent graft for central Arterial Occlusive disease Stent graft for peripheral Arterial Occlusive disease
Others K L
Arterial trauma Angio access for dialysis
132
L.P. Jensen, J. Laustsen / International Congress Series 1272 (2004) 130–136
in other neighbouring countries, demanding the same good treatment all over Europe. Since patient outcome reflects the provided quality of care and treatment, the tools of outcome research are increasingly being utilised to evaluate this. So, even if the first purpose of the registries is to support professional knowledge, development and rising quality, we have the patients supporting the same purposes, together with the wish for extensive public transparency. Vascunet is, and will only be, for a long time, a forum for discussions. We are still far from the demand of common European standards for treatment. That is not to say that there is not already a need for some European standardisation in collection and reporting of data to enable comparison of results, exactly as is the case for reporting of randomised trials. Since vascular surgery is based on a scientific basis all over Europe, we of course also must acknowledge the need for proper, uniform methodology in reporting standards from the registries. Much work is still needed in this area to accomplish the task, stressing the need for the work done in Vascunet, and stressing the need for tight bonds to the scientific organisation. One obvious need is to establish a minimum data set (MDS) to be included in all the European registries. The MDS should indeed be a minimum data set, a set of core variables, but most registries would choose to include more variables. First of all, the MDS should describe the vascular procedures and indications, but in time also include basic quality parameters, e.g. result and complication parameters, especially in areas where the scientific basis for setting limits is present. Discussions should, of course, not be limited to the minimum data set, but also exchange of views and evidence for suitable parameters for different vascular procedures is important. Another very important question is regarding validation of data, where common ground and development are needed. See below for further details. Only with valid data it is possible to have reliable reporting. Vascunet can play an important role in defining minimum standards for reporting of results. Until now, the only fairly reliable reports for comparison across Europe have been the number of certain index procedures, but this will surely be extended in the future with the increasing demand of transparency in the health sector. There is an obligation to monitor results and quality, and to make data available to the public to some extent. In several countries, e.g. Denmark, patients have the right to freely choose the place of treatment. In this respect, patients are asking for information on ‘‘best quality,’’ and the politicians are moving along the same line in demanding the same information. This demand is not without problems. If reliable reporting is to be achieved, the whole chain of information should be of high quality as discussed below, that is, in the areas of data collection, data validation and case mix correction. From a political point of view, this focus on transparency also seems a little narrow. The goal for all countries should be to deliver health service of high quality at every place to every citizen. The best way for a registry to serve this purpose would be as the foundation for scientific discussion and development among colleagues, not by publishing ‘‘top 10’’ lists of best and worst. It is to be remembered, that for any quality parameter, half of the departments/surgeons will always be below average and half above! If too much emphasis is put on public presentation of individual results, the consequence could be defensive surgeons protecting their statistics rather than accepting the challenge of especially weak
L.P. Jensen, J. Laustsen / International Congress Series 1272 (2004) 130–136
133
patients. The balance of public presentation with the need for collecting data for quality improvement will be one of the major challenges in the coming years in Europe. 5. Establishing and maintaining a vascular registry As mentioned, there are huge differences in the different European countries regarding vascular registries. Some have had registries for many years, and some have not even started yet, but everyone recognises the need for them (see the chapter by Troeng et al. in this book for an overview of the current status of each country). Several issues need to be addressed to ‘‘do it right’’ when establishing a vascular registry, and the same issues need to be repeated from time to time to maintain a well-handled and reliable registry. 6. The data set The proper set of data to include in the registry provides a foundation, and should be given a lot of attention. Some countries, like the Nordic ones, have a lot of experience, as well as knowledge that can be drawn from the literature. Some general lessons have been learned by the existing registries:
Keep it simple! It is easy to be excited with the possibilities of having an extensive amount of data for analysis, but remember that the data should be collected in the busy, everyday practise of each surgeon. Keep the amount of data as low as possible, but still enough to allow case mix correction and risk adjustment of the patients’ ‘‘health profile,’’ together with information on procedures, results, complications and followup. Also, keep each data entry point as simple as possible, easy to fill in, and robust to inter-observer variations. So, try to keep it as close to ‘‘yes/no’’ questions as possible and try to pose self-explanatory questions [2]. Always have a patient ID, if at all possible. Some countries have huge difficulties in this matter (e.g. Finland and Switzerland), but if even a 30-day follow-up on results and complications shall be possible, there is need for proper ID. The patient ID is also the basis for validation of completeness of data towards other registries, operation protocols etc. Another question is whether to include all of the vascular procedures of limiting the registration to a few index procedures [3]. The latter is naturally not as time consuming as the former, but then leaves the question if the index procedures are the right ones to evaluate overall performance. 7. The collection of data The technology is so advanced today that one should aim for electronic transferral of data to the registry, but there are still huge differences in this matter in Europe. Some still use paper forms sent by ordinary mail and fed to the computer by a secretary and a few, like the Danish Vascular Registry, have for years had internet-based software with the database placed on a national server. A vascular registry is probably often seen as a continuous collection of data, but this need not be the case as one can choose to collect data at certain intervals as samples. The
134
L.P. Jensen, J. Laustsen / International Congress Series 1272 (2004) 130–136
advantage is, of course, that less time and effort is needed, but the disadvantages are that the same organisation is needed for both solutions, the same training and maintenance, and most importantly, changes over time will be more difficult to detect due to pauses and reduced amount of data in a given time period. Another important question is who is going to type the data into the computer. The best quality is obviously obtained by the surgeons doing it themselves, but this is also time consuming. Combining data, with incorporation from various electronic sources and the possibility for automated, continuous check and filtration by the software, will ensure adequate quality for some of the data, while others will be dependent on clinical judgement and honesty. Especially in the USA, it is common to have trained nurses do the registration with the help from the surgeons, thereby securing complete and accurate data, but this is costly [4]. One of the difficulties for the registries is that it is still very little recognised that there should be money and resources allocated also in the health sector for this task, as it has been for many years in the private business sector.
8. Validation of data This is probably the most difficult of all the problems of maintaining a registry [2,5]. Many reports are published without addressing the question of completeness and the quality of data [3]. This task is very time-consuming and takes a lot of resources, which is often used as the reason for not prioritising this in the end [3]. The problem is obvious: if you do not catch, for example, data on complications, the results will seem fine but will, in fact, be unreliable. Several validation questions should be addressed:
Completeness of data. If at all possible, data should be checked with other sources to see if all patients are included as intended. If not, efforts should be made to include the remaining parts or to verify that the inclusion/exclusion is random and not an important source of bias [6]. The question of completeness has always been related to how difficult it has been to obtain data—the nearer to the source, e.g. filling in operation data right after leaving the operation theatre, the easier it is to include all the data and to check if it is done. By sending forms by mail to another address, completeness is seldom above 90%. Reliable and reproducible data. The first factor to obtain this goal is, of course, to design the data set properly, to avoid any misunderstanding of which data to include and how to fill in the forms. The next is to have extensive validation built into the software itself, especially in the case of direct typing into the software by the surgeons, instead of by a dedicated person. There are three levels of validation by the software here: the first and most basic, that only correct values are allowed and you are warned in case of empty required fields. Second is validation across the different fields in a form, e.g. that time for end of operation is after time for start of operation. Third is validation across forms, e.g. that dates are correctly ordered or status for limb amputation affects questions about ankle blood pressure in future forms. Finally, it is important to secure that each surgeon/ department uses the same criterions for putting data into the registry. To obtain this goal, there is a need for external review of the practice, something that should be done on a regular basis [2].
L.P. Jensen, J. Laustsen / International Congress Series 1272 (2004) 130–136
135
9. Reporting from the registry Feedback to the surgeons and departments is, by far, the most motivating factor in keeping a healthy and reliable registry. The main purpose of a vascular registry should always be to promote discussions and dissimilation of principles for best practise and, only to a lesser extent, to provide data to the public. If focus is primarily on the latter and not the former, engagement will rapidly drop and reports will become unreliable. The basic reporting is not much different from basic scientific reporting. It should include figures on number of procedures, demographic data, results (including standard deviation), and complication rate. The methods used should also be the best scientific and proven ones with emphasis on the very robust ones, since lesser-controlled data create greater risk of bias [6]. It is extremely important that reporting from a registry is standardised and collected on a regular basis to serve the surgeons in the best possible way. Since data in a registry are often collected continuously over a long period of time, this allows both for cross-sectional analysis, e.g. benchmarking, of results, with comparison between different clinics or surgeons, and for time series analysis, where fluctuation over time for each clinic or surgeon is presented. For the latter, much development has taken place in the industry, creating control charts with fixed rules for alarming and reducing the risk of false alarms due to statistical uncertainty, but that the alarm is sounded if suspicious trends are evolving [7]. These methods are currently tested by the Danish Vascular Registry, but many other methods are tested by others too at the moment [3] (Fig. 1). It is often said that reporting can only be fair if it is risk-adjusted for differences in case mix. The basic assumption is that some surgeons or departments handle the very difficult cases, and thereby may accept a higher complication rate and poorer result than those handling the more fit patients. This is, of course, true and risk-adjusted data should be generated under most circumstances, especially if the aim is publication or benchmarking. However, it is not always the whole truth; there is a range of problems when trying to interpret even valid and well-adjusted data. It is crucial to accept that an outcome better
Fig. 1. Control Chart from the Danish Vascular Registry, showing mortality after elective aneurysm repair.
136
L.P. Jensen, J. Laustsen / International Congress Series 1272 (2004) 130–136
than an accepted standard does not necessarily reflect a better performance [8]. Another problem is that one department may have risk-adjusted mortality at an average level for elective aneurysms with a crude rate triple the average, then revealing that very ill patients are offered surgery despite the fact that there may be an unfavourable outcome compared to the natural history. So, crude rates are equally important as the basis for discussion among colleagues, but may not be fit for publication in a broader sense. Reporting in itself is not enough; it should be scrutinised through an audit process in order to draw learning from the reports and implement it back into the system. Also, in the border between science and quality improvement, the reports can be used as tools for generating theses on different ‘‘seem to be’’ problems, to be tested later in a randomised, controlled trial. Lastly, but equally important, reports can be used for specialist training, as a tool for documenting training in different surgical procedures [9]. 10. The future of Vascunet The coming years will emphasise the need for documentation of results, for standardisation of treatment as an instrument for improvement and for transparency to the public. All these goals can be achieved by vascular registries among other tools, and Vascunet will play an important role in connecting the different national initiatives and discussions, promoting even faster development. The process is already running, along with the increasing awareness in the scientific society. References [1] W. Paaske, EUROVASC report 1999, Eur. J. Vasc. Endovasc. Surg. 22 (3) (2001 Sep.) 282. [2] J. Laustsen, L.P. Jensen, A.K. Hansen, Accuracy of Clinical Data in a Population Based Vascular Registry, Eur. J. Vasc. Endovasc. Surg. 27 (2004) 216 – 219. [3] S. Ashley, B. Ridler, National Vascular Database Report 2002, The Vascular Surgical Society of Great Britain and Ireland, Report published by Dendrite Clinical Systems, [email protected]. [4] A.S. Fink, et al., The National Surgical Quality Improvement Program in non-Veteran Administration Hospitals, Ann. Surg. 236 (3) (2002) 344 – 354. [5] L.G. Fine, et al., How to evaluate and improve the quality and credibility of an outcomes database: validation and feedback study on the UK Cardiac Surgery Experience, BMJ 326 (2003) 25 – 28. [6] L.P. Jensen, O.M. Nielsen, T.V. Schroeder, The importance of complete follow-up for results after Femoroinfrapopliteal vascular surgery, Eur. J. Vasc. Endovasc. Surg. 12 (3) (1996) 282 – 286. [7] J.C. Benneyan, Number-Between g-Type Statistical Quality Control Charts for Monitoring Adverse Events, Health Care Manage. Sci. 4 (2001) 305 – 318. [8] P.S. Basnyat, et al., Mortality from ruptured abdominal aortic aneurysm in Wales, Br. J. Surg. 86 (1999) 765 – 770. [9] J. Sandermann, L. Panduro Jensen, The Danish Specialist Training in Vascular Surgery, Eur. J. Vasc. Endovasc. Surg. 23 (2002) 353 – 357.
www.ics-elsevier.com
European vascular registries: Vascunet and Eurovasc Leif Panduro Jensen a,*, Jesper Laustsen b b
a Department for Vascular Surgery B, Gentofte University Hospital, Hellerup DK-2900, Denmark Department of Cardiothoracic & Vascular Surgery T, Aarhus University Hospital, Skejby Sygehus, Denmark
Keywords: Vascular surgery
1. Introduction Vascunet is a forum for vascular registries, mainly European. It has until now been an informal forum for exchange of experiences between the existing vascular registries, and has served as inspiration for colleagues involved in the creation of new registries. Recently, Vascunet became a more formal and permanent body under the Vascular Society. The concept of vascular registries began in Europe in the mid-1980s. The purpose of a vascular registry is provision of data on everyday performance, providing the basis for promotion of advances in the treatment of vascular diseases, research, statistical purposes and surgical audit. The strength of a vascular registry is that data reflect results and complications of everyday practice more realistically than a series published in the medical literature. At the same time, this is the weakness of a registry that data collection and validity are less controlled and the case mix of patients often leads to difficulties in interpretation of data. If used the right way and with proper caution to these aspects, registries have proven to be valuable sources of information over the last decade. 2. The history and organisation of Vascunet Vascunet was founded in 1996, at an informal meeting between a few European registries, at the ESVS meeting in Lisbon. It has since then grown in size; in 2003 more than 10 countries contributed actively to the content of the meeting. The Vascunet meetings have always been in connection with the yearly meeting of the European Society for Vascular Surgery. At first, the Vascunet meetings only took place as small lunch meetings, but in recent years with more extensive meetings and even scientific symposiums on matters related to vascular registries. This development will probably continue to grow and extend in the future. Up till 2003, there was no formal leadership of Vascunet. Thomas Troe¨ng from Sweedvasc chaired the first meetings, which later on continued with Leif Panduro * Corresponding author. Tel.: +45-3977-3403; fax: +45-3977-7674. E-mail address: [email protected] (L.P. Jensen). 0531-5131/ D 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.ics.2004.05.079
L.P. Jensen, J. Laustsen / International Congress Series 1272 (2004) 130–136
131
Jensen from the Danish Vascular Registry (Karbase), who also arranged the first symposiums. In 2003, at the meeting in Dublin, it was decided to create a more formal organisation with a steering group and a spokesman. Martin Bjo¨rck (spokesman, Sweden), Leif Panduro Jensen (Denmark), Jesper Laustsen (Denmark), Rosa M. Moreno Carriles (Spain) and Simon Ashley (UK) were elected. The purpose of this group was to further develop the concept of Vascunet into a more permanent group or committee under the ESVS, and to further strengthen the cooperation between the existing and developing vascular registries. 3. Eurovasc Along with the creation of Vascunet, Prof. W. Paaske did some work on collecting information regarding vascular surgery in Europe from volunteering registries. These data were published in the Journal of Vascular and Endovascular Surgery as the Eurovasc Report. The Eurovasc Report only contained figures on number of procedures for each of the defined index procedures [1] (Table 1), and not any other parameters. Even with this limited information, the figures showed remarkable differences in practice among the different countries, thus raising interesting questions on the basis of differences in how often various procedures were performed and the proportion of endovascular to surgical procedures, etc. There has been a pause in these publications since 2001, where the figures from 1999 were published. Their continuation will be a task for the reorganised Vascunet in the future. 4. Why is Vascunet needed? The world is getting smaller, Europe as well, and also in the field of Vascular Surgery. At the same time, as the professional organisations are becoming increasingly important, the patients are becoming increasingly alert to the conditions and possibilities Table 1 Eurovasc index procedures Open surgical procedures A B C D E
Carotid Artery disease Abdominal Aortic Aneurysm Central Arterial Occlusive disease Peripheral Arterial Occlusive disease above knee Peripheral Arterial Occlusive disease below knee
Endovascular procedures F G H I J
Central PTA F stent Peripheral PTA F stent Stent graft for Abdominal Aortic Aneurysm Stent graft for central Arterial Occlusive disease Stent graft for peripheral Arterial Occlusive disease
Others K L
Arterial trauma Angio access for dialysis
132
L.P. Jensen, J. Laustsen / International Congress Series 1272 (2004) 130–136
in other neighbouring countries, demanding the same good treatment all over Europe. Since patient outcome reflects the provided quality of care and treatment, the tools of outcome research are increasingly being utilised to evaluate this. So, even if the first purpose of the registries is to support professional knowledge, development and rising quality, we have the patients supporting the same purposes, together with the wish for extensive public transparency. Vascunet is, and will only be, for a long time, a forum for discussions. We are still far from the demand of common European standards for treatment. That is not to say that there is not already a need for some European standardisation in collection and reporting of data to enable comparison of results, exactly as is the case for reporting of randomised trials. Since vascular surgery is based on a scientific basis all over Europe, we of course also must acknowledge the need for proper, uniform methodology in reporting standards from the registries. Much work is still needed in this area to accomplish the task, stressing the need for the work done in Vascunet, and stressing the need for tight bonds to the scientific organisation. One obvious need is to establish a minimum data set (MDS) to be included in all the European registries. The MDS should indeed be a minimum data set, a set of core variables, but most registries would choose to include more variables. First of all, the MDS should describe the vascular procedures and indications, but in time also include basic quality parameters, e.g. result and complication parameters, especially in areas where the scientific basis for setting limits is present. Discussions should, of course, not be limited to the minimum data set, but also exchange of views and evidence for suitable parameters for different vascular procedures is important. Another very important question is regarding validation of data, where common ground and development are needed. See below for further details. Only with valid data it is possible to have reliable reporting. Vascunet can play an important role in defining minimum standards for reporting of results. Until now, the only fairly reliable reports for comparison across Europe have been the number of certain index procedures, but this will surely be extended in the future with the increasing demand of transparency in the health sector. There is an obligation to monitor results and quality, and to make data available to the public to some extent. In several countries, e.g. Denmark, patients have the right to freely choose the place of treatment. In this respect, patients are asking for information on ‘‘best quality,’’ and the politicians are moving along the same line in demanding the same information. This demand is not without problems. If reliable reporting is to be achieved, the whole chain of information should be of high quality as discussed below, that is, in the areas of data collection, data validation and case mix correction. From a political point of view, this focus on transparency also seems a little narrow. The goal for all countries should be to deliver health service of high quality at every place to every citizen. The best way for a registry to serve this purpose would be as the foundation for scientific discussion and development among colleagues, not by publishing ‘‘top 10’’ lists of best and worst. It is to be remembered, that for any quality parameter, half of the departments/surgeons will always be below average and half above! If too much emphasis is put on public presentation of individual results, the consequence could be defensive surgeons protecting their statistics rather than accepting the challenge of especially weak
L.P. Jensen, J. Laustsen / International Congress Series 1272 (2004) 130–136
133
patients. The balance of public presentation with the need for collecting data for quality improvement will be one of the major challenges in the coming years in Europe. 5. Establishing and maintaining a vascular registry As mentioned, there are huge differences in the different European countries regarding vascular registries. Some have had registries for many years, and some have not even started yet, but everyone recognises the need for them (see the chapter by Troeng et al. in this book for an overview of the current status of each country). Several issues need to be addressed to ‘‘do it right’’ when establishing a vascular registry, and the same issues need to be repeated from time to time to maintain a well-handled and reliable registry. 6. The data set The proper set of data to include in the registry provides a foundation, and should be given a lot of attention. Some countries, like the Nordic ones, have a lot of experience, as well as knowledge that can be drawn from the literature. Some general lessons have been learned by the existing registries:
Keep it simple! It is easy to be excited with the possibilities of having an extensive amount of data for analysis, but remember that the data should be collected in the busy, everyday practise of each surgeon. Keep the amount of data as low as possible, but still enough to allow case mix correction and risk adjustment of the patients’ ‘‘health profile,’’ together with information on procedures, results, complications and followup. Also, keep each data entry point as simple as possible, easy to fill in, and robust to inter-observer variations. So, try to keep it as close to ‘‘yes/no’’ questions as possible and try to pose self-explanatory questions [2]. Always have a patient ID, if at all possible. Some countries have huge difficulties in this matter (e.g. Finland and Switzerland), but if even a 30-day follow-up on results and complications shall be possible, there is need for proper ID. The patient ID is also the basis for validation of completeness of data towards other registries, operation protocols etc. Another question is whether to include all of the vascular procedures of limiting the registration to a few index procedures [3]. The latter is naturally not as time consuming as the former, but then leaves the question if the index procedures are the right ones to evaluate overall performance. 7. The collection of data The technology is so advanced today that one should aim for electronic transferral of data to the registry, but there are still huge differences in this matter in Europe. Some still use paper forms sent by ordinary mail and fed to the computer by a secretary and a few, like the Danish Vascular Registry, have for years had internet-based software with the database placed on a national server. A vascular registry is probably often seen as a continuous collection of data, but this need not be the case as one can choose to collect data at certain intervals as samples. The
134
L.P. Jensen, J. Laustsen / International Congress Series 1272 (2004) 130–136
advantage is, of course, that less time and effort is needed, but the disadvantages are that the same organisation is needed for both solutions, the same training and maintenance, and most importantly, changes over time will be more difficult to detect due to pauses and reduced amount of data in a given time period. Another important question is who is going to type the data into the computer. The best quality is obviously obtained by the surgeons doing it themselves, but this is also time consuming. Combining data, with incorporation from various electronic sources and the possibility for automated, continuous check and filtration by the software, will ensure adequate quality for some of the data, while others will be dependent on clinical judgement and honesty. Especially in the USA, it is common to have trained nurses do the registration with the help from the surgeons, thereby securing complete and accurate data, but this is costly [4]. One of the difficulties for the registries is that it is still very little recognised that there should be money and resources allocated also in the health sector for this task, as it has been for many years in the private business sector.
8. Validation of data This is probably the most difficult of all the problems of maintaining a registry [2,5]. Many reports are published without addressing the question of completeness and the quality of data [3]. This task is very time-consuming and takes a lot of resources, which is often used as the reason for not prioritising this in the end [3]. The problem is obvious: if you do not catch, for example, data on complications, the results will seem fine but will, in fact, be unreliable. Several validation questions should be addressed:
Completeness of data. If at all possible, data should be checked with other sources to see if all patients are included as intended. If not, efforts should be made to include the remaining parts or to verify that the inclusion/exclusion is random and not an important source of bias [6]. The question of completeness has always been related to how difficult it has been to obtain data—the nearer to the source, e.g. filling in operation data right after leaving the operation theatre, the easier it is to include all the data and to check if it is done. By sending forms by mail to another address, completeness is seldom above 90%. Reliable and reproducible data. The first factor to obtain this goal is, of course, to design the data set properly, to avoid any misunderstanding of which data to include and how to fill in the forms. The next is to have extensive validation built into the software itself, especially in the case of direct typing into the software by the surgeons, instead of by a dedicated person. There are three levels of validation by the software here: the first and most basic, that only correct values are allowed and you are warned in case of empty required fields. Second is validation across the different fields in a form, e.g. that time for end of operation is after time for start of operation. Third is validation across forms, e.g. that dates are correctly ordered or status for limb amputation affects questions about ankle blood pressure in future forms. Finally, it is important to secure that each surgeon/ department uses the same criterions for putting data into the registry. To obtain this goal, there is a need for external review of the practice, something that should be done on a regular basis [2].
L.P. Jensen, J. Laustsen / International Congress Series 1272 (2004) 130–136
135
9. Reporting from the registry Feedback to the surgeons and departments is, by far, the most motivating factor in keeping a healthy and reliable registry. The main purpose of a vascular registry should always be to promote discussions and dissimilation of principles for best practise and, only to a lesser extent, to provide data to the public. If focus is primarily on the latter and not the former, engagement will rapidly drop and reports will become unreliable. The basic reporting is not much different from basic scientific reporting. It should include figures on number of procedures, demographic data, results (including standard deviation), and complication rate. The methods used should also be the best scientific and proven ones with emphasis on the very robust ones, since lesser-controlled data create greater risk of bias [6]. It is extremely important that reporting from a registry is standardised and collected on a regular basis to serve the surgeons in the best possible way. Since data in a registry are often collected continuously over a long period of time, this allows both for cross-sectional analysis, e.g. benchmarking, of results, with comparison between different clinics or surgeons, and for time series analysis, where fluctuation over time for each clinic or surgeon is presented. For the latter, much development has taken place in the industry, creating control charts with fixed rules for alarming and reducing the risk of false alarms due to statistical uncertainty, but that the alarm is sounded if suspicious trends are evolving [7]. These methods are currently tested by the Danish Vascular Registry, but many other methods are tested by others too at the moment [3] (Fig. 1). It is often said that reporting can only be fair if it is risk-adjusted for differences in case mix. The basic assumption is that some surgeons or departments handle the very difficult cases, and thereby may accept a higher complication rate and poorer result than those handling the more fit patients. This is, of course, true and risk-adjusted data should be generated under most circumstances, especially if the aim is publication or benchmarking. However, it is not always the whole truth; there is a range of problems when trying to interpret even valid and well-adjusted data. It is crucial to accept that an outcome better
Fig. 1. Control Chart from the Danish Vascular Registry, showing mortality after elective aneurysm repair.
136
L.P. Jensen, J. Laustsen / International Congress Series 1272 (2004) 130–136
than an accepted standard does not necessarily reflect a better performance [8]. Another problem is that one department may have risk-adjusted mortality at an average level for elective aneurysms with a crude rate triple the average, then revealing that very ill patients are offered surgery despite the fact that there may be an unfavourable outcome compared to the natural history. So, crude rates are equally important as the basis for discussion among colleagues, but may not be fit for publication in a broader sense. Reporting in itself is not enough; it should be scrutinised through an audit process in order to draw learning from the reports and implement it back into the system. Also, in the border between science and quality improvement, the reports can be used as tools for generating theses on different ‘‘seem to be’’ problems, to be tested later in a randomised, controlled trial. Lastly, but equally important, reports can be used for specialist training, as a tool for documenting training in different surgical procedures [9]. 10. The future of Vascunet The coming years will emphasise the need for documentation of results, for standardisation of treatment as an instrument for improvement and for transparency to the public. All these goals can be achieved by vascular registries among other tools, and Vascunet will play an important role in connecting the different national initiatives and discussions, promoting even faster development. The process is already running, along with the increasing awareness in the scientific society. References [1] W. Paaske, EUROVASC report 1999, Eur. J. Vasc. Endovasc. Surg. 22 (3) (2001 Sep.) 282. [2] J. Laustsen, L.P. Jensen, A.K. Hansen, Accuracy of Clinical Data in a Population Based Vascular Registry, Eur. J. Vasc. Endovasc. Surg. 27 (2004) 216 – 219. [3] S. Ashley, B. Ridler, National Vascular Database Report 2002, The Vascular Surgical Society of Great Britain and Ireland, Report published by Dendrite Clinical Systems, [email protected]. [4] A.S. Fink, et al., The National Surgical Quality Improvement Program in non-Veteran Administration Hospitals, Ann. Surg. 236 (3) (2002) 344 – 354. [5] L.G. Fine, et al., How to evaluate and improve the quality and credibility of an outcomes database: validation and feedback study on the UK Cardiac Surgery Experience, BMJ 326 (2003) 25 – 28. [6] L.P. Jensen, O.M. Nielsen, T.V. Schroeder, The importance of complete follow-up for results after Femoroinfrapopliteal vascular surgery, Eur. J. Vasc. Endovasc. Surg. 12 (3) (1996) 282 – 286. [7] J.C. Benneyan, Number-Between g-Type Statistical Quality Control Charts for Monitoring Adverse Events, Health Care Manage. Sci. 4 (2001) 305 – 318. [8] P.S. Basnyat, et al., Mortality from ruptured abdominal aortic aneurysm in Wales, Br. J. Surg. 86 (1999) 765 – 770. [9] J. Sandermann, L. Panduro Jensen, The Danish Specialist Training in Vascular Surgery, Eur. J. Vasc. Endovasc. Surg. 23 (2002) 353 – 357.