How to choose a child heart surgeon
One key recommendation of the Bristol Inquiry 11 years ago was that ‘patients must be able to obtain information as to the relative performance of the trust and the services and consultant units within the trust.’ The Inquiry concluded that between 30 and 35 more children had died after heart surgery between 1991 and 1995 in Bristol compared to a typical unit in England at that time. So how are England’s child heart surgeons performing now?
The official figures for 2010-2011 look superficially reassuring. Over fifty procedures are listed and if you know which one you child is having, you can look at the results in your hospital and compare it to other units on a graph. If you look at, say, the results for the arterial shunt operation, you’ll see that half the units are above average, half are below average but none of them breach the ‘significant statistical outlier’ line that would trigger an investigation. The same is true of pretty much all of the procedures, but the problem is that for many operations, the numbers per hospital are so small as to be statistically meaningless and you’d have to be truly shocking to trigger an investigation.
Generally, the more of an operation you do, the better you get at doing it and teaching it, and the easier it is to statistically prove your competence, which is why the Bristol Inquiry recommended a reduction in centres performing surgery. The fact that we still haven’t managed this 11 years after the inquiry and over 20 years since the Eye broke the Bristol scandal is, in the words of NHS Medical Director Bruce Keogh ‘a stain on the soul of the specialty.’
Health secretary Jeremy Hunt has just ordered a review into the current review. So the Independent Reconfiguration Panel (IRP) will now decide if the Joint Committee of Primary Care Trusts (JCPCT) has made the correct choices in its proposed reduction in the number of surgical centres from 11 to 7, each with a critical mass of 400 operations a year (Eyes passim). Just about everyone agrees that child heart surgery in England would be safer if expertise, resources, research and training were concentrated in fewer centres, but no-one wants their local centre to close.
Meanwhile, expertise is spread too thinly and units are buckling under the strain. The Care Quality Commission (CQC) has investigated after a recent child cardiac death and other safety concerns in Bristol, has issued a formal warning to University Hospitals Bristol FT and concluded the post-operative ward is understaffed.
The JPCT chose not to use any outcome data in reaching its decisions because the numbers were too small. But if you add together the data for four or more years you get a much better idea of which units are performing best. Last year, Professor Sir Brian Jarman, who heads the Imperial College Dr Foster Intelligence Unit, did a comparative analysis using the best data available to him and the same techniques that uncovered Bristol. He sent it to Keogh but got no response. He has just repeated the analysis for 2006-2012, sent it to Keogh and put it on his website. For open heart surgery in the under 5’s over the last 6 years, there are six units that have lower than the expected mortality ratio of 100: Leicester (45), Bristol (47), Brompton (65), Southampton (67), Newcastle (68) and Great Ormond Street (73). Leicester and the Brompton are marked for closure despite being in the top 3 for overall survival. Of the others chosen by the JCPCT, Birmingham has a mortality ratio of 110, Alder Hey 120 and Guys and St Thomas’s 128. Leeds (120) has not been chosen and Oxford (160) has been stopped from operating. The JCPCT argues that when (if) their proposals go through in 2014, the amalgamated expertise in the 7 units should ensure excellent results whatever the hospital site (if the staff work together and put their rivalries behind them). But if I could choose where my child went for open heart surgery tomorrow, I’d check out the staffing levels in Bristol but otherwise opt for any of the top six.
E mail correspondence post publication
To: email@example.com Subject:
RE: Outcomes of heart surgery in children
Date: Wed, 31 Oct 2012 15:43:01 +0000
Enjoyed your article in Eye 1326 about Childrens’ Heart Surgery. In the CCAD Portal you can, as you say, observe data for each year. However the funnel-plot graphs (which are just about to be updated) show a THREE year view of the data, not just a single year – because, as you say, numbers get just too small. We thought three year was a good compromise between avoiding small numbers and using data that was too old to reflect current practice. Brian Jarman has chosen two cuts of the HES data, 6-year and 4-year. The casemix adjustment, or lack of it due to the inadequacies of HES codings, may explain some of the apparent differences. For instance Brimingham and Guys do most of the Norwood procedures for hypoplastic left heart syndrome, which may explain their relatively high SMRs. In our non-casemix adjusted analysis of the same 6 year period, inclusion or exclusion of Norwood ops for HLHS made a difference of 20 to the SMRs of Guys and Birmingham.
It is extremely difficult to risk adjust properly for congenital heart surgery – there is no adequate risk model existing in the world – but we are actively working on that with the Clinical Outcomes Research Unit (CORU) at UCL. Early results look fairly promising and the work is ongoing. In the meantime that is why we display results for over 50 relatively common procedures.
Brian mentions that they have applied for CCAD data, which is correct. The audit commissioning body, HQIP, rejected the application and offered DFI advice on how to reapply successfully for the data. We currently await the revised application. I would personally like to work with DFI in achieving an optimal analysis and would welcome discussions with them. It is really important this is done correctly – since Bristol there have been some false trails which have taken a long time to sort out and they perhaps could have been avoided without compromising the public interest, in my view.
I cannot comment on the wisdom of the choices made by the JCPCT but I understand at least some of the recommendations are being reviewed by the new Health Secretary. It is always going to be difficult, and highly emotive for so many people, to make these difficult choices. I hope this makes some sense. Rgds David Cunningham
Dr A D Cunningham Senior Strategist for National Cardiac Audits 07753 682686 . firstname.lastname@example.org NICOR . Centre for Cardiovascular Prevention and Outcomes . University College London 3rd Floor . 170 Tottenham Court Road . London W1T 7NU
From:Jarman, Brian [mailto:email@example.com] Sent: 31 October 2012 17:43
To: Cunningham, A Cc: Phil Hammond
Subject: RE: Outcomes of heart surgery in children
Dear David, Phil copied me into his email to you. I am pleased to make contact. It has always seemed strange to me that, with my having been on the Bristol Inquiry panel, and Paul Aylin, and our Imperial College unit, having done the calculations for the Bristol Inquiry, we have never been able to get the CCAD data and yet there is no risk adjusted overall model published for PCS units (I don’t mean for individual procedures, where the umber of cases and deaths is not likely to lead to significant results that the public could use for making decisions about the units).
As you will see on my website, I say that our data is risk adjusted but we still use the case mix of procedures that we used for Bristol and they will have changed now. Paul has tried over the years to have access to the CCAD but HQIP has rejected the application and Paul says that they keep coming back to him saying he needs ethical approval, or ECC clearance. As you probably know, we have full clearance to receive HES data from the National Information Governance Board (NIGB), previously its predecessor the Patient Information Advisory Group (PIAG).
I think it could add to the information available if we were allowed to do our normal risk adjustments, which I describe on my website, using the CCAD data with the up-to-date case-mix. We would like to work with you to achieve an optimal analysis. At the Bristol Inquiry the parents of the children who died asked us why they weren’t told that the Bristol PCS, under 1 year, open heart surgery mortality was 29% (over 3 years). They could have driven an hour or two up the motorway and got a third of that mortality. We didn’t have an answer tor them then and, as far as I know, we don’t have an answer now. We concluded then (Bristol report, 2001, Chapter 12, “Concerns” paragraph 6):- “In short, there was no effective national system for monitoring outcomes. This situation was compounded by the assumption by a number of the respective organisations that it was not their responsibility but that of some other body. This meant, in turn, that the absence of, and need for, a national system was not recognised nor acknowledged at the time.” Also: “It would be reassuring to believe that it could not happen again. We cannot give that reassurance. Unless lessons are learned, it certainly could happen again, if not in the area of paediatric cardiac surgery, then in some other area of care.”
Best wishes, Brian.
From: Cunningham, A [mailto:firstname.lastname@example.org]
Sent: 31 October 2012 19:13
To: Jarman, Brian Cc: Phil Hammond Subject:
RE: Outcomes of heart surgery in children
Dear Brian, Paul’s application to use for data has been rumbling on for a while now (but not ‘years’, but that’s by the by). I think HQIP’s point was there seemed little in the application that demonstrated any value to patients. I have no comment about that. Our point was that the application suggested the use of RACHS-1 methodology, which is quite definitely outdated and just doesn’t work. I do not personally see why Paul can’t word his application to keep HQIP happy. I would lose the reference to RACHS-1 and instead suggest that you/he will use or generate your own casemix adjustment. The adjustments you used in your analysis of 2006-12 data were necessarily limited due to you not having access to enough information in HES.
I don’t personally think valve disease is the way to go for risk adjustment in this very young group of patients – but here’s the thing, YOU may not agree with that and you may find that you can build a risk model that does work, which would of course be of value to everyone. I do not know about the ECC’s position – simplest way would be to email Claire Edgeworth (Claire.Edgeworth@nhs.net) – she is the Deputy Approvals Manager and could advise. Certainly we always have to have specific ECC approval when we seek to receive new data even if we have similar approvals already in place. I would refute any suggestion that all-cause, all-procedure mortality is a valid comparator UNTIL we have a properly working casemix adjustment tool.
My mentor Tony Rickards always said if he was going for an op he would look for the centre with the HIGHEST overall mortality and the lowest for his risk group – in other words they weren’t afraid to take on high risk patients. Not everyone would agree, naturally! It will be a condition of releasing data, should you/we convince HQIP that this is in everyone’s interest, that one of our Steering Group (probably me) works with you on the project and that publications and presentations are cleared with HQIP. I know the latter might be seen as restrictive, but no “veto” has ever been applied to any of the research projects we have collaborated with so far. I hope this is helpful, and perhaps signals the begining of a long overdue collaborative effort,
Rgds David Dr A D Cunningham Senior Strategist for National Cardiac Audits 07753 682686 . email@example.com NICOR . Centre for Cardiovascular Prevention and Outcomes . University College London 3rd Floor . 170 Tottenham Court Road . London W1T 7NU
From:Jarman, Brian [mailto:firstname.lastname@example.org] Sent: 31 October 2012 To: Cunningham, A Cc: Phil Hammond
Subject: RE: Outcomes of heart surgery in children
Thanks David, I love their comment “there seemed little in the application that demonstrated any value to patients.” Tell that to the Bristol parents. Such a typical attitude, one that I find appalling, simply appalling.. If you don’t mind I’ll email your comments to Paul. I’d love to be able to work with you on this data with you – I think there’s a chance that we’d produce something that would be of “value to patients. Congratulations to Phil and Private Eye – there’s a chance that they will again have done something valuable for those parents. Best wishes, Brian.
From: James.Ford@grayling.com To: email@example.com
Subject: the Jarman data Date: Fri, 2 Nov 2012 10:49:29 +0000
I suspect you may get a few letters about this week’s column. The Jarman data tells a partial picture (eg complexity of caseload) and is therefore seen as misleading. Can I suggest we set up a briefing (face to face or phone) for you to speak to one of the experts. I would suggest Prof Martin Elliott. If not: Leslie Hamilton, Shak Qureshi or Roger Boyle?
Best wishes, James 020 7025 7523 Managing Director – Public Sector Grayling 29-35 Lexington Street London W1F 9AH 020 7025 7523 www.grayling.com
From: firstname.lastname@example.org To: email@example.com Subject: RE: the Jarman data
Date: Fri, 2 Nov 2012 11:48:16 +0000
Have already had a helpful reply from David Cunningham (attached) which I’ll be using in my next column. Let me know if there’s anything you’d like to add.
I will also point out that Brian Jarman has now sent his data to Bruce Keogh and the DH twice, without any response, and that Paul Aylin applied for the CCAD data to improve the risk adjustment of their analyses but was turned down by HQIP. I find it inexcusable that statistical experts aren’t sharing data to come up with the best combined analysis to guide patients and spot avoidable harm. As David Cunningham says, he’d like to work with Professor Jarman and DFI, and if the Eye column facilitates this collaboration and gets HQIP to approve the data share, I believe it will be a good thing. It seems churlish to complain that there isn’t a robust method of risk adjustment for PCS if you aren’t enabling your best statisticians to collaborate on it.