Although cardiovascular disease is the leading cause of death globally, an increasing proportion of patients are surviving with heart failure (HF), which is substantially increasing in incidence and prevalence.1,2 In the management of HF the indications for implanted devices have widened, resulting in more patients living with an expanding variety of sensor-enabled implanted devices than any other patient group.
Patients with HF can now also take advantage of the ever-increasing availability and affordability of consumer electronic devices – both wearable and environmental. All these devices generate massive amounts of data, and the connectivity of these devices has created opportunities for pooling data from multiple sensors – so-called interconnectivity – and for artificial intelligence (AI) to provide new diagnostic, triage, risk stratification and disease management insights for the delivery of better, more personalised and cost-effective healthcare.
AI is also bringing important and previously inaccessible insights from our conventional cardiac investigations, which are becoming increasingly accessible outside of the hospital setting.
This article reviews this convergence of AI, sensor technologies and interconnectivity and how this combination is set to change the care of patients with HF. This decade is tasked with the significant challenge of frontline implementation of technology-enabled care, which will first need rigorous clinical trials to validate what we have learned so far.3
Overview of Artificial Intelligence
The established and emerging technologies outlined in this review all share the opportunity to collect low-cost data, passively obtained and at massive scale across populations. The subsequent datasets qualify as ‘big data’, characterised as high volume, high velocity and/or high variety information assets that require new forms of processing to enable enhanced discovery, insight, decision-making and process optimisation.3
The combination of possessing massive amounts of data alongside advances in computing power has marked the resurgence of AI, composed of a set of powerful tools that can analyse big data to confer previously inaccessible insights. AI is a broad term that encompasses machine-based data processing to achieve objectives that typically require human-level cognitive function, such as recognising images (Table 1). Complex datasets can now be mined with potential to identify patterns and novel representations of data beyond direct human interpretation. Through AI, paradigms across all sectors of society are being disrupted. In medicine, the most high-profile research has been across ophthalmology, dermatology, radiology, intensive care and mental health.4–9 Many of these studies focus on algorithm-enhanced risk prediction, diagnosis and treatment selection, but there is also significant enthusiasm for AI liberating clinical staff from tedious administrative tasks in order to spend more time with patients.10 However, despite much hype, most applications for AI in HF and medicine, in general, remain theoretical and have yet to be validated at scale in routine practice.11
Automation is not a new concept in cardiology (attempts at automated ECG interpretation date back to the 1970s).12 However, this last decade has seen significant AI breakthroughs through the use of machine learning (ML) and, more specifically, deep learning (Table 1). ML aims to learn from data in order to correctly answer a question, which is different to conventional computer programming, that is, handcrafting the answer into the system.
The subfield of deep learning is modelled on a conceptual representation of networks between neurons in our brains that are exquisite at soaking up information containing data that help us generate predictions. Deep learning is responsible for nearly all currently tangible daily-life advances of AI, from image interpretation to spoken word recognition. It is a very powerful technique that is very data hungry, requiring a lot of data to work well. The power of deep learning lies in its ability to circumvent the problem of finding meaningful features in the data. Deep learning systems are capable of learning from complex data without much preprocessing (labelling) beyond the essential clean, uniform formatting that ‘denoises’ and normalises a dataset.
Abundance of Implanted Sensors in Heart Failure
Cardiology has long been at the frontline of pioneering and adopting new technology. After the first cardiac pacemaker was implanted into a patient in Sweden in 1958, rapid iteration on the original prototype has made this life-saving technology available to a much larger population.13 Now cardiologists implant more sensor-enabled devices into patients than any other specialists, encompassing 1.4 million pacemakers per year.14 Patients with HF are among those who have benefited most from device-based treatments. ICDs, pacemakers and CRT have revolutionised both mortality and symptom control and comprise standard guideline therapy around the world. Looking forwards, HF patients will continue to have significant representation in the implanted device population, and thus likely stand to become test cases for new sensor technologies (Figure 1).
New Opportunities with Established Implanted Sensors
The addition of further sensors to devices that are already being implanted poses a significant opportunity. Every cardiac pacemaker or defibrillator provides such a platform. However, clinically unutilised data could also be captured from existing pacemaker sensors. As established, physical activity is highly predictive of cardiovascular outcomes. Internal accelerometers, incorporated for the primary purpose of rate-responsive pacing, passively generate low-cost data that can serve as a surrogate of physical activity, highlighting an as-yet untapped clinical and ML opportunity that could inform interventions.15
CRT devices have already demonstrated capacity for additional sensing features, including intrathoracic impedance, which has been clinically available for over a decade. Rising impedance can be used to stratify patients into varying mortality risk and serves as a superior measure by preceding weight gain by 2 weeks, predicting an increased risk of HF hospitalisation sooner.16
The clinical utility of these current and new sensor technologies is manifest when combined with the increasing interconnectivity of implanted devices. These are mostly already capable of transmitting data to healthcare providers, with nearly 70% of patients with CRT devices using a remote monitoring feature.17 However, this is currently limited to, at best, once-daily transmission from a home-based transceiver, which in turn transmits the data through landlines or data networks to the manufacturer’s server and then on to clinical teams for action. Therefore, a rich datastream exists, but is underused, with manufacturers capable of collecting a wealth of clinically meaningful data that could be built into preventative care.
Little of this datastream has been built into clinical pathways and feedback loops that benefit patients, in part because of challenges with false positives (FPs). Specifically, the high FP rate of telemedicine initiatives has significantly hindered their scaling, which could be addressed by an algorithm-based method to refine identification of those most at risk.18
One approach that has navigated the challenge of FPs is testament to the value of using different data sources to inform a patient’s true clinical state. Research by Ahmed et al. in the Triage-HF Plus study showed that among patients flagged as being at high risk for HF decompensation based on CRT-D physiological data (Heart Failure Risk Score, received via CareLink, Medtronic), FPs can be mitigated through the addition of a simple telephone triage questionnaire.19 The achieved sensitivity of 98.6% demonstrates the potential of patient-centric pathways that leverage sensors, interconnectivity and data variety, where further augmentation with AI becomes a natural next step.
Several previous attempts to bring rising impedance alerts into clinical pathways have failed.20,21 These studies highlight several challenges to implementation, including physician and patient adherence to remote and telemonitoring systems, reinforcing the need for human-centric design for any technology-enabled clinical pathway.
New Implanted Sensors for Heart Failure
Several newer implantable sensor technologies deviate from the traditional box-and-wires design, including two for measuring pulmonary artery pressure (PAP) that are directly deployed into the pulmonary artery – one established (CardioMEMs, Abbot) and one emerging (Cordella, Endotronix). Using home transmission of PAP with an implanted pressure sensor, long-term hospital admission rates for New York Heart Association Class III HF showed significant improvements. Rates of admissions to hospital for HF were reduced in the treatment group by 33% (HR 0.67; 95% CI [0.55−0.80]; p<0.0001) compared with the control group.22
Raised left atrial pressure (LAP) is the most specific and earliest sign of impending HF exacerbation, long before clinical symptoms occur. At the vanguard of new implantable sensor technology is a new digital, wireless, battery-less device (V-LAP, Vectorious Medical) capable of transmitting a high-resolution waveform that represents LAP.23 This device is due to undergo its first clinical trial in patients this year.
Remote PAP, LAP and other monitoring carries significant setup and running costs, but this may also be a problem amenable to a ML solution by training an algorithm on the data from previous PAP-monitored patients to ‘learn’ who benefits most, thus enabling targeting the intervention to those most likely to benefit. 24,25
Use of all such devices poses significant technical challenges, particularly optimising CRT-D function, but clinical benefits could be maximised without the need for new hardware by using AI to build models capable of enhancing decision-making around implantation and optimisation.26,27
External Sensors For All
Since Norman J Holter’s achievement in 1949, substantial progress has been made away from the weighty backpack that acquired the first remotely recorded ECG trace. The increasing prevalence of implanted sensor technology is now far outweighed by the ubiquity of consumer-directed wearable sensor technologies, a rapidly growing market set to achieve a net worth of US$34 billion by 2020.28 There has been rapid deployment of powerful smartphones, wearable sensor devices (e.g. smartwatches), and the healthcare Internet of Things (IoT), together providing unprecedented levels sensor feedback.29
Self-monitoring and the transmission of signals of cardiovascular status to healthcare providers is one of the defining strengths of this technology (Figure 1). Initially this was limited to measuring simple parameters, such as step count and heart rate, with subsequent progress extending to most other vital signs. Wearable health-monitoring technologies are now usually accompanied by or integrated into a mobile phone app – technology that itself is now heavily integrated with a wealth of sensors.30 This increased access to and visibility of health and vital signs data is accompanied by a wider shift towards promoting self-management by giving patients online access to their health records.31
Current and New External Sensors for
Heart Failure
There are several potential early warning signs that may predict acute decompensation in HF, but many of these are not clinically detectable. The most basic, that of early morning weight, requires a patient to manually document this (often on paper). This data point has the potential to be exploited for monitoring and timely prevention of decompensation. This is already being done in some instances; Bluetooth-connected scales can upload data to a patient’s electronic health record (EHR) for accurate self-monitoring and healthcare provider oversight, acting as an early warning sign of deterioration that has been integrated into successful care management strategies to reduce HF admissions.32,33 Physical activity measured with implantable devices has already been shown to predict risk of hospitalisation.34 Pedometers have been surpassed by more informative accelerometers, present in most smartphones. They offer a non-invasive opportunity to monitor a patient’s activity level, particularly useful in the HF population where comorbidity, including the risk of falls, is the rule. 35,36
Newer external sensors for haemodynamic measurements have been developed. For example, remote dialectic sensing (ReDS, Sensible Medical Innovations) is able to use electromagnetic signals to give a numerical measurement of the degree of pulmonary congestion, allowing extrapolation of lung fluid concentration that correlates well with CT assessments of lung fluid concentrations.37 ReDS is yet to be validated for benefit against more simple (and cheaper) technologies for predicting deterioration. However, surface electrodes adapted to measure transthoracic impedance, a marker of intrathoracic fluid levels, have been shown to precede worsening HF prior to the usual go-to measure of weight gain, with a sensitivity of 76% versus 23% (p<0.0001).16 The authors of this study make several references to thoracic impedance as an important diagnostic tool. However, it is important to note that the success of this technology has been thwarted by its poor sensitivity.
Peripheral, wearable equipment has seen some of the most drastic price drops in the history of the electronic goods and services sector.38 This increasing affordability also extends to hardware, such as ultrasound equipment, offering the opportunity for wider use not just of echocardiography, but also of lung ultrasonography, which may improve diagnosis of acute HF episodes.39 All of these noninvasive technologies have the potential to complement established monitoring methods and widen the capture of patients becoming sick in hospital and at home, with the promise of reducing rates and duration of hospital admission.
New technologies have emerged at the intersection of implanted and external sensors, such as a pill embedded with a miniature sensor (Proteus Digital) that, when it enters the acidic environment of the stomach, emits a signal to a wearable sensor patch.40 This highlights an opportunity to monitor adherence to the medication regimens at the centre of HF management.
The Commodification of ECGs
In a short space of time, consumer technology has developed to not only be able to measure most vital signs, but several technologies also now exist that enable accurate single-lead ECG traces. The development of portable AI now makes it possible to automate detection of AF using these traces, offering opportunities for early intervention to prevent progression to HF.41
AF is the most common arrhythmia in HF. If AF with a fast ventricular rate can be detected and treated early, this may reduce episodes of decompensated HF. In addition, those with new AF can be identified early and anticoagulation promptly instigated to prevent stroke, which causes significant morbidity and mortality.42 The diagnosis of AF has been made easier by the availability of commercial, AI-powered AF-detecting wearables (AppleWatch 4 and 5) and external sensors (Kardia, AliveCor). The latter has since advanced to six-lead ECG (KardiaMobile 6L), with 12-lead detection in development. These technologies enable patients to have more agency over their own health through prompt, automated feedback on the presence of arrhythmias. This has broad implications for enhanced diagnostic accuracy and rhythm determination – an important area for further study and evaluation.
The opportunity for near continuous ECG monitoring offers particular promise for improving early diagnosis of paroxysmal AF. This aligns with the ambition to limit development or progression of the HF syndrome by leveraging user-friendly, increasingly low-cost technologies to anticipate potential triggers (for 50% of patients with AF and HF, the arrhythmia came first).43,44 The study of the risk of short bursts of AF has mostly been limited to patients with implanted devices; population-wide representation of AF bursts, enabled by cheap wearable sensors, may help to reach clinical consensus for what constitutes a significant burden of AF activity.45
Opportunities for Heart Failure Using
Artificial Intelligence
Outside of implantable devices, a successful approach to external sensor technology will be one that simplifies self-monitoring by means of user-friendly hardware that integrates collecting a variety of actionable health data, allowing AI opportunities to follow naturally. Industry is already pushing ahead with this, with some large companies (General Electric) and smaller start-ups (Current Health) vying for their AI-driven multisensor monitoring devices (patches and armbands) to be adopted to facilitate the liberation of both patients and providers from the burden of actively recording and monitoring vital signs.46,47
Understanding the Heart Failure Population
Achieving the goal of personalised medicine will require a granular understanding of subgroups within a population. Improving on traditional linear models, ML methods can process a diverse dataset, including sensor outputs, to unearth complex, higher-level interactions among a multitude of features to improve discrimination and predictive range with respect to HF outcomes. This can address the significant challenge of heterogeneity in the populations that make up the HF syndrome. Changes in weight, ECG, impedance, PAP and LAP can be viewed with a patient-specific reference range, instead of one-size-fits-all averages.
Already, notable AI successes in HF include a ML model that draws on phenoytypic data, including echocardiogram images, to identify patients with HF with preserved ejection fraction (HFpEF).48,49 AI is not necessary to make the already relatively straightforward diagnosis of HFpEF, but could serve to better segment subpopulations into previously unidentified clusters. This could inform participant selection for clinical trials and, therefore, increase the chance of observing a genuine disease-modifying effect of a treatment for HFpEF, a diagnosis currently lacking in any prognostically beneficial medications. Such an approach would involve unsupervised ML, using unlabelled data, designed to find hidden patterns. This approach could also add new value to clinical trials in HF that have fallen short of expectations; for example, by identifying a subclass of patients who might benefit from specific drug treatments, including spironolactone, enalapril and sildenafil.50–52 Importantly, any AI-driven hypothesis would of course still need to be tested to a high standard using randomised controlled trials.
Artificial Intelligence for Heart Failure Imaging
Studies that, for example, have used cardiac MRI in the past suddenly have renewed value by being able to offer a potentially high-quality, labelled dataset with which to interrogate new AI powered research questions. Echocardiography, the diagnostic stalwart of HF, shows much promise for being enhanced by AI. Data quality will be key, and specifically with image recognition there still remains a need for humans to annotate and label the images that become the training set.
ML algorithms can subsequently assist in the discrimination of physiological versus pathological patterns of hypertrophic remodeling.41 AI’s impact on echocardiography could see a convergence towards a real-time, ML-based system for automated capture and interpretation of echocardiographic images, drastically expanding accessibility, accuracy, consistency (on second scanning, the same operator will change their categorical assessment of left ventricular [LV] function 30% of the time) and affordability.53 AI may also enhance the diagnostic utility of more advanced echocardiographic techniques. Global longitudinal strain (GLS) can serve towards early detection of myocardial changes and prediction of cardiotoxicity in patients receiving cancer therapy, but is a technique that manifests the common challenges of reproducibility (operator dependence), which could be improved by AI’s potential to automate GLS calculation.54,55
Combined with a degree of AI-enabled automated interpretation and ever-cheaper ultrasound technology, the diagnostic power of echocardiography could be made accessible to a much wider pool of patients. More broadly, the advances across all modalities of cardiac imaging have been myriad, continuing to produce rich databases of diverse images and thus highlighting a wealth of opportunities for cardiac imaging to be enhanced by AI.55
New Insights Using Old Investigations
AI is unearthing ways of deriving unanticipated physiological and other insights from established investigations and sensor inputs. For example, it is now possible to predict 1-year mortality from normal-appearing ECGs.56 The ECG is already known to reflect elevated potassium levels in the form of tall T waves; deep learning has taken this to the next level by being able to quantify potassium levels after the model was trained on over 1.5 million ECGs.57 This has highlighted the opportunities for ‘bloodless blood tests’. HF patients taking significant diuretic doses, as well as their clinicians, may welcome the prospect of being able to monitor the electrolytes of otherwise stable patients non-invasively and remotely through the use of ECG-sensing wearable technology.
A further revolutionary application of deep learning can accurately recognise – on what to the human eye looks like a sinus rhythm ECG – patterns that indicate a propensity towards AF, therefore by proxy highlighting a cohort also at risk of developing HF.46 Furthermore, the ECG has traditionally not been considered a good diagnostic test for asymptomatic LV dysfunction (affecting 2–5% of the population), but researchers have now trained a deep learning model using pairings of ECG and echocardiogram images, achieving good performance in the detection of LV dysfunction (sensitivity and specificity 86.3% and 85.7%, respectively) when subsequently predicting this using ECG alone.47,58 A recent study highlights an even more impressive achievement: 100% accuracy in categorising ECGs as healthy or HF, by analysis of a single ECG heartbeat using convolutional neural network models, a form of ML that can visualise to researchers what morphological features are important.59
As ECG and other technologies become increasingly commodified, these noninvasive tools will become more prevalent in community settings. The UK’s National Health Service has highlighted community ECG facilities as a priority addition to standard care.60
Lastly, ECG data (QRS morphology, QRS duration, presence of AF) was included among a set of common clinical variables to build a ML model capable of predicting outcomes for CRT. ML demonstrated better outcome prediction than guidelines (area under the curve 0.70 versus 0.65; p=0.012).27 This could improve shared decision-making and better patient selection for a procedure with inconsistent impacts on clinical outcomes.61
Catalysing AI with Novel Data Sources
It has been reported that 38% of patients will die within the first year of diagnosis of HF.62 Understanding this population on a more granular level will enable tailored disease-modifying therapies that maximise individual patient outcomes. The goal of redefining HF into clinically meaningful homogenous subclasses using AI will be aided by the burgeoning stream of data derived from sensor technologies. These inputs may be combined with other novel data opportunities, including ‘omics’ (spanning but not limited to genomics, metabolomics, proteomics and environmental exposures), along with patient-reported outcome measures and social determinants of health, thus refining how HF is characterised beyond just an echocardiographic- and symptom-based classification and facilitating a more personalised diagnosis than ever before.63
Several countries are advancing towards the mass digitisation of health records. Much like the unanticipated insight of being able to derive age and sex from ECGs using AI, access to a richer digital patient profile through EHRs could deliver AI-based insights into propensity towards developing HF that we currently could not anticipate.64 With this abundance of data, AI can serve as a means to contend with the risk of information overload, which has marked some of the present criticism of EHRs, by processing and highlighting the most salient points and assisting in workflows.10
Catalysing AI with Interconnectivity
To achieve a vision of personalised medicine, the integration of sensor technology and AI will require the addition of a third feature: interconnectivity. This is not just of medical devices, but also everyday objects. This health-impacting IoT is becoming increasingly commonplace in patients’ homes, adding a wealth of new sensors, data and, therefore, opportunities for insight. The next decade will see the mass rollout of 5G internet, setting a new precedent for powerful interconnectivity between different digital technologies. This will draw heavily on cloud computing, which provides data storage and computing power at scale, on-demand without direct active management by the user.65 5G is predicted to enable a move towards real-time health services becoming the norm rather than the exception.66,67 Connectivity of objects used in daily life can create new sensors that can inform how patients with HF are faring; a smart, IoT-enabled environment can infer when patients are not preparing meals, monitor if they are suitably mobile and if they are in safe environmental conditions.65
Challenges with the Adoption of AI and New Technologies
Algorithms continue to prove themselves to be diagnostically more ‘accurate’. However, for many of these ML methods, their ‘black box’ nature makes it difficult to infer any diagnostic reasoning. This lack in interpretability of AI models marks a challenge to adoption, which is understandable when considering the risk of hidden biases in training datasets being learned by models whose output can exhibit discrimination without us realising.68 HF transcends all socioeconomic and cultural divides, thus requiring AI models to be drawn from a dataset reflecting this diversity. Ensuring training datasets are generating among a representative population is therefore essential to minimise biases. Further dealing with this Achilles’ heel of AI requires development of fair, accountable and transparent ML techniques, augmented by improved algorithmic literacy across society.69–71
As an example of a potentially more acceptable ML model, a generative adversarial network (GAN) is capable of generating synthetic data that resemble the real data. GANs are trained to capture the most defining features of the real dataset and, without compromising patients’ identity, can produce new ‘generated’ datasets capable of training a ML model for arrhythmia detection.72
Governments are slowly catching up to the new legislative questions that AI poses. Patient safety, data protection and evidence-based action should be core tenets of law-making in this area, requiring a rigorous approach that at the same time avoids reactive regulation that could stifle innovation.
The likely natural progression to the capability of a real-time monitoring ‘feed’ will – especially for implanted devices with a pacing or defibrillation function – prioritise the agenda for discussing significant safety, privacy and ethical implications of being able to adjust a device’s function remotely (a capability that already exists). This will need balancing against the benefits of early intervention for adverse events, which AI will be able to predict with high accuracy. However, developing such models may be limited by the often proprietary nature of data; data sharing between researchers and device companies, in a way that incorporates informed consent from patients, needs to be prioritised to realise the full potential of AI in HF and beyond.73
New healthcare technologies are often treated with suspicion by clinicians, wary of the risk of an even greater workload with further decisions and actions to be taken. This is just one of several challenges acknowledged in a recent WHO report on transitioning from innovation to implementation of digital technologies.74 Responsive reimbursement models are needed to increase adoption of new healthcare technologies, success of which also depends on a viewpoint that encapsulates tech-agnostic, patient and user-centric design.75
Future Outlook
The trifecta of interconnectivity, diverse sensor technology and AI tools sets a course towards the ultimate goal of cost-effective, clinically beneficial closed-feedback loops. AI models can offer decision support and could be enabled to run autonomously in some instances. It will be a sum of different sensor technologies and their varied data outputs that will be able to realise the full potential of any interconnected AI tool. For example, modification of therapies in HF, such as adjusting diuretic dosing, may be amenable to decision support from a ML model, trained and continually iterating at superhuman levels of accuracy by incorporating datastreams from sensors. Eric Topol, cardiologist, geneticist and digital medicine researcher, outlines how these insights will ultimately converge towards a fully automated, individualised AI-driven virtual health coach.10 Though such solutions are years away, they could not align more with the necessary paradigm shift in HF and medicine in general – away from a model of break-and-fix and towards predict and prevent.
In the more immediate future, machine vision interfaces could revolutionise how proponents of HF are identified. Deep learning has demonstrated that mass screening for AF is possible by algorithmic interpretation of video from a smartphone camera.76 The capability of accurately measuring blood pressure using a smartwatch, that is, without a cuff, has been realised and is destined to become a standard feature of health-related wearables.77
Beyond the scope of this review, the convergence of genomics, digital medicine (encompassing sensors), AI and robotics will enable staff working within an ethical and legal framework to deliver a more holistic approach to personalised healthcare and disease prevention.78
Conclusion
The next decade of advances in HF care will need to confront several challenges: leveraging an exponentially growing repertoire of interconnected internal and external sensors for patient benefit and processing massive, multimodal datasets with new AI tools. The opportunity lies in fostering a greater degree of empowerment for patients and improving the accuracy and efficiency of HF management. Success will depend on a human-centric approach that makes use of new technologies appropriately, without assuming they are always the right solution.