In research, policy, regulation and clinical practice, there is an audible buzz around healthcare artificial intelligence. A flood of papers in high-impact journals repeats a single message: healthcare AIs – particularly machine learning based systems – can now outperform humans, and their scope and accuracy are ever-increasing. In the future, many argue, clinicians will inevitably be augmented, or sometimes even replaced, by machines; these technological innovations should be expedited, as they will make healthcare more accurate, more objective, cheaper and faster. So why would anyone want to put a handbrake on such remarkable breakthroughs? In this presentation I will suggest that we should be excited about the potential of healthcare AI. However, we urgently need to reflect—on the purpose, implementation, and ethical, legal and social implications of healthcare AI—and put in place effective boundaries to protect patients, prevent injustice, and preserve trust in healthcare.