Compliant, and Still Wrong: The Human Side of Risk Decisions

Hello and welcome, everybody. I'm Sean Smith. This is Friday in Focus. Those of you who would like to be. elevated to panelists, please raise your hand. We encourage lots, lots, and lots of participation today, because We have, Two of my favorite speakers from IMCS 26, the International MedTech Safety Con… Medical Safety Conference, whatever it was called. In June, they did two sessions there because they had so much to say about this topic that I thought it was worth bringing them back for an encore performance. Of compliant and still wrong. The human side of…

The.

Risk management.

The human side of risk management. So we're here to talk about risk. Bijan, welcome. I would like to give you an opportunity to tell us about next year's IMCS. If you will, come off mute. Tell us what's coming up. Ireland, right?

Yes, IMSC27, International MedTech Safety Conference is what it stands for. It's going to be in Dublin, Ireland, in 2027, June 29th through July 2nd. Our tradition is now we are going back and forth between the US and Europe. We were in Boston in 26, and we had the pleasure of having both Steve and Jean there, who are longtime friends and supporters of the conference, and now we are going to Dublin to be more accessible to our European friends. And it's gonna be amazing. It keeps getting better every year. I'm so excited about it.

That's great. And while we're waiting for… everyone to join. I just want to talk about what's coming up for a moment. Let's see, I'm not going to share my screen. I am just going to.

Okay.

Go to my notes, which…

I'm just gonna put all the events in the chat, so you just go ahead. I have it. So don't worry about it.

So I'm just gonna read Nope, wrong button. Hmm.

I'm also gonna send the PDF, the deck, so you guys can all download it, there's links in it, so you have it.

Beautiful. Thank you, Francesco. Those of you who don't know Francesco, he is, the co-pilot of this event. Let's see, so next Friday, we have active medical devices, R&D. Uruguay is closer than you may think. So, those of you who know where Uruguay is on the map and can pinpoint it, my hat's off to you. We're gonna learn a lot more about Uruguay next week, and why a U.S. or European company might want to explore working with an R&D team in Uruguay. Not something that comes to mind right off the bat, but, you know, I think it's gonna be a really interesting session.

I've met the team, they're, really, really good. And we're talking about implantables and wearable. devices at that point. So then the next session is Demystifying the FDA, Five Steps to the U.S. Market. With Michelle, and that's going to be on October 14th. Followed by Clinical Project Management, the Hidden Complexity of Medical Device and IVD Studies on the 22nd, with Daniela Carrere and Elena Arcangeli and Matteo Mosso. Daniela and Mateo are from Donawa Life Science, and they are going to be presenting

An in-depth look at project management for… Clinical trials. It should be interesting, and… Then we have more stuff coming up, but I don't have it offhand. What did I want to tell you? There was one thing, oh right, and then I'm going to be headed to Europe. mid-October to the end of October. capping that off with, Team PRRC. One thing I'd like everybody to do while we're online is, just… I'm gonna drop this link to Chris Kelly's Latest post. So we did a video of last week's call, 3 minutes, so that you can…

Catch up on… Let's see, copy, link, and then go to… Chat… everyone… I want everyone to go and drop a like. On Chris's video, you can watch it later. And that's what you missed last week. So, with that in mind, we're gonna go ahead and get started. Guys, did we agree that I was sharing the slides? I think we did.

Yes. We gave you a bit of control over us.

Just a tiny bit.

Yeah.

I mean, whose show is this, anyway?

right?

Sean? I sometimes get called Sean, so maybe it's a bit mine.

Okay, so… Slideshow… sorry, not prepared today. Lot of webinars this week. So where do we start? Standards versus reality? Go.

Yeah, so we weren't going to rehash the presentation that we did at the IMSC in 26 in Boston. But we did think it's smart to symbolize the things we were talking about during that presentation. So we took the effort of not wasting any human power, but just a bit of AI. And this one symbolizes an aspect where we stated that standards assume everything works systematically. On the left side, we have a lovely clockwork. Clock. But anybody who's run risk management knows that in reality, part of the system at least looks like the right side of this image. It's tied together and you don't really know why certain things look the way they do.

And this is important because anybody. Stephane Trying to learn risk management starts with the standards. They start with text articles, and until they actually are brave enough to start doing it, they don't realize that you do need to tie it together as a human. This is not something that functions only by virtue of mechanistic aspects in the text.

Yeah.

the.

But I don't know that I agree that, people figure out the right side pretty quickly. I think they just… Sling to… Especially since a lot of us are, you know, technical types, we love a good formula, right? Anything that is gonna just automate something for us, and we try to simplify everything as much as possible. And then I think it takes a long time, and that's kind of the gist of what we're talking about today. It takes a while before the light bulb goes on and go. How many times do I have to bang my head on that before I realize that's… that's not gonna move? But I think the problem is, we tend to just keep sticking to. I've got this standard. I got 14971. I got it. If I just follow that right, I just comply with what it says. Then, magically, I'm going to get good risk management out of it. And then, you know, those of us who have been at this for a long time, and I got tired of banging our heads go. It doesn't look like what's on the left.

There's all these nooks and crannies, and I'm sorry, it's hard. And I often tell my students, I said, you know what? It's supposed to be hard to get a medical device on the market because of what they do. So not a surprise that all the supporting standards and tools are not that simple, right? And boy, we wish they were, but That's just simply not going to be reality, right? It takes… a broad team to get this stuff right, and you're gonna have to challenge yourselves. And just because you did it before, it's not going to be the same result. There's nothing stable about any of this is kind of the the gist of it.

And that's why that human side remains important and how quickly somebody realizes it, I think also depends on. The environment in which they first get to know of risk management. If you're lucky enough to start out learning about risk management, while you are talking to people who've already done it. They will get you up to speed quite quickly on assumptions versus reality. We deal a lot with startups on my end specifically, and They often have tried to get to know risk management from those techs and they take longer until they get somebody who allows them to believe that, no, it's not. Strange that you're not getting this.

And you need the examples and you need somebody to run you. Through some actual risk management before you believe that, yeah, the table is not just a table and a lot goes on in the background.

Yeah, but that depends. You talk right. Do you find the right person to learn anecdotally from right? Are you in the right tribe to get that knowledge? I do. John, I see you got your hand up. Yeah.

So I have a question for both of you. Who are you?

Why should you listen to us? Why should you care? Trust us.

is.

well. Sorry, yeah, it's a good point. I'll go first. It's great, I see a lot of names I know, so that's wonderful. But I'm Steve Gompertz, I'm with QRX Partners, we're a small consultancy in Minneapolis, Minnesota. Focusing particularly on early stage MedTech companies. I have been in the MedTech industry now 35 years, and that was after seven years in the computer industry as a software developer. So a lot of trips around the sun, and a lot of chance to, create opinions, and learn from banging my head against things, whether, intentionally or accidentally. So, but, you can see my, my kind of tongue-in-cheek title here, The Quality Heretic. I'm fond of…

challenging common notions, you know, things that people have been saying for years, this is the how you do it, because, like everybody, that's how I started out learning, as we were just kind of touching on. And then I started realizing, maybe I'm not listening to the right people, you know, because when I would do my own research, I go. That isn't what I was told. Actually, what I was told was actually wrong. So, anyway, that's me.

Thank you.

So, I'm the other side of the conversation, and based on what we did in the IMSC, I am the European side, where Steve is in the US. I also provide the European perspective. I started out and stayed active only in Europe now 14 years ago in Medtech. Due to a lot of coincidences that are too detailed to discuss. But I started out at a startup, luckily with people who came from Medtronic, so there was a lot of veterancy and knowledge in there. But at the startup, there was a resource-constrained environment. And you get inventive, and you get to learn the details a lot when you don't have overhead or any outsourcing capacity to do it to.

After 4 years in there, I went to a larger company that was active internationally. Dutch Ophthalmic Research Center. I always summarize it as the company that cannot say no to suggestions from its client base. So they did everything from single-use disposable travel instruments to reusable active devices to substance-based devices, which is a different way to get to know everything, even though you have the support of a larger company. as I am prone to do halfway through, I got bored, and I did a stint as a technical assessor for the Accreditation Council of the Netherlands. So that's the Dutch Accreditation Council, where I had

the good fortune to not only be subject to audits from certification bodies like BSI and DECRA, but also, get to sit on the other side and assess them on how they interpreted and applied the ISO 13485 for the accredited certification audits under ISO 17021-1. So, I have that double perspective of knowing all of the background details, so the back of the clock, and knowing what needs to be checked from the front of the clock, double over. And that's also what I found interesting, because where I am at now, at MedAce, I came back to my home region to help out startups again, because it has become immensely hard

To understand this if you're just starting out, I was lucky and I realized I was lucky and I want to share that luck.

I'd be interested to know if there are, anybody who's working with startups or any startups on the call, just raise your hand or, you know, put a, hey, me, in the, in the chat, because, you know, it's funny, you know, so I've been publishing MedTech Leading Voice for a number of years now. And, we are… Nearly, nearly, nearly all of my clients say, oh, we work with early stage or small or small to medium start startups, and they're the hardest people in the world for me to reach. You know, like, I mean, getting to the startups, if you're not offering the money, right? It feels like, you know, you're just…

And, you know, we have all these, you know, experts on the call who are like, oh, well, if only they'd start their quality work earlier, if only they'd speak to the R&D people earlier, if only they'd speak to the clinical people earlier, if only they'd, you know. engage with human factors or usability earlier. If only they would, you know, and it's… and every single person is, like, earlier, earlier, earlier, and so then we try to get the message out to the startups, and they're like, do you have money for us? You know, like. I don't know, is you know, you guys work with startups. Talk to talk to us about working with startups, talk to us about you know how how you get through to them risk.

You know the value, you know the importance of understanding risk to startups. I'd love to hear that.

I've had these discussions with several of them, and while Risk is prominent, it's not the way you reach them. The way you reach them is progress. And they've stumped their heads on. Tens of hour-long introduction webinars, and they've spent time on day-long events where the same thing got said, or it stayed so superficial that they didn't gain any time advantage, and that's why they stopped going there. I had one discussion with a fellow who Disagrees with me in a lovely manner a lot of the time, which is very educational. And he stated that depending on the time an event, takes. the startup decides whether the engineer level goes, or the founder or executive level goes. If it's too long, you're not gonna reach any executive level, because they say, I have better ways to spend my time, I'll send an engineer.

So there's a lot of nuances there, but it's not based on… they are not concerned about risk, it's more about Learning about risk levels out quickly if you look at the typical resources.

I was gonna link it to, Talking with startup, you know, entrepreneurs is, to me, is very much like. you know, talking to executives at any company, and I come back to, I think it was Duran, right, in his book, talked about the need for quality professionals to be bilingual. In that you always have to talk in terms of money. You have to do that when you're talking with your executive management in a larger company, but with the startups, as you alluded to, Sean, they have little money, and all they want is they need more, and what they have, they gotta be really careful about spending. And so anything, whether it's risk, or cybersecurity, or reimbursement, and we, yeah, we were constantly telling them, you're already too late to start thinking about this, you need to get moving. And the easiest way to explain that to them is, you've done some interesting work.

If you don't put some controls around this, or do the right analyses on it, you're gonna do it again. Right? And they think, what did I already spend, right? I already borrowed $50,000 from my relatives, and you're telling me I might have just wasted that? And going, well, yes. Right. Because yeah, the age old, if it isn't documented, it didn't happen, kind of approach, right? So I think if you tie it to, you know, you're we want to help you protect your investment and money you've already spent. Then that's why we need to start talking about doing this In a more controlled, formal fashion, right? All of the aspects of getting, you know, to market.

But risk is certainly, you know. Eventually you can help them understand. Right? That's what FDA cares about, or any regular you're going to talk about is right. Is your device safe and effective? That comes down to risk. So that's got to be foremost.

It has got to be foremost, but many startup founders don't start out with that. It's one of the standard assumption versus reality things again. The standard assumes everybody starts out with the use specification and drafting soft layouts, but usually by the time they get to this, they already have had some physical prototypes or proof of concepts going on. And they're stuck, and they haven't even gotten in touch with which materials are you allowed to bring onto the market and use on a human. It's. Frustrating at times, and you do need to be early, but what they are lacking of is not people talking at them, but the people showing them examples.

One of the founders, we worked with, stated that he worked on the… I think it was an open source ventilator. project during the times of COVID, and that he pitied that that didn't get to a level where it could serve as a general example. To startup founders on everything that comes around the corner when you try to bring something like that to the market. Because a lot of what they do have access to is webinars, some free materials, the standards, but the examples of this is how it actually looks like when it's operational, that's missing. And that's what they flock to once you offer it.

That's a very interesting point, and I, you know, one of the things that I've been sort of circling for a few years now is. The value of human intelligence over artificial intelligence. And have been trying to elevate that within this forum and give people the opportunity to actually say something useful. And, you know, not just sort of hand-waving, you know, or here's what the standard says, I'm gonna read the slides at you. So I want to thank both of you for being willing to have a candid conversation about risk today, and sort of the implications of it.

And so, you know, I would like to ask if any of the audience members have any questions or comments for the speakers before we move on. There's Someone just wrote, there is no money for very little money until basic proof of concept has been shown. Yeah. I would agree. Maria, you have your hand raised? You're muted. No. You're not muted. Why can't we hear you? Can't hear you.

I love Chris's question.

Checking activities around Risk.

Maria E. Donawa, MD: Now?

Yes, Nick.

Well.

Maria E. Donawa, MD: good.

Kind of goes back to what I was talking about, right? We love, right, a good formula and a checklist is wonderful, isn't it? How many checklist consultants do we have out there on the net who will sell you some sort of checklist that'll solve all your problems? They drive me nuts because it's not a checklist, you know, activity. None of it in quality system as a whole regulatory, it's sort of good as a final. Did I remember to do everything? But that doesn't mean you automatically get the right results. And You know, I'm all about education, you know, and we talked about, you know, entrepreneurs, right? They tend to have this very narrow focus and narrow understanding.

And I think it's just, you know, I hate to say it this way, it's like raising a child. this repetition that has to occur. You need to put it in the simplest possible terms, and you know, and then tie it to business success again, right? It's all about the money. So speak bilingually. We love as quality professionals to have all these wonderful terms, and like I said, difficult concepts, but at the end of the day, the people we're talking to are gonna have to influence care about dollars, you know, and I'm fond of telling people, and basically everything we do in quality can be tied to money. There is a way to do that. So, so that's the challenge for us. But…

That in the box checking, though, Steve, that the box is being checked is not the final answer. A name, a signature goes with that, and that's a license people must know for other people to call you up and explain why that box was checked.

Oh, why we have to talk about why.

Totally wrong. It's the name that signed the thing that gets to stand up. Receive the question and give the answer. And checking the box simply to pass a gate isn't what the box checking does. You're signing to take a position that signature of yours is a tiny you standing on that page saying, I will speak up if there are questions. And that's both for the auditor side as the approver side.

Thank you, Maria.

Maria E. Donawa, MD: Okay, hopefully you can hear me now. Yep. Okay. Yeah, I think I'm gonna tag on to what Steve was saying about education. I think that many of us, or most of us, who deal with startups, there's a lot of education involved. Maria E. Donawa, MD: And risk is such a basic, important requirement. Maria E. Donawa, MD: It's not only necessary for just, you know, risk aspects for the device, which are obvious, but in Europe, it's a really important aspect of clinical evaluation. Maria E. Donawa, MD: I mean, the biggest section in some CERs, including ours, can be the safety section, where you talk about risk management. So what happens is you see

Maria E. Donawa, MD: Risk analysis done in a certain way. Sometimes you see procedures that are not Maria E. Donawa, MD: Correct. It doesn't mean that you need to be dogmatic about it, and that they need to do exactly everything the way that you yourself Maria E. Donawa, MD: does it so how you would do it. So there's a judgment involved. But you really need to make sure that they're not making big mistakes in terms of risk, because it's going to affect everything. So I'm sure that many of us, or most of us, have gone through that. You're working with them. You're explaining, you're educating. You cannot let it go by.

Maria E. Donawa, MD: There may be some we've seen where there has been a tendency to just say, well, whatever. Maria E. Donawa, MD: And companies have not had the input and help. Maria E. Donawa, MD: That they need. But it's… it's… Maria E. Donawa, MD: as far as risk is concerned, it just… also, I don't know if I said this, and I'm repeating myself, but rigidity is not helpful. So, if, you know, you can… you can comply with risk management and have.

-Oh.

Did we lose her?

Don't know, she froze up. Diane, I wanted to hear the rest of that, because I love where she was going with this, about the rigidity thing. I talk about this a lot in various places, that… We get still too focused on the word control, quality control, right? And old school quality professionals, it's about control. And I teach in a master's degree program, and I was just telling my students the other night that it seems counterintuitive, but a better quality system is a flexible quality system, right? And a lot of old-school quality people would just shudder and go, no, no, no, no, you can't give them options. You can't have flexibility in the quality system.

And I think maybe where Maria was going is, well, you know what? The universe is not rigid. Entropy is real, and you have to be able to adapt. quickly, and a prescribed, a heavily prescribed, whether it's a quality system or a risk management approach, or a template or a checklist. It works against that, right? Because now it only gives you the one path to follow. And you go, oh, we got time constraints, we have money constraints. we'll make it work, right? Versus we allowed for the possibility that we might have to take some turns here along the path, and how will we make the right decisions there? Which, again, of course, is a risk-based approach. So, very, you know, tightly tied concept with risk management, so…

So I don't know, I'm trying to assume where Maria was gonna go, I, I, but she caught my attention when she said the word rigid, that's for sure.

I'll go off what you and Maria said, Steven. I'm also going to pull in what Jill stated. Because money and time as pressures are present, they're present due to investors a lot when it concerns startups. And… I pity the fact that many investors are very unaware of the complexities and the iterative nature that you need in medical device development, but also the fact that there is no such thing as a minimum viable product. And then you get to earn a little bit of money. Well, there are tiny ways, if you plan ahead, to get money, but they're not in the medical device space. You would need to look for a function of your device that exists outside of the medical device space, and that's not so much the case for medical devices as it is for IBD.

Because some diagnostic tools. function on chemistry, and that chemistry can also exist outside of the human body. So we've had some startups who have success there in finding a CAAS stream, but usually it's still not enough to cover everything. And I wish investors would have an avenue to get to learn about what it takes and what to evaluate in a startup to see actual advantages.

Okay, so Alan Marinelli had a comment that he wanted to make. Alan, can you hear us?

Whoops. Yes, hello, could anyone hear me?

Yes, we can hear you.

Good, good. I guess you can't see me. You know, this platform sometimes. I'm more of a Microsoft Teams guy. That's what I've been using most of the time. So I basically have about 35 years experience. Most of it is in the pharmaceutical industry vaccine. I have some years of experience, a good, let's say, 7 or 8 in the medical device. What I've noticed is that a lot of startups in the medical device A lot of people are always in a hurry, they don't understand the theory. If you don't understand the theory, then how could you make it you know, basically apply it out there in the real world. So, there are some people that they had experience with 45 years experience, allegedly in charge of gauge R&R studies, they could do this, they could walk on the moon, they could do all kinds of stuff.

And then I had no experience. I never even heard of gauge RNR studies. But I really, I get up to speed very quickly once I put my head into anything, whether I've done it or didn't do it. And somehow the protocols that the other fellow had designed, because as they say, sometimes all talk and no action doesn't mean anything. I'm more of a guy who's a little bit more conservative. I say, I've never done it, give me a chance, and you'll be… and you'll be happy. So, the client that actually Michel Lott, you know, had actually worked for, at Numerex. I designed gauge iron studies from the ground up because their templates were not very good.

So, so I actually had to learn what Gage INR is. And I was able to be up to speed very quickly, design protocols, design templates, and do the execution, also train people. So, what's the lesson learned and all that? not to pat myself on the back, but what Steve was saying, everything that Steve was saying, and everything that the other person was saying from the Netherlands, it's a combination of that. People have to understand the theory and the academics, as well as the hands-on, and then do the execution the QA people, a lot of QA people don't have the hands-on experience of understanding design. You know, they say they do, but there's a difference between doing it and actually understanding. So I think somebody who's a very good QA will have an overall synopsis

hands-on experience, as well as understanding what the regulations are, because the regulations are only A guidance to basically follow the law. Okay.

Requirements. Standards are the minimum, not the totality.

I don't mean to cut you off, but we're only on slide two. And,

I wanted to point out Maria's back, so I don't know if we want to let her…

And Maria's back.

Maria E. Donawa, MD: No, I apologize. Isn't technology wonderful?

isn't.

Maria E. Donawa, MD: No, I'm fine. I mean, I think I made the point. Thank you very much, Steve.

Actually, I tried to finish your point based on what I thought you were going to say, so you'll. the recording and see if I got it right.

I.

I think it's like you said the word rigid, and that set me off, so I…

The point that we're all making is compliant and still wrong. I mean, it's actually the name… it's the name of the event. You can check all the boxes, you can, you know, be on paper all correct, and still have a product that fails in the market, or, you know, falls short.

Maria E. Donawa, MD: Yeah.

the.

Maria E. Donawa, MD: Talking about when it was wrong on paper. Maria E. Donawa, MD: So you're, you know, starting at a big disadvantage.

Yeah, that's true. Sean, I also…

I just wanted to point out, in the Q&A, Carrie brought up something, and I don't know if we wanted to go down this path, because we're… We're talking about risk management, but I did allude to risk-based, you know, approach, which is related but different, and she brought up, she was curious whether we could have some discussions around that. I don't know if we want to expand the conversation in that direction or not. Technically, it is your show.

It's up to you, you know, you're the guy.

The slides we had, they were conversation starters, and as long as we stay on the fact that compliance doesn't mean you're actually doing risk management, I'm fine with any direction.

Okay, well, hit it, Carrie.

So it caught, again, it caught my attention because actually just yesterday my business partner Mark Swanson and I did a 3 hour workshop for, the ASQ medical device division on risk-based approach. Actually it was kind of fun. We, we, we based it on lessons learned from the movie The Godfather. Okay. Obviously these things are two very connected, right? Risk based approach is about your quality system and considering risk based decision making. Risk management, I always talk it as risk management is one of those possible approaches that you typically would go. what approach should I use when I'm designing and developing my device? Oh, that's where I should do formal risk management, which is more the basis for what John and I were talking about at the Risk Management Conference.

But of course, you can't separate those two too much, right? And so, Carrie, yeah, I think if you had some specific questions, you asked about establishing and maintaining guidelines and policies. I'm curious if you have a specific issue there you wanted to talk about. I don't know if she's got the ability to just talk.

section.

Other…

And that issue is the FDA just goes straight in and goes right for it now. So it's just a comment and.

Yeah, and I'm sorry, Carrie Britton, this was at Carrie Blandford.

Oh.

You asked the question, but Carrie, I'd love to hear from you, by the way, Skoll.

Hi.

Love your shirt. Thank you. Skol Vikings. Skol.

That's fine.

But yeah, if Carrie Blandford's still on, it'd be great to see if she had something more specific she wanted us to look at. So.

Got a good one coming in from Kris there. That's a classic question. When are you done? It's always a bit…

So…

Dubious. Because when people start out, they think that the checklist and having covered it means you're done. But there is an immense depth to risk management, and you can always go back to the Big Bang as a cause. So you're never truly done, and I think it's more about. Realizing what you can explain and what you cannot explain about your device, and whether you've explained it to a degree that you feel nothing will surprise you anymore, or at least that you will not be able to Deal with the surprise. Because as long as you leave a certain level of uncertainty over, but you can deal with that level.

That's fine, I think. Because we don't live in a universe where, in the factory, you can determine exactly what happens with your device at any time in its lifecycle. And that means that coverage, and this is where top management comes in, and their policy and laying down what their risk appetite is, in essence. Is about, are you happy to take the risks both for your company but you're also your personal liability in cases of Mismanagement for what you send out into the world, and that doesn't feel nice. Because. we would prefer that nobody ever releases a device which hurts anybody. But I also believe nobody truly intends to do that, except for the tiny percentage of psychopaths.

And that's the core of when are you done, when you feel you can defend what you did. And in the beginning, you know so little, you don't think you can defend it. And then you get a bit of tuning.

Yeah, and the key is to remember, right, risk management is intended to be a device life cycle approach. Right? As to your point, John, it's not, oh, here's where we're done. You and I get asked this and I go, you're never done. But right, you you're certainly using it as a you know, a prerequisite to do certain things. Like, how do I know I've done enough? So that I can now submit, right, to a regulatory body or regulatory authority. And I think it comes down, it's prioritization, right? And that's why we might, in 14971, we just use two characteristics, severity and the probability of harm. And at some point, right, you get your your use RPNs, you get your RPNs, or you get your risk levels to a point where I've addressed all the major risks.

as much as I can, as far as possible, right, for the European approach, But understanding that I'm not done, I'm going to continue to observe, right? I'm going to look, like, to your point, John, right, your internal environment is a little easier to understand and maybe get to actual root causes externally. You're never quite sure what you're going to be surprised by, but understand it's because that's why you're not done. You're going to continue to monitor in the post-market, you're going to continue to watch the feedback. One, test your original estimates of severity and probability, and 2 look for new ones that you, you know the unknown unknowns, or the the reasonably foreseeable misuse that wasn't reasonably foreseeable yet, and just know that you have that ability to cycle back. But you have to get to a point of

have I addressed the most significant risks, you know, to, you know, what's the current state of the art out there, so that a regulatory body would go, yeah, you've at least matched today's known Intelligence on this, or something like that.

It's that, but I think it's also in the beginning, to harken back to theory and practice need to be combined. Until you've absorbed enough theory to know that you're encountering the same theory again and again. You're uncertain what people are gonna hit you with. But after you've read. 4 separate regions regulations, you realize, well, safe and effective is core. And it's usually that. And after you've read the 14971, and you've added the 24971, and then you've attended a few webinars or courses, you start realizing that finding more knowledge isn't helpful, because they keep telling you the same thing, and that's when you need to do practice. And after you've done enough of the execution, and you think nobody can use anything from the text they've shown me to hit what I've actually done, then you feel done.

And that's why some things like, data security or biological risks feel iffy because their definition is also iffy. We've are very quick to understand something mechanically going wrong, but something biologically going wrong is more complex, and devices that deal with that thus also are longer in that zone of uncertainty of, am I done? And that's where you need to consult experts who've been in it longer who can tell you, no. You're either done, and you're covered for anything they can hit you with, or… No, you're not done, there's still some pieces you don't know, because there is more than 20 standards in the 10993 series.

So I have a question. So when we talk about medical devices, we're talking about everything from a wooden tongue depressor to a large machine that goes inside of hospitals in a system that is very complex and runs on all sorts of software and all sorts of things like that. And so it's difficult to talk about risk in the abstract. Thinking about, Class 3, and implantable devices in particular, you know, where the risk levels are a lot higher, there's such thing called the SSCP. summary of safety and clinical performance. And I'm curious to hear you guys' take on the connection between clinical performance and safety and risk.

as it goes into the SSCP, you know? Because I think that is an interesting summary of residual risk. In… in the… in the CER, And the other thing that I would say is that a lot of the discussion, you did mention the post-market previously, that the reality of medical devices is really seen once they're… once they've achieved the… the clearance or CE marking, and are… and are out in the field. And in… You know, how… how risk people are actively monitoring what's actually happening with

with patience, you know, with performance, and so forth. So, those are two completely, wildly different questions. I apologize.

And they're so narrow.

Yeah, and they're so narrow, so take your pick. But, you know, one was narrower than the other.

Yeah, that's a lot to think about there. But again, I was going back to what I was saying before, thinking about going back to the idea of this is a life cycle approach, right? And there's. E. you kind of start with, and I think John was touching on this as well, you know, you start with a lot of estimation, right, and data gathering, until you think you've addressed things enough, and then when you talk about clinical, right, that comes in a lot of different flavors and phases, right? And there's kind of the clinical evaluation you do to shore up your estimates. where you're sort of taking a a sample of the world of your market right? Your target population.

and then seeing if you can confirm your estimates. But like I said, it's a lifecycle approach, and then you get to a point where you've created enough confidence by the regulators and yourself that, yes, I think we can go out to the market, but you're not done, because in the post-market, whether it's additional formal clinical investigations, or just simply keeping your eye on your device in the market, right? And making sure you continue to watch the trends. The clinical part… evolves and becomes more of the real world for you, right? And I think that's just the natural life cycle of all this, is you start from the theoretical, you then…

You know, basic scientific method, right? And then you prove your hypotheses, and then you continue to monitor over time and see whether or not your experiments were right. Do you need new experiments? Do I need new approaches? Do I need to think about the data in the wrong way? Or did I get everything wrong in the first place and I got to start over? But again, that's why, again, it's not a checklist. Here's where we're done with mismanagement. Or here's where risk management occurs in the process. It doesn't work like that.

Have you ever seen an example where, the quality or risk people have to say, hey, we got a real big problem here in the market?

Yeah, I mean, that happens quite a bit. You know, you make certain assumptions, you know, like, component failure rates or something like that, right? Where you've… you've designed a component, you've made some estimates based on experimentation of durability and reliability, and then that feeds into the risk management, right, probably of occurrence of a failure there that'll lead to harm. And then, right, you watch Service reports or something like that, and you start going, oh, our field techs are replacing this particular component at twice the rate we expected them to fail. Right? That just begs the question of, one, hopefully, you have designed

risk management to fit into your quality system, that that feeds back to something. It's not just a bunch of service reports that somebody goes, oh, that's interesting, that it actually triggers a need to go back and go, where did we cover that in our risk analyses, and how did we get to the original estimates, and now we have real-world data. So I've seen that happen. And that kind of raises questions of okay. So that one component now fails at a greater rate with the potential for harm than we estimated. What does that mean for the device overall? And so you get into… this is a good way to segue into the risk-based decision-making. So what do I do about that? And I can remember when I was at Medtronic, we had that issue pop up. It was just… we were able to tie it back to the supplier issue, there was a component failing at a greater rate.

but the rate at which that component was failing was lower than the overall rate we expected the device to fail right? So meaning. that component was at a less failure rate than all the other possibilities, right? So what do we do? Do we go, oh, well, the device is going to fail for some other reason than that component, so we don't need to take a whole lot of action on that component. right now, risk-based decision making which maybe feeds into your development of ethical culture. There's a tie into the new version of Iso 9,000 and 9,001. But you know, then, yeah, you find out what your company's really about.

At the time, Medtronic did the right thing, and they said we could have easily hidden behind the math, but that component did not meet our expectations. We take action. Right? we're not going to hide behind. Oh, the device will fail for some other reason at a much greater rate. Ignore that. But this is again back to the original concept here of this is not formulaic. Right? This is people making decisions and multiple functions getting involved, right? That wasn't just an engineering decision to do that, because they probably could have done the math and gone. oh, on a statistical basis, that's not really a concern, right? But then you have the clinical people come in and go, yeah, but it is gonna cause harm, potentially, right? If it occurs, it does cause harm, that kind of thing, right? So, again, this is, you know, when John and I did this originally, we talked a lot about the way you solve some of these problems.

plus.

I'm sorry, what was that?

And These things feel natural to us when we're veterans to it, but when you're Starting out, whether as a company or just as a person in an ongoing company. when somebody tells you, analyze the field service reports for pillars, they don't tell you what they intend to achieve with that requirement. And in the same sense, the question that started this off, so the summary of safety and clinical performance, we are required to do it by the regulation. But what the regulation intends to achieve by having us doing that isn't stated. And that makes it very hard for people, either starting out as company or as persons in ongoing companies, to actually, when they do that thing, hit the target.

How do you know you've hit the mark if they only tell you what to do, but not how to evaluate what you've done? Well, what you get is, this thing is in that report, okay, but it being in the report, does it achieve what you want to achieve? And that's what I think is a difficulty, because we've been adding on requirements and requirements and requirements for loads of companies and loads of people without telling us, well, we evaluated this, there was a shortcoming, and that's what we're trying to hit. And I think that's a pity that a lot of training and education that's going on doesn't address, because a lot of the time we discuss on what you need to do, but not what you're trying to do it for. It's the what-for that we had the discussion on during the conference. Bingo. I was gonna remind you of that. Well, you only know you're done when you know what you're doing it for.

Yeah.

I was going to remind you, Jean, that you had talked about that at the conference presentation. Right? We don't do risk management, again, like a checklist, for the sake of doing risk management. There is an objective to be achieved here, right? Begin with the end in mind, as Stephen Covey would have said in the 7 Habits. And that's always the key is that like anything in quality systems and regulatory, it's never because you're just trying to check it off. There is a… there is a why behind every line of ISO 1345 and ISO 14971. And smart quality professionals understand the why. Anybody could read it and then be able to repeat back to you the what and the how.

That doesn't make for good consulting, it doesn't make for good professionalism in any discipline. It's those who understand why are we doing this and set objectives and go, remember, this is the end point, not getting the work done.

That's the thing, but it's also the who, because the risk management standard says the risk management team needs to be competent. The QMS also does, but they never stayed. When do you know you need to lay down a competence requirement? If Carrie asks in the Q&A.

Yeah, I was gonna say, perfect segue.

The ability for post-market confirmation. She suggests risk engineers or other staff, and then it's like, okay, but by calling it risk, it seems as if it should be the responsibility of a single entity within an organization, and it's a shared responsibility, and that's the issue we have when we discuss like this. We have quality management systems, so we have quality engineer. We have a risk management system, so we have a risk engineer. And all of these standards are very agnostic as to who needs to do it, and there's only a few regulations which are specific, like in pharma, you've got quality unit, or we've got management representative, but then what they need to do is very…

high-level and all-encompassing. There's very rarely an answer for who. And again, you only get a feeling for when it's been done effectively by a person, or whether a Function is staffed well enough and competently enough. If you have an idea whether they're hitting the target or not, and when we don't state what the target is, companies cannot evaluate whether they're doing it well until they get hit by a 483 or by an audit. And. if you have an ongoing cash stream that is survivable as a company when you're a startup, that's frightening. The idea that you might get a lock on your factory, or that you are going to be so busy with remediation that you're not going to be able to hit investor targets.

Hmm.

Yeah, and just to wrap up, because, yeah, you pointed to Carrie's question, and I said it was a good segue, because she was asking about, and I, how you get this done, but she was asking about who leads this, you know, and I think we've made the point of who participates has got to be broad. In terms of who leads it, I tend to think of… The common logic would be to your point, John, right? Oh, we have somebody who's got a title Risk Engineer, so they should obviously lead the risk activities, right? But I think it's like doing design reviews or risk management reviews and all this stuff. You want to find somebody who could be more independent, right? Who can stand up there and guide the conversation. Again, avoid a checklist approach.

And… say things like, hey, the emperor has no clothes. That's challenging. Now, in my mind, that should be quality's role, but again, it depends on where you've positioned quality in the organization. and I get it. This could be expensive, or maybe that's you bring in an outside expert right? Somebody who has absolutely no stake and no ties to you who can just ask you all the really hard questions and risk management should be about hard questions to do it right. But somebody else. Mahesh has got a question there right now about how do you navigate the cost of this. But you mentioned cost of quality, and I think that's what it comes back to, again, right? Turn everything into a money measurement, and then go, well, if we don't do it, right, the potential cost of poor quality is this.

The cost of good quality is this. Hopefully, you know, you're avoiding a bigger cost of poor quality by spending something on cost of good quality. right benefit risk. Same concept that we play in risk management. If the benefits outweigh the risk, it's money well spent.

Excellent.

I think you've addressed Mahesh. Charlotte typed something in the chat as well. Yeah, so at MedAce, we have a GMP site to our clean rooms, and we have got a medical device site to our clean rooms, so I get to deal with the GMP side of healthcare quite a lot. I always call them monks and medical device people priests, because they write a lot, and we speak a lot. But the issue is neither of those actually get things done in the world. We like to talk a lot. But risk management truly actually only matters when it comes to the risk control measures that are. Well, implemented, executed, and evaluated.

All of the rest of risk management leads to those decisions and whether those are effective. So, when you have people who work on the paper side of it, I sometimes also get… Questions from my startups, like, how do I know I get the right risk person? Well, you need to evaluate whether they've got an interest in how stuff went wrong, whether they're going to actually look at something or not. And. That stays with me over everything that we did. The human side in risk management is ever present. But it's rarely written down or discussed when you look at the way standards or regulations are written.

That brings me to a question that I have about AI. So, I was at RAPS Convergence recently, and probably 60% of the vendors that were there were presenting AI compliance tools, AI tools for this, AI tools for that, and we're seeing a lot of new entrants into the market there that are saying they're gonna You know, streamline your compliance, get you 90% of the way there, human in the loop, you know, human judgment, indispensable, but we can do all this with AI. you know, and, I… you know, listening to this call. And knowing this audience very well, and working with people with decades of experience know that the value of doing the work

Is what teaches us how to do the work. And if we hand over 90% of the work. To automation, how are we going to teach? the people coming up What the real purpose, value, and goal of the work is. Awful. I'll leave it there.

not. You're not going to teach them and even if You try to apply AI to solve this, you're still short-circuiting what actually needs to happen for learning within human brains. So in that sense, I think that we are currently going through a boom of AI and I hope it doesn't last too long. It's a tool, as any other tool, but it's not a tool that's going to do the work for us. If you go to a garage and you get a jack that you don't need to jack up to lift the car, it's very nice, but just lifting the car. Isn't fixing the car. So, until we start realizing that humans still need to take that responsibility and be the ones in court, you get the old IBM slide, the computer cannot take accountability, so it shouldn't be the one to make a decision.

Until we realize that remains in play, or we make AI liable and able to pay out damages from the accounts of a company. Could go that way, though I don't think it ever will. I think we should be very careful about endorsing those kinds of companies and those kinds of activities.

Yeah, it comes back to Maslow's quote about, right, if the only tool you have is a hammer, every job starts to look like a nail. And right now, that's what we're doing with AI. It appears to be able to do anything and everything, right? And so that's how we're trying to do it. And I go, Yeah, I can absolutely use a hammer to get a screw to go in. But I'm not really gonna get the right result from that. And I think that maybe, you know, thinking about what Jean was trying to say is, at some point, this will sort of level set, and I think we'll eventually figure out what is the tool really for. You know, yeah, there are things we could make it do, but ultimately, it's for hammering in nails, not for screws.

And is it great at bringing together a whole bunch of information and summing that really quickly for me and showing me the… what's in there? Yep. Is it great at actually telling me how to think? Probably not, right? And the idea of, you know, Sean, where you were kind of going is how do you sort of give it the ethics and the human… consideration in there. somebody pointed out, because I use ChatGPT a lot, I've been using it for 3 years, and it's… I treat it like an assistant, and I said it's sort of scary what it knows about me, and that's kind of the point, is that It's reflecting me, right? Which I have to remember sometimes means it's not necessarily telling me what others might be thinking.

It's understanding what I want to hear, and that becomes dangerous as well. I think there is an important role for AI in a lot of these tools. In our practice, we have a regulatory intelligence tool that uses AI, but again, it's… be smart and just go. We don't automatically assume the answers it gives us is correct. You have to have a conversation with it and challenge it. And the same thing you would with any person you walk up to at work or, you know, in your career and, you know, ask an opinion on, you're probably gonna ask some additional questions, and then you wanna maybe check their credentials and see, do I want to take everything they said at face value, or I will do some…

grabbing.

What I'm gonna add to it on my own? Go ahead, John. Yeah.

We're in the last bit of the presentation. I still want to address Carrie Blandford's statement there. What is competence and risk management?

Oh, yeah.

what you were saying. the FDA focused the MDSAP update on that aspect. It says, read and understood is not good enough, but read and understood always was the same as that signature I mentioned earlier. Do you dare to stand up for your understanding? And what we haven't been doing, what we haven't been taking the time, and this is the efficiency disease which existed even before AI. We haven't taken the time to have people defend what they did. And. It's not up to short quizzes or the way you evaluate, it's People who are competent. Are comfortable going into an arena and defending what they did or using what they did.

I would prefer a person who made the FMEA and who made the risk management documentation going into an audit with confidence, not because they had a certificate, but because they know how to explain it and why it reduces harm to a level that they think the user and the buyer and any survivor Of a mishap we'll be happy with. Because things happen, stuff can go wrong. One of the things that goes wrong during surgeries a lot is that people die due to circumstances, due to combinations of things. And it's not every time that this happens that family will say, you could have prevented this. Sometimes they go, like.

we took a risk, we took the gamble, because there were upsides to it, and there are downsides. But that's different from somebody who… Mismanaged the surgery or who did something stupid or indecent. And to me, competence and risk management is the most human of things, but it starts with If you were bereft of all of your tools and you could not change anything you did in the past. Would you still feel comfortable? Standing there, explaining what you did. So competent confidence, to be honest, but it's rightful confidence. It's not fake confidence.

That's a great summary. Martin, you had a final comment? You're muted.

Yes, on the discussion about AI. AI is not cognitive. And if you use governed AI, you can put your standards and your regulations in, and if you set it all up, you'll get great documents. But what you won't get… Two words. Best practice. And the whole… Basis of successful submissions is best practice. The whole principle of Efficacy, safety, and security. is best practice. That's why AI won't take our jobs away until it becomes cognitive.

Well, let's hope it doesn't take our jobs away, because we need jobs. Yeah. Thank you, Martin. Thank you, everyone, for your time and attention today. I would especially like to thank Steve and John for. Coming to talk to us again. Are you guys going to be in Ireland for the next conference?

That's my plan, that's if they choose me to speak, you know.

very.

So…

Okay, well then I look forward to seeing you there. Thank you everyone for your time today. And we look forward to seeing you again next week for Uruguay. Uruguay. Please come. Thank you.

Thanks, Rob. on Sean.

Bye-bye.

Having me on, Trump. See you.