Spotlight on Enhancing Risk Management Throughout the Product Lifecycle

Yeah, great, thanks, John. Okay, everyone, yeah, thanks for joining our topic today, as Sean mentioned, is Spotlight on enhancing risk management through the product lifecycle, and we'll just maybe just do some quick introductions. Yeah, my name's Dr. Andrew Gibson, I'm a Senior Managing Consultant at Accra Team. If you don't know Accra Team, we are a regulatory quality and clinical consultancy based in Germany, offices in Germany and U.S. And I'm based in Germany, my background's in biotechnology, and I manage our European team of consultants, covering those topics. We've been around since 2021, and I've been around with Accra Team since founding.

And we've grown to a team of about 20, supporting many manufacturers in different jurisdictions. So our main focus is European regulation, but we We'll also do all the global regulations, including U.S. and Canada and a lot of other markets. So I think we have an all-Canadian crew today, so we have Ari and Greg on the call as well, so I'll maybe next give it to Ari to introduce himself.

Thanks, Andrew. Hi, everyone. My name is Ari. I'm the Chief Clinical Officer at Omniscient Neurotechnology, where we're focusing on developing neurosurgical software for helping clinicians perform surgeries better, more safely, avoiding areas that could cause surgical deficits. I'm also the CEO and co-founder of OneComply, which is focused on developing generative AI solutions and workflows for regulatory compliance and regulatory professionals. Excited to be here and talking about this. while Andrew is looking for the unmute button, I think, Greg, you're probably gonna.

There we go, Craig. Yeah, thank you.

Yeah, yeah, hi everybody. My name is Dr. Greg Griffin. I'm a technical specialist and clinical evaluation specialist at BSI, so for those who are unfamiliar, BSI plays a similar role to the FDA in the US, except for the European and UK jurisdictions. So I'm a reviewer here. Before this, I worked at Abbott Neuromodulation, developing spinal cord stimulators and deep brain stimulators, and now, I'm on the Regulatory reviewing side.

Thanks, Harry. Thanks, Greg. And, have the mute button now. Great, so we'll get into it now, and just quick disclaimer before we start the… getting into the background. This presentation is intended for educational purposes and does not replace the legal and legislation standards guidance documents. This… this is… our interpretation of the regulations, and, we should not be made liable for, you know, differing opinions, and basically this is the… our… view of the regulation as of today, but make sure to keep up to date and to check the actual legislation and standards and guidances. We know regulatory… from regulatory, documents are always changing, so be sure to check the,

current documents. So, for today's agenda, I'll be giving a background on risk management and a primer for our panel discussion. This will be about 15 minutes before we get into a poll. And our panel discussion and the Q&A session that Sean mentioned at the very end for about 10 minutes. And, And… our focus today is not only from a European perspective, but I'll discuss also FDA. But, you know, having a 15-minute background for this topic, there's a lot to cover, and we'll be at high levels, so feel free to ask me any specific questions in the chat. So, from a background, MDR, we know there's requirements for risk management directly from MDR, but, and we know the standard 14971, but

It's important to consider not only the standard or MDR, but, you know, they have to be considered together, as well as any regional legislation that's applicable to your organization. But from an MDR perspective, we have to consider the harmonized version, so ensure in your documentation that you're considering ENISO 14971. That is the state-of-the-art risk management standard. And, although not, you know, specifically required, this is, of course, an expectation, of notified bodies, and just the state of the art for risk management throughout the organization, and we'll see as globally. So, ISO 14971-2019 is the international standard.

And this is recognized by FDA in the consensus standards list. And same way, FDA does not mandate ISO 14971, but if you do not do that, you have to show an equivalent process, which Yeah, I would just say it's easier to use 14971 as a basis rather than to justify why an equivalent process would be sufficient. So anyway, from our… clients and companies that, we've worked with at Accra Team, of course, 14971 is, always used. And it's the most effective way to meet the FDA's risk management requirements and streamline your regulatory submissions. So, by streamlining, I mean, you know, reducing the questions from FDA or during your notified body assessment.

So, when we're talking, we'll be looking not only to 14971 from a device specific, like your technical documentation and specific devices, but you should also consider it from your quality management system perspective. From 14971 specifically, you know, this is… not covering all the expectations of 14971, but at a high level, we need to consider the risk analysis of the various risks applicable to your device, and perform an evaluation of those risks, look at the risk controls, are the mitigations effective, and Are the residual risks acceptable when weighed against the benefits of the device and your benefit-risk analysis? This should be clearly documented and, you know, provide a clear justification for why those residual risks are still acceptable.

And the evaluation, of course, where usually we're seeing, hazard analysis sort of documents, and it just results in a report where, you're, you're summarizing, the outcomes of this risk analysis, and Becomes part of the life cycle of your device, where information is constantly being… Systematically and being fed into the risk analysis process, and you're looking at your occurrence rates, your severities that you've assessed for your device, and make sure that everything is in alignment with what you're seeing in the field. So this is an iterative and continuous process that is constantly Developing for your device.

So, when we're talking about the product lifecycle, I'm sure you've seen, you know, terms like this, but, you know, startups are looking at concepts, concept, of a device and the design and development activity. This is really the focus of their risk management activities and the risks that they're looking at and into prototyping. But when you get into launch, manufacturing, commercialization, of course, those are different sorts of risks that you're looking at. So, as you can see, it's important to consider what is your life cycle of your device to understand what risks you need to be considering.

In your risk management pile. Going back to more MDR-specific expectations, where we're looking mostly at risk management is in Article 10, which is your general obligations of manufacturers. Annex 2 and 3 are the technical documentation. Annex 1 is your GSPRs that are the general safety and performance requirements for your device, and … and, you know, are the requirements that are showing conformity with the expectations of the EMDR. All of these are tied in, definitely, with risk management. from Article 10, this… covers your obligations and, is entirely interconnected with risk management. We can see, design and manufacturing, risk management itself, your technical documentation, clinical evaluation.

These are all feeding into each other, and evaluating, the risks for your device. Post-market surveillance, is… Gathering risks from post-market setting, and feeding back into your risk management file, and being continually updated in your clinical evaluation. And, as I mentioned, when… when you're tying together all these… these requirements, you need to consider what roles are applicable to your company. Do you sterilize medical devices, or are you a software producing company, or are you only designing the device and it's being manufactured by another entity? This gets very complicated and needs to be considered by your risk management team.

And, this helps to determine what needs to be considered, what processes you need to have, to manage your risk management activities. So both your considerations from a process Like, a quality management system process, and, the device-specific risks. The life cycle of the device we mentioned, and that's both of the company and the device, and what are the applicable legislation? So, do you need to consider FDA requirements? UMDR, others, Health Canada, TGA, you need to consider all the different legislations that are applicable and envisioned for your sale of your device, especially if you're a startup.

To future proof. Your, processes and… And documentation so that you don't have to keep, you know, adding on to it or adapting it. Which… which is okay as well, but if you future… if you plan and know what markets you're planning on entering, it makes it easier if you're considering them during the first drafting of these documents. So, you should ask yourself the question, do you have records and processes covering your obligations? And I mean, this might even be less obvious to you than you think, you know, from Swiss perspective, or from UKCA, These will have different reporting requirements and are collecting,

data risks for your processes and should also be considered, as well as specific documentation requirements, like the PSUR for Canada, is, can largely use the requirements of the EMDR, but you should also be reporting on Canadian, specific numbers, like how many incidents and your rates for Canada. So this is why it's important to consider what markets you're in and really understand what requirements regionally apply to you from a risk management perspective. So, maybe keep in mind, this is really high level now, but we're thinking from a PMS and technical documentation, or DHF, and risk management, how these tie together and play together and make sure they feed and

Talk to each other, From our technical documentation, we have the device description, which has their intended purpose of the device and claims, what risks are presented by what you're intending the device for, what is the purpose, what populations you're going into, and how you're communicating that in the information to be supplied by the manufacturer. So on the label itself, there might be symbols and warnings in the IFU. These should tie back to your risk management file, and we're seeing a lot of issues during review about traceability from what you're communicating to the user, what you're intending the product for, back to your documentation, how that's being verified and validated.

and how that's being insured from a post-markets perspective. So that's really… you start seeing the overall picture, and … The narrative that needs to come together, and you need to be able to demonstrate to the regulator. And it's really that entire, Process that feeds into itself, and you're making sure that the risks are acceptable in the end. … And that's where one key part of the benefit… the technical documentation is the benefit, risk analysis and the risk management documentation, of course, has all that information and is continually updated with that. I mentioned the GSPRs, definitely in GISPR 1 to 8, we're seeing a lot of risk management, and then

after, GSPR 10 and up, these are more device-specific, GSPRs, that will be kind of device per device, but, GSPR1-8 should be considered for every device, and a lot of them, and highlighted here, they all tie into risk management from one perspective or another, and you can directly see risk is mentioned in several of the abbreviated titles. So… … We see a strong influence, and… Here are a lot of the pitfalls that we see from during review, that I've mentioned. We have the traceability of risks. Maybe you have a risk noted in PMS that, you know, you're reporting or that you've identified that doesn't have a clear trace to your risk management, and you're not able to demonstrate that the hazard analysis has adequately identified and analyzed that risk.

And, this, this trace is not apparent, and when that comes up during review, say during your clinical review of your clinical evaluation. this cannot be demonstrated. So you need to make sure that risks identified, either from PMS, your clinical evaluation in literature. If you deem those applicable to your device, that they're being, not only identified and analyzed in your risk management file, but also being fed into from PMS, and you can show that Some sort of review is happening at a regular frequency. that could be quarterly. I think annually could also be acceptable. I mean, I'm not a notified body, but from my experience, I think annual

is a bit too far for me, but quarterly review of complaints against your risk management file is something that I've used to do it in past organizations, and this was a good cadence, and you don't lose track of occurrences or trends. It should be at a more frequent interval than annual. Or not at all. So, and again, if the risk management file is not updated or interconnected, or you can be able to trace risks through the various documents, this generally leads to problems. And now we're speaking a lot about device-specific evidence, but also you should have processes supporting these things that I'm mentioning, so this is where the quality management system ties in.

So, from a post-market surveillance, perspective, from EU, because that has the more robust PMS requirements. You must, establish a post-market surveillance system. and set that up by your quality management system, and base that on a PMS plan, which you must have. If you have, CMARC under EMDR. There's requirements to ensure that that's, updated as part of your technical documentation that we can see here, and this interface as well, and we have you know, we can do Dave's presentation on risk management training, but we can see a lot of interconnections, of course, between 14971 as a harmonized standard and EMDR, so it's important to check for these interconnections.

Other major interconnections I've, mentioned in passing, the CER, SSCP, PSUR, your PMS report for Class 1 devices, ensuring your incidents and trends flow in to these processes, and … and any field actions. So, from a risk management perspective, we often recommend, as a consultant, that you consider ISO TR 20416, and this provides a lot of good information. Some manufacturers, know about this, some don't, so if you… if you haven't looked into the standard, it's a good source of information. And the outputs are… I've listed here on the slide, but you can use it both from a pre-market perspective.

and the post-market perspective, and can be used for, you know, your regulatory intelligence processes, for example. But make sure to check out the standard if you're not aware of it. the proactive regulatory intelligence that I mentioned is another good best practice, that I've often practiced in organizations that I've worked at, that we have at a defined frequency that you're systematically identifying inputs to both regulatory, and you can adapt this for, obtaining risk management-related data from various sources. So… so I have some examples here, but basically, if you make a process around this regulatory intelligence and can gather this data on the defined frequency.

defined in regular frequency, then you should be fine from an organization to make sure that you're not missing standards updates and key risks that might come from, you know, high-profile Recalls that are happening on similar devices, and questions why you haven't considered that for your device. Great, so yeah, that was 15 minutes that I mentioned as a background, so now we'll run into the participant poll.

That's me. As soon as I figure out how to do this, as usual. There we go. Are you seeing the poll?

Not yet.

Oh, you may not be able to see it, because you're a….

It came up for me, Sean.

Okay. At any rate, so while this is running, I would like to remind everybody, if you would like a certificate of attendance for today's call, please put your name as you would like it to appear on the certificate. and your email address in the chat file, and we will try to get it to you as quickly as possible. We have to make these by hand, so please be patient. And, let's see… and I would also like to remind you that, to attend the Friday calls, the Friday in Focus calls, Aria's gonna be speaking in 2 weeks, well, a week and a half. And, talking about AI for…

Regulatory affairs, and showing us some neat stuff that he's been working on. So, that's a live call, and it's interactive. I encourage you to join at medgroup.biz slash calls. You'll see all the calls listed there. Thanks, back to you, Andrew.

Thanks. So, while you're doing the poll, we'll just ask the same questions to our panel. So, I'll start off with Greg. Maybe, can you share what you think is one of the most important things to consider when talking about risk management in the med tech field?

Yeah, yeah, sure. You covered documentation, so I'll reiterate that, Andrew. You know, I've seen, I think, several examples of a lot of smart people within a lot of knowledge doing a good job of risk management, in practice, but then the documentation can be lacking, you know, tracing between the PMS system and requirements, things like that that you were talking about. And then one other… I know you asked for just one, but one other thing that I want to stress to the audience as well is that there is a difference between a harm and a hazardous situation, and one thing I can see a lot, too, is that manufacturers consider

Their device, and how it's used clinically, and then… you know, if the device malfunctions, gives some incorrect piece of information to the doctor. That's a hazardous situation, but it's common to kind of stop there and not go ahead to what… what's the actual harm… potential harm to the patient? How severe is… is that potential harm? So that's an important Thing that manufacturers consider, not just the hazardous situation, but what's the potential actual clinical harm outcome from that.

I think that's a great point, because I actually should have mentioned that as well, but the definitions, because if you have your concepts incorrectly defined from the beginning, then… then you're set up for failure with the whole risk analysis, because if you don't have your harms correctly identified, then the risk analysis is fundamentally incorrect from the beginning. So, yeah, that's… Can totally relate to that. And Ari, from your perspective, what is the most… one of the more… most important things to consider when talking about risk management in the med tech field?

So, I'll, … I'll take this in a different direction, maybe. … I'll say that… the time when you do it, and especially the time when you start thinking through risk management and putting things on paper, to Greg's point. But even if you… even if the documentation is not perfect, and you're not filling out a, you know, wonderful-looking risk management worksheet or whatever it is, but you're… you're thinking through the risk profile of your device, the benefit-risk profile, etc. I think it's critical, and the more that you can introduce that into your organization as part of a regular process, I think the more successful your risk management program is going to be. So…

In my mind, whether you're a startup or you're a large organization, start as early as possible with as many resources and methods as possible. So, if you haven't done a fault tree analysis, try it. If you haven't looked through literature and… in-depth for both your device and similar devices, try it, right? So cast a really wide net so that you can try and understand what's worked and what hasn't worked in the past. And the example that I'll give you here is … Everybody on this call, I assume, reads books, right? Whether we enjoy reading books or we don't enjoy reading books, it's sort of besides the point, but we read books because they represent somebody's accumulated knowledge and experience over time, right?

And by reading books, we try and shortcut that accumulation, so that we don't have to spend 30 years, trying to learn whatever it is, the topic that we're reading about. We're trying to glean some information that allows us to shortcut that accumulation of knowledge. And risk management is the same way, right? So, there's a lot of information that's out there already, both on your device, even if it's an incredibly novel device, I'm sure you can find a simile or a, … A parallel enough example or function, that you can draw on. Once you have those resources, really take the time to think about how it applies to your product, how it might inform workflows, how it works across various other topics, cybersecurity, human factors, etc, and really avoid what I would call compliance theater, right? The act of checking a checkbox.

For having a worksheet or a report, instead of actually going through the, the thinking process behind the, the exercise.

Great advice. Thanks. No. Now we'll go into our panel here. So the first question is a bit of an opening question, and it's setting the stage, in this question for both, Ari and Greg. When you hear designing for safety, what does that mean in your day-to-day work? How does it go beyond just meeting the MDR requirements? And we'll start with Greg.

Yeah, sure, so, … you know, the MDR has requirements that risks are reduced as far as possible, and a bunch of specific risks listed, and it's pretty common for there to be some standard that even is device-specific and governs specific risks. So you could, in theory, design a device and test it according to the standard and say, oh, the test's passed, therefore our risks are well controlled. But I think designing for safety requires an understanding of how your device works, how it would influence those risks, and the failure modes that may lead to a risk, and in order, you know, understanding

the… sort of the root cause of why a test would fail, even if it did pass, you know, where the failure point would be. That's really important to understand the mechanisms, how the device works, why it's passing the test, why it might fail the test in some different scenario if you're, you know, maybe in vivo it would fail if it wouldn't in vitro. So, it's not just Testing to a spec and making sure the test passed, but it's really important to understand why the test passed or why it would fail, that type of thing. And that's, I think, paramount for making a really safe device, like a deep understanding, not just, a list of specifications that you can pass with a certain test.

Right. Yeah, I think that ties in well with the checkbox, treating it like a checkbox activity, where you're just not thinking about what you're doing from a high level, and yeah, it… yeah, I think that makes total sense, and If you're not completely understanding the whole, device and why you're doing certain testing, then it can't all work together. Exactly.

Yeah, one other thing, if I can add, before we hear from Ari, this… I think, you know, it can… we can really get focused on safety and consider, oh, is this device going to burn a patient, or is it gonna cut a patient, or something? But something that… that's also super important is does the patient feel safe and comfortable using the device, or having the device be used on them? So it's possible that, you know, you can make a device that doesn't strictly speaking, harm them, but if the patient is scared the whole time that they're using it, it's… I wouldn't call that safe or effective. So, you know, patient comfort and just general anxiety and things like that, the actual patient experience is an important

aspect to consider, generally, in terms of safety, too, I think.

I think that's an important focus these days. And… Ari, from your perspective, ….

Maybe I'll… I'll build on that. I think that's a… that's a… I mean, fantastic point. … I think the separation in our mind between, life cycle safety, sort of Some of the post-market activities that you're doing, and launch safety, so the pre-market activity that you're doing. … making sure that you are applying the same amount of vigilance and hunger for information in both of those, I think, is critical. I see a lot of companies focusing on the pre-market component, because, frankly, they need it in order to get to a clearance, or an approval, or get through an audit, or whatever it is. But then once we get to post-market safety, we sort of largely rely on PMS processes, and that's about it.

not thinking through what are some of the other sources that we might have in your organization, that are maybe captured during PMS, maybe not captured as part of PMS, that could inform the risk profile of the device. not only that, but the frequency with which you're doing it, and I know that we all need to justify it to regulators and auditing organizations, but thinking through, really, how often that data source is updated and how that could inform your design decisions, and building in the hooks Into your process that, forces the wrong word, but, Really, really try and prompt somebody to think through, looking at a source and inputting that into your risk management file and making that as smooth as possible.

I think is integral to, Designing for safety.

Great. Thanks for that. So… now we'll be moving more into questions about integrating risk into your… into the QMS lifecycle. So, first with… for Ari, what is the biggest challenge you face in embedding risk management principles from concept through to post-market?

Yeah, I mean, … I alluded to this in my previous answer, in the previous discussion, … Really thinking through what are the sufficient hooks in the various processes, that we have in an organization. And the larger you are, the more complex you are, the more different types of risk management.

That you might be doing.

And how that actually drives updating of the risk management file, I think, is a really big challenge. And Greg mentioned traceability, and really, it comes down to that for sure, right? So, these days, you have a critical task analysis, you could have a cybersecurity risk analysis, you're gonna have an AI risk assessment, you're gonna have PFMEA for service, you have a PFMEA for… your design process, potentially, you have a safety risk analysis, etc, etc, etc, right? So… Has everybody thought through the feeder mechanisms for all of these processes? So how do you get risks in there? what triggers inclusion in those risk processes, and the interaction between these, right? So when… when is something elevated to a different risk process, and who is responsible to mine that and look at the different sources of risk and make sure that updates occur, I think is a real challenge.

Makes sense. And, Greg, from your perspective, what are the common gaps when you see companies trying to demonstrate this integration that we're talking about into their QMS?

So there's a few. One that I think is a common pitfall is Taking risk control measures for granted, or, … you know, it's common, like Ari said, almost every medical device is an iteration of some previous device, or some similar idea that maybe humans are doing manually or something. So, there's often a lot of background knowledge that designers and people performing the risk assessment can have about a certain device, and so they might You know, inherently know, oh, this risk is conceivable, but it's not really foreseeable because of how the clinical practice has been going on for a long time, and

Or even if they have a specific device, they maybe have a design requirement that was put in place to mitigate a certain risk, but years passed, and the engineer who Design of the old device isn't there anymore, so that… Knowledge can get lost, and sort of taking those things for granted and not… thoroughly documenting them, or, you know, writing down the thought process, like I was talking about, casting that broad net and thinking about what past devices have done, etc. So that can be a big gap of not sufficiently Writing down the thought process and the baseline that you were starting from.

Another big gap that… or common gap that I see is a lack of evidence of verification of effectiveness, and that's a big phrase, you know, but the risk management standard Perhaps the most important part of it, or one very important part of it, is After you've come up with an idea of how to control a risk, it's really important to validate that that idea was effective. You know, it's not generally sufficient to just say, oh, we put a warning message in our software that pops up, or we wrote a requirement that says this thing shall last for at least 5 years, and we tested to confirm that, but It's really important to validate that the risk control measure has

Being effective at really reducing the risk. … And then. The final gap, big gap, is just comparison to the PMS data after the post-market. It should be a relatively straightforward thing, I think, to, like you were saying, Andrew, check every quarter, that the PMS data continues to agree, but it can be a big gap. Just comparing that, I've seen, I think, at least one or two questions in the chat about, oh, if there's nothing new in the PMS, do we have to document that? generally, you do have to document some amount of review. That's what Clause 10 of the Risk Management Standard says. It's not like you have to go rewrite the whole risk management file, but it's important to periodically do that, and at least record the results of the

To the review… of the review to… to demonstrate that it's been undertaken.

Yep. Well, thanks, thanks for that insight. And maybe I'll take one of the gaps you mentioned specifically, so kind of a gaps that are gen… happening because of a knowledge loss, because maybe with a startup, you know, someone's left… well, it could happen with any organization, but where the current team now doesn't understand where, mitigation… how mitigation works, or where some information came from, and I mean, this is a bit of a ticking time bomb, because when this comes up during an audit, or during a review, and the current team doesn't understand and can't explain Why something exists or how it works, then that's a big problem.

From your perspective, Ari and Greg, and I'll start with Ari, is there any, What are some ways of avoiding this knowledge loss? In an organization to ensure that, you know, that this doesn't happen, and teams still understand How decisions were made.

Yeah, so I'll, I'll avoid HR. Processes here in terms of, how do we do onboarding and offboarding and all that sort of good stuff. So focusing on just the risk management component, … I think this is where, … and again, this is a concept that came up previously in the conversation, but the concept of traceability, comes up. So, in my mind, one of the best ways of best ways of guaranteeing this continuity is, number one, you have a big enough team that participates in your risk management activities, so that there's an understanding of what are the discussions that were had. You're documenting those discussions through risk management reviews, etc, so you have an idea of

Who contributed to what, and what was the thinking process, and this is incredibly simple to do these days with, note, automated note-takers. And lastly, once you do have a risk mitigation, and very often I see companies come up with sort of a single line that says, oh, well, we're gonna, you know, do X in our device, and it's going to reduce risk by Y amount. And the question is, well. where… how is that X implemented? Where is it implemented? And sometimes it's traced to the requirements level. To various degrees of success, but really thinking through, number one, where is it traced to from a design input standpoint? Number two.

how does it actually interface, or how does it get incorporated into your design spec, or into your architecture, wherever it is that it lives? You can create that additional hook that sort of explains, How that mitigation affects the overall design of the product.

Right? Makes sense. And Greg, from your perspective, do you have anything to add for reducing those knowledge gaps that can happen?

… Fundamentally, not really. I think Ari did a good job, you know, large team, redundancy in the knowledge in the team, making sure that there's not just one senior engineer that designed the thing and understands it, and everybody just takes their word, as gospel. You know, it's important that, To have a large team of people who understand it, and it's important to question assumptions, too, as, you know, moving forward, making sure that, if there is… if something's being done, somebody's got to be able to explain why, you know? So, not taking assumptions for granted, making sure that there are good explanations for everything that's done that everybody understands.

I know, from our consultancy, we also saw some good, kind of, … we saw some risk reviews that also, kind of, documented and, It seemed to, you know, have some history, but these can add to a lot of additional documents in your quality system, so, … I think there's a good way of doing it and a bad way, so we see a lot of risk reviews, and if they're well documented, and you can understand the decisions from them, they're documented well, but, … Also, you can really document them, so they're also not understandable, so there can be pros and cons of doing that as well.

Yeah, and one thing I just remembered, Andrew, to help others, too, is I've seen some good examples of requirements management, where there's some written specification, but also a little commentary about why that specification's there, where it came from, and stuff, and that commentary is not, I don't think, required necessarily by any standard or regulation, but it can go a super long way to helping a regulator understand, or future people at the company.

Right, yeah, that's a good tip. … So I think we'll jump ahead a little bit. So, going beyond regulatory minimums, Ari, what, what does… proactive risk assessment look like in practice, and how has it helped you stay ahead of compliance issues from taking a more proactive approach?

Yeah, so in practice, For me, anyway, it means incorporating small risk management activities throughout your development, or whether it's pre-market or post-market life cycles. So you start as early as possible, whether you're doing a new product introduction project, a big new redesign, or you're doing change management. having hooks in those processes that, reinforce concepts of risk management, thinking through risks, reviewing what has changed, even if you did a, you know, a PMS cycle. … is there anything that has changed about that particular functionality or that particular feature? Because very often, when we're doing PMS, we're looking at similar devices, but fundamentally, we're looking at the same functionality as you already have implemented.

If you're doing a brand new iteration on a product, whatever it is that you're introducing, you might consider doing, I'll call it a mini PMS, but it doesn't have to follow the same exact rigor. But really thinking through what has changed and how you incorporate that. And then, I think… Equally as important is involving as many different groups as possible in this proactive effort. So I mentioned a few different risk management processes between PFMEA for service, cybersecurity, human factors, etc. All of those fundamentally are risk management activities. And so doing those as early as possible with as broad a team as possible, across the organization, I think, is, is critical for productivity.

It's… And Greg, from your side, what are some good practices that you've… or are there any good practices that you've seen that help… have helped go beyond regulation and the requirements and… but… and have, you know, positively improved safety and performance of the device?

Yeah, I basically said this earlier in terms of what's really important, but testing to failure, rather than just testing to the specification. This isn't as directly applicable to software, I'd say, but for physical devices, there are often many fatigue tests and things that, you know, needs to meet a certain minimum specification, but Once you get there, that's great, you can document that verification, but keep the test going to see how long it fails, or see how long of accelerated aging you require to you know, make the sterile barrier break, or something like that. So, testing to failure, it goes beyond the minimum requirements of the regulation, but I have seen,

certain scenarios where manufacturers have done that preemptively before the market and show that their device is going to last two and a half times longer than it needs to, or I've seen some other unfortunate scenarios where manufacturer tested to the minimum specification and the standard, and their device was, you know, a relatively uncommon device within that category that's covered by the standard, and the standard was a few years old, and those writing it hadn't really thought about this Potentially novel device, so they tested the standard and then, needed to, you know, have a design change and a minor recall to

to improve their device and make the reliability really what it needed to be. So, I think testing to failure, understanding the failure modes and stuff is super valuable at every stage.

Interesting. Great, so we'll jump ahead. So, looking… looking ahead, I think on a lot of people's mind are kind of digital health, AI-enabled devices, so this is a question for both Greg and Ari. So for AI-enabled devices and connected systems becoming more common, how do you see the QMS, or the risk of the patient safety relationship, how do you see that, evolving as the nature of devices are… I mean, or getting additional and popular devices that are AI-enabled or connected digital health … We'll start with you, Greg. How do you see that evolving?

… So, … Beneficially, the design cycle is a lot shorter and faster with AI-enabled software than it is with a physical device. An unfortunate consequence of that is there's less time spent in the pre-market phase in a lot of cases, and perhaps less time spent on the risk management. So, specifically… well, I guess it's kind of general and specific, a lot of these AI devices are claiming to Interpret some image, or maybe interpret some data from a physical device that's monitoring, put things together, and predict a risk of a certain disease for a patient, or try to help a radiographer identify something on an image.

But in general, they're mostly saying. we look at a bunch of data and information, provide some interpretation that might be a piece of advice for a physician or something like that. And this relates to what I was talking about, difference between hazardous situation and harm. I've seen a lot of AI companies say, oh, we're not making a diagnosis, we don't claim To replace a physician were just a useful tool that they might use. And then they sort of downplayed the severity of providing incorrect information or cybersecurity breaches that could compromise patient data and stuff, so… it's important

for companies like that to consider, oh, if the doctor comes to trust this thing over a year or two of using it, they don't spend as much time double-checking the results, and then there's some hallucination in the future. What's the potential severity to the patient of that? regardless of what the manual says, even if it's got all these disclaimers that said, don't trust this, we might be wrong, etc, I think, in reality. People will come to trust them over time, and it's important that we don't lose sight of the possibility of incorrect information's malfunctions in the software, and the severity of that information being presented to the doctor.

Right. Great, and are you… From your perspective about the… do you have, thoughts about that relationship evolving for digital health and AI-enabled devices?

Definitely agree with everything that Greg said. The only thing that I'd add is, … we're moving, or we already live in a world that's very rich in data. All of our devices, or I shouldn't say all, but many of our devices already produce or log something, in the background, especially if there's a software component. How… how visible that is versus how deep it is in the architecture is… is, … varying, but I think… In the future, you're going to see more devices that are elevating that information directly to the manufacturer, potentially. Obviously, there's cybersecurity and other considerations, but that will really allow us to move from this world where, once a year, we sort of look back at all the logs and think through.

what could have went wrong to a world where we're continuously, we're looking at trends and what's happening with our devices, and are there any tweaks or changes or, you know, the big R-word recalls that we need to, perform, as part of the performance data that we're getting?

Right? Yeah, definitely agree. We'll have so much more data, potentially, for certain devices feeding into our PMS and PMCF, potentially, even. So, yeah, might have a lot more visibility than we did before when we knew PMS has so much biases in terms of reporting and under-reporting specifically. So, yeah, it'll be changing with these devices. And so, Sean, I think we're moving into the….

Yeah, that's….

portion now?

Let's move to the Q&A. I'll let you guys look over the Q&A in case there are any questions that jump out at you, but the ones that jumped out at me, were, for example, how do regulators, FDA versus MDR, IVDR differ, and how they expect risk management to be documented and maintained?

… I mean, I'm a regulator, so I feel like I should be answering this, but I'm honestly not that familiar with the FDA expectations. You know, I've worked for American companies and gone through FDA approval before, but I don't… I'm not familiar with the inner workings. As Andrew mentioned, I think with the EUMDR, the requirements and expectations are generally probably more rigorous in Europe than the US now, from what I've heard. So, I can say in Europe, we expect you to follow 14971, that standard that Andrew mentioned, and it's not, strictly speaking, legislatively mandated, but all of the GSPRs do align very well with it, and so…

That's the expectation in EU, is basically that you follow that standard and… and don't just follow it in practice, but document it well, also, so that people like me don't need to ask a million questions to confirm that you did… did the risk management.

So become well-versed in GSPR.

Yeah, yes, that's… that's definitely a good piece of advice.

Great. So here's an interesting question. When people leave an organization, risk knowledge can get lost. How can peer review and clear record keeping help maintain continuity? And anything else Ari might like to add to that little topic there?

… hold on, I'm trying to find that question. How can… Do you have the name of the person who asked it?

It's a… it's a summary of one of the questions. So, really, the question is, you know, when people… when people leave an organization, how do you ensure continuity and risk management?

Yeah, so we chatted about that a little bit, and my understanding was they're asking about peer review in particular, I think that's a fantastic method of ensuring continuity. Of course, you might have, hopefully, support, with some of the software solutions that you're using. If you're using something like an Excel sheet or, you know, Word documents in order to manage all of this, then it necessarily doesn't, plug in, or it doesn't help with that continuity component, unless you make the effort of recording your meetings and all that sort of stuff. But there could be solutions out there, on the market that, might do some of that work for you and help you with, knowledge-based, maintenance.

Let me know if that doesn't answer what they're looking for.

I think it does. I think this is a good answer. Oh my goodness, there's a lot more questions. Let's see, what is this one? Can IFU be considered as a ultimate risk mitigation for a Class 3 implantable medical device? That's a highly specific question.

I don't mind, giving a bit of input on that, Sean. So, I assume by IFU, the question is asking about information for safety, whether that's in the IFU or not. The MDR requires that you consider risk control measures in a certain order of priority. It's the same as 14971, so… First, you gotta figure out if the risk can be eliminated through design in the device. Then there's possibility for protective measures or alarms to alert users, and then the final option is information for safety. So, in theory, information for safety may, in certain cases, be the only risk control measure, depending on the risk, etc, but you do need to be able to

demonstrate that it wasn't practical to reduce the risk through design or through a safe measure. And then, if you do reach that point that information for safety is the primary risk control, it's really important to consider The 62366 standard, which governs usability and human factors testing, that type of thing, and Clause 413 of that says, if information for safety is a risk control, that information shall be subjected to the usability process. So, this relates to what I was talking about, evidence of verification of effectiveness. If you get to the point and say, oh, we can't design this thing out without compromising the benefit of the device, so the only thing we can do is

tell people what to worry about in the IFU. It's really important that that information goes through The usability process, and you can demonstrate that it's effective to the typical user, and they… they see that warning, they understand what it means, they change their practice accordingly, based on the information, that type of thing. So, short answer, yes, in some specific cases, but there's a lot of, there are a lot of Requirements to… that you need to be able to meet to get to that point.

And Greg, as you said, the ultimate one is eliminating the risk completely, which is reflected in the standard rates, and then… And then it's reduced… Without compromising the benefit, that's the important thing, because, you know, you could….

If you don't put a device on the market, all the risks are eliminated, but then there's no benefit, so….

There's that good example where you have a scalpel, but… and then the risk is about the sharpness, and then you remove the blade, and you've eliminated it, but you've….

Risk-free scalpel, but it's also benefit-free, so, it's important.

Gotcha.

Consider that, too.

So, there was an interesting question about benefit-risk analysis guidance. Is there guidance available? What, for example, MDCG guidance, I think 2020, 2022, 6 and 5, 14971. Where… where can people find guidance on… … Risk-benefit analysis, … that… Yeah, that's the question.

Yeah, I don't know from the European standpoint, but I linked one guidance from the FDA, and I saw a comment somebody making around there being other guidances from the, CDRH guidelines, search website, so yeah, I encourage everybody to do that search, but certainly, I've used the, factors to consider regarding benefit-risk medical devices, and it's not a horrible guidance.

Yeah, somebody else wrote in, FDA has 5 guidance documents on benefit-risk with examples that are the only guidances that were published by a regulator that I'm aware of. Search FDA CDRH guidances for benefit-risk, and they should pop up. Thanks, Edwin. So….

Don't forget 24971, so the accompanying guidance 214971 is another good source, but yeah, there's a lot of good Yeah, you'll have to dig around, and those are some good examples.

And take a look at the SSCP as well. Okay. So, we're in the final minute. There were 15 questions, 2 got answered. We'll try to follow up with you, with answers to your questions in the coming week. And I'd like to thank everybody, especially our speakers, for giving up their time and their expertise and sharing it with the rest of us. For free, which we love. And, thank you all. Please, please complete the survey. Give us your feedback, we'd love to know what you thought. Any final words, gentlemen?

I think I saw a question about sharing my LinkedIn profile. I think I imagine it would be in the invite to the, to the meeting, so please take a look there. I'm happy to share my email, too. It's just greg.griffin at bsigroup.com, so simple first name.lastname at bsigroup, if anybody has follow-up questions or, anything else.

Very good. Thank you very much. Please follow all of today's speakers on LinkedIn, and I look forward to seeing you all next week for the Webinar on… … Startup funding, yeah. There we go. That's it. Thank you all.

Okay, thanks, everybody.

Thanks. Bye.

Dude.