MLV Exchange

Great.

Nice.

Okay. So

Hi, everyone, I'm I'm just trying to figure out how to let all these people in. I'm looking for Chris. I'm just gonna panelists invite. I'll be might like, sorry.

Okay, well, while I'm promoting hundreds of people to panelists, I'm gonna go ahead and start so hello and welcome everyone to Friday and focus. Call little disorganized here today. But fortunately my guests are hyper organized. I would like to go ahead and and introduce Jose and Mohammed from cyber Med, and let them start talking while I deal with promoting everyone. If you don't want to be promoted, just decline thanks.

Great. All right. Well, Sean, thanks for thanks for having us on. Welcome, everybody. I'm hoping to have a good conversation today about cybersecurity and medical devices. I thought I'd start with just a brief history of my own background, the company background, Mohammed. That sort of thing. so I'm the founder and president of Cyber. Med started the company around 8 years ago, and we originally started as an internet of Things Development Company and it was, I was like a solopreneur just myself doing consulting, and one of the earliest clients that I got was high security product. So it wasn't a medical device, but it was a device that required very high

levels of cyber security. They were developing equipment to be used in in, in various facilities that required security, and so they needed somebody to help implement it. And I knew that wasn't my strength. So I went and started looking for somebody, and I met Mohammed, who's on the call here. He's he's 1 of my 2 partners and and we met. We headed off. I could tell he was brilliant both in in software development and cyber security. So I asked them to become a partner. His history is, he was formerly the chief software architect at Motorola's global communication systems, two-way radios. And so he was responsible for global communication systems whose people's lives depended on them. So cybersecurity was top of mind.

He's got a tremendous cybersecurity background. He did the 1st fips, 142 certified in browser crypto module, and the first, st you know, software system to thwart spectre and meltdown, which are 2 major cyber security threats. So just tons of experience in that space. And so all along, we we had this medical device practice. And then we had this security side. And about 7 years ago we decided to really focus on medical devices. But we always retained that that one client where we were always kind of pushing the envelope on cyber security. And then, recently, last 2 years, really started focusing on helping medical device. Manufacturers make sure that their devices are secure, and then make sure that they're compliant with FDA regulations. So that's just a little bit of our history.

Why, why does cybersecurity matter? Why does it matter in medical devices? I was thinking we could do like a brief history in time, you know. Go back to Adam and Eve in the garden, eating the forbidden fruit, you know. Get kicked out. Then you've got Cain killing Abel, and we see that there's this global, eternal battle between good and evil. And if you fast forward, then to the 19 nineties. you've got the Internet being invented, world wide web computers throughout the world being connected together. And now that evil doesn't require physical contact, right? So you've got the good side you've got, you know, Tommy and his basement in Ohio, being able to chat with Pierre in Paris, and that's great. But then you've got Timmy Tom's evil twin brother being able to steal Pier's.

I don't know. Lotus 1, 2, 3 files, or something, or send them a virus. It's the nineties. And so you kind of see that that those bad things that you could do previously requiring, like physical co-location, can now be done across the world, and you've got the advent of this new kind of security, threat and crime that that is related to cybersecurity. We oftentimes call it hacking that kind of thing. So you know, that's the global context. And then, of course, in the 2 thousands and 2,010 s. We start to see new paradigm, which is the connection of devices to the cloud and the connection of devices to each other. So you've got companies like Nest coming out with thermostats that connect to the cloud, you know, ring coming out with cameras. You've got Fitbit with pedometers, and you know we take it for granted now. But it's really not that old like. It's it's not that long ago that we didn't expect physical devices to be connected to the Internet. But now they are.

and medical devices being a little bit of a laggard when it comes to technology took took some time, but before you knew it you had glucose monitors connecting to iphones, and you know, infusion pumps connecting to hospital networks and that sort of thing. So of course, you've got dramatic improvements and benefits that come from that. You know it's great if you can monitor your loved one's glucose levels and get an alarm if you know, levels are out of whack or something. But, as you might imagine, you've now got greater threats around that. So that's that's the context of the world we're living in. And the context of where FDA and and frankly, Congress has

really driven medical device manufacturers to have a greater sense of urgency around cybersecurity. So so far, so good. Everybody can hear me. And stuff I don't wanna talk for okay. Good. Alright great. So why would somebody want to hack a device like, why would you want? Why would a threat actor want to do that sort of thing. The obvious thing, of course, is money, and that comes in a couple of different ways. So one of them is ransomware. So where a hacker can hack into a server, let's say, encrypt all the files and then demand, let's say, bitcoin payment. That's literally a case that I'm aware of happened to a medical device startup. So you know, you can see how threat actors can use cyber security attacks to demand ransom. But you've also got things like data theft where healthcare data is a lot more valuable than even things like credit card and social security numbers.

And then you've got things like nation, state or terrorist attacks. So you can imagine a rogue country targeting critical infrastructure in the Us. And the healthcare system is part of the critical infrastructure to cause harm. And then you could also have people who are attacking a specific person. So an interesting story. And this is actually one of the drivers of cybersecurity regulation, in a way, was back in 2,007 with Dick Cheney had to have an Icd implanted so a defibrillator. So they actually required that the wireless connectivity of that Icd be disabled because of the concern that a terrorist could attack him, you know, remotely

through an attack. So there's a lot of reasons why threat actors might try to cause harm, or at least threaten to cause harm, and in response to that both Congress and the FDA have pushed to have companies just implement better cybersecurity. So from a regulatory standpoint, you have regulations going, or at least guidance documents from FDA dating back to 2014. But I would say that the you know the the pedal hit the metal in 2023. Congress passed the Patch Act which updated the Food and Drug Food drug and cosmetic act, added the section called section 524 B. So sometimes you'll hear that term being used in cybersecurity

which essentially said, if you have a cyber device, and they gave a definition of that. then you have to do certain things. And they were a little bit vague about those things. but they gave FDA Enforcement authority and interpretation latitude to essentially set the boundaries of of what companies needed to do. FDA took that on, and within a few months. In 2023 released the draft guidance around cybersecurity. and then, by September of 2023 release the final guidance with enforcement started in October of 2023. So it's been just under 2 years that FDA has had this guidance out, and that companies have to to abide by.

And it really applies to any device that has software. So they took the definition of cyber devices that's set forth in Section 5, 24 B, and they essentially expanded the scope. So they said, You know pretty much any device that has software and any conceivable means of connecting to the Internet, whether on purpose or not on purpose. So you can imagine. If you have a device and it has no interfaces, it still has to get programmed at the manufacturing facility. Well, the computer programming it could connect to the Internet. So there's a conceivable path for it to connect to the Internet. So essentially, their their view of it is, if your device has software, you have to follow this guidance document.

And they took the the 3 bullet points in 5, 24 B, and expanded it to a little over a dozen documents like 13 or 14 depending on how you count them related to cybersecurity. So that's that's the context. You know, as I mentioned, I run a company called Cyber Med, with Muhammad and another partner. and you know we view our mission. We kind of have 3 pillars of our mission. So the 1st one is that we help medical device manufacturers understand their vulnerabilities and and we help them document those so understand them so that they can fix them right so that they're launching products that are secure and then document them so that they can submit to FDA and get cleared. So that's 1 pillar. The second one is doing the testing. So we do fuzz and penetration testing to help ensure that the device is, in fact, secure. And then the 3rd pillar is community education. So even if companies aren't going to be clients of ours.

you know, we believe it's important for medical devices to be secure. And so we still want to help demystify cybersecurity for medical device manufacturers. And in that vein one of the projects we've been working on for about 6 months is a book. So we we launched a kind of a quick start guide as a starting point to help companies understand at high level. But we wanted to do something more comprehensive. So we've been working on a cybersecurity book. and the approach we wanted to take to it was rather than just writing it and launching it. We wanted to prelaunch it as an electronic book that allows commenting by the community, so that if we miss something, you know, if you think we got something wrong, or if you think something's not clear, or if you're like, Hey.

I don't see a section talking about this, and that's an area that we struggle with. We want to get all that feedback. And over the course of the next month or so we're going to incorporate a lot of that feedback and hopefully launch a better book that's helpful to the community. You know, our goal is not. This isn't really a profit center for us, like when we do go to print, you know, we print to price it at a relatively low cost. and then for anybody who participates during this period of time, they'll get the free version of the electronic version, and if it's something helpful you'll be included in like an acknowledgement section of the book. So right at the front end of it, we want to acknowledge people who contributed. We kind of see this as a community project to really help the industry launch more secure products. So I'm going to pause there, see if there's any preliminary questions.

And then what I'd like to do is show you the book. We've launched it as a web app. So there's there's a little web app that we designed and built, that has, like a way of navigating through the book, and a way of commenting on the book, so that we can get all that feedback and incorporate it. So I'll show you that. But before I do that any pressing questions.

So I just want to interrupt. This is a really really great opportunity to preview the entire book, to search it, to use it online. The app that he's built for this thing is incredible. I logged into it last night. And these guys, they just blow me away. I mean, you know they're putting my marketing company to shame, so I'd have to say that if you want to review it, give your feedback, please do so, you know. Put a put a message in the chat to me, or whoever. If you think that this is a good idea, love collaboration. And I'd also like to say that I need speakers for the 20 second and the 29.th So if you want to speak on the 20, second or the 29, th you should raise your hand and let me know.

or send me a message that says, Speak, please. I can't promise those spots to everybody, but I can promise that we will pick somebody good.

Awesome great. And I did see. There's a question from Dorothy. What about a bluetooth device that does not connect to the Internet? It still applies. I mean, even if you don't have Bluetooth honestly like the guidance. If you look at the scope of the guidance document, it says medical device that has software functions, including firmware or programmable logic. So even if you just had like a an Fpga or what's called a Cpld, it's like the scope is anything that has software. I find it amusing that they even have an example where they say, if all you have is a thermometer, right? Of course we don't expect the same level of rigor than if we're talking about, you know an implanted

Icd. Or something, or an infusion pump. But you still have to follow everything. So it's just, you know. That's that's the position that FDA took. And so that's the world we live in. You know. I'm not saying it's it's how I would do it, or how you might want to do it. But it's it's how it is. But yeah, keep the questions coming, and I'll I'll try to address as many as I can.

The best comment was, is your device entirely made of wood? If no, if if if no, then chances are the connected device regulation applies.

Right? Right? Right? Can you guys see the book? Now, the kind of a website that says cybermen on it? Okay, great. So I'm going to be kind of looking to the side in front, because I've got my larger monitor on my left. But but I'll try to maintain some level of eye contact. All right cool. So this is. This is what your experience will be like. You'll create an account. You'll be brought to this page on the left hand side. I'm going to orient you to the experience of the book, and then we'll get into some of the details of the actual book, and my hope is to really only spend about 30 min on it. and then the rest of the time to just have a conversation. Questions you might have. You know anybody on the call? I know we've got a lot of regulatory advisors, so I'd love to hear about what you're seeing out there and really just sharing our content and our experience for the benefit of of the group.

So all right cool. So on the left hand side, you're gonna have a table of contents. The book is divided into 6 chapters. so the 1st one's an introduction of medical device, cybersecurity. Chapter 2, is the regulatory history and framework. So looking not only at the guidance documents, but also at some of the standards that apply, and and some of the, you know, even non-standards, non guidance documents that exist in the in the community, like from example, from like mitre that are helpful. And then we get into the more technical part of things. So you know, we always talk about. There's like 3 main phases of cybersecurity

in a product. There's pre-market architecture phase where you're designing the system architecture. Then there's the implementation of the system. And then there's post-market. So we break that down into those 3 chapters, 3, 4, and 5. So planning and architecting for security, secure development and then post-market responsibilities. And then, chapter 6, is the Eastar submission artifacts. So you know, we're trying not to be redundant with the other chapters, but saying, Okay, now you're getting close to submission. How do you take all of that content that you've prepared as part of the process, and then

put it into the format that FDA is looking to see. So that's that's the gist of it. Okay, cool. So you know, Chapter one, I won't go too much into Chapter one, because it's essentially a lot of the stuff that I discussed in the in the intro presentation. But looking at Chapter 2, you know, here's what I was what I was saying earlier. So at the at the heart of this are the 2 guidance documents that are still relevant. So there was a 2014 guidance document that was superseded by the 2023 guidance document. And you know, I should probably update, say 2025, because FDA did release an update to the guidance a few weeks ago.

you know, honestly, like nothing really changed. It was really in. The intention was to try to clarify some things, I think in some ways they muddied the water a little bit, but but the the bottom line is that nothing really changed. So you've got that one. And then you've got the 2016 guidance document that is still relevant. And it's more focused on post-market responsibilities. And then, you know, I look at kind of the top of of the the map. Let's say a north star is the Ansi, Amy, SW. 96, which Mohammed and I are both contributors to that and that's the the standard for medical device security. So

prior to that, there were 2 tirs. So tir 57, which is principles of medical device, security, and then principles of medical device, security for post-market, risk management, and SW. 96, partially supersedes Tir 57, but Tir 57 still has valuable things that are not in SW. 96. So they're all good references. and then kind of at the bottom of this as sort of a foundation to all risk analysis and medical devices, is Iso 1, 4, 9, 7, 1, which is the applicability of risk management to medical devices, particularly things related to safety. So even though they are different, they are related. And, in fact, in both the guidance document and in SW. 96.

There's there's a requirement to look at that interrelationship between security risks and safety risks. Okay? And then, beyond that, you've got things like Mitre's playbook for threat modeling and medical devices. You got Mitre's rubric for applying Cvss to medical devices. So cvss, it's a it's a scoring system for essentially prioritizing vulnerabilities. So that you know where to focus the mitre attack framework. The mitre defend knowledge graph Iso 2, 7 0 0 1, which is kind of a foundational document related to to security. And then one of my favorite documents, which is the medical device, health. It joint security plan, which is a really helpful

kind of community document that just goes into details in a very practical way on many of the things you need to do for cybersecurity. And then, beyond that, you know you've got things like the NIST, cybersecurity, framework, and etc. I won't go through all these, but we go through these in the book and provide some details. Alright. So moving along from here, let me see if there's anything else in here that might be interesting. You can see the chop sub chapters. So why the regulation, the evolution of FDA cybersecurity, key standards and their roles, you know. So you kind of clicked some of that. See more details on what we're talking about.

Alright, and then getting into the more technical piece. So I'll I'll pause here and take a little bit of time to talk about this diagram. I think it's a helpful diagram. So this is looking at the process of developing a secure system from an architecture standpoint. So you know, a normal medical device product development. You know, you always think about user needs. You know, you've got your planning. And then you've got your user needs and but of course, your system requirements might also take into account business needs and and other needs. But at the end you create your system requirements. And you architect your overall system based on those requirements

when you get to the software piece. So we're dividing it here from between hardware and software. You know, you can divide it differently, of course. But in the software subsystem you would prepare your software requirements. And then what we're calling a proposed software architecture. So they're just to say, Look, this is going to be an iterative approach. So the 1st architecture that you come up with is probably not going to be the final one. because that architecture design leads to subsequent steps that cause you to revisit whether you're using the best approach. So you propose an architecture. And then you might ask the question, Does the system and hardware architectures, or do the system and hardware architectures support the software architecture? So for example, if you're doing an embedded type of system, and you know

I don't know you need Bluetooth connectivity and the the chip that your hardware team selected doesn't have that. Okay? Well, then, the system architecture does not, or the hardware architecture doesn't support your software. So you know, you might say, Okay, well, can we? Can the software architecture be adjusted to work with that system or hardware? If the answer is yes, then you might choose to do that, or if it's no. You probably bubble that up to the system architecture system requirements. So that then the hardware people, or whoever else needs to make that tweak. So that's just general product development. Eventually you'll arrive at some software architecture. It's really still not final. It could still have changes.

And this is where the security piece enters. Now, that's not to say you're not thinking about security at an even earlier stage, as as you become more seasoned than this, and you have people on your team who understand security. You're thinking about security even when you're doing your system requirements. But you're gonna find new requirements as you go through the process. And so one of the tools that helps you come up with those new requirements is doing things like generating the security views for your architecture. So things like your global view and your your patch ability, those kinds of things, or your like, your multi patient harm view

and using those views, you can then kick off threat modeling. Now, threat modeling isn't a once event threat modeling really happens over the course of product development. But during the architecture phase. That's a good time to start that process. and you may discover a number of of vulnerabilities in the system, or understand threats that could exist for the system. you would then try to quantify them and evaluate the risk. So there's there's a difference between estimating the risk and evaluating it. So estimating is like essentially trying to quantify it, and then you'll have some criteria for evaluating. Is it acceptable? Is it not acceptable?

If it's not acceptable? You'll probably need to implement some controls to make it acceptable. And then you'd ask yourself, Okay, is it a software control, or is it not a software control? If it's not a software control. So as an example, you know, there might be, there might be something in the instructions that you provide the user that they have to do. You know, it's not something you're doing in the software, but it's instructions for the user. Or it might be that you know, the hardware has to incorporate a crypto engine that's that's governed by the hardware design, not the software design.

So you'll you'll feed some of those controls back to the system level, and then some of those may come back to your software requirements, and you sort of operate in a in a couple of loops until you've got an architecture that is pretty solid. And then at that point you would proceed to your software design and software testing. So there's other activities that aren't shown in this diagram. but at the architecture level this is the kind of process you'd you'd want to follow. I can pause here in case there's any burning questions, and then we can move on to some other stuff. any anything burning? Let me see in the.

There were a lot of questions.

Okay, cool. Let me see. Alright. Well, it looks like Mohammed's been answering some of them, which is cool. That's good.

Jose, I have a question.

Yeah.

So let's say, a medical device company is remediating a device because most of the devices on the market had to go through this exercise. The question always comes, when is it enough? And they look to regulatory to be like? Is this good enough? Regulatory? Cannot give you that answer. You have to almost provide that justification. So could you give some guidance here of where you feel in your experience. And Mohammed? Obviously, please chime in that is achieved. And it's like you're telling us. This is a cyclical over over and over and over, and then the other follow up. Question is, you know, certain other Iso standards that come into the risk management. 14, 9, 7. 1 is the big umbrella one. But the other aspects, especially around software, get pulled into this, too, as part of cyber security.

Yeah, great questions. Angelina. So so yeah, one of the things that is a little bit confusing in the guidance document. But it's really not that confusing is FDA's requirement, that a company provide reasonable assurance of cybersecurity, and they actually did a presentation about this after the release of the guidance document. and you know the summarized version of that is, you provide all the documentation they request, and it's well prepared. Right? Now, as to the question of like, when is it enough that really boils down to, you know, in this diagram? And then you know this, this little box that says estimate and evaluate risk appears in many places, because over the course of the process, you're estimating the risk. And you're evaluating the risk. So in your cybersecurity management plan just like how you're doing your risk management plan for 14, 9, 7, 1, you're going to establish some criteria, and you're going to say, you know, when the risk is below this threshold, then it's acceptable

right? And so the the short answer is, it's enough when the risk levels are acceptable, and you've properly documented them to show that they're acceptable. So I would say, that's that's the gist of it. As to your other question about other regulations, another really good question. So there's a couple that come to mind right off the top of my head. So one of them would be 6, 2, 3, 0 4, right. So if you're doing software development, you should be following a process. And, in fact, if I jump ahead to Chapter 4, which is the implementation, one of the things we did is we borrowed from a diagram that's in 6, 2, 3, or 4, like the classic V model, which is just like this V here.

and we we tried to add to it like, where in the V model of 6 to 3 or 4, does cybersecurity fit in? So you know, like in in that software development process. As an example, 63, or 4 tells you you should have. You should have a planning document. You should have software requirements and software, architecture and software design and that sort of thing. Well, as as you're doing the software architecture, you would then put together those security architecture views and do your threat modeling and risk estimation. And that might lead to you going through another cycle where you update those requirements like what we talked about at the architecture level and then at the implementation level.

You know, you're pulling in new tools like software bill of materials, analysis and same thing. So, as I was saying earlier, like this little box that says estimate and evaluate risk, you know you see it here in the architecture. You see it here in the Sbom analysis. You see it over here in the you know, when you do your penetration testing and your fuzz testing there. There's different times during the development process where you're going to see these vulnerabilities or these risks, and you're always going to evaluate them and determine if they're acceptable. So there's going to be definitely a tight link between software development 6 to 3 or 4

and cybersecurity. The other one that comes to mind is usability. Right? So 6, 2, 3, 6, 6, in fact. when you look at 6 W. SW, 96, and Tir 57, they they actually bring up usability or human factors engineering as a as like a parallel or almost like as a precedent for what they're doing with cyber security in the sense that classic safety risk management is predicated on this idea of like, you know, stuff happens right and like failures occur, and they might have some probability and and some severity to the harms that are caused right. But in usability it's not like some random thing. It's the fact that if you design usability poorly, you might drive confusion which could lead to to hazardous situations. So there came a time where people recognized that the way you look at

risks associated with usability just needs to be a little bit different than how you look at it. For standard, you know, failures just like component failures. Let's say that frame of mind carries over to cybersecurity because cybersecurity, risk analysis is also different. So in cybersecurity, you're not thinking of a random event that has some probability you're thinking of a threat actor purposefully choosing to look for vulnerabilities in your system and exploit them to cause harm. And so so that's 1 way in which usability and cybersecurity relate the other way is that as you're implementing controls for cybersecurity.

you have to incorporate and bring in your team that has expertise in human factors. And your team that has expertise in safety because you might inadvertently create new risks related to something like usability. So you know. Imagine, for example, if you've got a I don't know surgical instrument that's used in an emergency situation, and one of your security features is that you enter your password, and it locks you out after 10 min of inactivity. Well, you can imagine how that could have significant usability implications when a doctor's in the middle of trying to perform surgery, and the security control causes them to have to go back and put in their password, and they're already gloved up and stuff. So you know, it's just kind of like an example. But you have to consider how all these things interact with each other.

That's that make sense.

There's been a couple of questions, Jose, on on guidances and overlaps and guidances and standards. And so, you know, this is something that we deal with all the time, which is the different standards. you know, 81 81 0 0 1 5, for instance, is actually older. It was published in 2021 is actually older than the 2023 guidance from FDA. The FDA borrows from that enhances it. We're actually just voted on reviewing. reviewing the Iec standard again, and it's probably going to go through another update in the next year or so. So these standards and guidances will constantly piggyback, and and they'll overlap, and then they'll put new additions, and then they'll change for the most part they they also target a little bit different areas. So some, some of the standards and guidances are are more more targeted to particular aspects of things. So you know, again.

I I think you just have to look at all of them as a totality, and and try to see how they all apply to you, to your situation. I mean there's always little holes that you have to close, but there's no, there's no perfect answer, unfortunately, and there's no one guidance. That sort of supersedes all of them, or one standard that encompasses everything. You, you just. It's just. It's just a way of life that you've got these multiple standards and guidances from different organizations, and you have to figure out how to how to make them all work together.

Yeah, we, I mean frankly, that's part of the purpose of the book is we're trying to kind of pull in from these various places and provide, you know, kind of instructions. I'm not saying, Read the book, and you don't need to read everything else like you still need to like this isn't. This has no enforcing enforcement power. But but the idea was to try to pull in all those concepts and then make a practical guide to how you would go about this. I see there's a question, too, about should we perform risk assessment for all findings identified through Sas and Dast? And you know, I mean, I'll say the answer is yes, like really any vulnerabilities

that you discover whether they be through. You know your threat modeling like doing like a stride type of analysis. or whether they be from yeah, from your your static application, safety testing or your dynamic testing or fuzz testing or penetration, testing or S bomb analysis like these are all different tools. You're trying to be comprehensive, right? So it's almost like like, if you're building a ship, and it's got 30 holes right? It's like, how many do we have to patch up? Well, probably all of them. So you're kind of looking for different tools to see. Like, you know, this one helps us at the architecture phase to understand those vulnerabilities. And now this one helps us in the S bomb phase. And I'll speak to that in a minute. And this one. So

yeah, so every vulnerability they discover you have to evaluate and consider what impact it could have. If you go back.

Back to what Jose said earlier about being comprehensive and part of this, and again, this is a little bit FDA centric, but I think it applies in general. Of this, of this reasonable assuredness of cyber security. It's it's it's certainly preferable to document and then rationalize and accept, than it is to simply throw away. And don't document. If you, if you just simply ignore things, there's a chance that you know it can be found, and someone will come back and say, Why didn't you think of this? So we tend to take that sort of more comprehensive view of saying, Just capture, capture what you find, you know. Be be split, be be be thorough.

Analyze it, and then, you know, rationalize it. And to the earlier question of that was asked about. When's enough enough? Well, you set your criteria, you set what your acceptability criteria are, and then, as long as you are meeting those acceptable criteria. You're fine, so fine in the sense that you at least have some justification as to why you're doing it.

Yeah, I mean, obviously, they need to be reasonable. Right? FDA will look at that. And if you're setting the bar too low you'll get pushback. But if you set a reasonable bar, and you know, as Mohammed saying like, that's the reason you always do. The security, estimation and analysis is to determine if you're at an acceptable level.

I'll give you one data point just very recently. So I don't know how many people are aware of this or not. I'll just bring it up that FDA are now employing automated tooling to basically analyze submissions. I'm pretty sure they're doing it very specifically in cyber security areas because they are able to pick out very tiny discrepancies in documentation. So they they are they. They are not just hand waving on this they appear to be very targeted and looking at the documentation being submitted and asking very, very pointed questions. you know. towards that. So if if you, if you, if you don't do a thorough job. There's a probably a very good chance that they're going to.

Yes, they're using. They're using AI. It's called Elsa. It's it actually came online early. It was targeted to come on late in June, and they actually deployed it second of June. And we've already had reviews go through with Elsa, and I've seen else the responses. So you know, it is ex. They're extremely detailed. They're they're they're literally drilling down on individual sentences and saying like, why are you claiming this? You know why? Why is this? Okay? And so so you need to be thorough.

But isn't Elsa like also hallucinating a little bit.

Might be.

I think they all do. You know, I I actually found the questions that were being and and it's changed the dynamic because we were literally going back and forth and getting response replies, within a day, I mean, we would. We would get a question, we would submit an response, and the next day we already have a response back. So it's very rapid now. I I found very similar to Mohammed that.

Yeah.

The questions and the responses all seem to mirror exactly what you got in the last response. So.

Similar. Yes.

Yeah, we've seen that. So clients will come to us like after they've submitted. And they get back a list of deficiencies.

And you're like, I just got this exact list. It's a calam. What are you? What's going on.

Exactly exactly. Yeah. I mean. And you would imagine that to be the case, because even before, when it was human reviewers, you know, they've got they had. It was very clear.

Reviewers. Yeah.

Yeah, they had a checklist now with Elsa. It's it's even more automated. So I mean, there's pros and cons to that right? I think one of the one of the potential downsides, you know, as Angelina alluded or mentioned is, you know, you're going to have the AI hallucinate, or maybe misunderstand something, or whatever else fine, you get an opportunity to respond to that. So you know, it's a very minimal downside. In my view, the upside is you get. You get responses very quickly, right? So you know, like we said, we've had clients come to us, and they're like we submitted it on Tuesday, and by Friday we had this list of deficiencies. And we're like, okay, let's dig into it. See? See how we can help?

So yeah, it's very clear that they're they're implementing it.

Are you cool getting the sense that the human reviewers are giving any oversight to the Elsa output before forwarding it on to you or not?

Sure. No, Carrie, you probably see more of these. We do just general responses. What's your take.

I mean. Well, when you call it on it all of a sudden you do. During their their phone calls. They kinda had a few where you call it. You call them out on it during the phone call, and then they're kind of like, oh, oh, oh! And then you get the human human response. But that's been I've had. I have had a couple in the last couple of weeks where they've kind of all of a sudden. Well, we'll get right back to you. And then it seems to take a couple of weeks to get the human response, and then you get the old fashioned.

Yeah, that's kind of the fear.

Because they're green, that, as you know right now, all the all the reviewers.

Properly.

Now we're pretty.

Big problem.

So that so that would actually, I mean, that would support. I mean, my view would be, is that there what you're getting as the initial responses probably are not human screened there. It's it's.

And.

Right.

And the output is the exact response. You just got on the same product code.

That is, and I'm guessing.

You're seeing a lot more questions coming out from it.

Yes, that that. Yeah. If I compare submissions earlier than June to what we were seeing in the June, you know, it's through June and into July. They're they're far more, far more questions and far more detailed.

And are there? Are there themes that are coming out in.

Awesome. I I would say the theme. Well, if there is a theme well, I'm going to give you the theme from the perspective of what I found as the answer that stops more questions. How about that? Right? So so what? As Carrie said? If you answer insufficiently, you're going to get the same question again. and I think the answer to that is, you have to be assertive and very specifically make it clear what the safety and efficacy. You know, considerations are for whatever issues being brought up. I think you have to frame thing. I found that framing things in those terms is what it's looking for as an answer. It doesn't want like just hand waving. It won't accept. It won't accept, just.

you know, vague answers. If you if you answer in a vague way, it'll just ask you again. It'll it wants a sort of definitive, assertive answer. and the.

Way to prevent a lot of that is to just be be rigorous. And in your original submission, right? So just have a good.

And it wants you to hand it to him in a silver platter and point it. Give it to you. They're not looking through the submissions. I mean, that's the whole point of Easter.

I'm sorry.

Hear me. I'm sorry because I've been. I wanted to jump in quickly. Are you able to hear me? Yeah.

Okay, great so I just wanted to kind of bring a little bit of not necessarily devil's advocacy here, but a little bit of balance, maybe, to this conversation. So recall, you know, not too long ago, when we were all complaining that reviewers gave us questions that weren't even related to our devices, or reviewers would respond back with the same questions they'd already asked us. I would argue that reviewers themselves pre AI hallucinated. So I hesitate a little bit. I see a lot of alarm and a lot of hands in the air about oh, my God! FDA is going to AI, and I respect that I respect the concern, but I don't.

I don't think it's extreme. I don't. I? You know, FDA current administration, whatever it is. FDA is built of intelligent people. Intelligent people make mistakes. AI makes mistakes, and for me, somebody who sat through 2 years waiting for an FDA clearance because the P. The reviewers just kind of kept shuffling it on and off. If I get AI answers within a week or 2, even if they're wrong, and I can call them on it and get somebody's attention. It's a win. So I I know that I am biased towards AI. But I don't want to see us in industry over indexing on this caution and this, oh, my gosh! This is not right, it can't be, you know, where there's not ready.

I feel like we need to support it, and we need to help FDA move it forward in a positive direction. Cause I hear what you guys are saying, and it's absolutely legitimate, but it's a step forward, and it's positive. And we had complaints about FDA before AI. So, anyway, not to call.

I mean, we're, I know, on on our end. We're definitely in agreement there, Monica, and and that's our position, too. I'd rather I have. I mean the same same exact line of reasoning? Right? It's like AI hallucinates. Yeah, humans make mistakes, too, you know. But if it gives me the answer rapidly. And it's like 95% good. Then we know what to address. And it's the same kind of thing from from a submission strategy standpoint. It's like, you know. follow the process correctly, document it well, and then submit it. And yeah, they might come back to you with. They're going to come back with fewer questions if you did a good job, and if you did a good job again, like what matters is like the bones right? That the thing is actually properly designed and and secured. Because if FDA, whether it's a human or AI, comes back with a question

and you did things right, all you have to do is explain it right. The nightmare scenario is when you didn't do things right, and they come back with a question, and it's a legitimate concern that you've got a vulnerability that you haven't properly addressed. That's the nightmare scenario, because then it's like you're not gonna be able to give them an excuse. You're gonna have to go back and re-architect your system. Redo the you know, whatever the encryption, your authentication, your whatever whatever thing you missed potentially re probably redo your verification testing redo your pen testing, you know. God forbid hopefully, you don't have to redo your usability or your clinicals. But that's that's the real risk. So it's like, make sure you follow the process. Well, so your system is actually secure.

And that in that that scenario absolutely a hundred percent. True. But if you've got a problem like that, it should be caught, whether it's AI or or reviewer. So yeah, totally aligned.

In fact, you want you. In fact, you want it caught, because the last thing you want is for something to get through and get through FDA and get cleared. And then we have a problem in the field that that's the last thing you want to happen. So you know, I I take it, as I take FDA, I mean, it's almost like how I do with my building department. I take FDA as being a gatekeeper, that their purpose there is to ensure that there are problems in the field now. It may be a little painful sometimes, you know, to get through reviews, etc, but it's, I think I think the spirit of it is the right thing, which is it's it. They're trying to prevent failures in the field. That's ultimately their goal.

Yeah, good stuff. Good discussion. I'm glad we're seeing it, and I know we've got a few more questions. I'm gonna let me wrap up just a couple of things here, and then I'll shut down the screen and let's just have a good conversation about it. Just a really quick thing that I realized I didn't show you guys. But as you're looking through this, by the way, the link is right up here, so it's book, dot Cybermedai. You don't need the dashboard, just the book. Cybermedai, and that will allow you to register and then start doing it. And then in the document, let's see. You'll you see kind of how like, as you hover over text it, it highlights that allows you to just click and then leave a comment like

this is confusing. Add more details right? And then that'll get added here, and we'll be able to see everybody's comments. And again, the the purpose of this is to really take your feedback and and hopefully at the end have something that's that's better. I won't go into the details of this. You'll get a chance, you know, in these star submission artifacts, we try to break down the various artifacts and just give you tips on things that we've learned. Having gone through this, you know, kind of from like when you're developing it versus when you're submitting it. What are some extra things you'll want to look at, you know as an example like you'll want to make sure you have sufficient details. So this is the kind of thing that could get flagged. Like, if you're just saying, okay, an attacker could spoof authentication messages like, well, when you're submitting, you want to make sure you say, okay.

just provide more details. Right? So authentication, spoofing leading to unauthorized therapy changes. An attacker could craft a fake authentication message to bypass user verification. What's the clinical impact? What's the likelihood? Those kinds of things. So just make sure that you have good details. and you know good, good traceability and that sort of thing. So I'll stop there any. And then, yeah, let's let's have the conversations I'd I'd love to hear also from other people who've done recent submissions. and some of the things that you're seeing, or some additional questions you might have.

Hi, Jose, this is Fred.

Thanks.

For the conversation. It's great. So I just wanted to go back to the submission strategy point that you were making right. And I'm curious about your experience, because obviously, you know, I've gone through submissions, and especially recently, not so recently, because obviously, after June, things have changed a little bit. But, more importantly, when do you think, is it time to do a submission when it comes to like, you know, preparing that cybersecurity documentation package just being enough to submit as opposed to. Just wait until you completely, you know, validate and have every single documentation complete. So when do you think you know it's the right time, you know, so that you could just kind of like strategize to

maybe assure, like an earlier clearance if you know what I mean.

Yeah. So I mean, look from a cyber security standpoint, you'll want to make sure that you complete everything that's required up to and including your penetration testing and fuzz testing right? So at the system level, you know, we didn't cover that here. But it's in the book you'll want to do fuzz and pen testing of the system to make sure, like any final vulnerabilities, that you haven't caught through all of your other processes. that hopefully you catch them at that point. So at minimum, you need to have that beyond that kind of like, at a more global strategy. I know we've got some awesome experts here, Angelina, and you know Carrie and others

who do a lot of more submissions. We don't do regulatory submissions. We just focus on cybersecurity. Well, you know, if we get clients that need that help, we refer to them to other experts like that are on this call. But yeah, from a cybersecurity standpoint, you know. Make sure you've got all your documentation ready. Make sure you finalize all your testing, and that your devices is secure, that you've got evidence that it is, in fact, secure. Beyond that. I don't know, Carrie, what what do you think in terms of strategy, of when to submit.

I mean, there's many strategies that you have to have all of your documentation ready. You know there's it depends on the size of your company, and if it's your risks and one to go, you know there's a lot of smaller companies that'll end up doing your design history file while your submissions in the queue to get everything else ready to go to market. I don't know.

True. You want to create a reputation with the agency, and having that reputation with the agency, builds that trust. Even if there's an AI bot giving you deficiencies. So my rule of thumb here is. are you set up for success? That should you get any sort of deficiencies, does not derail the business completely. That is the rule of thumb here for me. So if you get an AI letter that you say, gosh! We took this risk, and now it's going to be 6 months for us to fix that. That's not a good ring strategy. perhaps strategy like that. There are businesses that want to do that right. They're like, just submit it. We'll deal with the questions later. You have to be savvy enough to go. Where is that risk, and how derailing will it be to the actual submission, and to your reputation to the agency? It is key.

A lot of one, a lot of agency, or a lot of companies want to get it in for their stockholders, just to say we submitted it. So their stockholders are, you know. Say, we've submitted it, and yet the device isn't ready to go. They're still in the process of doing their 6 0 6 0. 1 testing, and then they'll, you know. But you need the way the East are set up. You have to have all your testing done now, or it's gonna get kicked back. And you're gonna get a deficiency letter and you have 180 days to submit. And if you wait till the end of it. You know, you're going to get an Nse non significant equivalence letter, and you're gonna have to resubmit. So.

There's another question here, too, on kind of submission of devices. So I'll tell you a quick story. One of our clients has a device or had. Yeah, still has a device on the market that had been cleared like 5 years ago. They got new clinical data that supported an expanded indication. And so they submitted that to FDA and FDA came back with 14 cybersecurity deficiencies, because, you know, in their original submission, it was before that October 2023 deadline. So just be aware that just because your device is cleared doesn't mean you're good to go. If you ever have to go back to FDA for any reason whatsoever. At that point they're going to require all of the cybersecurity documentation. So it's not like this grandfathered notion of like, okay, you're in the clear, you still. And frankly.

this is all stuff you should do like. This is stuff that's important to do. To make sure your device is secure. So, putting aside the the requirements from FDA, you want to make sure that you're doing this stuff so that your system is secure. so I hope that answered your question. Who was that, Matt Bontrager? I think alright.

Want to call either Lisa or Monica into the Reg discussion, because they're seasoned as well. And personally, they're both my mentors.

I mean just the one thing that I was going to interject there to build off of what you had said Angelina was in considering when to submit. think about what are the potential questions that the agency is going to have. So you should know before you submit. What are your potential downfalls. What are the warts that they might ask a question about? And if they did ask a question about it, what is it going to take to respond to that. And, as Carrie said, if they do ask a question, you have 180 days to respond. And so, if you're if the response isn't able to happen in that time. Frame

that one's not a good risk to take. So absolutely get that duck back in a row before that submission happens. You know Jose was absolutely right, you know, having your existing clearance changing that device. doesn't grandfather you from the from the updated requirements. And that's something that I see often with people with biocompatibility. Another great example where people maybe haven't changed things. But the standards continue to update. And as soon as you put a new submission in. Now you've got to react to what has changed out there. And and that's a good thing for all of us, because even though we're industry, we're also all our patients. So we want

good product to get out there. And you know, I think Monica had said it really? Well, you know, FDA is out there trying to do a great job. And you know, I going back to the conversation about the Elsa tool. our Monica and I both serve on the Twin Cities chapter for wraps, and earlier this year, when the the 1st round of cuts happened to FDA, we we led a session to to support the FDA people who had been cut. We've been trying to support them throughout the year, and

we're we're we are out there. We're trying to find ways to to support FDA. I think the use of AI is one of those great ways to help the review staff be more efficient. There will be some growing pains as with any change that happens. You know I'm encouraged that it is kind of a closed system that it's only being trained on FDA documents. So you know, if there, I'm sure there are some hallucinations. I expect those to be minimized because of that. But you know I ultimately I expect that that will be a positive thing moving forward.

you know. There we do have an issue with some green reviewers at the at this point, and we'll always have an issue with some green reviewers. We we have tactics to to deal with that. But we were a complete ecosystem. We need all work together. I'll get off my soap bucket soapbox, Jose. It i 1 question I had for you is when you were talking about your mission who? It's clear that you guys support the test, the testing documentation. It's clear that you guys have been involved in helping with some remediation of issues from FDA. Do you guys get involved in all of the documentation for cyber security for companies? In all the 1314 documents.

Yep, yep, I mean, in fact, an engagement with us. We usually prepare all the documentation. So our approach, our team is made up of like seasoned software and cyber security architects. So the approach is we come in. We learn your system very quickly. We put together the security architecture views, and then we go through a process where we do the you know, we do the threat modeling the risk assessment everything. So we yeah, it's pretty white glove. So yeah, I mean, definitely, if anybody on the call needs help with that, please reach out. And I know Shawn has on this call like likes to have an AI minute at the end. I told him I'd be happy to share kind of one of my little tricks that I like

to to use. AI. I use all of them for many things. and you know, kind of a fun trick if you haven't created diagrams with AI is is to tell it to create a diagram. So there's there's a couple of different scripting diagramming languages like Uml and Mermaid, and and a couple of others, 3D. 3 js, so you can tell us something like, All right, here's another tip. I use a tool called whisper Whisper flow, and I'll show you what that looks like. So create a diagram using mermaid that lays out design controls for medical devices in a straightforward way. Okay, cool. So it's like text to or voice to text

that works really nicely. And then what you can see is, you know, here I'm using Claude, you know, to to an earlier point made that they're using a closed system different ais kind of tend to have different levels of security, or, or, at least assurances that it's not gonna leak out into into the world. train the models on it. But you know, here's like an example. Now, this diagram is fairly complex. Kind of hard to read. So you can tell it right? This is too complicated. Give me a simpler version and hopefully simplified for us. There you go.

So now you know, it's telling us design controls, design inputs. Review. Now, this doesn't guarantee the answers are right, you know. I like to say that AI has brought the term. I know enough to be dangerous to a whole different level right where it's like you. You really need experts that are looking at the outputs and that sort of thing. But here it is. It's telling you, you know. Plan, input output, review, verify validate, transfer etc. So pretty good. You could tell it to do the same thing with things like uml or drawio, which is another one I like. So something to play around with for AI,

yeah. And with that, yeah, thank you guys for for joining no sean any last words. You're you're muted, Sean.

Yeah, so thanks everyone. For your time everyone that contributed if you didn't write to me. And you want to write to me. My email is Shawn at. oh, my email address is so hard to type. B, 2 b.com sean at view. 2 b 2 b.com. It's in the chat. Feel free to message me. Book a call with me. Talk to me. I love talking to experts. That's my favorite thing in the world that in mind next week. Nelly Bushman. Oh, and if I don't need to point out the obvious, if you need your cyber security documentation done hire cyber. Med. You won't be sorry. And then next week Nelly Bushman is going to be helping us look at the costs of quality management systems from a whole new angle. It's going to be really interesting and really good for small to medium sized companies.

Awesome.

Anything. I forgot.

Just a quick little announcement I put in the chat. But if you're trying to register for the book and you kick back, get an error, saying that too many people are doing it at once. Just try it in a little while we'll increase the throttling. But in the meantime, just try again in a bit, and you'll be good. And yeah, Sean, you'll you'll share the link. There's something yeah. And I want to thank everybody for showing up this week and bring a friend next week. Awesome! Great thanks, everyone. Bye-bye.

Bye, everyone.

Bye. Thank you, Jose.

But I think comment.

Thank you.