How to Develop, Secure, and Get FDA Clearance for Software-Enabled Medical Devices

Hello, and welcome everybody to how to develop, secure and get FDA clearance for software enabled medical devices. Our speakers today are Casey Armstrong, Mohammed Fastook and Rob Packard, moderated by Jose Barquez. So before we get started, I would like to share a few housekeeping announcements with everybody, and one small advertisement. So the recording and slides will be made available tomorrow. You should get an email via zoom with the link in it. Unfortunately, you're gonna have to cut and paste it, because zoom doesn't make anybody's life easier. Second, use the Q&A button vigorously throughout this

webinar to ask questions of the panelists. We want you to ask a lot of questions. We welcome your questions. use the chat function for comments to me, or to request certificate of attendance, we will provide a certificate of attendance to people who are here that request it. You need to give me your name and email it your name as it should appear on the certificate and your email address in the chat. Please give us about 48 to 72 h. To get these out. We have to make them by hand. The last announcement is, if you're going to be in Southern California next week anywhere near Orange County, Newport Beach, please contact me in the chat about

participating in 10 x 2025, both Jose and Mohammed will be speaking there. So this is your opportunity to meet with them in person. Even if you want to come for an hour, I'm fine. With that I will give you a pass. Just let me know if you would like more information, and I will provide it. This is an excellent opportunity. Now getting started. I'm going to hand it over to Jose, who is our host?

Good morning, everybody. I'm really excited about this webinar you know, even though I'm moderating, I feel like I'm also going to be learning. It's 3 people who are experts in medical device product development. And they're going to each be talking about different facets of specifically medical devices that incorporate software because there's particular challenges to that. So Casey is going to kick us off giving us just an overview of some of the challenges related to software in products from a product development standpoint. Mohammed's going to speak to the cybersecurity challenges the things to have in mind common pitfalls.

Rob Packard, who heads up the Medical Device Academy. Some of the best content on medical devices you'll find online will be speaking to the regulatory side, and some of the unique challenges around medical devices with software. So with that Casey, maybe just a brief introduction to you and nocturnal. And then you can speak to the product development software challenges.

Sure. So I'm Casey Armstrong. I'm the president and founder of nocturnal product development. Nocturnal develops primarily active, implantable, and wearable medical devices. So we specialize in ultra low power, hardware and software, very compact. Small things that do a lot. My background is mostly medical devices. I've been in the field for 27 years, started in drug delivery, moved pretty quickly into cardiac rhythm management and then started nocturnal 14 years ago. And we've been mostly, as I said, implantable and wearable. So we're heavily focused on the product development side. We also force edge into regulatory approval because we have to prepare documentation and do testing that supports regulatory approval. But my, my focus here will be largely on the product development process and creating the thing to begin with.

So with that Jose, do you want to just throw up my my one slide here with

If you have.

And I can share it. That also works So I'll just. I'll just do that share right now. I don't wanna share full screen apologies. Just want to share that there. Okay, so kind of 5 key points here, things you might not have thought about about product development that we see all the time. I'll kind of go through these. So first, st this is, gonna be an unpopular opinion, maybe in this crowd. But, contrary to formal teaching. It does not always make sense to follow the full formal medical product development process right away. especially for our primary clients are startups and early stage companies where money matters and milestones matter and the early milestones are not usually not usually documentation. They're quick functionality. Does this even work as a product. Can I make a business out of this?

So if you follow the formal process, then that would tell you, you need to do your full design. Input, first, st you need to establish all your requirements and your hazards and your user needs. We tend to do that for a lot of customers informally, so that we can quickly get to something that works that meets most of the requirements. And once we find that out it's easy to justify spending money on the formal documentation. You have to develop it right from the beginning. So you can do that later, but doing it from the start can consume a lot of money and a lot of time creating documentation you may not need for quite a while.

Number 2. Don't try to do everything. If you want to. Your product everybody tries to put the kitchen sink in their product. they try to do what other companies do and their products when really they need to be focusing on the things that are special about their product. What's new to the market that they're doing, there'll be time later to add other features. But it's so tempting to add things. If you, if you're not careful and don't stay focused, you'll run out of money before you get what really matters to the market in your v 1, you can always improve it later. So don't try to put everything in the beginning complicates regulatory approval complicates testing and documentation complicates development. Stick to what makes you unique.

Another item here. Poster system components are to the patient. We do a lot of aside. We do. Almost everything we do is cloud connected these days. Implantables, wearables. They all have an implanted component or a wearable component, a mobile component and a cloud component. You want to push analytics and business logic as far up that stack as you can the closer devices to a patient, the harder it is to get approved, and the harder it is to update later. So you want, for example, one implantable device to be able to gather the basic data you need and perform the therapy you need. But you want it to do as little thinking as possible. You want to move that up to mobile or cloud. Because if you want to improve your system later, or you have a change, you want those things to happen far away from the patient, because it's much harder to get the FDA convinced the FDA that it's okay to go update firmware in a bunch of implanted patients than it is to update something in the cloud.

well, what we do is very, very low power. And we've learned over the years that ultra low power is really a state of mind. We get a lot of customers who ask us, Hey, what are the few things we can do to to make something low power. And it's not that easy. It's not a bullet list. Everything you do, especially the software, everything, a single thing you do. You need to be thinking low power. It's an art as much as it's a science, and it's an approach to doing things that starts from the ground and moves up. this is here. It's driven a lot by software. It's a very iterative process. Add a feature, reduce power, add a feature, reduce power, and set your data sheets. Lie trying to figure out how much power something you're gonna draw. Just by analyzing data sheets for a microcontroller or other hardware will get you part of the way there. And it's okay for comparing one approach to another.

But in the end you need to build hardware and get software running on it. Before you really know what your power consumption is. So the sooner you have hardware that you can develop on real hardware and not just a development board the better. Finally, the details of the business case don't just matter to the finance department. They matter to engineering a lot we spend a lot of time early on the development process, talking to our customers about where they're going with the product. Not just v, 1, but v, 2, v. 3 and where the value is in their in their company and their product. you have to architect from day one for what you want to do in V 3, or you're going to end up redoing a lot of work later and having to do a lot of work with the FDA to approve why your thing is so different than it was to start with.

Okay, that's probably probably good for me.

Casey. Those are. Those are all excellent points. I'm nodding my head up and down because, because, having been in medical device product development for 20 years. Yeah, there's so much wisdom in all of this, I mean, and especially number 2. I tell people that all the time. So you don't need to pack every feature, but it's like Number 2 and Number 5 kind of go hand in hand. On the one hand, you don't need to pack every feature, but on the.

Correct.

Some thought to the long term architecture, so that you don't box yourself out of maybe a future feature that's important to you. And so, yeah, this is, this is really excellent. Thank you. And we're gonna open up for questions here. After the the 3 panelists give kind of a brief intro first.st And so with that, Mohammed, if you don't mind, let's let's learn about cybersecurity and some of those key concepts that people need to have top of mind.

Sure I'll just give a quick background. So my name is Mohammed Hustok. I am the cso of cybermed I've been in software for a little under 40 years now, focused a lot in mission critical, secure mission critical communications, did some work in nation, state security. and now, applying all that in the in the medical device, cybersecurity, space. So with that, let me share my presentation should be enough full screen mode already. This is a quick contact info. You will get all the slides, so you'll have all these details. Let me just jump right in to some of the things to consider right up front. And this is FDA focused because primarily, you know, you've got to get your devices cleared by FDA.

Fundamentally, what does FDA ask for? These are the 5 top bullets. This idea of reasonable assurance of cybersecurity is something you have to convey to FDA. They have to get a sense that you know you're you're confident of this. As a manufacturer. They expect you to look broader than the medical device, and this may not have been true in a lot of the functional approach to medical devices previously, but with cybersecurity. FDA stresses this that you need to look beyond your medical device. How does this medical device interact. As Casey said, a lot of medical devices these days do communicate and are part of a larger ecosystem. So how does that ecosystem affect the security of the devices themselves.

Designing for security. I'll touch on that a little more in the next slide. and then they expect a very, very aggressive post-market approach. So cyber security is a constantly changing landscape. Attackers are constantly changing their techniques. New issues or vulnerabilities are found regularly. So you need to be on top of it. Be aware of what's happening out there. Once your device is in the market and show FDA that you are actually keeping track. I'll touch on that a little bit later as well. And then finally, to sort of address the fact that this is a dynamic situation. They really expect devices to be updatable or patchable once they are released into the field. And there are some strategies around how to achieve that.

Okay, with that in mind, I'm just gonna zip through these pretty quickly. Another aspect of of security to consider that is, that is really critical to the whole approach is thinking of security. Along with the rest of the software development lifecycle. So it's really all about integrating security into the development lifecycle. And that really starts at beginning with architecture. Once you once you reduce down your user needs into requirements. And you've developed your software architecture. That's the time to start bringing in security. And start thinking of the security considerations to that architecture. And I'll again. I'll get into a little bit more details in the next slide on that once you're through that phase and you're looking at the actual design of detail design of the product itself and its implementation. Again, it's keeping security 1st in mind, as you're performing your

projects so that you don't run into problems at the end. You're addressing them as you go through the development. And then I already mentioned post market. So it's again integrating an approach for how you handle post market surveillance from a security standpoint, and it can be tied to the post market surveillance from a functional standpoint, no reason to do completely separate things. But there are different inputs and different things to consider post market from a pure security perspective. So with that in mind, I'll spend a bit more time on this slide. This, this slide basically depicts the entire flow of what happens from a cyber security perspective? For your product development

on the right, what you see there is a list of all the various artifacts, documents, and other information that FDA expects you to provide. So this is what you would actually be submitting to FDA. Your views, your models, your safety assessments, your controls, your risk assessments. the management plan, all those initial ones that are shown in red there. That's considered. Sort of that. The 1st phase of documentation that you do, that you're really preparing upfront in your project. And then what's shown in blue is is this of the outputs of your development process. So your your bills and materials on your policies have to be machine readable, your assessment of unresolved anomalies. If you have any left over anomalies that come out of your actual functional testing need to be analyzed for their security impact.

And then obviously, your cyber security testing penetration testing plus testing being 2 typical ones. Software levels of support is a little bit of a tricky one. Sometimes it's really determining and understanding the support level of the 3rd party pieces that you're using from a security perspective metrics, risk management reports and labeling also round this out. So if you, that's on the right side. If you go on the left side, it shows so shows the flow. And it really begins with your design inputs that you would normally start with. And in that 1st phase of architecture. So this is happening in parallel to your

actual software architecture that you're developing for your product. You're looking at the security, architecture, aspect, and you see a little loop there that happens. So you begin with the security architecture. You perform your threat, modeling on it, establish what your threats are, perform a risk estimation, determine what controls are needed and controls become then requirements that drive back into your architecture and security architecture, and you will basically iterate through this loop as you are devising controls and reducing your your threats to acceptable levels. Once your threats are acceptable, you move on from there into your design and development. You may have some residual threats that you considered acceptable. That's fine, you document all of those

you enter your design development phase. And this is where, again, you're repeating this loop where you are performing your design and implementation, generating S bombs out of it, performing analysis on the S bombs. And this is a dynamic. This will change if your development is 6 months. It's possible that what you find in the early months changes months down the road. As I said, 3rd party pieces are constantly evolving. There's new threats identified with them. So the more 3rd party you have, the larger iteration you may have here. And sometimes it's it's desirable to know early upfront, what issues you have with potential 3rd parties. You get into the the mode of of screening 3rd party libraries and 3rd party subsystems to determine which ones are most secure, and perhaps sometimes you'll make the choice and say, I'll I'll choose a component or subsystem that is more secure, or has less issues

in favor of one that isn't, even though perhaps doesn't give me quite the functionality I need. And you do these trade-offs eventually, once you've gone through and achieve the level of stability in your project, and you're basically ready to perform what would be your verification and validation. So again, this is all running in parallel to your normal development flow. You get into your cyber security testing and you're doing basically, as I said, penetration testing, buzz testing and to the typical ones. And if you're finding issues, you're going again going back into your design and implementation phase, making adjustments, perhaps updating components and performing your analysis again

and and looping around. Eventually you reach a level where that is stable. You get your final test reports, which is what you're going to be submitting. And again, you may have some residual threats that you've decided are acceptable, and you document all of those. And now all of the all of that flow basically feeds into what is your post market? And in post market you are. You're monitoring what's happening. You are required by FDA to submit to them a list of metrics, their performance metrics on cybersecurity, on your ability basically to track and and stay ahead of cybersecurity issues. You would. You would actually present that as part of your management plan and initial template of security metrics, and then FDA would expect you to keep track of that. So you are tracking your metrics as you're going along. Your incidents are happening. You're reacting to them, and that that may then, you know, result in it. Feeding back in where you need to do updates to your product. And then the loop starts all over again.

So this is just a very quick overview of all of the activities that you would do to address cyber security as per the 2023 guidance from Fba.

Thank you. Mohammed.

Yep. And here, just to finish slide, and that's it for me.

And in full disclosure. I'm the president of Cyber, Med. So Mohammed and I are partners, and, in fact, just today on Linkedin, if you follow me? We posted something on kind of that post-market process in more detail if you want to check that out. But thank you, I think a lot of nodding of heads there as well, Mohammed on. Really, you know, cybersecurity has become kind of a hot topic with FDA in the last year and a half since their guidance document in in late 2023. And so I know that's something that a lot of companies developing products or software are maybe scratching their heads a little bit about so that's a great overview which moves us now to Rob.

who gets to see not only the challenges that companies face when they don't do the software and cybersecurity right? But he gets to see all the challenges that come as related to the regulatory space FDA and even Europe and other regulatory locations. So rob quick intro on yourself. And yeah, I would love to get your thoughts for the team on some of those challenges that companies face from a regulatory standpoint.

Sure. Thank you. And every single time I listen to your team talk I I learn a little bit more. So it's it's a real pleasure. In fact, just before this I was talking with a client he's like, Oh, yeah, I'm already working with Jose. So just are the clients that are well prepared have great help, and and that starts with you know what the requirements are. So when a client comes to us and we're a full service quality and regulatory consulting firm. They come to us, and they say things like. how quickly can we submit, and how much will it cost? They don't know what their regulatory classification is. They don't know what their predicate is. They don't have a design history file. They've never heard of 62, 3, 0, 4, and they want to get out of cyber security. But it's a web based software.

So all these are bad. We start out with really simple questions like. Do you know what? 62, 3, 0, 4 is? And when they they tell me 63, 2, 0, 4.

Like.

That answers my question. You don't even know the standard number. That's a great answer. So you have to have people that have a procedure and a process that understand the requirements for medical devices. That's not the same as I know how to code in Python. They're not even remotely the same. There's people that code. And there's people that design software and engineer the software. That's what Jose's team and cyber Med does. They? They engineer the software, not code they can code do. But anybody can code if they spend a little bit of time online and studied hard.

But that won't get you through the FDA. So I say, do you have a procedure? Do you have the standard? Do you have anybody that knows how to do this? The documentation? And they're like, well, how much documentation is needed. And I'm like, well, one of our submissions was 6,000 pages, and it was software as a medical device, and it was a mobile app. Now granted it was for something that was a de novo, and it had a clinical study, and so there was a lot there, but it was still a mobile app with 6,000 pages of documentation. So when people ask me like, How much detail do I need? And they give me 10 page documents.

Nope, you got the wrong number of zeros. You need 10 different attachments for software, 12 different attachments for cyber security, and that's just 2 of the 20 sections that you have to fill in for a regulatory submission. Our company will help you with a quality system. Our company will help you with regulatory submission. But you've got 2 other things that go in between those 2. One is you should have a presup with the FDA, and there's a lot of dissenting opinions like it's a waste of time. It's not a waste of time. The purpose of the pre-sub is not to have the FDA design your device. The purpose of the pre-sub is to forge a better testing plan.

And I don't just mean software and cybersecurity. There's also this section where you have to convince the FDA that your device actually performs a medical device function. So that involves benchtop testing animal testing human clinical testing. If you don't have any performance testing, you don't have a device. You're not gonna submit. That section will be left blank and they'll reject it because it can't even be submitted without it. So you need to forge a better testing plan with the FDA. And I use that word forge intentionally. You're going to take your plan. Stick it in the fire, called the FDA. It's going to come out molten orange and red, and you're going to hit it with a hammer many, many times, maybe hit your thumb a couple times, and then you're going to put it back in the fire again with a second pre-sub, with a revised plan. And you might even need a 3rd pre-sub because you're going to do software testing cybersecurity testing and maybe even clinical studies.

So that's the performance. Preparation is you're coming up with a plan for your testing. You're going to do once you get agreement with the FDA in your pre-sub and you forge that test plan. Now, you actually have to do the testing. and that takes time and money. So I put it in rough terms. Let's say I spend a hundred days doing a pre-sub? I do 100 days of testing, and I spend a hundred days reviewing with the FDA. That's 300 days. So if your plan that you've told investors is, you'll be launching your product next month, and you're going to get a 15 K. But you don't know. 62, 3, 0 4. We have a slight disconnect.

And can I fix the problem? Yeah, I can I. I have a magic wand, but it doesn't accelerate time. It just stops the bleeding. So I can introduce you to people like the speakers on this call that can help you with your software and help you with your cyber security. We can do a precept with the FDA, and we can come up with a better testing plan. We can do the testing. But when you submit. our specialty is the submission. And when you give me a list of deficiencies from the FDA, people are like, well, we want no deficiencies. We want this to be absolutely pristine when we submit it. Well, that doesn't exist.

The best case scenario. You're going to have one AI hold deficiency. That's a major that's the best case, because that's the best I've ever done, and we're talking hundreds of submissions. The worst that I've ever seen. A company came to us after they submitted on their own, and they had 48 major deficiencies, and we were joking earlier before we started about how many subparts for them, like, I'm not counting subparts like subparts a. Through M. That was on number one number 2, number 3, or just the major points, and then we had all the sub points below that, and I still got that company. 5, 10 k. Cleared.

So the company, with one deficiency, and the company with 48 deficiencies. They both got cleared. and at the end of the day the investors only cared about did they get cleared. but it took a lot of work to get 48 done. And so there's the 3 phases there's we come up with a better testing plan with the FDA and a presub. We do the testing with a team that knows how to do cybersecurity testing and software performance testing. And then the last part is we submit to the FDA, and we have to fix all those deficiencies they identified, and they will. All 100% of the submissions I have submitted in my life have had

an AI hold 100. I have people that work for me that have magically, somehow managed to get one that was interactive and didn't have a hold. but the the person with the most experience, she only had one of those. The person with the least experience only had 2 of those, and I've had 0. So I think it's luck. When you're looking at odds of single digit. the chances are you're gonna have an AI hold, and you're gonna have to fix things. So a hundred days, a hundred days, 100 days, they answered. Question number one, how long is this going to take 300 days, and you budgeted a week. The second thing is, people ask how much money

I'm the smallest piece of your budget. I'm just the guy that prepares the form that you submit to the FDA. The people that are going to do the real work are the people that are designing the software, architecture and retesting and retesting and debugging and doing cyber security, and then re testing. Those are the people that are doing the bulk of the work. And that's where the money goes. It's the testing of your device to make sure it works, and it's safe.

Hey, Robin? For the Newbies in the group. Ai! What does that mean?

Artificial intelligence, and it it's not going to write your 5, 10 k. For you, no matter how many want to.

We're talking about holds from FDA.

Yeah. A. Oh, the. So the AI hold in that context is our additional information. So the FDA had they reviewed your submission and they found the following things wrong, they always find things wrong, because the default answer is, Give me more data.

They? They feel like they haven't done their job. If they don't send you at least.

Yes, same thing with auditors. If I don't give you a nonconformity, I I haven't earned my keep as an auditor.

Perfect. Well, thank you, Rob. I mean, that's a great summary. And some of the horror stories that I feel like, you know, when I launched my 1st Medical Device company. I don't know 15 years ago, or something. I remember going into the patent attorney's office, and they said something about like a 510 K. And I pretended like I knew what that meant. But you know, being a grad student, I had no idea, and I went back and googled it. There's so much to learn. There's so much to learn just on digital health. There's a nice page in FDA's website that has 24 guidance documents. If you're building a product that's related to digital health, and it's like, you know, that's the benefit of bringing people onto the team as consultants, advisors, partners that have gone through this and have the the scar tissue. You can borrow

before we go much farther. I know Shawn just put up a a poll. So if you could take a minute, just kind of look through that really quickly, it helps us as we're gonna continue the discussion here in the next half of the webinar to make sure that we cover some of the topics that are going to be important to you. So please take a moment. Fill that out. and as as people go ahead and do that, the second thing I'll ask is in the chat. Please put any questions that you might have. So I've got a few questions of my own, but of course our goal is to help. You have your questions answered that you might have from either for any I mean any of the 3 panelists. So I'd like to transition. Now, Mohammed, if you want to come back on

to the interactive part of the webinar and really start digging into the weeds on some of this stuff. And maybe, Casey, we could start with you, can you maybe talk about like just an example of a mistake that you see companies make when they approach software development. What's kind of like a big one that you oftentimes see. Oh, you're muted good job.

Actually, I have. I have 2 2 things. I'm I'm thinking between. And I'm actually gonna gonna change the last second here, because someone asked a question about sales of fake devices on Amazon alibaba, and how much that? How do you address that kind of related to that? We get a lot of. We get a lot of customers who want to take a less expensive path to getting a product to market and say, Well, hey, I want this to be a wellness device, or I want it to be some wearable thing that isn't a medical device, so that I can get it to the market, sell it, make money, and then turn it into a medical device.

And what we have to explain every time is, those are 2 very different development paths. And you have you. If you want to later turn into a medical device, either you're going to develop it as a medical device. Now, in which point, in which case you may as well get it approved as one, or you're gonna develop it as a consumer device and trying to make them a medical device later, just by writing some documents doesn't really work. They're 2 very different paths. One requires a lot more rigor, and you can't just add it later. So we we've seen a a surprising number of potential customers with, Hey, I wanna I want to go measure this physiologic parameter with a watch or something, and I want to tell the patient these things. But have it not be medical information, just like cool a data for them to have.

You can't really do that either. Right? So So, to sum that up, I think one big mistake is thinking that you can get around the medical approval process by making something and saying just saying, it's not a medical device, but have it be a medical device when you want it to be. You've really got to pick a lane.

Yeah, no, that's a great point. And it comes up, I think, in software and in cybersecurity, and probably in other areas, too, is like part of what FDA is looking for is process, not documents. And so if you process and you're just trying to throw documents at it at the end. You're going to run into problems so completely agree with you. There? Kc, that's that's really good advice, I mean. And and to companies to your point, I mean, if you're going to develop it as a medical device, maybe the strategy is while you're going through the clearance, you launch a version that's more consumer or whatever. But you've built the product from the get. Go to to meet those requirements.

Yeah, you can. If you structure development, right? You can drop a not a consumer ish thing out of the process somewhere along the way, but you have to develop it as a medical device in the beginning. You can't. You can't shortcut the development effort for sure. Because it's a lot cheaper to write software for a consumer device than just for a medical device. But if you write it as a consumer device, you have to start over again later. So there's there's some parts of the process. You just can't get around.

That's great. So I've got a question here from George. He's asking if the cyber security risk assessment results in a low risk value, ie. Acceptable to plan is cyber testing like pen and fuzz, still required. Mohammed, you want to take that one.

Yeah. a good question, I mean, and I think I'm gonna lean on Rob a little on this. He can give some anecdotal data on this, but fundamentally I'll say it this way. As I said, FDA expects you to give them a reasonable assuredness of cyber security. So you have to convince them that you're comfortable. You've achieved that level. You you certainly would do your risk analysis, and you would come out with with low scores. That's final. A lot of devices can do that. Now you're now playing. This is the gamble a little bit. You you can maybe convince them, or at least offer to convince them that you don't want to do any testing.

And in your mind you can maybe justify that because the threats are low, and then you're playing the gamble of will they accept that? Or will they just basically, will you waste 60 days? And then coming back and saying, No, you know what we expect you to do some testing. I tend to believe it's probably a lot easier just to do a little bit, at least a minimal amount of testing to show that you have tested and that you haven't found anything obvious in your testing. Then sort of run that risk of submitting and getting a rejection because you've just now extended the time cycle of everything. So I don't know. You know. I'll offer interrupt.

Just before, Rob. I'll just add to that, too, that I think part of what happens during fuzz and pen testing is that you'll discover new vulnerabilities. So just because you've kind of done the exercise of evaluating the system and looking at your S bomb, you know, in your architecture, and done your threat modeling, and you've discovered some vulnerabilities through that path. Part of the purpose of doing your fuzz and pen testing is to see if there's other vulnerabilities that you didn't think of. So I yeah, I tend to agree with Muhammad. It's a fairly risky approach to try to get away without doing fuzz and pen testing. I don't know, Rob, have you had that experience yet of where companies have tried that? And.

It usually goes one step before that. It's they try to say why they don't need to do any cyber security, because it's not a cyber device.

Hmm.

They have a small window. It's sort of like threading a needle of showing that this device is not a cyber device, but if it is a mobile device or a web app. It's a cyber device, and it's not optional. I have 0 experience of a company that was successful convincing the FDA. They don't have to do some kind of cybersecurity testing how extensive the testing is another matter. But I would also say, I've seen 0 devices that had no vulnerabilities found at final test. which means they all had deficiencies. They had to go back and fix and had to retest. So if you have 0 cleanly through the testing on the 1st round.

Then the FDA expects the same thing like, where is your list of vulnerabilities that were found? And how did you fix them? And when you say we had none, they don't believe you. It's just. It's not believable at all.

Yeah. So there you go. So a guy who probably submits over a hundred submissions a year is telling you he's not seeing FDA come back and and you get away with it. So I would agree. I mean, I think you you kind of just need to plan for it. By the way. Just a quick plug. So Mohammed and I are writing a full ebook on cybersecurity. If you're interested in a kind of an advanced copy, just put ebook in the in the chat, and we'll send a copy over to you. So we'll go into all sorts of details on how you want to do that. But there, there's another cyber security question here that came up. It says, can you explain threat methodology like stride and

attack trees, kill chain, dread. Maybe, Mohammed, if you can give a quick, I know it's hard to address all these things in a quick.

And we. And yeah, and we're trying to stay a bit of a high level. So I mean, you sort of, I'll start with the 1st 2, so both both stride and attractions, I would probably say, is, are good to approaches, to use, to identify their own methodologies in terms of how you find threats. So you're doing threat modeling. You've done some data flow diagrams, for instance. And now, so you have an understanding of your system of how information is flowing through it. And now you're looking at. Okay, well, where are? Where can spoofing attacks happen? Where can? Where can tampering attacks happen? You know things like, where you know, where, where will attack source, where can attack

services be, and what types of attacks can come in those points. Other types of attack. Now, there's other methodologies. You know. Kill chain is more around if I you know, if I found a threat. And now I want to trace it back, to try to determine like, how does it originate, or where did you know what other aspects of that attack are also important to consider you can approach it that way. So there, there are a variety of different techniques you can use to perform analysis on threats. but I think I would probably say in your projects right now, you're probably more focused on threat discovery

than necessarily threat analysis in the immediate case. But you can go. There are lots of approaches you can use. And experience is a big one, too. I mean, I think that that's the more devices you do, the more systems you create the more awareness you will have of the types of attacks, the types of threats that exist in systems. And that's very valuable also.

And some of these models are meant to help guide the process. Right, like stride, just gives you like a variety of, you know, essentially 6 different types of attacks that you might look at like spoofing, you know, or or tampering, or that sort of thing. But at the end of the day. That's where it's really helpful to have somebody who's got experience and can think through like, well, wait a minute. What types of spoofing could happen, and where in the device and the architecture could that happen? And where you know? What if? What if there's a 3rd party component that you're pulling in? So yeah, lots lots that we can discuss.

On those, though I will mention just a quick thing is that I think it's a mistake to try to remove threats too quickly. I I've seen that that approach of you know, occasionally done where people will look at things and start discounting threats and say, Well, you know, this can't happen. This is too difficult to happen, etc. I think that's that's sort of defeating the purpose. I think you're you're better served. Identifying as as many possible threats as you can. documenting them, justifying why they score low and shouldn't be considered further. But don't try to just wave your hand and sort of ignore threats, because I think that all it takes is for FDA to review it and say, Okay, well, we just thought of one threat is not listed, and they'll stop your submission. So you know you're again running the risk. So I tend to say, be more comprehensive.

you know. Show show your sort of show your paperwork in terms of what you've done. Give your justifications, and then move on.

Great Casey, real quick. So just looking at another question here, like, what are some of the mistakes that are surprisingly common when integrating software into a device. And maybe you can give an example of something you've seen.

So I'll go go to low power on this one, because, again, short of the price products we work on need to be low power for one reason or another. It's common that people go and develop their algorithm or whatever it is there that makes your product special on windows or in python, or something like that, and then they just try to drop it into an embedded platform. and they're shocked that their battery life's terrible like. Well, I don't understand. Let's see what's the deal. As I said before, in my kind of intro, that low power is such a state of mind. There are. There are lots of ways to accomplish the same thing.

We've for example, we had a cardiac product. That customer had written a fairly complicated algorithm in in C in windows, and they tried to port it directly into their embedded processor. And they had, like a really high powered processor, because they needed to run this complicated algorithm and then they had terribly high power consumption. They handed it to us. We were able to take their same algorithm, write it differently in the embedded platform and cut the power consumption by a factor of 5. Now that was still too high for their product. So in the end, over the course of the next many months we approached the problem they were trying to solve completely, differently, with an embedded state of state of mind, and we were able to do the same thing with 1 20th of the original power.

But it. It required that we we understand the high level requirements for what they need to accomplish and approach it completely differently. They were doing very complex linear algebra, matrix manipulation stuff which makes no sense in embedded processor, and we approached it very differently, modeled it in Matlab, showed them that it would work, and then implemented it in the embedded world, and got much lower power. But not everybody knows how to do that.

Yeah. And and you know, really, there just aren't any shortcuts right? Cause you, you try to. It's and you're either gonna pay the price in power, or you might even pay the price in cyber security. Like, you know, people try to use an operating system as all these libraries and their software bill materials ends up, being.

Oh, yeah, you'll you'll pay it in both. And you do need to. And you to point everybody else making. You've got to think about security very early on, especially as it relates to complexity. And yeah, power consumption. We, we tend to minimize how much you know soup we use. We have our own operating system. That's ultra low powered. It's very simple and straightforward that we've used many times in the past. So we know we know what it means for cybersecurity and for power, and for 63, or 4 in general. So yeah, thinking about all that early on matters.

Yeah. Now you hit on a word that I know. I hear Muhammad talk all the time about, which is simplicity, right? Like there's almost like an elegance like if you can take a system and figure out what's the simplest way to achieve what you're trying to achieve rather than throwing like. You know, the kitchen sink at it. You're going to have benefits in everything. Power, security, simplicity, documentation, etc. Hey, Rob, we've got a question here. That I think you the best suited to answer, says, is a device that inspects closed packages in a Pharma manufacturing line, considered as a medical device, would like to understand if I also need FDA clearance.

Okay? So the answer, as I understand the question or the description. No, it's not a medical device. The the basic questions you ask are? Does it diagnose an illness? Does it treat an illness? Does it monitor an illness? If it doesn't do one of those 3 things? It's not a device. So when we we were talking earlier today with a client, they were talking about an exercise software that was used for rehabilitation. You're rehabilitating an injury. You're monitoring the rehabilitation. You're treating a patient. You're diagnosing what the injury is. And you're tracking the treatment. So it's doing all 3.

So it's definitely a device. But this case. it falls into software tools used in manufacturing. So they still require software validation. But it's software tool validation, not software device validation. And it doesn't require a 510 K. It's part of the quality system for a drug manufacturer. It would be Gamp 5 that they would be normally talking about, which is, I think, 5 times as long as the requirements for how to validate software. Tools for medical device. That's a risk-based approach. Camp 5 is more paperwork by a factor of 5, the standards 5 times thicker. It's a 400 page document. Yeah, just the standard.

Good news, bad news, good news. You don't need a 5, 10 K. Bad news. You need something.

This is your. You need a handful of procedures to understand this.

Yeah. Yeah. So I mean to. To the person who asked that question. I would say, call Rob, because I mean, even if you don't need a 5, 10 KI mean. And presumably, if this is something that you're gonna be selling to pharma companies.

But I I actually used to work in biotech and Pharma. And that's where a lot of my software experience came from. Because we were working on Icon based control systems for bioreactors that were 8 stories high. So that's where you know, mine was real old school PC, Plc, controllers and and Icon based control systems. And we were. I was monitoring the bioreactor from home on on my laptop. Now I can do it on my phone. But that just changes what the devices. It doesn't change what I'm doing. I'm still monitoring it from home.

Yeah, that's great.

Jose, I saw someone ask a question. I just want to make a quick comment. I keep. I keep throwing around the terms low power. And someone asked, What what do I mean by low power? In the case of implantable devices? We're talking about micro watts of power. So it's like really, really small, even for wearables. We're in the Microwatts, maybe milliwatts. So just put that with some scale to that correct me, I hadn't really defined that terms. This one asked.

No, that's great. Yeah. I mean in, particularly with implants where you want to get 10 years of life of the product. That's where you're yeah. It's it's really key.

There was also a question, Jose and I, and I've seen this before, and I answered it. I'm not sure it went to everybody or not, but those that have worked in medical devices that are familiar with things like the Fmas, and the probabilistic way of assessing risks that have been used in functional development in the past, would be a little surprised by sort of FDA's stance on risk analysis and scoring for cybersecurity threats where they specifically do not want probabilistic techniques to be used. And so I pointed out that ultimately you need both. You need a methodology for identifying threats. But you also need a methodology for assessing those threats, the risks of those threats.

And you want to use something that's non probabilistic. So Cbss is an example of that. So I just want to point that out.

Great. Thank you, Mohammed. And you know there was another question. There's a lot of questions on cybersecurity here, but one of them that came up. Maybe, maybe, Rob, you've seen some of the. Some of this is like, what are the types of deficiencies that you're that you're seeing in AI hold questions that are related to cybersecurity. But I mean, throw in there the software ones, too. What are some of the common deficiencies that you're seeing on both topics?

Well, we had a client that went through 2 technical holds, and now they're on AI hold. All 3 had the same issue, and it was the way the company wanted to do updates. They wanted it to be compliant with the oldest supported version of the operating system. So we don't. We don't still use windows 8. But if Windows 8 was still supported they'd want to. They'd want to be compliant with windows 8. And the FDA's policy is, if that you've reached the end of support date, it will be put on technical hold right away. They won't even review your submission.

So that's how to get on hold within 14 days. If you're if that's your goal. They did it twice. because both times their way of explaining how they were going to update operating systems wasn't consistent with the way the FDA wanted to do it. And when we finally had a submission issue review meeting with the FDA 60 days into the review, and we had our AI Hold letter that wasn't 48 items long. It wasn't anywhere near that long. The FDA said. we understand how you want to do your operating system, and we have no problems with the. Your plan to update our problem is that we can't clear something that's already expired.

So if your operating system is already past the end of support date, it's too old. You have to give the FDA something to work with that it won't expire within a few days of clearance. So if if you're working on windows 10, you gotta upgrade, you gotta get the at least a windows. 11. And by the time you watch this video it might be windows 12.

Makes sense that makes sense. And how about on the software side? What kind of deficiencies like, what are some of the common.

So I think that's a great one, the the one it wasn't that client. Only almost all of our clients ask the simple question, upfront. What is our classification of our software? Is it A, B or C, or is it basic or enhanced documentation? And I think you answered this question better than anybody else. I've heard answer this. It's not really the right question they're asking. What they should be asking is, how can I take my architecture and have as little of that B class C as possible and as much of it as possible. Be non-device function. So I minimize my documentation so as much as possible. It's non-device. Some of it's class A, a little bit of it's Class B, and ideally, none of it's Class C,

and if you're trying to design a defibrillator, you know, that's not possible. But it depends on what you're designing. And so you have to think about it. If your algorithm is a real time monitoring device for Life threatening arrhythmias. You're gonna have a class. C algorithm, there's no question about it. But you can save yourself a whole lot of time and money by having as little of the software in that category as possible, and that's done at the architecture level. So when you provide an architecture diagram, it's not just a diagram. It's the explanation that explains how you designed it for security and how you designed the classification of the different modules. So the minimum amount of rigor was required

for each of the modules and the part that is the core. The algorithm to do the analysis of that arrhythmia that is very secure and has lots of documentation. But it's just that little piece.

Yeah, no, that that's great. And I know Mohammed as as a as a software architect with many years of experience, makes this point quite a bit, too. And really, from a software architecture standpoint for medical devices, there's almost like 2 types of 2 types of categories. You should think through one of them is what part of the software is medical versus other functions that's non-medical. And then within the stuff that is medical. What portions of it are class A, class B class C, and and yeah, the goal should be to try to minimize that that footprint of the stuff that's class C, but you know, Mohammed, maybe you can speak to that a little bit more, you know, both from

general security, standpoint and risk, and also from a cybersecurity standpoint. What are some of the things you can do at the architectural level to reduce the threat, the threats that are that are possible, and and even from just a general risk level. And even from a documentation level.

Yeah. And and I think this speaks to this of the broader question of the interaction between just general software architecture and then security. And I think that sometimes people look at these 2 things as sort of separate things. But really they're not you know, a good architecture is one in software, architecture or device. Architecture is one that incorporates all of these different viewpoints together sort of harmoniously. And it's so. So I would say that if you tend to approach this a system overall by and narrowing it down, so the parts of it that are critical are very are very concentrated, and, you know coalesce you, you create a very small security boundary that you have to protect effectively.

and and you make it much more robust. And so this analysis, where you look at your overall system and start to partition functionality and FDA use the term device functionality. So medical device functionality. So if you, if you are able to migrate and distribute the or say, redistribute your system so that your medical device functions are small and and well located and protected. and then things that are outside of them or don't affect the behavior of those medical device functions. You start to create more functionally robust systems as well as systems that are easier to secure. And so so these go hand in hand. And when we look at the analysis of your architecture, you really should be looking at all these facets. It's not one or the other.

Yeah, that's a great point, I mean, and it's gonna save you to Rob's earlier point. It's gonna save you money in the long run, because if you if you architect it, and then you make a small change to a portion of the software that's maybe not even part of the medical device software. Then a letter to file is fine. Maybe you don't even need that. But you know, if your whole thing is Class C, because you architected in such a way. Where there's interdependencies between all the different components, you can't make any changes without having to go through full regression and testing. So that's yeah. Those are great points also important to why you want to make sure you're looking at this early, you know, to Casey's earlier point, you know, if you design this kind of haphazardly, because you're like, it's just a consumer thing.

And then later, you want to turn it into a medical device. But you didn't architect it right? You're you're in for a lot of pain.

And then the concept, Jose, of what is a medical device you have to kind of. Think of carefully, because I know that the term is used that way, and maybe historically, in the past. A medical device was this little box that you had? That actually is the medical device. But Rob mentioned, in case he mentioned it, that now medical devices are these interconnected systems. So so a medical device could be a little physical box with an application running on a phone with a back end cloud interacting with potentially other devices. And so it's like, where really is the medical device, you know, do, and if you can, if you can remove the medical device functions from your app, and your app is no longer part of your medical device that can save you a lot. Conversely, if you don't do it carefully, and you spread your medical device functions all of a sudden. You're doing this admission. That's a device, an app, all of your cloud software and everything else. And you've got this

huge surface that you now have to protect. So again, it all ties together.

Yeah, and I know here we're running out of time. And I don't know, Sean, if there was another poll you wanted to throw up there. So we can kind of have that up there, as we're as we start to close things up. But there's a couple of questions that are all sort of interrelated around the classification. And I'll give a quick response. If any component of your software is class C, then your software as a whole is Class C, in the sense that you know when you go to file, you're going to need to do the enhanced instead of the basic documentation. Correct me if I'm wrong, Rob, I'm pretty sure that's right. But.

for example, when you do your software design specification document. the details are, you know much more rigorous in your class C, and then less in your B, and you know, technically not not required for your A components. So that's that's where really it's going to help you. Do you see, Rob, have you seen? I know you've been involved in a lot of audits, and in fact, I want to give you a chance to plug this, because I know you have a question about audits for software as a medical device. But yeah. maybe give that plug first, st and then maybe speak to when you've been involved in audits around software, if, like that sort of thing around the architecture or the design description, and that sort of thing comes up.

Yeah. So what I was going to plug is that for a very long time I've wanted to teach a lead auditor course on software as a medical device because they teach you how to audit medical device companies, and they use examples of sterile products. Sometimes they use examples of a let of software in a medical device like a a glucose reader. But those have. Those are physical devices in half of our submissions end up being software as a medical device. So when half of your submissions or software as a medical device, but there's no training on how to audit those companies. They don't have purchasing traditionally, because there's nothing. They're not placing purchase orders for pallets or boxes of anything they're not doing, receiving inspection of anything physical

they don't have a manufacturing production floor. They don't have any controlled environment for sterility. All these things that you normally look at as an auditor just don't apply. So how do you do an audit? Well, you have to know something about software development in order to ask the right questions, to understand a little bit about how to audit software as a medical device. And I want to teach that. So the question is, anybody that's interested in that. It's something I I can teach, and I do it all the time. But the question is, does anybody want to sign up for it? And so, if you don't want to sign up for it. Say, no, this is a waste of my time. And if you, if you really are interested, like, sign me up now and on our website we have, we have contact us page, if you want to contact us. But you can put right in the chat just yes, I want to sign up for this.

But what we look for when we audit is traceability is probably the most valuable concept to learn. When you look at a traceability matrix for a company. If you can't see hazards in software design specifications, in testing requirements in the document where the test requirements have the results. you don't have traceability, and one of those common AI hold items is lack of traceability. Well, if you had somebody that was auditing you for that particular requirement, they would show you. I can't trace from your risk analysis of software to your documentation.

So you have lack of traceability, and you wouldn't even get to a submission before we fix that.

Yeah, that's great. Because I know a lot of people who are on this webinar are probably in that Qa internal Qa, or maybe they're doing independent audits, and if they haven't trained on this, it sounds like there's interest. But just right on the chat. Just put audit webinar or audit course, just anything around audit, and we'll know, and we'll make sure. Rob, gets that feedback and we're gonna close it out because we're at the top of the hour, and I know some of you maybe have appointments. Next. as you come out of this, you're gonna get like a pop up questionnaire. Please fill that out. Included in that will be whether you want to have a meeting with any of us, with Casey, with Rob, with Mohammed, with myself, you know, would be all of us would be more than happy to jump on a 1 on one call with you. You can check out each of our websites. I know, Casey, yours is nocturnalpd.com right.

Correct.

And Rob, you've got a couple. So what's the main one?

Medicaldevice academy.com.

Medicaldevice academy.com for us. It's Cybermedai. It's been mentioned a couple of times. We do both software and cybersecurity. So on the software side. It's boldtype.com. And then on the cybersecurity. It's Cybermedai. We can help on either end. But with that I want to thank all 3 of you. Excellent conversation. I learned a lot hopefully. The folks on the webinar learned a lot. And, Sean any last announcements before you jump off.

No, I'd just like to thank everybody for attending. Please complete the survey. It'll pop up on your screen automatically if you're asking to be contacted by somebody. It would be helpful if we knew who it was. Feel free to put that in the comments. And with that I'd like to thank gentlemen, for wonderful presentations. I'd like to thank the audience for giving up an hour of their time, and everybody who has submitted comments requesting the ebook requesting the audit course. We will follow up with you on that in in due course, along certificates everything. If you have any questions, email me at sean@virtubtob.com.

That's hard to write, anyway. Or contact me on Linkedin. You know where to find me. Gentlemen. thank you.

Thank you.

Thank you. Sean.

Take care bye, bye.

Bye, bye.

Everyone.