How to AI ML FDA Clearance in 20 minutes feat. Yujan Shrestha

Hey! There!

Hey Sean!

How are you?

Going? Great! How are you?

Good good to see you.

Likewise.

You all ready for the holidays.

For yeah, for break, it'd be great. How about you.

Oh, I'm gonna be working.

I'm good. I will be working as well. but it's good for a break from everything else. Yeah.

Well, it's yeah. It's good that everyone else isn't. You know, calling, you know that that you get a break from the

Yeah.

Thanks, Sean. I couldn't find the switches to do muting and all, and whatever this is called.

Yes.

Video.

Can you hear me? I'm I just trying to promote everybody to panelists. Sorry, I'm yeah, yeah.

Welcome to. She's like, Yeah, don't worry. It's a bit dark.

How y'all doing.

Good morning!

Good morning!

Morning! Hey! Michelle!

I am not home for you. Hmm! Nice to see you.

Yeah.

Hey? Sean.

Look it up, drink.

Sean.

Yes.

I'll have you lead today. I'll be on and off.

Okay.

Thank you, hey? I had a thought. Folks, I I've been saying how Lucas's 21st birthday would be the one day sale for 10 x, and that's still true. I realized that some of you may want to tuck the expense in this year versus next. So what you can do is write me a check and send it to me, you know, electronically or over mail. so that you send it in 24 and I cash it in 25. So because I'm on my phone. I can't see everyone at once. Oh, actually, I think I can. Yeah, Andre, come on

Does anyone want to do that? You can? If you want, I'll put it in the our weekly call thing, too. If anyone hey, Nikki.

Hi! Good morning!

Hi! Good to see you, mark.

Hey! Everybody.

Mark wrote me and said, he has some clean spaces. News! What is it?

So as a lot of you know, we've been developing this environmental controls app that takes into account all the data that needs to be put together for collecting out environmental controls for medical device, pharma, biotech things like that. So just recently, we've been thinking of creative ways to try and come up with what we need to finish off our development costs. We have to trip across a company who is a cleaning company. That cleans medical device, clean rooms or pharmacy, clean rooms. And they said that their customers came to them and said, We need electronic version of this. So during during a place in Alachua, Florida, just north of Gainesville. They have a big biotech get together every year for the the companies there. This company approached us and said, Hey, we've we've got hundreds of customers that want us to digitize all this stuff right now, can we do a deal with you? So instead of us doing it, we're thinking about doing it. But since you guys are like 85, 90% of the way there, maybe we should sit down and talk to you. So we've sat down and talked with their principals.

and we still have a another couple of rounds of talks, because they need to know who we are. They need to, you know, have confidence in us and things. Then we're you're you're on. You're on mute, Joe.

I excitedly said, you're gonna be rich, is what you're telling us.

So here's what we're thinking. We need to get a hundred $50,000 somehow, some way to finish off the development of this. So we can finish it off a hundred percent. Get it all back point over. There's a 1.0 version is exactly what they're wanting, because that's all they wanted to develop for their customers. As soon as we get that done we can step into a market immediately with hundreds of customers who have already told us they want this. So what we're thinking is the Jeff Bezos model of. When he 1st started Amazon he went out and talked to hundreds of people. He got 20, some people at $50,000 each. He raised a billion dollars to get started. So we're saying, if there's anybody who has an appetite for something like this.

We're looking for 3 investors, 50 k. Each that can get us the money. We need to finish off the development and get the legal, all the IP all wrapped around for all the countries we're gonna go into. And then, as soon as that done. we could do that sometime in January, by the end of the 3rd quarter everything should be done. We're ready to commercialize. We'll be in business before the 4th quarter of next year, and we'll start realizing revenues almost immediately. And this is going to be a subscription base. So we'll we'll be generating revenues every month from all the customers that are there. So we.

That's very exciting. Do you have any projections and numbers and pro formas and things.

Got all of that my CEO has been on that. He's been readjusting the numbers based on this new information to see exactly where we're going with this. But we're thinking, if we start with this cleaning company and have this many customers, we have hundreds of customers we're talking about. one to 2 million dollars in revenues just the 1st year, and if we don't stop there we look at other cleaning companies that are doing the same thing getting directly to the the companies that need this, we can do the same thing there. We can just replicate this across all the companies that are already working with the clean rooms need.

Then we have a ready made market. So again, we're looking for 3 people, $50,000 each, so that we can get the money. We need to finish this up, and if we can finish this up, get in January. We can finish up by 3rd quarter we'll be revenue ready. We will be commercial going out with the commercial version of this by the beginning of the 4th quarter of 25.

That's great news. Go ahead and put whatever you have in slack in the what do you need? Channel?

Okay, will do.

Okay, great good luck. We have a few faces on here. I rarely see Ms. Rodriguez. It's good to see you always. and a couple of bases. I don't know very well, Cooper, I don't know that we've ever spoken before, have we?

Haven't quite been on one of these before, but, you know, saw that it was an exciting topic, and figured I wanted to join, so happy to be on.

Well, what do you do? Tell us about you?

I'm a quality regulatory lead at a Startup Med Device Company in Philadelphia. We are in the neuro space, and AI is of particular interest to me and my company, and so I figured that I'd hop on today.

Okay, welcome. George. I certainly recognize your last name. I don't know that you and I have ever met. However.

Yeah, I am the notorious George Duval.

Known and associates.

And associates right

How many associates do you have?

I have. I have none right now. I'm growing my side hustle because my front hustle actually went away. And I'm actually really getting involved with growing consultancy and advocacy towards applied sleep therapy for cognitive decline and aging adults. I'm trying to.

Interesting.

Gaining some momentum in the sleep band arena. And with what little funding I don't have, I mean have. I'm trying to get things started.

Are you not working with mark anymore.

I'm a free agent right now, and hoping to land somewhere in January.

Okay, There are a few folks in the group who are working on sleep related things. I'm working on keeping my cpap on for more than 4 HA night. But let's let's talk later. Sure. At this point, Shawn, let me have you lead. I'm going to put myself on mute and off camera. And you, John, me, Joe, good to see you and thank you for doing this. You are backed by popular demand. Tell us what you did.

Yeah. So today, we're just gonna we're just gonna continue the conversation from last week, and during the demo, I'm also gonna go into just kind of some more details about some highly opinionated ideas of how to get aiml. Cmd. FDA cleared and also given the nature of this of this group, I'd be very happy to entertain questions and panel discussions during that presentation as well. I thoroughly enjoyed last time so hopefully. This will spur some more conversations as well. so yeah, let me know what you know. Should I go and get started, John, or any anything else from the house?

So those of you who missed the previous session. With you, John? You know, they're doing some really innovative stuff. With automating the the 5 10 k process and he was gonna demo the thing and then we then we sort of ran out of time. Is that accurate.

Yeah.

So I'm gonna let him talk and shut up.

Let me share my screen. alright! Let me start that. There can everyone see my screen? And can everyone see my mouse cursor.

Yes.

I've got my eyes and ears alright cool. So from the audience I would like to give, or like to ask you for a favor you could. Please give me your favorite SA. Md. Device that's already in the market. I'm going to pull the 5 10 K. Summary of that device. And we're going to put it into this AI agent and watch it come up with a a proposal list of requirements and user needs. So anyone from the device or from the audience. Give me something you've got that's your what's your favorite SAMD.

Hmm.

Sleep. Monitoring is popular in this group.

Alright. And I know one just released by apple. I believe I think that was a sleep monitor. Is that right? Yeah, yeah, it is. So let's take their 5, 10 k. Summary and I'm just using the 5 10 case summary, because this is a publicly available document. However, it's more likely you'll be using something like code, or or like internal documents that have already been written for the context that you would give to to an AI like this. And I can prompt it with some specific actions that I wanted to do. or we've got this feature called. I'm feeling lucky which will go through and execute this pre written set of commands. These script.

That that will ask it to create a bunch of requirements that that kind of follow my own personal style. So this is really like it. It helps me perform this specific task. and I prompted it in a way that kind of captures my design philosophy. so this is. This is a tool that could be used could be used internally as part of your service delivery. It's not something that we like are making really publicly available. Yet this is still like really much like an internal tech demo. But just to kind of get you guys an idea of where I think. generative AI tools in like the regulatory intelligence space could be headed

where you have context that you give a tool which would include all of your documentation which would include FDA guidance documents could also include code and then you you prompt this knowledge base with a set of Llm. Prompts. and then those Llm. Prompts can come back with actions that your agent can then perform. So in this, in this instance, the AI agent behind the scenes is adding requirements to this, to this database that we have here on on the right? Based on the prompts that we give it. So yeah, this is what I was demoing last time I was here. We didn't have time to really talk about like any of the details or any questions, or you know any any ideas

about about this. So I'd be happy to entertain some questions, and you know any, any, any thoughts on this so far? And then, after we kind of talk about this, I'll go into the other topic about some more opinions about how to get Semd. FDA cleared so I'll open up the floor for for anything.

This is awesome. I I see you know, in in the in my last corporate role we were looking at having a software cluster bundled together for a wearable application we were that they were trying to get through. FDA. Had we had this. had we had this tool that would have saved months on the the filing and application, because I think this would have been a good precursor to understanding pathway. So yeah, somebody somebody pointed out this, this could be sold right now.

No, that was me. That was me. I can tell you right now that that I the reason I put together the software company to do what we're doing with the environmental controls, because there's nothing out there. And it's needed really badly. If you're already doing something from a commercial venture. It's just like when they were putting out the Esg. It was a cluster for the 1st few years of trying to get drugs through the Esg portal, until finally I had to go through 3rd parties to get all that validated. They created their own validations to get things through that software. If you have a way of apiing into and getting the FDA to say, we will accept this information.

You get a deal with them to say, if you do this. This is not like quickbooks before report, you know, for for an audit report this is needed right now. I was thinking about doing the exact same thing. You just beat me to it, but that would have been on my list of things to do. Have that part of my company. So you have an excellent idea that is commercially needed right right now.

And I'm and I'm going into different directions. That was one of the ideas I had put on my on my shelf that I'm gonna I need to sit there and use open AI to kind of automate this process. So this is, thank you.

Yeah. And and really like this, this concept is a lot more general than than just the use case that I had. I just demoed. And really, it's like a couple of key components. I think the most important one is context. So context, it's kind of like short term working memory, and and sorry to kind of sorry to kind of antomorphize, but like it's kind of the easiest way to explain some of these concepts, even though it's not exactly the way that these Lms really, you know, quote, think. but the the context gives you what what you would have loaded into your short term memory in order to make make some sort of action

right? So for for, like this could be 5, 10 k. Summary, which is currently in the context right now. it's also the state. It's also all existing requirement which gets which gets updated dynamically. And you could, you could imagine that you would add all sorts of things to this context. Right? FDA guidances. You know, probably, like the Tplc Tplc and all sorts of things, and in this context goes to a prompt that you generate and the 2 combined.

We'll then go to this Llm. And this Llm. Could be one of many things, and he, here's the other beauty about a system like this. as the Llms. Get better like you go from Chat Gpt. 3 to chat, gpt 4, or Gpt. 5 or whatever. It's essentially a drop in replacement so pretty much overnight. Whenever a new Llm version comes out. this entire system gets smarter and smarter. And then this, this Lm, then generate set of actions. And then via, like Apis, you kinda take these actions into like. you can even do like Google deep in our case. We use notion a lot.

But and also, I think, Mark, you'd you'd mentioned an Fdi FDA. Api.

Yeah, the yeah, these are all actions that you could program into the prompt.

And say, like, you know, please give me back a set of actions that I can execute. based on this context that I give you, and based on this prompt that I've designed and then, once these actions confirm execute. you essentially can repeat the loop. Don't turn it into this like chain of chain of actions. Basically. So I think this this overall pattern is what's being coined as like the the agentic workflow where you have Llms that can that have tools that they can use to not only generate Llm. Responses for you, but also act upon

the data model via Api calls like this. So this is like the agentic workflow. And you, you see a lot of these coming out where they kind of do this research or do it like like in this case here, like a 1st pass through all the requirements and user needs and other documentation artifacts. But yeah, that's that's basically the pattern that that is used here. And it's quite general. If you if you think about it in this way.

Can I? Can I throw something in real fast.

Yeah.

One of the things that would be immensely needed is, if you had a tie into all the different literature searches where you can actually check different databases and pull any type of predicate devices. Anything that's going on. The MoD. Anything in the Mdr. Anything, any problems out there, and any kind of literature searches. If you had that already pre-programmed in with some sort of filter to look for certain keywords, and then you're documenting those keywords because that is also what goes in your report. Here are the keywords that we use to do the literary search. If you're doing that from one place in more like a wizard format, like what you've got right here, and it automatically pings. And here are all the places we go to, and it's all the stuff everybody does. Anyway, if it has, AI is beautiful for something like this, because it can go through and rip through like a spider through the entirety of the Internet and find stuff like this. That would be an immense help

to regulatory and quality. People who have to go and do these kind of searches. If your AI can do it, it's worth its weight in gold right now.

I want to use it for teaching, because this is. it's because my students they see. Well, AI is here like we want to learn the latest thing I teach design control. And so once I teach the basics of design control. It's like, why should they go through manually, and the the only thing is is that you need to have a good amount of knowledge to understand like, okay, does this really apply to my device? And also in terms of the requirements. you need to make sure you don't have conflicting requirements and things like that. So, but I would think, like you could do with requirements. You could say, All right. Now, please apply the 41 rules of systems engineering to make sure. Because, I use software from Qra Corporation right now for that.

So I'm just thinking that just another step further to make people's lives early, easier on things that are so administrative. That's what people need to be able to learn to do to get medical devices, approved Quicker. You know, I mean, the testing is like really important. But things like this, data searching. This is what AI is perfect for, rather than just manually googling and searching and.

100% agree 100% agree.

Definitely and and and I think I think we're not you. You touched on several important points. There is that this AI is a tool. It's not going to replace any any human expertise. In fact, it's only it's only here to augment and gap. Fill a little bit of your knowledge, but not extend past your your current level of comfort or your expertise. Like I've I found that like when using AI to try to convert someone into an expert. It's it's usually doesn't work. But usually it's use useful to amplify an expert to really to really amplify their their their skills in that so? Certainly something like

you know. Please apply 41 rules. The the person using this prompt this tool should be, should know how to do that manually themselves, so they can evaluate the output. This AI to make sure that it's not hallucinating, or make sure that it's operating as intended. but it can't be used by someone who doesn't know how to do the underlying prompt themselves. That's kind of my. that's been my experience so far with the current generation of AI.

Well, I guess you're really trying to.

Through for a while.

I actually have an example where there's somebody I don't think knows what they're doing, and so they'll come in and say, Well, this is an requirement. This isn't a requirement, and I'm knowing it is. I've gone into AI. And I said. Please give me language to explain why this is important. Why this is required, because I know it. And then, when I read the language, I can validate its accuracy, whereas if I'm somebody who doesn't know what I'm looking for, and I get something. I don't know if it's accurate or not. I think that's really what you're trying to say. It's like you've got to have the knowledge to know what you're reading, that you're getting output.

Yeah, does it make sense? Yes, exactly. The other thing is that AI is great, for is coaching. So with which is why it's really nice for students. Is that I say, all right, you're learning this. So if you're wondering if this is an appropriate, you know, you can ask like, is there something missing is? If I if my product has this other feature, you know, would I need another, you know. So the coaching is what's really helpful. The other one don't forget other than the 41 rules. You might also be able to apply ears, which is the easy approach to requirements, syntax, and so it could also then check and then suggest appropriate syntax in the ears. Format requirements.

I've actually been using it for labeling, review and verification. Right now I literally had it. Help me write up. I was teaching somebody how to do labeling specifications yesterday, and sometimes, like you get tongue tied. You're you're explaining it. And I said, so I went to AI, and I said, okay, this is what I'm trying to say, you know.

7.

Make it simpler, and it outputted the entire specs and explained it. Why? And I was able to walk the person through it in 5 min versus it would have taken me hours to make a presentation to explain exactly why we were doing what we were doing.

The other thing is like it would be good, is like, I wonder if it can like? Okay, here's all your requirements. But of course, these also all have to. Software is only slightly different. But I mean, still, with all requirements, they need to trace back to user needs. So it'd be good to say, like, all right, if these are all the requirements. I mean, it's such. It's so like tedious to go back and trace to user requirements. But a lot of times, that's what you need to do. Because you've you've kind of like, unconsciously been saying, Oh, yeah, this is a user. Need we gotta have it do this and this. But then people as I come into to startups or as I'm teaching, it's like people don't think of in, you know, the proper order of user needs. And then this and then a design requirement. People think all different ways. But then you have to connect them all with the trace matrix. So.

Well, you know what's funny about that is that that's that's the other part of the the software boom that the things that need to be electronically resolved, because you've got so much interconnectivity between the different documents that you have in a design history file, or, what they call a medical device file. Now, in 1345, all these things are interconnected. So you are ultimately this would be a module of a much larger piece of software that would roll through all the requirements, interconnect everything, and make the the whole Dhf. Which is now MDF. Process. One large spider web interconnected, easy to put information in once that's in. It's interconnecting to the different places that it's supposed to interconnect. I'm doing it manually right now on excel spreadsheet. And and Microsoft Project. But this is something I've been wanting to develop for the longest time. So once this company, our company with the environmental controls gets done and we start generating revenues. I want to take on the big task of doing all of this, and maybe including this.

as that literary search module within the entirety of the Dhf, so that everything is connected like you said through a full trace matrix beginning and end inputs outputs, verifications, validations and then transfer to manufacturing.

The one thing I struggle with when I'm having AI compare something, and it's more of my visual issue, because I I have 3 forms of lazy eyes so like, if if it's all black and white text. I can't see the difference. And so it would be really great if AI could actually start doing some sort of, you know, red lines or a yellow highlight. Hey? You have a problem here, because if I get a huge document, I'm I have to sit there and side by side still do the compare so that I'm still going to my Microsoft words and doing. Compare documents to figure out where my gaps are. So that's 1 of the things I find that's a miss in AI is. It's not really good at flagging things to be obvious

e. Either a gap or what you're trying to compare, or differences that kind of thing.

Yeah, normally, what I'm doing it might be might be color code things. So I've got. I've got the, you know. The stoplight approach red is I got critical. Got to jump on. It's behind. Yellow is. I'm I've got some problems. Green is. It's okay. I don't have to worry about that sort of thing. So just from a high, level approach, that's kind of what I'm doing with this and be nice. If AI could do the exact same thing, letting us know where the problem areas are from a project management standpoint.

Exactly. That's that's the gap that I find. And if I you know and what I I think, what you were guys are saying, and I want to just paraphrase, this is like, I could literally have requirements. Upload them into AI and say, Please trace these out, and that's what AI you can. Is that a way you're doing that cause? I know that a lot of people do it manually, or they use the system like matrix to do it manually, and that would be useful.

Yes, thank you.

That that is one that is one action, that that this particular AI agent does have access to. So so.

And.

A tool that it can use to link a specific requirement to a specific username. And I, I found that that is a very good use case for AI right now. It's pretty good at doing like a fuzzy language match. Or it kind of knows that, hey? Most likely this this user need is implemented by these requirements. You obviously still have to check it like you see here that there's some duplicates so like, and there's and there's another action that could go through and deduplicate requirements. But these are like. These are all like ingredients, and like a big recipe, and you kind of have to chain them together.

But yes, linking is a great use case. Also, I think your point about adding comments to like word documents and stuff. I think that's just a matter of adding that as an action to this agent that perhaps you'll have in here, like your Quality Manual could be as part of the contacts, let's say. and you might have a action, or you might have a prompt that says, like, find find gaps. a quality manual or find 1345 gaps in the quality menu.

Yeah. The other thing, I think.

Yeah. And then this, this thing could go through the Lm. And one of the actions would be to flag or to add comments to a word document where it's found. The gap.

Yeah.

So yeah, like the like, I think, like the core, like the core Llms themselves. probably have the capability of doing this gap analysis, but they don't have the tooling to execute and to like add a comment to word. And so that makes it significantly less useful because we also the Llms don't have a good way to to assemble the the context, which is usually a separate tool.

Well, like.

And yeah.

If I've uploaded a document. And I said, Hey, do a grammar check on this. Then I copy and paste it. I'm still not seeing the errors, I still have to do a compare to the original word document, but because you've lost your styles, your layout, it really doesn't. It's not helpful in that way, or I haven't figured out how to make it helpful. Let's put it that way.

Have you tried using the template.

Using the template in the AI or.

Yeah, a lot of times. What I do is, I'll I'll say, using this template, you know. Tell me, give me feedback on this document.

Oh, no, I did not. That's a good idea. Thank you.

Yeah, yeah, that's how you and that's how it's used. Well, 1st of all, you can give it a rule, and then it'll apply the rule for you. So you know but also you can ask it about the template, and like oh, like would you know. make sure that it's you know, more formalized or more professional looking. Or you know, I use it sometimes for emails. When I'm like doing a massive number of emails. I'm thinking, okay, I'm getting frustrated. Make, can you just check the tone of my email to this student? You know.

So many.

And then like, Oh, yeah, that's much more calm down.

And and I have the same affliction with the lazy eye as well as well as adding to that add, So you know I. I find that this tool of AI is very helpful in keeping me focused and catching things that I might miss.

And to be honest with you, if you sit and you design it right, if you have it functionally perfect. the the actual graphics sort of fall into line with being functional. If you can talk to people like ourselves, the people who are out here using it manually right now, what we're using. If you can incorporate the AI that takes away all these manual processes, you have it functionally perfect. It's functionally doing what it's supposed to be doing the the pretty doesn't really worry me too much. I'm looking at functioning, being able to do something that takes me weeks, months. whereas if I can sit down. And I can input all the data. And just as I go it, just input the data, and it's learning, and it's putting things together. And it's building. By the time I get done. If I have put in everything, I should only have one review where I review everything and have to make minimal changes to the functionality of that. 5, 10 k, you have the perfect 5, 10 K AI type, software.

Thanks. Mark. Yeah, certainly. Like, right now, I think the technology like the state of the art right now, it still is much like it needs to be a tool used by experts.

Yeah.

And it needs to be edited like, like each kind of step along the way. there should be a break point and say, okay, well, here's some draft requirements. Now. An expert should go, and like make sure they they look good or edit them. Add more delete more. That's another shortcoming. I think of a lot of AI tools that we that we see like the the chat bots and every website, I almost never use those I find them not that useful. and the the reason is like they don't give you a chance to edit and, like guide the AI.

T.

Down down a certain process. Which is why we we built in, like the ability for the AI to write back to notion which is a which is really easy, like like in notion. It's really easy to change change any of these artifacts generated by AI and via nature of the context. That that change then gets incorporated into subsequent calls to the Llm. So that that allows the the human expert to course, correct. You definitely need that. Otherwise. like, it becomes like a giant Jenga block that you know. Eventually it just it. Just you could let AI go on its own for a little bit, but it's gonna get derailed at some point.

Well, you do. What you'll have is you'll have to have the ability to allow the experts when we get new information in, to be able to make those changes to our initial assumptions and requirements. So there has to be that iterative process of saying, if I have to make a change. Because then I have to go back and make those initial requirement changes. User need changes which AI should pick up to say, Okay, if you make that change. There may be a change to a risk. There may be a change to procedures. There may be a change to documents I need to supply because of that new information. We need the AI, because most people, if you don't have the expertise up here.

if you're if you're not already experienced to know that these things have to happen based on the new information just like we were talking about it being a teaching tool. This, with our help. should be able to pick that up to say, if this user need changes, it changes the risk, it changes documentation. It changes all the things that need to be changed as a result of that initial change at the beginning. And AI, if completely developed, to do that sort of thing like what humans are doing now and taking a long time to do if we could replace that whole human effort with a prescriptive. This is what we normally do. Iterative type approach and smart enough to pick that up and catch it. It helps us as professionals to say, oh.

I completely forgot about having to do this, because we made that change. It is definitely something that we need to support our effort out here in the field.

Yep.

So you, John, is there a way that the people on this call can play with this tool?

Jeez.

That's a great question, and the like. This tool is still very much like an internal tool that we use to help us with our services, which which, by the way, we're a software engineering firm. So so we so we build software medical device and also help our clients get. FDA cleared. and this is like a kind of internal tool that we're currently using. So it's not really easily, publicly available. Although if anyone's interested in potentially funding that. Or, you know, helping us raise some money to turn this into a standalone tool. We'll be happy to explore options. There.

Oh. I think that you could go for you could look for a grant. I mean, this is constantly in terms of getting things through. So I would say, yeah. If you want to chat offline, you should look for an Nih or a grant of some sort. The government, because this would be beneficial, not only for industry, but for the FDA.

They would probably want to help fund it.

That that's that's great to know. Thanks, thanks for.

Absolutely.

Yeah, yeah, cause we're we're trying to solve. I think just breaking this down into how, how we're probably currently using Llms. The 1st step you do is you if you assemble a context right and mine currently, most of these are are being done manually. So then you assemble the contact you formulate. The question to the AI, and then you you then ask the AI or say, Send, you know, send prompt, send prompt plus contact to Llm.

And then you manually create actions. The take, and then you manually execute those actions afterwards. and and for those of you use AI heavily in your work. You probably find yourself doing this a lot right every day. So, and the the the issue currently with a lot of AI is that this is all a manual process. The the context assembly, the

formulating questions, and then taking the actions on the responses, is all a manual process. But what but where? I think the industry should be headed is to turn these into an automatic process, so that this this loop that you're doing goes from taking 10 min per prompt action cycle to taking 1 min or even less by turning some of these manual aspects. or what what were manual aspects, to turning them into something that is automatic. But I think, like fundamentally, this is the. This is the core idea of of what I presented. Turn it something like this to these will be automatic.

That's for anyone.

I was just getting ready.

Creating.

For sure, but 5, 10 K process is prescriptive. Really, if you look at it from a standpoint of here are the requirements we're only talking about. If I were to use this, say fully functional. If I put any class device in and say, I want to know everything I need to know about this, to be able to submit a 510 k. If we've done this a number of times should be far enough to learn to say, here's the thing, have to have this device, this specific device, this class device, and what the FDA is requiring to have an entirety of a 5, 10 k submission. The AI should automatically be able to say, you enter in whatever the device is, what class it is, and then automatically go out ping all those sources bring in predicate devices. Bring in all this stuff that you have to have, anyway, and start filling up things and then produce the Gap assessment to say, here are the other documents, supportive documents you need to have.

Go get those upload those once they're in. AI can generate those links between everything, and should be able to guide the people doing the work. As to what's left to do instead of hey? We got a whole bunch of stuff. We're not really organized, we can't get it together. This could actually, as a teaching tool, direct people to do things in a certain order, put things in line with doing it automatically. But certainly, if I put in some minimal information at the beginning, I would expect AI to come along and say, here's everything you have to have for a 5, 10 K. Submission for that particular device.

Mark you, you hit on something as well the the predicate devices that would be filed with with this, so that so having having the AI come back and have a like a rank percentage of, you know, this predicate versus this predicate versus this predicate that would be most likely comparable that you could use it for, for the submission would be ideal.

All the predicate devices are all on the FDA website. You go and see exactly where all the data is through all that get the language. See how close each of these are, and at some point in time, maybe make a subjective call as to how similar other devices, and get give you that ranking because it's all already set up. We just needed an automatic way of going through all these manual processes to do. They're basically paying us by the hour. For now AI could certainly not really take my job. Now I'm in the role of I'm helping people get started. Put all this together. I'm gonna teach what you need to help you create the documents

so long time before there's a whole bunch of experts pick up, use this, and then they don't need consultants to help them out. This would be an absolute boon for my industry, because I could go out and talk to people and say, Oh, you want to file a 5, 10 K. Guess what we got. You gotta see this.

Yeah, like, basically what you described. Actually. this manually querying the external resources. That's probably one of the most time consuming parts of this.

Absolutely.

This is often not done. This is often not done, which is, I think, why a lot of folks like, whenever you do, have suboptimal results from from using Llms is because the context isn't fully there, and then you're relying on the Llms internal memory, which you never want to rely on that internal memory, because that's that's how Llms hallucinate. But because this this step is so time consuming, we often just skip it right. And we just ask chat to Beto. Just find me a productive device for this. That's almost never gonna work. Because the internal memory is only limited to like September of last year, or whatever the whatever the training frozen time was.

But because this step is so time consuming, I'd say. really, if you did it correctly. This would take you a hundred minutes per iteration, maybe even longer. I don't know but with the with the automatic method that brings it down significantly, because this can be all scripted right.

Looking at a source document that is already pre-format it a certain way. Just that. Much easier to go out and find the documents, parse out the information and put it into a a report that's easy to read in a matrix style, where you can see across predicate devices and do the basic comparisons between them.

Yeah, what the next step would be like would be nice to go from the requirements to comparing design specifications like, obviously, you're just looking at historical stuff here. But then you've got your own design. So then you'd want to see like, where do your design requirements fit into these? I mean, design specifications fit these product requirements. And then I just say, well, then, the next step is then they want. I want to test stack. And I want protocols and.

Because that's take so much time. So much time. That was what I was saying about the whole dhf thing is that you have an entirety of activities from beginning to end that include and include the trace. Matrix includes Gsprs and post-market surveillance and risk management, and all these other documents and design requirements and inputs and outputs and and verifications and validations. You have a trace across all of this. If that were a big program that included this one we're talking about right here as a line item that did the search, you can start at the top and say, I want to build a certain product. Let me put that in. And then, if we had a larger dhf that were fully AI developed to say, Well, you need risk management. Here's what you need to put into place and do. The literature searches

and be able to file. We're saying it's a 5, 10 K. Well, here's all the things you'll need if it populated a an activity list just to say, here are the things you will have to really do. And then, once that's done. Put that in and have the program. Just take over saying, Thank you very much. I'll take it from here. Now you've got list of activities you have to do. It's telling you prescriptively what you have to do. And it's just that much easier for us as experts to focus in on the activities that we have to do, and let this do its job of doing the grunt work that takes us forever to do. This is amazing. You, Jen. I'll tell you right now. This is really I'm I'm drooling here about this.

Too. I have to. What keep wiping them plus. It's not only fun, it's I mean, it's amazing. But it's like fun.

Yeah, yeah, yeah, yeah, yeah.

Because all the stuff that is so like like this is, gonna be cool when it's done. But the process.

Right.

So manual and long. It is like, okay. Our brains could go much faster than this if something else could just do this like this does. That's awesome. The other thing that.

More of like a philosophy.

This is like great wave brainstorming. But the thing that I've been wondering is like, does anybody use triz or have used triz before. I'm not seeing any any. Yeses. No. So if you look up, tris it, yeah. So it's for so basically, it's for I've used it before. In industry. And I've had colleagues, too, for coming up with new inventions. But it's it's very It's it's kind of a counterintuitive, because rather than just sort of out of the box. Brainstorming. It goes and this has been around for a long time. But, like AI, be perfectly applied to this you go, and mine all patents that have ever occurred.

and then what it does is it applies these 40 principles so that you can say, All right? I've got this problem. What? How can I apply an analogous situation that's out there in the world to solve this problem? And usually it's a a type of problem that is like, okay, I need something like, here's kind of, I don't know a random one like, Okay, I need like something that needs to go really, really small, but it's got to create a big effect or something. And then it goes into all these principles. So if AI could be used to help prompt people for designs. This is kind of on the other end of what what your product does. You could help come up with ideas and then see all right. Does this connect to something that's already a 5, 10 k. Or how or other ideas that were out there to help me?

Problem solve and iterate designs.

Yeah, yeah, that's yeah. That's a great idea. and you can kind of see, like, I think a lot of that lot of the work going into that is, contact Assembly. Going out and getting getting a state of the world. filtering it down to what's relevant for that specific problem that you're trying to solve. So kind of like decoupling. getting the data and doing the computation and getting the data is like the most time consuming part of that process.

Yeah.

The the prompt itself is going to be really trivial, like, you're just gonna basically ask it. Just a question.

But.

The majority of the difficulty is the context assembly, and also taking action is pretty easy, too, like you. Once you have the the answer. you can then use that to ask it yet another question, or you know something else. I I think, typically like taking action is actually less work than than the Context Assembly.

So much of what you're doing here is is feasibility you're talking about. If you could do this and find out what's out in the world. You're thinking about going out and and making something, you say. Well, how feasible is it? What else is out there right now. if this were functioning right now, the way I think we're talking about it functioning. I could put in like I said I could put in a device a class name of the device punch a button. Have AI. Go look around the world, find out what's going on. Bring that back in and say, here's how many predicate devices. Here's the way they work. Here's the classes they are. Here's the risk that's going on with them right now. You've done the literature search first.st That tells me whether or not I even want to get in the business of making this device. That's a great asset.

Well, also you could. I mean there must be, are there? I haven't even looked. But I mean, are there products out there? There must be right now that are doing intellectual property and say, All right. Do I have freedom to operate in this space? you know. Is this a design, you know, because then it's that takes a ton of time, or the other things like that. Take a crap load of time when I was an engineer is with devices as opposed to software, with devices to develop a test stack, so that I'm able to build the least amount of product to do all of my tests across. And so you have to think about in a catheter is what comes to mind, because I've done crap load of catheters. It's like, okay.

I could do this test. If I do it in this order, then I don't damage the product. And I can use this other part of it to do this other test later on. But you, it takes a lot of time to like map that out. Figure out, okay, what needs to be sterile device versus non sterile you know. Cut off the catheter tip to do this test. So it seems like that should be something that AI be able to do with a lot less time as long as you input the design. And obviously before that, you got to input all the tests that goes back to the requirements. But if you're able to lay all these things out, it's just a huge time saver.

Then you spend the time in the lab, like actually running the test, gathering the data.

Be careful. That's what we're paying the IP lawyers for, so they might.

They'll use it, though. I mean they I mean, I would get bet there's probably I know there was software out there years ago. But I mean, now, AI is just gonna make it better, because you can just comb stuff. And you know it's a little bit better than say regulatory stuff, because, you know, these patents have been out there for a long time, and they're, you know, active, for you know. What is it? 1517 years, or whatever. And so so it's not like you have have to have the latest and greatest, because a lot of stuff doesn't, you know, stuff doesn't publish, you know, until a year after. It's. you know, been in this initial patent. So so, anyway. So

you, Eugene, because I want you to come and lecture in my program, because this is what students are hungry for. It's like, what's the latest. And we have so many clinicians and founders and faculty.

With, you know, starting products that they want to be part of like, what's the latest and greatest. And what's you know happening now. So how are you? Yep.

I'm at Ucsf and Uc. Berkeley. So I run master of translational medicine program. So it's similar to Biodesign or John Hopkins. C. Bid program. So let's connect.

After this, and talk about some things.

Yes, that'd be great. Sean. Yeah.

Thanks.

In the meantime, yeah, that's don't leave Mark out because I have some ideas about where this can slot into a bigger type program. Using AI just from what I know industry needs. I have the overall, the big overall I would love to talk to you guys about. And if we can get something going over here, do some sort of strategic partnership, or align ourselves to build this sort of thing. I already know what industry needs, and they need this badly.

Yeah, let's let's do it. Let's do it. That that sounds great. I mean, this has been a super helpful discussion for me, like I think I've I've come to the realization that this context assembly piece. probably where most of the market market needs are right now. and there's probably gonna be lots of verticals right in like different different industries. There's probably one in law. There's probably another one in like in like building construction, you know. but they're, I think they're all generally solving this this part of this like AI feedback loop process

and like a lot of the pieces are there like I'm I'm pretty sure there's maybe an Api where you can. You can query all the patents that existed for a certain question. and that that Api will give you back some context about relevant patents in that space. And there's yet another one where you can query the 5, 10 K summaries for for suitable predicate another one. Maybe you can query cms for replicable reimbursement codes, and then you can assemble all these things together in a context. and do something useful to that useful with that. But I think the market inefficiencies right now is like finding those, and turn and kind of

aggregating them together. I think that's the the current market in inefficiency. And I think that'll be a great series of a product that potentially could solve it for each vertical. So yeah, it's just just kind of interesting to think about teasing apart, like, how how we're using AI day to day. What's the pain point. I think context assembly is the biggest one right now.

So I wanted to get an unplug for couple of things. One save the date May 6, th through 8th in Newport beach. We're gonna bring together really dynamic program this year. We're bringing in more startups, more more investors and more experts. And our goal is to create a highly interactive networking opportunity for everyone involved. So that everyone basically meets everyone else. We're gonna be changing the program up with shorter presentations and more working groups. And I think what was really nice about this call today was seeing the way that the collective brainpower came together around you, John's tool, and everyone was able to contribute their own perspective to this. I think that when groups come together and work as teams to solve problems, it's much more effective than a

person standing at the podium handing out information that could be handed out in any number of other ways. And so what we're looking to do is to create a really dynamic event this year that everyone is gonna walk away with something of value, and and make it as as accessible to as many people as possible, so please stay tuned for further announcements. And as Joe said, You know, if you want to get in on writing the check this year. He's more than happy to take your check this year. And but the so we're we're gearing up for a launch. The we're gonna take some

couple some time off, and the next event will be Friday, January 3.rd At the usual time. The topic is, gonna be very new to me. I don't know how many of you know about this post. Quantum cyber security and Med tech compliance and the new NIST requirements and understanding how that applies. Apparently my understanding is that most devices that that a that a large section swats swath of devices has been on the market for an average of 12 years, and that their cyber security is long in the tooth making them incredibly vulnerable to cyber attacks. And so you know how to hospital systems

prevent themselves from, you know and protect patient safety and protect themselves from liability. In this continuum of of you know, evolving threats. So I encourage you to log in to this and and to bring a friend. That's it for this week's edition. Any questions or comments. Please feel free to reach out to me or Joe on on slack or by email. I'm Sean Smith, and for Mdg premium and medtech leading voice. Until January 3.rd I hope you all have a very happy holiday season.

Happy Holidays.

Yeah, he's great. Thanks. Bye.

Take care. Thank you.