AI FOR PMS CHALLENGES AND OPPORTUNITIES featuring TIBOR ZECHMEISTER & BASTIAN KRAPINGER-RUETHER

Hello!

Hey, guys.

Hey!

How's it going.

Good thanks. How are you.

Very well.

That's good. Can you hear me? Good!

Yes.

Alright, perfect.

Joe, how are you.

Good evening, afternoon.

That's a it's evening. It's like 5 5 Pm.

Will you see one another in Brussels?

No, we didn't, I think. Sure, you met 2 colleagues of mine right.

No, no, I'm asking if you are going to see Shawn, cause he's gonna be there in a few weeks.

Oh, sorry!

I don't think he's going. You're not going to the your conference. The European authorized representatives.

No, no, maybe t-bor is going there, but I'm not quite sure.

Could you co host me.

Yeah, sure.

Maybe make yourself a little sticky note that that's gonna be my 1st question every Friday.

Yeah, I actually remembered. And then I forgot. It's funny the way that works

Manuel, I must say you're a very well appointed young man.

Thank you.

Yes, quite professional and well groomed.

Yeah, I try my best every morning.

How long have you been wearing that beard?

Actually more than 10 years.

Oh, okay.

Yeah, but it's always the same. It never changes. Just.

Nice city.

What's his name?

They must have a really good trimming system.

Yeah, you know, after 10 years of practice. Fair enough.

Are you a husband or parent.

Yeah, I have 2 boys 6 and 3 years old.

Oh, very nice.

How about you?

My boys are 23 this week, and 21 earlier this month.

Alright, that's very different. If.

Different life stages. I was telling a friend, and, interestingly enough. that you're sharing a conversation about AI today. personally and selfishly. I am so glad that I am not 3 or 6. I don't know what your kids are gonna do for a living. I believe the threat is very real.

You think.

I really do, I really do? Here's a i i shared with another friend yesterday a video that I discovered in 2015, and you can find it on Youtube very easily. It's called Humans need not apply. And it was talking about AI. Now the fascinating thing about it, watching it now is. It's 10 years old, and it was showing like the 1st Waymo which is, are you familiar with that brand? It's a self driving cars that are like.

Yeah.

And it shows like a robot very slowly bolding a shirt. and it said, it may be pretty slow, but at a hundredthth the cost of a human It's pretty effective, I imagine. They've sped it up in the last decade. It said that. You may be ambiently familiar with. The history of the Great Depression, which is 96 years ago. at the time in America, the Great Depression, 25% of the population was unemployed.

And it said, If we look at just these categories, transportation, this service industry, this this you know, going down all of the things that it's not too much of a stretch to imagine the that AI would take over that would represent 45% of the workforce.

Thank you.

Do you ever read Reddit at all? You ever go on, Reddit. It's like a twitter. Kind of thing.

Not read it. Yeah.

So there's a couple of subreddits that I've been looking at. One of them is singularity. Another one is Chat Gpt. Another one is, you know. Chat Gpt prompts but as a marketer I stumbled upon a subreddit for copywriting, and it was this guy saying he just doesn't see how he's going to survive with his job, that that most companies and I would think, certainly and medical device they're content with good enough copy for their website.

And then the delta between this.

Fantastic, subtle nuance that the copywriter can come up with relative to what you know AI could just churn out was so relatively imperceptible to the people who pay a salary that he's getting squeezed and squeezed and squeezed. and he doesn't know what his fate is. I really I really don't know, I think about. I was up Director of Marketing Communications for Publicly Traded Medical Device Company until 2011, when it was bought out. I've been on my own ever since, and I tried to bamboozle Andre on the call here into paying me $50,000 to do marketing for him, and he almost did it But then Covid Hit and his supplier had parts in

Wuhan, China, of all places, so we had nothing to market, and I couldn't believe that I was out of a job because of Covid. I'm just sitting here writing copy. Anyhow. There's no way in his right mind that he would pay me that kind of money. Now he could get some kid to AI generate copy for his website and his. Would it be as good as me. Of course not. Would it be worth a $50,000 swing? I'm pretty sure he would say so, and he is a huge Joe H. Fan, the biggest, even though he absolutely, categorically refuses to come to our annual event. Isn't that right, Andre?

I'm glad I give you something to talk about every week, Joe.

Every week. It's just so confounding.

I'm gonna be infamous for all the people on this call. So.

The he he used to every week, you know, behind him. He has this Adm engineering thing. It's a little hard to read, but he used to even like print out a special welcome for Mdg. Premium. People.

Good.

It out every week like he's like a major fan. and I just he doesn't. He doesn't fly. Well, I think he's afraid of afraid of flights and God after this week that was scary.

Yeah, tough.

Dboy, you have the coolest name I've ever heard t board Zack mindset.

You're the Zeckmeister.

Absolutely been enjoying your name all week. You so much. I can put another thing inside.

I can put another layer on that in in Austria. If you're the Zackmeister, it literally means that you're the guy who who takes care of the tab. If you're going to a bar or something similar.

So you're everybody's friend, you you're very.

I mean, except I'm not. I'm I'm not going to into too many bars but it but I hope it's still people like me, even without that.

Sure, I'm sure after today you'll have a few more Zackmeister fans. Well, Sean is my co-host, of course, and the reason we've gathered today because I knew neither of these gentlemen before some 5 min ago. So, Sean, would you tell us how you found these gentlemen, and why you've chosen to share them with our audience today and coming off mute first, st of course.

That would work. The dog was making noise. Sorry I had to mute the phone. Hi, everyone! Welcome thanks for dialing in today I'm super excited. That Emmanuel and and Tibor are gonna share with us. I 1st met Tbor. I don't know. 6 9 months ago. He started popping up on my Linkedin feed a lot more, and, as you know, I like to follow the people that are active on Linkedin and you know we recently featured him as as well as

One of these and others.

Yeah, well, 99 others, and and and and number of people that you all recognize and in in the Mlv. 100 show. Then I thought, Well, why not have him on the show? And I've seen a little bit about their platform, but I really haven't seen that much? I. So I'm looking forward to learning about what they're doing at Flynn. AI from a manual. And Tbor. And I'm just gonna turn it over to them. But what I would like them to do is give us a 1 min introduction to who they are as human beings before they launch into their you know, biz the business side of them because we we get a little personal here, we get a little, you know, we wanna we wanna know who we're talking to.

I am going to invite. I I have already, and some folks just out of habit. We'll just say No, thank you. But in order to get to know our family here, we'd love to see your faces. So I invite everyone to come on screen. Some people are just not comfortable doing that. Okay, I respect that. But I am going to go down the list again and invite you each to become a panelist, whether you speak or not, that's all. Good. Why doesn't why don't we have the Zeckmeister go first.st

Thank you so much. And then, Emmanuel, I'm also looking forward. You're providing in a short summary. I hope the quality of my audio is okay. If it's not okay, just tell me. And then I will try to to switch the Internet connections. Just yeah. 1 min about myself. Approximately 13 years ago, after studying biomedical engineering, I thought that I'm going into the medical device industry full time. I've built my own medical device companies, mostly focusing on medical devices with neurodegenerative and neurological focus measurement devices, differential diagnostic devices. So this was how my career in Medtech started. And 30 years later, I could say that.

on the one hand, I'm still an entrepreneur in the medical device space. This is also the reason why I'm the head of regulatory and quality at Flynn, because we are a fast growing company that is really trying to revolutionize the concept of yeah regulatory automation. And AI supported regulatory automation. So this is one part for me. The other part is that you know During my 1st companies nobody wanted to do regulatory affairs. Everybody was afraid of that, and I was somehow crazy enough to say, you know what I'm interested about that I know it's complicated. I know it's not easy. I know it's complex, and you have to put a lot of puddle pieces together.

But that's the interesting challenge about that. You have 1,000 solutions, how you get your certificates or your approvals or clearances. And that's what makes it really interesting from my side. So I've been also for several years into regulatory affairs, and I'm trying to cover my knowledge with different approaches. So one, of course, obviously, is a blend, because we are talking to many different customers and corporations. But we will be having a little presentation about that soon. On the other hand, I'm also an auditor at the notified Body Laundry or in Europe, meaning that I'm actually going to European medical device manufacturers and checking if their technical documentation, their quality management systems and so on, are complying with everything. So I'm trying to keep.

I'm trying to keep up to date from different perspectives, because. especially in Europe, we are getting more and more challenges in regulatory affairs. But I have somehow this mentality of challenge, except that the more regulations you give us, the the more how should I put it? Creative? We will be getting to comply with everything and still have innovative and safe medical devices.

And Emmanuel.

Effective.

All right. Hi! Emmanuel is my name. I am a proud father of 2 boys. I have a dog, so that's that's my private life. I live in Germany. Professional wise. I've been in the medical device industry for about 16 years. I started with a company. It's a rather smaller company where we developed picture archiving system and radiology information systems. I worked at a pharmaceutical company in the medical device department there as one of the first.st That's quite some time ago where we implemented the Tir 45 for Medical Device Software development. And after that I became part of the Chowner Institute, which is a consulting company very well known in Germany, at least eventually became CEO of that company. But what really drives me is, you know, there's so much regulation, and especially in Europe, that medical device manufacturers, the whole space is heavily overregulated.

And what I, what really drives me is automating and digitalizing regulatory processes. And that is why I joined Flynn. Because, like, we have a pure focus on software development, and we believe that we can have a real impact for for this, for this industry with what we are doing. And maybe we can give you a glance today on on what we're doing and how we think we can automate things. And hopefully, you you enjoy what you see. maybe if it's okay, I I would quickly share a couple of slides. Don't worry. I will not bother you with. Enter slides to show you who we are and what we do. Would that be okay?

Of course. Now, how many folks do you work with.

Pardon.

How many people are at your company.

I will. I will tell you this in a second. When I show the slides. Just give me a second. No. Can you see our slides.

Yes.

Alright perfect. So our company is called Flynn, we, we develop software in the medical device companies only. So nothing else. Our focus is medical device companies. We're not doing consulting or anything else. We have a very highly experienced team with really profound industry knowledge. For example, our founders. They have built up medical device companies in the past. Our principal data scientist. He has more than 10 years of experience in data science. And you met Tebow already, and myself also. And to to your question, we have right now about I think it's 35, more than 35 full time employees. Not counting, like external or not full time employees in.

And the company was started. In what year.

Tibor. Correct me if I'm wrong. But it's about 2 years ago, when the company was founded.

Wow! That's some growth. Congratulations!

Yeah, it's it's really growing fast. That's right. So maybe to show you what we developed in those 2 years. So what we have already available is a software module for observing vigilance databases like, for example, the FDA MoD. Or recall database, but also other European databases like B. Farms, with medic. Like all Md. Sub countries, are already included. Our second module is a literature search module, that where we help search databases like pubmed Pmc. Cochrane, Google scholar just to name a few. This already includes some AI features. Maybe we can show you them later that help. For example, extract data from the full text like manufacture, data or device data.

Our 3rd module is regulatory change monitoring. So here we observe the whole regulatory landscape and provide our clients. A monthly update on what has changed, on regulations and laws, for example, and the 1st 4th module we're working on the complaint management. This is really early phase, but also very complex topic where we tackle the whole complaint management and try to find also using AI, try to find features that make use of the whole data and complaint management, which is very complex, and we hope that we can show something. maybe towards the end of the year. But for an outlook. So what we are aiming to is we call it a quality management system co-pilot. So we want to have a system that gets all the external and internal data and make use of machine learning or AI to inform intelligent risk management basically.

And just to show you a few names that we are allowed to show to to show you that we work with a whole variety of different medical device manufacturers. So we're not focused on any niche like we we provide services for the whole medical device industry. and that's about it. For for this presentation. So what we thought we could offer you next is that Tibor talks a little bit about our opportunities and challenges in AI, and after that we could have either a discussion, or, if you want, we can also show you our software in a live demo. So that's up to you. We can decide when Tibor has this presentation.

So Tebow, can I hand it over to you.

Absolutely. And Shawn, as you mentioned, and also John Shawn. Thank you so much for inviting us today. If we understood correctly. This is today an informal session. So if anybody is having any questions or just wants to ask something right away, please just interrupt me. So even if somebody says, after 5 min, oh, we already know about all these lunches and opportunities. Let's talk about something else. Then just.

Don't worry about. I'm big into interrupting. People from New York. and Germans are kind of rude, too, so I think we could.

Direct. The word is direct. Joe.

Oh! So!

Great. So then I would be just showing you a few slides. Give me a moment, and I will start sharing. Hope it's working. Can you.

Yes.

You see my slides? Yeah, perfect. So what I really want to talk about today is a little bit about the fact that even though, for example, we, as Flynn, are working with AI on a daily basis, we somehow have the feeling that companies believe or like to communicate this, that this whole topic is quite easy, highly efficient, that you can have 1 million features in one week. It's super reliable, and so on. And I really want to show a little bit a more nuanced showing. Yeah, what's possible with AI, what can we do today? What would be probably really realistic in the next 3 to 5 years what we could be doing in regulatory with AI. But at the same time. And this is where maybe my role as an optical body auditor comes in place. Also the challenges.

So how can we check if it's really reliable? How can we check if the features are really working correctly? What's the accuracy rate we want to expect, and so on. So that would be a little bit the idea about that. And I would really like to to focus on 2 topics and and present a little bit about that today. Because this is also what we are working on. Number one is, we are providing a tool for literature search, and I think most of us know the pain you go through pubmed Google Scholar Cochrane. You name it all different databases. Enter your search terms. Then you get dozens of different papers.

or even more, depending on how large the company or the medical device complexity is. And then you're starting and saying, Okay, I need to go through these abstracts. I need to include, exclude my papers for the included papers I then have to check. Where is valuable content, what's actually relevant to my medical device, and so on. And of course, this is a lot of copy pasting work and a lot of repetitive work, and of course, anybody who has done that before knows this is not done in 8 h. You're spending weeks, sometimes even longer, doing that. So this is highly technical. And where we are right now at AI. And this is really interesting is that on the 1st level

we are already able to mimic inclusion and exclusion behaviors from human experts. So let's assume I would give you 100 different papers. Say you have a certain search criteria. You're looking for. You say you have a special medical device and start including excluding these papers. You say I've included this one, because yes, that's exactly the medical device we're looking for. You have excluded that one because it was only an animal study and not a human study. So we are not interested in that one. And the really interesting feature which is already now possible is that after, let's say, 20 inclusion and exclusion criteria AI is already able to make suggestions for you, for the next papers, meaning you included, excluded 40 papers, and from paper 41, for example, it says.

based on your past decisions. I would recommend you to include or exclude the next paper, and also provide you some information. Why, it would suggest you do that. So this means that you still have to go through. Let's say, 100 papers or 200 papers, but at least you get a little bit of an assistance and the smart assistance, saying, Okay, you seem to have some repetitive behaviors here. I'm trying to support you and include and exclude these papers altogether. So this is one thing we are currently capable of. And I must say the the accuracy and the the yeah reliance on that is quite good. So we have tested that with multiple customers, and they said, Yes, it's actually helpful in providing some value because it speeds up activities. Another example, maybe let's pivot a little bit to to vigilance cases or severe adverse events depending. If we are more us based or European based. And here we have a quite similar story you get, maybe.

Yeah, you have an FDA MoD search, and you look for similar devices and check what has happened to other medical device manufacturers and their devices, and could that also affect my risk management, my medical device technology. If you if you get a certain search term into FDA MoD, it's not improbable that you will be getting 10,000 plus different hits, meaning. If you find 10,000 plus different hits, somebody has to go through these 10,000 plus different hits. And I mean, again. If anybody has done that in the past, I had to do it at least a few times in my career knows this is not an easy Chore, and you have to do that for weeks, ongoing in an excel sheet copy and pasting.

But again with AI we could be do that a little bit smarter. 1st of all depending on the database and the information, provided we could be doing certain highlighting. This is indicated here on the left side. For example, if you go to the German database, the beef farm, you don't get this nicely structured piece of information, as by the FDA. You get a Pdf report, and these Pdf. Reports can be 4 pages. It can be 20 pages, or even 40 pages, and you have to reach through that. And unfortunately, and again. I'm quite sure all of you have done this before. If you go through some of these 1015 page severe adverse event reports or reports, vigilance reports

in some of them. You know exactly after 5 seconds, what the issue was, what happened to the patient and what the overall issue was, and in other cases you read the same document for 5 times, and you still have no clue about what they offer wanted to tell you, because either they are really not good at writing these reports, or they just simply want to hide some information to not get into legal trouble together. And here AI can support goes through these documents and gives you the most important information. Were there any deaf? Were there any deaf cases? Was there any injury? Was a patient involved or not. Was it an actual incident, or was it an maybe this could have been an incident situation? This is one possibility.

Another possibility is summarizing these events. I mean, you have 40 pages. You don't have time to read 40 pages. You want to get a summary, maybe 2 to 3 paragraphs stating the most important information. And that's also something especially large language models are super efficient. So this also a good use case. And another thing is what's happening quite often, if you are a medical device manufacturer operating in 60 plus countries, for example, what could be happening is that the same reports of similar devices are showing up in different databases, meaning that while they get a new, unique serial number, it's still the same report. And here also, AI could be so.

And let's move maybe a little bit into the future. And I know there is a lot of debate about that. What can AI do? Can AI not do? But if we really assume that AI is growing in the same pace and speed, what we are seeing currently also with the new activities of Openai, but also deep sea from China. And so what we are seeing from the speed. then it's not that far away in the future that we say that AI could be collecting multiple data sources on its own. For example, getting information from severe adverse event databases afterwards going into clinical literature data, and maybe even do complaint, handling, and say it intakes complaints from actual customers and users of the medical devices. These are already pre-filtered.

and the results are processed for the human expert to check. Then you even get information for your risk management, clinical evaluation activities and maybe usability issues or biocompatibility issues. You have pre-generated text out of the incidents and findings the system has been doing. And finally, you just get a list of to do's from the system. So you get your complete co-pilot or regulatory artificial intelligence support. Where you, as a human expert, can really focus on the most important things. thinking creatively about what new risks could be emerging out of this medical device. Where could I leverage my time to make the device safer? Should I spend more time in R&D checks or in production. Where could I really impact something and leave this repetitive work? I would say

to the AI system, now this is now the big, I would say the big pitching about how? Where we are going in 3 to 5 years. But now I would like to go to the Flip side. What that, or or Sean, do you have a question? Or, Yeah.

I'll interrupt here, Sven, if you wanna come on camera and unmute yourself. You had a question to ask, and, Priya, I've invited you to come on as well. I see you have a question. Spend.

Oh, Hi, thank you. if you hear me. My question was, I wanted to know whether it's dependent on a domain specific. Lmi Api, or it is some kind of self hosted llama model that you use.

So at the moment we are having self hosted models, we are building on existing models, not inventing our own, but heavily modifying training it. So we are having a combination, especially in experimenting. If we are looking for new features, especially I features, we are testing multiple models against each other. We are going with llama. We are going with Chat gpt different models, and so on. And we're just checking which model is the most suitable or efficient for.

No, thank you.

It seems Priya is a bit camera shy, so I'll read her question. Curious as to what model you're using to triage the vigilance of the Pms literature data? Is it open source or domain specific. large language model.

Yeah, I think this is a similar question here. So we are using a mix of proprietary and open source models. And we're doing several benchmark tests and deciding which is most efficient. What I can tell you right now is that for the vigilance case, what we see that Gpt based models are still superior, not that much superior, but still certain level superior to the other models that provided right now, but this is something that could be changing over the next 3 to 6 months. For example.

Nikhil, your order. You don't have to type in your question. You just come off mute, and ask.

So I saw that your tool sort of looks at keywords and summarizes, provides a brief summary of the adverse events. But does it actually contextualize the whole picture like, does it come? Is it able to reason it and come to a conclusion from these keywords? Because keywords can mean different things. But can it actually draw a conclusion from that fact?

This is a really good point, and this is where validation comes in place. Because if we are testing and validating these features, first, st we need to understand why a certain conclusion was made, especially if an incorrect conclusion was made. I have one slide about an example where we tried for months to improve our system, because, as you say, it's context specific. We had one experiment where we gave the system obvious case of reports where death case happened or were mentioned, and the simple task by the system was, no matter if somebody died, or even death was mentioned, or a synonym was mentioned.

You then tell us that this is a deaf case, and I can tell you it wasn't such an easy task you would assume. This is a 5 min, chatgpt, prompt, that it's done. But there were so many weird reason it was starting to reason about yeah. But nobody actually died. So this is not the death case, and even though the word deaf was mentioned 5 times, I still didn't believe that it would be valuable for the reader to get that information. So I have some time prompts became was 2 pages long. And this is one example that it's highly context specific. And we need to understand how the system is reasoning, otherwise it wouldn't pass our validation tests and activities altogether.

Thanks.

Thank you. Please continue.

Thank you so much. So let's move to the other side. And now we, with the questions we already had some interesting topics. I want to lead now into. And this is about okay. Now you're introducing AI in your regulatory affairs processes. You're either a manufacturer or a consultant, but you decide to use these tools altogether. For example, Flynn. then, of course, some of the most obvious questions auditors will be asking, what about hallucination? Can we really rely that, for example, if summaries of severe adverse event reports are summarized correctly, does it invent any information? Because let's be honest, even if we do something simple, with, for example, Chatgpt, such as

telling the system. I'm researching a topic and give me 50 papers and the original links to these papers about this topic, you will be surprised how many fake topics fake links and fake dois chat gpt is still generating. So this is still a big issue. We we haven't resolved yet. If you're using the oh, sorry if we are using the vanilla version or the basic version of the system. Another big thing is, how was the system validated because no matter if it's an AI software or a Non AI software, if the manufacturer is, for example, complying with Iso 13, 4, 8 5. Then this software will fall in the scope of section 4, 1.6 computer software validation.

And we have to validate that. And we are not talking about the simple input output testing, because that would be easy. If AI would be just simple output output testing, then we wouldn't have any problems with validations. But here the system is generating content, interpreting information is checking context, specific information. And we need to have completely different validation approaches. And of course, what data sets were used for training? Is it reliable? How many data do we have available? Can we reproduce the tests we are doing so? These are only just 3 different topics where I believe that anybody who is working on AI, if they cannot at least think about general approaches, how they will tackle this problem or solve the problem.

Then you, then you have a system that might look great, but at the same time I'm not sure if it would be compatible with the medical devices. Because let's be honest, if I'm asking for 50 papers out of curiosity because I'm interested in nuclear fusion. Okay, if I get fake results. That's 1 thing, but not if we are talking about patient risk, patient safety and also medical device quality. That's a completely different story. We have to check for. And what we have seen really in the last 2 years altogether, since working with AI features, is that especially if you're talking about validating these features. There is not one size, fits all approaches where I can create one template for all the AI features we or anybody else is having it. It has to be really highly individual approach.

And this is why I always want to communicate with people that validating such systems is not only the job of regulatory affairs expert, because, for example, we, as regulatory affairs experts, we know what validation means. We know how to set it up, document it protocol that. But at the same time for AI features, we need data scientists. We need these experts and the multidisciplinary team to really understand what feature are we developing? What are the potential risks? How could we deal with them? And how could we test them? What would be actual realistic use cases? So, for example, at our company. If we are doing validation activities. I'm not in my silo department saying, Oh, I get to know new feature. And now I'm thinking about the best option. I'm asking the data scientists. Would that make sense? Is this approach meaningful or not?

And just show you some examples? I mean, this is not an exhaustive list. By the way, this list was generated by an AI to show all the different possibilities how you can check AI. The the thing is, you see. And you see, there are dozens of different elements. There are so many ways how to validate AI in a software environment. And this is a good news because we have approaches. Maybe the bad news is. we still don't have proper standards or regulations, or any best practice frameworks where we could be saying, Oh, wow! Okay, I have a decision. For if this is my feature and it has these inputs or these outputs, or is using this model, then, out of the 50 different options I'm having here. I should be using option number 25, or number 12, or something like that. So we are still in a quite experimental stage.

meaning that there is not really a right or wrong answer how to validate as long as it's meaningful. But it's still a learning phase where you might say, as a company. Okay, validating. This feature was meaningful 12 months ago, but after 12 months and gaining new knowledge, you say we have to redo validation because we have now much more experience with the feature or the model we are working to get her. and maybe to to get to the regulatory side of that, that this is really interesting and and our focus. And my focus is is on the European Union and also on the Mdr. The medical Device regulation, I mean AI is not addressed by the medical Device regulation at all. You will not find a single requirement or a statement about AI. There.

then, we have a complete European Union. AI Act. I mean, I think Europe is the world's 1st yeah. Supranational organization hosting such or organizing such a law regulation. But if you go through the hundreds of pages again, you will not see any concrete recommendation. If you're having this system going to that direction for validation, or in another way. You won't find that even if you go into best practice right now. and British standards has, for example, published. Bs 34, 4, 0. It's great framework. Or you should do your documentation for validating AI software and features. But at the same time it still doesn't give you a certain

framework or decision tree. What method to use. So we see that there is an emergence trying to harmonize our approaches with how to validate and really build trust for the AI. But at the same time I sometimes call it the Wild West of AI, because there are so many solutions. There is no established framework yet how to do that properly. maybe just to finish with 2 examples where I can really tell you from. Yeah, from our experiences. What we had. The thing is we mentioned detecting keywords, as, for example, tell me if there was a death incidence in a report. and of course we have here ground truth testing, because if you give me 10 papers or reports, I will be going through them reading through them and saying, Okay, there was a deaf case here. There was no deaf case there. This was maybe a deaf case, because they mentioned deaf cases, but not in relation to a patient or something like that. So the obvious solution for here validating this AI feature would be to a human expert who is also double checking. But there we have now other challenges.

How many checks should we do if the system is going through 100,000 reports? Should I go through 100,000 reports. Is that meaningful? Is that efficient? Where's the cutoff? So this is really complicated and not easy to answer, because, especially the question remains, how accurate should the system be. should it be 100% accurate, and we know AI will never be 100% accurate. Should it be at least as accurate as a human expert, because I can tell you, when I did, my FDA MoD searches with these 10,000 plus hits, I can guarantee you that I, as a human, made mistakes in these 3 weeks, because if I'm staring at that excel sheet and making inclusion exclusion

decisions for 3 straight weeks I will be getting tired. I will be making mistakes, copy pasting mistakes, and so on. So what should be the accuracy? So this is one challenge we're working with. Another thing is that was really interesting. And here we tested human intelligence against artificial intelligence. The basic test was that we got reports from the German farm competent authority. These these Pdf reports about severe adverse events and what we said we give 3 trained auditors. One of them was me and the AI, the task that we use the Imdrf coding system for severe adverse events. You know the Nxa to F. 4 GI think, with the different classifications. And we started with Nxa and said, Explain to me the medical device problem.

And what was really interesting is that even though you had free human experts who had, I think, accumulative knowledge and experience of 50 to 60 plus years at least. The problem was that this Imdrf coding has 400 different options. meaning that we, as human experts for each and every case, had to remember these 400 different options to choose the best one. And of course the system here was much better than the human expert. We then gave the results to another independent assessor, and asked them, do you think that the human or an AI did these assessments, and unfortunately, or unfortunately depending on the interpretation the auditors heard

less efficient or worse than the AI. But this should be also logical, because for me, as a human memorizing 400 different options, and going through all of these options in my mind, for I don't know 200 cases 0 adverse event report cases. I mean, I don't say it's impossible for a human brain to do that.

Probability.

Sorry.

It's beyond the scope of human probability. Yes.

Yeah, absolutely. And this is the issue. And so AI was much more efficient here. So this is also what I wanted to show with this presentation. So yeah, I just want to really summarize. Yes, AI is definitely, in my opinion, here to stay. It really depends on the level, because, you know, there's always a hype cycle in the beginning, saying AI will solve everything in the medical device industry. Now we are a little bit in the face where we say, Okay, it cannot solve everything, but it is still good for many use cases. But at the same time, I think the bigger question is not, what can we achieve with AI? But how can we do it? More reproducible, more reliable, and more accurate, that

we don't have the situation that the AI is working efficiently, but introducing more potential risks to patient safety or medical device quality than the human experts are currently doing, due to fatigue or burnout, or having too much effort or too many projects at the same time. Yeah, this was just a quick summary, I hope. For for many of you. There is at least a little bit of new information there. Hopefully, none of you say we already know that, or we knew that from the beginning, maybe maybe feedback from your side. What? What's your take on that? Do you say these challenges are real? Or do you say, yeah, we believe it's already solved in the industry that this this can be easily validated. What's your take on that in general?

This AI versus how to test and check AI reliably.

George.

1st of all, I wanna thank you for your presentation. And the the software and and activity that you guys doing and and tremendous growth. I, I, for one, am also a fan of making research a lot more efficient have been down that path before with the cut and paste of thousands of of literature error and all that stuff. So what I've used in the past or most recently for for AI is bio bird and it's its ability to, you know, help with the screening for the material that I'm going to include in the research, or or what have you? And and whatnot so sell me on? Why, this, your approach would be probably an optimal solution versus the solution. I might be looking at off the shelf with Bio Burt. And you know the name entity, recognition and transformation model. What that is.

Emmanuel, do you want to answer the question?

I I can try. I'm to be honest. I don't. I don't know by word maybe what we could do is if if after, there's other questions answered. If you're interested, we can quickly show you an example. We prepared like a very small demo of our capabilities. In in our software.

That would be illustrative. Let's jump to that.

I can wait.

Any other questions before that, or should I just go ahead.

Please go ahead.

May I ask one.

Yeah, of course.

Yeah, it's you how I'm actually from Finland, north of Germany. But I have a very trivial question that when I do, literature searches, I have to purchase the the the articles from from publishing companies, so I have no idea how it works with the with the AI. Do you 1st buy the articles, and then you let the AI read them, or how does it work?

No, I think this is a really important question, and also a big burden for many medical device manufacturers, because you have to spend thousands, if not 10 thousands, of euros and dollars, into buying and purchasing these papers. So we have 2 approaches here. Number one. The abstracts are always available, so at least some information. It may. Okay. at least some information. If the abstract was well written, and we know that not every abstract is well written. Unfortunately, we can already get some information, and the second part here is, some of the papers are open access. This is much easier to collect information, and if it is not open access, then, as you said, the manufacturer would have to provide

the full text access paper to the system, so it can make more interpretation on that. But that, of course, creates some legal questions about. Is this within the intended purpose of purchasing papers and then using papers in the organization. So the technical answer is, Yes, of course you buy the papers and give it to the AI. But of course we are also discussing different approaches of licensing models with partners, and so on, and publishers how it would be still in scope that the AI can check these texts and provide meaningful information, and be so in a legally compliant framework, so that we don't hover within a I would say gray area.

All right. Maybe I'll give you a quick run through through our software. And also, if you have questions, please interrupt all the time. So when you log into our system, it looks like this. You have an overview of all the different searches that have been done in the past. You can see here. It's a mix of vigilance and literature searches. And I'm now going into one literature search, just click that one. And it's basically you can can name the search. You can attach some labels to it. That's for sorting and searching for and grouping the searches. You can decide if you want to search a fixed date range. For this example, we choose the whole year of 2023

you can choose an ongoing search. That means the system searches like you put a start date and the system searches every night for you and sends you a report monthly with what? What has changed within this month. If there's new publications available, for example. And here you can choose the databases. For this example, we choose 2 basic databases like pubmed and Cochrane reviews. and you can write your query here. So for this example, we choose to look for intraocular lens and publication Type Review in the databases pubmed and Cochrane. and for the date 2023. And if you click on, see results, the system searches in the selected databases for this time range using the keywords, and we get a list of all all the hits we we received. It's in total 137.

So let's click on the 1st hit. Here you get the abstract, all the information that is available in the abstract. And the system works like this. It's 3 parts like you do the search. You do an abstract screening where you quickly go through the abstract and see is it relevant or not relevant? And later on you do a full text evaluation for all the relevantly selected papers. So here you can quickly select if it's if it's relevant or not relevant. And you already see one AI feature working here. That's 1 we prepared for this demo. Specifically, it actually looks at the full text and extracts the device names and the manufacturer names. And so that's when you do the abstract screening. You already get the device names and manufacturer names. You don't have to look them up manually in the full text at this stage.

and we also have this. We call it pure highlighting. For example, if you click, show me the intervention, it shows you it highlights you. I don't know if you can see it. It's light blue. It highlights the parts that are relevant for the intervention in in the abstract.

The manual, so it's already tagged it for you. So you don't have to go in and manually tag this article.

What do you mean with tagging carries.

So so your AI insights. It looks like it's already tagged this article with the device names you've already tagged the manufacturers. You've already given it. The pico.

Yeah, it depends on how we configure the system. So in this demonstration here, it only displays the information. So it's not automatically tagged because the system. So there's 2 ways, for example, go to the appraisal or to the analysis. We could configure the system that it automatically puts the information in the corresponding fields. In this demonstration here, you would have to copy and paste them. So it's up to the human basically to decide. Is this correct? And if yes. we put it here.

So like, if I was doing a Cer, for example. And I needed to do all of my competitive research. So I had the okay. So it looks like you have it set up here for a cer.

This is basically a setup for for this demo. Specifically. So, it's not specific.

Sorry to interrupt. Okay, go ahead.

In fact.

Shut up!

But did I answer your question?

Carrie, you'll tell us a little bit, because I believe this is the 1st time you're talking in the Forum, I know, but for the benefit of the folks at home and how, if at all. You see, being able to use a product like what you've seen today.

I I don't know what I do. I help manufacturers get products to market and post market surveillance, and I just help what I don't know what I help. I help them do whatever they need me to do.

You can work on your marketing pitch.

Yeah, I know I do. So.

Although I think you've done pretty well before we met, and you'll have a chance to meet her in California in May.

Yeah, yeah.

So. So you see a use case for you to use this product.

Oh, yeah, yeah, I've used nested knowledge before. I'm not sure if you guys have seen that one before. But to me the tagging piece of it gets to be a pain because I have to set it up. And it's they just keep updating the software for me. Which then means I need to go back in, and anyway, go ahead. And manually, I'm sorry.

Sorry. No, no, ask questions like if you have further questions, please.

Maybe I don't apologize here.

Michelle, did you have a question.

No, yeah, just a quick question for Carrie. Are you based in the Us.

Yes, I'm in Colorado.

Okay. Thank you. Sorry to interrupt.

But you may be hearing from her.

All right. So so this is one of the AI features for this demonstration, which extracts on like on the abstract level, you already get information that is in the full text. I'm just to quickly show you. The full text, of course, is available here as well. So you click full text, and you can scroll and see review the whole text. And yeah, you can do the appraisal like, if you have done the abstract screening. So we are now in the full text evaluation. For example, you can do the appraisal here for this demonstration. We put a met dev appraisal template in here, but this is highly configurable, so you can do your own templates, or we can provide you templates that we already pre-created here. So you can do the appraisal, and with the analysis tab, you can

also. This is highly configurable. So you can put your own fields here. For for us, for our demonstration. We put in patients, intervention, comparison, outcome, patient gender indication, and so on. and maybe show you another AI feature, which is our recommendation feature. So this starts working once you included and excluded enough papers that the model is confident that it understands what is relevant for you. And here it recommends you the 5 most relevant papers it could found which would not show up using your keywords. But the system thinks they are relevant. so you can go through them and see if they're relevant, for you include them or exclude them. And if you press, get 5 more recommendations, you get the next 5 recommendations that the system thinks are relevant for you.

but the the more you click, the the less relevant the the papers are getting. So this, with that feature we make sure that in the literature search. You're not missing anything which is relevant for you which is not showing up with your keywords or your query.

I really like how it searches all the relevant databases. I'm just going to tell you that I'm impressed. Thank you. Thank you for the share.

Show, and so also working on adding other databases. Right now we are including Pmc, for example, and we're just in discussion with other manufacturers that have specific databases. Like, for example, that we are looking into. Yeah. So once you're done.

Will it link to the actual, free, open source so we can download the papers from there? Yes, no.

Yes, yes, like the papers. You have them available here.

See? The open access? Perfect. Okay.

We are also working on a.

That's very helpful.

Yeah. And we also working on a purchasing feature. So if there is an article, where you, which is behind a pay war, you can buy that in the future directly in the software. And it's not our business model to make money out of that. So you'd pay basically the same amount if you would buy it from the original source.

Yeah.

That leads me to my next question, what is your business model?

Our business modeling is also not consulting. It's we develop software and we do research and development in AI, and we provide software. And our business model is license fees. Basically.

Is it a flat rate? Can you give us a sense of what it would cost to, are you? How many, if you're willing to share. I know it's private information. How many customers do you have now? Do you have a goal for the year? What kind of growth rate do you have beta testers. Whatever you're willing to share with our.

Yeah, here. So we have about it's it's almost 50 customers. Every size. We are providing was a worldwide customers. I showed you a slide before that had some names like, or was international names on it. We do have like beta customers like there's a certain amount of customers where we develop new features. For example, the complaint handling features. We do this with a selected, with selected partners to explore. If we can create something that provides value to to those customers. And if we think that at least 5 of those customers get value out of this, we create a something that we can provide to all of our customers from the price range. I cannot give you specific numbers. This is really depending on on basically heavily depending on the complexity, like the the classification of the devices, and how much effort it really is to observe the databases and and do the literature, research

and also of what kind of modules would be included.

It's big enough, does it? Is it a function of how many seats you have.

The seeds are less relevant. It's more the technical documentations, for example, this is more of a driver. This also the seats, but less threatened, I would say.

Okay. Let me see if I can ask this question. Then I'm a 1 person company, and I want the simplest off the shelf thing you got. How much am I paying.

We don't have- we don't have off the shelf, basically like this. So.

Okay, well, whatever I I want, I want what he's having. I want the same thing that you already built for for Jose. And I'm 1 person. I just need that. How much is the.

I would say, for a let's say, for a smaller company. That is looking for only doing the literature research. So it's it starts low. 5 digit, I would say.

Okay.

And of course it depends on the how larger or how more the more technical documentations you have, the more complex the whole thing is to. of course, the.

So you're not getting out of here for less than 10,000.

No.

But we should talk to based on that. You can talk to Emmanuel, and maybe negotiate with him a little bit, but we have a general idea, and thank you for hearing me.

Thank you for the question.

Test it out.

Why so sorry? My name is Frank Volley, and I have a question. So you said you make license, and this especially, the rate for the license is depend on the class from the medical device. So now I have a class 2, a product.

And.

With a complete whole documentation. And I need, and I mean. I need the complete technical documentation, risk management, vigilance, and so on. So what is the price for the license boy? 2. A product in Germany.

Thanks for your question, Frank. So this is so it's it's it's not about the prices. So what we usually do with with potential clients is, we find out if there is a work we can automate, and this heavily depends on the amount of devices and the classification like, if you have one class, a device, if you do like up, for example, once once a year or every second year. And it's not a lot of work for you. The system probably doesn't provide so much value if you, if you have more devices. And you say, Oh, we have a lot of work observing the databases and updating Ocrs. Then our system probably provides more value. So first, st usually we start the discussion with our customers finding out, can we really provide value or not. Because I said, like for lower classes, devices, or if you have less devices, our system probably doesn't provide so much value for you.

Yeah, yeah, sure, it's clear sorry for this one for the interruption. So because now I have a 1, 2, a class product in Germany for blood pressure devices and manual blood pressure devices and a digital one. So I have 2 technical documentations. And I need to complete things for Pms. So for the weekly ends. for everything, for my technical documentation. So I make it now handly. And you said, Okay, you can automatically this one for Aa. So, for example, arrange what is it? Is it for the license? 200 500,000 per month per year? I don't know.

I mean the the. As I said, Frank, I think if you're really interested, we can. We can explore, if we can provide value for you, but to give you to give you a number as said before, like it probably starts low 5 digits.

Thank you for

Quick one sorry, sean. Does it integrate with any existing workflows this? Can we use it with any other systems? Or does it need? Sort of it needs to work on its own.

Good question. Nikki.

Yeah, very good question. Thank you for that. So we are Api open. We can integrate basically with any system that provides an Api, and we already did this with with customers in the past.

Thanks, Emily.

For example, I don't know if you know Polarion, but this is one of one of the systems that's heavily used by our clients, and we have an integration with them. For example.

Yes, George.

So I I saw that you had, like Phillips, one of the big players in the Med tech industry, and all that stuff either that, or if you could give a a case study, perhaps on the value, add that you were able to help present for for that company in regards to the services that that you offer here.

Yeah, thanks for the question, George. So I don't have anything to present you. But what a what? Typically, what we could achieve. With the vigilance module which I didn't show you so far. That we increase like efficiency in the team with about 80%, because the everything is completely automated. I can quickly show you the vigilance module, and you only need to export an excel document and attach it or or provide it to your notified body or auditor. But if you're interested I can. I can provide you a written case. Study afterwards, if you, if you want.

I have a question. If I'm sorry I.

Go ahead!

Yeah.

Okay, here's a scenario to companies that are competing companies. They basically are doing similar lit searches. And is there because you're using AI and the language learning module model. Is there a chance that it could? There is a chance. You can't tell me now there's no chance. But you know, because there's always a chance of coding errors. But What's the how? When you're going through the testing? And you know, quality of your product. What's the likelihood that you might cross and potentially present something that it was actually the other clients data and presenting to a similar client.

So that's not a hallucination, but it is like that would be an error which would be a costly error because of the breach of privacy.

Very good question. Thank you for that, Patricia.

You're welcome.

Keeper, you can correct me if I'm wrong. But so this, how we set it up is that for each new client there will be a completely, a new environment. And the models are also for for for this environment. So they are not shared models, like every client gets his own not his own server, but his own environment on the server. And the model is like part of this environment. So it's the models are not shared between clients. Does it answer your question?

Yes, yes.

Absolutely, maybe, that this is the current status quo. But in the future we will have the situation that we want to use the data from multiple clients to improve the overall system. But there the topics of anonymization, not only student not only creating creating pseudonymized information, but really anonymous information, would be absolutely crucial. So we are already doing that right now, some of our clients would like to provide us with information where you get patient data, health data, and so on. And this is where we are heavily pushing back right now, saying, we don't want to know about the actual names of patients, their patient data. If it's anonymous for us, that's a whole different story. But you're absolutely right. In the future. We will be having

many more discussions about this topic how we can maximize the safety and not accidentally make 2 competitors data accessible, that another competitor brought to us so definitely a highly relevant question for us in the next few months.

Thank you.

Show if I may. This is this is a topic that I love. I've worked for a number of technology companies directly and and as clients, and have been actually working with a number of these issues since 2,012 and you know, to the 1st point I worked for a company called Quoso, which was bought by Elsevier, because they were indexing scientific literature, and by indexing it they were not violating the copyright. And so Elsevier bought the company because they wanted the indexing technology for themselves to stop them from interrupting their business model. That's number one, number 2 is

you know, have written extensively about post-market surveillance, and you know the clinical and the literature search are well established domains. that and even the adverse events to a certain extent, if they would ever get you to med working. That would be an an interesting thing, although I understand that you know the data in it is already suspect. The you know. There, there's there's stuff that has already happened. And then there's stuff that's happening now. So signal detection. monitoring social media. And and more importantly, probably most important of all is real world evidence. And the ability to use real world evidence. As

the most important indicator of actual product performance in the marketplace. And to incorporate all of all of those data going forward, you know. Give us a glimpse of your your thinking. As you know this, you have this past. This is what's happened with your product. This is what we already know. This is established truth and the ability of AI to interpret that versus the ability of AI to interpret things that are developing and are happening in real time.

I have to excuse myself. I have a hard stop. I'm so grateful for today's session. Thank you both very much, and you're in cable by hands with Shawn and Shawn. Please let the folks know what they have in store for next week. See you next week.

Sorry I didn't realize much. But yeah, I can take this sorry.

So, maybe to quickly answer your question or not even quickly. I think this is an excellent question, because what we are struggling with in the whole industry, I believe right now is that regarding complaint management and collecting real world evidence, we have a huge problem from garbage in garbage out. I remember a few months ago one of my data scientists came to me and said, Tibor, could you please explain me this complaint by an actual manufacturer? Because I have no clue how I should train an AI model or interpret it on my own, and I told, Sure, show me the example. The example was that the customer wrote back in an email, the device is defective.

And I was like, Okay. we we have to call the customer because I have no clue. What this means is the hardware defective. The software is there by compatibility issue. Was there a problem with the patient or only a technical malfunction, and for the future. What we are working on right now is that we say the current approach is probably generating a lot of garbage because hospital staff doesn't have time to fill out 20 page reports to the manufacturer. They don't have 2 h of a virtual call, so what we are working on is again with AI support to create, for example, smart intake tools, saying that if the next time the customer is writing an email or using a chat interface, saying, that device is defective.

The system is asking questions. Could you further elaborate on what defective mean is meaning. was there a software issue? If yes, could you please tell us if there was a patient involved, or this was just a technical issue, and so on. So this is one approach we are trying to little bit disrupt in the system, saying, the old mechanisms are maybe not too efficient. but maybe this new approach could be some kind of solution to get at least high quality data. And with the high quality data also, the assessment and the interpretation becomes much easier than with the current status quo we are having with the devices defective, or the device is malfunctioning, which is not a bad information, but also not a good information to interpret

for a large language model.

Wow? Well, that's exciting. So we're over time. I apologize for that. Next week we're gonna have another technology provider compleer from Portugal is gonna be presenting their Eqms and e-technical documentation. I think the e-technical documentation aspect of their product is more interesting than the Eqms. Because that's a well known territory. But the I think it would be worth exploring what they're doing uniquely in that space. So I hope you all will join us next week. I'd like to thank our speakers today for doing an excellent job. And I invite you all back.

or any any future ones. Thank you very much.

Make that so much for the invitation.

Thank you. Bye, bye.

Bye, thank you. Alright!