Mastering Literature Review for Medical Devices From Compliance to Strategic Insight

Hello, everybody, and welcome. My name is Sean Smith. This is Medtech, leading voice and Accra team present. Oh. medical device, literature, review, panel discussion. I'm pleased to introduce today's panelists Diane legere from Dnv. Romney Adams, Demetria. I'm not going to try and I'm Sean Smith. A few housekeeping notices while we're while we're queuing up and everybody is logging in. If you want to go ahead and put in the chat where you're where you're calling in from that'd be awesome. That'll give you something to do while I'm blabbering away. So use the Q. And a function to ask questions. Please use it rigorously. Ask any questions that you have. We may not get to all of them live, but we will try to follow up with everybody

after the fact. Secondly, this will be recorded and the recording will be made available to all registrants. Tomorrow the email goes out automatically from Zoom. 24 h later, also, the slides will be posted. Note, the slides are posted in sales, navigator, and you have to actually hover over the title of the slide deck in order to see the down arrow in order to download it. That's just one of those fun things that sales navigator does for you, at any rate. So the slides will be available there. and there will be certificate of attendance created. That takes us about 72 h to put together. We have to do them manually, so please, if you would like a certificate of attendance, please put your name as you would like it to appear on their certificate along with your email address

in the chat, not the Q&A. The chat. So use the chat function for communicating with me. And the panelists use the Q&A function for any questions. that said, I would like to point out that tomorrow we're doing a cyber security webinar and next week we will be live in Newport Beach, California. Anybody in Southern California area that would like to attend. Please put a message to me in the chat, and I will make you the deal of a lifetime, I promise you. You will be thrilled, and we have an excellent 27 speaker session lined up last last promotional item is, Accra team is doing a fantastic 2 day in person event, June, I think 24, th and 25.th I'll put the details in the chat.

It's the quality accelerator program with 18 speakers live 2 days in Munich in June. So plan on going to Munich in June. For this event I will put all of the details in the chat and send them during the follow up. We have a special 10 discount for Medtech leading voice. People, and we will put that in there as well. So you want to take advantage of that. That said, I am going to turn it over now to Ula, who is? Gonna take it away with the presentation. and I will be interrupting later. Bye.

Great. Okay. Can you see my screen? Great? Okay. Hello, everyone. And thank you for joining today's webinar on mass reading, literature review on medical devices. It is a pleasure to be here, and before I begin I want to thank our guests again for joining this presentation, and also for sharing the insights later on. So I will start with a short presentation, giving some background information and highlighting some critical aspects. and afterwards we will dive deeper into burning topics like software solutions and AI and Literature review during our panel discussion, which will basically starting shortly after the presentation.

So books, I actually have to start with a brief disclaimer. So the content of this presentation is only meant for educational purposes. only, representing the opinion of Aqua team, and not necessarily the opinion of other stakeholders. Okay, let's start with the agenda now. So while I could actually talk for hours on the topic of literature reviews, I'm limited to 50 min today, and I have to admit it was so hard, so hard to limit the content to 50 min. Really. So within those minutes I will talk a little bit about where literature reviews are essential across the product lifecycle, how to design efficient reusable processes.

I represent some fundamental tools that you can use today. and we will also touch base on how AI and large language models can save time and elevate your strategy, whereby the last part will be mainly addressed during the Q&A session and the panel discussion. Okay, let's start with the yeah product lifecycle. So 1st of all, the life cycle of a product or medical device is actually not defined in the EU Mdr. However, we are lucky because there is at least an Imdf document existing which defines the life cycle, as all phases in the life of a medical device, from the initial conception to the final decommissioning and disposal.

Thus the life cycle of the device is basically understood as the period between the 1st conceptual ideas during the research and development process. and it ends with the last device which will then disappear from the market. So when we now look into the question, where are literature reviews essential across the product lifecycle we are really studying during the R&D cycle. So during the R. And D cycle, literature reviews are usually used to identify the competitive landscape, also to identify or define the added value of the device, and also to search for relevant safety and performance parameters which can be used for preclinical and clinical investigations.

This is especially important when comparing your device data to state of the art data later on. It is not that unusual that the endpoints investigated via extensive and expensive studies do not really reflect what is usually used as an outcome in the state of the art missing important parameters. So a proper literature review during your R. And D can really prevent you from redoing clinical investigations or from performing additional Pmcf activities. Just to address those missing endpoints of relevance. Next, as you probably all know you need a literature review for the clinical evaluation in which you describe the state of the art in the respective medical field, the therapeutic alternatives, gold standard benchmark devices, and in which you also define the input

yeah, to find the endpoint of those parameters and in which you're also looking for your subject device. Data. Yeah, so sorry. Next. You, of course, also have literature review during your Pms and Pmcf activities which most of you probably know. So the goal of the literature search is basically to identify and describe the state of the art for a particular therapy. Thereby it is important to keep in mind that state of the art is not the most recent hype or trend in technology. Sota is rather understood as the current knowledge and good practice acceptable for medical treatment of a certain condition using medical devices.

So the literature search is also used to identify meaningful, measurable, and clinically relevant parameters on safety and performance and to define the acceptance criteria that have to be achieved by your subject device. So the literature search is basically providing the basis that enables a proof of compliance for your subject device, showing that the device is in line with acceptance criteria established from the state of the event. Okay, so how to design efficient and reusable processes. So to perform a literature review, you need to have your defined sources of information. A team responsible for the search and also for appraising the literature. You need to have detailed and correct key data of the subject device, a software that is organizing your literature, and of course, also good and sound planning.

structuring and a good search strategy. So when you consider the literature search cycle, it is basically starting with defining a good literature search plan, in which you are extensively describing what you will actually do. Okay. next, the literature search has to be executed. So you are basically doing the search and everything that was planned as described within the literature, search, plan, or protocol, which means screening of literature, excluding and including articles, and also appraising the articles. and lastly, you will, of course, then extract your information into various documents, and you will describe all your results in the literature search report, and where you basically really present, how you assess which paper and where, how are the assessments of all the included articles?

Okay? And even though this is something. That actually needs quite some time to do. That's therefore we are really recommending you to use standardized templates and processes. If you are a manufacturer with more than one or 2 devices, templates can really save so much time. They are enormous time savers, and you can add all generic information inside, so that you only need to adapt device specific information, keeping the remaining content consistent throughout your documentation at devices. So in case you do not have a template in in place. Take the time to create your template or simply buy one.

I'm working for aqua team. So I'm using the aqua team template on a daily basis, and I can only recommend it. So if you want to know more about that, just feel free to contact me via Linkedin. Just drop a comment, and otherwise just feel free to create your own template. So usually you have one protocol and report for the sota. So the state of the art data and one protocol and report for the safety and performance search of your device. And in the Lsp, you are usually defining your search terms, the data sources that you are using, like pubmate or embase, the strategy that you are applying, which is usually pico, the timeframe, your inclusion and exclusion criteria as well as your selection criteria for the literature appraisal

in the literature search report. You are typically repeating some of those key elements described in the Lsp. But you are also adding the results like who carried out the literature search, and when? What were the exact search strings used, and how many outcomes were obtained per search string. which references were excluded, for which reason, on which level of screening and which references were included? And how is their appraiser? And while the Lsp and Lsr template can really decrease the time spent on the whole literature process. there are also certain aspects which require special attention.

So, for example, the creation of the search terms, when you create the search terms, please make sure to take your time, as search terms are especially susceptible to errors. Just this morning I had one of those errors again in my search terms, and it took me just ages to figure out where it was really so. You can see a search string here on the left side that we are typically using in our template. and one missing of wrong letter or character, or one space on the wrong place, can really mess up everything. Thus we would really advise always to double check for correctness. It is better to take extra time to request a second pair of eyes to have a look than to notice later, when you're basically done with literature review that there was a mistake that you have not noticed earlier. And, believe me, those errors happen more often than you might think.

Another important aspect is that you should really design the search terms in a way that fits your needs. If you have an unreasonable high amount of fits, you can just consider to exclude unnecessary data sources, or you can limit your search, for example, to major topics, which is an invitation that can be used by pubnet and mbase. which is limiting the data in which your keywords basically appears as a major topic only and based on experience. I can also recommend to always exclude certain data sources like comments or abstract. So while we are in our search, focusing here, for example, on including just guidelines and consensus documents, we notice that a lot of conference abstracts are appearing in the search when you are not specifically excluding them, and

while it is not hard to exclude them later, you often really need to wait until your full text level to exclude them, as it is not clear before that those abstracts are actually just conference abstracts, so by eliminating them through the search directly. You will not end up with this problem, and you will save some precious time. Okay, so yeah, here's an overview of the search subjects and filters that we are typically using. So our 1st search covers usually the clinical condition and indication. And we are typically restricting the search to guidelines and consensus documents. Only guidelines are really a great basis to understand and describe the condition and to identify the typically used gold standard and other alternative treatment options.

We are also typically limiting the search time to 5 or 10 years, depending on the number of hits and sometimes depending on basically the yeah, the age of your device. So if it's a really old device, you even need to go back to 15 years sometimes to get any data at all, because it's just not the focus of research anymore. Sometimes it's really ridiculous. Yeah, okay, so when, after looking for the clinic condition and indications. We are typically checking the clinical intervention and also alternative treatment options. Here we limiting, we are limiting our search on guidelines, meter analysis and systematic reviews as we do not. Yeah. Expect to extract any. Yeah. Safety and performance parameter related data.

The time restriction that we are applying here is usually the same as used for other searches. So if you decide to go for 5 years in the clinical condition search, we typically have it through throughout all the searches that have a time restriction. Next, we look for information on generic device, group and benchmark devices which is very similar to the previous search. Just that we have now included some yeah, or basically all groups of clinical studies. As we can already get or extract some nice detailed information on safety and performance. And lastly, we have a similar device. So it's always good. If you have similar devices defined, sometimes you don't have it, and you need to be more detailed and so generally biased group. But if you have similar devices, we are typically do not restrict the time.

And we really look for all things that was published for the devices to get as many safety and performance related information as possible. So yeah, I have more slides with a bit more detailed information, but due to the time I will not go through them in detail. So yeah, we are usually having some highlights. Oh, yeah. One thing that I have to mention. Actually, please be aware that you really need to consider variance and spelling and writing when you make your search terms. So, for example, some people are using Johnson and Johnson. Some are just writing with a complete, and some do not even mention it, and some just use abbreviations. So you need to consider all those variants within your search term as well.

When we go for the safety and performance, search where we are only considering device or equivalent device data, we are also not doing any time restriction. So the search is basically the same as we are performing it for the similar devices. Yeah, so what else do we actually need? So next, we, of course, also need the software. Then for organizing the literature and the Pdfs and Pdfs, and probably everyone knows and uses pubmed and embase, which are basically search platforms where pubmed is for free mbase is actually a fee-based service. Yeah, those 2 softwares are typically used to perform the search and to end up with a specific number of relevant publications.

Apart from that, you also need to use a citation and reference management tool. These are, for example, Endnote, which is also a fee based. yeah, software. So Terra, which is, I think, at least when I used it for free and citavi. and apart from that, you can also use other tools which are available. For example, the literature review. Software distillers are or adobe AI assistant, which is basically yeah, it uses AI to. So you can use AI to ask questions to your Pdf, and then the AI is basically giving answers based on the content of the Pdf. It can also be quite useful.

So we at Aqua team are mainly using distiller as a review tool. So I will give you just some more details on this software. For now. So in distiller, you can basically upload your list of search results from pubmed and embase, and whatever other search database you are using, you can use an automatic deduplication function, and you can screen your results on title abstract and full text level, and you can, of course, also appraise your included publications. All of those steps are customizable to your needs. So once you have your literature, search, report, literature, search, protocol, or plan ready.

You can just use the inclusion and exclusion criteria, and also the present criteria that you defined there and add those as options to select from within your screening, and to present a distiller which is really nice. So the purpose of distiller is basically to create a literature database to support the selection process. And it is specifically meant for a team of persons. And I have to say that it's really fairly easy to work together on one project. At the same time, we are doing that quite frequently. And yeah, distillers also using some AI and intelligent workflows. Throughout their yeah, platform. And what we are actually using most at the moment at least, is, 1st of all, keyword highlighting. So you can add specific keywords specific to your indication, device.

or whatever you can highlight, you can let the distiller highlight those words within different colors. And then it can. Yeah, basically highlight all those keywords in title, abstract and also full text screen which is really nice, especially for full text. Because you can actually upload your Pdf. To distiller. So every reference can be attached with the Pdf. And then you have it all colorful with your highlights, and you can really focus on what you need to look for, which is quite nice. but distillas offering much more. For example, automatic Pico and adverse event detection on abstract level, and much more. It just

not a lot of things that I'm using at the moment. So I would stop describing it here. And yeah, so, yeah. So that's actually everything that I wanted to use. I have just one more slide left, which is basically the key takeaways. So remember, the Literature review starts in the R. And D phase, and it's really important to be used there. Actually. if you are a manufacturer of more than 2 devices, create and use literature, search, protocol, and report templates. always double, check your search terms and adapt them to get a manageable number of searches, and, if possible, utilize software solutions which can really accelerate your revenue process.

And with this I'm actually done with the presentation. I need to figure out how to stop sharing my screen. Done.

So there is a poll on the screen. You should be able to see it. Please take a moment to answer the poll questions. We can move into the next part. While this is running we'll let it run for about a minute and a half more, and give people a chance to answer the questions. Couple notes. The webinar will be recorded. It will be made available tomorrow by email will be sent to your email by zoom, and there were a number of links and opportunities and stuff put in the general chat. So please take advantage of those as well. That's it for me.

Okay. In the meantime, while the poll is still online, I would like to ask Diane, Romney and Demetria, would you please like to share one or 2 key takeaways for the audience.

Sure I'll go first.st Thank you for that presentation. That was very helpful. And I'm going to say, as a notified body clinical reviewer, I really hope lots of people really listened to it. For me. Literature searches. Tell me how much you know about your device, and they are probably one of the most important elements of your clinical evaluation that helps me build trust in your review. So that's my take home.

Shall I maybe go second from manufacturer's perspective? I'm very happy to hear the notified body perspective, and from a manufacturer perspective we would like to be able to design devices which would really bring benefits to the patient and to the user respectively, during a procedure. So to be able to do that. Of course we have to have very good understanding of the current context in which we are offering the device. The state of the art was mentioned throughout the presentation of Ula is a very important input to the design of new devices. And then, of course, throughout the lifetime of the device, we would like to make sure that it is still performing as expected, or even better.

or capture any safety signals. So I think literature search is key for that, and having it well done, is very, very important indeed.

Round this out. Thanks all for that presentation. I feel like you did part of my job for me by talking about distiller so positively. One thing that I would say is, you know, you talked a lot about kind of templates and processes and things. And for me, once you've got those processes kind of standardized and perfected. There are so many opportunities you have to augment and expedite your literature reviews in a metered, human, validated way, and it doesn't necessarily have to involve AI either. So that would be something I'd like people to consider going on from here.

Great. Thank you. So, Sean, how is it about the poll? I don't see it so time over.

Yeah. Are you seeing it now?

Yes, now I see it.

So it's it's ended. I just gonna remove it from the screen and thanks everyone for participating. Yeah, I'm done.

Okay, wait, then, let's just start with the panel discussion. Okay? Yeah. The 1st question is already related basically to the medical device lifecycle. So you hear that, at least from our consultation perspective, it's really necessary to start with that and the research and development phase. But we would like to know at what specific point in the medical device lifecycle do you consider literature reviews to be non-negotiable? And why? So? This will be a question for Diane and Demetria. If you would like to say something about.

Shall I go 1st this time, Diane? Sure.

Okay, of course. So a literature search is conducted at any time. Of course, you mentioned that it's non-negotiable at some point of time, but of course it is happening all the time, because it is also bringing insights for new product development. So it is inspiring innovation. And there, that may not be really a must have, because information can be collected by different ways. But it is a powerful source. However, as a non-negotiable, or, I would say, even mandatory step. This is when we enter in the design control stage. And when we start defining the user requirement specification of the device in a very formal and quality managed way. So from that point of time the literature search should be done in a way which is exhaustive, extensive, specific, and giving us all the relevant information that would allow us to build our user requirement.

And of course, at that point of time we also start the risk management. So any risks that may have been captured from systematic research on the state of the art would be then input to the risk management process and to the design control in this case as well. So that stage of the device, lifetime and life cycle is the start of the non-negotiable phase.

Great. Thank you.

Right. So that's perfect for me to jump in, because as a notified body, we are involved, once the device is ready to go on to the market. So from then on it is a non-negotiable. So for me, I need to see a literature review that's planned for at least annually depending on how stable the state of the art is for your device. It might need to be more often. Or if there's a lot of novelty in your device. So there wasn't much on the market, and so you'll need to continuously be. But reviewing it. But I had one comment on the poll. I found it really interesting that I mean the obvious for question number 2, clinical evaluation report 66%.

I don't understand why that's not 100. So that was one thing that was interesting for me, and then only 10% for post-market clinical follow up. So I'm not sure if that's just because you consider it part of your Pms. And not so much your Pmcf. But they're speaking of non-negotiable. For both of those parts of question. 2 literature searches are non-negotiable.

I agree.

Exactly.

Okay. I mean, from from early design phase until end of life. Literature. Search is happening all the time throughout the lifetime of the device.

Okay, great. I think that is very important here for some other manufacturers. Okay, coming to the next question, what metrics do you use to measure the effectiveness and efficiency of your literature review processes. So this is now a question for yeah, again. maybe this time the end. Do you want to start.

Sure I found this question very challenging, because for me, I just look at when I feel saturated. So when I see the Literature review is not finding anything new, or I start looking through reference lists, and I start to see that I'm recognizing authors and topics. And when the literature review aligns with you, talked about professional guidelines as part of your part of your review, so I'll review those and see if what I'm seeing in the literature review aligns there, or if there's something in there that's missing on either side, really but and when those 2 align, then I start to feel very comfortable with the review, and feel that it was maybe not exhaustive, hopefully. But I don't know what that even means with things on coming on all the time, but comprehensive is the goal.

Yeah. So again, from manufacturer perspective, I'm really happy to have this kind of very diverse panel, because we also learn from each other from our stakeholders. Thank you so much. Acro team for setting up this panel, because now I see what is the perspective of a notified body from manufacturer's perspective. Actually. of course, we would like to be exhaustive. We don't want to miss data and to be able to do that first, st we have to define. And Ula, you mentioned that during your presentation a very well structured search term line or or search them criteria so that we don't miss any information.

And that's an input to the literature search process. So this is actually validated with several different stakeholders to make sure that we capture the proper search terms. The proper keywords. But once we are doing the search, of course, for us, it is important that we identify the right sources of information, but not too many sources which are not applicable, because then our screening stage would be really extensive. So you would see also certain metrics in the literature published on the topic like specificity and sensitivity. These are also applicable for literature searches. So how many data sets do you retain? How many do you reject? And the ones that you reject.

were they really not relevant? Maybe your search terms were not properly designed, or maybe your tool didn't work. So it depends on on the input that you're providing to the methodology. Then, of course. we would like to save time and and gain in efficiency. So that's why automation of the process is very helpful in this case, or digitalization of certain at least certain stages of the search process to scale up, to accelerate. Because now we are, we set when we start the literature search. But we have to remind that for some devices we have to do it annually, or even more often depending on the novelty of the device.

and if the research process is very slow or not very efficient, you may easily lose a lot of time or be late with timelines. So time and the quality of the outcomes which are identified are essential metrics for us.

Great. Thank you. So that gives directly a bridge to the next question. Actually, so, I remember that you used also a distiller. Sr, so I would like to know which specific features have proven most valuable for your medical device. Literature reviews Demetria.

From the, we are still in the process of implementation of a digital tool. To be honest, so far we have been doing that more on a manual basis, I'd say. But what we really found helpful with distiller as a digital platform is that it helped actually the communication with our stakeholders. We are using it not only for the clinical evaluation, but we are using it now. Also, we are trying actually to use it. It's not yet the case, but to use it for market intelligence. So we are partnering with our colleagues from different functions, who may be actually conducting literature, searches also for their processes. So it is a great platform for collaboration within the different cross-functional stakeholders

regarding the actual systematic review process. What we also find very helpful and scalable and very sustainable is that once a literature search is completed, the data is quality auditable. So it's helping us standardize the process. So from quality perspective. It's fantastic. We also can reuse certain data analysis of an article. If another user from another function would like to assess the same article, they don't have to redo everything they can jump in into that record and just complement it with new questions or new data. So it's really a scale up an accelerator of stakeholder engagement and promoting sustainability in the literature. Search.

Great. Thank you. So Romney, as a distiller expert, would you like to add something.

Absolutely sure. Yeah, Demetria. She hit on some really great points and features that we do have a couple of the more sort of basic ones, I would say that we have in terms of features is distiller, has an Api connection with the pubmed database, so rather than needing to search, pubmed externally, and then bring everything into the platform. You can just do all that searching from within the platform itself. and as well as that you can capitalize by scheduling those pubmed searches and scheduling the imports that come in and automate that process. So in the case of if you're doing like a living review. Or if you're updating your post-market surveillance review or anything like that, you can just kind of set and forget almost those updates, because that can all get taken care of in the background, and distiller will bring in the updated literature from pubmed for you.

and then it's just ready there for you to screen and extract the data from which is, yeah, a huge time saver in that regard. We also in terms of working with full text. If you have Pdfs for your references, or for those papers that you are bringing in as part of your Literature review. There's a matching functionality in distiller where you can just upload your Pdfs. And distiller will again work in the background while you're off at another meeting or doing something else in the platform, and it will actually match and attach those Pdfs to the correct references. So again, a huge time saver there as well. And Dimitri mentioned to our data, reuse layer curator. Cr.

so that's where, if you're reviewing the same paper again, or if somebody else is reviewing a paper that another person in your team has already reviewed as part of a different literature review that human, validated, extracted data will be available for you to call upon. So you don't need to comb through the full text of Pdf. Again to find out all that information. It's already done for you. It's been done by a human reviewer, so there's a trust element there, and you can just bring that across. And one other thing that I would say, too, is not so much a feature of distiller, the software platform, but distiller as a company, we do have an extensive training suite and also an onboarding process. So if you're new to distiller, and you would like to have some assistance

with setting up your templates and for getting those built into the platform. That's definitely something that can be available to you, and is a big help in terms of getting started and getting underway and getting those templates and processes bedded down.

I have to add that distiller has also a really great help support. So whenever I really need some help, because I just cannot figure it out myself. And I, yeah, make a ticket. I will have the answer on the same date. I just made one this morning, and I have already received 3. Fly with you.

Oh, excellent! That's great news!

Yeah, I used to be useful lists. So, yeah, yeah. great. Thank you. So. And now I have a question for all of you basically. So I would like to know what complementary tools do you recommend for managing the review process beyond those search platforms?

From manufacturer perspective. I may say that, of course, as Romney mentioned, tools like distiller, are connected to the peer, reviewed databases like pubmed in this case, but we also search in other databases which are not yet connected to the tool, and where we actually do our manual search. But there's powerful source of information in the clinical trial. Databases like clinicaltrials.gov. Of course, competitors, databases. These are all the standard sources of data which are also recommended as part of the regulatory guidelines that we search. We also have a very extensive communication with peoples who are assisting events like conferences where they would capture actually any novelty which is coming which may be important, not maybe immediately, because it may not have been published yet, but at least would keep us open and vigilant and careful.

But of course, on top of the peer reviewed publications, we are monitoring clinical trials, competitive databases, guidelines of health associations, books, all of that is input, of course, to the review process.

I'm going to say for this question. Thanks, Jamit, for that. I had to go to Copilot because of my role. I'm not. I mean I do. My own literature searches to correlate with those that are provided. But for me it was really, I found, something called illicit consensus connected paper. So it was a real eye opener for me, because I haven't. Well, I haven't been in academia for a little while, so not doing it, but looking at all the tools just to get an idea of what's available and the pros and cons of those to be used. So I'm going to have to leave it to to the rest of the panel to talk about other platforms.

Well, obviously from my end here, representing distiller distiller, is something I'm very passionate about, but in terms of a couple of other tools that you can integrate and use with distiller. My other hat is as a librarian, so I'm a big advocate for free access to information and all of that sort of thing. So 2 tools, one of which has been mentioned by Ula already, that can integrate with distiller, and also make your life so much easier. Zotero, that is free. So the reference management software. So for anybody out there who's watching a budget which I feel like is everybody at the moment.

Zotero, if you're not using a reference management software, that's a really good option. And then something else as well is a browser extension that some people on the call might be aware of called unpaywall, which will essentially look for free legal versions of full text Pdfs housed in various repositories online. So it can be another way to save time in terms of that process of getting the Pdfs into your review. And if you're using distiller as a platform getting them into the platform. So yeah, big big time and money saving benefits that can come from that. It's a very simple little browser extension. So yeah, they're the 2 tools that I'll often recommend to people as well as well as distiller. Obviously.

Yeah, I remember, we are also using that because you recommended it to us. So.

There we go, perfect.

Super. Good. Okay? So coming to the next question. what specific tasks in the literature review process are most suitable for AI assistance today. So here. I'm very curious about your opinion, Romney, and also your opinion, Diane.

I'm happy to leave with this one. I think, right at the moment. What we're seeing the strongest developments in AI have really been on the screening side of things on the screening review process. So we've mentioned already, or Ula mentioned in her introduction. The keyword highlighting that distiller has. So some of that is kind of manual, human input but then we have others as well for things like flagging Pico terms in an abstract flagging, drugs and adverse events that are mentioned as well. So those functionalities are powered by AI, and they're just drawing the reviewers eye to those key terms that may be useful for you and help you make that screening decision faster.

You can also use AI in distiller to label your references in a particular way. So to add contextual metadata to those references. It can be useful for you down the track. So things like, you know what type of paper it is like, whether it's a just a literature review generally, or whether it's a systematic review or a randomised, controlled trial. Things like that. And you can even use AI to automate that screening process entirely and kind of use it as a second reviewer. If you're working as part of a smaller team, or if you're under particular time pressure. To complete a review, you can actually help automate that review process in terms of screening.

So that's what I would say. The focus has been on in the industry to date. But we're seeing some really exciting movements in the last, I would say 6 to 12 months, particularly in AI and data extraction. So helping to automate some of those processes and helping to kind of expedite the extraction of those various data points you need for your reviews. So that's really something that distiller has some capabilities in at the moment, and we're working to build upon. But it's a really exciting area, for sure.

Thank you. That's a very comprehensive answer, which makes my answer very easy. I was going to say exactly what you said, the searching and summarizing, because to me that seems like the bulk of of time that would take in a literature search, especially if you're, I guess, lucky enough to have a device that has a lot of hits. I was going to say also that the ability of it to keep sort of searching and ongoing search, and just to see, not wait a year for you to do your search manually as part of your post market plan that it's going on in the background while you're sleeping, and when you wake up in the morning you can see that there is something that is germane for your device to take a look at. But from my end. What I see in my

reviews that use a product like distiller is how organized the the report is I love how it's standardized. It's just the way it's presented honestly for me is it makes it easy. It makes it pleasurable. I can follow the logic and.

You know it puts me in a good mood when I'm reviewing.

That's the best part.

That is the best part.

Wonderful great. Then I have now a very specific question to Dimitri. Yeah. So from a manufacturer's perspective, what are actually your biggest concerns when you think about integrating in an AI solution.

Yeah, thank you. I mean, that industry has to follow certain quality standards. We have to be very careful when we implement new processes, equipments, procedures, etc. So extensive validation, qualification is key. And of course, the more complex a system is more challenging. It is to get it implemented from quality perspective. And that's valid also for artificial intelligence systems, not necessarily for systematically treated search. But any artificial intelligence which may lead to decision making that may impact the safety and performance of a device or the safety and the health of a patient is very thoroughly scrutinized. So the implementation of digital systems in general is quite a quality controlled process.

and when they imply also artificial intelligence tools, it's even more difficult, but not impossible. We have now a very huge program in place about artificial intelligence, integration at all levels, not only clinical evaluation and literature searches, but any aspect of the business. and there is a lot of empowerment of going in this direction. But of course, quality standards have to be covered. Risks have to be mitigated in order to avoid erroneous decision making which may impact the safety of a patient.

Wow! I'm impressed how many efforts you're already doing in that direction. It's really cool.

Yes, and it is very interesting. Believe me. it's not only a buzzword. It's really our new normal. We have to implement properly. a biggest challenge actually is unlearning the AI. So it's an input to Romney and the distiller team. Once we have trained AI. Of course, we are training it with the data which we have today. But it is evolving. And maybe what is right today is not going to be right or maybe not complete tomorrow. So the unlearning process is a challenge, and then very important one.

Okay, great. I think we have one. Yeah, one time for one last question. Actually. So, Diane, I would like to ask you from your perspective as a clinical review at a notified body. What are the most common deficiencies that you're actually observing in literature, where you submitted as part of the technical documentation.

Oh, gosh!

Let me!

See, where will I start? I'm gonna say my number one, because I noticed the time. So my number one deficiency is literature. Searches are much more than a find, and summarize that what I really need to see is, and often don't see is a synthesis of that literature that you found. So you found it. You maybe use a tool to grade it for the methodology and whatever. But there's no critical narrative on how this relates back to an existing body of knowledge. If there's any more gaps. If anyone actually looked at those gaps and did another search or sorry, another study to try to fill in those gaps.

I really, I really sort of taking it to the next step. So you know, you're going to have all those dots that you have of your literature. Search if you will, and then you connect the dots, and then what does it? What does it say? I want? I want to see a synthesis from from the literature review.

Okay, great. So I think then we can actually already go to the Q&A session just in time.

Say I have lots more deficiencies if you want, but I noticed the times.

I can only believe. Yeah, sure, you have quite some stories to tell.

This would be a powerful input to the new guideline for clinical evaluation. Diane, like any observation from notified bodies which can help industry and the clinicians to comply would be welcome. So.

Okay, thanks for that. Yeah. I try very hard to put my thoughts on Linkedin whenever so follow me, as I am often giving a giving my thoughts there as well. But yes, thank you.

Yeah.

Thank you. So we have a lot of questions. We're not going to get to all of them, obviously, and some interesting ones. What is the difference between alternative treatments, generic slash benchmark devices, and similar devices.

That is a good question, because sometimes it's very blurry. Actually. So alternative treatments. Okay, let me let me think of a certain we have, for example, a device which is treating cancer and alternative treatments are, for example, you have a liver transplantation, or you are treating it. First, st we have a bridge to transplant. Yeah, alternative. First, st before you do something or even just medical treatment. So those could be alternative treatments to device treatment. It's a generic device group.

Can you.

We then have, for example, local regional therapies which could be, for example, I'm sorry. I'm repeating, for example. something like radiofrequency, ablation or cryoablation, or whatever. And then you have your generic device group and your benchmarks from the genetic device group, which is then a very specific type of therapy. For example, then just a certain type of cryoablation, for example, and then your similar devices are really devices which are using the same principle as your own device. So it's it's not. You do not always have similar devices. Sometimes you really need to go back to

alternative treatments, because there's just nothing there which is.

Okay.

Really treating everything that you want to treat, for example. But that's how we are usually describing the difference.

Can I add? Add to that? I would say that all of them, the alternative treatments, similar benchmark, all come down to the bottom line of benefit risk. And so when you speak to alternative treatments, you need to talk to me about what populations might be better to, as all said, to use medicine, pharma instead of a device where those that fail at Pharma maybe, then come onto a device, or for certain age or comorbidity, or whatever. So it gives me something to understand. The current state of the art, so similar also gives me something to see how other devices are doing. What are the acceptable risks. All devices have risks. Don't tell me all of your risks have been mitigated, and so you don't have any residual risks, because I'm going to come back with? A No. So what are acceptable risks

that practitioners and patients obviously willing to accept? And then the benchmark is again. How Ola talked about in her presentation. That's how you're going to meet the Gspr. Number one. That says, your safety and performance needs to be a state of the art. So this device is similar to mine. It has this many risks at this percentage, and I'm going to compare mine, and it's going to be at least as good, but maybe better. And where it's not, you're going to tell me why. Maybe the target population is sicker or whatever. So all of what those 3 come down to the assessment of benefit, risk.

To complement a little bit on both Ola and Diane's response. I actually, that really depends on the novelty of the device. Of course, we have always to screen all these options. But as Ula mentioned, some devices are so novel, so you may not have a similar device. A similar device would be typically similar technology, similar functionality, same target population, same intended purpose, same indications. But you may not have such a device. And then again, you have to screen. What are the other options? What is the current standard of care for that disease or for that indication? And that is your comparator. In factually. So we have to

have an open vision on all of those, but depending on the maturity of the technology. Some may be more relevant to the device under evaluation than others. For example, for very well established technologies, like needles and syringes. You may not even look for a lot of alternatives, because they are so long time on the market, so they have become the the only possible way. Or yeah. In some cases there are some exceptions, of course.

In a similar question. If no benchmark articles are available for a medical device, what alternative steps can be taken to define safety and performance, acceptance, criteria.

I'm happy to jump into this. Yeah, that happens more often than not, especially to meet you talked about. What do you do with a syringe and needle, those very sort of standard of care devices. Well, they'll have similars, but but not necessarily published, so it still puts you in the same place. So if you're not novel, you're going to have to go back to your own risk assessment and use it and justify it. If you're completely and utterly novel, and that you have no risk management to rely on. It is a very challenging position, and you're looking for devices that are in the generic group, and as close as possible, just to get a sort of guardrails if you will. And then you're going to really have a top notch, post-market, clinical follow-up plan that looks much more frequently. That's how I've managed it with clients

previously.

Benefit risk is something that we have to revise at any moment. So it is not something that we fix once, and then it stays for life. No, it is throughout the design control process, also from early ideation until certification of the device. This is very regularly reviewed, actually. and may change, because maybe during the design process we have new information which may change the benefit risk profile of a device. Of course, having pivotal studies before regulatory approval is giving you this kind of specific device specific information which may allow you to calibrate your benefit, risk and risk management documentation. So it is not something that you set in stone, and it's there for life. It is something that you have to challenge every time and throughout the entire lifetime of a device.

but again, depending on the novelty, you may have it as an information from similar benchmark devices, or even if you have a broad portfolio of products, maybe you have your own predicates, which are giving you already a lot of information that you can use as an initial input.

So we're about out of time. I would like to thank the speakers and the attendees for your time and attention today. Please remember to subscribe to medtech leading voice and medtechleadingvoice.com to make sure that you're updated on all upcoming webinars, and I would like to give the speakers an opportunity to say goodbye to everybody and thank you again. Please complete the survey. At the end of this we look forward to your feedback.

Thank you.

Bye. Thanks everyone. Great discussion.

Bye.

So much.

Bye-bye.

Bye-bye.