PMCF Surveys Under MDRWhen They Apply, How to Structure Them, and WhatHigh-Quality Looks Like

Hey everyone, welcome. My name is Sean, and welcome to MLVX, Friday in Focus call. We are… I'm gonna share my screen for a moment. So that you can see just how many tabs I keep open at all times. To talk about, what's coming up while we're waiting for everyone to join. We have some, really interesting things coming up. Next week. the team from Pure Global is going to be joining, talking about global regulatory submissions, authorized representatives, and in-country,

regulations, in… around the world. So if you're interested in entering new markets, South America, China, or Asia, or… Africa, then this is the event, for you. Go… let's… let's go global. The next, week is gonna be a really, practical learning. experience from Leon Dorn, who's going to be talking about a real-life, FDA AIML submission. In some of the… problems that you're likely to encounter. And then, UTIMED, you know, the UTIMED deadlines are, are, end of May.

For, product registration, and a lot of things are missing from Utimed. So, this guy. who I just met, Sam, has, developed a platform that helps people with this. So that's good to know. Next, the International MedTech Safety Conference, June 2nd, Through the 5th, in Boston. I'm going. I'm going because it's important. I'm going because this is a not-for-profit conference. This is volunteer-driven, it's organized, by Bijan Alahi, and, it's a great group with,

a lot of really, really great speakers. If you want to go. Here's discount code MLV200, or you can go online, which is even cheaper, but then, you know, you don't get to press the flesh. There's also special startup discount codes, so if you're working with a startup in the Boston area, please share this information with them, share it with your network. And then the last thing, which Anna's still working on the slide. as we speak. But, BSI is doing a roadshow next week. Monday in Massachusetts, just outside of Boston, Wednesday in Minneapolis, and Friday in Santa Clara, California.

And they're gonna be talking about, the, what's going on right now in the EU. They're gonna be talking about, a number of topics, including Annex 7 and wet technology, and I'm going to report on it, and plan an Annex 7 event with, Rich. Holboro. So, I'm excited. I have to get up at 3.30 in the morning to get to the airport on Sunday night, Monday morning, and then I'll get home at 11 o'clock at night. So, if you would like a copy of my slides, Sean's slides, and the upcoming events, please just say, you know, hey.

drop… send me… send me your slides. And as soon as they're done. It's… I like this real-time action. Also, I want to announce that at the end of this call, I'm gonna give, Jose a few minutes to introduced this cool new feature that writes LinkedIn posts for you from your profile, if you're on MLVX, and so he's gonna demo that, I think. And now, without… further ado, I'm gonna stop sharing, and I don't know why I'm seeing Fraz as the main speaker on this, but, I'm here, and I'm going to turn it over to Marcus. I've known Marcus for a few years, and, see him at lots of events. Purdy Pasco is a really, really interesting outfit. They've been specialized in PMCF,

surveys, for a number of years now, and have really done remarkable work in this area. He's very knowledgeable about it, and I want him To do the talking from now on. Marcus, where are you located?

Hey everyone, I'm in London, in sunny London at the moment, actually, but the weather's about to turn, which is a bit of a shame, but firstly, yeah, thanks, Sean, for the nice intro, and really looking forward to be able to present, on Friday in Focus today, and thanks for everyone who's giving up your time today, especially those over in Europe where it's eaten into your evenings already. So let me just share my screen, and we can get started. Okay, hopefully everyone can see that. Please let me know if not. And I know, obviously, these sessions are slightly different than the traditional, webinars, you know, people with cameras on and able to chime in and ask questions throughout, so please, you know, feel comfortable

to do that, should you need to, there will be opportunities throughout this, where we'll be pausing and potentially asking the audience here to, you know, answer some of my questions, as well as me being available to answer yours as well. So, without further ado, I can get started. And I'm going to be speaking about PMCF surveys under the MDR today. But firstly, again, with this, you know, slightly more informal style, I thought I'd just, share a bit more information about myself, firstly. So, my title is Head of Post-Market Surveys. I work for a company called Pearly Pasco. We're based in London, in the UK.

And over the last 6 or 7 years, I've helped, deliver over 600 post-market surveys, supporting the medical device IBD, industries to generate real-world evidence for MDR, IVDR, and global submissions. A few of those photos there. My professional headshot on the left-hand side, a big thank you to Sean and MedTech Lead and Voice for including me in the top 100 last year. The right-hand side, me and some of our team at RAPS. We will be in Lisbon in a month's time, at RAPS Convergence, and hopefully at a few more conferences. throughout this year, so hopefully I can meet some of those on this call today who I haven't face-to-face already. And then the picture at the bottom is our wonderful post-market survey team, which is now 19 people strong.

However, you know, that's a bit more about my professional sense, self, sorry, a little bit more about me as a person. Outside of being head of post-market surveys. I am a husband, a dad of two, a keen traveler, and a sports fanatic as well. I may have delivered over 600 post-market surveys, but many more nappies and diapers that have been changed over the last three years or so. I'm a frequent traveler to my favourite spot, Menorca in the Balearic Islands, and regularly skiing down the French Alps, where I was a few weeks ago. I say sports fanatic, it's mainly football, a bit of golf as well, but a picture at the bottom of my,

old football team where I was player manager for 5 years, and then, an avid Manchester United and AFC Wimbledon fan, and the pitch in the middle is, me taking… well, my little… my eldest son at his second game on Monday earlier this week, which was great fun. So, how'd I get here? My, sort of, career or, start in life sciences began at the University of Bath, where I did natural sciences, a chemistry and pharmacology degree. That was split up by a placement year as an analytical scientist at AstraZeneca. Before returning back to the University of Bath to finish my master's degree. I then had to embark on my career, thinking about what was going to be next. I briefly flirted with patent law, and quickly realized that that wasn't for me, or I wasn't for the many people I was interviewing for.

And then kind of fell into, healthcare market research and predominantly med tech market research. when I joined Purdy Pasco in 2015, A fresh face, myself down there as a graduate with our old logo and branding. And I was working in the market research side for around 3 years. However, an opportunity came my way in 2018, When one of… one of our med tech Market research clients passed on our details from their marketing team across their clinical team. I wanted to leverage our experience in quantitative surveys, but instead of collecting information to inform marketing decisions, it was to, obviously, much more focused, help collect clinical data, and predominantly PMCF data, through PMCF surveys.

And I've been leading that team for the last seven and a half years. So that's where we are today, and that's why I'm here, talking to you guys about PMCS surveys.

Any questions about diaper changes, skiing, football? Or any related topic, or PMCF for Marcus before we get started? Okay. Okay. I'll let you dive in.

Thanks, Sean. A little bit more just about the company here. I think I've mentioned most of this, especially in the top left. You know, team of 15 people strong, 600 surveys, and we're a full-service, end-to-end agency. A little bit about our services, you know, we have a GDPR-compliant, regulator-approved approach. We have in-house programming, in-house medical writing, and in-house quality control to ensure that all the data we provide is, one, fully compliant, Accurate, and then also, especially with the programming, we have, you know, real quality and cost efficiency and cost effectiveness as well.

And with the 600 surveys we've conducted, the large majority of those have been PMTF-focused. There's been, you know, the last few years, a little bit more around, sort of, wider real-world data. when, you know, the MDR was around the corner, we started doing some surveys where there was very little guidance, and they were very much more in that sort of general Level 8 approach, and that probably lasted for around, sort of, two years, where we saw the majority being in those Level 8 surveys. However, over the last two, three years, you know, there's a lot more scrutiny on surveys, a lot more guidance as well, and we're seeing a lot more of the

sort of high-quality, Level 4 surveys being implemented, and I will focus on those a little bit later in terms of what, you know, what makes it high quality and how can you best design those. In terms of what we do, I said we're full service, you know, we can help with the planning, the design, the implementation, and then the back-end data analysis and reporting as well. But just before I sort of start looking at, PMCF surveys, in… more detail. I think it would be good just to start looking at, the benefits of, sort of, real-world data versus traditional clinical studies, with real-world data being a very hot topic at, you know, on both sides of the pond at the moment.

And I think, you know, real-world data gives, as it says, real-world clinical practice insights outside of those more controlled, you know, clinical study conditions. And this allows for, you know, many, I guess, benefits or things that can be taken from that. So, it can allow you to evaluate a broader range of clinical questions. Include populations that maybe are otherwise underrepresented in clinical studies. Able to look at longer-term outcomes. And we know there are those issues when it comes to, you know, full-blown clinical studies, you know, impractical, challenging, and the, you know, the ethical side of things as well.

And what this allows you to do is collect that naturally occurring real-world device usage, potentially looking at patient experience data as well, which would all feed into new insights into device performance and safety. which can inform future device modifications, new tech development, and also align with device innovation cycles. And ultimately, what you're looking there is to assess that device risk-benefit profile for real-world populations. And I'm just going to pick out a quote that was taken from some, you know, FDA guidance, quite a while ago, but I think it's important, and it chimes with me quite,

quite heavily, where, you know, analysis of real-world data using appropriate methods, which I think is, you know, the critical word there, appropriate methods, may in some cases provide similar information with comparable or even superior characteristics to information collected and analyzed through a traditional clinical study. And, you know, we're going to be looking at the MCF surveys, throughout the rest of this, hour, and, you know, that falls into that category of real-world data as well. Okay. So, just a first disclaimer, this is the… I've got a few slides here, I promise this will look… they'll be the only slides where I'm going to run through, you know, MDR documentation or guidance. No one needs me here repeating stuff that is known already, or they can find, elsewhere. But I think it is important just to set the scene around where PMCF falls into the…

the EUMDR. So, you know, the MDR Article 2 introduces the term post-market clinical follow-up here when it looks at, you know, clinical data and where it could be sourced from, stating that it can be sourced from clinically relevant information coming from the post-market surveillance, and in particular, post-market clinical follow-up. And it's free, also mentions that, the PMS plan should cover at least a PMCF plan, or at least a justification as to why a PMCF is not applicable. And then, under MDR Annex 14, Part B, it states here that PMCF should be understood as a continuous process that is used to update the clinical evaluation.

And then there's a few other references list there at the bottom. And then just some guidance, which I will touch upon some of these later on in the talk, later, talk today, but, you know, good, guidance or resources in MedEv, and also, relevant MDCG guidance as well. But moving back, sort of, again, taking that guidance from NX14, I think it's important, again, to reiterate that, you know. MDR considers PMCF as a continuous process that is used to update the clinical evaluation. It doesn't see it as just a one-stop, you know, tick box. So really what it's asking is manufacturers to be much more proactive in assessing the safety and performance of the medical devices continuously throughout their product lifecycle.

I put together this sort of cyclical diagram on the left-hand side, just simplifying it, but also showing that you need to create your PMCF plan, go out and collect your PMCF data. analyse that data and feed it into your PMCF report. And then, very importantly, is using the learnings or data there to update relevant clinical documentation, such as risk management, design, your PMS plan, and then back into your PMCF plan, so it can help inform upcoming cycles of PMCF data collection as well. The objectives down the right-hand side, with the main objective really being, you know, confirming the safety and performance of the device, but also looking to identify any unknown side effects, any emergent risks, looking at that benefit-risk ratio, and also, to help identify any possible systematic misuse of off-label usage as well.

When it comes to data collection methods, I've listed just a few here, but we've mentioned, you know, clinical studies, clinical investigations, giving you that rigorous data collection, high reliability and control. Which can be essential for high-risk devices, or when other methods, such as surveys, for example, are insufficient. Maybe there's a lack of preclinical data as well. However, we know that it can be costly, time-consuming, and, you know, face, feasibility constraints as well. Look at registries, again, you're getting that, you know, robust level of data. It really helps with longitudinal data collection, and again, for high-risk devices, particularly implantable devices.

But again, can be costly, slow, and sometimes misaligned with PMCF objectives, but they can play a crucial role in post-market surveillance by providing that long-term safety and efficacy data. Surveys, which we will focus on today, they give you that flexible, scalable, and case-specific, or general usability, depending on the survey approach, data collection method. There is greater regulatory acceptance at the moment, but importantly, only when the methodology is robust and sound. And it can give you that fast, cost-effective data collection tool globally as well. However, we need to be aware that they aren't appropriate for all devices, there are limitations, and it can result in lower-level data as well.

Other methods I've littered there, you know, complaints, feedback, literature reviews, quicker and cheaper, but, you know, potentially, again, there's lower levels of evidence, so… I think it's important to know that, you know, it isn't just one of these. For each of your device, it might not be, is it a registry or is it a survey? You know, there could be multiple data collection methods that you take on for that device. It's just making sure they're appropriate, well justified, and, And that methodology, again, is robust in sound. So, to sort of walk along a very simplified clinical data decision tree here, you know, firstly, what do you need to collect your clinical data for? Is it PMCF purposes, PMPF if looking at IVDs, or is it wider real-world data? So, you know, label expansion, or something like that?

Once you've determined what the need is, you need to think about what your approach is, and it could be, you know, one or several of the approaches listed here. you know, if you've got a high-risk device with a lack of preclinical data, a clinical study is going to be needed. A longitudinal implant… an implantable device that needs longitudinal data, potentially a registry is needed. But surveys, you know, we're focused on them now, so I'm going to take you along that decision tree if a survey is selected. it's not as simple as going, okay, I'm going to do a survey, here's 10 questions, let's send it out and get the data. You really then need to think about what survey approach is appropriate. So, is it a high-quality

Level 4 survey, or a more general Level 8 survey. And I'll touch upon the differences between the two, shortly, as well. But again, once that's selected, it's not as simple as just pulling together some questions and sending them out. You need to think about a very thorough, rigorous, survey plan or survey protocol that will outline and justify your methodology. Look at the design and think about how you're going to implement the survey as well. And then, again, this is just listing how it can help. So, PMCF surveys, you know, the traditional sense of what we do is, you know, helping manufacturers proactively gather that real-world data on the safety and performance of their devices.

PMPF similar can be a bit more complex in terms of who are the touchpoints and who are the, you know, the actors or the end users that you need to get data from, but help and provide real-world data on the performance of diagnostic devices. But we are seeing a lot more requests come in around wider real-world data, you know, such as devices outside Europe for FDA or NMPA submissions, or indication of label expansion as well. So I'm going to pause, firstly, just to see if there's any questions on what I've mentioned already, but also, it would be great if I could just see… I'm just gonna make sure you guys are all back on my screen, and I can see any reactions coming in.

But just a show of hands or a reaction to any point on who has used or implemented PMCF surveys.

So I asked that, in the chat a little bit earlier. Has anyone ever conduct a PMCF survey with a client or employer, and we have a number of responses. Yes, I've developed PMCF surveys for Class 2 and 2B… 2 and 2B catheters, and have been ongoing discussions with Notified Body. Till, we reached final approval. Which brings up one of the questions, which was, you say in your About Us that you're in partnership with Notified Bodies. Could you explain that?

Yeah, I'd say, you know, we, we do a lot of work with, several notes of my body's webinars like this, for example, but also conferences and, you know, we presented, alongside the likes of BSI at, you know, many conferences over the last few years. We've also got an article That is about to be published in the JMDR, so the Journal of Medical Device Regulation, later this month. that will be behind a sort of paywall or subscriber wall for a while, but we will hope to have open access where we can share that with the wider community. That's alongside BSI. I will have a QR code later to download some of our previous articles that we have, we have published in that publication. But yeah, really, you know, we have a very sort of strong relationship with several, you know, we've worked with the likes of DMV and TUB as well on other webinars.

So, yeah, the feedback we have received, you know, I would say the partnership is more so in terms of that sense, rather than working directly with them. But we, yeah, we're sort of aligned in terms of. the approach that should be taken when it comes to, you know, PMCF data collection, and particularly survey data collection as well.

Will your slides be available? Can I share them after this?

Yep, Cecil.

And then, here's another question. How do you handle, different languages when doing PMCF surveys across Europe?

Yeah, so we would always recommend surveying in local language. You know, one thing is, you know, a more general survey to the public. You could probably get away with doing, you know, keeping them in English, but when you're talking about, you know, medical terms and so on, I think it is important to try and ensure that the local languages are used, one for data accuracy and so on. our platform that we utilize, we can always program the survey usually in English first, get it tested and validated, and then, we can get the survey translated. We work with partners that are specialists in medical or healthcare translations, and then we can relatively quickly overlay those translations directly into our survey platform before distributing to the relevant markets as well. Same token, when we get data coming in.

We can do back-ended translations on any open-ended responses that come in, but we also make sure that with our outputs, we always provide the open-ended response in the local language and our translations to make sure that there's, you know, no issues with, you know, incorrect translations or anything like that as well.

So here's an interesting question, which I don't really know the answer to. Has anyone used surveys for a Class 3 device that has clinical trial data? Is that even possible? Can you do… can… Can you use PMCF surveys for Class 3 devices?

Yeah, we… we… We have… we have done. I can see if anyone else got answers there. I've actually got a case study later on, which is an interesting one. It's on a Class 3 guide wire, so relatively, you know, simple, you know, well-established device. but is deemed clash-free. So I think that it was an important point when we look later, when we're trying to define, you know, which levels of survey do you need? You know, you've got that guidance in MDCG 2026 stating that, you know, well-established legacy, or legacy devices, especially Clash 3 or Class 3 devices must have that minimum level 4 data, so if you're going to go down that approach, you're going to have to justify it, and you're going to have to ensure that it falls under that high-quality level 4 survey, if appropriate.

Maria E. Donawa, MD: Yeah, let me… can I just, say something briefly? Yeah. You… to Sean's question, you can use these data for just about, Maria E. Donawa, MD: Let's put it this way. You can use the data to support safety or clinical performance. Whether or not it's considered substantial or pivotal is another issue, but there's nothing to prevent you from actually using it. The quality level is gonna impact how important it is to the particular device. Maria E. Donawa, MD: And the relevancy in terms of conformance to safety or clinical performance.

That's awesome, thank you for that. So, you want to move forward?

Yep, sure, thank you guys, really great questions, and hopefully some more to come. But I think, you know, my question is usually here, who's used surveys, and then also, you know, who's had success using surveys? And I ask that question because we still see, you know, going to lots of conferences and, you know, presenting at lots of webinars, is there's still mixed opinions and limited success when it comes to surveys. And I think, you know, the main issue we see when there is limited success is a lot of people understand, well, you know, I will talk about and shout about surveys being a cost-effective tool, and I think that is true, but I think lots of people forget about the word effective when they're setting up surveys. They quickly see, you know, it being a cheap option, so they rush to that.

So those, you know, those mixed opinions we hear, asking whether surveys are appropriate, I think the simple answer is, yes, they are, for the right device, and if you can justify it for that device. You know, I'd love to be sat here as a… a service provider saying, you should do a survey for every single one of your devices, but that just isn't appropriate, obviously. So the issue and challenge we see from our experience is those that have limited success, there's sort of one key similarity, is that poor and adequate planning at the early stages, so not thinking about whether a survey is appropriate. If it is appropriate, not thinking about how to

sort of set up a proper, robust protocol with, you know, statistical sample sizing, properly defined endpoints and objectives. And then also, even if you design a really strong protocol. then designing the questions that just don't align with those objectives that you've set out in your… in the protocol, so you suddenly get data that doesn't align back to what you set out originally. So, that's definitely where we see that, mixed success. And without that proper planning consideration, you know, a survey which you think is pretty cheap quickly becomes expensive if you need to repeat it, or you need to disregard the data you've collected from it.

So, yeah, I mentioned the two different, sort of, survey approaches, and just where does this come from? I've list… I've got here the… the… table of the, sort of, clinical data hierarchy from MDCG 2026, and it's free. And surveys are mentioned, at two of these ranks or levels. So, starting with the Level 4. These… what it states, firstly, outcomes from studies with potential methodological flaws, but where data can still be quantified and acceptably justified. High-quality surveys may fall into this category. The one issue is there's nowhere that actually defines what does it mean by a high-quality survey, so there's still little guidance out there around that.

However, I've just got a little bit of a blurb here, just… what we sort of see that as meaning, and we'll go into more detail on later slides around what a high-quality survey is, but, you know, these are going to be used for higher-risk devices or those with larger clinical gaps. Again, just to reiterate, in MGCG 2026, it states that, those Clash 3 devices must have that minimum level 4 data. Here, each survey represents one individual patient or case in which the device is used, so you're asking respondents to complete a survey following each usage of the device. They may complete multiple surveys if they've used the device multiple times for multiple cases, could be a mix of retrospects and perspective, and it enables you to meet those objectives and endpoints through much more structured data collection. So, you know that if there was an adverse event, for example.

that was encountered, you can tie that back to a certain patient age group, for example, or a certain variant of the device. Whereas the Level 8 surveys, which I'll touch upon later, are going to give you a much more, sort of, aggregated, high-level understanding of the data, which doesn't give you that structured, specific data collection. However, they can be more clear to what you're asking. Also. More expensive in addition as well. So, Rank 8, it takes proactive PMS data, such as that derived from surveys. We sort of term these as more general usability, or sort of HCP surveys. Used usually for lower-risk devices, or those that have smaller clinical gaps.

Here, each survey represents multiple device usages, so you're asking respondents to recollect information based upon usage across a set time period, which could be the last month, it could be the last 12 months. So it's a one-off survey, it's solely retrospective, and because of that, you get a much lower level of evidence due to that recollected nature, and also the data is going to be much more aggregated, in a sense, as well. That being said, it is quicker and less burdensome for respondents to complete. But definitely something worth considering when you're… you're thinking about a survey, is what type of survey is appropriate for your device and can be justified for it.

So, just again, those of you who did raise your hand and say, you know, you've worked or implemented surveys. It'd be great if anyone wants to drop in the chat, or if anyone wants to speak up, just about… have there been any obvious challenges when you've been conducting or implementing surveys? response rate is coming in. Yeah, lots of people saying that. Okay, that's interesting. Okay. Well, as more come in, that's great, keep going, and I'll try and review them as they come in, but I just… in case no one did answer the question, I went to my, favored AI agent at the moment, Claude, just to see what, if I asked this question, what came out.

And it was quite interesting. So, defining a meaningful clinical objective, so more broad, generic survey goals rather than anchoring to a specific clinical question. that comes out as one of them. Achieving a representative or sufficient sample, so leading into that response rate, getting the right respondents, enough of them is difficult. Over-reliance on a single-site KOL network is definitely something we see, manufacturers struggle with as well. designing questions that generate that defensible data, so I think, again, vague questions that don't tie back to those clinical objectives. So, you know, if that is the case, notified bodies are going to challenge whether that data actually evidences that device performance or safety.

integrating survey outputs into the broader evidence chain, so treating it as a standalone deliverable rather than tying it back to your PMCF plan and wider clinical documentation. And then just making sure that the survey is sort of fully compliant, I guess, around documentation and audit readiness. So, you know, is there a process behind it for, you know, methodology, respondent selection, data handling? is there an audit trail, you know, validated survey software, those sort of things. So, you know, it was interesting to see what came out when I asked that question, and there are definitely a lot of the challenges that we see, on our side as well. All clients come to us,

querying those, and I think I'm gonna try and walk through some best practices on later slides, but that, you know, that response rate one is definitely one that we see a lot of, and I think… I don't know if I touched upon it too much later on, but we… I think it's very important to understand, if you can understand where your user base is, so look at sales, do you have your own contact as a manufacturer? If you don't, how are we going to identify those users? We work with third parties and these panels or networks of healthcare professionals. Which, you know, they're signed up for knowledge-sharing purposes to these networks, or market research surveys, but now more so these post-market surveys.

And if we're looking for orthopedic surgeons, for example, in Germany, Spain, and France, we can go to these networks who have them profiles, and we can screen them on their inclusion criteria, or we can validate their usage of the device, and then get them to complete the survey from there. These are people who are willing to, and also very comfortable with, you know, taking part in surveys as well, so you see a greater response rate, and it can definitely help Further down the line, if you don't have those contacts yourself as a manufacturer. Also, you've got to bear in mind, if you do have those contacts as a manufacturer, are they, you know, high-volume users? Are they KOLs? You know, is that going to bias your data as well at the back end? Because, one, you're the one reaching out to them, so do they want to report on, you know, negative outcomes?

But also, are these just the more experienced users of the device? Is that giving you a representation of, you know, the true usage of the device both in experienced users to your… those KOLs as well. So, yeah, we definitely see… see a lot of that, I will move on just because I'm looking at the time, but I know lots of other challenges have come in, and I think I will hopefully touch upon those on the next few slides as well. So, just quickly, why to use surveys? You know, they are fast, they are quick, they're cost-effective, they are accepted by notified bodies, they're scalable, so you can reach healthcare professionals globally and virtually.

enabling that proactive data collection can also potentially help support wider business decisions as well. You know, I will always say, don't turn a PMCF survey into a marketing survey. But, especially for the SMEs where, you know, you could get additional data that can help feed into other things, then why not ask a couple of extra questions that can help you as a wider business and add value? I think if you look at the alternative methods, you know, you've got that rich, robust data through clinical studies and registries, but they are expensive, they are slow. Literature reviews and general user feedback, you know, low cost and fast, but low level of data. I think surveys really hit a good balance between the two, especially Level 4 surveys that can give you that

High level of data, while also staying as being, you know, relatively quick and cost-effective as well. But, as I said, it's really important to know their limitations and when not to use them, so… Lower level of clinical evidence, you know, compared to the other methods I mentioned there, and also noting that those general usability level 8 surveys are not appropriate for clash-free implantable devices. There is longitudinal data difficulties with surveys. Much harder to capture long-term safety and performance data. only really possible through a high-quality survey if certain device criteria are met. For example, follow-ups are performed, performed by the same physician, and there's documented in, you know, in records, so you can go and collect that data, through a survey.

It's not suitable for all devices. High-quality surveys must be used for higher-risk devices. Surveys are not appropriate for devices likely with limited pre-market clinical evidence. There's those endpoint statistical challenges we see. You must build your surveys upon endpoints and acceptance criteria. Ideally, statistical sample sizing, feeding off those endpoints and acceptance criteria, but it can be difficult to set these and to collect data to align against these with surveys as well. Potential for bias, there's selection bias, especially if there's manufacturing involvement in the recruitment process, and could potentially then be a lack of representation as well. There's also recall bias, especially with those Level 8 surveys that are based solely on recollection.

Then feasibility constraints. Recruitment can be challenging, especially for low volume or specialist use devices, and response rates can be low. I think one thing I mentioned there with the panel networks that we use, we also do in the most part, use incentives, to reward people, not reward, but compensate people for their time. We try and keep surveys relatively short, you know, 15, 20 minutes maximum, so the rates aren't crazy when we make sure we work within fair market values, either set by the manufacturer or the going rates in that market as well. So hopefully that, again, looks at some of those challenges that some of you guys may have had.

So, I guess the question here, again, feel free to, add any comments in the chat, or jump up, you know, feel free to speak, and respond, but has anyone got any ideas of, you know, what makes a high-quality PMCF survey?

I'll go. I mean, for… For me, it's… it's… actually answering a question. It's providing real insight into something, rather than just performing a survey. One of the things in marketing surveys, for example, they ask questions that they don't really care what the answer is. You know, or they're, they're asked to make the, the, people feel good. about themselves, like, oh, I wanna, I wanna hear compliments, but… It's not really driving toward, a decision. And since you're specifically trying to support clinical evidence, you know, it's really important that the questions be, you know, few and targeted and going to generate an answer that's going to

Drive a future decision. Otherwise, you're just asking questions for the sake of asking questions, and that's… A waste of everybody's time. That would be my opinion.

Yep. Nice. No, I think that's, definitely a critical key point, and I think, you know, similar things have come into the chat as well, you know, well-structured questions that give you that valuable data, you know, ensuring that the questions align with those, Safety and performance endpoints as well. So I think, you know, validity, reliable data as well. So, yeah, they're definitely, key points when it comes to those high-quality surveys, and I'll, I'll just move on to my next slide, which sort of outlines some of those points, I think, that we just touched upon. But, you know, scientific rigor in the design, I think I've mentioned protocol design. It's not just jumping into asking questions, it's thinking about, you know.

what Notified Bodies are wanting to see now is, imagine if you're doing a high-quality survey, you're setting it up with a protocol that is very similar to a clinical study protocol. Obviously, where there are differences, because it is a survey versus a study, but you do need to try and think about all of that before jumping straight into it. So having that pre-specified rationale to tie to clinical gaps, having a formal plan and protocol that Talks upon that is really critical. Linkage to specific PMCF objectives, which we've just spoken about, you know, those safety and performance endpoints. So you ensure your questions, your data align with robust and clearly justified safety and performance endpoints. You know, they should be pulled from state-of-the-art reviews, any clinical data you have already on the device as well.

And then that really ties into that statistical sample size justification. So, you know, calculating… calculating utilizing those acceptance criteria and expected rates that are determined from, you know, state-of-the-art or information on the device and the evaluation. you know, quality data, valid is your data we mentioned, data quality integrity controls, bias, you know, has just been mentioned in the chat as well. So making sure that you have proper monitoring and quality checks in place for the data to make sure that you have respondent validation, so you know who's answering… the people who are actually answering the survey are users of the device, you know.

critical. There are limitations to surveys. You can't always be there as they enter the data, so you have to try and think about how do you mitigate all of those risks that are… that go alongside those limitations with a survey. You know, ensuring, is there a possibility for missing data? If there is, how do you handle that missing data as well? make sure questions have appropriate clinical context, you know, Level 4 surveys, you know, they can be based upon patient charts or records. Very, you know, it's a… it's an area as well you need to make sure you're… you're following any, you know, local regulations and so on, you know, does it need EC approval? Does it need IRB approval? Can you avoid it by not collecting any patient identifiable information, and so on?

But also ensure that if you are trying to do a Level 4 survey, where you need respondents to go back and look at specific information on these cases, is it even possible for that device? Are they recorded, or are you still relying on recollection, you know, on those group physicians to take… Completing these surveys, which would, again, potentially, you know, risk that survey being defined as a Level 4 survey. Having validated or clinical… clinically meaningful instruments, and when I say instruments, sort of this validated survey software, you know, thinking about relevant guidance, you need to, you know.

comply with ISO 14155, Section 7.8.3, you know, CFR 21, Part 11, and make sure you build in critical user acceptance testing, and that is documented as well. you know, SurveyMonkey doesn't… it doesn't work, you know, you need to have… you need to have another, tools that are properly validated, and you can… you can sort of justify them and make sure they're audit-ready as well. Representative and adequate sampling, so make sure your population reflects the intended use. Also, coverage of age populations. If, you know, if you state that, your device can be used for the pediatric population and neonates, do you have data that can feed back to that? And also, you know, you might have different variants or sizes, making sure you've got data across all of those.

And then finally, transparent reporting. You must… your output must report findings honestly, you know, include negative data, include the limitations or any deviations from the protocol as well. So… Some best practices, sort of looking at quality and quantity, really. So, the article I mentioned that, we're just about to publish alongside BSI, one of the quotes which, from Breeder, Kearney at BSI in that article states that, you know, real-world data collected by PMTF activities must be of sufficient quality, so completeness, accuracy, consistency, reproducibility. Transparency and relevancy, and also quantity, and to enable those valid and reliable conclusions regarding the safety and performance to be made.

So how can… how can we ensure that there is that quality and quantity? I sort of said this a few times now, so I know I'm repeating myself a little bit, but planning and design is critical. Clear survey protocol. You know, list out the device information, objectives, endpoints, acceptance criteria, what's your inclusion criteria, what's your statistical sample sizing, and what is your recruitment plan, and then make sure your survey aligns with that protocol. that recruitment plan, you know, what are the inclusion and exclusion criteria? What countries are you going to target? You know, your device might be sold across 20 European markets. Do you really want to go to all of those markets and have translation costs and so on? So, you know, you want to have representation, but you want to make sure it's cost-effective as well. And then think about how are you going to get the survey. You know, your sample size might come out and say you need 200 cases.

Yep. what happens if you've only sold the device 100 units in the last 2 years? You know, it's not feasible. Similarly, if you need 200 cases, how are you going to get the survey to that amount of respondents? So, do you need external support, for example, or can you manage that internally? Validation and quality assurance, so use a validated survey software, have user acceptance testing, and have quality control of the data and respondents. And then also, critically, in your protocol, have backup and shortfall plans. You know, list the countries you're going to target, but have an option to expand to other countries.

you know, potentially the US, for example. I know PMCF is a European you're collecting data for European submission, and European data is the gold standard, but you can justify use outside of the… outside of Europe, for example, the US, where patient characteristics aren't dissimilar, and use of the device shouldn't really differ from, you know, Europe to the US, so you could put that in place. And have flexible quotas as well. So, you know, have target quotas, but if you don't hit them, you know, how can you justify that, and then make sure you elicit their deviations in your report? So here, it's just saying that that article I mentioned, so look out for that article. It is going to be sufficient clinical evidence for medical devices and what the role of post-market real-world data is. We did have an article, last year, which looks at high-quality surveys, so feel free to download this article, that's open access. You can scan the QR code, and it'll be in the slides at a later date that Sean can share, and at the end of this presentation as well.

But the overall article will be published over the next couple of months. quickly, I know we're sort of 10… well, 10-15 minutes left, so I'm going to quickly just… the survey process here, and then I want to run through a case study as well. You know, it's that sit through a process. Get your clinical documentation, your CR in place, get your survey plan and protocol designed. survey design, programming, data collection, data processing, reporting, and then make sure it feeds back into those relevant documents. We usually work on a sort of 3-6 month timeline. There's variability in that, mainly due to what is the sample size and how easy it is, is it, to recruit the end users, but that's what we're usually trying to work towards.

And finally, what do you do with that data once it's collected? So, you know, align it with those endpoints and acceptance criteria via statistical analysis, review the data for complaints and adverse events. You might need to pass it on to complaints handling. How are you going to do that as well? Is that an ongoing basis as data comes in, or is that at the back end to make sure that, you know, there's cross-functional buy-in in terms of when that data comes in, they're ready for it. Feed it into clinical evaluation risk management, and then also those next rounds of post-market data collection. It's a continuous process, and this data can be reused, or not reused, the survey can be reused, but the data and the learnings from it can help update it for next cycles as well.

Okay, just before I move on to this case, I should be able to run through it in about 5 minutes or so. I just want to see if there's any other questions. I know quite a lot have come in, so I might not be able to answer all of them right now in this time, but if there's any ones that people would like me to… to touch upon, please do feel free to shout.

Maria E. Donawa, MD: Privacy, please, if you can do that briefly. If not.

Yeah, no, sure, I think, you know, privacy is a critical thing. Obviously, you know, GDPR, for one, also, you know, patient privacy as well. So, you know, majority of the surveys we're doing are with healthcare professionals based upon cases that they've used a device, you know, obviously in a patient. So. Firstly, the thing is the privacy of the healthcare professional to make sure they know you know, what data you need from them. So, you know, we're collecting adverse events. A lot of manufacturers need to know, would like to know who it is who's completing the survey. So you might need to collect their information, their name, their contact information. So you need to make sure you very clearly list, you know, or allow them to consent for what information you're collecting from them, firstly, to be compliant with GDPR and so on.

When it comes to, obviously, the… Further down the line, you know, collecting data on cases. depending on what data you're going to be collecting, that is going to, you know, feed into, do you need informed consent? Do you need IRB approval? Do you need EC approval? You know, we try and set up most of our surveys where we're collecting information on that case, but we're not collecting any information that is going to be patient identifiable. You know, we're minimizing as much information that will be able to identify a patient. If anything, it's age group and gender, but, you know, it wouldn't be what is the age of the patient, it would be age ranges to, again, minimize things like that. So, you know, there's times where you might need to collect more information. If you do, then that's where you need to start thinking about that approval process. How does that work? Do you need, informed consent and so on.

And how is that going to be logistically possible? Because that could really impact, one, feasibility and timeframes as well. Okay, and just one more question that came in, how do you assess survey feasibility? Across the issuance and in… Yeah, so feasibility, I guess, we… when we cost, you know, we look at our fixed costs, it's pretty easy. The harder thing is, you know, okay, you need a sample size of 200, but how… is that possible? We always ask for as much information as possible from a manufacturer, so sales information. We have a good feel for if the device has a certain amount of sales, whether that'll allow you to get to the number or not.

But also what we're able to do sometimes is do some sort of mini feasibility surveys or pre-surveys. So we can go out to those networks I spoke about and go, okay, let's just go to a subset of them, maybe 100 of them across 4 markets. And, you know, let's send it out to, general surgeons who use a specific device, for example. ask them what procedures they perform. Okay, in those procedures, what devices do you use, and so on. So we can have a good idea of, in that 100, you know, subset that we've looked at, 20% of the respondents use the device in question, and then we can look at, okay, on the wider panel, how… what numbers do we have for that specialty? And it gives us a good idea of the numbers we'd be competent of achieving as well. So, there's definitely ways that we can help with

You know, looking in feasibility and trying to ensure that we're not, you know, the worst thing is to over-promise and then to deliver, so, that's key. Okay, so finally, just a quick case study. It's that Class 3 Guide wire I mentioned earlier. So. You know, a manufacturer here was looking at collecting robust PMCF data to confirm the ongoing safety and performance. The study was designed to evaluate two key endpoints, so procedural success and freedom from adverse events. There was also a challenge that data was required across all available guide wire sizes and age populations to address some clinical evidence gaps that had been identified.

So, step one, determination of the survey approach. So, what do we need to consider here? Well, let's look at that guidance. You know, MDCG 2027 and 6, you know. firstly, MDCG 2026, you know, class-free devices must have a minimum level 4 data, and then also, you know, 2027 says that the methodology must be scientifically justified and proportionate to that risk as well, and tied to clinical evidence gaps. So, because of that guidance, we've got a clash-free device, we want to do a survey, a Level 4 survey was selected as the most appropriate approach. This provides that structured clinician-reported patient-level data, where passive surveillance was insufficient, and a full clinical investigation was deemed disproportionate.

We need to then design a robust survey protocol and questionnaire. Looking at that guidance again, Anisal 14145, you must have a formal protocol, pre-specify things as objectives, endpoints, and then eligibility criteria, sample size justification, and methodology. So we designed a thorough survey protocol, listed out safety and performance endpoints and the associated acceptance criteria that had been determined through a literature review of the DUE and the SOTA. The performance endpoint was deemed as procedural success, so successful advancements of the guide wire to the target location, an acceptance criteria of greater than 90%,

And we were looking at freedom of adverse events with an acceptance criteria of less than 4%, or less than equal to, sorry. The sample size was justified feeding in from that information, so we took the acceptance criteria, we took the expected rate of the device, and looking at the 95% or 90% confidence intervals. What do we need to… what amount of data do we need to achieve to… to be, you know, confident that we're going to fall within that… that criteria? And the survey was designed to ensure the data was collected to align with these endpoints as well. validated survey software was utilized, so again, having audit trails, controlled data access, GDPR compliance, user acceptance testing as well, to ensure that the survey logic, routing, and question formats are working as they should.

And also, a balance between having a survey that is also easy to use and understanding for respondents as well. So, that validated survey software was used, and user acceptance testing was performed with, any changes to the survey formally logged as well. Data collections and sampling, we need to consider the recruitment strategy, as I mentioned, and how we're going to get the survey to the end users. We set minimum quotas for those guide wire sizes and age group populations to ensure that those clinical gaps were met. We targeted high sales markets in France, Germany, Spain, and the US. US was used as supplementation if we struggled in the initial markets, and then we could go to the US afterwards. Benefit there is, as well, there's no additional cost, because no translations.

And also a pretty cheap market to recruit in. We did use the panel network I mentioned, and relevant users were screened to ensure the inclusion criteria were met before completing the survey. The data was collected, we needed to analyze the data, it needed to align the pre-specified statistical analysis plan, as per the protocol. And results were collected with procedural success of 95%, exceeding the 90% target, and adverse event rate of 2.5%, falling within that 4% threshold. And also, data was a subset We looked at the, sort of, cross breaks of age populations and sizes, and there was consistent performance and safety across all age cohorts and device variants.

Final thing then was a manufacturer, really, was to think about, you know, what next. So, under the MDR, you know, the findings must feed into their CER, the PMSR, the PSUR, and the SSCP. So the survey findings were incorporated into the CER, confirming the safety and performance across the full intended use, and the structured, protocol-driven subgroup stratified approach, fulfilled, obligations, and provided a repeatable model for future PMCF cycles. So that's just a, sort of. simplified case study of, you know, how a Level 4 survey can be used, and the steps you need to consider when doing so.

And that takes me to the end just about in time. So, thank you very much, everyone, for sticking with me today on a Friday afternoon over here in Europe, and And, yeah, much appreciated. Here again, feel free to download the article here. I'm happy to answer any questions in the next couple of minutes, but, you know, you can also, add me on LinkedIn. If you want to connect on LinkedIn, you can scan the QR code here, and I'm, available. Yeah, LinkedIn, email, just feel free to drop me a line, and we can jump on a call or discuss further.

I'm also going to send the chat file, in the Q&A to you, along with, you know, attendance report, so that you get an idea of, you know, what… What questions were being asked?

However, that'd be great.

And, I… we only have a couple minutes left, so I want to turn it over to Jose, but before I do that, I want to mention that Andre, who's on here, his product, is, been nominated, for… one of the coolest things invented in New Jersey last year, and there's voting, and I put that in the chat. So please turn out the vote for Andre's, VetSonic, vet… electronic, vet, whatever, it's this cool thing that helps, pets with arthritis. So if you're a pet lover, and you've ever had a dog, or cat, or animal with arthritis, please, vote for his cool thing. And you have until the 13th, so, vote early, vote often. Now, Jose?

Oh, and I also put a link in, Michelle Lott now has a Substack publication, it's awesome, you should check it out, you should subscribe, it's free. The link is in, the chat, it's also LeanRock… leanraqa on .substack.com.

Nice. Alright.

Okay.

Here's your quick update. It's been a little while since we've looked at this. So, I'll give you one quick update on the chat itself. We're now tied into all of the various FDA databases, so…

Amazing.

Can you guys hear me there?

Yeah.

Okay, cool. Yeah, so if you just go on a new chat, you can say something like, hey, what FDA databases do you have access to? And it'll list them for you, and then if you ask it questions about a particular code, I don't know, like… You know, what code is related to… Defibrillators. it'll look that up, it'll give it to you, it's tied to, like, the mod database. And so, it's cool, because it'll pull that data, and then it'll put it in the context of a large language model. It'll also pull information from, like, other experts who might have thoughts about your particular product code. Etc. So that's kind of a cool feature you can play around with, let us know if you run into any issues or have questions.

And then the other one, you know, remember this is, like, a two-sided system, so there's… one side of it is, like, just, medical device professionals who can come on here both to get information through the AI, or to connect with experts, like, for example, if we come on here. We can see that Marcus, our wonderful speaker today, has his own little profile here, and you can see he's got some content That's associated with it. So that's great. And so for those experts, if you have an expert account, one of the things you can do is go into your dashboard, which you'll find on the right, and there's this new feature called PostWriter.

Which is meant to help you write LinkedIn posts, and the cool thing about it, you know, how it maybe differentiates from something like Just Claude or… ChatGPT is that as you upload your own content into the system so that it can be drawn, you know, in the chats, it knows your content. So, for example. like, if it says, okay, let's brainstorm for your next post, what topic do you want to cover? If I say, like, okay, threat modeling. Right? Then it's not just gonna give me some, like, vanilla, you know, thing about threat modeling. It knows that it's in the context of a medical device, it's gonna look at my own stuff.

And so it's gonna say something like, okay, here's a few options, threat modeling isn't a one-time event, where to start? Threat modeling, stride for cloud devices, so I'm just gonna tell it number 2. And then… So it's gonna propose a hook for me, you know, what's the single biggest mistake teams make, when deciding when? No, I'm just gonna say it sounds good. And… Okay, it's asking me one more thing. When you say start doing architecture, what does that actually look like in practice? draft post. So it's meant to be kind of interactive, and you can add your own voice to it. You can tell it, like, oh, I want to focus on this, or I want to focus on that.

But yeah, I think the coolest thing about it is that it's pulling from your own content that you've uploaded into this over time. And then the other thing I like about it is, if you've ever used, Authored Up, which is a tool for LinkedIn posts, borrowing some of the ideas there, so you can see what it's gonna look like in terms of like, the width, and you can see what it'll look like in desktop versus mobile. You can actually do, like, like, bold, which… LinkedIn is funny, you can't actually put in bold, so this changes it to Unicode, so you can copy and paste it, and it'll look bold on LinkedIn. And then the other cool thing about it, I think, is that you can actually edit this. So, you know, like, I'm a big proponent, I like large language models, AI, but I'm a big proponent of

when you're writing LinkedIn posts, make it your own. Like, don't just copy something from an AI. It's like, humans are very good at pattern recognition. When I look at something that's 100% written by AI, I, like, I'm turned off by it, personally. So I don't like to do that myself. So this is meant to be an aid. So, you know, I can take this and go, okay, they finished the design, you know, most teams start with threat modeling too late, they finish the design, write the architecture document, and then say, blah blah blah, alright, that's all fine. But you can come in here and, like. You know, hit enter, you can, you know, create a list of stuff, and…

It's a little bit buggy right now, but, etc. So you can, you can actually change it, and then if you tell it, like, hey, let's make this… shorter… For example. It'll take your content that you've now edited, and it'll add on top of it, how short do you want it? Let's make it 30% shorter. Let's see what it does… Okay, and then it'll make the changes, and you can kind of see what was there previously. And, like, what are the differences? Like, what did it change? You know, here's what it removed. Because what it cut out, whatever. And then if you go back to the clean version, you can make additional edits.

And then, once you're done with it, you can copy it and just paste it on LinkedIn. And also you can save it, like, if you're not ready to post it yet, you can just save it here, say, Brett Modeling. And then, yeah, later on you can come in here and look at your various drafts. So… Hope you find it useful and play around with it. If you, again, if you want additional features, or if you find bugs, if you find issues, let us know. And our goal is to continue to make this helpful for you guys, whether you're just a medical device professional or you're more on the service provider side of things.

To help you, you know, get the word out, get more clients, that kind of thing. That's it.

Yeah, on that note, I mean, the goal behind MLVX.ai and our collaboration with BoldType is, to create an interactive community, a supportive community in the med tech world for aligned service providers, experts, and solo consultants, because really, you know, the lack of support, the lack of tools, the lack of understanding around what it takes to run… run… run a small business in… in the sector and successfully get clients, you know? It's really… important that we support one another. You know, I put some links in the chat that said, you know, please show some love, you know, to the latest posts, and it may seem like, you know, thirsty behavior on my part, but it… and to a certain extent, it is.

But at the same time, you know, it's really important that we come together and share knowledge among experts. In a meaningful and valuable discussion. And that's not something that… I find happening very often, and when it does, it usually costs you $2,000 to go there. And… and plus travel. So, help us build this community, spread the word, follow along. And, please give an eye to MLVX.ai. We want your feedback, we want your collaboration, we want your help developing this product. This is not, hey, here's this great thing, sign up and pay us money. This is…

you know, a group effort. So, I really, you know, applaud the beautiful work that Jose has done. In helping this come together, and, you know. Figuring it out as we fly the plane, so… Please… Be part of this. contact me, contact Jose, contact Marcus, contact Maria, contact Michelle, you know, contact Andre. It's, it's, you know, we're here for you. So, thanks, everyone. I hope you have a great weekend, and I look forward to seeing you next week. for Pure Global.

Bye.

Maria E. Donawa, MD: Excellent. Thanks.

Thank you, Sean.