When Enrollment Stops: Diagnosing a Study That's Already Gone Wrong

Hello, and welcome, everybody. My name is Sean Smith from MedTech Leading Voice. Our topic today is when Enrollment Stops, Diagnosing study that's already gone wrong. Today's speakers are Daniela Karrer, Vice President of Operations and Clinical Affairs at Dono Life Science, and Matteo Mosso. Director of Clinical Affairs at Donoa Life Science. Daniela and Matteo, I'm really happy to have you with us. They bring, real clinical experience into today's discussion. They're going to walk us through four studies. Where enrollment stalled, and what the sponsors could see at the time, what was happening underneath, and how teams diagnose these problems before that reaches the critical point.

Before we get started, a few quick housekeeping announcements. This webinar is being recorded, and slides will be available, as well as the recording tomorrow to everyone who registered. If you would like a certificate of attendance, please put your full name as you would like it to appear, and your email address in the chat. Use the chat function. to communicate with me, and if you have questions for the speakers, please use the Q&A function. There will be a brief poll, I don't know, 20 minutes in, I'll interrupt. And then, there'll be a survey afterwards. Your feedback is really important to us. It helps us understand what's helpful to you, and what you're looking for more of or less of. So please be sure to complete both the poll and the survey.

What's coming up this Friday on, Priya In Focus, Etienne Nichols from Med, Medical Device Guru, or from Greenlight Guru, will be presenting the Modern MedTech Growth Playbook, Why Great Service Providers Are Struggling to Win Business Today. And… Then on August 26th. The business case for quality, how business culture puts your market access and revenue in jeopardy under QMSR with Michelle Lott and Tiana Benson from LeanRAQA. With the announcements out of the way, see a lot of people have joined the call. I'm gonna go ahead and hand it over to the speakers. Daniela.

Thank you, Sean. Welcome, everybody. Happy that you'll be here, I don't see how many of you are here, but that's… part of the modern game, apparently. So, welcome. We hope that you'll enjoy the format chosen for this webinar that's It's not based on conceptualizing things, but just showing how things went, and making some considerations that we made and we are sharing with you. So we'll talk about four studies. For actual case studies, we selected them based on the… diverse feedback that they can give, and all these studies had some kind of trouble having to deal with enrollment or initiation, you will see.

3 out of 4 were not ours to start with, and one was. You will see diversity, I mean, there were sites that were activated and not enrolling, sites that… did not perform. Sites that could perform, but for other reasons didn't. And the enrollment stopped. In all cases, the sponsor came to us and giving us this question, what is that I'm not seeing? What is that I'm missing? Why things are going wrong? Can you help us? Can you help us fix the study? And because enrollment was the key element, the key trouble.

Either low enrollment, but in most cases it was no enrollment. Without enrollment, you have no study. Enrollment is the vital part of any study. And as such, one of the messages, we start here already, saying that any issue with enrollment should be really addressed fast. So why is enrollment so important? This is quite obvious, based on what I just said, but, There are significant costs involved. And in delaying a study. And Enrollment is the biggest question mark, it's the biggest uncertainty.

In any study, because what happens before regulatory approvals, drafting documents, etc. What happens after? data cleaning, statistical analysis, etc. It's… Reasonably. You can have a reasonable forecast of that based on experience. The big Question mark is enrollment. So, all companies, make… An estimation of how enrollment is gonna go on, and base the number of… Costs and deliverables and deadlines on that. So this is why this is so important, and All attention to keep the study on track, Ideally.

If not, be able to catch up. This really early, okay? I've been looking around on the internet, I found this cost, it's additional study operational cost, so every day, that's the estimate, every day. of delay in enrollment. Enrollment, which, again, means that everything that happens afterwards, unless you can squeeze it and make it shorter. Will result in a delayed study, okay? So, every day of delay has a direct Additional operational costs, which it's estimated to be between 1,000 and 7,000 euro per site per day.

Clearly, this varies based on the type of study, the type of product, the milestones, the objective of the study, etc. You will use money faster, money that you have, investors' money, and you'll probably have some questions from investigate… from investors. You may entry market Later than you would like to, and this you may lose competitor… an advantage that you may have on other competitors, in case your device has, high, degree of innovation, and there may also… patent expiry can also be involved. So…

One of the biggest uncertainty of the enrollment duration derives from the fact that this is not under your control, but this is under the control Of a partner that is a clinical site. We'll touch repeatedly on this concept, because clinical sites are not vendors. You have a different type of relationship as compared to CROs. CROS can also be partners, but basically, their scope in life is to serve the client, right? But clinical sites are different. They have their own goals, their own objectives, their own routine. And they accept to cooperate with a sponsor for various reasons. We'll touch on that.

Later on. But this is… we think it's an important… Element. Sope. The four case studies that we worked on will move on this scheme, so we'll describe what we saw, what the sponsor saw, what we held them see. Try and guess what was the reason for the issues that the sponsor was having. How data and the event was analyzed, and root cause, in most cases, identified, and how this was resolved, and the Key message. to take home. Okay, case one.

Case 1, that's a simple title, so there was an MDRC marking deadline, and it was a raise. The company did not have, much time. 4 sites and 2 patients enrolled after a very short period of time, that is 3 months, but still, with 4 sites, you expect more. Some more detail on the case. This was a post-market clinical follow-up study, of a special kind. It was randomized and controlled. Why? Why? Because this was a legacy device, that, the objective of the company was to achieve MDRC marking.

And there were clearly obvious gaps in clinical data generated to date on that device, so the notified body had made some requests, and the company was forced to run a very complex study in a post-market clinical follow-up setting. The deadline? So, this… this is why we talk over race, and there were 4 sites active, one in the UK, 3 in Italy. After 3 months, 2 patients enrolled. And… In the horizon, the deadline for notified body submission. So…

You can think of the site resources problem, they don't have enough resources, they're not interested, or maybe there's a protocol problem. Or… who knows what. So… I'm happy to have here online my colleague Matteo Mosso, who's Director of Clinical Affairs, but he directly acted also as project manager for the study. And worked with a team. So, this is the question that he was posed, where's the problem? And now he's gonna tell us… What… what they found. Matteo, you tell me when to switch. Yeah.

Yes, thank you, Daniela, thank you for introducing me, and welcome, everybody. Just to add some additional details, I wish to mention that the company did, submit an application to the notified body, presenting, results, data on, studies conducted on questionnaires, so a rank 4, type of studies, and of course, the application was rejected. This is why the, let's say the sponsor had to, address some, major gaps to the… to the data that they were presenting. In addition to that.

The device was… it's a device that needs to be used on muscle skeletal disorders, and the sites that were involved to participate in the study were private physiotherapist clinics. So this gives a little bit of additional context. to the… the situation that we had to face. So, actually, what did we find? So, first thing is that the sites that were initially involved were all selected by the marketing department of the sponsor. So, we understood pretty early that they were not able to clearly, explain the complexities of conducting a study to people who have… who are naive to studies, who have never done a study, so they were not aware of the workload, of the responsibilities, and so on. So this was

a first, let's say, blocking factor. Another important aspect was that we found out that in the protocol, there was an exclusion criteria that was blocking the enrollment. Actually, a non-significant criteria, but that was, important enough to… To restrict the patient population, to narrow down the patient population, as it required to have the patient enrolled within 3 days of the occurrence of the, let's say, of the disease or the problem. So, that's why, actually, the situation was pretty, critical at that, at that stage. And we reacted pretty quickly.

By, of course, consulting the sites, in, in order to understand what I just mentioned. So, where were the problems? And we acted, by, let's say, applying, remediation actions, by, moving forward in parallel, so we not only amended the protocol to remove this blocking exclusion criteria, but we also Added additional sites. in Italy and in the UK by conducting a proper site selection. So, visiting the sites, explaining them exactly what was going on and what are the requirements to run a study, and getting the insurance that they were engaged and aware of what they were going to

phase. So, let's say that after all this, the enrollment, restarted, and let's say, as an outcome, you can move to the next slide. Yeah, we can see that, actually, the enrollment was completed. We hit 62 patients. The criteria was… the success criteria were met even early, because the protocol did foresee an interim analysis, which met the success of the study, so not only the enrollment moved up Quickly, but also, given the success of the study, they were also able to,

close it, close it earlier. So, what does this tell us? Actually, the, that, First of all, in certain circumstances, you need to react quickly, and not on a, you know, step-by-step process, but you need to implement several corrective actions at the same time, simultaneously. Fun fact that I just want to add is that after the study was concluded, a couple of investigators that we involved were so enthusiast of conducting the study that they came back asking if we had more. So, that's an interesting aspect that,

I… we were not expecting to see. Yeah, so this is it for this, for this stage.

Yes, and the motivation, this is something we'll touch on, but the motivation that brings sites to perform is not the same for every type of site. I mean, in these cases. they were naive, but they were properly followed, and once they were, well prepared to know what was coming through, and they said, yes, I want to do it, they jumped on it, very happily and performed very well, so…

Yeah, having a collaborative approach and granting full support by the CRAs did help a lot in this kind of situation where, you know, this unusual setting was, I mean, we have to go with this unusual setting.

Okay, good.

So. For a PMCF study that is already underway but enrolling poorly, how do you determine which strategy is ideal? A protocol amendment, adjusting the site-specific strategy, etc? I guess… I think…

Yeah, yeah. I was just saying what you probably were just about to say, that it depends on what the root cause is. So, there's not a fix for everything, and In this case, maybe something that's detailed that was not given is that The sites initially chosen because it was a PMCF were known by the marketing, which can do a fantastic job as long as they know what the clinical study is, and often they don't. Meaning that they're not aware of the efforts, and paperwork, and precision, and time. That each study requires, no? And that's one thing. The second thing is that all initial sites were already using the device, whereas those that were added later

and that resolved the issue, they were not using the device. They received it specifically for the study, and used it in accordance with the instructions for use. So… So, this, this, this second part, it's also, a way to analyze what the… what the events are… are telling you. So, there is no guarantee that the site, only because they buy a lot of of your products. They are interested in clinical studies, and they are aware of what it requires, etc. Naive sites with no experience, if you provide them

with the device, clearly we are talking here of a relatively simple device, but it still required training by the manufacturer, not the CRO, huh? Training on the device. Still, it was worth, because it resolved the issue.

Great. So I'm running the poll, as you can see. Feel free to move on to the next part.

Yeah.

Minimize that by hitting the yellow button at the top, so it's not blocking your screen.

Okay, then we have… Case 2, a study received from a sponsor who had been working with two ZROs. Not one, but two. They had enrolled, they had a single excellent site. Very nice. They had enrolled a lot, but… Part of their data were entered in the CRF, and the remaining 400, were staying in source document with data not entered. So, practically, they were not visible. Enrollment was stopped. what was the case? It was an IVD study with a US company. for the detection of colorectal cancer. And…

they started, with a small CRO in the… well, that's… that's what Matteo is gonna mention. I'm… I'm staying up. So, it was clearly a rescue study. In addition to the backlog of data, there were ongoing monitoring problems, because the CRA that was assigned to the single site was not following up properly, and then because of a number of Issues with the… switch between the first and second ZRO that Matteo is gonna talk about. Practically no one had visited the site, I think, for months. As a result, the enrollment had suspended, and clearly budget had been largely overspent, and the study was in big trouble.

So, again, what's happening here, you have a very nice site, so the site resources we can, exclude. you know, things can happen. A site can close, and a very good study coordinator can leave, or be pregnant and leave, or whatever. There are issues with the CRF, Who knows? let's… let's hear what Matteo has to say. Matteo again! You were working with a local CRA, right, here? So you… took over, this… this is what your role was. Yeah.

Yeah, correct. So, yeah, let's…

Project manager, yeah.

Yep, yeah, yeah. Yeah, exactly. So, in this… the situation that we had to face was, as Daniela mentioned, particularly, challenging because the company, the sponsor, did initially engage a… being a U.S.-based company, they engaged a small, U.S. CRO who did not have the CRA, their own CRA, because the study was conducted in Spain, so they had to rely on, I believe, freelance or some kind of associate. In any case, what we became aware of is that the ZRA was not fully engaged, so really underperforming.

And to this extent, the sponsor entered into a litigation with this first CRO and decided to move forward and hand over the study to another CRO, which in this case was one of the big players, you know, those that deal with big pharma studies and so on, trying to see whether they were able to fix the problems and move forward. Unfortunately, the second CRO made the request that they did not want to use the first CRF, so they wanted to use their own vendor. And this caused additional problems, because there was a huge, let's say.

problem for transferring the data from the first year… the first CRF to the second CRF. And as Daniela mentioned, still, all the 400 patients' data were still standing. And the study was not moving forward. So, they came to us to see whether we were able to, to resolve the issues, so we were the third CRO dealing with this study. What we did, actually, we did not request to change again the CRF, but we discussed with our data management department whether they were able to a deal with another vendor, the CRF, and this was the case.

In fact, they were able to fix all the issues for data transfer and reestablish, let's say, the connection, and make the CRF available again. We did implement a kind of remedy plan by involving two CRAs. In parallel, actually, which, visited the site, either on-site or remotely, but given that the site had good, capabilities in terms of staff, we were able to, act in parallel and, and, let's say, increase the volume of data that could be entered into the CRF and monitored, and so on. So we were able to, clear up all the backlog, pretty quickly, and the enrollment was,

was resumed. So, after all, let's say that the study could move forward. And if we see the next slide, we will understand what the outcome was. So, the study was concluded with over 1,300 patients enrolled. What does this case tell us is that not always the problem is within the site, or with the performance of the site, but in certain circumstances is the sponsor selecting not the correct partner. To run… to run a study. So this is, the lessons for the second case.

So, we had a question on the first case, the 62 people enrolled. How long did that take? And then… extending the question to this one, 1300 subjects achieved, you know, what is the timeframe for remediation on these projects, generally speaking, or specifically speaking, like, this 62 people, how well… how long did that take? to get them.

I think it was less than 6 months.

First one.

Yeah, the first case, I think it was, yeah, around… 3 or 4, something like that. I cannot exactly… Provide a date, but, for sure, less than 6 months.

Okay, yeah, because a couple people had asked what the time frame was, and So, when you're talking about remediation, when you're talking about going in and rescuing a study, you know, you obviously have to go and you have to diagnose the problem, and then you have to figure out what you're going to do about it, and then you have to turn that thing around. So, time takes time, you know? And depending on, you know, in this case, it was, you know, data transfer, the files, everything. Very interesting, thank you.

Yeah.

Yep.

Okay.

Okay, yeah, we can move to… Let's come to the third…

one that Matteo will introduce this time.

Yes, exactly, because in this case, Daniela was the main player, having to deal with this situation. So, we are talking here of an international company. We can just move to the next slide, please. it's an international IBD company that was trying, not running, but trying to run an FDA study to obtain clearance in the US market. So, the company was thinking that they were able to manage in-house the study, and they tried to do that for, I think, over a year. before we were, involved. And, within the timeframe that they were managing the study, of course, they were…

supposed to have a few sites initiated, but not yet active, because in the meantime, they did not fully execute the contract, so they were, let's say, partially initiated. I think that there was some kind of mess with the protocols, because the company did have multiple protocols. Approved for different sites. And they tried to initiate the study while they were still discussing with the FDA on certain aspects of the statistical analysis. So I think that the smart move that the company did was really to Understand that something they were doing wrong, and they, came to us to ask for conducting a gap analysis to try to understand what could be the solution of the problem.

So, if we move to the next slide, please, So, what do you think was the problem here? So… underperform… maybe underperforming sites, maybe not. Delays in regulatory approval, or are we speaking of a rare indication? So, Daniela will tell us what actually happened, and I will just hand the word to Daniela.

Yeah, thank you. Yeah, the… I went to do the gap analysis because this is… was where everything started, with a colleague that was a senior statistician with large, past experience in interacting with the FDA, so very familiar with their processes and with their requirements. So, we spent a couple of days. In the headquarters of the company, and it turned out that the site initiation and enrollment was really the last of the problems here, so we started from the end, and then… went back to the top of the pyramid, if we can call it this way. So, to the root cause.

So… the management was the issue. This is the solution. So… to give more detail, so the company is a European company with a quite large U.S, affiliate. And the two organizations, because there were two, when they're big, they become two, although they have the same name. They had run studies independently, local, independently from each other. Apparently, they never did something together. So, the group in Europe did not have a grasp or even authority on what the US was doing. Everybody was going through different objectives.

So there was no coordination, and no clear project ownership, okay? One of the big, notable aspects was that the people involved in the study I mean, managing the study at the level of study monitor and the project manager, they never… Besides the monitor, who in this case was, as I see it, less relevant, because they did the even get to the point of monitoring. But the management was someone that never managed an international study done for regulatory purposes. It was a person coming from marketing with very limited experience and local experience only. So, it was really a bad choice.

So they had no structure, to the point that they did not have no coordination, poor structure. And not even… SOPs, which, in the lack of everything, you know, you can rely on a procedure that someone else wrote. But in this case, they were missing. They had chosen two vendors, one for the CRF and the other for the statistics, so the CRF vendor was coming from the academic field. I don't remember if it was a spin-off, but there was some connection, so they had experience in studies done for publication purposes, so mainly academic type of studies. So when we started making some questions as part of the gap analysis that we were asked to do.

In terms of quality, how is your data cleaning plan, the data management plan, they… they looked at us as if we were asking for something unusual, whereas in this type of study, this is absolutely standard. Statistics. Another big, risk that the company was going through was to start the study without having finalized the interaction with the FDA. In particular, the interaction was about statistics, but as I think all of you know, the statistics is closely linked with sample size. So… This can happen because you can start a study, you can start enrollment, and if FDA does not agree with a certain element of the statistics, maybe you can amend the protocol later, while the study's ongoing.

However, this… this can be a risk, because what if sample size, and this happened, increases? So, they had in mind a totally different sample size from what it turned out to be. So… how we acted. So, first of all, we completed the gap analysis that was our first remit. We wrote a long report, and being very honest and transparent. And I want to say that this request came from someone, and I'm still in contact with this person. He was… he was a regulatory person in the company. He was the person that saved the study.

We helped them, clearly, but Without his action and his, wisdom, I would say, the ability to say, okay, guys, let's stop. Let's bring in someone that can see this from the outside. And someone that knows… that's… expertise in what we are trying to do, because I don't see the problem, but they may. And, and we saw a lot of problems. I mean, for us, it was incredible. It was a… They were really doing, kind of, all mistakes that you can think of. So, we were given the full management of the study.

and reorganize things, so the interaction with the FDA… we stopped any activity going on. the focus was on finalizing the interaction with the FDA and obtaining their acceptance. sample size was much higher than expected as a result of FDA feedback. So, then we, had the protocol rewritten and amended, submitted. We had to scale up to 18 sites from the 8. 8 sites in Europe, in Spain, France, Italy, in Belgium, and the rest in the US.

The study started. And the final goal was, that the company… that the group, the sites, enrolled around 2,500 patients. In the indication from emergency departments. At all sites, and after the 510K submission, clearance was given after 4 months. 4 months, 4 months and a half, so pretty quick. So… The take-home message is that… you need to know, this is a sentence that my boss likes, but I like it as well, and I think she,

mentioned it the first time, she thinks that I did it, but never mind. You need to know what you don't know, and that's critical, and someone from the outside can help. Really, because they have a different perspective. Even someone within your company, as long as they… Understand enough of the topic. And management is essential. Study management and project management. That's what I want to say.

Can I ask a question here?

Yeah.

It seems like the FDA meeting with the FDA was helpful in reorganizing this study. What do you have to say about pre-sub meetings and protocol review and sample size and the FDA? With that regard, should companies be doing this? You know, what it would… what is your advice?

Well, pre-sub is really a precious process, I think that all those that have been through that, it's free. Still, it's free, although I read around that it's no longer… that may not be… Free for long? And I well understand why, because in Europe, there's no, you know, there's no such a thing, so… And they give substantial feedback. I've attended a couple of these meetings in the last 6 months, and they don't… They're not, I don't have the English word. I mean, they give a big feedback. So you ask for 3 questions, and they say, by the way, I've noted that there is this, and maybe, who knows, and this, you should send us this, otherwise we cannot comment, etc. So.

I think that their feedback is really precious and generous, and Prior to starting a clinical study. that everybody knows it's very expensive, and it has big consequences. The recommendation is to go through the FDA, yeah. For sure.

And would you say that even if, the study is not going to be a U.S.-based study for… for an EU-based study? I mean, I, you know, I mean, or do you go structured dialogue with the… with the notified body?

Structured dialogue! Good luck!

Sorry.

So, let's say that FDA has its own standard, so… and their requirements are not Especially the regulatory requirements are not overlapping with those in Europe. But! from the… Clinical study design level, their approach, it's still… Although they sometimes change their mind, this has happened to us, that 6 months before they said something, and 6 months later, clearly they're authorized, I mean… We accept that they changed their mind, I mean, but what I'm saying is that the difference between Europe and the US at the clinical level is reduced, so feedback obtained from the FDA certainly can be used very

Nicely in Europe, although their standards is pretty high, especially in terms of sample size and statistics.

So, two questions, one from case number two, can a CRO Support data entry in a regulatory compliant manner.

Yes, we have done it, maybe Matteo…

Yeah, responsibilities and roles need to be very well defined, so… people who do data entry cannot be… The same who monitors the data, so you need to have some procedures in place that prevent, making mistakes, or having data not perfectly reviewed. So, it's something that can be done, for sure. We did it in certain circumstances, but you need to be careful.

Okay.

Yeah, and also, I'd like to say that there are CROs, big players, very big players, one of the biggest big players, that already do it. So, they pay study coordinators, meaning that not they pay, they provide study coordinators, so it's not only data entry, but selecting patients, I mean, acting. And reporting to the principal investigators as if they were hospital staff, but they are not hospital staff. They are provided by the CRO.

Lynn?

this kind of gray zone, and I don't want to enter into this discussion, but this is already happening, and for data entry. Data entry, this is, Yes, something that, as long as you manage accurately, with transparency and with honesty, and separating exactly who monitors and who… performs the data entry, it's… Certainly can be done.

Okay, and a more general question, how do you help clinicians understand what good candidates are for a study without taking wind out of their sails? For example. High number of comorbidities or age patient… age of patient could significantly negatively impact the potential outcomes.

So we're talking about patients here, not… I mean, candidates, it's patients. Yeah. It's not sites, okay.

Hmm.

Patients should reflect the intended use of the device, and… Patience can vary, as we were discussing today, Matteo, between a pilot study, a small study, where you have to understand What's happening, how the device works, and you need to have top-level, but key information to plan the pivotal study. So, the variables… Shoot… be limited. What are the variables? Comorbidities, etc. However. Consider that if you select a certain population, that's gonna be reflected in instructions for use later on.

This is more relevant for a pivotal study than for a pilot, but if in a pilot study, you only select, let's say, white people, and then your intended use is not in… will not discriminate between white and black. skin, then you have a problem, because you have data generated only in a population. How do you plan your pivotal study? So. It's a matter of discussion. But in a pivotal study, you should reflect what you're gonna… What you are going to put into the intended use. Into the instructions for use.

Yeah, I agree with you, Daniela, and of course, The key point for pilot studies is, have robust data that you can rely on to then build up. the pivotal study. So you need to limit the variables and the possible confounding factors. So you do… it is true that you narrow down a little bit the population, but this allows you to have a better and clearer idea of The effectiveness and safety of your device that will allow you to plan and execute, then, a wider study for getting into the market.

Or maybe extend the population later on.

So…

you can plan on a certain population for the C marking, and then you do additional studies trying to extend, for example. widening the age. You roll up to, I don't know, 65, and then… and I've seen this several, several times, companies do that. They extend, that they can demonstrate later on that What was, valid. For younger than 65 is also valid in people that is older than 65. These are additional studies, but in the meantime, you enter the market. Alright.

Okay, so we move on to the last case. Implantable cardiovascular device, disaster. 5 sites active, sorry, 7 sites active. Already activated with all… Regulatory approvals and contract signs, and zero patients enrolled. Okay, I'm sorry, I need to connect my laptop, I forgot. Sorry, hold on a second. Yeah. So… Talking of a pre-market study, highly innovative device, unmet need.

a CRO already in place. They've done, up to a certain moment, maybe, a good job, sites were active, documents were in place. One… Small detail, no patience, a disaster, and Surprising thing when we had the study proposed to us was that half of the budget for monitoring visits had been spent. If you have no patients, how is this possible? Because the ZRA was asked to Repeatedly visit the sites with no outcome. I'm rushing a bit, we don't have much time, so… What was the problem? It was sites? No, sites were very famous. I can tell you already, that's not the case. Sites were famous, large hospitals.

But protocol design, not at all. Recruitment, they did not see patients. No, no, no, they did see patients, but they decided not to enroll them, and… This is the case where Matteo was involved visiting each site with a sponsor representative, and the word to you, Matteo.

Yeah, thank you, Daniela. So, exactly as you were mentioning, the previous CRO was acting, let's say, on a standard mode by just trying to push on the sites to encourage them to enroll, but without having the CRA clearly understand, and the sites also, disclosing which were their concerns about the study. I need to say that, The implantation procedures was particularly invasive because it requested open-heart surgery, so we can understand a kind of reluctancy by the investigators to initiate the study. Sorry, around the patient.

So, what did we do? Actually, as you said, we personally visited, I personally visited every single investigator, together with a senior member of the, of the sponsor, so the director of clinical operations, and together we had face-to-face discussions in order to perform and understand and have an open, an open-heart discuss… discussion with the, with the, with each investigator. And… Exactly, this is what we understood, that they were reluctant because not being familiar with these procedures, and, let's say, somehow a bit afraid of starting the enrollment. Together with that, we also had the opportunity to involve one of the sub-investigators who was one of the cardiologists, and who was particularly enthusiast of this new technology, which acted as an advocate. So we had, you know.

Doctors speaking to other doctors, cardiologists speaking to other cardiologists, so they are on the same level. And, in addition to that, we also organized, training sessions on live animals, on pigs, so each investigator had the opportunity to train and get familiar with the implantation procedures. And in addition to that, we also understood that some of the sites, mainly were very famous key opinion leaders with too many studies ongoing and not sufficient resources to run the studies, so we decided to, let's say, reduce

the number of participants that would have never delivered, to focus only and concentrate on those that were able to move forward and contribute significantly to the study. So, the outcome was, next slide. that the enrollment was completed, 45 patients reached, so the… here, the message is that in certain circumstances, it's not a matter of sending multiple times a CRA who maybe, the investigator is not happy to discuss certain topics. But you need to involve senior staff, senior people, and And open a discussion to try to understand where the problem really is hiding.

So, yeah, this is it.

Yeah, something that was, it is unusual, after you have sites initiated, to go back to animal… to an animal implantation, but this was necessary, this was one of the actions that was decided in this case, because of the… Block that apparently could not be…

Hmm.

overcome. Usually, in different situations, I do not know if and why they didn't do this before. We took the study at a certain point. But this was another, unusual thing, thinking out of the box in a way that we think Helped, because they eventually enrolled.

That's a great case study, and I, it brings to mind, the… the difference between, doing a study on A non-invasive device where you have interventional cardiology devices, and high-risk Patient group that the, the… the… that the structure needs to be different, and then turning to peer-to-peer enrollment, was a particularly clever solution. How did you come up with that?

Ehh. I don't know, how did you come up with that, Matteo?

I don't know, I… I mean… Thinking out of the box and trying to figure out, you know, solutions that could, somehow… Improve the situation, so…

Yeah, because.

Experience, I don't know…

Well, you see that, I mean, but this happens even today. We have investigators signing a protocol, and a week later, they say, no, I do not like this. Why don't you change that? You say, but you, you, you signed… A week ago. Why didn't you tell me before? Well, because they don't read. And that's similar. You've been involved, and then you don't enroll. I mean, if you're not certain, why don't you tell us before? Because this is reality. They don't… Tell you. Unless you deep dive and ask et cetera, et cetera. So, what saved the study, because again, the study was saved, in this case. was the perseverance, not only ours, but I have to say, even from the sponsor, to work together and sit down, and long discussions, and to visit each site and extract

The words that, helped us resolve, in a way, the situation, yeah. Okay, so… these are final considerations, these are not rules. Mmm… so… What we think, it's relevant that the clinical sites and clean investigators, they're not service providers, they're a partner, and you should treat them that way. So, select them carefully. inform them very well what the study requirements are. Based on their understanding. I mean, there are sites that do everyday clinical studies, and there are those that have never done it.

And those… Sites sometimes can… they have the interest and the motivation to do very well, as long as They are informed, and they know what to expect. And then, after they're in… they're involved, and they accept, you should support them adequately. Sometimes this is not the case, as we've seen. Sometimes they're very well supported that they do not deliver anyway, so close them. I know it's easier to say that rather than doing it. There are several diplomatic things moving on, but still. The motivation of the sites to perform well, it's really variable. It's also geography related, it's culture-related, it depends on how big the site is, the… where it's in Europe, and it's Southern Europe, Northern Europe, US,

It changes. An investigator's motivation can be financial, scientific, glory, prestige, or just a publication. Or even they're unable to say no, because I think that sometimes we've even seen these cases, they say, yeah, yeah, but they… will not. Enroll. Fine. Whatever is the thing, it is important that you understand. Now, you may decide to have a very famous name into the sites group. Knowing that they're not gonna enroll. Fine, but they have the name, they have one patient, and the others have enrolled docents, and they have one, but they're part of the game, you wanna say that?

Fine. As long as this is known, it does not… does not come as a surprise. Move on early, not all studies have the same pressure to end. Sometimes there are deadlines, and when there are deadlines, then the companies are more active. Sometimes we see studies that go on for years. We know patients, but Usually, it's big companies that… for that particular study don't have a specific deadline, so they roll over very slowly, and but when you… when you do have a deadline, investors, regulatory, notified body, then you should move on as early as possible.

And analyzing, the root cause. Sometimes it… who is within the team is not in the best position to see what the problem is. Sometimes this happens. So you need someone from the outside, that doesn't know, but has experienced that can tell you what is… what is wrong. And maybe can help you fix it. Okay, this was the last… oh, yeah, please, yeah.

So, if anyone on the call would like to speak to Daniela or Matteo about what you're working on, you know, please put that in the chat, just write call. You know, that I'd be happy to talk to you about what your, challenges are, and, you know, if it would be appropriate to work together. I think that would be a… You could… Summary, right? You'll talk to them. Thanks for volunteering us. And, I want to thank both of you for your time today. And I would like to say thank you to everyone who attended, who gave up your time. And, the slides will be delivered tomorrow.

And, yeah, thank you, and I hope you have a great time.

Yes. Thank you.

Thank you.

Thanks, Sandra.

Hi everyone, thank you for attending.

Great job.

Thank you. Goodbye to everyone.

Bye.