AI and MedTech — Insights from the Portugal Unconference

And so we're just gonna have a discussion about, what happened, They are what was covered, and, hopefully have some surprises for you. Reveal the survey results, and, and hopefully spark a conversation with everybody today about, where we are with med… With AI in… in med tech, particularly in regulatory, for regulatory, and compliance, and where we're going, and how we're gonna get there. That's basically…

was the goal of the conference. On a personal note, I would like to add, having just returned from, RAP's conference in Pittsburgh yesterday, which was a very large conference, a very, you know, normal vendors and sessions and everything, that the one in Portugal was very different than that. And the reason that it was very different than that is because everybody was focused on a single topic. Not, you know… A tiny topic, but a broad topic. Which was, how are we going to… integrate AI into our… into regulatory compliance. With that, I would like to introduce, Miguel and Celia and Mariana from Complear. If you guys want to say a few words, feel free. If not, just…

No, I think just a quick word from our side, so we had this opportunity to try to do a different kind of event, discussing AI in a physical format, so it's not what we think normally, but it really made us… Discuss things that otherwise may be harder online, so it's quite interesting when we go into a more digital and automated world where the physical presence becomes more relevant. And because of that, also this opportunity to really discuss things in a level of depthness that we can do in the physical world. So, the same way AI is good for some things, bad for others, I think a lot of… of these kind of setups shows that the more we see AI, I think the more we value humans.

We had a big discussion on the relevance of the humans in the AI world, that we just need to understand what is our spots. AI will never replace us, will replace most of the things we do, but will free us to do quite amazing things that we are not doing before, so… it's a bit of mistakes on being able to stop doing more kind of retreats, not… no one was there trying to sell anything. We had completely already developing a platform to try to automate a lot of this journey, but the reality is that we even discussed VIP compliance with normal quads, or normal tools, so not trying to say our tool is the best.

or the other is not good. We had different consultants coming together, so… some of us competing for the same clients elsewhere, but we enjoy some good meal together, and it's a bit of this spirit of human connection that's… AIMCP cannot fix.

Absolutely. Thank you for that introduction. Celia, do you want to say anything?

just reinforcing, I think, the key point, and while discussing AI, I think sometimes we miss is really this, like, the human in the loop, and how the human interactions in the end are what give the best output to everything, to the AI, but also to humans in general, and how we can use AI to make the most of it, and not having AI doing everything without the point of the critical thinking of the originality that humans have. and also free us, so that we can have better human interactions in the end. Yeah, just wanted to, to, to reinforce that while discussing AI, how humans are,

Are important and are not the replacement, but how we can improve ourselves and enjoy more yourselves because of that.

I want to highlight to a moment that I think was quite… relevant to me, that was we had a music concert, violin and cello, in the middle of the park, just for us. In the reality. could AI play better music than those amazing young talented musicians? Probably you have already AI doing so. But being there, the presence, having someone smiling when they were playing to use, feeling like there was more than just people playing music, or someone really feeling how they're playing. That moment, that was, I think, a bit of humanity kind of reality, when we were just discussing AI all the time, so it's a bit kind of interesting, kind of, of not typical events.

we are not discussing the future of technology and forgetting humans, we are really going back a bit on, okay, where the humans are relevant in the world of AI. And this presence, being able to be present, being able to really keep up with the world, I think, is the biggest challenge that we have with AI, is that, how can we evolve AI at the same pace we need to allow people to still keep up with it? Because we need to keep the human component on it, because it's what makes things special.

Exactly. Mariana, did you want to introduce yourself?

Okay, sure. Hi guys, my name is Mariana. I am a PhD fellow at Complear. I don't think I have anything to add. Miguel and Celia have said pretty much everything. I am very pro-AI, but if it's used for the right thing, I think it should not replace us, it should aid us. So, yeah, let's… let's move on and hear what… what Sean has to say.

Okay. So, as I said, the core questions were, you know, the, obviously, how are we going to use AI in regulatory and quality systems, and what is the readiness of organizations across, across the sector, presented while I was there the, survey results, which I'm happy to share with anybody, in… in summary format. But what we found was that, a good 50% of the regulatory audience that responded to the survey, really is not

very well prepared to deal with AI, nor are their organizations. They don't have policies in place regarding the use of AI by the employees. They don't have protections in place To, safeguard their intellectual property, against. You know, being shared with the world, and so there are plenty of opportunities, for both individuals and organizations to Come together on this. That was my… presentation, and a lot of people are still very suspicious of AI. That would be the other summary finding that I would share with you, is that a lot of people just keep mentioning hallucinations.

And, and difficulties with getting the AI to perform the way that they want it to. the way that it was structured, as I said, everybody was there with a single purpose. There were interactive solution circles, we would split into groups, collaborative workshops, peer-led problem-solving outputs, which are, gonna feed into a white paper. the key themes that we discussed, and I'd like to open it up to the audience a little bit here, is, you know, AI governance and validation, human-in-the-loop responsibility, predictive and data-driven quality systems, and AI literacy. And I just read a slide at you, which is something I swore I would never do.

So, on that note, you know, from these themes, is there anyone, that would like to comment on any of those areas? I'm happy to… Go off script here. to pipe up.

Sean, this is Monica. I know, I wasn't able to attend, I'm sorry, Portugal sounds a lot more fun than where I was, San Diego. So, I was in, I was at the Advamed MedTech Conference, and there… we… it was just… AI was all over the place, right? And it was very much the things you're talking about here, so human in the loop, responsibility, the, governance and validation and hallucination. People just kept saying it over and over and over again, and I think what's happening right now is humans haven't caught up to the thinking yet, because yes, yeah, hallucination is a risk, but we got it, right? We're on it, we're working on it. So, at this time, I don't think, in my opinion, and some of the folks I've talked to, hallucination

is not the boogeyman it was maybe even a year ago. I think the bigger problem is cybersecurity. And so, that's just kind of my takeaway from 3 days of being immersed in… what should have been med tech and wound up to be AI, so…

Mmm. Thank you. So that was Advamed. I was not able to attend. I was at RAPS. So then, Elizabethan. Might need to promote her. At any rate, okay, so… We will… we will circle back to her.

But, if I can add here a bit, this human in the Loop is… we… we brought in this session, so Marina led the session on… with a journal club, where we look on the studies already on the use of AI, On the human side, so… so not necessarily if it works, does not work, is good for good compliance, but the human component out of it. And we… we… we are all from healthcare, so the… we start also having these studies from… from doctors using AI, and how not necessarily AI plus a doctor is better than a doctor. While AI may be better than a doctor. In efficiency, the combination is not necessarily better, because the overloads of information that AI provides to the doctor or to an expert, like us, make us to spend more time to review and assess things if you really want to do a critical decision.

that in the end, will be quicker if you… we were the one doing the research. If you were… we were the one probably compiling some of that information. At a certain point, there's a bit of this discussion where the human-in-the-loop aspects is a big challenge, because we really want quickness, then you just skip… are you really doing a critical judgment on the, on the content? And if you really want to have a human in the loop. are we really being more efficient with AI, or are we just being pushed to do a kind of… of… checking, and I often say that AI is the best liar in the world, so it's really tailored to lie to us. Modern hallucinations is how we write things, how we convene things, how convincing it is trying to tell us that that is the truth, that makes it really hard for us to understand what is not.

true or not. So, it's easier to have a junior person, and you really understand it's not secure in some things. The AI becomes quite secure on himself and becomes a challenge. But yeah, I think this human-in-the-loop aspect that we have been learning with the med tech industry, how really AI is not the holy grail for healthcare, with any flavor, so there's specific flavors, how this for our profession of quality assurance and regulatory is also something to have in perspective. But yeah, so this is my point. I think, Faisal, you wanted to add something?

Yes, I mean, I think it's… it's a very interesting… to go a little bit more into detail about this, I mean, AI, obviously, when we talk about AI, it's machine learning, or are we talking closed algorithm or open algorithm? The world is certainly moving towards open algorithm, it seems, more and more. When it comes to validation, that has, you know, that plays a role a little bit. What are we talking about? And then, with regards to the human in the loop. I mean, there are… every technology has built into, you know, the way that it operates some endemic flaws, or what makes it work good is also what makes some things be, you know, flaws, you could say. And with AI, what I'm hearing… what I'm not hearing enough, perhaps, is

you know, we need human beings… well, there's something called AI drift. So, certainly, humans, we have a tendency to pick that up very fast. If we're working with AI. So, I would love to hear that more, that we need humans to be looking at AI drift, and this is going to be an area. And then there is the fact that the world changes, so retraining, AI retraining. If you have a physician, but now there's a pandemic, and the physician doesn't know there's a pandemic, or doesn't have data. So there's that aspect as well. So I think that… I love that we're talking about AI, but in the kind of the more detailed, grainy area.

when we're discussing it, I would love to kind of lift out some of those human-centric areas to work with, to say, this is the area, we need humans here. and regulatory can do this or that, and this is another area, so that it isn't so wishy-washy, in some sense. We're all bedazzled by AI, and I love using AI for a number of things. I think it's fun. But I think we're all sharing those, like, kind of questions and concerns. Just not hearing enough of those… those things, I think.

So, I wanna, speak on behalf of Elizabeth Ann. She had said, that she was having technical difficulty joining, but, perspective… From a… from a lawyer's perspective, is how can the use of AI be validated, particularly where it's used in activities such as post-market surveillance. And I think that's… that's a valid… point, because what I see is that if you ask the same question twice in a row, you often get two different answers. And so the idea of, creating a

query or something that you're gonna submit to the FDA, and you're gonna say, well, this is what I submitted, and they're gonna get a completely different answer, or at least somewhat different answer from the answer that you're getting, is how are we gonna reconcile the fact that you can't replicate the results? It seems to me to be a huge problem from the perspective of something like post-market surveillance, where you're saying, this is the research that I did, here, replicate the thing, and they're not able to replicate it. You know, and they go from Claude to ChatGPT to Gemini to whatever, they're gonna get, you know.

you know, either wildly different results, or, you know, inconsistent results. So, it depends on the platform that they're using, and the level at which they're using it, and the prompt that they're entering. So, that would be my concern.

Yeah, something we explored, so I organized a quick workshop on what we call VIP compliance. on, like, not that everyone is a developer with VIP coding rights, that's how we can do the basics of VIP compliance. And the reality, and we at Compliat, we really embrace that, is our field is quite plagued with unknowns and knowns, and we as experts, many of us as experts, we have this rule of coming to SMEs, to companies, and helping them to think about questions they may have not thought about. For me, the power of AI is that it allows a PG student, when it's doing an initial thesis, to think about, help me to do a risk workshop.

And think about risks, that's probably only 10 years later, they will be thinking about when they reach to a more expert person to help them through this stage. Are those risks correct or incorrect? Probably those… this is not the question, but just making people question if that content is relevant or not for them is already, for me, the biggest power of AI and how we can use AI today for compliance in a more free way. Of course, cybersecurity's being an issue, but But I don't care about hallucinations many of the times, but I say, I want hallucinations, this kind of thing that is good to making thinking. When we go to something we need to validate, to make a decision, okay, this is…

we cover all the risks. That is where probably AI is still shortcoming, and this is where the validation of these methods become a challenge, but the power of just filling a void that exists of knowledge, I think, is quite relevant. Not expecting that that becomes the only source of knowledge, but… Any AI knows more than any clients I haven't had, so if they can… if they just speak with AI before they're speaking to me, it'll save a lot of time just on teaching them the basic things of what is risk. Even yesterday, I was discussing with a client on how he's saying, oh, my AI model does not have a risk, does not have bias, and how could you, like, here are 10 examples of how that can have bias, and he had to agree with all of my examples, but for him, he was rocket solid, no bias.

this case doesn't have any risk, and this use of AI, of really having an expert in your pocket quickly, I think, is quite a powerful use of AI. With, of course, the limitations on the full cycle of compliance.

That's great. So we had a few comments, in the chat. If I use OpenAI, I ask for the source and response to the answer at the same time. As asking for the question, yes, definitely helpful. When I'm using transcripts, I ask for a timestamp just to make sure that they're not hallucinating the quote. And then I ask them to verify that they're not hallucinating the quote, because it's really important when I quote people that I quote them accurately. And Luke said, whereas AI is used to do or help with human work over time, it will probably be tempting to use AI. For the human part of the human in the loop.

Yeah, that's a… that's a distinct possibility. The, the co-founder of, OpenAI had raised his prediction recently to 20% that, human beings will actually become slaves to AI at some point in the future. And so that is a distinct possibility. I think that we have to be aware of the fact that, we're playing with fire in some respects. This is a… this is such a powerful technology. That is gonna have a sweeping… Impact on everything from education to how we learn, to how we think, to how we structure ideas. how we present ideas. I've noticed recently that a lot of people have started saying deep dive, as part of everyday English. It's not part of everyday English. It is now, thanks to AI when they're deep dives, but, you know, frankly, before AI, that was not a common English expression. I mean, yeah, it existed, but

Did people say it? Every single day? No, absolutely not. So, that's just my own… Personal thing.

Can I add here? I think it's, it's, it ends into how… we implement critical thinking through what AI gives, and of course, and we sometimes can get lazy, and we ask AI, we don't verify enough, or we don't do this validation, or like Luke was saying, we can use the AI to validate itself, to be like the human in the loop. And maybe even asking other AI to validate what another AI said could be even a good strategy to Add an extra level, but then… should add still a human in the loop, also because of… and that was also one of the points, one of the sections of our… one of the workshops of the retreat, is that, is AI making us dumber, or, or not? Because if we get… we ended up always being too lazy to do this critical thinking, like.

what will we have in the end? And like, like, like Sean, you were saying, Kuch, from… from the co-founder from the OpenAI, like, will AI, like, we… will we be slaves to the AI? Like, in the end, if we won't be able even to do critical thinking, and we are dumbifying just ourselves, and just being another loop in the, in the… in the AI, like, what is the purpose? And I think AI can be a very good tool for us even to improve our critical thinking, or to improve how we learn and get to know, learn more, because we have a huge access to information. the thing is, if we don't know enough, we don't know if that is correct or not. Even if we ask for reference, we double-check everything and all that, but,

up to a certain point is… it's a bit up to us, and the worry thing is we typically end to be as lazy as we can, so we will tend into, like, reduce this critical thinking, most likely, and that… that is the worry of… of, Of this, of how… instead of using AI to amplify us, and so that we can improve ourselves as humans and our knowledge, we ended up just being another loop within… within that, and not adding our extra value or our extra critical thinking to the… to the loop, and our extra, interpretation of something to the loop of the AI, because that's what the AI is not able to do, this extra personality, because it just merges everything, and everything starts to be a little bit the same, because it's the same rationale all over, again.

Yeah, that's what I wanted to… to add here.

Yeah, we did, we did have a discussion about, you know, the impact on human thinking and the studies that have been done on, the few studies that have been done on, you know, does it make us lazier? Does… does it… does it actually diminish critical thinking, or… or does… does it raise it up? And I think that… and Luke has his hand raised. Go ahead, Luke.

Okay, I think it's not a question about people getting dumber or smarter, but that AI is getting dumber. I mean, AI is built on a limited data set that's in the world, on the internet. And if you remove porn, cat movies, and grocery list, I think you have 50% left. And of that 50%, a lot of it is bullshit. So, it will be an, I think, a challenge how to filter out valuable content. And… the bigger models you make, and the more data points you use, it will not make your AI smarter in the end. Especially because it's a kind of self-reinforcing, like, there's a lot of more AI content coming onto the internet, which builds on more AI content, and it doesn't make it smarter.

You could argue, well, if you only use, credible sources, like, like, scientific papers or something. But even in the scientific papers, there's a lot more bullshit going in, because people produce more, and even if you have… I can make an example for my field of work. there was a breakthrough article on how to measure EEG, and they used a head cap with the ground electrode in the middle. And that's bullshit to use it there, because it doesn't improve your signal. But because it was a breakthrough article, everybody else wants to do it as well. So you have a kind of flywheel of

bad information, bad data that perpetuates throughout the internet. And I think with AI, that's even gonna get worse. So, yes, I think AI at the moment will… Help a lot, because you get, you get new insights in a fast and easy manner, but to go beyond that level, I think it will be almost impossible. So I'm not afraid of AI becoming, Better than humans, at least some humans.

Thank you, Luke. Nellie, you have your hand raised.

So, I… I think it's kind of the age-old kind of… if you think about it, if something makes your life easier and you don't want to use that part of your brain, then you need to compensate it and use the other part… other parts of your brain. So I think it's just really the people who are gonna be… dumbed down are going to be dumbed down, and the people who aren't are not going to be. When I use AI, I use it in a project capacity from… load it with something I already know the information, just to verify I wrote it correctly. So, I'm not… I'm still using my critical thinking, I'm just making my critical thinking more effective. And so, I think that's really… it's just human nature. You're gonna have the idiots, and you're gonna have the smart people, and the people who want to work and know things will work and know things.

Is that… I said… I hope I said that right.

Yeah, no, no, I agree with you. I use AI quite frequently in my work, and what it allows me to do is do things that just took a lot of time before, like create this PowerPoint presentation. And, you know, this PowerPoint presentation was created in a very short period of time. Based on a large amount of content that was produced at the event, was carefully considered, ingested, and so… But yeah, in the old days, creating a PowerPoint presentation could take days. And, when we used to have to write research papers, we'd go to the library, we'd go to the card catalog, we'd go to the stacks, we'd go to the books, we'd look up in the index what page

That comment was on, and then we'd find out that it was meaningless. You know? So, a lot of that manual research that was done before is being short-cutted. To Luke's point, the amount of Nonsense being, sort of, flywheel effect that's happening from that. That is true to a certain extent. So, why does MedTech Leading Voice matter? Why does the exchange matter? It matters because it is the voice of the expert. that we capture, and it is the voice of experience and knowledge that has to be somehow preserved, and brought forward. Otherwise, we do risk…

perpetuating misinformation on top of misinformation. And so I think that the role of the human being is shifting, and in a conversation that I had with Miguel and Celia over lunch, We had a long discussion about the idea of the things that machines do quickly and well, and the things that human beings do. Equally well. And the overlap is… is not a lot. There. And it is in the creative process, it is in the judgment, it is in the original thought, it is in the decision-making, where I find that human beings excel. You know, a lot of times.

I ask AI to summarize something, and they just miss the narrative spine that I'm looking for. And I go back and I redirect it to the narrative spine that I want to draw out of it. And then ask it to redirect. Is it the best writer in the world? No, it's not, you know? Not yet. It will get better, you know? But… And I see that Leonard Leo has his hand up.

Once I get to the mic. So, things that AI is not good at is things like when there's discussions in the room for developing standards, it's not going to know information like that, for sure. So, for standards developers, you know, they're… For 4th edition of 601, we're writing a lot of rationale, but there's even more information than that that's discussed that's never going to be in the document. That's in the people that are developing it, the people, the regulators, the manufacturers, the academics, the trade associations that are sitting in the room making those discussions and making those decisions. That's never going to be out there in the public.

And even the standards themselves, they're never written perfectly. That there's full, clear, knowledge how… the requirements are written. They're written as good as they can be, but there is room for interpretation. So you need the experts in the room still, at some level. And IEC60601-1-2, the EMC standard currently, is written so poorly in some aspects. You have 3, 4, 5 pages long for one requirement. You can scratch your head and look at that requirement and go, what is it saying? And interpret it probably 5 or 6 different ways.

Excellent point. I think human discussions are absolutely the thing that set us apart from the machines in our ability to interact with one another, and that's why the event in Portugal was so valuable for me. Faisal, you have your hand raised.

Yes, I mean, I was thinking about what Luke said, and also what you said, Sean. And really, you know, we are using AI, all of us. more and more of the data that AI is going to consume will probably ultimately be from AI, or a large part of it. And so, one of the issues, we're maybe touching it right now, but it'll probably get worse, is when AI is, to a larger and larger, you know, degree, consuming its own data. So, that also is probably… why human beings will constantly be necessary to engage them, to activate them, to have them speak and not just consume information from the internet, but actually, you know.

create content and have discussions, so just to… I wanted to make that point as well. I see that in this discussion, certainly.

Thank you. So, we had a gadget-thon with AI tools, and there were 7 presentations at the thing, I think it was seven, and… We looked at the things that people are doing with, digital twins, and prompting, and, human review. I did a post about that, but I had to take it down because I actually got, one of the people's names wrong, and it was, so I'll try to resurrect that, and share it again in more detail.

Cool. You're the part of digital twins. That has been promising for years, and were there actual results presented there?

No, it was more like a how-to use the gadgets, you know, like how to create a digital twin for yourself, and How to, how to create videos, how to, you know, it was, it was like, it was like different AI tools that were explored.

Yeah, not on digital twins from the healthcare perspective sense.

Oh, right. Okay, also in production, in machinery, a lot of people say we make digital twins so we can predict maintenance. And it was always very promising, and I've never seen a commercial Digital twin taking off yet. Have you?

Jumping on this, some of the times, I've been working in innovation for a long in startups, the moment you start speaking about the buzzwords is the moment, probably, those things are being used, you don't even… are aware of this. You're speaking about digital twin. Does Facebook have a digital twin of you when they try to send you ads? This does not end to be a digital twin in some way, so the full extent of this kind of rationales probably they blur a bit when they really get into practice. Of course, we have this hype of blockchain is the next thing, and I remember when you, with a piece of paper.

Gladly, AI was not present there, because just doing a white paper back then, we could raise $20 million, 30 million, which is a business plan without nothing to support it, so imagine just being able to do it with AI. But the reality is this, today, we have blockchain being used in quite different applications. Probably not the same thing we foresee back then. And I'm saying this because probably we don't heard about digital twins in some promising areas, because they are just being used in some capacity in certain things, probably not with the full extent of magic, of… you have a digital twin of the city, and you can predict the outbursts of flu in real time kind of stuff, but honestly, I've been seeing a lot of what can be considered a digital twin application.

Okay, I stand corrected. I think you're right that Meta and Google, they do have Digital twins of our lives.

Yeah. But it's an interesting topic, because when we speak about AI, it's because it's so novel, but in the end, we may just be speaking about content creation, or compliance… automatic audits. For instance, why do we have internal audits with humans, when probably you can do internal audits every week with AI, that probably delivers better results in compliance than having someone going one year to a company, and you pretend some of the times you're doing certain things that you are actually not doing. Just giving examples of… we speak about these technologies, but the moment you stop speaking about them, often it's because they are… it's where they start really having applications, and they become secondary to the discussion.

I actually think the example of internal audits is a dangerous one, because now then, how do you get to the point where you qualify your AI? Because if you're getting a different answer with every AI, you're creating the same exact issue that you have with humans, whereas with humans, they can do an intellectual assessment of it and do deeper dives. So I think… I wouldn't use internal audits as an example, because then you have the whole realm as, how did you qualify that AI? AI to do the internal audit. But I understand what you're saying. I would just, I would just, you know, tread lightly there.

Okay, let's move on to AI and design and development. You know, what were the key takeaways from the event? How can AI, accelerate… Design and development. Any… any comments? hear from… You know…

Is it for physical products, or more for, Like, software things, like software as a medical device, or…

I haven't used either, really. I think it's either one.

To me, I'm a research and development engineer, and I haven't found any useful application for AI yet. Especially for the large language models. And it's what they say they are. They predict every next word, so it's especially suited for applications that work with language, whether it be human language or computer language. You can see that it takes a lot of… it takes off on programming, and also in writing documents. But for designing physical products, I… I… I don't know, I haven't seen a good application yet. And you can say, hallucination is not a big thing anymore, but that only counts for the hallucinations you recognize directly. It's the more subtle, hidden variations that are in the answers that are more dangerous, I guess.

So, my two cents, thank you.

Andre?

Yeah, Luke, to take off on what you said, you know, we're also developers of hardware and software. We make a lot of physical products in the medical device field. And what we've been doing now with it. is something like what Nellie had mentioned before, is that, you know, we're creating, let's say, a bill of materials or a schematic for an electronic product, and if there's some question on it, instead of just going in cold into the AI, is showing what we've done and asking for, for example. alternate components, or alternate design considerations. Now, it's not always great, but it does give you some different… some different alternates, some different perspectives. So, you know, we're implementing it now on the physical side for actual hardware units.

That's very interesting.

Actually, jumping here, this is the promise of the digital twin. It's how really can you do a representation of… in information of the physical worlds, and then you can deploy AI at a larger scale. Probably not so sexy for ChatGPT and OpenAI and Google, because they are software companies. But all this hardware space, the moment you… we already have a quite digital world, right? The components, the diagrams being vectors, so they are quite digital, a lot of the information. We probably are not investing the same way, on that space where Probably… these companies are investing on the content creation kind of space.

But I don't see why the physical will not… probably will be much easier, because at least we have some constraints that you can put on the physics of certain things that, of course, you cannot do it with words. That's, of course, why the videos fell so much with AI, is because they are built into models. But there's a lot of research on AI being done now on new ways of framing concepts that are beyond LLM, so LLMs just… is a revolution, but it's not the only technology that we should expect with AI at this stage, so…

So… Luke, one of the things that, I find most helpful, when I'm trying to create something new with AI is to go to it with, an hour-long transcript, Of a recording of myself. Saying, this is my problem. This is the problem that I'm trying to solve. These are the things that I've tried before that didn't work. This is the outcome that I'm trying to achieve. I want help thinking through this problem. And I'm not able to come to an answer on my own of exactly how to move forward. And what I find with AI is that although the answer that I get at the end is not that it develops a new product for me, or it creates something out of whole cloth.

But rather, it helps guide my thinking In a way that is constructive, directed, and is able to bring in pieces of information. How many times have we thought, oh, I should make a t-shirt that says. this, and I'm gonna make millions of dollars off it because it's so funny. And then you go on the internet, and you find out that 100 people are already selling that t-shirt, and that, you know, your idea wasn't nearly as original as you thought it was. So, what I like to do is I like to ask AI to validate how original my idea is. Or if it's just commonplace knowledge that I just thought that I thought of all on my own. And so, the fact that it is ingesting the entire World Wide Web

The fact that it is generating content off itself, the fact that all of this is available for me to double-check my sanity against, that, and to ask it not to blow smoke at me, but rather to critically analyze the content that I've given it, and to help me, create something new from that. I find it very helpful. Personally. But I'm not… an engineer. Caveat, I am not an engineer.

But jumping here as an engineer, yesterday I was doing an exercise, we had some… some technical challenge in our platform that requires a bit of architecture thinking as engineer, and we are gathered with all the team, like, we are getting nowhere, or at least we had more needs. And I just jump into clothes and say, here's my issue. Propose implementations on how to sort these out. But propose a lot, so… and then… and I kept asking, like, propose more, propose more. present the pros and the cons, and in the end, that was a way, as in 15 minutes, I was able to get inspiration. When I say inspiration, I was able to… I think, Andre, you were saying, like, finding a component that probably I didn't remember from the first place does not mean that the moment I see it, for me, okay, this makes sense.

Because I'm an expert, and I know this makes sense, but… the reality is that opening… this brainstorming, so the deep dive, Shadi, you're mentioning, I think that's the powerful thing of AI these days, is how can it give us a lot of information consolidated that accelerates a lot of what we could be doing. on this inspiration, or anything about the products is this kind of things, thinking outside of the box, because the reality, most of the things we think is not new. It's what I say the average starts supply work, like, if… already… don't… you don't need to explain much further. I know exactly what they're doing. I have 10 other clients that do basically the same. Is the reality that, that many of the cases, a lot of the problems we face.

And the approach was already documented somewhere, and AI is an amazing tool for that. But we need… this can be the tool for the full. If you don't know what we are doing, or it can be really powerful, because we can do in probably one morning, which may take a moment of re-engineering, of testing things, exploring things, just to find out that someone already tried that and didn't work. But on the engineering side, I see huge potential, but I see really the challenge of And actually, I think we were speaking on that also on lunch. The issue with the junior person, like, because if you do not able to have critical thinking.

this is a hard tool, to evolve, but if you really know a lot, and you can keep on the side of learning things, and being… keep the critical judgment, this is a tool that exponentially, allow us to do things, much bigger. Speaking about internal audits, Nellie. how about human… human AI internal audits? In one day, you can probably do, an audit of everything. And yeah, okay, find the things that probably are just missing, and you go directly to the things that you… you will not miss a lot of things on the sampling process that probably AI helps go through that process. But again, does not mean that you will look into that. It's a matter of accelerating and scaling what we can do.

And that is a challenge. How can we keep producing exports if we now subject a bunch of people that no longer need to think about this FDA submission, they just use a platform and click next, next, next, next, next, and they have a FDA 510K submission ready to FDA. on the other side, and now probably FDA just, okay, we just want to promote innovation, have a next, next, next, next, next evaluation AI system on the other side, and you just have a someone that I have no clue what was even approved, or someone that just created documentation without critical thinking, and in the futures. Those will be the people that need to replace Leo in the standards, right? And where there's critical thinking that we need for creating new content? For me, is the biggest concern, is how we really need to think as a society beyond just individuals.

how AI really can… Can be sustainable long-term.

Yeah, I think… I think you actually hit on what I was saying before. You have to have the smart human working with the AI and controlling the AI. If you don't do that, then you kind of run the risk of, how did you qualify it? Like, for example, when I do an AI project on ChatGDP, I have projects where I've front-loaded standards and procedures, and I ask a question. When I go to use it, I will go back and say, please remember the guidelines of this project, and then I start moving forward. Because I've literally put the same thing into what Sean was saying. I've asked the same question 3 days in a row, not because I…

was testing ChatGP, my brain didn't remember that I asked it already, and then I got 3 different answers, and I went, oh, wait a minute, I've got 3 different answers. They're all about… they're all in the same venue, enough to where I can take and shape it, but most people cut, paste, and just don't think about shaping it.

Hmm. Good point.

And perhaps you could even ask the same question and have different outputs, and then you could compile on it. Maybe it has a, like, having someone with experience and knowledge, and use that different outputs as a way to… to better know or know more deeply on that subject, like Miguel was saying, for example, in terms of brainstorming a topic, okay, give me more options, give me more options. Also, as a way, okay, is there something else that I'm not seeing? So that you, with the knowledge and with the critical thinking, can better work on it, or map it, but for sure, this person that is an expert

is a critical thing here to know and to avoid through the hallucinations, or the just things that are just filling the text, because AI loves to write a lot of stuff just for the sake of writing, and it doesn't add anything, so you can go through and add something that is valuable on top of it. Yeah, and I think that that's the key point of how we can exponentiate our work or our knowledge with AI, with the support of AI.

We became experts because of the hard manual labor and all the errors we made and learned from it. I don't think it's feasible to become an expert just by using… AI.

That is a tricky thing, and I think was what Miguel was saying, how… for us that have already some experience, like you're saying, with failing an error and learning from that, I think the tricky thing is how… how should we train new professionals If they, first, they don't know what should be right, because they don't have yet that critical thinking or specialized or critical knowledge on a certain topic. And how… and they will not pass through this fail and error, at least as we did, because they can have AI that can guide them more. So, I think that that's a big key question. How can we train new people, into… into this era of AI? Because I think that that's what we don't know. Like, what is the future of

People that don't have that much experience yet, and how will they be able to The experience to be able to do this critical thinking and support on top of it. Yep.

Okay, so I wanted to get through… Ugh. Ready to go.

You stopped sharing, just…

I stopped sharing because I didn't want anybody to see how messy my desktop is, and how many tabs I have open, but… what the hell? So, we were here. I want to, you know, just quickly touch on, AI and post-market surveillance. AI, human cognition, learning and career development, leadership and culture. These were the topics that were covered. I'm happy to share this deck with anybody, not that it's… super explanatory. I can also share my notes. But this is the stuff that's gonna be, in the… in the final report, from the event, is like, you know, these were the things that we came together to discuss.

I think, one of the things that was evident to me from going to RAPS, from going… from all the events that I've been to this year, is really the importance of human discussion, and I would really like to, expand that idea of, you know, yeah, you're… you're… you're probably not… gonna be really well off with AI as your only teacher. You need to pull in all of the tools that are available to you as a human being and use all of your senses And that was sort of the point of the cooking post that I did recently. I asked AI to explain to me why is it that, you know, after a long day of thinking all day long, I love nothing more than

than cooking, and it said, you know, it's because it's this time-bound activity, you're in a space, you're using all of your senses, you're using… your perception, you're using your timing, you're using… you know, we have, you know, AI is not exactly one-dimensional, but it's closer to that, and the ability of the human beings to think in time and space and to weigh cognitive, emotional value versus, you know, dopamine versus a lot of other things. We have an advantage over the, over the machines in that sense. And then I wanted to share, something that, came up during the thing, which was,

you know, what… when we talk about AI literacy, and that was one of the scopes of the… of the event, you know, what are we coming up with? And, What was introduced was this idea of, you know, it's a lot like driving a car, and so you start with, you know, what do you need to know about it? As a pedestrian, you know, recognize AI in daily operations, understand core terms, see where automation influences outcome, versus when you get your learner's permit and you start to learn how to drive a car. And, operates AI tools for defined tasks, validates results, maintains judgment and human oversight.

Right? Whereas the skilled driver, when you become a skilled driver of AI, explain how the model generates results, identify data or prompt bias, train colleagues on effective ethical use. And then, the mechanic. You know, these are the people who are, actually able to do coding, unlike me. And, you know, adjust the prompts, document governments and validation. design training materials for consistent use. And then finally, you have the architect. behind the whole thing, who is, coming in and doing something at a very high level. And I thought this was an apt analogy, I don't know if anybody else did, but,

You know, when we think of a car, it's a tool. It helps us get to a place that we want to go to. You know, a lot faster than walking or riding a horse. And, so, how do we use that judiciously? And, You know, that's, to me, the larger question. that I would sort of leave everybody with is, you know, and welcome your comments on in the few… well, we don't have any remaining minutes, but, you know, feel free to stay on the call if you want to talk about it, and I will stop my sharing.

I'll even add, probably, that there's a missing rule that is… I'm not… I don't know if the regulator or, like, our rule of the compliance many of the times. Because the reality of the car can kill people, the reality is AI can also do a lot of harm. And in this kind of aspects. It's so easy to do things with AI that many people do it. It does not mean that they can do it out in the public. That they cannot do it… like, you can drive any car that you want in your own farm, at least I think it's a common rule in most of the countries. You cannot drive your own made, self-made car out in the streets, but we do it with AI, and that is the reality of challenges that

I love to push through regulation. I'm an engineer, so I'm not a lawyer, but I love regulation because it's something… it's a rule of society that allows us to trust each other. That's why I walk in the street and not being killed all the time by a car, because we have regulations, so the cars… work, the people have some basic capacity for that. And the reality is this literacy, this capacity to run things, this… being… who is the engineer that's signed off that AI model into the world and becomes responsible for it is the kind of thing that… it's not just AI model, the use of the AI on certain services, the kind of thing, probably, that is also a key component of this journey.

Well, I'd like to thank everyone for, joining today, participating in the discussion. I'd like to remind you that next week, Robert Van Buxtall and Yellow Schwar and… are gonna be, presenting on, state-of-the-art. claims and strategy, and the connection among the three. I think it should be an interesting presentation. I've spoken with them at length. about what they're going to be presenting, and I hope everyone will join us. In the meantime, please support one another, and Reach out if you have any questions, comments, or concerns. Thank you so much, everyone. Thank you, Miguel. Thank you, Complier, Celia.

Mariana? Have a great day.

Yeah.

Hi, everyone.

Bye.