AI for Regulatory Affairs Unlocking Insights from Unstructured Data Featuring Michelle Wu, CEO, Nyquist AI

So while we're waiting to get started, I'm just gonna do some shameless self-promotion here.

No problem.

I'm gonna look at… So… I just did this post with the video. Me… Posts and activities… There it is. So I'm gonna take… the link, and I'm gonna say, copy link to post. Go to the chat. Go to… Everyone. Please. Show some love.

So… Love. And then there we go… So next week, we're doing…

Let's see… go here… go to events… Mapping IEC 62304 to E-Star, fast track FDA clearance for medical devices with software. with Jose. And… Copy link to post… And they're already, 300… Almost 300 people registered.

Bring that in…

So, while we're waiting to get started, michelle. How was RAPS? Did you enjoy it?

Yeah, it was great, like, it was fantastic meeting you, and we've been going to RAPS for years in a row, and, have seen a lot of changes. It's great to see the old friends, and it's also, like, a mini renewing, and meet new friends. I need more coffees in my system. And, I actually just come from Chicago, another conference, so, very interesting. You know, some constant themes, there are also some, like, different themes.

what, what conference were you in in Chicago?

It's called, Generis American Medical Device Summit. So, the participants are more like the decision makers, like the, the VPs above.

Oh.

Okay. Yeah.

And how many people were there?

It's a much smaller conference, about 200 people.

Well, I mean, you would expect that from a decision-maker conference.

Yeah.

Did they have booths?

Yeah, they do, they do.

Did you sponsor it?

No, this time, I got lucky. One of my clients, who, ECI, they are a consulting firm, they are the primary sponsor, and, the… so they have a booth there, they have the largest booth, and they just invited me to talk about, you know. AI in regular affair.

Oh, okay. That's cool. Yeah. So, we're gonna get started in about a minute. I just want, to… make some announcements. So, next week, we're doing a webinar, with Jose and Rob Packard, who many of you know. Please make sure that you register and, share the, invite link, with anybody who wants to go. It's on… mapping IEC62… 304 to E-Star, it should be, fairly detailed, and, and always fun to have Rob and Jose, in the same room together. So I'm really looking forward to that.

Melissa's super excited about the, new program that we launched yesterday with Donowa Life Science, CRO, based in Rome. If you're interested in, talking with Donna… Donawa about their referral program, and having them refer business to you, and you refer business to them, just write a referral in the chat, and I'll connect you with the right people. At Donoa, but we launched the 5-day email course. And what's interesting is the idea of doing a webinar, and then following it with a 5-day email course, which not only reinforces the lessons in the webinar itself. but also adds another dimension. It adds resources, it adds the slides, it adds the video, and additional detail that couldn't be covered.

In… in the course of one hour. So it's a way of taking the content, expanding it, and getting more out of it. Ultimately, where I see marketing going, and Michelle, I think this will interest you, is the, it's really gonna become a lot more AI-driven, it's gonna be a lot more interactive, it's gonna be, a lot more multidimensional. I think one of the things that, I was talking with a… a prospect about the other day, was the idea that, you know, the whole idea of our brand kits being 2, 3 colors, certain fonts, that all comes from printing.

That all comes from the restriction of 1 color, 2 color, and four-color printing. We're living in an era where we're not bounded by any of that anymore, and I think that people really need to sort of free their minds and start to embrace the, possibilities that AI represents, not just in terms of marketing, but also in terms Of regulatory affairs, daily workflow, how we define our jobs, how we write, how we research, how we… Consider ourselves to be experts. I think the brilliant thing about AI, and this is sort of a thesis that I've been developing, is the idea that it allows human beings to be slow.

Because it's so fast. And by being slow, I don't mean stupid, I mean… it allows us to go deep into the subject matter areas that we're actually experts in, and to take that expertise and share it with the world in new and more interesting ways than we've ever seen before. And so, I'm hoping that, what we're gonna see today, what Michelle is gonna share with us, is gonna inspire you as to how, You can incorporate AI in regulatory affairs. Without another word from me, please make sure you see the links that are in the chat, show some love to the post, and… I…

I'm gonna turn it over. Michelle.

Hello, everyone. Hi, I'm so, happy to join Sean's, like, Friday chat, and, to talk about something I'm very passionate about, AI and regulatory affairs. how we can help regulatory affairs to be the superstar, by the power of AI, and I'm so happy to see some familiar face. Not surprising, like, Xiao is a super connector in the med tech, and I saw, like, Carrie, I saw David, and, I see a lot of, like, new and, old friends, like, joined the, joined the webinar. That's fantastic! It's a great way to celebrate Friday, and especially today is Halloween, so, like, I prepared some candies, as sent out, and also.

I make this, our talk less… formal, everyone know me, knows that I love fun, I like to, you know, make the talk very engaging, and today is Halloween, so it's a little bit spooky, and I welcome everyone, come out of, you know, mute and chime in. This is a dialogue, and rather than a one-hand presentation. And before I start, I want to quickly introduce about myself. I'm the CEO and co-founder of Nyquist AI. Recently, I had the pleasure to meet Sean and, work with him, and also, like, learn about his tricks of using AI to create influence and education, even though there's already a lot of education about AI, but when it comes to regulatory affairs or, like, medical device, they're still allowed to learn. And I feel like I'm a student all the time, and would love to hear your thoughts, would love to hear

your experience of using AI, or, like, your confusion or feelings about AI. And, as for my, professional careers, like, I recently got recognized the top 100 women in AI, which is a big honor, and, super excited, and I, I owe, all my… gratitude to you, and, thank you, EA, and, thank you, everyone. Like, I learned so much from you guys, and prior to That I worked for a Boston Consulting Group and, Novartis in medical device and biopharma. So, a big nerd, and would love to let the nerd inside of me shine with the nerd inside of you. Let me quickly open up the presentation.

Bye.

How's everyone doing today? Go ahead and put it in the chat.

Yeah Can everyone see my screen? If you can see my screen, like, please type 1 in the chat, so that I can see that, or not, so you can see the screen. Okay, awesome, awesome, awesome. And, be, like, I want to see some, like, show of hands if you are in medical device, can you, you know. can you send a, emoji? Like, if you're a medical device, I think there's some, like, okay, reaction… And, and if you are in pharma, maybe, like, type 2, yeah. I see a lot of, medical device folks, that's excellent. Oh, some farmers, friends, that's awesome too. So welcome, welcome, welcome. Today, we're going to talk, have some fun, spooky, Halloween theme of, discussions, like.

top 10 mistakes, AI mistakes, medical and pharma leaders can make with AI, and how to avoid them. So, a couple of, rules, so this… this is not a presentation. This, yeah, like, this is more like a discussion, so it… it, involved a theme of, like, Halloween and AI, the two most, like. spooky and… but also fun topics to discuss. And number two, like, in order to break away from the evil. I encourage everyone in the audience to be brave and join the conversation, either put in a chat or, like, just come out on mute. I do much better when I have a conversation, and if I'm just talking to a screen, I feel very, very scared.

And number three, and be aware that each spooky question comes with a special trick. If, you answer it in, it incorrectly. Are we ready? Okay, let's buckle up and go! So, what will be the number one mistakes that, you know. people make about AI. Like, they will think AI is a headhunt haunted house. Have you ever hear or questioned that oh, AI is going to replace all my people, diminish my team size, budget, and power. Do you resonate with this, or have you heard that like, people question this. Sean, you are the experts in med tech. You have such a strong connection with all the leaders in med tech field. Have you ever heard people talk about this?

So, I actually hear it less in med tech than I do in other areas, but I.

Oh, interesting.

the… the… the observations that I've seen are that, people are still pretty suspicious of it, and it has a lot to do with the hallucinations, and so maybe we'll get to that.

Yeah, yeah, yeah. I'm going to call on David Chekwitz, and David, if you are there, like, can you chime in a little bit, since you are the leader and pioneer of AI? No? No, you know? Okay. David, you're on mute. Okay, nevermind. So, the reality is that AI does not, replace teams. On the contrary, it unlocks the potential. So, I actually create an example, and, I come from Stanford, and previously I was, like, and Stanford is known to be the center hub of the AI, so on the left-hand side is probably a gentleman, we recognize his face quite often, like, when

AI first come out in 2016, people were talking about, like, AI is going to eliminate all the radiologists, because AI can read all the radiology scans. But now, 9 years later, like, we can see the demand of radiologists is even bigger. It's because that, you know, AI makers, the radiologists, they're so efficient, they are much more efficient and effective, so the demand for the radiologist is higher than ever. So, fast forward to, regulatory affairs, I think, the common theme, so when you, you, you're afraid of when people or leadership get afraid of AI, think AI is going to eliminate jobs or eliminate roles, is, the key… there's three, three things that we can do, is first.

define success as impact. It's not about quantity, it's about quality. And also, see AI as your team's amplifier, rather to, see it, oh, AI can replace the human beings. We've tried this. We've worked with 200 customers globally, and we've been around for 5 years. We haven't seen AI really eliminate any job completely from end to end, and quite frankly, recently, there is a lot of, news about AI eliminating jobs or people, especially from the tech world, where people cut jobs, that I… my personal opinion is that AI is a convenient excuse

for, like, a large organization like Amazon or Google to trim down their team. The reality is that AI still cannot replace human 100%. Let's leverage AI to celebrate speed, execution. And, to come up new measurement of value. So, I'm curious to see, like, you know, any comments and feedback from the team. Yeah, Daniel, yeah.

Yeah, I think, I think there's a misconception about the whole concept, and I think that in part is, when you, you know, when you talked about Amazon, Amazon does a lot of stuff that is repetitive in nature, so that these are basically automating things for which decision It's really not implied, it's already assigned to the computer, you know, this item or that item. When it comes to the pharmaceutical industry and medical devices, I see AI to be supplementing, to a certain extent, a lot of the work that we do, especially when it compiles. Information. But the decision-making process is something that is so complex and so unique, because we're dealing with so many different issues on a day-to-day basis.

And as you see in the pharmaceutical and medical device industry, there is more complexity that is going to be coming up as part of the natural progression of the industry. I… I think that when people tell you that… that they… the AI is going to replace, people. I don't think that the people who are saying this, and I just put that comment on the chat, I don't think they have played with AI. I've played with AI a lot, and I tell you, the more I play with AI.

the more I see that the need for human beings to be part of the decision-making process is just…

becoming more and more important. Will AI help and supplement? Absolutely. But AI will never replace human being decision-making.

Yeah, yeah, yeah, fantastic. To make the conversation more fun, I will put, like, a reward. Anyone come out to the mute, Chime Min will send you a box of chocolate to trick-or-treat, and if you are silent for too long, then I will, like, you know, call you out, and then ask you a question. So, thank you, Daniel, for your comments. Do you have, like, any superpower or psychic, you know, power? Because my next, topic, ta-da-da-da… is about mistake number two, phantom of the boardroom. AI will diminish my leadership, my decision making, and take over the whole thinking process. You literally, you know, just read my note. So, I don't know how, like, whether you use AI superpower to

to, like, get, like, you know, guess my next, topic, but, I totally… yeah, go ahead, Daniel.

I can't… I can't tell you, Michelle, otherwise I would have to leave the group.

Okay, okay, yeah, yeah, yeah, yeah. That's, like, that is… I totally agree with you, and that is, like, the number two mistakes I constantly see people, make, and especially the leadership teams, they're, like, thinking, oh, AI is going to take away all my decision powers, they are going to make, make us less impactful. I had a… I just… I was talking with Sean, I just came from a conference in Chicago, and when I was having coffee chat with the VP at the multinational corporation who has 33 years of experience. He's like, Michelle, you are the young generation. For my generation, when I think of AI,

And the image I have is a ginormous calculator that's linked to the Terminator, that they're going to wipe us all out and take away all our decision powers. I said, like, well, I can totally understand your fear, but a fear should not be a reason to stop using AI, or give it a shot. Like, if you delay, if you procrastinate, if you, avoid it, it's just going to make the whole situation even worse. I think, like, as long as we as medical device and biopharma leaders understand our fear or our anxiety and walk towards it, figure out how to use the AI and understand it, it will actually help us, rather than to continue to eliminate more of the decision power. So, I think for all the leadership.

this is my personal calling, is to learn how AI think, and where it helps you think better, because AI cannot replace your thinking process. AI is almost like your thinking companion. You can, instead of having a conversation with I, me, myself. And AI can really help you to, you know, bounce ideas off each other. And also, you can use AI insight to shape your strategy. and not to, you know, surrender to it all, because AI said so, like, I, I, I also teach at Stanford. Fortunately, unfortunately, Stanford kids tend to, you know, be the first one to just, like, eat off AI, and sometimes I question them, like, hey, John, like, why you write the… your homework this way? He's like, well, because ChatGPT says so, or Microsoft GoPilot says so. I think it's very important to check the results and use AI as,

to find an answer rather than asking questions. And for all the leaders, we should model out curiosity and competence. The knowledge today can be an advantage for tomorrow. And I saw there's a question in the chat, like, I, Arlene asked, how are your or your clients raising the patient AI IQ, and, And then, like, I'm curious, Arlene, does it mean that, you know, like, we help to improve patients, like, AI literacy? If so, like.

I think…

Yeah, go ahead. Okay, for… I would just assume, like, your question is, like, my client raising the patient's AI literacy, I think, currently, there's not, like, be… Be… there… there's not a very good, way to… measures, AI's IQ. I saw, like, you know, ChatGPT and all the… in the tech world, we have different benchmarks to evaluate AI's IQ. There's a… in terms of number of parameters, how big is the model, how fast it calculates. I have a couple papers, big papers, about

AI's, IQ. But when it comes to patients' literacy, I think most importantly is to help patients understand, like, what are the comments, what… which steps, the doctors or the leaders evolved in AI, and how AI derived into that answer. Did I answer that, your question? Yeah, I saw, like, you… thank you for coming up.

Thanks for… thanks for including me. My experience is that there's this ill-founded focus founder for entrepreneur AI IQ.

And by that, I mean…

Data literacy, data dexterity, etc. But… Too often, the neglected stakeholders.

Mmm.

are… Now, increasingly, physician and health professionals are getting into the game, but… I think there are still significant gaps in board members of these institutions who don't have high AI IQs Or understanding, and secondly, more importantly, patience.

Yeah. So my question is.

I mean, this is not an easy… task to increase patient AI literacy and dexterity. So my question is, what are… what are you folks doing, or your clients doing. to improve patient AI literacy Because… As a clinician. There's a lot of pushback. I get…

A lot of… and particularly when it comes to the patient experience, Chatbots are driving people crazy. Yeah, it is.

You know, and there's so much money being spent on the so-called patient experience.

In my opinion, the cheapest way to do that is just pick up the damn phone. And talk to them.

the chats are driving people crazy, so my question is, and I don't know how many of you have had a patient experience recently where you've had to deal with that, so my question is, as it applies to reg affairs, getting ideas to patients, and all the other stuff you guys do. What are you doing to fix that?

Yeah, this is a very pointed and loaded question, and for… ensures, like, how our clients address the AI, literacy of patients. Currently, they… it's very… I want to manage your expectation, and they are… very early in their journey. Most of the time, they come to us, like, the tech strategic partners and say, like, hey, I need… I'm facing more pressure from, you know, patient advocate groups or, like, from my board to improve the patient journey and use AI. How can I do that? I think if we are looking for improving patient experience. Overall, I do agree with you, Arlene, that there's… there's AI…

currently cannot… does not understand empathy and compassion. It's… tech is… it's some degree cold, and so you, there are still elements that a human cannot replace. And second, in terms of the chat, AI chatbot experience that drive people crazy. I personally have an experience with that. Like, a lot of the Stanford hospitals use AI audio to replace the receptionist, and I have to repeat the same question, same answer again and again to to… to go through the next step, because I have a coding experience, I cannot understand where the product design is going. But to your point, Arlene, the product is designed with

by the coders and engineers who does not have deep domain expertise. Well, I do see in the future Actually, this is good news for everyone, that with the diminishing, entry barrier of tech. people who do not need a lot of tech experience to use Lovable, and we are rolling out a new tool where you can actually build your AI agent yourself. Imagine, like, I have an in-air agent. Help you manage your own patient documents, manage your own workflow, you know, doing business development, deep dive on the customer, understand their clinical and regulatory strategy. I do feel that current leaders overhype on the tech side, the…

engineers have been paid way too much. They think technology or AI is going to solve the problem. Actually, I have a contrarian thought. I think people from the industry, aka everyone on the call, you have such a rich experience of real-life needs and domain expertise, you just need to code out your thing, and if you and your product, solving capability is much more reliable than engineers. So I… I do remain very hopeful in the future that maybe someone from the audience can use Lovable or using our AI agent tools to build out, you know, patient recruiting platform, or clinical trial management platform, or regulatory submission platform.

That will be so much better than the ChatGPT of the world. I also have another topic that I'm going to touch on regarding the technology literacy. Please, please stay tuned. So for the mistake number 3, I would see is, like, I hear this a lot of times, like, from VPs of regulatory affairs and clinicals, like. I don't have time for AI. I've been through the dot-com bubbles, big data cloud, my team is too busy to learn AI, so I call this, you know, the ghosts from the past technology. But I think what really, differentiate AI, even though AI is a loosely defined term, what's differentiated AI from the previous dot-com bubble, the app hyes.

and the big data and cloud is because it's not just another technology. It's actually creating, operational or lifestyle shift. When we first start the… today's conversation, Sean says, like, you know, I've seen AI changing people, how… how… how people work, how people communicate, write the email, making decisions, or how they leave their world. live their lives. So I do feel like, you know, AI is already, redefining a lot of the regulatory and clinical process. Take Novo Nodesk as an example. And, we have seen Previously, they have 42 medical writers, and now with AI, and to Sean's point, a lot of the work that the people do can be repetitive and manual. Now they only need 3 medical writers to write all their labeling and regulatory submission, and they upskilled and moved the people

From medical writing into more strategic and, impactful. growth. So, I do feel like, you know, excuse of no time for AI or seeing AI, just another, you know, shiny object is a little bit, It's a… it's a… kind of, disvalue or discount the true impact AI is going to make. Okay, so… what we, what Way promotes, like, you know, for leaders, if they are early in their AI journey, or they really want to leverage AI to revolutionize their clinical, regulatory, or post-market process, they can identify one or two high-value workflow, and that is low risk and high value, and benchmark the industry peers, and, you know, stay in

formed. Even some… I understand some organizations move very slow, or, like, there's… the boards or the C-level IT still want to see people make change, like, change, how others do it, and then copy it, which is very typical for medical device companies. I would still encourage everyone to stay informed and have a dialogue like this, join Sean's webinars and, you know, read his post on AI to see, like, oh, this is how Novo is doing AI, oh, this is how Medtronic's doing AI, like, we can… maybe somewhere along the way, you know, opportunity arise. and you can bring AI to your workflow. the number 4 mistakes I would see is, like, this actually gave me the most headaches. That's why, like, I have this, you know, magic magician here. A lot of leaders come to me, it's like, oh, Michelle Vernachus.

AI should surprise me. Can I just, you know, dump a bunch of data on AI, and it's come up with new clinical trial protocol that nobody has thought of, a new drug, a new medical device, and, you know, something that we haven't thought about? I would be like. you know, sir, or like, you know, Mr. or Mrs. VP, if you meet a company or AI startup that can surprise you with things that I have never seen on this planet, please tell me, I would want to be the NGO investor for that company. So, we need to get into the nerdic part of how AI, or how generative AI, like the ChatGPT or Copilot, were, is trained. So, number… step number one.

is provide AI with the training data. And, number two. is to provide AI with feedback, you know, yes and no, is the picture, like, take Google… how Google trained AI to read the image, as example. Google spent 7 years to send all sorts of image to its AI algorithm and say… and label it. Is this a cat? Is this not a cat? So the number two step. with AI training is to give AI, feedback, and number, label the data, like, yes or no, yes or no. And number three is to give AI feedback. And, so… so now it's, and because AI is not…

learning from everything around the world every day. All the chat GPT and Copilot, they have their dedicated data sources, so AI cannot come up with an answer that he does not know before. In some sense, it can help you accelerate getting to the answer, but you have to ask the question. So in, so in short, AI cannot create knowledge it hasn't learned in the past. Even though… yeah, yeah, Daniel, please.

Oh, Michelle, one thing that I wanted to mention is that AI will never tell you how mature it is for the functions that you want AI to do for you. This is something that you have to figure out yourselves.

Right, and I think that's part of the problem that I think that I see in the industry, which is…

that nobody knows exactly how mature AI is, or what they want it to use. And the only way you can do that is by doing your own validation of the capabilities of the AI function that you're using, your own verification, and then for you to continue to feed whatever input it needs to go into the AI, so the AI can become more mature as you go. But just to your point, to expect AI to do something for you that you didn't think about, it's the wrong way of thinking. It is you who will eventually make AI to think and to behave and perform the way you want AI to think, behave, and perform. And I think that's part of the problem that we have. It's a cultural problem, I think.

Yeah, yeah, Daniel, I couldn't agree with you more. I wish all the VPs and SVPs have your maturity and sophistication when it comes to AI. They expect this magic button. They expect, like, you know, the slogan of Apple, who is in my backyard, is surprise and delight. But with AI, you actually don't want AI to surprise you too much. You want AI to be your companion. Like, for me, I don't have time to read all those, you know, 5-10K submissions, so I want AI to very quickly read it for me and fetch me the answer, rather than, like, AI surprised me. And I also want to highlight the Sean and, someone in… earlier in the conversation mentioned about the concern of hallucination.

The reason that caused hallucination is that the current gen AI model is designed in the way that AI is rewarded to please you, so whatever question you have, AI is like, okay, I must come up with an answer. Even though I don't know the answer, I will make up an answer. So that is when, there's law… there's a law firm that their associate uses AI to write up some case studies, and then AI made up a case that does not exist at all in the history to help the associate when the wins the trial. But overall, I think AI, like, I want to manage everyone's expectation as of today, or the next 6 months.

I do not see AI could come up knowledge that it hasn't learned from human beings. Yeah. I see Gaudi came out. Do you have any questions or comments on this, like, mistake number 4?

Yeah, it's… thank you, thank you, Michelle. Very interesting topic, and, you know, I just did not want it to ramble, but I really loved how Daniel phrased it, and I really love how Arlen asked the question. For me, I am a heavy user of AI, and for me, understanding the back end, is it like, you know, a database? Like, you're talking about agent.

I have.

AI agents are what? They are… whatever data sets you feed, there is a design behind it, the question you ask, it will give it to you. So it's almost like a replacement of your superficial memory. Right? So, I don't have to remember everything, here is the answer, right? So that's a simple one. AI agent is not a problem. I think… There are senior people, especially. They are looking for efficiencies. And those efficiencies usually come from a very complex dataset and the design behind that data set. So if you are creating something in a regulatory, you have to really understand Which, I think Daniel alluded to, is like those interpretation and the decision-making process, and, you know, so you will have a validation protocol designed very quickly.

But then you read it, and you say, oh, this is not how I intended to use my spec, this is wrong, right? So that human mind, how we connect the dots, AI can't do it right now.

Come on.

And senior people don't see it. Senior people will say, reduce regulatory people from 6 to 2. Use AI. Right? And now those two people still have to do that level of work to kind of make those dots connect. So it's a hype in my mind. It's a real hype. People don't fully understand. They are looking how it can be very efficient. They are looking for how they can reduce the cost of a department. But I feel it's really not there yet.

I agree, I agree. I also saw Fred, like, come out of the chat, yeah, any comments, Fred?

No, I think that, you know, just to reiterate, I think that AI, it's still kind of, like, shaping up to be that great, you know, tool that we're all looking for, but in the meantime. is just assisting, you know, on what we do. I think that the potential is there. Some people talk about the AI bubble right now. They're just dropping a lot of money into developing these tools right now. And, you know, companies have not seen yet, you know, the revenue coming back. So, they're hoping that maybe in the next few years, they get there, but in the meantime, I think that it's just a, you know, an addition to our capabilities to be a little bit more efficient, but not to replace, you know, exactly everything we do.

Yeah, yeah, yeah, yeah, I fully agree with you. So, like, moving on to number 5, Okay. So, like, you know, number, four, number four, it's like, I, I think a lot of you has mentioned, you know, train your team, and also the leadership team, communicate with your leadership to write better prompts, and, provide AI with specific data sources. If you want AI to come up with, you know, like, help you find clinical trial insights or regulatory insights, then you have to feed it specific data. You cannot expect AI to come out of something… come up with something that it hasn't learned in the past. Okay,

Number 5, okay, we are halfway through, so we're doing great. I call it the Frankenstein's Monster Syndrome, like, we have Frankensteins, like. what I've seen from the technical perspective, that all our customers want AI to solve everything. So it's like, oh, can we have, like, for, you know, for the Medtronic and Sanofi of the world, Sanofi's like, can we build one AI platform that feeds all our function, cross-function needs, you know, from clinical to regulatory to medical to post-pharma, post-market surveillance, and also needs it required from oncology to diabetes to hypertension to cardiovascular

Vascular to everything. And, the reality check is that, you know, AI is not a magic wand. I think Daniel and, Gaudi and, you know, Fred mentioned, it's still not there. It's… each model is designed very, very differently, even though, like, people think Copilot, OpenAI's ChatGPT, and Gemini are, you know, all the foundation models are kind of same, but each tool, AI tool, is designed with… should be designed with specific data, specific prompt, and a specific goal in mind. You cannot… it's almost like, with humans, you cannot expect one person to be the marketer, to be the regulatory expert, and quality, and expert, and medical experts. It's the same for AI. Like, I would strongly

Caution, even though… human wishes are also natural, like, of course, it's not wrong for companies, they want one solution to have, you know, solve all their problems. But currently, there's no… there's no magic wand that, you know, you can pick one solution that fix all your problems. Okay, excellent, yeah. Yeah, and also Phil mentioned, like, you know, the infrastructure challenges with GenAI, where running out of space and electricity to power all those data centers, and maybe, like, GPU would be running out. So, there's also a notion of small small AI model instead of large language models, there are small language models that could potentially

feed to the, problem even more. Yeah. okay. Next to one… So, for mistake number 6 is to think, you know, I think Daniel mentioned this a little bit, like, I need a… as a practitioner in life science or in medical device, I need a technical background to use AI. Effectively. So I see, like, you know, it's almost like you have a cold ghost that is haunting you, but in reality, it's like, modern AI tools are designed for domain experts rather than in the, you know, the coders. You actually do not need

a deep technical background to learn how to use AI. I saw there's a lot of training on Coursera and, you know, Stanford, MIT, Harvard, they all push out the training to, you know, understand AI, how AI is trained, how to. build an AI model, but that may not fit for our purpose. We are practitioners, we are domain experts, so the key is to just about the technical skill. AI literacy is a far more important skill than knowing how to code. So the most successful regulatory leaders we find who, you know, really fly in AI is someone who knows how to frame questions to interpret the outcome and validate the results, and with… within the complexity of their work.

Their strength, is in understanding the real need of the industry, and also the regulatory nuances and the language, rather than writing code, or, like, you know, understand the million differences and nuance of the large language models. Okay, awesome. Okay. Mistake number 7. So, this is another thing we have seen on the challenge with AI adoption, especially on regulatory affairs, where we see, like. You know, everyone, the leadership team's like, okay, we invested in ChatGPT, we invested in Copilot, we invested in Nyquist, we need everyone on our team to adopt AI at the same time.

is, I call it, like, all hands on the Haunted Ship. And in reality, it's… it's really, really hard, like, even with or without AI, when is an organization ever adopt one thing or do one thing simultaneously? from a team. The answer is almost never. So, many regulators or the leaders in med tech assume that the fastest way to transformation is for team adoption. But, you know. the… even though their wish and their goal is… is very ambitious and is very admirable, but the execution can overwhelm staff and dilute the focus. That's why when I, you know, help a…

help the medical device leaders to think about AI, you know, think about the AI journey. compare, you know, on contrary to all those other tech teams, like, you know, go big, move fast, and disrupt things, I would suggest people to start small, very small, and very gradual, have a very realistic expectation. We have seen successful AI adoption grow from smaller teams and empower groups. It's almost like you have take military as an example. You have a squad team, or, like, one or two champions who are really passionate about the technology, and then you… they learn how to use AI, you know, with… in previous discussion, we talked about AI is… not all AI is created equal, and also not all team is created equal.

It's better to create a center of excellence where your team really learn how to bring the external tool into your existing workflow, and see some ROIs, and then have controlled pilot. Educate your staff about safety of data, educate them about Privacy and security, what, what documents should be put into the… ChatGPT, OpenAI, or even our tool to write a prompt, when is not to put, you know, your internal documents, or to make sure that your your generative AI to have your proprietary instance, so everything's on your end. People are not using your data to train their model, people will not have access to your own data.

Oh, Michelle.

So.

Yeah.

Michelle, I wanted to supplement what you just said, because I think that you're correct in… looking at the perspective and the perception that the industry has, which is about, in this case, who should be involved in working with AI. But I think that what's really most important is that who also has to be supplemented with, what am I going to be using AI for? when do I need to use it? Why do I need to use it under… and under what specific circumstances? So… so because, again, AI is a tool, and you don't use a tool all the time. Because that's not the case. So you really have to, in addition to who should be using AI, you really have to look at this discussion from a more holistic perspective. Otherwise, we're gonna be losing the ability, really understanding that the tool has to be used wisely.

Yeah, well said. I cannot emphasize enough that when it comes to any technology adoption, it's not about The technology is about the people. And, for… especially, we have seen companies' adoption in different phases and at different scale. With larger companies, because the SOP compliance requirement, the employees expect the… leaders to be more prescriptive on what they can do with AI, what they cannot do, at what process, and how to validate. So, instead of just giving… providing your team the solutions, it's more important to educate them, to help them to build

to Daniel's point, AI literacy, and how they can You know, demonstrate value through technologies. I think the, one of my, clients, ECI, they are a consulting firm, they use our solutions for regulatory consulting, and the CEO has one of my favorite quotes, there's no point to buy a Ferrari and let the Ferrari, you know, park in the garage. And also, it does not make points for everyone on my team to drive the same Ferrari at the same time. So, it really requires organizational change to be more intentional on how you introduce technologies to your team.

I have a question, Michelle.

Yes, please.

I don't fully understand this, so there was a news I read that, somehow, the private chat GPT content got indexed by Google. And companies didn't know about it. So there is no real protection or copyright on things that you add to ChatGPT, even though it is, you have subscription to it.

So as soon as engineers start putting.

attachments to any LLM model, do you know how they get, like, how they got indexed? Like, what safety companies have to say, this is my patent design, or this is mine, and not anybody can use it?

Okay, how much time do I have, Sean? Like, this, like, I will try to, give, very brief answers. If you use Copilot and ChatGPT on your personal subscription, I would strongly you go to the settings. and, you know, click do not use my data to train, and do not use my, I will not authorize OpenAir or Copilot or anyone to use my data to train the model, and, email the service team, say, like, is there any way you can delete my data periodically? And, if you use it on a personal level, and there's some prompt, and there's some, like. answer you really like, just copy it in your own computer. But you can email to them and ask for them to delete your data periodically.

And.

And it's in the settings.

The setting only includes do not use your data to train. You need to email them, and whether they will grant it or not, it depends. I've seen I've seen people have different levels of success as a consumer, as an individual user. And from a company level, I want to intra… before I discuss, like, you know, how Google get hands off your, of users' data and indexed it, I want to introduce a concept, technical concept, called unlearn. Basically, it's, like, very easy to feed the data into AI and let AI to learn, but it's very difficult for AI to unlearn what they have learned. It's almost like

Everyone on the call, if you know how to ride a bike. If you already know how to drive a car, it's almost impossible for you not to learn, you know, to forget how to ride a bike or drive a car. So, it's the same for AI. Once the data… your data got leaked, Into the training set. At least, to my humble knowledge, currently there is no good way for the algorithm to unlearn that data. And so I want to… I want to answer your second question on how to… how… how enterprise protects its data by introducing this… this… this technology term, so unlearn, so that we have the…

knowledge background to move forward. Then, if the… then how to avoid the data leakage? There's a couple of ways. The easiest way is, like, legal binding. In the contract, it's like, oh, you, you cannot use my data for training, you cannot use my data for da-da-da-da-da. It's… it's kind of easy, but very… actually very hard for enterprise or consumer to prove that the Google and the OpenAI of the world haven't used your data to train their algorithm. And, so this is number one. there's legal protection, and number two, there is part-a- protection. You can specifically, if your company adopts AI tools, you can specifically write, like, I want everything to be stored on my own cloud, and for my own

in, like, for… on my own instance. So if anything leaks, it's within your… cloud. It's within your organization. It probably, you know, for example, I shouldn't have access to Sean's file, but since we're in the same company, the damage is still contained within the same company. So you can you can have some, some, like, product protection. And number three, is from organization perspective. Have proper training and periodical training with your team to to, to… to know how to use different tools, what data to put, what data not to put, and how to periodically delete the data. So, if any… anything happens, at least, like, they won't have the record of you.

Thank you.

Yeah, but I…

I think it's very loosely understood, people I talk to. Especially within the engineering team, engineers probably have better understanding.

But because they're looking for answers, they can attach things and…

Other people probably would have no clue.

Yeah, yeah, yeah. So this is, the fear of, under-trained staff is also one of the barriers for AI adoption. So that goes back to, you know, what I discussed previously. Find a low-risk and high-impact workflow, like. getting regulatory intelligence. To formulate your regulatory strategy, you need all the regulatory intelligence. That's all publicly available information. It's just, like, no one wants to spend a week or a month curating the mold database or, like, the clinical trial, that's where AI can help you, to achieve ROI. But for your internal say, 510K submissions, or your clinical trial drafting protocols, you do want, you do want some protection so that your data will not be leaked out.

Thank you.

I think we need to get through the last 3.

Yeah, yeah, yeah.

One minute. Yeah, okay.

6 minutes. Okay, so, thank you, Sean. So, number 8 is, you know. this zombie work. AI is all about technology. We actually touch on this a lot in practice. When you want to have AI adoption, it's 20% technology, 80% of the people. Even though we talk about technology and all the development, the reality is that it's all about the people. And number 9 is, like, you know, we already have an AI tool, so we are all set. It's, like, the tombstone of all the tools that you bought and left in the basement. the reality is that, you know, the technology is evolving constantly. Do not treat this as a disruption. It was like, oh my god, this is a new test, my boss asked me to try new tools, but take this, hey, this is a great opportunity. Constantly learning how to use new tools could be a must to have

skill for your career. So how can you get your… the most out of your career by adopting new technologies? And, Number 10, the last one is, you know, the sleeping monster in the basement. We already have an AI tool, we don't want to learn, we're all set, but the reality is, like, you know. the technology is evolving so rapidly. I feel like I'm losing so much sleep just by keeping up with all the technology development, reading all the papers, the publication from the Google of the world. So, it's… it's like an evolving partnership. You… you are… you… it's some way, like, you are married to the same person, but that person constantly changes, constantly learn. So, it's the same with AI.

So, with that, thank you so much for your time. If you have any spooky questions and, you know, you want to connect, or you have any other questions, like, happy to answer them and invite everyone, come out of the mute.

So, I have a question. How can, the consultants on this call, work with you in partnership to introduce Nyquist AI, in your services to their clients? Can they book an appointment with you, or is there a demo? I mean, how can they… What is the best way for them to get ahold of you?

Yeah, this is a great question. So, like, here is my… let's connect. I believe in the power of networking, even if, currently you may not find immediate use, like, you know, we develop very quickly, and, we probably target at the same market, like, you can connect me via email is mw at nightquestai.com, or check out our website, book a demo. from our website, or, like, connect with me on LinkedIn. And I also saw another question, who are our competitors? Like, there are some AI companies and reps, and we have 3 tiers of products. The first is the low risk, which is regulatory, clinical and regulatory intelligence.

And, we… we have, like, there are clarified global data in that sector. What sets us apart is that we are probably the most affordable solution and with the most comprehensive global data, for example. clinical trial regulatory from China, from Japan, and the largest emerging market, and everyone loves Japan. I love food from Japan, so that's the differentiator. And we also have AI workflow automation tools for medical writing and AI agents. So, when it comes to enterprise build, we do you know, compete against OpenAI and Copilot, but because we have the domain expertise, we are very focused on just regulatory affairs and, just quality, just medical, so that, differentiates us from the other competitors.

That's great. So, one of the things that I'm looking forward to in the new year is to do, country-specific webinars, like, around MedTech Italy, right? Everything you need to know about Italy, and I had talked with Michelle about becoming a data partner. In that program to, provide some baseline information about what's happening in each of the countries. If any of you, are interested in participating in any of these, they're going to be multi-speaker and, if you have specific countries in mind, I'm planning on starting with Italy and Ireland, which represents my own heritage, and In, in Europe, but then, you know, everything else is fair game. Certainly want to, map it out, and, I would, I would love to do,

an Asia-specific event, because they do have such great data on, China and Japan, I think that would be really interesting. Considering how hard that information is to get. If anybody has any comments, please let me know, either here or by email. Are you participating in any upcoming shows? You going to be traveling anywhere, or…

Yeah, yeah, I'm going to, Texas in, like, by, November 20th. Like, I will, I will give a talk to a local reps chapter from Texas, and, talk about, again, like, AI for regulatory affairs. And more on the leadership, you know, mindset change, because when we have the fundamental thinking framework. when it comes to specific use cases, whether it's AI for regulatory intelligence, find your predicate, or file for 510K, that's easy, but getting the you know, get the courage to face, you know, the evil or spooky side of AI is the most challenging part. It's a… the first step is most… is always the most difficult, but after the first step, everything is a breeze.

Cool. Well, I want to thank you for your time. I want to thank everybody who joined us today. We had, quite a large turnout. I saw 70-something people on the call at the high point, and you know, fun. I'm glad that you, took the thing, the, the Halloween… theme so seriously, and put the effort into this. I thought that was really clever. Anybody can come off mute at this point that wants to chime in, talk about how they're doing today, or, you know, what they ate for breakfast, or, And, or has any feedback for me or for Michelle, happy to, you know, just keep the call going for a few minutes to let you check in, as they say. Carrie, how are you doing today?

I'm good, how are you? Hi, Michelle.

Hi! Hi again.

It's interesting, you guys… so it's Nellie, sorry, I've been on my phone taking care of the dog, Sarah.

you know, AI, what I find interesting with me for AI is I use it as to validate what I already know and frame it in better language sometimes. But, you know, I was hearing of a case where somebody did a paper, didn't know the acronyms and the terms, and ended up referring to something that was completely irrelevant to the actual subject, right? And I think that that's the key factor, is AI is a validator and a verifier of what you already know, and maybe a better framework of how to say it sometimes.

Absolutely, couldn't agree more. I, I use AI very, very much. Instead of reducing my staff, it's caused me to hire more people. Because it allows us to get more done, and that means getting more done means helping more people. And the more people we help, the more people we need. And, because AI isn't going to reach out and help the people. You know? And so, really, in product and service. Provider business, it's really gonna be, A sort of a… a productivity multiplier, take care of a lot of the things that I don't like to do. For example, I received an NDA yesterday, and I put it into ChatGPT, and I said, is there any reason I should not sign this?

You know? And they said, here's a couple things you might want to note, but otherwise it looks like a standard NDA. you know, feel free to go ahead and sign it. I… when I tell you I had no desire to read that entire NDA, I mean it. I'm really, you know… I use it to read 55-page guidance documents and say, is there anything in this worth reading? You know, is there anything in this that's new?

Or summarize it quickly for me, and it'll… yeah, some, you know, some documents.

But the other thing that I used it for yesterday, and this is a brand new workflow, because I had 6 hours of calls, I recorded all the calls, I transcribed the calls, I uploaded them, and I said, for every single one of them, I need you to create a Trello card for me to give to my team. Because I'm at the point now where I'm getting further and further behind every single day of the week on the number of tasks that are lined up. And if I don't get more organized. then this whole thing, you know, starts to come apart at the seams, you know? Which… You know, some of you might think that, you know, it already is, but that's not… Yes, Daniel.

No, I was gonna say, Sean, you just said something that's very important. We were told a long time ago that computers would simplify our lives, and that we would be bored. And now with AI, we were told that AI would also simplify our lives, and that we would, you know.

Be exempt.

Exempt. And you were saying that you spent this whole week, you know, finding yourself using AI because you generated a lot of work. And I think that's something that's very, very important for everybody to understand. AI will not reduce work. will create more and more work for everybody. That number one. Number two, I don't want to spook anybody from regulatory affairs. But remember, the data that you're going to be using for your submissions is coming from QA and manufacturing people who will be using AI, so… So, the quality of the data that you're going to be getting internally, it's something that it's going to be dependent on how good that your people use

AI. From a decision-making perspective. So it puts a little bit more of a challenge on regulatory affairs, because you get a lot of output, and are you going to use AI now to verify what you know, the quality of manufactured people gave you, which they also used AI, so it's a little bit of complexity that comes with this.

Look at radiology. They said every… it would get rid of all the radiologists. Well, look at the quality of radiology, look at.

and…

I mean… breast rate, you know, you're… look at your mammograms now. You're getting more and more high-quality output of any type of…

Correct.

scan you're getting of your body, you know? It's just…

Absolutely.

That's great. Any closing thoughts from anyone? Anybody want to chime in?

There goes my doc, sorry.

Well, thank you, Sean. It was really a very, very good, discussion. I mean, we are still in limbo, and many questions still needs to be answered, but these discussions, really refreshing.

I'm glad you enjoyed it. Thank you, Michelle. I do wanna… I do wanna say something. You know, for those of us who don't regularly attend all of these big meetings, industry meetings, these sessions… They actually provide incredible value.

Huh?

That it comes for free. we don't have to travel. In the intimacy of this circle of professionals. We actually get much more than attending, you know, going to get on a plane, going to a hotel, you know, driving there, etc, etc. This is exactly what the industry needs. I hope more people can take advantage of these opportunities.

I echo that, yep.

Yeah, yeah. And I could be… Sorry, Sean, like, and I could be, like, you know, the bridge between Silicon Valley and, you know, the industry. I'm nowhere are experts when it comes to regulatory affairs, but I can… I think my superpower is translate what Google says, like, unlearn, you know, the new terms and new technology, and very quickly, almost, like. seamlessly feed it to the real experts, the domain experts. So, hey, this is what's happening in the AI world. This is, like, what it means for your day-to-day work. And also, I travel quite a bit. I talk about AI adoption almost every other week, either from the tech side or from the, the industry side, so happy to continuously share

the knowledge and also the megatrends I see from different, places around the world. Yeah, please, Sean, yeah.

The only downside is when you sit next to Michelle in a booth, you get chocolate.

Right. Yeah. Yeah, no, so I'm gonna end the call soon, but, you know, we are looking at a new platform, it's called Demio. And the downside is, is that everybody doesn't get to be on camera at the same time, but if you want to be on camera, you just raise your hand, and then you come on, camera. Only 10 people can be on screen at a time. But everything else about the platform is much more user-friendly than Zoom, so maybe we'll give it a try next week or the week after that, which means that you'll be receiving an invite. to a different platform. There's no… nothing, no downloads, nothing you need to do differently, just access the…

I don't know, I'm… it makes me anxious, these technology changes, because, you know…

One way, and then… You know, okay, that's enough of the behind the scenes. I will, let you go. Enjoy your day. Thank you, everybody, for your time.

here.

Happy Halloween!

Hello, happy Halloween.

Halloween.

Thank you, guys.

Thank you, thank you, bye, bye-bye.

Thank you.

Thank you. Thank you. Thank you, Shaw.

Bye-bye.

Bye-bye. Bye.