Generative AI & You Work smarter, faster, and more securely

Hello, and welcome everybody to generative AI, and you work smarter, faster, and more securely presented by Medtech leading voice. I'm here today with guests. speakers, Mark Omo and Martin King. Many of you know them. and some housekeeping announcements before we get started. I want this session to be as interactive as possible. So please use the chat and question and answer functions vigorously throughout this webinar, and we will try to get to all of your questions and answer them while we're here, but if we don't we'll follow up with you afterwards. There will be a recording of this event.

And it will be sent tomorrow. To all everyone who's registered along with the slide deck, a link to the slide deck. So here's the agenda. Here's why we're here today. We're gonna ask you about you. We're gonna launch a poll in a minute. Talk about what qualifies us to talk about this explain what an Llm. Is, and why you should care, and the other topics that are that are listed here. I'm not going to read them to you, because that's no fun. But moving forward. Is there anything that I forgot? Oh, there's going to be a survey at the end of the event, so please

complete the survey. And right now we are going to launch the poll question hopefully. More polls and quizzes. I think I'd learn how to do this by now and launch the poll question, are you seeing the poll? Question.

It's showing.

Yeah, I see it.

Okay. So while we get started with this you know, our goal here is to evaluate. You know. How many of you are using generative AI or Chat Gpt, or any of those tools at this point. And what your experience has been with them while we're doing that, I'm gonna ask Mark to introduce himself and tell us a little bit about how he got started with chat. Gpt. Mark.

Yep. Yeah. So my name is Mark. I'm the engineering director and general manager, Marcus engineering. We do. lots of medical device development design, cyber security, that kind of stuff. And I guess about a year and a half ago, you know, I had a lot of colleagues using chat gpt, and I was pretty dissatisfied with it. I played with it a few times, and I was like, Oh, this is a a machine that generates just rubbish output, and it, it turned out that I was using the the bad versions of it, the free versions of it. And I didn't really understand in what ways it was useful versus the ways that it maybe wasn't useful. And so, after spending some time.

I using the using the paid versions, after some cajoling from some people that I trust and really understanding about what it. what it's good at, and and really what it's not good at you know. Really. Help me understand how it helps. And now it's something that we regularly use to help us just do our jobs, you know, faster and better and easier than we were doing them before.

That's great, Martin. What's your experience?

So my my story is much easier. I just saw this stuff on the news, and I thought, This is cool. I have to play with it and try this out? When I really, I suppose, what? How I really began to use it was when I wrote stuff, I would ask. can you write this better than me, or use it as a sort of prompt, you know, for ideas. And I realized, Oh, this can really speed things up, you know it's it's like a companion working together, and in that mode it works very good. If you just sort of ask it a question. and and you haven't a clue what you're expect going to expect back.

or you have no expectations. Then you're probably gonna get garbage back, and you're probably then going to say, this is garbage. It doesn't. It's not useful for me at all. So now I use it well. Why am I prolific post days? Because I can get Gpt to give me key points in a document really quick or in a page that's been published, and it tells me where to focus on guidance and regulations. You know, we have the new liability regulation come out in Europe this week. Well, it's going to read it, you know. A lawyer, maybe, but you need to know what's in there and what it's about and what the key points are, and it's and it's really good for that sort of thing you can just say, tell me all the key points are of this document, and it will tell you.

and if you is that all you've got. Then it will say, No, I've got a bit more. so you sort of have to encourage it a bit along the way.

That's fantastic. I'm going to end the poll. Share the results.

Are the results visible.

Okay, so while I'm sharing that I'm gonna tell my story. My story is a little bit different than most people's. In February of 2023, I had brain surgery, and prior to the brain surgery I had 0 interest in Chat Gpt. In fact, when people would mention it to me, I would get irritated and respond poorly. Most of the time. however, after the brain surgery, I wasn't able to get my fingers to do what I wanted them to do anymore. And I started to have to rely on workarounds. such as voice recordings and transcriptions, and I found that with the transcriptions I was able to incorporate, Chat gpt into summarizing them very easily. And from that point it's just been sort of a steady progression of all of the applications

with Chat Gpt to the point where I don't use Google anymore. I use Chat Gpt search. which is faster and better. They released it about 2 weeks ago. That was the impetus, for the idea of this call was to talk about Chat gpt search which was released, and to then see how quality and regulatory professionals and other med tech professionals are using tools like chat, Gpt, or copilot, or being in their day to day work. So thank you so much for everyone who responded to the the poll questions and from there.

We are going to move on. There are instructions here in getting started if this never done this before. So yeah, fewer keystrokes. This is the thing. Simply talk into chat, gpt like you're talking to a friend. It will transcribe the information. but not only that, it will organize the information and clean it up and clean up your English. you know, or whatever language you're speaking in. And yeah. So that that's where we are. I'm gonna turn it over to Mark to talk about what is a large language, model error.

So I'll start the the high view. So Llm. Large language model, that's what we call the that's the general name for these kinds of tools chat. Gbt, Gemini, things like that. These tools that take in text or other information video photos, videos, that kind of thing and transforms it and outputs new things, and the way that they work is they will try to predict the most likely next word in a stream of thought. And so they're they're large statistical models. So they don't really have lots of real intuition or thought processes built into them. They

they're they're really kind of transforming one type of information to another. But they they can do this in a very sophisticated way that will emulate that kind of thought process and thinking. And so you know, one of the one of the ways to think about it is they're really context driven because they transform information from one form to another. And then they're trained on all the information on the entire Internet. So they have this kind of fuzzy knowledge of lots and lots of things, but especially in regulated areas like Raqa. You know, medical devices, specific regulations. Their knowledge is going to be really broad and high level.

and especially about what you want to know. They don't know. They can't read your mind, they don't know what you want. And so the way that I like to think about Llms, you know, these text output models is, imagine there's a someone who's pretty smart sitting in a blank white room or in a little box, and they pass a piece of paper on the floor with an instruction, and they try their darndest to give you back what you want, and if and if you don't really tell them what you want. They can't do what you want. So if you say to them, Hey, write a risk management plan. They're going to try. They're going to write some stuff, and you're going to get it back and be like, well, I'm not designing surgical devices. I'm designing wheelchairs. This is complete garbage. Well, they they don't know that unless you tell them that information.

and the more information you give them to work with the better they are. So let's say you're in the Us. And you want to focus just on the Us. Regulations, just on the say, the cybersecurity risks. If you give them all the context, all of the cybersecurity information. all about your product, all about your current processes. And here's an example risk management plan for a different project when you ask it to give you something for you, based on all the context, all the things that you already know. It's going to perform so much better at giving you back that information. and there are a bunch of different ways to use. The tools is

you can upload documents. You can give it information. You can give it large blocks of text, and it'll interact with all of those things differently. And one of the nice things about that is, you can also kind of nudge it in the direction that you want. So you know, one of the silly things you might say is, you know, like, oh. transform this regulation into pirate talk, or something like that, and it's really good at changing the tone and the context. And so, if you said, Oh, make this simpler for me or explain this more. Or if you're like, yeah, this is the picture book version. I need the like real professional version. You can say, Hey.

assume you have a knowledgeable technical audience. Those kinds of things. Those are really good ways to help nudge the output and the tone to what you need, because it doesn't know what you need unless you tell it. That information.

Excellent So now we want to talk about some use cases for raqa. I'm gonna start with the general idea of proposals and meeting notes, you know, sort of day to day tasks. As I said, you know, I've stopped taking notes because I still can't type very well, and or write very well. And I record conversations, I transcribe them using auto AI, or fireflies, or any other transcription tool. I upload them into Chat Gpt and this has allowed me to attend

meetings, record the meetings, upload them, write articles about them in no time flat. If you want to convince your boss that you should be sent to the next conference that you're going to go to tell them you'll record the sessions, transcribe them and summarize them using chat Gpt, you know. I think that would be an awesome way to get permission to go to a conference. Then data analysis which one of you is going to talk about data analysis. Martin.

Yeah, yeah, I'll talk about data analysis here. So one of the one of the ways that I like to use one of the the features that I really like of Chat Gpt is, it can do it can write and run code for you. So you know one of the things that I like to do is do analysis of the 5, 10 k database to find things like predicate devices. And if you're in a if you're in a large field and you're trying to find the right predicate device. you know there might be, you know, thousands and thousands of of submissions. Or still, if there are hundreds or dozens of submissions there is

a ton of information to review, and so using it to cut down on that information is is a really useful feature. So I'll show you guys a demo here of analyzing the some of the the 5 10 k database information with chat. Gpt Let me share my screen here. All right, Sean. It's can everybody see my screen here?

I'm seeing mine.

Okay. Martin.

Yeah.

Screen.

Awesome.

So this is the, this is the 5, 10 K database you can download and export from the FDA's website. And so whenever I analyze data. And I always wanted to ask Chat Gpt to look at it because all data is different. It's all. It's all going to be formatted differently. It's going to be really annoying, which is the kind of thing that takes time. And you know, we'll stumble. People who are experienced in writing code or ingesting or analyzing data will stumble over. And that's also something that that chat Gpt is going to stumble over. It's not, it's not perfect. It's it always needs some guidance. So here we can print the 1st 2 lines. The FDA has this weird piped, eliminated format for their data.

So now, if we will ask it to load it into Python here. So it's Gonna.

Screen again. I think the audience wasn't seeing it. I think.

Oh dear!

Sorry.

It's still the slide deck I'm sharing. Let me stop sharing and share again.

Okay. So Sandra says she can see it.

I shared it again here. Okay, we could see it all right. Great. So it wrote some code here for you to load it into Chat gpt, or to load it into python and then, which might be really fast if you know how to write python. But lots of people are not that experienced. So that we'll do is we'll ask it to list all the product suppliers for this product code. So this happens to be for a powered wheelchair, the powered wheelchair product code. So maybe I want to do some competitive intelligence. See who is doing? Who's in this market segment? Who are my competitors. Who should I be reviewing for predicate devices? And so I'll ask them to do a little bit more information I want to sort by the number of 5 10 ks, so maybe a startup made a single device. That's not as interesting as who is the most prolific device manufacturer.

So it's gonna it wrote some more code, so I can see it'll give me a table of the most prolific manufacturers of the devices, and the 5 10 k database has lots of great information about companies, addresses all that stuff which I could get out. But what I'm interested in is, let's say I want to see how hot is this market segment? Are? Are lots of people doing powered wheelchairs? Is it? Is it less important these days? So I'll ask it to make a chart for me here, based on this data. and so if you were to go download this database and try to make it an excel or load in a python. You know. This would have taken you a good part of a good part of an hour, probably to get it all set up and running.

and in just a few minutes here we can ask Chatgpt to make this chart for us. And then try to figure out how you know, what does this look like? And so we can see here that in the last part of 24 and the last few weeks there's been an explosion in the number of 5 10 ks. For these powered wheelchairs, you know, over the last few years it's been pretty flat. but there's just, you know. We jumped from 50 to 70 devices just in the last half of 2024, you know, there's a very, very popular market segment. So this is just an example of how we can use chat gpts. data analysis. It can write code for us. It can transform data. And it can do that without you needing to be an expert, you might know what information you want to get out of it, but you may not have the skill to do that analysis yourself, and that's something that you can delegate to chatgpt. It has really great tactical skill at manipulating data and making charts and doing this stuff, and it can enable you

to get the information that you need out of it.

I assume the same would be true for the adverse events database.

That's exactly right. That's another great example of the type of data that you can ingest you can ingest all kinds of tabular data. You can even do things like copy paste a bunch of data off a website and ask it to make charts based on that or request that information, those kinds of things.

Someone just asked his copilot the same as Chat gpt.

Yeah. So that's a good question. So the I so Openai makes chat, gpt, and they're partnered with Microsoft. So all of Microsoft Products copilot being searched, those kinds of things on the back end. They're using the same kind of fundamental large language models as Chat Gpt. It doesn't work exactly the same. The interface isn't exactly the same, but it it's it's it's very similar on the back end. And so other teams like Gemini have a competing model. I've used it before. And I actually used to work at Google. and it is not as good as Chatgpt. It's very good.

But I found it to perform slightly worse. the other. There's some other alternatives like Clod and anthropic. who have other models that are also pretty good. But chat gpt today continues to be the stand out.

Okay.

and let's see. So there's some questions about the data analysis. So usually Chatgpt can't download files from it, can't download files from the Internet. It can search the Internet. It can look at things, but you generally need to upload the files to it. And this is a feature that's only in the paid version of Chatgpt. As far as I'm aware. it certainly performs best in the paid versions of Chat Gpt. And there's some good questions about I you know, reading documents, and it outputs being totally wrong. And I think we're going to talk about that next. So.

Okay. So I'm gonna go back to sharing hopefully, bring up the right? Oh, murder screen, desktop share, and then go back to this and go back to present sorry folks remedial you seeing it now, making the case for our Qa data analysis. So making a Gpt, how does Chat Gpt read a document?

Yeah. So one of the great capabilities that Chat Gpt has is you can upload files and ask about it, like Martin mentioned that at the beginning, uploading new standards, or if you're a consultant, you might come into a company that has a lot of great documentation already. A lot of processes that kind of stuff. And it's it's it's a ton of information to sort through every time you need to find one little process or procedure or set. And so Chatgpt doesn't really read documents in the way that you might think about it. It can only look at a given size of information and it for for chat. Gpt, it's about all right.

16,000 words, and so that's that's only a cup, you know. That's several pages of information. But that's not an entire 400 page standard. And so the way that it does it and on the inside, as you upload a file, it breaks it up into a lot of little sections. You might think of them as kind of like note cards with the chunks of text on it. And when it asks when you ask a question about a document like, say, you have a I standard, and you're trying to find information about the some particular subsection. It will. You can imagine there's somebody else with all the note carts. Our Llm. Is going to ask someone that someone hey? Give me the most relevant note cards about

electrical safety, for example. And so it's going to sort all the cards to figure out which ones are most related to electoral safety, and then hand off that stack of them back to your, to your large language, model back to your person in a box who's trying their best to help you. and they're going to take that information and try to answer your question. And so the the things, the kind of the important nuances in that are, it's not looking at like the page numbers. It's not necessarily looking even at the section titles, unless the section of text that got back happens to contain the section number and the section title.

and when you ask Chatgpt to look at page say, 6 or 612 and it asks for the relevant note cards. Page 612. It may not work the way that you expect. It's not like a person who's going to flip to that page, and then they're going to be like, oh, you meant Pdf, page 612, not page number on the bottom, right? 612 those kinds of things. It's going to try its best to find the relevant information and answer based on that. So you know things to keep in mind as you're asking about documents, or if you try to say, tell me everything about a document. It's not really looking at the whole document. You can kind of shortcut this by trying to paste

the whole document or the relevant sections into Chat Gpt and that kind of forces it to read those sections and look at them in its entirety. But like I said, it's not, it doesn't have a big enough. They call it the context window. A big enough chunk of data can look at it one time to read very large documents using the tool, more complicated tools. You can read larger documents, you know, like maybe 100 or 200 pages of text or so, and some specialized ones can read thousands of pages of documents. But those tools aren't generally available in like the easy web interfaces. Those are more developer, focused, more expensive tools.

So somebody wrote, but how are you sure that the data are correct? I had an inquiry based on Chapter 1, 5, 1, think this happened? 8, 9, Colon 2,022, and their title a simple inquiry. But 50% of the chapter sub number were wrong. And so that's exactly what you were talking about is that it's not looking at the information the same way that you or I would look at the information and the I think the workaround for that is to cut and paste the actual chapter into Chat Gpt rather than asking it to read a Pdf document.

Yes, no, maybe.

Yep, absolutely. And I think you know one of the one of those standards that I like to do this a lot with is 60, 61 you know, it's really long. It's got a ton of they. They have their own creative terminology set that they use inside of it. And so I will often use it to do things like, hey, how did they define this term? Instead of flipping all the way back to the glossary and trying to flip around to see where it's used. It's very good at those kinds of inquiries, and if you're looking at it like Martin said, like, oh, what are the most important sections of the standard? And you're like, Okay, well, tell me about in the cyber security

post-market guidance what's required in an Sbom. It's going to give you some information, and then I would encourage you to go check that information, ask it to say, what section should I go to, and you should scroll to that section and read it and understand it. It's not going to replace you doing a hundred percent of the work. It's not like someone who's going to be there and really, carefully research and output. It's going to do 50%, 80% of the work for you. And you need to fill in 20%.

Oh, go ahead. I think that's a really important point, because your your demonstration earlier on really sums up this ability to take unstructured data and put it into a structured form in a presentable form that people can follow and understand. And the key thing is is that if I do a search just on windows, or Mac, or whatever. If I do a keyword search, I'm using a keyword. If I'm doing a key sentence search, it will only find that sentence, but if I'm using Gpt, it will find a sentence with a similar meaning. Now I saw a question earlier up pop up about validation and things.

all of the applications we've talked about or or referenced. and exactly the point you were making there is. You can go and verify you. Go and read the document. But if the if it's if the information is in the middle of a 200 page document, then Chat Gpt is going to help you find it much quicker. You're not going to have to read all 200 pages in order to go and find it, and you might not find it just with keywords, and sensitive and and sentence searches, but if you find it with a meaning chat, Gpt is looking for the same meaning for the same implied statement. And this is where it comes in really useful when we're talking about

exactly the example which you have with 5, 10 k. And the predicate devices. And look, we need to find similar devices. When we're doing our clinical evaluation reports. We need to find similar devices when we're doing Sota, we need to show. Why are we state of the art? If there's a if there's a whole lot of devices coming out that indicate that we're not state of the art. Then perhaps we have a problem, or perhaps we need to come up with a new rationale. So I think the key point or the key take home is, don't use a tool blindly in the sense that oh, I validated it. Now, I don't have to go and check anything anymore.

I mean, even when you validate tools in in an environment, you revalidate them every now and again. And in this case you should be revalidating every output. So you should be checking every output that you're using. There's also a question about gap analysis. Yes, if you put 2 documents in. If you put Mdr. In, and you say, is my is my document. Compliant? Is my intended use statement compliant with Mdr. Then maybe it will help you. Maybe it won't. But if you say to Chat Gpt, where in Mdr. Should I be looking in order to check that? I'm compliant with this clause?

It might, it will probably tell you where you should be looking, and then when you, when you go there and look, you can then say, Look, I've got this clause in Mdr. And I've got my my statement. Are these, are these compatible? Am I missing anything? Is are there any gaps in my statement? And that can save me a lot of time. It's a little bit like the person who runs around a large organization asking the same question to many, many people waiting to get the answer and confirmation they require. You can actually use Gpt to do that job for you. It can act as a large organization when you don't have to ask a hundred people before you get the answer

that you're looking for, or the correct answer is probably more important, an answer that makes sense and confirms your intuition or confirms your expert opinion. And that's the point. A tool will only be as good as the user. And I think I'm a lousy carpenter, and I can't cut straight right, and I can have a great sore, and it doesn't help me. So that's a classical case. Yeah.

And I think, Martin, you know, I really like that. The you know, somebody asked about doing a Gap report, and it was very different. And if you ask it to do a a gap report, and you know, Chat Gpt will only give you back a small chunk of data. If I said, write me a gap report in 4,000 words for a very large system, you know, you're gonna you're gonna hit some things. But you're gonna leave a lot out. But if you say, Okay, where is your, where is your nonconforming materials procedure? And okay, does it meet this chunk of sections after I give it the whole thing to you. You know, those kinds of really targeted, focused inquiries is where it's going to provide the most help

asking it to do the job of someone that's going to take several hours or several days of review is is too big a chunk for it to do a good job on the smaller the chunk of work you ask it to do the better it's going to perform. And so someone actually asked about foreign language documents which I think is actually a really just, excellent use case for Chat Gpt. It is extremely good at translating documents and for finding things in foreign language documents. I use this very frequently for looking at, you know, information that's only available in foreign languages or standards or data sheets that kind of thing, and that's a very good use case. It's very good at word for word translating chunks. It's also good at doing the same kinds of things in English that it does in foreign languages. It performs best with the most common foreign languages. You know German Chinese

things like that. It's going to perform poorly for very, very niche foreign languages where it just doesn't have a lot of data and information to work on. But the most common foreign languages that I operate in it works extremely well, and having worked with you know, people who are native speakers in those languages. You know they they will often, you know, describe the translations as better than than they would have done by hand, you know, especially without taking a lot of time and being a professional translator. So I found it to be very effective for those kinds of for those kinds of barriers.

Before we move to reviewing large piles of documentation, I would just like to say, Hack number 2. Summarizing, you can summarize a large volume of information. Say, a 200 page document. Somebody asked about uploading that. Yes, it will do that, but it'll do a superficial job. What you then need to do is go into it and ask it exactly what you want section by section, and to expand on that and tell it what you want, and tell it what you don't want. Then tell it how you want it to write it. and then tell them what tone you wanted to take, and who the audience is. The more information, the more context that you provide to chatgpt, the happier you're going to be with the results. Now, reviewing large piles of documentation standards, regulations, quality systems, predicate devices, adverse event reporting. Martin go.

So if we if we're looking for a comparison situation, we're looking for something similar. We're looking to match things. because we, as soon as you're matching something. You're working in the contextual domain. And that is where Chat Gpt excels because it's a context machine. So it's looking for for comparison. So when I'm working on my, when I'm looking for adverse events which are may be applicable to the same device that I have. then it's an ideal tool for that. It can go through a lot of information, and it can confirm or not. It can at least give me a list of things to look at and say, you know, maybe you need to look at this. It can be useful. Now.

how do I deal with that in a Qms ice? Actually, when we do. When you do a clinical evaluation report or any of these post market surveillance reports, you're supposed to document the the method you apply. and there's nothing wrong with saying I use Gpt in order to search through the data and identify predicates. And then I examined those predicates presented to me. Document. The method you apply when you're using Gpt as part of your Qms. Or part of your regularly reg regulatory affairs

system of regulatory intelligence. Put a sentence in there and say that because if you have to formulate how you're using Gpt for these kinds of things. then that will automatically tell you how you should be using it if you're just trying to use it to do something. But you cannot formulate that in clear words. in a document that a reviewer is eventually going to look at. Then probably you don't really know what you're doing. So that's and you could ask. you could actually state what you're trying to do to Gpt and say, I need to formulate this text in a document. Can you help me? And it will help you, and it will help you give you give you a a sentence or a few sentences to describe that.

so that that kind of thing will work very well when when you're taking that that approach. So all these tables that you have in these Mdcg templates you can think when you look at them and like, how am I going to fill out this? Or how am I going to do my post market work? Just think, what tools can I apply to be quick and effective at sorting this information. And why should I be doing it? Because if I learn to apply Gpt tools to help me do this. I will. I will remain state of the art. How does a professional today in their field remain state of the art?

They have to remain one step ahead of every graduate coming out of university because every graduate coming out of university has played with this stuff. and they're not gonna understand why their bosses haven't got a clue when they and like come to work with them.

Martin, why don't you show us the quick example that you did, or you're gonna.

Yes, yes, okay, let me let me. So let me show you something that does that work. Am I sharing.

I think so.

Does everybody see that.

Yes, yep.

So you will know that I do the regulatory roundup every week. Well, there's an index. And I had a big problem with the index at the beginning of the year, I thought, this is very simple. I will do a page. I will create a page in Linkedin, and every week I put in a new. I put in the put in the file. and then I figured, Oh, but you can't search this information so originally. So it's like, okay, I I needed to write a program to do this present code. Well. I'll tell you that Gpt created this. I'm yes, I'm an engineer, and many, many years ago I wrote software 30 plus So

yes, I can read code when I see it. And I can read the code when when Gpt presents it to me. But basically, all the functionality here that you see is created in Gpt. And all I have in the background is basically an excel table with this information in it. and so I can search by organization. and I can then go to. Then I've got a link to my the actual article that that appeared in, and then I can go in there and find the actual information I'm looking for. So that's a very simple example of of how you can get Gpt to create a tool that can be useful for you. And then I was playing with the Bing button.

The Bing button takes the information and creates us a search in Bing. But if, because bing is linked to copilot, I can actually go to copilot and co-pilot will come up with the information with additional information. So if the stuff wasn't in regulatory roundup. you can actually simply click on the link and get additional information. So the organization of this data. It's not so simple. It's in all of this terrible stuff called code and script. But Gpt wrote that not me.

And I'll show you in a minute how to how that, how to approach that. And we have a similar story with the with the standards right? What we got here 8,000 entries here. So it's great being able to look for a standard for a specific requirement. I might know the standard right. I might be able to say 13, 4, 8, 5, and I want to look for 13, 4, 8, 5, and pull it, pull it up, and I can select it, and then I can say, export it. I can select a couple I can, I can, I can sort them, I can export them.

This is all created with Gpt. I can hide the abstract. I can open the abstract, but probably the more useful function is what happens when I want to check. If I have the the current standards in my document or the right standards in my document. So, for instance, I can go in here and I can load the document, which is basically a list of standards. And I can I get the links to those standards that are matched to the current document? Now, how do I know they're up to date. Well. we can look at another file with an unmatched standard in it. I know it's unmatched because I put it in there. But basically, we see immediately

149-71-2007 is the wrong year. This is much quicker than going through these files. Now the interesting thing is, someone asked about security earlier on. So there is an issue you have to consider if you're working for a company and uploading their documents to the cloud, basically to do this kind of analysis. But all of this work is done locally on the end users machine. So you're not uploading any document here. You're just asking the document to look at a doc asking the tool to look at a document locally. So so there are a couple of tools. We can say sophisticated tools that you can create using Gpt now.

Great.

You want me to do the demo.

Sure.

Okay, so.

I just wanna say that Martin Martin is gonna share both of those tools, and they're at the end of the deck that you'll get links to them.

So you will see we have a speech. I icon here so we can pull this up and gpt, we're we're live. We're in a webinar, and people are listening and and looking at you. That's fantastic. Hello! To everyone tuning in. I'm excited to be part of this webinar. Whether you're here to learn about AI discuss regulatory intelligence, or just curious. I'm here to help make this session informative and engaging. Let's dive right in. So so what I'd like you to do for me is to create an HTML page.

Let's try with a black background and some animated colored text, and and and the text has to read generative AI, and you work smarter, faster, and more securely, and thank the audience for being here, and also thank the speakers for speaking. got it? Here's a simple HTML page with a black background and animated color text with generative AI, and you work smarter, faster, and more securely. Let's dive right in. Here's a simple HTML page with a black background and animated color text. It would be great if the audience were to turn the pink off.

Let's turn to graphics in adobe easel. We're about to dive into something useful. Let's turn to graphics in adobe easel. We're about to dive into something useful. Thank you. You're welcome. So you will see that in that conversation it created me some code. There's the code. So we can copy the code, and I can put it in this window here. and I can save that window. And I really have no idea what's gonna come up. I hope it's I hope it's gonna work. And here we go, there we go. That is what Gpt did for us

today. Right now, as we were watching and listening.

Very nice, very nice. Should we go back to the deck.

Yes.

See, I am gonna share my here.

And I saw a bunch of questions about privacy and legality. We're going to talk about that after this, so we'll we'll.

There we are exactly.

Alright, so this is a, this is a really important topic. And the the answer is, it's complicated. So it depends on the model that you're using. So, openai. unless you opt out, may use your content to improve and train its AI models, it will not share it for other purposes, except for boilerplate, like, you know, if it require, if it gets valid legal requests. Those kinds of things, you know, all companies have to respond to subpoenas that kind of stuff. And I. But you can opt out of your data being used to train the models so that keeps it from being ingested, and if you have a I company.

if you have a company controlled Gpt. Account where you pay, and you have seats for all of your users that setting is off by default. And so the data retention for Openai, you know, which uses chatgpt things like that is, it will retain the data for as long as the users keep it plus 30 days, and you can create things called temporary chats. Those are also retained for 30 days, and so Openai will retain that information for abuse, for anti abuse purposes, things like seeing if people are trying to do illegal things, seeing if people are using the service inappropriately, that kind of stuff.

So Gemini is a little bit different. It's a Google product. So its data retention is tied tritely into Google's global data retention policies. And so it's very similar to the privacy stipulations are very similar to Google's regular data retention policies. So something to consider is if you email this document using Gmail. It has broadly the same privacy and data security implications as if you use Gemini. However, there are a few additional things when you're using Gemini unless you set settings in Gemini. Similarly, it may use they call it user content, which includes everything you say or do to Gemini to train models.

and it will and you can opt out of that globally. And so they will retain your data for a specific period of time for 18 months, and if you have workspace plans, you can adjust that between 3 or 36 months, and they have a similar thing to temporary chats. They have this global do not track setting in their privacy, and so they'll retain it for up to 72 h. And so both Openai and Gemi may have human reviewers of their data. Those human reviewers are bound to the same confidentiality and privacy, sharing as the whole company is. And so for people who are who have more strict privacy requirements than that.

There are things services like Amazon bedrock, which are much more complicated to set up and use, but they are. Oh, Sean, you want to go back here? But they are all right. Totally custom. You spin up an instance. Compute that you told that you own and so they don't store any outputs or data. They don't ever use anything to train any models. No human will ever review it. It only gets run through automated anti-abuse, which runs and then immediately discards the data. They don't retain any user content. So you know, as far as somebody asked about what should we have in our general guidelines? And so for for companies?

is I, I would generally say, you know, never, ever upload things like trade secret information anywhere outside your company, especially to AI tools. I would. Probably I would. Generally companies will disallow use of AI tools except for those that are specifically vetted. So you know, you work with your legal department to look through the terms. And usually that usually that'll say, Okay, we're going to add the Gemini enterprise to our Google Workspaces, or we're going to stand up an open AI company workspace for that stuff where we control those policies. And we're going to prevent that from

from changing. You know, people have a copy people talked about copyright infringement. So copyright infringement occurs, and I'll say I'm not a lawyer, and I'm not an expert in this area, so you should go check with your counsel on these matters, but generally copyright infringement is when you distribute material outside of what you're allowed. And so when you're uploading it to chatgpt to use it, and especially when you disallow it to be training. You're using the copyrighted material you may. You're not necessarily distributing it to people so that would probably be an allowed use. and people also talk about things like

should we be uploading like customer documents, things like that? And that's a that's a question that you should work with your legal counsel, your confidentiality agreements generally. It it generally you'll meet most confidentiality obligations. There's a whole bunch of details and data retention, information that kind of stuff that can get really complicated as you start as data moves into different tools. So and I'll definitely say that the free versions of tools. You don't get nearly as fine grain control the free versions. I think you okay, can almost never output the You can almost never prevent them from training on you that kind of thing. So certainly this is something that I'd only recommend that you do for work on

the work. Focused versions of these products? in conjunction with your legal team. And so people talked about like, Do I own the output? You know? What is the copyright status of the documents. And the answer is, today, it's an open question. It's pretty complicated. The I, you know I would generally view. Oh, go ahead, Martin.

Yeah. So open AI has a statement that says the user's own the output as long as it complies with applicable laws.

Yes, and I think that that's the second part. I think I think everybody allows you to open it. Whether you do is complicated, I think, generally say you. You probably do. If you, for example. uploaded an article and said, Spell, check this, you substantially created it, and you use a tool to spell. Check it, you know. Word spell check does not mean that you don't own the tool right? Chatgpt wrote an entire article, and you published it. you know that that's a really complicated gray area. It may not materially matter for most things. But it's it's certainly an ongoing area of law and research.

There's 1 last topic that I'd like to get to. We're not gonna get to everything. But will Jen replace you.

Yes.

No.

Let's it would.

Well, Martin, do you want to start on this one.

Yeah, I I think so. Basically at all right now, the way the tools are right now. No, because it's it's generated pre-trained. So it needs a trainer. It needs a Pre. Trainer, and it needs an expert to use it as we've seen today. If you, if you don't use your expertise in asking the right questions and getting the right information back and checking that that information is what you really want. It's gonna be very difficult to just expect that someone's gonna put in generative AI sitting at a desk

to to replace the quality management or the Ra manager. It's it's you know, I'm ready to receive your calls.

And I think it's you know my, my general thought is, it's gonna let you work smarter, faster, better than without it. you know when there used to be a whole bunch of people who would spell check documents before there were spell check. You know, if you typewriter wrote documents, people. There would be people who were reviewers who would go and help spell, check, grammar, review those kinds of things, and as those automated tools got better, it enabled individuals working on that stuff to do more, achieve more independently. And you know it's certainly going to push out the very lowest level of work as people who are skilled and efficient are able to become even more skilled and even more efficient. And people who are doing

the very bottom 10% of rote work. That work is going to be good targets for automation. And so, you know, I always encourage people to grow with it, to enable to enable it to enable you to be better, faster, more efficient at your job. You know people doing the very lowest level information is always going to be the easiest and 1st to to be able to be replaced. You know you're like, Oh, I spend all day every day doing document review. you know of just bulk documents that may not be as efficient as people who are actioning on those documents.

I I think there's a really key point. And that's this efficiency with efficiency comes expectation. So if people are becoming more efficient in their work. The bar is being raised. so you need to embrace these tools. otherwise you will not be meeting the expectations. It's as simple as that.

Right. And we're gonna close on that. I'm gonna say, I encourage my clients that you don't need to be an expert at chat, gpt, or generative. AI. You just need to be a little bit better than the next company or the next person. So what I encourage you to do is to set aside your resistance to change which is natural to all of us, and to start playing with the free versions, and to start using it in your day to day life. and then try to incorporate it into your work, and to take it one step at a time without letting it be overwhelming. I'm glad that everybody showed up today. Please take a moment to complete our survey. I'm going to sign off, guys. Do you want to say goodbye?

Yeah, thanks to everyone, it's been fun.

Thank you very much. Everybody hope to speak to you soon.