3Sixty Insights Dylan Teggart on How to Evaluate AI

[00:00:00] Du bist immer am Puls der Zeit und gerne up-to-date? Dann ist der Now Brief des neuen Samsung Galaxy S25 Ultra genau das Richtige für Dich. Stau auf dem Weg zur Arbeit? Dann wird Dir automatisch geraten, früher loszufahren. Es regnet am Zielort? Dann empfiehlt es Dir, den Regenschirm mitzunehmen. Mit dem Galaxy S25 Ultra kannst Du einen Schritt voraus sein. Neugierig? Erfahre mehr auf samsung.de.

[00:00:26] ServiceNow unterstützt Ihre Business-Transformation mit der KI-Plattform. Alle reden über KI, aber die KI ist nur so leistungsfähig wie die Plattform, auf der sie aufbaut. Lassen Sie die KI arbeiten, für alle. Beseitigen Sie Reibung und Frustration Ihrer Mitarbeiter und nutzen Sie das volle Potenzial Ihrer Entwickler. Mit intelligenten Tools für Ihren Service, um Kunden zu begeistern. All das auf einer einzigen Plattform. Deshalb funktioniert die Welt mit ServiceNow. Mehr auf ServiceNow.de slash AI for People.

[00:01:41] PeopleTek. Hi, Dylan. So you spend a lot of time looking at AI and where it's going in terms of, you know, as a business, in terms of product development. How do you think that people who do not have a tech background should evaluate AI?

[00:02:04] I think the most important thing is to kind of stop looking at the trends and what is trendy and what like the cool kids quote unquote are using and focus more on it as a tool. It's not necessarily going to revolutionize your business today, but it could over time.

[00:02:23] And thinking of AI as a business tool and not just technology is a better way to invest in it to better meet your business goals rather than just kind of jumping in on the tech jargon bandwagon, which I think. In a day and age where there's new tech around the corner all the time and you want to seem like, yeah, I know that technology. Of course, I know that technology. You don't want to seem like you're behind the eight ball. It's really easy to get caught up in the fads, I guess you could say.

[00:02:53] Right. And looking at it more as a business tool, just like having a new timekeeping system or a new computer terminal at work is a tool is the best way to kind of think about how it belongs in your business, if at all. But I think the way it's looking right now, AI probably has a place in every business, especially if you do want to work at a modern pace.

[00:03:15] But really looking at how your business functions and where it could fit in and not trying to have your business fit to the AI, have your AI fit to your business. You know, I'm reminded of an old saying that nobody goes to the hardware store because they need a quarter inch drill bit. They go because they need a quarter inch hole.

[00:03:38] And I wonder if that's the way more people should approach AI, not looking at the technology itself, but looking on the results they need. Absolutely. Yeah. Outcomes should be really the most important thing, if not, you know, number two on the list of most important things. Like you said, what problem are you trying to solve? How will it impact your employees, operations, customer experience?

[00:04:06] And are there going to be ethical risks or compliance issues with you using AI? AI powered, you know, talent acquisition tools can screen resumes faster. But if not designed properly, you introduce a whole other layer of very complex bias that could be your next problem to deal with. So it may be faster, it may not be the most effective. It really depends on how you train it, how you position it, really.

[00:04:36] If you let it run wild, it's going to do whatever it thinks is right. But you need to really frame your usage of this just like you would any other tool. Like, why are you buying a new operating system for your computer, for all your computers at work? You know, you need a justification for it, not just this is what other people are doing. It should be addressing a problem, addressing an issue or addressing how you're going to approach operating your business in the future.

[00:05:03] Do you think people are beginning to do that or are they still hung up on they need AI, you know, that implementing some kind of AI completes the mission? I think these days now that, you know, it's been in the culture, I guess you could say, the culture of, you know, in Western and any, well, I guess any modern technology culture for at least a year or two.

[00:05:30] And everyone's interacted with it in some sort of way, even if it's as simple as doing like a search online. I think people's expectations are a bit more realistic as to what it's what's possible. It's not going to be able to do certain things like highly complex tasks or write, you know, the next great American novel yet. But it could help you design some certain lists, help you with note taking, translate things, you know, get little projects done.

[00:06:00] And I think also, if you look at the economy at large, I think people are beginning to realize that this tech is potentially a bit of a bubble akin to perhaps the dot-com bubble. While that's a huge concern for the American economy, I think it is also having people develop a bit of a hesitancy towards what are the outcomes? Is it going to replace my job? Is it going to replace all of our jobs? Probably not. Is it going to enhance our jobs? Absolutely.

[00:06:28] But also, people are kind of pumping the brakes a little bit about adding AI to everything that they do because are they just throwing money down the hole when this could potentially be a bubble that's about to burst? Or is going to adjust in a way that has everyone reevaluating their investments and expectations and values around this new technology, if that makes sense. You know, there's so much buzz about AI and there's so many people saying so many things about it.

[00:06:57] But how do you put that in perspective? I mean, how can somebody who's a smart person, a business leader, but not necessarily an IT leader, how can they figure out what's true and what's not? That's very, very hard. It comes down to, I think, maybe creating parallels when you're training someone on this and understanding that it's not akin to a person.

[00:07:23] It's more akin to a highly advanced maybe search engine in a lot of people's cases. You know, it's a grouping of multiple functions. So it's Google Search. It's Google Translate. It's a chat bot simultaneously. It's not a replacement for an event being, but it's something in the middle. And I think explaining it in a way that just makes it more relatable, that it's, yes, it's a technology shift, but it's also a change in the culture of how we do things.

[00:07:53] So just like when Google was around, first introduced, people were, their minds were blown. This isn't so different from that. And it's baby steps. I was skeptical myself with AI, and now I use it all the time because sometimes you just need a little assistance. And I'm sure someone back at the library was thinking that Google Search was cheating back in the day.

[00:08:18] But now someone who loves Google Search is probably thinking AI searches are cheating. It's just the natural evolutions of things. We don't use an abacus anymore for a reason. And, you know, we have to spend time doing other things. And this is just going to help you offload some of that work that you would have been spending elsewhere doing more manual tasks. And that applies to the HR world in a big way.

[00:08:44] It's taking stuff like timekeeping, bookkeeping, potentially, you know, double-checking records, flagging things, stuff that, you know, if your eyes are tired and it's the end of the day and you're looking over your employees' clock in times, you might miss something. But this is taking that out of it. It never gets tired. It's going to flag information for you. Should you double-check it? Sure.

[00:09:06] But it's really just offloading, redundant, mundane work that you would have done and that you would have to do if it was not around. And I think that's the most important way to look at it is have it do basic tasks that you hate doing and then move on from there. Just think about the lowest-hanging fruit. You know, I hate entering in timekeeping numbers. I can't, you know, it's like my least favorite thing to do in a week. Try it out with AI. See how it goes.

[00:09:33] And I think just kind of testing the waters very slowly for someone that's not familiar with it or is a little scared about it is the best way to do it. Let it do the number one thing you just don't like doing first. If you're happy about it, have it maybe do something else or make it, ask it a more complex question in the same sphere and then go from there. And let it help you. Have it work for you. Don't work for it. And that's where kind of coming back to it being a tool again.

[00:10:01] Whether you're a payroll pro or an employee wanting to understand your paycheck better, we've got you covered. Tune into It's About Payroll for expert insights on payroll trends and compliance. Or check out It's About Your Paycheck, the go-to podcast for employees looking to understand their pay and rights. Two great shows, two great hosts. Listen now, brought to you by Work Defined, where payroll meets clarity.

[00:10:30] Well, it has the advantage of being pretty simple to use, you know, from the end user's point of view. I mean, most of the large language models out there, you know, a truly conversational interface. So you don't have to have a properly structured query like you do in, say, Boolean search or even Google. You need to have a detailed prompt to get the information you want. But it seems like it's easy to get there.

[00:10:59] Do you think that's driving a lot of the excitement? Absolutely. You know, I did some early on before I worked for 360, I did some AI troubleshooting. So I think it was with a company called Data Annotation, which I think was an offshoot of ChatGPT. So we'd take, they wanted artists and writers and other people and stuff like that who have a broad range of expertise to kind of work part-time.

[00:11:26] Looking at issues that were flagged within the system that were, it gave bad results. And sometimes it's user error. Someone, you know, types in something that's extremely vague and doesn't make sense. But the more questions you ask it, the better it gets at analyzing how humans ask questions. And you can ask a question in not a million ways, but maybe a hundred ways or structure it in a way or misspell things.

[00:11:56] And I think that's where its strengths comes in is, is it take common parlance or take slang or take misspellings and still spit out the correct answer? Was something that I think was largely manually done by people like the hundreds, if not thousands of people that were in my position, just part-time looking through these queries. Why did it answer that question incorrectly? Oh, as well, because someone put you are instead of, you know, Y-O-U-R.

[00:12:23] Or training it to look through those is really important. And I think, you know, with ChatGPT and some of the other ones, you can just speak to it now. And that's opening up a whole other can of worms where, you know, people speak with different accents, different inflections. That's going to be a whole other can of worms.

[00:12:40] But I think with time, the more it ingests, the easier it's going to be to ask a questions to the point where it can maybe read through the question you're asking. And I think it's already kind of doing that now. It's like, you're asking this question, but maybe you also mean this. Because in my past experience, you know, it's not telling you this, but in my past experience, 10% of the people who asked this question are also interested in this.

[00:13:09] So I'm going to assume, just like we assume in conversation that, you know, some things are implied, that you might also be interested in this as well. And I think for low tech or less experienced tech users, especially in the HR technology space, you know, with the implementation of ChatPots and things like that, not everyone's an IT expert or not everyone even has an email address. But they do know how to ask questions.

[00:13:34] And being able to ask a question in either your native language or in common parlance, like just speaking the way we're speaking right now, or the way you would speak to someone at work that you've known for 10 years is really important because it just removes that barrier. You just talk to your phone, ask it a simple question, and it's going to hopefully give you an answer. And the more questions we ask it, the better it's going to get.

[00:13:59] Now, you're out there talking to a lot of people and a lot of executives about AI and what they're doing with it. But when you're talking to them, do you find that there are common themes to what they get wrong? I mean, are people just sort of naturally all making similar misassumptions? Good question. I think it really varies. I do find any field in the world, there's fads.

[00:14:27] And I think we just finished, we're wrapping up the AI fad, I feel like. It's starting to taper off in a way that, not that the technology is going anywhere, but that the expectations are being tempered. Because it's not, like I was saying earlier, it's not going to make your lunch for you. It's going to maybe give you a recipe for how you can make your lunch. And I think that's the natural progression of things.

[00:14:55] So I think, I guess if there's any advice I have to business leaders and people who work on this type of thing, or are in charge of the ability to bring on new components of technology to their software platform or to whatever the vendor is selling, is to really justify your investments to people. Be able to really explain it in something that is, in a way that is unique to your company.

[00:15:25] That makes it seem like you're not just spewing out the same information you've heard from a bunch of people, because you feel like you have to. I would really respect a business a lot more if they said, you know, yeah, I know that's a big trendy thing, but when I look at our way we do things, it doesn't really align. I'd be like, oh, great. You've thought about this a lot. But making thoughtful, considerate decisions about technology, I think, is almost more important.

[00:15:51] And explaining that thought process in a way that is respectable and relatable is important, I think, for customers to hear, because it makes it seem that every decision you make in your business is based upon practical decisions and outcomes, not so much optics. And I think a whole lot these days, because things are so quick, you know, trends come and go so quickly, faster than ever. Optics has almost become more important than reasoning.

[00:16:20] And reasoning ultimately is where you build trust, I think, in a company. And justification as to why I should spend all this money working with you. You know, you want something that has depth to it, not that it's just surface level. And I think a lot of the times people are focusing too much on that surface level, saying you're doing something, adding AI to the end of a sentence.

[00:16:44] I think sometimes it's more important to just say, we're going to really examine this and then see where it fits into our business. And then making a really strong case for it, I think, is very important. And it helps legitimize your position as a business. So how would you advise executives to sort of sort through all the noise when they're looking at vendors? Like, who are they going to spend their money with? You know, these are people who know their business.

[00:17:13] They know how to run a business. Again, they're not technical people. How can they tell who is coming at them with, like, real solutions as opposed to somebody who's just talking about it? I think, you know, the vendor should be selling to the business. So I think asking, okay, what is your product going to do for my business? This is a case study of what we do in a day. How can you help me? This is where I'm suffering. How can you help me?

[00:17:42] And then beyond that, asking how the AI makes decisions. What data is it trained on? You know, is there a risk of bias? Is there a risk of data breaches or data being, you know, put into a system where that could compromise your compliance or legality in some situations? You know, the AI model is ingesting information that is private to your company and then gets thrown into the mix with other companies.

[00:18:11] You want to avoid that. Most vendors are very good about preventing that from happening, though. And then how transparent is the model? Can you audit its decisions? Things like that. This really nuts and bolts things because ultimately you want to be focused on the communication training and transparency of the AI because people are going to be really skeptical of it. You know, staff is going to be skeptical of it or even the business owner is going to be skeptical of it.

[00:18:39] And it's going to bring a culture shift. You know, the size of that culture shift is going to be varying depending on how you use it and where it fits into your business. But I think, again, just keep viewing it very practically. It's a tool that it's meant to give you results and start there. Keep it very simple.

[00:19:01] Build from the ground floor and then really see if it fits into every level, every layer of your business, opposed to just having a blanket over everything and figuring it out later, which I feel like a lot of people, not everyone, but it's a common theme. You know, dealing, thinking about it after you've already paid for it. Dylan, thanks very much. It was great to talk to you. I appreciate all of your thoughts and I hope we'll talk again. Pleasure. As always, Mark. Thank you.

[00:19:41] My guest today has been Dylan Taggart, an analyst with 360 Insights. And this has been PeopleTech, the podcast of WorkforceAI.news. We're a part of the Work Defined Podcast Network. Find them at www.wrkdefined.com. And to keep up with AI technology and HR, subscribe to WorkforceAI today. We're the most trusted source of news in the HR tech industry.

[00:20:10] Find us at www.workforceai.news. I'm Mark Feller.