Ep 25: Increasing Fairness in Hiring Through AI-powered Interview Intelligence with Mark Simpson

[00:00:00] Feeling kind of left out at work on Monday morning? Check out The Barf, Breaking News, Acquisitions, Research and Funding. It's a look back at the week that was so you can prepare for the week that is. Subscribe on your favorite podcast app.

[00:00:25] Hi everyone, it's Bob Pulver. Thanks for checking out another episode of Elevate Your AIQ. Today I'm excited to bring you an insightful conversation with Mark Simpson, the founder and CEO of Pillar. Pillar's platform focuses on interview intelligence, which is a hot topic in 2024.

[00:00:38] And once you hear Mark's perspective on this space, you'll understand why. Mark and I dive deep into the world of AI and HR, exploring how it's reshaping the interview process and modernizing talent acquisition. You'll hear how interview intelligence is making hiring more efficient, effective and fair, and we'll dig into its impact on quality of hire as well.

[00:00:55] Whether you're a recruiter, HR pro, job candidate or just curious about the future of work, this episode is packed with valuable insights on mitigating human bias by leveraging AI responsibly in the hiring process.

[00:01:06] Please check out Pillar's free resources in this episode's show notes, including a hot off the press report on interview intelligence from aptitude research. Let's jump right in.

[00:01:17] Hi everyone, welcome to another episode of Elevate Your AIQ. I'm your host Bob Pulver. With me today, I have the pleasure of speaking with Mark Simpson from Pillar. How you doing, Mark?

[00:01:26] Very good. Hello, everyone. Bob, thanks so much for having me on. I'm a big fan of the show.

[00:01:30] Thank you. Thank you. Glad to have you. So, Mark, why don't you just kick things off with your background?

[00:01:35] And I've been working in AI and a bunch of different organizations for many years. So I'd love to hear a little bit more about that.

[00:01:43] Yeah, sure. Well, I'm British, so I don't like talking too much about myself, but we'll give it a go for a couple of minutes.

[00:01:48] But yeah, I've founded three companies now, the first one back in 2006, and all of which have been focused or used a considerable amount of machine learning and AI throughout the years.

[00:02:00] So they've all been B2B software companies and have developed those three companies as well as worked for a couple of very large companies as well, Oracle and IBM, specifically within IBM in the Watson team there.

[00:02:13] And my latest company, Pillar, is focused on helping talent acquisition professionals and companies as a whole recruit in a much more efficient, effective and equitable way.

[00:02:24] And we focus primarily on interviews and are revolutionizing the interview process just to make them take hours out of people's calendars, make better hiring decisions, and make sure those interview processes are fair and equitable as you go through them.

[00:02:41] So that's the current focus. I'm most excited about this business. We've had a couple of good runs at other businesses.

[00:02:48] But for me, I think this is the biggest problem that companies could solve right now.

[00:02:54] Excellent.

[00:02:54] Yeah, no, I do think it's a hot area, right? We've seen this kind of evolve over the last couple of years in part from folks like Pillar.

[00:03:03] It's interesting to hear you talk about when you started working with AI as well as the Watson connection, which you and I have in common.

[00:03:13] When I was at IBM and there were a lot of people coming up with new ideas for how to use Watson, a lot of the great ones and impactful ones seem to be in HR.

[00:03:23] And yet it took until much more recently for some of those to really take off.

[00:03:31] I'm curious to get your perspective on sort of the evolution of some of the AI capabilities.

[00:03:37] Obviously, you know, both of us predate, you know, generative AI.

[00:03:40] And so, yeah, I'm just curious to get your take about what was happening, you know, in the sort of HR space even, you know, a decade ago.

[00:03:48] I think it's super interesting that, you know, for most of us, we just think AI has been around for a year, right?

[00:03:54] When chat GPT sort of opened itself up in 2023.

[00:03:58] But if you look at the real history of AI, it's been around since the 50s.

[00:04:02] I think Alan Turing published in 1950 some computer machinery and intelligence, an article around that.

[00:04:08] And the first kind of workshop around AI was actually at Dartmouth by John McCarthy around artificial intelligence.

[00:04:14] And then AI has kind of developed through and then probably things that people of my age anyway remember where AI was actually in action was when Deep Blue, which was actually by IBM, beat, you know, Gary Kasparov at chess.

[00:04:28] And then, you know, that was in 97.

[00:04:30] And then this kind of speech recognition, which is AI and that, you know, Microsoft did that in 97 as well.

[00:04:36] But AI really sort of started developing through the 2000s.

[00:04:40] And, you know, it wasn't until, you know, really sort of Twitter, Facebook, Netflix sort of started implementing algorithms and AI for their advertising purposes back in 2006.

[00:04:52] And then in the sort of 10s, you know, Apple released Siri.

[00:04:57] And in 2020, it was open AI started, you know, chat GPT-3.

[00:05:02] So, you know, applications of AI have spanned decades.

[00:05:06] In HR and HR tech, it's really only over the last sort of decade or so that we've started seeing sort of AI and what have you come into play.

[00:05:17] And we see it quite a lot with things like assistance and, you know, organizing and scheduling interviews and organizing elements like that and using algorithms for advertising and marketing purposes,

[00:05:30] because it dates back a little bit further from those stages.

[00:05:33] So it's actually been around for longer than we think.

[00:05:37] But as I say, it was really only last year that suddenly everybody realized that AI was going to have massive implications on the way they work and for the future.

[00:05:45] So there's become much more excitement around it.

[00:05:48] But, you know, so been around for a while, but it's still a very young kind of baby and still developing.

[00:05:53] And we've got a long, long road to go from here.

[00:05:56] Yeah, I would say so.

[00:05:57] There's been, I was just reading something from McKinsey around the adoption that is taking place in the disparity between what employees are using,

[00:06:08] whether companies know it or like it or not, versus, you know, the actual official adoption.

[00:06:14] I think McKinsey's disparity is actually greater than some of the other ones that I've seen.

[00:06:19] Like instead of 75% employees using it and 50 or so percent companies using it, it was more like 85% employees are using it.

[00:06:29] And only like 15% of companies have actually put a strategy in place and started skilling and, you know, have governance in place and things like that.

[00:06:39] So we are very early days from a maturity standpoint, which is interesting because as you were talking about it, I was thinking about, you know, we've had the analytics, you know, sort of maturity curves.

[00:06:50] You've had an automation maturity curve and AI is still as young as early as we still are with the possibilities of AI compared to some of the robotic process automation or, you know, basic, you know, business intelligence and things like that.

[00:07:05] We have come quite far.

[00:07:06] I'm curious just about the, you know, the laser focus on interview insights, interview intelligence with Pillar and how you got to the decision that this is the sweet spot.

[00:07:18] Yeah, I've been fortunate that after each of the companies that I've sort of built, I've had time off afterwards.

[00:07:25] And it's given me sort of really good time for reflection about, you know, what are the big challenges that I face?

[00:07:31] What are the big challenges that businesses face, both large and small?

[00:07:35] And every time that I've done that, the number one thing on the list has always been surrounding people.

[00:07:40] You know, how do we reliably get the right skills in a fair and equitable way into the roles that I'm recruiting for and building the teams that I want to build?

[00:07:49] And, you know, I actually think that talent acquisition teams have done a phenomenal job with the tools that they've had at hand over the last 10 years or so.

[00:07:57] And it's mainly been focused at kind of top of funnel, you know, how to get people into, you know, the top of the funnel to then get into interviews and be interviewed by the lines of business to then recruit them afterwards.

[00:08:09] Where things have really fallen apart or where things have got stuck have been from that top of funnel where talent acquisition teams are supplying a really good sort of supply of highly qualified candidates.

[00:08:22] What actually comes out the other end when they start getting interviewed?

[00:08:26] Because two people have gone into a room in an interview.

[00:08:31] They've had a conversation.

[00:08:32] One person's come out the other side and said, I like them or not.

[00:08:36] And nobody really knows what's what's been going on.

[00:08:39] And it's been a very, very big sort of area where I just had, you know, issues with what's the quality of that interview and did the candidate get a fair chance?

[00:08:49] And are we really assessing skills in the right way?

[00:08:51] So I see it as probably the biggest, you know, weak point or challenge point in the talent acquisition process.

[00:08:58] And it wasn't really until COVID happened that I could find a way of actually sort of attacking that problem.

[00:09:05] And, you know, the one big thing that happened with COVID is that suddenly every interview went from being in a room where nobody knew what was going on to being on Zoom and being in Teams where, you know, suddenly, you know, if you can understand the conversation using AI, you can actually start understanding, you know, what questions people were asked, what skills they have, you know, how well they were interviewed, what the conversation was all about.

[00:09:31] And, you know, just with the ability to understand that, with the ability to pull data out of it, we can actually see now how to improve that process, how to make it, you know, massive amount more efficient.

[00:09:42] Because, you know, the first times we started seeing that, we could see that, you know, people were getting asked, interviewed five different interviews and they were being asked the same questions five different times.

[00:09:51] We were, you know, we could see that people were, you know, spending way too much time talking about their favorite football team or their dog or whatever it might be.

[00:09:58] And, you know, and not actually sort of getting into the real crux of actually finding out whether that person has the skills for the role.

[00:10:06] And, you know, just with all that data, we can now help guide the process.

[00:10:09] We can help make it more efficient.

[00:10:11] We can, more importantly, help make it more effective.

[00:10:13] We see, you know, things like first year turnover rates drop significantly.

[00:10:17] And we can make sure that everybody has a fair and equitable chance of getting that role.

[00:10:22] So we've attacked that.

[00:10:24] We've put, you know, a number of kind of free tools out to the markets.

[00:10:27] We want to help the market in general.

[00:10:29] But we obviously, you know, have a lot of customers which we're helping go a lot deeper in both providing a platform that supports interviewing now for really the first time ever, as well as providing a lot of those efficiencies back into companies as well.

[00:10:44] Yeah, I think there's tremendous opportunity there.

[00:10:47] I certainly agree.

[00:10:48] I've been on both sides of that interview conversation, certainly.

[00:10:51] And it is easy.

[00:10:53] I mean, it's just human nature to try to find, you know, commonality with someone else.

[00:10:58] Is this someone that even just from a, you know, a culture ad and a sort of relationship standpoint, it's in our nature to want to find hopefully the best in people.

[00:11:08] But maybe some interviews, interviewers don't do that.

[00:11:12] Maybe they're looking for reasons to reject as opposed to reasons to accept.

[00:11:16] And if you don't have the sort of documented, you know, evidence and traceability of how it actually went, then you're sort of trusting those interviewers.

[00:11:28] But you can't get better at interviewing if you don't do that.

[00:11:32] That's one aspect, I guess.

[00:11:34] The other is, you know, how do you mitigate?

[00:11:37] Hey, this is William Tenka, Work to Fun.

[00:11:40] Hey, listen, I'd like to talk to you a little bit about Inside the C-Suite, the podcast.

[00:11:44] It's a look into the journey of how one goes from high school, college, whatever, all the way to the C-Suite.

[00:11:51] All the ups and downs, failures, successes, all that stuff.

[00:11:54] Give it a listen.

[00:11:55] Subscribe wherever you get your podcasts.

[00:11:57] You know, some of those human biases of which we have dozens, as you know.

[00:12:01] So how do you mitigate that in some way with technology without technology actually, you know, making the decision?

[00:12:09] But at least it's presenting you with the right information, which is what we all want anyway, right?

[00:12:16] We're all trying to augment our own sort of judgment with some type of, you know, in your case, you know, an interview co-pilot or just somebody that can take, you know, meticulous, you know, notes.

[00:12:29] And then from there, you've got everything that you need to make a data-backed decision.

[00:12:34] Yeah.

[00:12:34] And I think that's the key where we put focus.

[00:12:38] I mean, I think with each of the companies that I've built, it's always, we've always had a real focus on just supporting the practitioner.

[00:12:46] You know, being that sort of, whether it's the interviewer, whether it's the talent acquisition team or whatever, you know, sitting alongside them and helping them do their job better.

[00:12:54] So I think that, you know, you know, that decision making, as you say, Bob, is ultimately the most important part of it.

[00:13:02] And just being able to make the decision based on a proper guided interview process, which we can provide.

[00:13:09] So you know that somebody has been assessed with all their skills.

[00:13:11] And then when you're actually making the decision, you know, instead of going into, I don't know, a lot of companies will do these debrief meetings around candidates.

[00:13:19] And, you know, typically they'll go in and, you know, the interview team will get there and everyone has their best intentions at heart.

[00:13:25] But they're looking at, I don't know, three-week-old notes on three different candidates and trying to remember the conversation.

[00:13:30] And really what they're remembering is whether they like the person or not, or, you know, some kind of guide in a few bullet points in their notes.

[00:13:37] But now you can just go back and use AI-generated kind of video clips to, you know, replay what people have said around their skills.

[00:13:44] And actually you're starting to be able to, you know, actually make a decision based on what someone has said around their skills and have an educated conversation about it rather than who you like best and those more fuzzy memories.

[00:13:58] So the quality you're getting in terms of hiring changes dramatically when you take that sort of more data-driven approach.

[00:14:07] And that's what pillars there to help support.

[00:14:10] Yeah.

[00:14:11] I think one of the things that has come up a couple times in recent conversation around AI's ability to not just transcribe a conversation, we've had transcription services for a long time.

[00:14:23] And a lot of them are getting, you know, much better.

[00:14:26] But one of the concerns I've heard and seen is around summarization.

[00:14:31] And so I was curious about how you have tackled that particular subject to make sure that nothing that would be an interesting and important nugget to a human as part of that conversation doesn't get overlooked in its own effort to give you the most concise summary.

[00:14:52] I'm just curious how you guys have approached that.

[00:14:55] Yeah, it's kind of interesting because you could take an interview and Zoom and Teams will do a pretty good job at transcribing that interview and then they'll make a summary.

[00:15:05] Or you could take that transcription and put it into, I don't know, chat GPT or any of the other models, large language models that are out there.

[00:15:14] What you'll get back is, it's okay.

[00:15:17] It will give you a summary of the conversation, but it's not particularly useful for talent acquisition, for interviewing type purposes, because it could have hallucinations in it, which are getting better, but they're still there.

[00:15:32] It probably will not look at things like bias.

[00:15:36] It will deal with how people identify particularly well and the way in which they want to be represented.

[00:15:44] So there's a number of different areas where we need to use the more human side of what we do day to day, overlaid with that more sort of technology side, which doesn't cope as well with that right now.

[00:15:58] When we're doing summarizations of interviews, which is a massive time saver, it's what a lot of people are interested in.

[00:16:04] We're probably saving five to 10 hours out of a recruiter's calendar every single week just by summarizing their interviews and then starting to coordinate the experience.

[00:16:15] And we are layering a lot of code and technology on top of base level, large language models to make it useful for them, to summarize an interview in a way an interview should be summarized, to make sure that the candidate and the interview are treated in a fair way in terms of the way it's summarized, to make sure that biases are removed and it's fair and equitable.

[00:16:35] So there's a lot that goes around it that would just be unacceptable if you just put it through a basic large language model right now.

[00:16:44] I think they will improve though.

[00:16:45] I don't think that that's necessarily the value that we add in five years' time.

[00:16:50] But right now you do need to be a little bit careful with some of the models that are out there.

[00:16:53] Right.

[00:16:54] So I know this is probably more on your client's side of things, but when each interviewer is done, do they typically get asked to review the summary of their own conversation?

[00:17:07] Are they the human in the loop of their own conversation just because they were there and they'll make sure that it's as accurate as possible before it's submitted and reviewed by others?

[00:17:20] Yeah, so for most of our customers actually serve a guide into the interview itself, into the Teams window, into the Zoom window, so that every interviewer knows skills that they are asking about.

[00:17:32] It will give you suggested interview questions perhaps as well, and it will have an area for comments and a rating scale.

[00:17:39] So it does two things.

[00:17:40] One is it will guide the conversation.

[00:17:42] So the interviewer is then guided in terms of having the conversation in the right areas.

[00:17:48] And the second is it's collecting data back in order to automatically fill out a scorecard back into an applicant tracking system or not.

[00:17:57] So after an interview happens, we will process everything that the interviewer filled out in the scorecard, which is alongside the interview.

[00:18:06] And we will then send them an email just to check the scorecard and check that what they've put in is being represented correctly after the interview happens.

[00:18:15] And they have the opportunity to change that.

[00:18:17] They can also check the summary.

[00:18:19] We do not have interviewers changing kind of summarization.

[00:18:23] The summarization is just really top quality now.

[00:18:27] It summarizes the conversation well.

[00:18:29] It's understandable.

[00:18:30] It's easy to consume.

[00:18:31] We can do it in a number of different ways, in paragraphs, in bullet points, in write-ups that can be programmed by our customers to be done in certain ways.

[00:18:40] So those can be reviewed.

[00:18:42] But, you know, the checking is not so necessary anymore.

[00:18:46] But just collecting the entirety of that data, the summary, the scorecards and everything in one place is just, you know, you suddenly have a complete set of data around every single interview.

[00:18:57] And the candidate is getting a much sort of fairer go through the process than it would be otherwise.

[00:19:03] It's almost comical to think back when I was interviewing for roles, you know, even, I don't know, five years ago.

[00:19:14] Just how haphazard it was.

[00:19:16] I mean, even when I was sort of acting hiring manager, when I was in a chief of staff role, I mean, I wasn't any better at this, right?

[00:19:25] I mean, we might have a couple key questions, but it was just like, who's available to talk to this guy?

[00:19:32] You know, I liked him.

[00:19:34] Joe liked him.

[00:19:35] Give him the, you know, the rundown and, you know, see what you think, right?

[00:19:39] And so that's just no way to run an efficient, you know, talent acquisition, you know, program.

[00:19:46] And that really is the only true opportunity to get to know this candidate off, you know, outside of the piece of paper that they submitted as their resume.

[00:19:57] Especially if you're going, not just doing maybe full-time hiring, but maybe you're hiring contractors.

[00:20:03] Maybe you're going through a vendor management system or something like that.

[00:20:07] You might have even, they might be pre-vetted in some cases, but what does that even mean, right?

[00:20:12] It just seems like we're not treating people, you know, consistently and we're certainly not getting the best sort of collective assessment of what this person's really, you know, capable of, you know, trying to mitigate any type of bias in the process.

[00:20:29] Yeah, Bob, I look back when I first started building teams and I was awful.

[00:20:35] Like, it's actually kind of shocking how, you know, the process that I went through looking back and in reflection around, you know, who I was choosing for the roles.

[00:20:47] And, you know, we all do our best.

[00:20:49] You know, we will have our best intentions going out into interviews and we're trying to be fair and we're trying to be equitable.

[00:20:54] But just the tools that were on hand sort of 10 years ago, 15 years ago, we're just not there to kind of help, you know, make that, you know, unbiased, fair, equitable, skills-based decision.

[00:21:08] And I think the talent acquisition teams have done a phenomenal job over the years in making that a lot better.

[00:21:16] And they've struggled as well, but put in place sort of guides and try to get people to use them and they're not sure whether people are using them.

[00:21:22] And they're going out there and chasing after feedback for interview and spending hours and hours doing those sort of those tasks, which, you know, they know make a big difference to the quality of the interview process.

[00:21:36] You know, now that there is sort of newer technology out there, it just makes their jobs a whole lot easier.

[00:21:42] Anyone can go onto the Pillar website right now, put in the, you know, put in the job description and we'll spit out, you know, good interview questions and an interview guide for free.

[00:21:51] You know, we put that out there and people can do it.

[00:21:54] We actually have a job description analyzer that people can put in their job description or come back and it'll rate, you know, changes that should be made to the job description to try and attract candidates in a fairer way.

[00:22:04] Again, for free and just having those tools and having the ability to go into an intake meeting, use AI to, you know, create your guides, assign them to your interview team really in minutes rather than hours or in seconds rather than minutes.

[00:22:22] You know, they now have, you know, great tools at their fingertips to coordinate that process so that everybody's getting asked similar questions.

[00:22:29] They're all being tested on the same skills and then people can make, you know, really informed decisions about who the best person is for the role.

[00:22:37] So, yeah, I go back when I first started, absolutely shocking.

[00:22:41] I go back even five years before I really started diving into Pillar.

[00:22:45] I still don't think I was good, even after, you know, 10, 15 years of experience of interviewing.

[00:22:50] And I think it's really only sort of now that we're actually, you know, hopefully doing something really good, both for companies that are recruiting, but actually also for candidates as well.

[00:23:00] Yeah, for sure.

[00:23:00] Everyone's looking for fairness in this process, right?

[00:23:04] And ideally, we do a better job with...

[00:23:06] If you like swiping, then head over to Substack and search up Work Defined, WRK Defined, and subscribe to the weekly newsletter.

[00:23:16] Hi there, I'm Peter Zollman.

[00:23:18] I'm a co-host of the Inside Job Boards and Recruitment Marketplaces podcast.

[00:23:23] And I'm Steven Rothberg, and I guess that makes me the other co-host.

[00:23:26] Every other week, we're joined by guests from the world's leading job sites.

[00:23:30] Together, we analyze news about general niche and aggregator job board and recruitment marketplaces sites.

[00:23:36] Make sure you sign up and subscribe today.

[00:23:40] Things like this, then maybe we can reverse this shrinking trend around tenure.

[00:23:46] Maybe we can improve engagement.

[00:23:48] I mean, I just feel like both parties should be looking for an optimal match, right?

[00:23:53] Because you want to be happy with what you're doing.

[00:23:55] You want to be engaged, and you want to collaborate with the team, and you want to be part of the mission that you've bought into.

[00:24:01] And so I think the better job we can do up front, the more impact we can have positively on what the organization seeks to do.

[00:24:11] We actively try and monitor sort of first-year turnover from our customers, for the customers who have that date, and we'll give it back.

[00:24:19] And look at manager sort of assessments and reviews, 90-day reviews, or whatever they might be.

[00:24:25] So I agree.

[00:24:27] I would love to change this trend of tenure.

[00:24:30] I think first starting with nobody should be leaving a permanent role within six months.

[00:24:36] Something's gone wrong on one side or another, or even 12 months.

[00:24:40] Something's gone wrong.

[00:24:41] And I think that's how we're trying to attack things right now, just starting at that sort of first period where there is significantly more turnover than there should be right now.

[00:24:51] Quality of hire.

[00:24:52] That metric, I know there's not a consistent way to measure that.

[00:24:57] And there's been a lot of conversations over the last 18 months or so about that metric, as well as who should own it.

[00:25:06] And I had a quick little back and forth with Tim Sackett about the fact that talent acquisition doesn't think they own quality of hire.

[00:25:14] They track enough metrics that actually relate to the recruiting process.

[00:25:19] Quality of hire is not their issue, sort of summarizing.

[00:25:23] But I just feel like the more we think about the talent lifecycle as one continuum, you can keep the organizations separate.

[00:25:32] But to say that the candidate became the employee.

[00:25:35] It's not a different person.

[00:25:37] Right.

[00:25:38] Right.

[00:25:38] So if they do leave, to your point, within the first six months, you know, putting on my process, you know, root cause analysis, you know, hat on.

[00:25:49] Like, you've got to go back to when we first interacted with this candidate.

[00:25:55] Or were there things that we missed?

[00:25:57] Were there signals that should have been picked up that said, you know what, this is not going to be a successful long-term relationship?

[00:26:07] And maybe nothing happened before they onboarded.

[00:26:11] And then that's great.

[00:26:13] Maybe they had an issue with their manager.

[00:26:15] Maybe they had an issue with you changed from, you know, work from home to return to office at the three-month mark.

[00:26:22] And, you know, this was sort of anticipated in some ways.

[00:26:27] But you can't ignore the fact that if you do everything that you can do to make sure that this is the right match, you know, up front, then you mitigate them, you know, turning around and walking out that revolving door.

[00:26:43] Yeah, absolutely.

[00:26:44] And look, I think the quality of hire firmly, obviously, should sit with talent acquisition.

[00:26:49] And I love Tim.

[00:26:51] Tim's always throwing things out that are controversial.

[00:26:54] I think it's just not been possible to do that analysis until recently.

[00:26:59] It's not been possible to look back to, you know, how someone was really assessed, you know, which for the most part is in an interview.

[00:27:06] Obviously, with technical roles, you can do, you know, specific sort of skills assessments and the like.

[00:27:11] And there are personality tests and so on and so forth.

[00:27:14] But where a lot of the decision is made is in the interview.

[00:27:18] And without being able to see into that, without being able to understand what the conversation was about, you haven't been able to look back and understand were things missed or were decisions made that were the wrong decisions or for the wrong, well, the right decisions or wrong decisions, but for the wrong reason.

[00:27:34] So, but you can now.

[00:27:36] It is possible.

[00:27:37] It's a new way, a new way of working and a new sort of insight and process that we're trying to help talent acquisition just get those insights without having to do too much work to see into them.

[00:27:50] And I think that is key with anything new.

[00:27:51] It's to this, to try and get access to it super easily.

[00:27:55] Same thing as we started out this conversation with AI.

[00:27:58] You know, AI has been around for decades.

[00:28:00] It's been very hard to access.

[00:28:02] Open AI, putting out GPT, because suddenly it became very easy to access last year and now everyone's doing it.

[00:28:08] We take the same sort of design ethos into the way in which we're providing tools to our customers in just trying to make that sort of data, that information super easy that if something has gone wrong in the process, we're trying to surface that.

[00:28:22] And we have a suite of sort of tools and analytics and insights that we're continually sort of looking at continuing improving to do so.

[00:28:31] But yes, it's harder when there is no uniform metric for quality of hire.

[00:28:37] It's harder when every customer measures sort of quality of hire or measures, you know, the success of an employee in a different way.

[00:28:46] But it's not impossible.

[00:28:47] It just takes a little bit more work on our part.

[00:28:50] Yeah, it wasn't instrumented.

[00:28:51] We now can quantify so much more and we've got the tools and the horsepower to sort of complement, you know, our own judgment and some of these more legacy tools, I suppose.

[00:29:03] So let's take advantage of some of these new capabilities.

[00:29:06] And as long as we're being responsible and we've got the right governance around some of these things, then, you know, we can use it with confidence.

[00:29:15] Absolutely.

[00:29:15] So just overall, I was just curious to get your thoughts about where you think, you know, things are moving with AI and HR.

[00:29:23] I mean, obviously, I know you're focused in the interview space within talent acquisition.

[00:29:28] But I mean, just I mean, you talk to, you know, clients and partners constantly.

[00:29:32] I mean, what's your sense of how things are moving and advancing?

[00:29:36] I mean, for me, we shouldn't underestimate sort of the power of this in the future.

[00:29:41] I actually think that artificial intelligence and emotional intelligence as well is part of what will be the fifth industrial revolution.

[00:29:47] You know, we've gone through kind of coal and gas and electronics and nuclear and then the Internet and renewable energy.

[00:29:53] And, you know, I would be amazed if, you know, the intelligence that we're going through now is not the fifth industrial revolution.

[00:30:03] So a lot of things that we just can't envisage now.

[00:30:06] I'll give you an example.

[00:30:07] If I thought even two years ago, certainly four years ago, whether I could see an AI assistant replacing human in an interview, you know, actually running an interview, you know, being able to answer the candidates questions, being able to ask intelligent questions or what have you.

[00:30:25] I would have said absolutely no way.

[00:30:28] Humans wouldn't accept it because, you know, they want to be part of it.

[00:30:32] And actually, you probably couldn't produce the technology to do it.

[00:30:35] Well, actually, that part I might have conceded that you could.

[00:30:40] Could I see that happening now in four years time?

[00:30:42] Yeah, I could see that happening.

[00:30:44] Absolutely.

[00:30:45] You know, I think that you're giving back time to the interviewer.

[00:30:48] The interviewer is probably getting the ability to assess the candidate in a way that they would assess it.

[00:30:54] The candidate could be getting what they wanted out of the interview in a fair and equitable way as well.

[00:30:59] So, yes, I could see that happening.

[00:31:01] So, I think that we, there are things that even a few years ago that you couldn't even fathom being a possibility that will be in only a few years time.

[00:31:11] Things are moving so quickly.

[00:31:13] In the more immediate term, though, particularly surrounding kind of interviewing, you know, right now we have areas where we are automatically training interviewers after an interview on bias.

[00:31:25] We're automatically surfacing the experience of the candidate when in the interview that experience sort of got better and got worse.

[00:31:32] You know, some really advanced, you know, capabilities that just help interviewers train themselves because nobody really goes through tons of interview training.

[00:31:41] And just sort of understanding how they give candidates a better experience.

[00:31:46] So, that's right here and right now, which a lot of people, when they see it, they are kind of shocked and amazed and awed, awed, awed, in awe about.

[00:31:55] The adoption of that will have to be sort of, you know, within enterprises and within organizations has to be thought about, about how you do that.

[00:32:04] So, I think that actually companies are lagging now, you know, in terms of their adoption, the technology that's already out there in the market.

[00:32:13] And AI councils and, you know, policymakers within companies around AI will have to, will have a heck of a job over the next few years in trying to catch up and trying to keep with all the new technology that's being developed.

[00:32:28] Because I think there's a big difference between what AI is out there and what ultimately companies are willing to use right now.

[00:32:35] Yeah, no, I think that's pretty good assessment.

[00:32:38] I do think about the people that want to talk to a human and how do you then bring that back together and make sure that you've got, you know, some consistency in the process.

[00:32:49] Because you do want to be as consistent as possible in terms of how you're evaluating, like you said, before asking different candidates the same question.

[00:32:59] So, you've got an easy way to sort of compare some of the responses.

[00:33:03] And I just wonder, you know, because some of the regulations say, I mean, even in New York City, which is not certainly the best piece of legislation.

[00:33:11] But one of the stipulations is you need to give candidates right in the application process an alternate path to submit their application.

[00:33:19] I don't know.

[00:33:20] I don't know if that's something that you can speak to that's happening already, or it's just something that we need to think about, you know, going forward that says I could be literally having and being interviewed by, you know, a bot of some sort.

[00:33:34] But, you know, I know someone else who interviewed and they.

[00:33:37] Hi, I'm Steven Rothberg.

[00:33:39] And I'm Jeanette Leeds.

[00:33:40] And together, we're the co-hosts of the High Volume Hiring Podcast.

[00:33:44] Are you involved in hiring dozens or even hundreds of employees a year?

[00:33:48] If so, you know that the typical sourcing tools, tactics, and strategies, they just don't scale.

[00:33:54] Yeah.

[00:33:54] Our bi-weekly podcast features news, tips, case studies, and interviews with the world's leading experts about the good, the bad, and the ugly when it comes to high volume hiring.

[00:34:05] Make sure to subscribe today.

[00:34:07] I spoke directly to a human and how to sort of reconcile, normalize that.

[00:34:11] We're not putting out there, you know, interview bots right now.

[00:34:15] And I think in regulatory terms, a fairly fast and fluid moving area right now.

[00:34:21] And, you know, we need to make sure that we're not using AI to actually assess a candidate's skills in interviews.

[00:34:29] We need to make sure that, you know, candidates are getting a fair chance.

[00:34:34] And I do, however, think that a human and a bot could interview side by side.

[00:34:39] You know, you can quite comfortably guide a human around, you know, the skills that they should be testing for.

[00:34:46] They can ask a variety of questions around those skills.

[00:34:48] And a bot can probably do the same thing.

[00:34:50] It could probably do it now.

[00:34:51] Or it could do it now.

[00:34:53] I know it could do it now.

[00:34:54] It's just not allowed to.

[00:34:55] I think that we have to think that and believe that.

[00:35:00] And we see that humans and assistants, bots, everything else are going to be working and living side by side more and more as we move into the future.

[00:35:12] I think that I look at the way my kids, who are 15 and 13, use technology.

[00:35:18] They're already used to it.

[00:35:19] They're growing up with it.

[00:35:21] You know, you look at college kids who take their phones into their lectures and their phones are transcribing, not just transcribing their lectures, making summaries for them so that they're doing their notes for them.

[00:35:32] And then they're producing exam questions or tests or whatever off that.

[00:35:38] And it's all happening, you know, almost in real time.

[00:35:41] So that not only is it helping, you know, save time and write notes and what have you, but it's also, you know, quizzing, you know, students around what they should be learning in those lessons.

[00:35:52] So, you know, just, you know, being able to work much closer, much more in sync with whatever technologies that you use to do your jobs is going to have to be a much more normal part of everyone's day-to-day lives, both in work and outside work as well.

[00:36:09] Yeah, no, I agree.

[00:36:10] As you were talking, I was picturing this scenario in my head where someone had literally created like a digital twin of themselves and that digital twin is interviewing with, you know, an AI interviewer.

[00:36:24] So now you've got AI interviewing AI.

[00:36:26] You could see a world in which that is happening and then you have to get your head around how do you actually assess whether the person is the right person or not.

[00:36:33] Let's hope that doesn't happen anytime soon.

[00:36:35] I think I'd like to retire before that does happen.

[00:36:38] Because I can't, I'm not sure I even, I can get my head around it just yet.

[00:36:42] Yeah, I don't know.

[00:36:43] I mean, somebody will do it just to make headlines, I think, right?

[00:36:47] Because the technology is definitely there to do it, whether, you know, it would pass muster or it would, you know, how quickly someone would catch it or whatever.

[00:36:57] I'm not sure.

[00:36:58] We see it would be a very base version of that, Bob, right now.

[00:37:01] We see interviewers being interviewed and they're using AI to, you know, live within the interview to come up with answers to questions.

[00:37:09] And it's not seamless yet because a human has to kind of take in a lot of information and repeat information and the like.

[00:37:16] But it's definitely happening.

[00:37:17] It's kind of interesting, you know, seeing that happen, seeing it play out and thinking where that might go in the future.

[00:37:25] I just think it's fascinating.

[00:37:27] And then everything you was talking about as far as AI and education is top of mind as well.

[00:37:33] I mean, my daughter's 16 and I have nieces in college.

[00:37:37] And yeah, I just think whether it's the use, you know, in school, making sure that they're actually learning, right?

[00:37:45] And then just being prepared for what's next.

[00:37:49] I mean, one's a senior, one's a junior.

[00:37:51] So it's not they've already had internships and like they got to be ready to roll.

[00:37:56] And so, you know, I want them to be comfortable, but I also want them to make sure that they're learning what they need to be learning.

[00:38:04] And they're not sort of outsourcing any of their critical thinking, just like I think using AI in a behavioral assessment is just beyond ridiculous.

[00:38:14] It should be abundantly clear without the words in front of you that we want to know how your human brain works.

[00:38:21] We know you're going to be working with AI.

[00:38:24] We expect you to work with AI as a co-pilot, but we don't expect AI to do your interviewing.

[00:38:31] The good news is, Bob, I don't think we need to worry about our kids.

[00:38:34] Kids have always helped the older generations with technology right back from when technology was really basic through the Internet era.

[00:38:40] And it will be the same through AI.

[00:38:42] So I actually don't worry too much about my kids.

[00:38:44] I worry about people like you and me more than them.

[00:38:47] Yeah, that's probably fair.

[00:38:49] It's funny.

[00:38:49] My daughter asked me for help with her Apple Watch yesterday.

[00:38:52] I'm like, you're kidding, right?

[00:38:54] Like, you know that you know more about that device than I do.

[00:38:58] So, yeah, it's pretty funny.

[00:38:59] So more generally, and whether it's, you know, personally or professionally, I'm curious what tools that you might be, you know, playing around with.

[00:39:09] Is there anything in particular that you're, you know, fascinated with or getting a lot of value from?

[00:39:13] Yeah, I'm not sure I have like tons of insight that other people haven't either mentioned on this podcast before or people are using.

[00:39:21] But, you know, I, you know, professionally, you know, in meetings, we use pillar, obviously, a lot through and I actually think that our sort of recruiting process is, you know, really sort of second to none.

[00:39:34] And we're really truly getting great, great candidates through.

[00:39:37] But then in meetings generally, you know, note takers and summarization and what have you.

[00:39:41] We can use pillar for some of that and more generally as well.

[00:39:46] Anything that needs to be produced from scratch, AI is a massive help.

[00:39:50] Whether that's research and whether that's writing, you know, from everything from, you know, blogs to posts to articles and what have you.

[00:40:01] But also within analytics as well, being able to, you know, just use AI to make sense of a ton of data and just surface it in a consumable way.

[00:40:12] I do.

[00:40:12] I use that a lot and use AI a lot to just help me in my day-to-day work.

[00:40:20] In my personal life, I know, I mean, you all use it all the time without knowing.

[00:40:25] You ask Siri or Alexa anything, you're using it.

[00:40:28] You know, so in your personal life, it's just, you know, various different tools that provide shortcuts and give you time back.

[00:40:34] The most important thing for me, you know, as you kind of reflect and start getting older is just how much time you have and how to make the most of the time you do have.

[00:40:44] So I look for any tools that will just give me time back in my day to do more productive things.

[00:40:49] But again, I don't think I have tons of insight that other people aren't probably playing around with or using at the moment.

[00:40:55] Yeah, no, that's fair.

[00:40:57] Yeah.

[00:40:57] I mean, just the fact that you're willing to use it, you know, sometimes it's just trial and error to see what's working or what's giving you the best, you know, output.

[00:41:07] Yeah.

[00:41:07] I mean, I've certainly played with four or five different tools just related to this podcast and then some others related to, you know, putting presentations together or doing my own market research and stuff like that.

[00:41:19] So, yeah, you just got to get out there and start playing with it, I think.

[00:41:24] Now's the time.

[00:41:25] Now's the time to find out, you know, what it can do well and what it can't and it will carry on improving.

[00:41:30] Yep.

[00:41:30] I agree.

[00:41:31] Just in terms of, I guess, just by extension, I'm curious because when I think of AIQ, I think that can be at multiple levels, right?

[00:41:41] It can be individuals, however we're personally using it, making ourselves more efficient and effective.

[00:41:46] It could be the team and it could be an organization, like how are organizations really embracing, you know, some of this technology?

[00:41:54] We talked a little bit about that earlier in terms of, you know, the adoption of some of these tools.

[00:41:59] Any high level advice for people thinking about, you know, upscaling beyond, you know, themselves or just any thoughts about, you know, growing your capabilities in terms of using?

[00:42:11] Yeah, look, and I think from a personal perspective, I think there's two different ways to probably think about it.

[00:42:18] You know, one from a personal perspective and the second from an organizational perspective of work.

[00:42:22] I think, you know, from a personal perspective, I would encourage everyone to just really grasp this and dive in.

[00:42:28] You know, fighting it or not being aware of it or trying to push it away is probably not the right way to progress into the future world.

[00:42:37] But it is coming at us whether we like it or not.

[00:42:40] So the more we understand about it, the more we read around it, the more we just get in there and use it, you know, in every part of our lives, I think is something I would encourage everyone to do.

[00:42:53] You know, just by, you know, using GPT every day, you know, typing into Google around, you know, whatever you're looking at AI, you know, look at AI around it.

[00:43:02] And it'll give you a sense for where it's going.

[00:43:05] From an organizational perspective, I actually think that a lot of companies are really struggling right now about how to, you know, consume AI, use AI day to day, what types of AI are allowed in the organization,

[00:43:17] but then also upskill their teams to be able to use AI to do their jobs better.

[00:43:24] So I think that on that, I would just be encourage people to dive in.

[00:43:30] I'm not part of a huge company anymore, though, that's going to fight back against me.

[00:43:34] So, you know, we obviously need to do it in a considered way and make sure that what you're using has the right checks and balances to make sure that you're doing your job in a very, very responsible way.

[00:43:47] But again, you know, take guidance from your organization about how they want you to use AI.

[00:43:52] And if there isn't any, then I think, you know, make sure that you use, you know.

[00:43:56] I get it. The podcast just isn't enough.

[00:44:00] That's all right. Head over to your favorite social app, search up work defined, WRK defined and connect with us.

[00:44:08] Common sense, fair assessment, make sure that you're doing everything, everything responsibly.

[00:44:12] So, but really, you know, now, now, now is the time to, to experiment.

[00:44:18] Couldn't agree more, Mark. It has been a pleasure. Thank you so much for coming on the show.

[00:44:23] Well, love, love the show. Thank you so much for having me. I really appreciate it.

[00:44:27] Thank you. Thank you. I really appreciate it. And yeah, thank you again. And best of luck to you and to the team at Pillar.

[00:44:34] Yeah. Thanks so much. Well, look, wishing you all the best. Thank you everyone for listening and look forward to hearing the next one as well. Thanks, Bob.

[00:44:41] Awesome. Thanks again, Mark.

[00:44:42] Thanks everyone for listening. See you next time.