Ep 17: Bias Mitigation for Better Hiring Decisions with Dr. Shiran Danoch

[00:00:00] Hey you with the podcast in your ear!

[00:00:02] Just a moment!

[00:00:03] Have you already activated your mobile happy hour?

[00:00:05] So you have an unlimited data volume every day for at least one hour.

[00:00:10] It's new and for everyone who is now getting mobile phone plus internet for home on the TV.

[00:00:16] Now Magenta 1 customers will activate the mobile happy hour in the My Magenta app

[00:00:21] and it's already off!

[00:00:22] On the 5G network of the TV.

[00:00:24] So don't wait, now get in and activate the My Magenta app of the TV.

[00:00:39] Hey everyone it's Bob.

[00:00:41] In this episode of Elevate Your AIQ I am joined by Dr. Shiran Danoch, CEO and founder of Inform

[00:00:47] Decisions and Interview Intelligence Platform to discuss the fascinating world of AI in the

[00:00:52] hiring process.

[00:00:53] We explore the importance of data driven and skills based hiring as well as the challenges

[00:00:56] and opportunities in using AI to mitigate bias and improve decision making.

[00:01:01] Shiran shares valuable insights on responsible AI implementation for favorite AI tools and

[00:01:07] of course offers advice on upskilling in this rapidly evolving field.

[00:01:11] Join us for an enlightening conversation that delves into the future of recruitment, how AI

[00:01:16] is reshaping the way we approach talent acquisition and management.

[00:01:19] I hope you enjoyed the discussion.

[00:01:21] Thanks for tuning in.

[00:01:23] Hello everyone and welcome to another episode of Elevate Your AIQ.

[00:01:27] I'm your host Bob Bolver with me today is Shiran Danoch from Inform Decisions.

[00:01:33] Hi Shiran, how are you?

[00:01:35] Hi!

[00:01:35] I'm good.

[00:01:37] Thanks for inviting me here.

[00:01:39] I'm very excited to be here Bob.

[00:01:41] Yeah, I'm glad to have you.

[00:01:42] Thanks so much for spending some time with me.

[00:01:45] As a way of introduction, why don't you just give our listeners a little bit of background

[00:01:50] about how you got into high psychology and people analytics and the HR tech space?

[00:01:55] Sure.

[00:01:57] So, I'm an organizational psychologist that actually went into the psychology world

[00:02:04] just from the basic desire to help people.

[00:02:09] That's what I knew, I think, from being very little.

[00:02:13] But what I didn't know is that I also love data and that there are so many aspects of

[00:02:21] the human capability that you can measure and assess.

[00:02:26] So learning psychology, my master's degree like that kind of opened up a whole new

[00:02:33] world.

[00:02:34] For me.

[00:02:35] So that's kind of, you know, it shaped already during my studies, but it became even stronger

[00:02:43] during my career, which I started as a consultant building customized assessments for

[00:02:50] different companies after several years of doing that and really owning on the

[00:02:58] expertise on how to design talent assessments, candidates, assessment selection processes.

[00:03:05] I did feel that something is missing for me.

[00:03:09] Doing consulting, you can create great impact but on a small, relatively small number of

[00:03:14] organizations and I felt, hey, I want to take my knowledge and I want to help companies

[00:03:22] at scale.

[00:03:23] And that's kind of when I started to understand that this could be done through

[00:03:28] technology.

[00:03:29] It was the early ages of machine learning and AI.

[00:03:33] It just started, but then I kind of understood the potential and kind of

[00:03:38] gradually made a transition from being a consultant to actually building

[00:03:43] products and technologies around people analytics and talent assessment.

[00:03:48] Excellent.

[00:03:49] So you had a startup before Informed Decisions, is that right?

[00:03:53] Right.

[00:03:54] Called Empirical.

[00:03:56] Empirical.

[00:03:56] Okay.

[00:03:57] And so this Informed Decisions is sort of an evolution of what you were

[00:04:01] doing with that or what do you see as the main sort of differences or the

[00:04:06] progression?

[00:04:07] Yeah, definitely.

[00:04:08] So Empirical was my first entry introduction into the technology world and

[00:04:15] all of the possibilities it withholds.

[00:04:18] And there we focused on pre-har assessment.

[00:04:22] So basically how to screening the best candidates based on

[00:04:27] relevant skills and building skills assessment, cognitive

[00:04:33] assessments, etc.

[00:04:35] My vision was always broader than that.

[00:04:38] As a consultant, the systems that I've built was always to incorporate data

[00:04:42] from multiple tools and multiple perspectives.

[00:04:45] It's not like I invented the wheel.

[00:04:47] There are more than 100 years of research around employee assessment

[00:04:52] and employee selection that says if you want to get an in-depth view and

[00:04:56] an accurate view of a candidate, you need to integrate multiple tools

[00:05:01] and multiple perspectives.

[00:05:03] And that's basically what we're doing at Informed Decisions.

[00:05:06] So at Informed Decisions, I'm taking the skills-based approach

[00:05:10] and basically applying it on the entire

[00:05:15] interview process, meaning how do you assess based on skills throughout

[00:05:20] the entire interview process, collect data, integrate data to make

[00:05:26] informed decisions.

[00:05:28] So you could use Informed Decisions at any phase of the hiring funnel, right?

[00:05:33] It's not just for on-site interviews.

[00:05:35] Yes, of course.

[00:05:36] At every stage, you should aim to, first of all, understand and clearly

[00:05:42] define what you're measuring, have clear questions or any types of

[00:05:49] assessment to assess what you're aiming to measure, the skills that

[00:05:53] you're aiming to measure, and the ability to quantify that.

[00:05:58] Because that will allow you as a talent professional or as a company

[00:06:04] to eventually be able to reverse engineer your entire process.

[00:06:09] You can't manage what you don't measure.

[00:06:12] You can't have the ability to actually understand what's working in your

[00:06:16] process and what's not working if your process is not quantified.

[00:06:22] So definitely it goes well beyond your interviews, starting from how you

[00:06:28] source candidates, from which sources are working better, convert

[00:06:32] better for you, the initial screening, whether it's done through

[00:06:38] pre-hire assessments or through a phone interview or through some kind

[00:06:42] of an AI algorithm.

[00:06:44] All of these components need to be broken down and can be broken down.

[00:06:49] It's much easier to break it down today when you can do a lot of

[00:06:55] these with technology and with AI.

[00:06:58] Yeah, absolutely.

[00:06:59] So what you're really describing is intelligence on the whole hiring funnel.

[00:07:04] Exactly.

[00:07:05] It's not necessarily just a 30 minute or hour conversation.

[00:07:09] It's how do you extract insights from each of those phases so that

[00:07:13] in aggregate, by the time you get to the end, you think you have

[00:07:16] the best view of that candidate as compared to other candidates.

[00:07:22] Exactly.

[00:07:23] And more than that, I think you can gain amazing insight just

[00:07:28] from looking at your hiring process data.

[00:07:31] But eventually, you have to close the feedback loop, meaning you

[00:07:36] have to compare your interview process data scores to actual job related KPIs.

[00:07:45] If it's retention, if it's performance, if it's engagement, whatever

[00:07:50] objective productivity measures that you have, that's the way

[00:07:56] that you close the feedback loop and actually understand which

[00:08:00] components of your interview process, whether it's kills,

[00:08:05] competencies, whatever you're assessing and also a whoever is assessing,

[00:08:11] which is actually what we're focusing on.

[00:08:13] The interviewers themselves, the decision makers, who is able to predict

[00:08:19] and not just because we want to tap them on the back.

[00:08:23] It's because we want to learn.

[00:08:24] We want to learn from what they're doing.

[00:08:27] Who are those that are actually hiring the most qualified and diverse candidates?

[00:08:32] And what are their interview behaviors that allows them to do that?

[00:08:37] And those who are still struggling, the ones that are not,

[00:08:42] that are missing the mark on the hiring.

[00:08:45] What are they doing?

[00:08:46] What are their biases that are hindering their efforts?

[00:08:49] And what feedback can we provide them in order in order to learn

[00:08:52] and improve for the next time?

[00:08:54] No, that's awesome.

[00:08:55] I think there's a lot of concern.

[00:08:57] Obviously people are concerned about bias in AI, but we can't, of course,

[00:09:01] ignore the long standing human biases and all the different human biases

[00:09:06] that could come into play when someone is asked to conduct an interview.

[00:09:11] And so if we dig deeper on that side of things, I mean, what kind

[00:09:16] of advice or guidance or coaching would you give to an interviewer?

[00:09:25] And how do you track improvements in their capability?

[00:09:33] Are you tracking from a performance management standpoint?

[00:09:36] Are you tracking each individual interviewer?

[00:09:39] I'll start with your first point where you mentioned a lot of companies.

[00:09:44] Like on the one side, they're very hyped about AI and how can we use it

[00:09:50] to streamline our processes to enhance the accuracy of our TA processes.

[00:09:56] On the other hand, there is a concern about the bias that goes inside.

[00:10:03] And it is a justified concern, but let's go back for a moment.

[00:10:09] Most of those AI algorithms are biased because they are trained

[00:10:16] on people decisions.

[00:10:19] Okay, if we're taking an algorithm and training them on recruiters

[00:10:24] or hiring managers' test decisions which are biased, and of course,

[00:10:28] the algorithms, decisions or recommendations will also be biased.

[00:10:33] That's why we need to go back to the source.

[00:10:36] If we want to unbiased our AI, we need to unbiased our people,

[00:10:41] our decision maker.

[00:10:43] And that requires an infrastructure.

[00:10:45] I know that a lot of companies are preoccupied with this

[00:10:50] and they're taking a lot of efforts in bias training,

[00:10:55] DEI training, etc.

[00:10:58] But we need to acknowledge that the key benefits of these types

[00:11:03] of training is raising awareness and it's not creating behavioral change.

[00:11:09] There is some research around that that shows.

[00:11:13] It helps to raise awareness, but eventually when I as a hiring manager

[00:11:19] I go and I do my interviews, I will probably default to my old ways.

[00:11:24] Not because I want to continue being biased.

[00:11:28] Before we move on, I need to let you know about my friend Mark Feffer

[00:11:32] and his show, People Tech.

[00:11:35] If you're looking for the latest on product development, marketing, funding,

[00:11:39] big deals happening in talent acquisition, HR, HCM.

[00:11:44] That's the show you need to listen to.

[00:11:47] Go to the work to find network, search up People Tech.

[00:11:50] Mark Feffer, you can find them anywhere.

[00:11:55] Most of us have good intention.

[00:11:57] We want to hire better people.

[00:11:58] We want to hire more diverse workforce.

[00:12:00] It's just because this is what I know and this is what I'm familiar with.

[00:12:05] So unless we introduce an infrastructure or a system

[00:12:10] that helps them to move from awareness to behavioral change,

[00:12:15] it will make it really hard for them to actually change.

[00:12:20] So my advice is not to the individual interviewer.

[00:12:25] I think individual interviewers, if they live within a context

[00:12:30] where they're not provided the infrastructure,

[00:12:33] they can do some things by themselves to create the infrastructure.

[00:12:37] But it will really be hard for them to create an ecosystem change.

[00:12:43] They might change their ways, which is a great step forward.

[00:12:47] But organizations need to think about it as an infrastructure

[00:12:54] that they need to provide.

[00:12:56] So they need to provide guidance both to recruiters and to hiring managers,

[00:13:00] which eventually it's not their day job.

[00:13:03] They're not experts in clearly defining what they want to assess

[00:13:07] and how to assess that, even if they do know what they need to assess.

[00:13:12] So that's the kind of infrastructure that we should provide them

[00:13:16] to provide them with the ability to standardize their process

[00:13:21] to assess accurately assess what they're aiming

[00:13:26] to assess, to provide them with data that they can learn from,

[00:13:30] learn from successes and failures and to improve.

[00:13:34] Because who can improve without feedback?

[00:13:38] Think about how do you improve in whatever you want to do?

[00:13:45] You want to lose weight, right?

[00:13:46] So you go on a diet or you start to exercise.

[00:13:50] You get feedback.

[00:13:52] You go on a scale you understand if you lost weight, right?

[00:13:56] Or now you can run for longer time periods.

[00:13:59] All of this is feedback that you're getting in order to understand

[00:14:03] if it works or it doesn't work.

[00:14:05] But this feedback rarely exists when we're talking about

[00:14:09] talent acquisition and hiring processes.

[00:14:13] So my advice, both to individual interviewers,

[00:14:17] but more organizations set up an infrastructure

[00:14:21] that will allow you to understand something, to collect data

[00:14:25] and to understand something about the effectiveness

[00:14:31] of your interview practices.

[00:14:34] What's the typical impetus for someone to decide that informed

[00:14:38] decisions is the right solution for them?

[00:14:41] I mean, are they not meeting DEI goals?

[00:14:45] Are they tracking quality of hire?

[00:14:48] And it's below what they expect?

[00:14:50] I mean, I imagine it's probably a range of things.

[00:14:53] But when someone comes to the realization that they need you.

[00:14:57] First of all, I wish the companies would have come to us after saying,

[00:15:02] hey, I've measured our quality of hire and it's low.

[00:15:07] Usually they don't have these types of measures

[00:15:09] or they have something very rough, but that's OK

[00:15:12] because we're here to help with that.

[00:15:16] Usually it's a sense of something is not working with my process.

[00:15:22] My managers are just a winning it.

[00:15:25] We have a lot of people living after a short time.

[00:15:29] Our candidate experiences of the process is very lengthy.

[00:15:33] People are living as I want to diversify.

[00:15:35] Like we're keeping on hiring more people that just look the same,

[00:15:40] sound the same.

[00:15:41] We're not able to diversify our workforce.

[00:15:44] So these are the types of voices that we're hearing.

[00:15:47] It's around eventually, yes, increasing the quality of hire,

[00:15:50] but more from an instinctive or an intuitive place,

[00:15:55] shortening the process or diversifying your workforce.

[00:16:00] Eventually there are basic principles.

[00:16:04] Again, that's something that we've invented,

[00:16:06] just something that we're applying at scale with our technology

[00:16:09] that they tell us these are the things that have been working

[00:16:13] across multiple contexts for years.

[00:16:17] And this is what you should be applying.

[00:16:20] Of course, every company, every position has their own

[00:16:26] a specific context.

[00:16:28] And that's exactly why we're measuring at that specific context

[00:16:33] and providing individual and customized recommendation.

[00:16:37] In terms of how AI is applied in your solution,

[00:16:43] is it helping with the guidance to the interviewer

[00:16:48] or in terms of feedback as well as what questions to ask

[00:16:53] based on, like you said, it's got to be specific to the role?

[00:16:57] Yeah.

[00:16:57] Is it also used in any type of scoring

[00:17:03] or you still rely on the human judgment

[00:17:05] to actually score the interview?

[00:17:08] That's a great question.

[00:17:09] The interviewee on that character.

[00:17:10] Yeah, that's a great question

[00:17:12] because it goes back to the fine balance

[00:17:16] that most of us are struggling with,

[00:17:19] on where should we put like the human emphasis

[00:17:22] versus where we should let AI or technology do the work.

[00:17:27] For us, we are a team of expert IOCycologists.

[00:17:30] Everyone that's working on my team has been trained

[00:17:34] and skilled to develop diagnostic questions

[00:17:37] in a specific context based on scientific principles.

[00:17:41] Sometimes it seems like asking questions is easy,

[00:17:45] but there is a whole science called psychometrics behind that.

[00:17:50] And that's the science of measuring the human capability.

[00:17:54] So that is something that we are not letting AI do.

[00:17:59] We definitely consult with AI or tweak our ideas.

[00:18:03] But in many times, what you're getting out of AI

[00:18:07] is more of the same.

[00:18:09] Again, it trains on existing data.

[00:18:11] So once you ask a chat GPT or another AI

[00:18:15] to generate interview questions for you,

[00:18:18] it trains based on existing data.

[00:18:21] And then a lot of what you're getting is what's already out there

[00:18:25] and candidates come well prepared today to interviews

[00:18:28] because everything is on Glassdoor, etc.

[00:18:31] So no, our team develops the questions

[00:18:34] and make sure that the questions are tailored

[00:18:36] to the specific position and organization

[00:18:39] and to the unique challenges of that position and organization.

[00:18:43] So that part is definitely the human part

[00:18:47] where the AI goes in is, first of all,

[00:18:53] to in detecting the different interviewing patterns

[00:18:57] that different people have.

[00:19:00] For example, the tendency to evaluate

[00:19:06] male higher than female on specific skills and vice versa.

[00:19:11] We actually see in the data that some skills are scored higher

[00:19:16] for male versus female and vice versa

[00:19:19] and also for different ethnicity groups.

[00:19:23] Monitoring is their specific times of the day

[00:19:26] where you as an interviewer tend to be more favorable

[00:19:30] words candidates.

[00:19:31] Also, we're seeing that some interviewers,

[00:19:34] they have what we call the time of day by.

[00:19:38] So when they're interviewing candidates in the mornings

[00:19:41] where when they're kind of fresh,

[00:19:43] though they're giving more favorable scores versus in the

[00:19:46] afternoon that the poor candidates that are being

[00:19:48] interviewed in the afternoons get significantly lower scores

[00:19:51] and that's consistent over time.

[00:19:53] It's not a one time thing.

[00:19:54] So that's where we use AI and also we transcribe

[00:19:59] the interviews, the ones that are being done,

[00:20:02] video conferencing and then transcribe it and summarize it.

[00:20:07] But not just a general interview summary,

[00:20:09] but a skills based interview summary that actually

[00:20:13] the same skills that you're assessing all throughout the interview

[00:20:17] you get an output that says these are the candidates

[00:20:20] strengths and points for growth in that same language

[00:20:24] by that same skills.

[00:20:26] So that's why we use AI as a tool for the interviewer

[00:20:26] So it's an example of the first step of the interview

[00:20:26] of the world of taxonomy, the interviewer's decision.

[00:20:30] It's the human decision.

[00:20:31] We amplify it with data, with insights, with feedback.

[00:20:36] But the decision is still a human decision.

[00:20:40] I think that's exactly how most of these solutions,

[00:20:43] Well, if not all of these solutions should be designed, right?

[00:20:47] It's you're putting the human first, you're keeping it human centric, but you have technology

[00:20:54] as part of a decision support system to make sure in part that you didn't overlook anything,

[00:21:03] but also maybe just to check your own potential biases, whether it's the time of day.

[00:21:09] That's really interesting observation that I hadn't really thought about.

[00:21:15] I know there are some interview scheduling tools that make sure you have, they do load

[00:21:22] balancing, right?

[00:21:23] So no interviewer is spending more than five hours a week doing interviews and you're

[00:21:30] sort of spreading it around and you're making sure that the right people who have been

[00:21:33] trained in certain topics or certain types of interviews are available depending on

[00:21:39] where someone is in the funnel.

[00:21:41] So there's a lot of logic that goes into that, but I like the tie to this concept of sort

[00:21:47] of the quantified self, right?

[00:21:48] Or if you don't measure it, can't know what and how to improve what you're working on

[00:21:55] or move towards the goals that you're trying to achieve.

[00:21:58] So it seems like there's incredibly valuable insight for the interviewer and the hiring team.

[00:22:06] Is there some output that goes to the candidate as well or this is purely for the hiring team

[00:22:13] to just make those decisions?

[00:22:15] Yeah, so currently we've made it really easy for the interviewers, the recruiters,

[00:22:22] the hiring managers to provide feedback based on output that we're giving them in

[00:22:27] the system because the output after every interview but also at the end of the interview

[00:22:32] process is something that we call the decision metrics that basically externalize in a very

[00:22:38] easy to understand way.

[00:22:40] What is the skill level?

[00:22:41] What are the strengths and what are the points for growth for each interviewer?

[00:22:46] So that is something that you can pick to the candidate and basically communicate to them.

[00:22:53] I will also say that I'm writing a lot of posts about it and like we're also

[00:22:59] recommending it to all of our clients, people that have taken the time to participate in your

[00:23:06] interview process.

[00:23:07] I'm not talking about maybe if it's just a phone screen or an initial assessment,

[00:23:13] okay, maybe you can send something written but if they have taken the time to participate,

[00:23:21] to interview, you at least owe them a call and some feedback about what they did well

[00:23:28] and also why did you decide to move forward with other candidates, not just the generic one.

[00:23:35] I will say that we are planning as part of our product roadmap to also provide something

[00:23:40] written but again, the technology human aspect of it.

[00:23:45] I don't think it's a substitute for a call where you're taking five minutes,

[00:23:50] ten minutes from your time to say thank you for taking the time.

[00:23:55] We really appreciate it.

[00:23:56] This is what we think are your strengths and you were really impressed by this,

[00:24:01] but you still need to improve upon one, two, three.

[00:24:04] I personally do it with all of our candidates and I find there are a bit that get defensive

[00:24:11] and kind of but most of them are super appreciative.

[00:24:14] It just gives them something that exactly the feedback group that we've talked about, right?

[00:24:19] So now they know how they can improve for their next interview experience

[00:24:23] and that's already something that they can work with.

[00:24:26] Yeah, I would say of all the times that I've been a candidate myself,

[00:24:30] I would greatly appreciate that type of feedback so that I know

[00:24:34] I at least have something to walk away with like sort of consolation prize, if you will.

[00:24:40] But yeah, anything that can help me do better next time

[00:24:43] because otherwise a candidate is completely in the dark

[00:24:46] and could have thought everything went well and they had a good

[00:24:49] they felt a good rapport and the interviews overall

[00:24:52] gave them a stronger sense that they were right fit for the role and then and then nothing.

[00:24:57] Yeah, right.

[00:24:58] So I think just an eye to candidate experience and respect

[00:25:03] for the candidate and the time to your point that they have sort of committed.

[00:25:08] Exactly. Why shouldn't it be a black box?

[00:25:11] You know why you've made the decision.

[00:25:13] It might be a hard to communicate, but at least some of it

[00:25:17] you can and you should it communicate.

[00:25:22] And another thing I'm actually using it to assess growth mindset

[00:25:26] with recurring candidates. What do I mean by that?

[00:25:29] I just had a case of someone that I've rejected several months ago

[00:25:34] or data analyst position on RT.

[00:25:38] I gave him the feedback and then another position opens several months later.

[00:25:45] He applied again.

[00:25:47] I interviewed him and he made such a significant clip.

[00:25:53] It was around communication capabilities.

[00:25:55] So I could actually see that he took my my feedback

[00:26:00] and applied it in our interview.

[00:26:02] And I said, this is amazing.

[00:26:05] Like you've made amazing progress and I'm happy to move forward with you to the next day.

[00:26:10] That is also a tool for companies in case of recurring candidates

[00:26:14] to actually see were they able to take your feedback and implement.

[00:26:18] What are your thoughts on what's happening in the space in terms of, you know,

[00:26:21] there's been a backlash with the way DEI policies and teams were created.

[00:26:26] There's legislation coming for all of this potential AI, you know,

[00:26:30] AI solutions in the hiring process.

[00:26:33] You know, just curious to get your general thoughts about

[00:26:36] how we're evolving in the space.

[00:26:37] Are we going in the right direction?

[00:26:38] Yeah, I will first talk about the positive things that I'm seeing.

[00:26:42] And then maybe a little bit.

[00:26:45] I don't know if to call it negative, but it is what it is.

[00:26:48] But first of all, I do see a general trend towards more

[00:26:53] tender dies and data driven hiring processes.

[00:26:56] A lot of talk around skills based hiring, structured interviews, et cetera.

[00:27:01] So this is something that I'm very happy to see happening.

[00:27:05] And also there is a lot of flourishing of an interview intelligence platform.

[00:27:09] We have a lot of competitors in our space and I'm happy about it

[00:27:13] because it shows that there is a need and that's an amazing thing.

[00:27:18] Also up until, I don't know, one year ago,

[00:27:21] DEI like it wasn't a dirty word.

[00:27:24] It was actually something people strive to achieve.

[00:27:27] I think it still is that way.

[00:27:29] But you know, there is a lot of backlash

[00:27:31] that kind of made people think what is actually the I

[00:27:35] and like a lot of other initials, the idea, the idea,

[00:27:41] now with SHRM doing the change inclusion for us.

[00:27:45] So there is a lot of wording and I feel.

[00:27:49] OK, I understand the essence of trying to make a difference with wording.

[00:27:54] But for me, the change is done through actions.

[00:28:00] So when a preplexed client comes to me and I can understand the confusion

[00:28:06] with, you know, with all of the buzz and the trends, the buzzwords, et cetera.

[00:28:11] What I try to bring it down to is eventually

[00:28:15] the essence or what are the basic principles?

[00:28:20] I'll give an example just to clarify.

[00:28:22] So for example, skills based hiring. OK.

[00:28:25] Also, I see some people like it.

[00:28:27] Some people say, oh, it's just it's just a trend.

[00:28:31] What is skills based hiring?

[00:28:32] It's knowing what you want to to measure in your hiring process,

[00:28:37] having focus and actually measure that and give more importance to that

[00:28:43] other than credentials.

[00:28:45] So there will be a lot of trends coming and going

[00:28:49] like this is the pace of the world that we're living in.

[00:28:52] And it's OK.

[00:28:54] Eventually what I try to ask myself and what I try to encourage others

[00:28:59] is to think about it in terms of what are the basic principles behind it?

[00:29:05] And do they serve?

[00:29:07] Do they serve you?

[00:29:08] Do they serve your goals?

[00:29:11] Ultimately, this is where I see responsibility,

[00:29:13] I specifically how you mitigate bias, how you make more fair

[00:29:18] decisions and the transparency that comes along with that to me.

[00:29:21] That can elevate all of these other,

[00:29:25] you know, somewhat controversial topics like the EI

[00:29:28] because if you can mitigate bias at a scalable level,

[00:29:34] then better decisions start to be made.

[00:29:38] Of course, human decisions,

[00:29:39] but human decisions augmented by AI and all the data

[00:29:43] and all the insight that can be derived from that brought to bear

[00:29:47] at the point of the decision for human beings.

[00:29:50] To me, if we embrace this and do it the right way,

[00:29:55] again, responsibly has more than just legislation,

[00:29:58] but certainly that is influencing people to some degree

[00:30:00] about how they govern the design and, you know,

[00:30:04] sort of construction of these solutions as well as how they're used.

[00:30:08] But to me, that lifts up all of these other more controversial

[00:30:11] subtopics because the data assuming the data is,

[00:30:16] you know, accurate and you've mitigated the bias in the data,

[00:30:20] then if someone still makes a biased decision, I mean, that's on them.

[00:30:25] But it just seems like you're not going to be successful with

[00:30:28] however you want to frame your DEI, DEIB initiatives.

[00:30:33] I don't see how you can be successful when AI will be used

[00:30:37] some fashion, maybe even throughout the hiring process.

[00:30:43] And it just seems like that's going to surface

[00:30:44] some of these areas that need improvement and have needed improvement.

[00:30:49] That's another infrastructure, right?

[00:30:52] Making sure that what we implement

[00:30:57] upholds a certain legal and ethical standard.

[00:31:01] And for me as someone that doesn't know anything about the law

[00:31:05] but still strives to be ethical, what is the framework?

[00:31:09] So if I have laws and legislations,

[00:31:11] if I have professionals like you that can guide me through it,

[00:31:17] if I have platforms like ours that can monitor the bias

[00:31:25] and gradually reduce the bias in our processes.

[00:31:28] All of these are infrastructures that eventually helps us

[00:31:33] to get to where we want to get to.

[00:31:35] And specifically in the context of possible AI,

[00:31:39] I would say it's about infrastructure for sure,

[00:31:43] but it's also about applying critical thinking.

[00:31:46] Because I think sometimes we see the capabilities,

[00:31:51] the growing capabilities of AI and we get blown away.

[00:31:54] We say, OK, wow, I have to use this.

[00:31:57] I have to implement this.

[00:31:59] Now, when you're an individual person sharing your own data,

[00:32:03] OK, get your own risk.

[00:32:05] But when you're sharing candidates data,

[00:32:08] when you're sharing your organization's data,

[00:32:12] at least you always to the organization, to yourself,

[00:32:15] to the candidates, to the hiring managers,

[00:32:17] all of the stakeholders involved to stop

[00:32:21] and just ask yourself some questions about

[00:32:25] how is this making recommendation?

[00:32:27] What goes behind the scenes?

[00:32:29] How are you using the data?

[00:32:30] What happens to the data once we finish our communication?

[00:32:34] There are so many questions that needs to be asked.

[00:32:37] And I really recommend in that context to read a book

[00:32:42] called The Algorithm by Hilke Schillman,

[00:32:46] which is an HR tech journalist

[00:32:49] that now kind of went and deep dive into HR technologies

[00:32:53] and wrote a book about it.

[00:32:54] It's an amazing book that basically investigates

[00:32:59] or kind of inquires the different technologies

[00:33:01] and how they're working and how they're making decisions.

[00:33:04] And it also has an appendix at the end

[00:33:06] for talent acquisition or HR professional.

[00:33:09] Now, when you choose an AI vendor,

[00:33:12] these are the things that you need to take into consideration.

[00:33:16] These are the questions that you should ask.

[00:33:18] So I highly recommend it.

[00:33:20] Absolutely agree.

[00:33:20] I actually was fortunate to meet Hilke a few weeks ago.

[00:33:24] I hope to have her on the show in the future.

[00:33:27] But yes, absolutely an important read

[00:33:30] for context about the broader space.

[00:33:32] And so, yeah, no, that's a great call out.

[00:33:36] As you think about AI and the fact that it's sort of on all our

[00:33:40] devices now or soon will be, certainly everyone has access to it now.

[00:33:45] I'm just curious more generally, you know, beyond your work,

[00:33:49] how you, you know, if you have any favorite tools or use cases

[00:33:51] for some of the AI tools that we've been seeing.

[00:33:56] I'm very curious to explore AI and what it can do.

[00:34:01] And I've been mainly experimenting with the church,

[00:34:05] you PTI assume they have, you know, since they came first,

[00:34:09] they kind of had that advantage, but also recently also with blood.

[00:34:14] So a lot of my use for it is around content writing

[00:34:19] because they do a lot of thought leadership on LinkedIn, on Forbes, etc.

[00:34:25] So it helps me brainstorm.

[00:34:27] It helps me refine my ideas and kind of take my more natural type

[00:34:33] of writing, which is kind of more, you know, I would say professional,

[00:34:38] maybe a little bit academic and turn it to something more engaging.

[00:34:42] So I think for writing content, it's amazing.

[00:34:46] Also, there are a lot of data analysis applications you can do on that.

[00:34:51] Of course, with being very aware of what you're putting in there

[00:34:55] and never putting anything that can identify anything.

[00:34:59] But there are a lot of data analysis and functionalities

[00:35:04] that actually just save so much human time

[00:35:09] in terms of, you know, calculation and integrating between files

[00:35:13] and just things that took us a lot of time to do before it came about.

[00:35:18] I would also say that for companies

[00:35:20] that currently don't have the resources to invest in interview

[00:35:25] intelligence or in hiring intelligence, I do encourage to use AI

[00:35:31] to generate interview questions, but not to just like you really

[00:35:38] need to kind of develop your prompt engineering there.

[00:35:41] Because like I said, initially what you'll get is standard

[00:35:45] questions, a lot of what's out there.

[00:35:46] Usually what I will tell ChagPT, when you develop a question,

[00:35:53] create some kind of a trade off.

[00:35:55] What do I mean?

[00:35:56] Like don't just ask when is the last time that you've achieved

[00:36:02] the goal or didn't achieve a goal.

[00:36:04] Now develop some kind of a dilemma where the candidate needs to decide

[00:36:10] or to trade off between being very process oriented to be a goal oriented.

[00:36:16] OK, so and then it creates a question that requires much more thought

[00:36:23] because there isn't a right and a wrong answer.

[00:36:25] All of a sudden you need to take multiple considerations into account.

[00:36:30] So you can use it to develop interview questions, but just tinker

[00:36:35] with it and challenge it in order for you to provide you with

[00:36:39] with more challenging interview questions and also scoring rubrics.

[00:36:43] scoring rubrics is something that GPT does a fairly well.

[00:36:49] Again, you should give it the exact definition of what you're aiming to assess.

[00:36:54] So that's I kind of went back again into the hiring context

[00:36:57] just to give something to hiring professionals.

[00:37:01] But wow, so many, so many possibilities really.

[00:37:06] It is amazing.

[00:37:07] Absolutely.

[00:37:08] And there's probably some people who have created some custom

[00:37:11] GPT specifically for interviews.

[00:37:13] I haven't checked out the catalog, but I'm sure they exist.

[00:37:16] Yeah, probably, probably there are.

[00:37:19] I did have one question about the future of interviewing

[00:37:23] and maybe it's not the future, maybe it's happening now.

[00:37:26] But have you seen people incorporating questions

[00:37:30] about candidates use of AI as part of the interview?

[00:37:34] Yeah, that's actually what we're doing now.

[00:37:38] So for our own hiring, I'll give a specific example.

[00:37:42] So we're hiring a data, a data analyst.

[00:37:45] So we give a professional give a professional task.

[00:37:48] Now what we're saying at the instructions is you are allowed to use

[00:37:54] AI or allows to use chat GPT, but tell us exactly where

[00:38:00] and how did you use chat GPT?

[00:38:02] Because for us, the ability to use it and to tweak your work based on it

[00:38:07] is an advantage.

[00:38:08] It's not a disadvantage.

[00:38:10] We don't look at it like cheating.

[00:38:12] It's another tool.

[00:38:13] Why won't you use it?

[00:38:14] Or we have another part of the professional task

[00:38:18] saying do something without AI, without GPT and now do it with GPT.

[00:38:24] And we want to see what is the difference.

[00:38:27] So definitely.

[00:38:28] And also for the processes that we're building

[00:38:32] to our clients, that's how we encourage them to look at it.

[00:38:37] To actually integrate it already as a requirement within the hiring process.

[00:38:43] That's a perfect answer.

[00:38:46] Because you're transparent with them, you're giving them explicit instructions.

[00:38:50] We know you're thinking about it.

[00:38:52] So yes, it's OK to use it.

[00:38:53] Just be transparent with us as to how you used it.

[00:38:57] So just anecdotally, my my niece was applying for an internship

[00:39:01] and she had to do an assignment.

[00:39:02] And so I didn't know about it until she was almost done with the assignment.

[00:39:05] But when I talked to my brother, I said, did she use AI for the assignment?

[00:39:13] And he was not keeping up with everything going on with AI at the time.

[00:39:17] And I said, let me show you this community assignment, put it in here,

[00:39:20] put in chat, GPT and take the output of that, put it in gamma and create slides.

[00:39:24] And he was just absolutely blown away.

[00:39:27] He's like, she spent like three or four days working on this.

[00:39:31] And it just did this.

[00:39:32] It's just did the entire assignment in less than 10 minutes.

[00:39:37] So I said she should submit and present the work that she did.

[00:39:42] But I would at some point acknowledge the fact

[00:39:45] that the technology has now advanced to the point

[00:39:48] where she didn't have to lose the weekend.

[00:39:51] Right.

[00:39:53] To get this done, I'm not saying you just have AI do it and submit it.

[00:39:58] But just to have that as your first draft and then just tweak it,

[00:40:05] you could have saved an incredible amount of time.

[00:40:08] And so it shows initiative.

[00:40:10] It shows honesty.

[00:40:11] It shows initiative.

[00:40:12] It shows creativity.

[00:40:14] And she got the internship.

[00:40:17] So so yeah, so she just showed at the end of the very last slide

[00:40:20] of her presentation, my understanding is she just had a side by side

[00:40:23] to your point.

[00:40:24] Here's what I did and all the time that it took each of these steps

[00:40:28] and then here's what I did in the time that saved.

[00:40:32] And my understanding is, you know, if you're a management consulting firm,

[00:40:37] that's exactly what you want to see because you're in the human capital business.

[00:40:42] It saves me time.

[00:40:44] It saves the company time.

[00:40:45] We're all becoming more efficient.

[00:40:47] I could do more work.

[00:40:48] I can bring more revenue to the company.

[00:40:51] Everybody wins as long as it doesn't jeopardize the quality.

[00:40:55] And then I get back to the critical thinking component.

[00:40:58] That's like when I talk to candidates, I tell them, like if you put your

[00:41:04] it's OK to generate your CV through a GPT, but review it.

[00:41:11] Make sure that it didn't didn't put there something strange or dishonest

[00:41:15] or just like completely amplified something that you did in a non realistic way.

[00:41:21] Be critical about it and make sure that you're in there, that you're represented

[00:41:27] in there and even more so in a professional task where eventually

[00:41:31] like your flavor, your creativity, your thinking needs to be there.

[00:41:36] So put it in there and then you can definitely tweak it with AI.

[00:41:41] Absolutely.

[00:41:42] So my final question to you, Sharon, is tied to what we just talked about.

[00:41:48] But when you when you hear that phrase, elevate your IQ, what's your advice?

[00:41:55] What do you think the keys are to everyone sort of upskilling themselves

[00:41:59] in this new technology?

[00:42:02] So first of all, be curious.

[00:42:04] OK, I don't think there is a person

[00:42:07] in the modern world that haven't heard

[00:42:09] Ched GPT or AI.

[00:42:13] Be curious, experiment with the different tools.

[00:42:18] There is so much out there that it is overwhelming.

[00:42:20] So I'm not saying go and experiment with all but experiment with one

[00:42:23] experiment with two in your specific for your specific needs and use cases.

[00:42:29] That's the way that you educate yourself.

[00:42:32] I think the key advantage here even even several years ago

[00:42:35] when you went into a new platform, you had to learn it.

[00:42:38] You had to study it.

[00:42:39] You needed to gain theoretical information before you were able to dive in.

[00:42:43] Today, you just go in.

[00:42:45] It's so intuitive.

[00:42:46] So just learn, experiment, play with it, learn from it

[00:42:51] and be critical about it.

[00:42:54] And that's the way that you can grow.

[00:42:58] I was about to say gradually grow.

[00:43:00] But I think what it allows it allows you to grow so quickly

[00:43:05] to grow your capabilities so quickly and it's so easy and quick to learn.

[00:43:10] So just do it.

[00:43:12] Excellent.

[00:43:13] Sharon, thank you so much for joining me today.

[00:43:15] I think there's a lot of takeaways from this.

[00:43:17] So I think the audience will really get a lot of knowledge

[00:43:22] and insight from this and some good advice.

[00:43:24] Thank you so much.

[00:43:25] Thank you so much for joining me.

[00:43:26] Thanks for having me.

[00:43:28] I really enjoyed it.

[00:43:29] Yeah, likewise.

[00:43:30] So thank you everyone for listening and we'll see you next time.