People Insights at HP & The Value of Data Security - Amy Stevenson - #176

[00:00:03] I agree with you. The part that's key is those relationships is actually how high trust information gets disseminated amongst the community. It doesn't feel like folks are just sharing some idea that maybe they haven't even tried out in practice and kind of falling prey to marketing and all of that.

[00:00:25] But, you know, I think you guys have been investing a lot into what it means to build a data lake and how do you work, not just with your HR counterparts, but with IT. Can you talk about that at all? Yeah, so I mean, I think there's a lot of flavors and no one size fits all, but for us it's been a bit of a journey. I mean, we definitely made the decision that we were a gold shop for various reasons.

[00:00:52] And, you know, you recently, you know, had a podcast trying to, you know, get back into keeping up with stuff. But anyway, it caught my eye and I wanted to kind of hook it here just because we have analytics leaders. So interviewing, oh my gosh, I just forgot our name. Is it Wilson? The Hydrogen Struggles, the code. Oh yeah, Jennifer. How do you think about data privacy as a part of your Gen. AI enterprise strategy?

[00:01:23] So I actually hooked on to something, another point you made though. Okay, go on, go to that point. So like, I do think security is separate from privacy. I mean, it's related but separate. Yeah, we try to use them interchangeably, but they're not. They are not, but it's a really important one and one that I think a lot of organizations struggle with. And here's the thing is I don't think it's a static, you know, decision and model. And this is what makes it so complex, right?

[00:01:52] So, um. Welcome to Directionally Correct, a people analytics podcast with your host, Cole Knapper, and today's guest, Amy Stevenson, Senior Director of People Insights at HP.

[00:02:22] Hey, Directionally Correct fans. This podcast is dedicated to you to help democratize people intelligence for the world of work. If you're looking to support the podcast, please make sure to listen weekly, subscribe to the Directionally Correct Substack newsletter, sign up for the Data Driven HR Academy at datadrivenhracademy.com, purchase Cole's book, People Analytics, or check out everything else at colnapper.com.

[00:02:49] Before we get into it, a quick word about HR Bench, the company powering this podcast. You know, when we all started in people analytics, we wanted to do strategic work. Building predictive models, workforce planning, advising the C-suite, and most of all, quantifying the impact for the business. Instead, we spend months building dashboards and reports that should already exist. HR Bench eliminates that entire phase.

[00:03:15] Your HRIS connects, your metrics calculate, your benchmarks populate. This is not novel. This is day one, not quarter two. That means skipping straight to prescriptive analysis, storytelling, and taking action for the business. Want to learn more? Book a demo at hrbench.com slash directionally correct. Find out more about the company powering this podcast and building the future of people intelligence.

[00:03:42] As always, all opinions are our own, and thanks for being a listener. Well, Amy, I'm so glad to have you back on the podcast. I have to do a little bit of a PSA before we get started because you were one of our first guests on the podcast and was actually the first episode where I feel like the podcast started to kind of blow up a little bit. Like it, it originally was just like we're on the friends and family plan, you might say.

[00:04:10] And eventually it kind of became something bigger than that. And I think you were really one of the early catalysts to that. But when we switched podcast providers recently, the first 50 episodes of the podcast got deleted. I didn't realize this would happen until your episode was gone. So I wanted to bring you back on and talk about all things, Amy Stevenson and all the good stuff. So I'm glad you're here. Well, thank you for having me back.

[00:04:37] It was a, it's a, it's, it's, it's a beautiful memory. It's getting a little more distant. How long has the podcast been a thing now? I mean, I'm losing track. It's, I mean, it's kind of crazy. It's definitely been four years, maybe almost five. I don't know. I don't remember exactly when we started, but we're 170 something episodes in. So it's quite a lot. It's so great to see. And, and I mean, it just is like the, that just sort of is mind blowing. But I mean, anyway, yes. Thank you.

[00:05:05] I mean, I'm happy to be back and sorry you lost early episodes. Cause I do remember in the early days, I was keeping up a little bit more than I do now. It's a little bit more like bits and bobs, but I do remember you had some amazing guests. So the fact that you're kind of coming back through the archives and, and inviting some of those old hands back is, is really nice. So I'm happy to be one of those. Thank you.

[00:05:29] We love our old hands, you know, since then, because we had that one, we recorded at PSYOP that you were. That's right. That's right. The only time we've been on, but yeah. And I think that one made the cut. I want to say that was after our first 50 episodes. I had to check to figure that out, but I think that one still exists somewhere out there, but. It's all a blur. My, my, we've come a long way since Anacoco and Louisiana tech. Have we not? I mean, yeah.

[00:05:56] Look at you published author, like, you know, just. I'm saying more look at you. Oh, no. I mean, I seriously, no, it's great. It's an opportunity to say, you know, someone who's been your friend for some years now. Um, and certainly you followed your career. It's, it's, uh, it's amazing to see and really proud of everything that you've established and how much of a, you know, resource you are for the community. Um, through, through these mechanisms, right?

[00:06:25] The podcast, the book, but like personally you show up, you're at conferences, you go to meetups, you're available to people on the phone, you know, in LinkedIn. And so I think that should be celebrated here because, uh, you you're pretty awesome, sir. Well, Amy, I'm here to like, try to compliment you today and you keep throwing it right back at me. So I'm going to throw it right back at you and see if it sticks this time. Okay.

[00:06:50] So you are my oldest friend in IO psychology, which eventually led into this whole people analytics space that we're a part of. Yeah. And I feel like, so I was just some kid that grew up in, you know, rural North Louisiana and hadn't been anywhere. And then in the graduate program at Louisiana tech, there was a student that came in who was so worldly. They had been everywhere. They had done everything. They had worked for amazing organizations and had done really cool stuff.

[00:07:19] And I got to befriend you. And gosh, I learned so much from you. Just like how to have a perspective beyond just kind of like my own little myopic existence. And so I just like, if there were a juncture in my life that sent me from being in one direction to another, I feel like you were a big catalyst for that. So I wanted to say thank you for that, Amy. Well, thank you. It makes me feel very, um, like, uh, I do feel a little bit like an older sister to you.

[00:07:47] So like, hopefully that's, that's why, but no, thank you. I appreciate that. I mean, it's funny. You say that you came from rural Louisiana. If you came from rural Louisiana, I came from like, rural Louisiana. You came from, yes. Where if it were a competition, you would win for sure. I would win that one, but, but I love it. I mean, you know, that's part of our history and story and, um, you know, it's all part of the fabric of who we are and, you know, where we, where we come from and where we go. And so it's amazing.

[00:08:16] I'm so glad our paths crossed and they continue to do so. Right. I mean, that's the beauty of it. Absolutely. Well, what are you up to, to now? I think so. Obviously I know what you've been up to, but what, like, uh, you know, I think you were at HP last time you came on the podcast, I believe, but what were you there then? Or were you still at AIG? Honestly, I can't remember. I was at AIG. It was between, it was in that inflection point. I'm not sure.

[00:08:43] I mean, I want to say so, but like, yeah, I've been at HP for a little over five years now, so probably. Um, and yeah, it's been, it's been an amazing journey and a ride here, you know, coming in and helping build the, um, the people insights group from, from the ground up. Um, you know, having done that a couple of times before in previous organizations, um, it's been, it's been great. It's been different. You know, I came in and of course it was that crazy time for everyone. It was like COVID.

[00:09:12] Yep. And so we were all just sort of figuring it out and, and, you know, so have, you know, to have weathered that and figured out how to continue to build and scale a team, um, you know, starting it and grounding it and that kind of frame. And then, you know, making sure that it had legs enough to, to stand and grow and scale, um, has been a great challenge and, and I've really loved it. I mean, um, I'd say that, you know, I've done this before, but in different areas. And I think that's a great, um, I've been working in a different industry.

[00:09:41] So it's always like a new challenge to do a build, um, somewhere new, different organizational context, different industry. And that certainly was the case here. It's been amazing. I felt very, very supported. Um, you know, I had a little bit of, uh, I'd say wisdom coming into interviewing for, for the role. And one of the things I did was lay out like, Hey, this is what a, like at this point in time, here's what I think a three and a five year plan looks like.

[00:10:09] And that's the, the, it's, it's kind of funny. It's ironic. Looking back on it, what I put on paper, we've actually like achieved and has been supported, right. By the organization. But the reality is, is the organizational and the external context changed so dramatically. Some of the things that was informing that were informing that plan on paper for me now, actually this week, I've been here with my leadership team saying we're ripping it up and starting a new, like no assumptions, right?

[00:10:38] Like everything that we have is great. How do we break it down and build it back? And that's a really interesting place to be in having gone through a cycle and then being able to kind of say like everything aside, if we completely tear up our structure, how do we need to change it? And so that's, that's, that's pretty cool. That is cool. And I think about, we had a recent episode with Gary Russo at Providence Health.

[00:11:05] He had talked about, he had been in his role for about 10 years. And one of the things that we discussed is that as like a lot of people analytics folks, no one would ever point a finger at me, right? Plinko around to different roles and don't, when they make a three or five year plan, very few of them are actually there to see it come to fruition. But you, you've been able to do that at HP. Do you think we're able to accomplish more as a consequence of that and kind of seeing the

[00:11:32] plans through and, and frankly, like do many of them still hold up? Like does it, you know, has, has the world changed so much with AI or everything post pandemic that's happened? Like how much of the plan actually holds up? It's interesting. Cause like part of the plan is, you know, you're, you're, you're turning a course for what you know to be true at any juncture. Right. And the world has changed quite a lot and technology as well, but some fundamentals stand, right?

[00:12:01] You have to make some determinations for your organization, the culture and different things that are at hand about whether you're going to go, for example, there's a whole like build versus by debate. And so like, that's a very foundational like direction, right? That you might go. And that does inform foundationally where, you know, you need to be thinking about, you know, structure, like skills, head count, those sorts of things. So you do have to budget and account for that.

[00:12:28] And typically most people don't get like a free plate of, oh, okay, you're going to decide to do one versus the other. Here's everything you need year one. And so I would say that, that, that, that decision is important for an analytics leader. And, you know, even if you inherit an organization, sometimes you have to take a look and give credence to the thought and, and weight that went into that.

[00:12:55] For, from a planning perspective, you do have to be nimble enough. Right. I mean, given none of us saw kind of the proliferation of AI and kind of the force and velocity of all that. And so you have to be able to figure out, okay, what's the new normal and how does this fit the context that I have? Like the writ large, right. Um, whether it's by versus build and all those sorts of things. I think generally speaking, the, the one thing I would say is like the stickiness that you

[00:13:21] have, if you, if you, if you sink your teeth in and you're able to make it and, and, you know, kind of from a tenure perspective and organization, what it does give you is kind of a, um, a sixth sense. I think about how to navigate and change course in, you know, like more nimbly, right. In, in the context of that organization, in the context of that industry. Um, and so there is some, I think value, right.

[00:13:51] Um, in, in having been there, you establish relationships and credibility. I mean, so even if like the plan on paper has to pivot, like, you know, pretty substantially, if you have credibility and those relationships in the organization you're established, I think that there's, um, more support for you needing to make those pivots in real time because of that, right. The relational aspect can't be underestimated, frankly. That makes a lot of sense.

[00:14:18] But I think like what role does the intellectual property that you kind of create in this process play a role in? Like, how does that make a difference? Because like, presumably, like one of the challenges I have bringing people on the podcast is everybody's like, well, I can talk about my work in generic terms. But I can't be very specific because everybody's trying to create kind of a competitive advantage for their firm using these types of tools and analytics.

[00:14:49] Um, but like, how do you know, you, you mentioned build versus buy. Obviously if you're buying, you know, that's not your IP, that's somebody else's, right. You build in house, that is your IP, but also you incur the cost associated with, you know, doing it in house. And, and frankly, sometimes it takes longer and so on and so forth. Um, but how do you know what, what IP that you have that's worth protecting or worth investing in versus, you know, something that you can just get off the shelf? It's a good question.

[00:15:19] And, you know, it's one that we have certainly, um, grappled with and navigated. Um, I would say again, relationships and I mean, I don't mean to harp on it, but they are really important here because if you are, um, if you have a safe space to sort of pressure test, you think you have something really great, right. And you're like, we built this thing. We think it has legs, right. We, we think that there may be an IP play here.

[00:15:47] Um, there are safe spaces to do that, whether it's through your consortium memberships, right. With, you know, sort of rules that help govern. We've had, you know, conference board, I for CP we've had, I'm forgetting some, but we've had probably four or five of them. Right. Whether it's even if it's in a more, um, you know, pointed space, whether it's listening or what have you, right. There are safe communities where you can go out to, um, peers and pressure test that and say, I think I have something really good here. Can you like level set with me?

[00:16:16] Because there is a tendency, you know, you're so inwardly focused, like, you know, of course what we're building is amazing. And, but you know, sometimes you need a little bit of a reality chat. Um, and then too, there's just, I mean, it's a very complex process to navigate, right? Like, so even if you get that external validation that, yeah, we think you should go for this, that doesn't necessarily mean that it meets the merits of, um, the requirements of that process for your respective organization.

[00:16:40] I mean, it's, it's a very, um, little more lengthy process typically speaking. So, but I do think the one thing I would say is like, I don't think that that should be something that, um, I think it's something we should sort of challenge each other as analytics leaders to do like actively, because I think there's amazing stuff being developed out there. And people either don't know or don't feel confident that it's like maybe good enough.

[00:17:08] And you don't know unless you, you know, float that balloon and, and, and try and have the conversations. So I think it's something that we as a community should be, um, discussing more, um, more in whatever safe fashion that we can do it. Right. We have to figure it, figure that out. Well, and I agree with you. Like the part that's key is those relationships is actually how information, high trust information gets disseminated amongst the community.

[00:17:35] And so it doesn't feel like, you know, folks are just sharing some idea that maybe they haven't even tried out in practice, um, and kind of falling prey to marketing and all of that. Um, but you know, I think you guys have been investing a lot into what it means to build a data lake and how do you work not just with your HR counterparts, but with it. And can you talk about that at all? Yeah.

[00:18:02] So, I mean, I, and I, I think there's a lot of flavors and no one size fits all, but for us, it's been a bit of a journey. I mean, we, we definitely made the decision that we were a gold shop, um, for, for various reasons. And, um, it was clear to me early on that, um, you know, I've known there was, there's a, there was a future state where as HR, we have enjoyed, I will use that term loosely,

[00:18:27] but, um, the, uh, we've enjoyed being walled off right from a data perspective. And there's a lot of reasons for that to keep that contained and sort of very, um, very locked down. Um, but when you're looking at certainly like complex, you know, enterprise wide initiatives, we, we see, you know, the shift toward a need for partnership with our inner enterprise partners, whether it's it finance and others, right.

[00:18:56] Everybody talks about like wanting to use these other data sets, but what does that really look like practically? Um, and for an organization as large as like HP, um, it, you know, you just, you do have to partner. You have to understand what is the roadmap and the strategy for the broader, um, technology, you know, organization, innovation, organization, strategy, organization. You have to figure out how do I fit into this and how am I helping make sure that our strategy and roadmap is, um, comporting with that. Right.

[00:19:26] Um, not that you're necessarily saying, Oh, we're going to jump, jump into that, but that you're like at least aware enough of what's going on such that you can, um, you know, chart a course for the, your HR people organization. You know, you can't, you need to be prepared if there's a situation where you're told that you're coming into a federated data lake.

[00:19:49] Like if you have to scramble and figure that out, you know, when you're informed, you're not going to be in a really great place. Right. So you need to kind of do some scenario modeling and planning around that and figuring out like what happens if, and what decisions that we might make now might make that easier or harder. Or, I mean, there's just this kind of conversations that you need to be having as a leadership team. But yeah, that has to, I mean, if you make the commitment and you, you have the right

[00:20:18] model internally, again, there is no one size fits all, but you have the right model for your organization. That has to set you up well for trying to deploy AI, right? Yeah. And what's interesting is again, there's, there's different, I mean, ways that organizations are approaching this. And I think how AI strategy and responsibilities get segmented or seeded within an organization is also just fascinating to, to, to watch how different organizations are doing it.

[00:20:48] And so, you know, I think there is a lot of value in a model where there's oversight for AI for functional areas, right. As well as kind of centrally for the organization or, or major business portfolio, because it does allow for, there's very unique needs, especially when it comes to like HR again, you know, bias, right. But just given the sensitivity of the data and some of the privacy and, you know, works

[00:21:17] counsel things that we have to typically navigate, it makes a lot of sense to have someone that's rooted as an expert in AI, but in HR, right. And, and helping, you know, sort of bridge that with the central, you know, whatever group it is in any organization, the people that own AI strategy for an enterprise. Um, yeah, and it's early days. I think people are testing and trying out different models. I mean, obviously I think a lot of people have been reading the news and how Microsoft

[00:21:45] has been being, been re-swizzling their HR organization to help kind of like think about these things, um, like critically. Um, and so I, I'm really fascinated by this and, and, and, you know, we're, we're early days with that too, but figuring out what, what's the right model for, for engagement and what's the right level of, um, governance that you as a function have. Feeding into an enterprise AI strategy. Well, and here's a part of this that, um, and feel free not to comment on this.

[00:22:15] I can, I can say something, you don't have to say this. Um, I think there's this notion out there that if you have an enterprise AI strategy, why do you need to have a carve out for HR? And, and because if you can have finance data in there and you can have ops data, you can have sales data or whatever kind of data your organization has. And you just have kind of like, let's call it this overarching bot that can answer any

[00:22:42] question that anybody has at any time in creating a frictionless experience. That's obviously the right end state. We should all be approaching. And the answer to that is no, we've never been able to give people access to all the financials of a company or all the sales data of a company or all the data about employees within some kind of nested permission structure, privacy, all of those types of things.

[00:23:10] So I'll bring you back in here again. You don't have to comment on that, but how do you think about data privacy as a part of your gen AI enterprise strategy? So I actually hooked onto something, another point you made though. Okay. Go on, go, go to that point. So like, I do think security is separate from privacy. I mean, it's related, but separate that there's. We kind of use them interchangeably, but they're not.

[00:23:38] They are not, but it's a really important one. And one that I think a lot of organizations struggle with. And here's the thing is I don't think it's a static, you know, decision and model, and this is what makes it so complex. Right. So I'll use a general example. Right. So for HR business partners, right, different organizations approaches differently about latitude of access to data.

[00:24:04] And, you know, that, that can, is really complex when you're securing and provisioning access to analytics and reporting right on the back end. It also changes really frequently. So like a business partner may be shifted around. They may do a hat. They may support, for example, like a country or set of it, or maybe both. Right. Right. And so I think the security piece is like, I'm glad you hit on it because to me, that's

[00:24:32] one of the pieces that's like the bigger struggle bus right now is, um, how do you think about data management strategy? And sustainability and stability with that when something like the, the data access piece is so, so fluid, right? Like based on role permissions. It is a huge, um, consideration.

[00:24:55] And for those organizations that have to support multiple provisioning systems, multiple platforms and systems, it, you know, this is where it kind of comes home to roost sometimes as what's reporting and analytics groups is like, okay, you got all the data from all the different systems and platforms. They're all using different, um, provisioning, you know, role based security, figure it out. And then that's where you, the partnership with privacy comes in. And so there is a hook there, right?

[00:25:25] It's like, okay, what should people have based on their job and role? Why? And the reality is, is that also isn't an easy answer because if you're looking at it from a siloed manner, it makes sense to give person X, Y, Z something in a learning system. But as soon as you start intersecting learning with listening, that changes the calculus. Right.

[00:25:47] And so I think where I'm going with this call is again, bias, but HR data is so much more complex in ways I think than other data, um, in the organization. And so it's boiled down sometimes with enterprise groups. Like it's just another data. It's just another group of data. It's a what, but the transactional nature of worker data and then bumping it up dimensionally against other data sets.

[00:26:13] I, you know, again, I just think there is a place for that carve out from a privacy perspective, from a data security perspective. There is an SME view of that, that, that I don't know exists. Yeah. Well, and I mean, I love the points that you made about security and privacy, but there's also just like you said about complexity. I can't tell you how many times I've worked with different data engineers at different

[00:26:42] organizations who came from areas outside of HR. And they were just like, ha ha ha dealing with this HR data is going to be a cakewalk. And because they're like, because we have all this it data figured out or all this, whatever other data figured out. And then they come into HR and they realize like this is a nightmare. This might be the hardest thing to deal with of all. Mm hmm. Not to mention above the complexity, you have the security and privacy considerations that

[00:27:12] in all the role based things that you mentioned a second ago. And so it becomes if you're a person who actually likes like a challenge, it becomes like almost someone's life's work to figure this out for an organization. But what talk to me about because presumably if you're if you're creating some kind of applications for people to use data, you're trying to make business decisions, you're trying to tackle different use cases, you're trying to, you know, solve some kind of problem that the organization is having.

[00:27:41] How do you navigate the trade offs between complexity, security, privacy and use cases, productivity gains and net value add from this type of work? I mean, it's it's it's always a consideration. Right. And I mean, you know, you talk about trade offs, but at the end of the day, we don't compromise on, you know, our principles.

[00:28:07] We have very, very strong relationships and guidance and guardrails, if you will, from legal privacy and others. And so like we we we come at it from our default is we adhere to those and then we figure out how we can creatively solution for our internal clients and the use cases that they have. But it is it's it's really.

[00:28:31] Again, I think the the inflection point is around like increased intersectionality of data sets and like the need to do that and build analytics on top of it or reporting on top of it. Um, it's a daily challenge. And I mean, I can't tell you we have a wholesale approach to like, oh, if it merits this, this is this. I mean, there is a discussion because sometimes with the leadership, you you have a conversation and it's like, here's everything. Here's the guidance.

[00:29:00] But there is a bona fide like business justification. And there is a, you know, there's a process to work through that to understand like what's an acceptable risk level. And like it doesn't mean that there's no risk because reality is, is there's risk in stuff we do. Right. I think that needs to be acknowledged. There's very little like no or low risk stuff that we do. Um, but if you come at it from an ethical and grounded perspective with the guard rules that you've been given privacy, legal, cyber and so forth.

[00:29:29] Um, you know, I feel very confident that we're ending on solutions that, you know, um, that are meeting the need of the business. The, the issue ends up being cold more than often isn't what we're delivering. It's the timelines, right? Because when you talk about all these pieces that come into play, I think that's one of the killers is how fast can all this get done?

[00:29:56] And like the more that you can work with your partners to get kind of standard operating like parameters, the better position you are as an analytics group to deliver rapidly against, you know, at least we're operating within these confines. That's the tricky space. Well, then there's another flip side to it, which is obviously you're creating value. How do you make sure that that value gets credited to HR and the contributions that you guys are making? Yeah. I don't know.

[00:30:26] There's this like, you know, saying that you're, you know, um, that your successes are private and your failure years are very public. And I think that that, um, no, no, never. Yeah, no, I mean, but I do think, but, but I think too, if you ask a lot of analytics professionals, they're like, of course you want recognition for the work you do. Right. And you want it to be more than anything, helping the organization.

[00:30:54] I think nine times out of 10, no one's gonna like, you know, um, argue that, but we also operate in a space where we, we kind of operate under the radar a lot. Right. We're entrusted with, um, access and use of data that is, you know, very. It's plumbing sometimes. Like it's truly the plumbing that makes the world go round.

[00:31:16] Exactly. And it's, and it's silent. Right. But, but, um, I think because of that, you'll, you'll see a tendency to maybe not over, over react to, um, not getting like blasted as the architect of like the most amazing thing. But I do think that the, the flip side of that for us as leaders is that, um, you know, we need to find out the ways to recognize that and elevate that work right within our organization to our leadership and frankly, outside of HR.

[00:31:45] Um, and you know, that, that can sometimes get lost in the weeds. What I will tell you is this is where building good relationships with your business partners is really integral because they are that frontline outside of HR. Um, and you know, if you have a good relationship, look, we've, we've had business partners that aren't shy about saying like, Hey, this group, look what they did for us. Right. Um, but it's, you know, I, again, it all comes back to relationships.

[00:32:14] I keep saying that and I'm like harping, but I do think it's really, really important because it's easy to like seed that work to the grind. And this is why, you know, I think as leaders, and especially if you're a leader that happens to have an arm that is client facing, um, certainly with the business community, but also within HR, how important that group is to your like branding, to your adoption, to your, like all of that. Right.

[00:32:45] Yeah. I mean, that's, that's one thing I have to give you credit for that I learned from you is the importance of relationships. Another thing that, uh, I always find so funny when I think back is like, I'd never heard the words like acumen or like remit until I met you. And these are such like corporate jargony words, but like, I was like, well, I mean, Amy taught me all these things. And so now I know some of the jargon because of you.

[00:33:09] So it's, yeah, I'm kind of known for saying, I maybe need to throw fewer sayings around, but yeah, I mean, um, but no, I mean, these are great questions, Cole. I mean, they're, um, you know,

[00:33:21] I, uh, your, your question was about how do we make sure we get credit for stuff. And I, you know, it's tricky because the reality is, is oftentimes we're a work stream in a complex, like enterprise wide initiative. Right. So let's say there's, you know, something that HR is doing from a transformational effort for the enterprise.

[00:33:45] We're a critical partner, but we're not like the only show. Right. I mean, you've got different parties, they're leading work streams. And so it's just making sure that you're showing up and, and ensuring that your work is getting the due credit because it is an enablement, huge enablement. Yeah. I can't remember who said it, but there's famous quote, like victory has a thousand fathers, but failure is an orphan. That one too. Yeah.

[00:34:13] Yeah. I think it's like, I'll have to use that one. But, uh, well, Amy, last time you were on, I don't believe the nerdery nor Cole's corner ever existed. Would you like to join me in Cole's corner for the first time? Oh, um, I think there was the nerdery cause it was you and Scott, right? Yeah. Back then. But, uh, but yeah, I mean, I'm kind of scared, but I'll try. Welcome to Cole's corner. Right.

[00:34:43] Well, you know, no need to be scared. It's a lot of fun, but, uh, let's start out with some rapid fire. Is it a poll? Do I get to push a button? Um, we could try it. Well, no, no, there's no poll. I was trying to think how I can spin that up on the fly really quickly, but no. Okay. Um, so the first question I got for you is if you weren't in the field of people analytics, what would you have done with your career? Oh, two. And they're really, really different. And I think you're going to.

[00:35:12] Okay. I would have either, um, made a, um, career in the military. Mm-hmm. Um, or. Doing what? Oh, um, probably like Jag or something like that. Okay. Um, or I would have been, um, an interior designer. You do have good taste, so it makes sense.

[00:35:37] That's like my creative outlet. Like I don't have a lot. I can't paint. I can't, you know, uh, a lot of the other creative arts, but like that one, I really do enjoy it because it's the intersection. Here's why. Design, the aesthetic aspect, but, um, the intersection with like architecture and like it, there is a like kind of a more technical intersection that your engineering build stuff. Absolutely.

[00:36:04] Well, you're a pretty well-traveled person, so this one will probably be a challenge, but what's the place you've never been to that you'd most like to go and why? Oh my goodness. Um, you know, actually this one's actually pretty easy. I, I'm blessed that I've been able to travel a lot, but I have never made it to Australia. And I want to, I, I'm a big snorkeler, diver, Great Barrier Reef, Australia. Yeah. I remember, um, when I was a kid, I said I wanted to be a marine biologist and go study the Great Barrier Reef. I didn't know how to track.

[00:36:32] And how did you know, though? Look at me now. I'm such a failure. Oh no. Um, if you were a character in any book, TV show or movie, who would you be and why? Oh my gosh. Oh, that's a good one. Um, ooh, my, my kids would love this young Sheldon. Young Sheldon. Why young Sheldon?

[00:36:59] I don't know. I love that he's like so intelligent, but awkward at the same time and like relatable. I don't know. There's something about it. I think he's hilarious, but, and smart and awkward and, um, just lovable. That is, now you're, you're refreshing my memory. We did cover this, I think last time. Didn't you go to college and you're like 13 years old or something like that? You were like a prodigy of sorts. A little, a little earlier, a little later. Sorry. Um, I was, I was late. I was older than him.

[00:37:26] Oh yeah. Just, just a tad older. Just, you know, I think you graduated with most school. Not quite in the same, same, same realm, but yeah, my kids love that show. So they would love that I, I offered that one up. But I do think that it's, it's very endearing. I think so. Absolutely. I've got one big question for you. Okay.

[00:37:46] We started out talking a little bit about bill versus buy and you've, uh, talked before about, you know, creating a product based model to doing people analytics. What happens to the product based model in the age of AI? Does it stay the same? Do you just spin up even more features? What happens to product based people analytics with AI?

[00:38:10] So in our case, and again, this is so timely because I've been with my leadership team all week kind of doing strategy planning. And this has been a topic of conversation actually. Um, I think in our, in our situation, we're lucky in that again, we have, uh, um, an AI partner in HR.

[00:38:30] So like, I don't have oversight of AI, AI, AI, and HR. My, I have a peer that does, but it's great because it does afford us the opportunity to, um, work together from, from a roadmap perspective and figure out like, what opportunities do we have to, um, layer that in? Right. And so, um, you know, again, I, I don't know.

[00:38:54] I can't speak cause this is actually something I was going to talk at PSYOP next week with some of our, um, compatriots about. Um, but you know, trying to scan the room and get a sense of how they're handling it. For us, we, we have some early cases. I can't speak to them in detail. Um, but that we're definitely planning to point that against. Um, but there's just layers of considerations, approvals and all that.

[00:39:18] So I think if nothing else, it's just, it's, it's more of, um, I think organizations trying to figure out and, and people analytics groups, making sure they're not running afoul of any sort of protocols and governance that are being put into place, um, around that, those deployments. I mean, even though you may have access to something through your tech stat, doesn't mean that you should be, um, you know, deploying that preemptively, you know, ahead of broader kind of guidance and strategy around what the organization wants to do.

[00:39:48] That's sort of where we are is like, we have some really compelling use cases and we think we could support it. Um, but we don't know that it's the right thing to do yet. Have you ever wondered what really makes a generation tech who gets to pick the name and why the slang keeps changing? Don't worry. I can help. My name's Dr. Megan Grace on hashtag Gen Z. I share the voices and experiences of Generation Z, how they're different from other generations, what moves them and why they do what they do.

[00:40:15] In each episode, we go beyond the buzzwords and the stereotypes to dive into real conversations and the insights that matter to making intergenerational collaboration a reality. You can catch hashtag Gen Z on the work defined podcast network and wherever you listen to podcasts. Absolutely. Yeah. Let's, let's do some, what am I reading? Let me pull something up for you real quick. So there was a recent paper in science that's been heavily cited, actually cited on the podcast a few episodes ago.

[00:40:45] I can't remember which one we're talking about recent discoveries on the acquisition of the highest levels of human performance. And it had this startling finding, which is if you're at the highest level of athletic or academic achievement, sometimes those individuals actually were not the most exceptional performers in the beginning. And, and so there was kind of this kind of correlation between how high you achieved in a field and how good you were at the beginning.

[00:41:16] And what this, this paper or this post by a guy named Mikhail Navard put out there is that this actually might be a case of collider bias or what's sometimes called Simpsons paradox, which is if you have kind of two groupings on a, on, on a single distribution. If you're not on video, you can't see this distribution on here, but essentially it's a linear chart that shows the relationship between early performance and adult performance.

[00:41:45] And essentially he creates a separate color for the people that are at the very, very top end of the distribution. And what it does is it does make kind of a Simpsons paradox where if you're at the top end of the distribution, it actually seems like the, the worse you were early in life, the better you were later in life.

[00:42:02] But in reality, if you put all of the distribution in there, you can see this very easy linear pathway between poor early performance means poor adult performance and great early performance leads to great adult performance. What did you think about this one, Amy? I, I need to read that apparently. There we go. She gets the podcast. I do.

[00:42:32] I'm going to have to, I, you know, I don't know. That one's like a mind twister. I, I wonder how they accounted for other stuff. Right. I mean, so like, you know, whether there were other attributes, I mean, we're just talking performance. I don't know what the, you know, the particulars are, but like there is at least anecdotally.

[00:42:55] I mean, I think an idea that, you know, you may not be good at something initially and then like through grit and persistence and all these other things that, you know, you sort of pursue and, you know, improve. But I don't know, sports. Sports. I'm not sure that holds. So. Well, one, one interesting thing that this article really did not rebut, but I thought was very interesting from the original article was this concept of if you learn across. And this is what we covered on the podcast previously.

[00:43:23] So this is not what got rebutted is if you're a person who narrowly studies or you play like one single sport and you try to get really good at it. So you're actually not as likely to reach peak performance. But if you go across a broad variety of things, you actually start out a little bit slower in sports and in academia, but you're more likely to have the highest top end of performance. And I thought that's very interesting because essentially like if you're playing sports, you should be a multi-sport athlete.

[00:43:52] And if you're studying academics, you shouldn't again have this narrow myopic focus on a singular topic that you're going to publish on for the next 40 years. Perhaps if you really want to be world class, you should study a variety of things that may take longer, but you'll be the most likely to get to the highest level. You know, that that actually makes a lot of sense. You know why is when you think about like the ability to like create like connections right between so systems thinking.

[00:44:20] I would think like, right, that could be a hook. Right. Because it's like the more that you understand about different areas of, you know, whatever it is. Right. You know, science and theology and, you know, you name it. It just allows you to bring a more critical like mind to bear to any one problem. And so you're coming at it from multiple angles and perspectives. And, you know, it's like diversity of thought. Yeah. You've always said gives, you know, better results, you know, at the end of the day.

[00:44:48] And so, I mean, I could see some merit to that. I'm interested if that holds for musical stuff too. Right. Like you're saying you study one instrument versus many. And so like I know some musicians who can like pick up anything. Yeah. And play it, you know, which is amazing to me because I'm like, that's I'm super envious of that. I can't imagine having that that skill, that gift. But yeah. So I wonder if that holds for the musical arena as well. I don't know. That's intriguing.

[00:45:17] I don't know if it holds for music, but it is related to the next topic that I have here, which is a recent article published by Google's Deep Mind called LLMs Can't Jump. And it's summarized by a LinkedIn post by a guy named Ian McCarthy. And he said there's an interesting paper that argues that while generative AI has mastered induction, which is statistical pattern matching, and it's on its way to conquering deduction, i.e.

[00:45:46] formal proofs and logic and reasoning, it lacks the abilities for abduction, which is the generation of novel explanatory hypotheses. So this is sort of what you were saying, Amy, of if you have this wide breadth of things that you can't study, you're actually more likely to engage in abduction, which is you can bring kind of new and novel ideas to the table, which is apparently the only thing that LLMs either are on the way, they can't do, and they're not on the way to conquering already.

[00:46:16] And so if human beings are going to continue to kind of have unique and individualized value in the world with, at least in the creative task, abduction is really where we need to lean into, which is fundamentally related to what you were just saying. So I thought this was kind of a really neat overlap here. That interconnectedness, right? I mean, it's that the richness that you bring, right? And context that I think is, again, I mean, you're just, you're going to be able to contextualize

[00:46:45] something from multiple elements, right? That, that's very difficult if you have a myopic view. Absolutely. That's, that's pretty cool. Interesting that we're on the music track, but. Well, I'm curious, from what you've seen, like, and this should be your own personal AI usage, what have you. Has it ever generated, like, an idea that you thought was like, wow, I've never thought of that before?

[00:47:12] Like, it, like, I think it's very good at, like, summarization and pattern matching and things like that, but has it ever done something you're like, wow, I think that just blew my mind? Not yet, but I do think it will probably happen in the near future. Okay. What, what leads you to that belief? Um, well, I mean, and just look at the, the, I mean, it's, it's really mind blowing, right?

[00:47:41] How rapidly the, the technology is improving, but I mean, I, you know, I think it could be, it is mind blowing that it can pass like the Turing test and all that. Like that is in itself quite an achievement. But, but I think it will likely be something more like seated, um, in like, so maybe you're leveraging technology to, you know, kind of automate, um, I don't know, analyses or whatever. Right.

[00:48:07] But it could be that, you know, it gives you this, like, I will say this, it will, it gives you the freedom of time to actually think, which is, I think something that we've lost. Right. And so it's like, I don't know that it's going to so much be that it gives you the generation of an idea, but like the fact that it's freeing time for your mind to sort of like turn and think and, and like, you know, the productivity that it enables that, that thinking, that deep thinking time.

[00:48:37] I think I remember you and I had a professor one time that talked about like flow, like being in that state of like creation and where we don't get to do that a whole lot. And I'm wondering, is that going to actually be our tool that unlocks that, that ability to just have like eureka moments? I don't think it's necessarily going to come from AI. I mean, maybe, I mean, perhaps we'll see, but I just wonder if it can get us away from

[00:49:06] the rat race enough to get back to a state where I think we were having more. Beautiful eureka moments, you know, in, in the distant past that, that have kind of gotten like subsumed and lost in the rat race the past 20 years. Yeah. And just to give credit or credit is due to that. It's a Chicks at Me Holly's work and Dr. Mitzi DeSellis who introduced us to that. So thank you, Mitzi, for, if you're out there listening. I think there's an interesting point there.

[00:49:36] We covered this a while ago and, but it showed that there was like a linear relationship between like how many novel ideas you can generate with AI and your levels of education. And so essentially the more ideas you have, the more ideas you can have AI generate, which I thought was kind of interesting. There's somewhat of like a compounding effect there, I would imagine too. Yeah. I don't know. I'm still going to believe in the, the uniqueness and, and, and beauty and value of the human mind and brain.

[00:50:06] And there's something that just like, you know, inherently can't be replicated. Oh yeah. Yeah. I mean, we're all experts at least on one thing, which is ourselves. Right. And so we do all have at least some unique value in some sense. Yeah. No, I just, I don't know. It's just fascinating to, to see and interact and be, be a part of that, you know, kind of evolution of that technology and sometimes scary, but yeah.

[00:50:36] So. All right. Well, I've got one more. What am I reading for you? Okay. So this is the last one is from a person we've covered many, many times, Dr. John Sullivan. And he's always writing something provocative. It's always got some nuggets of words in it. It's always got some parts where I kind of disagree with it. Um, and this is the case against hiring underqualified candidates with potential to grow into the job. Mm-hmm. And so he lists, um, a bunch of different reasons why you shouldn't hire people for potential.

[00:51:06] I'm just going to list a few of them because he's very, you shouldn't hire for potential. You should not hire. Yeah. It says, uh, today everyone has stopped listing potential as an acceptable substitute for qualification. Managers, uh, need the new hires maximum performance today. Defining and measuring potential is extremely difficult. Others will be forced to carry the load. This approach is not cheaper. Most managers aren't good at picking candidates with potential. The development of a new hire will be expensive and time consuming.

[00:51:36] And he goes on and on. There's, I mean, it's a, it's pretty, he's got a lot of reasons. All right. I am curious from your perspective though, Amy, do you think that hiring somebody for potential is a worthwhile endeavor, or should we only be, you know, hand to mouth with our hiring in the sense that we can only hire somebody who can do something for us immediately today? Absolutely not. I think potential is so important. And I mean, look, here's the deal. If nothing, what I'm looking for a mind at work, right?

[00:52:05] When I'm, when I'm interviewing like the four corners of a document, when you create a job description or, you know, your work that that's helpful, whatever. But if this is a checklist of like credentials and experience and whatnot, like, you know, again, I, that's not, that's not the way I operate. I think it's important to understand, you know, what skills people bring to bear out of the gate. And there has to be a water level, right?

[00:52:32] You know, there, there is a minimal, like you have to be able to do X, Y, Z on day one. I think it's more incumbent on the hiring manager to be able to say, here's the plan. Right. Here's your 30 day, you know, 69, whatever. Like, I don't think enough hiring managers are held to account for, from a performance perspective, creating a clear plan and then bumping that through the hiring process against like, do I think this person can meet this?

[00:53:01] And here's the substantiation for that. Right. And so that's where potential sort of, I think is, is really important. And I will tell you, I, you know, again, you see these job descriptions or whatever that say, you may not meet all of the, you know, qualifications for this. We encourage you to apply. Like, let's be honest, like the litany and laundry list of stuff that's in any one given job description is oftentimes ludicrous, right?

[00:53:30] I mean, you're asking for a unicorn and I get that. And so like, at what point is that like credentials and qualifications versus potential? If you can't check a box on every one of those, like 150, like bullet points you have in your job description, is that, that you're going, you know, you're moving. I don't know. This is, to me, I disagree with that. I love everything you said. One of the things I try to do is always try to like give a good faith disagreement with somebody.

[00:53:57] And so I want to give him credit where credit is due in the sense that I always think about, and this is what we learned in our graduate program of, you know, if you're not selecting for something, you need to be training for it once you're on the job. And essentially, if you're thinking you're hiring for potential, you're basically saying I'm hiring to train this person. Right. I'm dealing with that in some, a lot of cases. Well, but what I see is, and this is for my last role where we, we did a lot of research

[00:54:24] about what we called like the tale of two economies that's going on right now, which is like in the white collar economy, there is an excess of, of labor for the first time really ever. Like the war for talent is over in that space and a, a, a shortage of demand for new people. And so that means that employers can be incredibly choosy. And so that they truly can put 150 different requirements on there and wait for the unicorn to come along and then hire for that person.

[00:54:54] However, there's an entirely separate economy that most people who write in this space act like doesn't exist, but it's like the majority of the U S economy and the global economy, which is like the blue collar economy and the service economy and the healthcare economy and the, what you might even call the real economy. And in this economy, there's like one job seeker for like every five open roles. And you are, it is incumbent on organizations to select for potential because they cannot find skills applicants and they must train them on the job.

[00:55:23] And so I think what he's saying is right. If you're kind of in this, if you're selecting for white collar applicants, but if you're in the more of the real economy, I think that he's, he's wildly off. Yeah. But here's my counter to that is like for the white collar economy anyway. Um, like I bring someone in, I'm not bringing them in only for that role. I'm bringing them in. Cause I'm like, they've got potential to scale into other stuff. And so, like, so that is a really important consideration for me.

[00:55:53] And if it's only like you're qualified for this, but I don't see, you're talking about potential for the role I'm hiring you for, but like the traits that show me that you have potential to develop and, you know, those, those carry. Right. And so like, I do think there's a consideration there for scalability. Um, it's really important. Right. So that may be a consideration. Scalability and fungibility.

[00:56:20] Like you want people to apply them to multiple problems and create a more rounded human. And this is, I always like, it's like, I, you know, I am looking for people's career progression, both within my team and outside my team and across the organization. Because like, if you're just hiring, it's like, I want you for this role and I want you to sit in it forever, you know, like that's a, that's plugging a hole in a dam, you know? Um, so anyway, I, I don't know.

[00:56:43] I just take a little bit of a different like view there, but I need to read the article. To be fair. I need to read it. Yeah. Well, and when people put out like a very provocative thesis, obviously they're, they're asking to be argued with, frankly. Yeah. I miss the way I look at it, but, um, Amy, any questions for me before we wrap up? Yeah, two. So like on the musical vein, so one serious, one not serious, which one do you want?

[00:57:12] The not serious. Sure. Whatever you want. Okay. So like, uh, Cole, I, I've known you a long time and I know you love music. I do. And I know, you know, you can sing. Uh, somewhat. What's your karaoke song? Oh, I haven't done karaoke in a while. I used to do. You gotta have one. I mean, I've done a few and you've seen me do karaoke before.

[00:57:37] Unfortunately, I used to do numb encore, uh, by Lincoln park and Jay Z. Uh, that's a, that's a risky one. It drops a few, uh, foul words in the song, but, uh, it's a fun one. Okay. Um, and, uh, I've, I've done total eclipse of the heart before. Um, that's a good karaoke. That is a good karaoke song. Um, I don't know if we've probably even done one together before. Have we ever done one together?

[00:58:04] Uh, that's going too far in the, the, the trust tree. Yeah, we're not. Yeah. So that was that one. And then, um, I was just going to say, so you recently, uh, just kind of like really quick thought. Um, but like I, you know, you recently, um, you know, had a podcast trying to, you know, get back into keeping up with stuff. But anyway, it caught my eye and I wanted to kind of, um, hook it here just cause we have analytics leaders.

[00:58:31] So, um, interviewing, um, um, oh my gosh, I just forgot our name. Is it Wilson? The, the, the hydrogen struggles, the code. Oh yeah. Yeah. Yes. And so that was really interesting to me. It's something you and I have talked about for years and like just any kind of, um, parting words you might want to give and also a hook to listen to that episode for analytics leaders that might be thinking about what's my career path beyond analytics.

[00:58:57] Is it, you know, is there a path to, you know, executive leadership, CHR roles? Um, I thought that was really an interesting topic and interested in your thoughts there. Well, let me, let me get on my high horse here for a second. And real quick, that episode people do not because she's not well known in our field, but Jennifer Wilson is like, like if you're talking about gatekeepers for like the keys to the castle for any senior HR leadership role at the top companies in the world, knowing

[00:59:27] someone like her is like a key unlock. Right. And she came on the podcast and spit all this wisdom. And because people don't know her, it's like a very averagely performing episode. And so people go out and listen to that episode because she dropped so much knowledge. And just because people don't necessarily know who she is, cause she's not a people analytics person. Like I have gotten more kudos from like the top people like CHROs and top people analytics leaders and just people in general who were like, Oh my God, you got her on your podcast as a guest. I cannot believe that.

[00:59:56] And she shared all of this because the people who know, know that she has like this like secret proprietary knowledge. So if you want to know, like, and like, frankly, that's like, I'm sorry, I'm like getting super passionate about this. It's like one of the reasons I have this podcast is to try to share kind of like secret proprietary knowledge insofar as I can with the masses to do like this massive public service. Right. It's like one of those things that keeps me motivated. And so like having people like you on is amazing.

[01:00:25] Having people like her on is just like, you, you cannot replicate that. And so I'll say some of the things that I learned from her. And again, we've kind of used this word myopic a few times is a lot of people in people analytics believe that you should just become a deeper and deeper subject matter expert. And that's, what's going to take you to the promised land of moving up in your career. And she was very much not in the deeper camp, but in the broader camp of getting multiple

[01:00:54] critical experiences throughout the business and giving the exact playbook. I mean, exactly not like, not theoretically of what it would take for people analytics to become a CEO to grow someday. And I was like, and she did this in public and I was like, this is so crazy. I can't believe she's actually sharing this. And so go and listen to the episode. Thank you for asking that question, Amy, because it's just like, it is, it is just floored me, I thought it was going to be like the most popular thing I've ever done. Because it was like all this secret wisdom is just being shared.

[01:01:23] And I have thanked Jennifer profusely behind the scenes for being a part of it. But yeah, thank you for asking that question. What made you ask that? Just out of curiosity. You know, I mean, obviously, you know, it came up recently and I, it caught my eye and I'm like, you know, it's always been something that we talk about in our circles, right? It's something that we, I will say as analytics leaders, we know that we have wisdom and value to bring to bear. But I also think a lot of analytics leaders get pigeonholed.

[01:01:53] Right. And so I think it was really timely and it's something that we should use as like encouragement to your, for our peers to like, hey, you know, you may love what you do. And like, but for anybody that's feeling a little bit like I want to give more and do more, you, you do have a playbook for that. Yeah. Well, one of the things, and I recorded this and it hasn't come out yet, but we talked to

[01:02:21] J Ben Babel who published a book called The Power of Us. And he talks about the power of identity in the workplace and how much it plays a role. And he talks about like social conformity, but also the importance of dissenting viewpoints. And one of the things that was so eyeopening that you mentioned is that dissenters in a community are actually the people who care about the community the most. And so one of the things that I try to do is I feel like I have a dissenting viewpoint oftentimes

[01:02:49] from the community, but it's because I care about it so much. And I was so glad to hear him say that. I was like, oh, I'm not a crazy person after all. Thank you. Here's a dissenting viewpoint that I have. And it builds on what you're saying, which is I think people analytics folks, I think this goes for every profession, but let's say people analytics folks, because that's who we're talking about. They just want somebody to come along and say, what you're doing is perfect. You don't need to change at all. And it's like, no, the most helpful thing if somebody really cares about you is to tell

[01:03:19] you, hey, you're messing this up. Hey, here's how you can really improve. Hey, I have your best interest in mind. I care about you deeply and I want to see you succeed and you need to be changing X, Y, and Z about what you're doing. And I think that's one of the things that comes along with an episode like Jennifer's. She actually shares things that are not what people analytics folks tell each other about what it means to be successful. And, and, and therefore they're like, eh, I don't have to listen to her.

[01:03:47] It's like, yeah, but maybe you should. And so that's where I get kind of frustrated about these things is because I care so dang much about this community. It's like, somebody is giving you the playbook. It's like, imagine you're playing somebody in sports and the other team just gives you their playbook for how to beat them. And they, they just come out and they roll out the red carpet. They're like, here's the playbook. Please beat us. And people are like, no, I don't want to beat you. And that's what frustrates me. And so I just, I get super passionate about this issue.

[01:04:16] Well, I mean, I encourage everybody to go listen to the episode. And listen to Amy's episode too. I mean, you're listening, obviously if you're hearing it right now, you're listening, but I know it makes no sense, but Amy, you're fantastic. Thank you for having me. It's been great catching up and I don't know, like I just, it's, it's always, I mean, obviously I can say it's like talking to an old friend because we are, so that's why I can't wait to see you here soon. It's going to be good. I can't wait to see you.

[01:04:46] This will come out after Sia, but it'll be great. I know, I know. It'll be like old news then, but no. Absolutely. But Amy, it's been fantastic having you on as a guest. You've been listening to Directionally Correct, a People Outlook podcast with your host, Cole Knapper, and today's guest, Amy Stevenson. Thanks for joining. All right. Bye. Bye.