A Debate about Going Beyond Tasks: Redesigning Organizations Around Goals and Knowledge

[00:00:00] Welcome to The Debate. Imagine you are told to assemble a highly complex machine. You gather your tools, you sit down at this brightly lit workbench, and you lay out all the individual gears. But just as you start, someone flips off the lights. Which is always helpful. Right. Super helpful. The blueprint you were given suddenly bursts into flames, and the actual gear you're holding in your hands literally begin to change shape. Yeah, and honestly, that chaotic, totally unpredictable scenario, that is the exact reality of modern knowledge work.

[00:00:28] It really is. The classic, you know, mid-20th century approach of mapping out task interdependence, basically figuring out exactly who does what before anyone clocks in for the day, is completely obsolete. I mean, you're dealing with a software launch or pharmaceutical R&D, or these highly fluid hybrid teams. You just can't map task interdependence in advance when the tasks themselves are fundamentally ambiguous, right? They are constantly shifting. It is the defining organizational challenge of our time. Exactly.

[00:00:57] When you're operating in extreme uncertainty, the tasks aren't just predefined steps on a checklist. They're emergent. They have to be discovered as the work actually unfolds, and that completely strips away the traditional tools of management. You can no longer manage the work by, you know, just managing the tasks. Which brings us to the core of today's analysis. We are grounding today's discussion in the research beyond tasks, redesigning organizations around goals, knowledge, and the people who hold both. A fascinating piece of work, by the way.

[00:01:26] Yeah. Truly. The source material argues that if leaders can no longer structure their organizations around predefined tasks, they are basically left with two primary structural levers. The first is goal interdependence, structuring an organization around what people want, their shared outcomes. Right. And the second is knowledge interdependence, so structuring the organization around what people know, their specialized skills, and their expertise.

[00:01:53] Right. So the debate today centers on which of these two levers must serve as the primary foundational anchor for organization design. Now, I will be taking the position that goal interdependence must be that primary anchor. It acts as the essential motivational glue that keeps autonomous teams oriented in the same direction. It essentially prevents them from just fragmenting into isolated silos. And I take the opposing view. I will argue that knowledge interdependence is the true foundational lever.

[00:02:21] Because honestly, you can give a group of people the most beautifully aligned, inspiring goals in the world. But if you don't engineer the actual composition of the team, if you don't create what we call epistemic transparency, no actual coordination can emerge. The knowledge structure is what catalyzes innovation. Well, let me start by unpacking why goals absolutely have to come first, especially in these foggy, ambiguous environments. When the task structure is completely unknown, goals are the organizational equivalent of handing your team a compass.

[00:02:51] Okay. A compass. Right. They tell agents which north to orient toward when the terrain ahead is totally unmapped. And the empirical evidence for this is incredibly robust. Take Ruth Wagaman's 1995 field study of Xerox service teams. Oh, a classic. It is. She looked at what happens when technicians face these really complex, unpredictable repair jobs. What Wagaman found was that congruence between an organization's design and its goals strictly predicts group effectiveness. Mm-hmm. So let's look at the mechanics of that, right?

[00:03:21] If you have an ambiguous problem requiring collective effort, but your reward structure, your goal interdependence, only incentivizes individual performance, you get severe performance penalties. The technicians end up hoarding information. They compete rather than collaborate. You have to start by aligning what people want. I mean, I get the appeal of the compass metaphor. I really do. But let me push back on that. You can point a compass north all day along.

[00:03:47] But if you haven't assembled a crew that actually knows how to, you know, build a boat, navigate the currents, read the weather, you're not getting anywhere. Well, that compass is useless without the underlying capacity to act. And this is exactly where knowledge interdependence comes in. It's the complementary value generated from combining diverse idiosyncratic skills. Dorothy Leonard-Barton coined this brilliant concept of creative abrasion. Right, the friction. Exactly.

[00:04:12] When you are trying to innovate, you actually need that constructive friction that arises when you put agents with divergent knowledge bases like, say, a behavioral psychologist and a back-end software engineer in the exact same room. But wait, that friction is only productive if they share an incentive to actually resolve it. Without a shared goal, it's literally just two people arguing in a room. No, it's productive because their knowledge sets cannot function without each other to solve the problem at hand.

[00:04:36] I mean, in contemporary context, tasks can't even be defined until that right mix of knowledge is physically or virtually co-located. Henry Mintzberg's work on adhocracies backs this up perfectly. Adhocracy, sure. Yeah, in an adhocracy which is an organizational form explicitly built to innovate, coordination is driven organically by those who possess the specialized knowledge, not by whoever holds formal authority and certainly not just by top-down goals.

[00:05:02] If you don't design the knowledge structure first, the tasks that eventually emerge will be inherently constrained by whatever expertise is missing from the room. Okay, I hear you. But let's take your boat analogy and see how it actually plays out in the wild. Because I think putting brilliant complementary minds in a room without a shared compass doesn't get you a working boat. It gets you a highly capable mutiny. A mutiny? Yes. Every specialist will just want to build the part of the boat they care about and literally no one will care if the hull actually floats. That's a bit cynical. It's human nature.

[00:05:31] If you want to see what happens when you prioritize knowledge over goals, we don't even have to guess. We just have to look at Spotify. The famous Spotify squad model. Exactly. And the source material specifically critiques Spotify's model on these exact rounds. Let's look at how they engineered it, right? Spotify absolutely nailed cross-functional knowledge. They created these squads containing a perfect mix of designers, engineers, and data scientists. Then they built chapters and guilds to sustain the flow of that specialized knowledge across the entire company. Right. It was brilliant.

[00:06:00] On paper, sure, it was a masterpiece of knowledge interdependence. But they struggled immensely. Why? Because they granted all this autonomy based on knowledge, but they failed to install corresponding goal interdependence mechanisms. When you do not deliberately widen the incentive breadth to match the autonomy you are giving people, rational actors will do what rational actors do. They optimize for their local incentives. See, I would frame their struggle quite differently, but go on. The mechanism is super straightforward.

[00:06:26] Squad A is optimizing for a slick user interface because that's what their local metrics demand. Squad B is optimizing for back-end streaming speed. Right. Because they lacked an overall and goal interdependence to force them to care about the unified product architecture, the squads became deeply siloed. Right. They fragmented. The brilliant knowledge they possessed couldn't be stitched back together into a cohesive whole simply because they were optimizing for different Norths. Okay, look, I will concede that fragmentation is a massive risk in highly autonomous knowledge-based models. That's real.

[00:06:56] But you were diagnosing Spotify's coordination failure as a lack of top-down goals, and I would argue that is a complete misdiagnosis. The root cause of that fragmentation isn't a lack of shared goals. It is a lack of epistemic interdependence. Wait, hold on. Let me make sure I'm mapping this correctly. Epistemic interdependence. Meaning the predictive knowledge of your colleagues? Precisely. It is the degree to which your optimal actions depend on your ability to predict what your colleagues are going to do.

[00:07:24] When epistemic interdependence is high, you don't need a manager holding a compass. And you certainly don't need a collective bonus check to force alignment. Consider a firm like Bridgewater Associates. Oh boy. Right? They are famous or perhaps infamous for operating with radical transparency. They record every single meeting. Feedback is totally open and unvarnished. And the criteria for every decision are painstakingly documented. Which, let's be honest, people often criticize for creating intense burnout. They do.

[00:07:52] And we can set aside the cultural intensity for a moment to look purely at the structural mechanism. Because why do they do it? It serves a profound epistemic function. By recording and analyzing everything, it makes everyone's reasoning patterns and decision tendencies completely legible to everyone else in the firm. So everyone knows how everyone else thinks. Exactly. When you drastically reduce the uncertainty agents face when trying to predict each other's actions, that transparency acts as the structural glue.

[00:08:22] You can coordinate flawlessly without constant communication and without heavy-handed goal alignment because you fundamentally understand the knowledge architecture of the people around you. Hmm. I know how you will react to a market shift, so I adjust my behavior accordingly. The knowledge of how we think replaces the need for a shared goal of what we want. Okay, I see the logic there. But I really don't buy that it scales beyond highly specific financial or, you know, tech environments. I'd buy the Bridgewater example if their product was something other than trading algorithms.

[00:08:51] Transparency is great, but knowing how your colleagues think doesn't tell a hospital how to treat a patient. Well... Hear me out. If Spotify squads fragment, you get a clunky app update. But if fragmentation happens in an environment where the stakes are literal life and death, the theory gets tested in a much harsher light. Let's move to healthcare, where emergent task design is critical. Goals are the catalyst that allows complex task structures to merge organically. I mean, look at the Cleveland Clinic. A classic case of organizational redesign, yes. Right.

[00:09:21] For decades, they were organized by traditional, knowledge-based departments. You had cardiology over here, cardiac surgery over there, endocrinology somewhere else. They had grouped people entirely by what they knew. And mechanically, yeah, it optimized the tasks within those specific departments. But it created massive, dangerous barriers for the patient who had to navigate between these isolated knowledge silos. So how did they fix it? By changing the goal structure? Exactly. They changed the goal interdependence.

[00:09:49] They dissolved the traditional departments and created patient-centered institutes organized around organ systems and specific disease categories. Because the overarching goal became holistic patient outcomes rather than departmental efficiency, the necessary task structures organically emerged. Suddenly, new referral pathways developed. Collaborative care protocols were written. The goal was the structural intervention that changed the tasks. That is a very compelling narrative. But let's actually look closer at the mechanics of what happened at the Cleveland Clinic.

[00:10:18] Because I argue this example actually proves my point. Really? How so? Yes, the leadership instituted a new patient-centered goal. But what was the structural intervention that actually altered the daily reality of the medical staff? It was the physical co-location of overlapping clinical domains. But they... They literally put the physicians, the surgeons, and the researchers into the same physical institutes. You can hand a hospital staff all the patient-centered purpose narratives in the world.

[00:10:44] But if you do not physically or virtually co-locate those diverse knowledge bases, the tasks will not change. The friction of knowledge exchange is what prevents collaboration. But they only co-located them because the goal demanded it. The goal was the entire premise for moving the desks. The knowledge requirement dictated the execution. I mean, research by Gray, Simpson, and Vasudeva demonstrates this clearly. Physical or virtual proximity is required to reduce the friction of sharing tacit knowledge. And tacit knowledge isn't something you can just write in a manual.

[00:11:14] It's the intuition a surgeon has or the subtle pattern recognition of a diagnostician. Okay, but... If you want to see how this works when there's literally zero time for goal setting, look at Valentine and Edmondson's research on team scaffolds in hospital emergency departments. Okay, ERs are the definition of unpredictable task environments. I'll give you that. Exactly. When a trauma patient comes to the doors of an ER, the tasks are entirely fluid. The team has no idea what injuries they are dealing with until they start assessing. So how do they coordinate?

[00:11:43] They certainly do not huddle up to discuss their shared goal of saving a life. Because it's assumed? It's assumed, but it's totally useless for actual coordination. Instead, they use mesolevel structures, temporary team designs with predefined rule-based knowledge structures. The scaffold is a fixed architecture. An attending physician, a resident, a triage nurse, an anesthesiologist. Right, the slots are fixed. The actual human beings filling those roles might change from shift to shift, but the knowledge categories are fixed.

[00:12:11] When the patient arrives, staff plug into these scaffolds based entirely on what they know. Because the knowledge structure is so robust, it allows them to handle highly fluid, ambiguous tasks instantly. The knowledge structure is what allows the work to happen, regardless of the overarching goal. Okay, I am going to challenge that by bringing us down to the psychological reality of the individual people filling those scaffolds. If you focus solely on plugging people into roles based on their expertise, but you fail

[00:12:37] to align their goals with the strategic intent of the organization, you create deeply siloed, competitive, and ultimately miserable employees. You think that's purely a goal alignment issue? Absolutely. If you're listening to this and thinking about your own hybrid team, you are probably wondering why, despite having brilliant specialists on your Slack channels, every department still seems to be fighting a turf war. And you're arguing that stems from misaligned goals? 100%.

[00:13:03] Roger Gulati documented this extensively when studying firms that were trying to become more customer-centric. These firms had hired the right experts and put the right knowledge in the exact right places. But their incentives, their goal structures, still reinforce the old divisional boundaries. Oh, sure. The sales team was incentivized on volume, the support team on call resolution speed. So despite having the knowledge capacity to help the customer holistically, employees acted rationally

[00:13:30] according to their local incentives, and the customer experience completely fractured. Well, because the incentive breadth was too narrow. Yes, but it goes beyond just financial performance or customer satisfaction. Poor goal alignment fundamentally demoralizes people. Let's look at Andrew Carton's famous study of NASA in the 1960s. Ah, the moon landing. Right. Carton looked at how an organization composed of massive, disparate knowledge bases managed to pull off the Apollo missions.

[00:13:56] He found that employees who could not see how their highly specific local tasks connected to a broader organizational purpose reported significantly lower meaningfulness in their work. Think about the janital or the low-level technician at NASA. If their local goal is just clean the floor or calibrate this one dial, the work is just a grind. But when leadership actively builds a goal interdependence that connects calibrating this dial directly to the overarching goal of putting a man on the moon, everything changes. It's a nice story. It's data.

[00:14:26] The difference between feeling like a replaceable cog and feeling like an explorer is entirely a function of purpose narratives and goal interdependence. That is your primary lever for employee engagement and avoiding demoralization. Look, I don't disagree that purpose is powerful. But meaningfulness doesn't just come from a grand narrative handed down from the C-suite. Meaningfulness comes from agency. And in a modern organization, agency is fundamentally about knowledge. How so?

[00:14:53] You're pointing to employee demoralization as a failure of goals. But I would argue that frustration in today's workforce much more frequently stems from an employee possessing knowledge that goes far beyond their formal job description, but lacking the structural permission to apply it. Ah. They know how to fix a problem, but the hierarchy says, that's not your department. This is the entire basis of the job crafting literature by researchers like Reznowski and Dutton. Job crafting. Right. Being the process where employees proactively redesign the boundaries of their own jobs,

[00:15:22] regardless of what their formal description says. Right. They take on new tasks, or they alter their relationships with other departments, because they have idiosyncratic knowledge that the formal organization hasn't even recognized yet. True empowerment is about knowledge application. Let me give you a highly concrete example straight from the source material. Stephen Barley's sociological study of the introduction of CT scanners in hospitals in the late 1970s. Oh, this is a perfect example of technology disrupting established structures. Exactly.

[00:15:50] When the CT scanner arrived, it was a radically new technology, and it completely scrambled the organization of the radiology departments. But I want to be very precise about the mechanism here. It wasn't a shift in hospital goals that restructured the work. It was a sudden shift in knowledge interdependence. Walk us through it. Mechanically, the technicians, who were obviously lower on the traditional hierarchy, understood how to operate these new computerized machines far better than the senior radiologists did.

[00:16:17] The technology shifted who held the critical tacit knowledge. Right. The doctors didn't know how to use the computers. They literally could not do their jobs without the technicians explaining the computer interfaces to them. That friction forced a complete renegotiation of roles, tasks, and power dynamics between the doctors and the techs. The knowledge shifted first, and the organization had no choice but to adapt its structure around it. That makes sense. If leaders want to build adaptive capacity today, especially with the rise of AI and new tools,

[00:16:45] they need to legitimize these informal roles. They need to build transactive memory systems, which basically simply means helping everyone in the organization understand who knows what. That, not some corporate purpose statement, is what truly drives engagement and adaptation on the ground. Well, I will grant you that understanding who knows what is absolutely vital when new technology hits. But even in your job crafting scenario, we run into the exact same problem. An employee who crafts a new role for themselves based on their unique knowledge,

[00:17:14] but who actively works against the organization's overarching goals, is just going rogue. They're innovating. They might be highly engaged in their rogue project, sure, but they are actively harming the collective effort. That is why goal interdependence remains the indispensable North Star. As we wrap up our analysis today, I want to crystallize my position here. I completely agree that engineering knowledge configurations, you know, building those transactive memory systems and fostering creative abrasion, is critical for discovering solutions. Good to hear.

[00:17:43] But without deliberately widening the incentive breadth, without layering collective goals over individual ones, organizations just invite chaos. In ambiguous environments, rational actors will always default to optimizing for their local immediate incentives at the expense of the whole, unless leaders actively design the goal interdependence to prevent it. You need the compass before you build the crew. And to crystallize my perspective, goals certainly provide a destination, but the actual vehicle for navigating a foggy,

[00:18:12] ambiguous environment is knowledge interdependence. As I said earlier, a compass cannot paddle a boat. Fair enough. If you want emergent coordination, you cannot just tell people where to go. You have to engineer who interacts with whom along the journey. You have to legitimize informal roles based on actual expertise rather than outdated titles. You have to invest in epistemic transparency so people can predict each other's actions. That structural investment in knowledge is the true engine of adaptive collaboration.

[00:18:39] Well, while we obviously weigh the primary levers differently, I think our discussion highlights a vital, unavoidable reality for anyone listening who manages a team today. Both of our perspectives demand a profound, somewhat uncomfortable shift in how we view leadership. Absolutely. Organization design can no longer be treated as a, you know, mid-20th century exercise in top-down task management. The era of the watchmaker at the perfectly lit workbench is over. It is. Task interdependence is endogenous now.

[00:19:09] That means it is not a fixed property of the work that you can plan in advance. It is something that emerges from the people doing the work. And that requires leaders to view organizational structure not as a static blueprint, but as a perpetual, revisable experiment. Today's optimal reporting structure is almost guaranteed to be tomorrow's bottleneck. Which means we fundamentally have to recalibrate the psychological contract between employers and employees. If tasks are emergent, we are essentially asking employees to be co-designers of their own work.

[00:19:37] We're asking them to engage in continuous sense-making, rather than just executing predefined steps for a paycheck. Exactly. We have to structurally reward initiative, horizontal collaboration, and knowledge development, not just task completion. That requires a massive rethinking of how we evaluate and compensate performance. That recalibration is an area in the source material that is incredibly rich and absolutely ripe for further exploration as we all try to navigate this ambiguity.

[00:20:06] It is a completely new paradigm for how humans organize. So the next time you find yourself trying to manage a complex, unpredictable project, and you feel that familiar urge to just assign a list of tasks, remember the watchmaker sitting in the dark. You cannot map the gears before they are cast. You really can't. The question you have to answer is, will you focus first on pointing everyone toward a shared North Star, or will you focus first on ensuring the right minds are colliding in the dark?

[00:20:33] Think about what your team really needs to find their way. Thank you for joining us on The Debate.