Nik: Welcome to AI and Design, where we explore how artificial intelligence is reshaping the world of design. I'm Nik Martellero. So for today's interview, ⁓ we're joined by Tom Groendal, who is a Senior Product at CACI's Blue Intelligence Group. Tom was actually recommended to us by Jason Ogle, who's a designer at CACI and a listener of the show. ⁓ We had on an episode a few weeks back asked our listeners they knew anyone who was a product manager who was applying AI to their work. Dan: And I'm Dan Saffer, and we're faculty at Carnegie Mellon's Human-Computer Interaction Institute, HCII. And each week, we break down the latest AI developments, dive deep into topics that matter to designers, and talk with fascinating guests who are right at the intersection of these fields. Nik: Whether you're a designer working with AI or an AI practitioner interested in design, we're glad you're here. This week, some new research from the Wharton School at Penn looked at how AI is reshaping human reasoning, and the news isn't that great. In fact, they've coined a new term for it, cognitive surrender. What are they doing and what's it like? How is it in an organization where people are starting to really integrate this, not just at the design level, but at the management level as well. So Tom, thanks so much for being on the show today. We're super excited to chat with you. If you want to start with maybe a quick introduction. Thomas Groendal: I'm here. Dan: Then we'll look at an article that asks, if AI is so good, why aren't products getting any better? Thomas Groendal: Sure, for having me. ⁓ name is Tom Groendel. I work with the product team at CACI's Dark Blue Intelligence Group. It is ⁓ a team, smaller than two pizzas. It is ⁓ a ⁓ niche product. So serve enforcement, intelligence community, and other people who need to sift through an ocean of unstructured data ⁓ and sort of... Nik: to top off this week, we'll be talking with Senior Product Manager Tom Groendal from CACI's Dark Blue Intelligent Group on the real world experience of using AI on their product design team. Dan: But first, let's talk about surrender. Thomas Groendal: human intellectual filth that comes out of the dark web where people are doing bad things and try to de-anonymize some of those bad guys and also recognize threats, recognize rising trends in sort of crime and doing that are out there. So it's a pretty challenging and interesting space. I am lucky to have a very talented designer, Jason Ogle. he and I and the engineers worked together to kind of take all that messy data. can imagine forums where people are chatting or where people are dumping leaked data or sites that sell drugs and other worst things that we don't need to get into. But... Dan: So the term comes from a paper that came out back in February and is getting some traction. And the paper is from Stephen Shaw Gideon Nave and it's called Thinking, Fast, Slow and Artificial, How AI is Reshaping Human Reasoning and the Rise of Cognitive Surrender. Thomas Groendal: Just all that bad stuff and then kind of making it easy for somebody who doesn't spend all day in those environments to find which clues they could use to sort of pull apart the mystery and solve who is the mask and ⁓ the job done. And what we do. Again, it's pretty small team, so AI for us is a big accelerator. Dan: So you may recognize the thinking fast and slow bit from a book by the same name by the late great Daniel Kahneman. the paper kind of talks about two big concepts. The first is this idea of cognitive offloading. And this is when you hand off the how, but keep the what. ⁓ you are still judging whether the result is sensible and good. ⁓ Nik: ⁓ So regards to managing a team and especially working with designers, what have been using and what have you been trying out ⁓ that is that Dan: and you intervene when it's not. this is things like calculator or navigation or those kinds of things. Cognitive surrender, on the other hand, is what happens when you actually stop constructing ⁓ a good answer looks like at all. Thomas Groendal: Sure, so every product manager who's on LinkedIn is probably ⁓ in a white panic right now because they're going to be replaced, the engineers are going to be replaced, and ⁓ everything ever known is all invalid. That's the kind of messaging, the sort of anxiety farming that you see on LinkedIn. But the is more. Dan: the AI's output becomes the output no matter what it is. There's nothing that you're gonna override because you've never formed any kind of independent view or what a good result looks like to even compare it against. And so these two researchers, they did a series of three experiments involving over 1300 subjects. Thomas Groendal: disruptive than anything I can think of in recent history, far less disruptive than it's portrayed. We're not all vibe coding a brand new app every day. we all still have to solve hard production quality problems. In terms of the team dynamic, a lot is changing. We have made a new ⁓ sort ⁓ software lifecycle that is adjacent to our core workflow. Dan: and they discovered that the mere presence of AI was sufficient to trigger this human cognitive surrender. When presented with incorrect AI outputs, the participants just accepted the wrong information 73 % of the time. Even more concerning than that was the fact that the participant's confidence in the answer actually increased when using AI. Thomas Groendal: ⁓ And so I can talk about that in a minute. There's also just fundamentally the melting of roles, ⁓ it used to be there were there was a clear lane for the product person, for the designer and for the engineer. And then there were handoffs that kind of boundary those lanes into a nice little bento box. ⁓ The box has been through the microwave a few too many times and now ⁓ Dan: despite half of the provided answers being intentionally wrong. So they essentially adopted what the models typically high confidence level as their own. They felt like this was ⁓ the answer, even though it clearly was not. So. Thomas Groendal: The engineers are proposing design, the designer is proposing engineering, the product manager instead of maybe saying what the thing should be, maybe built the first one. And then, it's all a little bit backwards and ⁓ we're just on the fly to how do we do all that but still produce production grade software that solves really hard sort of problems for the customer. Dan: Cognitive surrender. Wow. One more thing us to worry about happening ⁓ with AI. Now don't this is really any surprise. At least not to me, but curious your thoughts on Nik I was imagining someone putting a lid on a bento box and shaking it and everything going everywhere. Nik: Yeah, so actually been ⁓ in my group a lot about cognitive offloading. ⁓ actually, we've thought, well, in a lot of cases, cognitive offloading isn't even really that great. Thomas Groendal: there's different ways to slice that pie. Some of it is in duties. Some of it is in who gets a bottleneck things. There's all kinds of ways that that change can occur, but the kind of through line for me is risk. So there's business risk. There is usability from a more sort of design perspective. Nik: because you're basically, if you were supposed to be doing amount of thinking ⁓ you allow a system to do that. But actually I do think that I like the term cognitive surrender here because it sort of adds a little bit of nuance to the idea ⁓ really pushes it more into this space of, I'm not even caring, adopting the AI answer. And I think this is really risky when you're especially operating in areas where... Thomas Groendal: I could vibe code something that has poor usability. could slip through, slip past Jason, go to prod, because he's on vacation or something like that. And then he comes back and there's some fundamental, I don't know, element of the information topology in the app that has just been violated and now it got way worse. Similarly, I could go on vacation. ⁓ Jason and the engineers could... Nik: you don't know, you don't have a good sense of judgment, does sound very competent. And I'm not surprised actually to see result. Actually, I'm pretty that the researchers themselves weren't surprised. It's probably why they thought, hey, this is a good paper, and we probably can get some really good data that'll show this, and that'll be a good result. The thing I've been about in ⁓ to this, though, is like, it just with AI, or do I do this? Thomas Groendal: throw something out there that solves one customer's problem, but absolutely creates new problems for other customers. And they just don't have enough sort of FaceTime with the customers to maybe catch that. ⁓ so when I come back, now I've got a new problem to deal with. Like there's ways that risk can manifest because you're moving faster. Nik: in other areas of my life. the two that I thought of, the first one was sometimes when I use a GPS and actually it was discussed how the GPS might be more of a cognitive offloading. I actually think sometimes I surrender to the GPS and I definitely have done something where I punched something in, didn't check it and just drove. And actually, because the interface design just shows me sort of the next, say, 100 meters, rather than showing me my overall view, Thomas Groendal: and a little vacation wouldn't have been enough to ship a major feature before, but now it sort of is. So that's kind of the risk management component of that. And so we just recently had a conversation around our new ⁓ of vibe code to production pipeline. which is completely divorced from the regular software CI CD pipeline. And instead one of our engineers brilliantly put together one where you can spin up a new version of the app that just lives in a little S3 bucket segregated from everything else. the codes on a branch. If you make a little prototype and you think it's cool, but it ends up being really bad for design reasons, for engineering reasons, for ⁓ business reasons, it Nik: I've driven to the wrong place and completely confidently just I and then just once I get there I go wow I am NOT in the right place I just clicked the wrong thing and I was moving too fast Dan: definitely experienced same thing. And I think when we had Chris Noessel on, he talked about this a little bit as well, when we were talking about assistive technology and this idea of you just letting it and you're not checking it. ⁓ And this seems to happen, I think a lot more Thomas Groendal: just away and there's no problem. It's not gonna clog up the pipes for something else that's critical. Or maybe ⁓ that it out there and now one of the benefits of this is Jason can see what I did. And when he sees it, ⁓ he that instinctual instant, man, this isn't right. This should be over here. The way doing this, the affordances are all wrong. ⁓ And that just been great because before it would have been a document. Dan: you don't know an area or you don't know a subject matter or you don't know exactly where you're going ⁓ and so you have less knowledge about things. So sure things that the ⁓ system provides seem authoritative ⁓ sure you're to cognitively because it because it is so confident it seems right and so you're like yeah. ⁓ Thomas Groendal: And he, I think, is going to respond better to software feeling wrong and looking wrong than he will to a document reading wrong. I think that's true for me as well. When engineers put something in front of me, even if I wrote the document it came from, when I can see it, it's much easier for me to get that reaction. So that cycle time, as as it doesn't go too fast. ⁓ Dan: That's good. you are using AI in a subject matter that you know lot about, can automatically pick out the flaws really fast. But when it's an area that you don't know very much about, well, sure, ⁓ it seems right. Let's just go for it. Thomas Groendal: and push a risk into production is great, as long as everybody's on the same page around making sure all the stakeholders have a shot at bringing expertise to the table. Dan: Are you starting with things like requirements documents now? Like where's, where's the genesis of features? walk us through some of that. Thomas Groendal: So we still have a roadmap. The roadmap is very much, as much as I can make it problem driven. And that really helps because leaves, that sort preserves room to maneuver for more innovative fast cycle development. What has shifted for me ⁓ is, Nik: one of the things that I've been seeing sometimes when I'm interacting students is you can sometimes when they're doing this cognitive surrender because you ask them a hey, why did you make that decision about your design work? And then ⁓ they're like, they don't have answer. Now that's actually, that's been common among design students for forever, right? Sometimes students and honestly, Thomas Groendal: This is probably a whole nother topic, but I created a thing I call product brain. It's basically a repo is memory, so I have a GitHub repo that is my playground where. I have ⁓ an ingestion workflow and I've ingested all of our marketing documentation, all of our help documentation, whole bunch of other stuff. I've set up an assumptions document and trade craft document that kind of helps it be a better spy and like all the things that it needs to know for context management. Dan: I just liked it. I liked that color blue. It seemed good. Nik: Right, or I didn't think about it, right? But actually, I'm seeing more and more when I ask students about their design work and they say, well, they don't have an answer. they're like, well, this is just what was output. Or they might even go back to the agent and say, why did you do this? when I really, ⁓ where actually it's this surrender term was like, they're not even trying to find ⁓ ⁓ or even try to explain the rationale. Like, well, maybe it did this because Well, that was a design system I chose or it has its own kind of house design. You know, that's why it's all purple. they're actually asking for this. And then, of course, it's going to give you a BS answer ⁓ ⁓ ⁓ confident in why it chose ⁓ palette or ⁓ layout. And, that is legitimately worrying me. ⁓ And so this Thomas Groendal: so it is ⁓ some ways ironically more documentation driven than it was before and the reason for that is twofold. One is I use a plugin called Superpowers through Copilot. Basically it enforces this sort of planning stage, a planning review stage before you go into the building stage and then I find that planning to be very efficient because it's ⁓ me, LLM, Dan: Mm-hmm. Nik: Then it makes me so as an educator, but also, I'm doing my own design work and ⁓ I can this sometimes like I feel sometimes when I let it do too much and I feel ⁓ ⁓ the difference surrender offload. I definitely am doing a decent amount of offload, but partly because I feel like I have an ability to control it and because I feel like at least when I'm working with it, I have an interface to check and to. Thomas Groendal: And then a bunch of context that I have built up in a GitHub repo product brain, which has a bunch of canonical documents in there. Those canonical documents are very carefully managed by me. And so that's become part of my job now as product manager and context manager. So I have ⁓ one document all of our assumptions are one document with a description of all the features. And nothing goes into those documents without going through me. have an agentic workflow that feeds me new updates, one little assertion at a time, and I'm very strict with it. I'm sort of the context librarian. using that when a new feature comes up, like, hey, we should add a new button here that does a new type of search. The first step that take now is I sit down with Product Brain and I just start planning. Nik: kind of make sure what I'm getting back is actually reasonable. And so this actually brings me to the second point, which was how you design interfaces that can lead you more or less to say surrender? Actually, in some ways, a chat interface is like, well, you're just giving me answers. Whereas other interfaces might potentially lead you to less cognitive and actually push you towards more cognitive engagement. Thomas Groendal: And then I take that document hand it off either to a prototype, which into an S3 bucket for everybody to play with, ⁓ or let's say, in a way, I would not have done a handoff before. But now it's so cheap to do rigorous documentation. And I've already all the energy to build the product brain to be rigorous that we're in a weird way both going faster and doing more documentation. Dan: there is a problem in here that is like, how do you start to steer people more towards cognitive offloading of things that aren't very meaningful for them or just kind of these rote tasks ⁓ that are just easy to put off. But how do you keep meaningful piece? How long did it take you to set up the product brain? and not fall into the cognitive surrender trap. And yeah, I think there are different interfaces that can do this. Some of my students have been working these concepts around and AI and what happens when you create things that are unfinished that the AI like basically to finish them ⁓ or to even them unless you start to Thomas Groendal: I say it was about two or three weeks. It might have been four, I don't really remember because it's all kind of a blur. Dan: Mm-hmm. Thomas Groendal: was sort of right around the time when I personally first did like Claude code in my private life and co-pilot in my work life and kind of had to climb all those learning curves simultaneously. So it was a painful couple of weeks, fun, but painful. And the, the ⁓ way structure a product brain is going to be very different for whichever product or. Dan: You give it some kind of direction. And then rather than doing it for you, it actually says, this is why I think this thing is wrong. do you agree with me or not? And you can say, you can override it and be like, nah, nah, I actually do like that. ⁓ Or I think that you've a valid criticism, make that happen. And so think there are definitely Mm-hmm. Thomas Groendal: you know, maybe it's a nonprofit or whatever it is that you are sort of managing, figuring out like what are the canonical documents? Does it make sense to have multiple feature documents or do you just want one document with all the features? How do you want to group them? What attributes should be recorded for every assumption and kind of understanding doing, learning by doing through that process. And then ⁓ once had kind of settled on a Dan: new paradigms that we could be doing that are Trying to keep people basically away from this cognitive surrender. And again, when Chris Noessel was here, we talked a lot about cognitive forcing functions. How do you put those speed bumps in those moments of friction where rather than just blindly accept something, have to like make it this, have to, you're like forced to make a decision about it rather than, I'm just going to do all this for you. Here's it really. Thomas Groendal: that seemed feasible. There was about two or three weeks ⁓ just grabbing every marketing doc from the archives all the way to the present. feeding them into an inbox for ingestion and then tuning that ingestion agent to be looking at the right things and then, you know, the routing tables so that all those assumptions go to the right places. And now, now those documents feel fairly stable. They get updated as I learn, of course, but that was after a giant burst of ingestion tasks. Dan: pushes you to make a human decision about something. And maybe some of that is, hey, here's why I think this is the right way to go or the right thing to do. Almost in the same way that navigation systems do that now where they're like, well, here's ⁓ the route that I picked for you. You could take this other route, but it's be three minutes ⁓ slower. Which one do you want? And you're like, okay, Mm-hmm. I understand your reasoning. I'm going to go with this one. You can still choose the other one. You can still override it. You can still say, no, that I don't, I don't want the way that you think is the best way because I know better or I know at this time of day that route's going to suck. Nik: I'm curious, how did of your AI usage start within the company and the group? Is it something that everyone's been doing in a bottom up and you're now forming these processes and maybe yourself as a manager? are really solidifying this or was there some kind of top down like, everyone, we should try to figure this out. I mean, everything's experimental, but yeah, I'm curious how it got started that everyone applied things in your org. Thomas Groendal: CACI writ large is a very large company. Dark Blue Intelligence Group is a two pizza team. so there's some ⁓ differences there. CACI writ large has a community of practice where information can be shared and all that. They have their own policies that I think are mostly at this point around. Nik: ⁓ I ⁓ really like how you bring in this idea of the multiple options routes, you're right, I do. ⁓ At least when I do this, often will look at that and typically I'll just pick the one that is recommended because that probably is fine. But then every once in a while you do say, actually, I know that given this time of day, by the time I reach a certain point and found like a long drive, this area is going to get bad. I do want to take this road or hey, this, this road is just an easier drive for me or there's just less potholes or something. and that sharing of options I think is really simple way. Thomas Groendal: kind of like security, like our customers are very sensitive to how things are done. And also just folks that work at CACI to try to get good ideas from one place out to other places in as much as you can share things when one person's project might be confidential or something like that. very different from what we're doing on the team, which... Nik: to basically push away from this surrender into a, you've offloaded the task of finding a route. I don't do that anymore and I'm fine offloading that because the computer is really good at that. But it hasn't. Thomas Groendal: try to be pretty bottom up. Usually that looks like somebody spearheading something, somebody else feeling uncomfortable with it, that leading to a conversation. And then our team working agreement in the sort of traditional agile sense is actually a GitHub repo. And we have a rule there where it requires unanimous to put something else in the team working agreement. And so when we run into something like Nik: set me to always accept the first option and this ability that I am still taking agency and okay, what route am I going to take? And I think we can do that pretty easily in a lot of these systems, like ask them to say, please give me three options for this. And then I'll look through that. And that's already a way to start fighting sort of the pure surrender. I mean, it's funny, I was working with Claude on something the day and Thomas Groendal: this new workflow where you can just prototype not only one, you could prototype three versions of the same thing going different directions, push them all up into working versions of dark blue and then compare. Again, it's real easy to start injecting risk because you're moving too fast. And so we sorted out the rules in the discussion. pushed up a summary addition, pull request to GitHub. Nik: I don't even remember what it was, but I remember the first thing I just for some reason hit the like redo button. I think it was like an accident. I just and it re-answered and the answer was different. It was and it was just as confident and I was like, ⁓ my gosh, it like it totally kicked me out and reminded me that like, yes, wait a minute. This is just word prediction machine and not actual like actually reasoning. Thomas Groendal: that summarize exactly how we want to go about kind of protecting ourselves from moving too fast and breaking too many things. And so now that's part of our policy. But it's definitely more ground up, more grassroots on the two pizza team level ⁓ with some sort of rational oversight coming from our corporate supporters. Nik: And I was like, ⁓ man. so actually one of the things that I've been thinking about has been how do I put in that variety and basically give me options to then work with. And I think as designers, right, that's often a very natural way for us to work. Like we often are like, let me look at a couple of things, if I'm doing a logo design, let me actually look at a couple ideas that I have or, Hey team, come up with stuff. And then let's actually make a reasoned decision from there to move forward. Dan: it sounds like you are still the hub for, things coming off the and then still being then kind of built by design engineering and you kind of together. It's not like they are building things and being like, look, here's this other that I made. Nik: And. Dan: ⁓ I definitely that this is one the biggest areas of exploration that we're going to have to figure out in the next couple of years ⁓ or ⁓ yeah, we're going to end up with these hollowed out tasks hollowed people who don't know what they're doing and why they're doing it. ⁓ That's doesn't sound like that's happening in your org. Thomas Groendal: Yeah, and I've never been much for handoffs. think handoffs are kind of one of the worst things in software development. It's better to give an under specced story and make them talk to each other or make them talk to me or make me talk to them than it is to spec out this beautiful 125 page PRD and then expect that somehow you were right about everything and that anybody would be willing to read that kind of monster. Dan: That just feels that just feels dark. Nik: Yeah, I agree. one of the ways that we also saw this was from Adi Osmani, software engineer at Google, ⁓ working on Google Cloud and Gemini. They had an article in a post where referenced the ⁓ Wharton article. And they actually had a really cool list in their article. We'll link to it. Thomas Groendal: The difference is velocity, the lubrication from AI means that you just need to be a bit more intentional to make sure that the directional energy is aligned. Nik: on engineering moves that resist surrender. Now this is really specific, I think, for software development, but I actually think that many the ideas here could be leveraged by all kinds of people working in different areas. So these include things like verification ⁓ as a exit criterion. This is like any ⁓ agent completed task to concrete evidence. And so this is about saying, you've got to have evidence to say that this is ⁓ good. Dan: does everyone basically have their own ⁓ really tailored to them workflow that works really well for them? Like for your product brain. Nik: Here's what the evidence is that suggests that this is done well. Another one that I thought was interesting was anti-rationalization tables, which basically means sometimes agents will skip ⁓ tasks, like they'll lazy ⁓ in the same that like a person might get lazy. And so you basically have rules that say, no, no, no, you can't skip this. Like if something says, well, that's too simple to need something, you say, no, no, no, you still have to. Dan: I mean, it's set up for you basically, right? I mean, it's got your agents, your way of working, your way of thinking. I mean, it doesn't sound like you set up so that like, hey, if you left the company, someone else could come in and plug right into the product brain or am I wrong? you Nik: do this and then you push it. Another one that I thought was good also just ⁓ ⁓ that So basically getting agent to basically argue with the answer. so using that, and actually I find that super revealing because when I'll do something and then say, okay, let me just open up a new thing, hand the output from... Thomas Groendal: Yeah, I've been pretty intentional about eventually wanting everything to be bigger than me, right? You shouldn't be irreplaceable. And I think a good heuristic for that is everybody brings something very specific to the table, whether ⁓ the engineer or the designer or the product manager. Nik: one session literally could be like, say in Claude, one chat session, I'll just literally give it to Claude in a new chat and say, argue with this or the flaws in this and it'll do it because again, it will so dutifully. But that can really push to say, ⁓ yeah, all right, I gotta ⁓ kick out of just accepting this. ⁓ There's other more ⁓ things here, working smaller, ⁓ but more sort of friction. Thomas Groendal: And generally that specific strong point is something that applies in other domains often more or less, the amount of overlap between product brain ⁓ and security engineer isn't that high. The amount of overlap between product brain and the designer, for instance, or sales ⁓ or marketing roles quite strong. Nik: to things. these are the ways to make these assistive rather than fully automating technologies. Thomas Groendal: And so that's where those elements of product brain is all out there. Anybody in in dark blue intelligence group in our private repo can go open up. Copilot in this case in a workspace in that repo and they inherit most of the skills. They don't necessarily inherit the integrations But inherit most of the skills that are built into the repo They absolutely could do the ingest workflow Although I have made it for Bowton until I you know Kind of have fully mastered how it should be and all that sort of stuff like that but for the first place that product brain really started to Dan: listened to ⁓ the big technology podcast with chief business perplexity and He was saying that they have a new agent out that is all about I think called final check and the idea is that yes It does a final check of the work that it no matter what you feed it. It's like looking for ⁓ Thomas Groendal: yield benefits was on business development and growth. So hey, the FBI has this request for a proposal. we have like 24 hours to turn around a nine page document. Well, this is perfect, right? ⁓ It knows. ⁓ testimonials that were forgotten in the marketing archive that, ⁓ people who write those documents now don't have top of mind. It churns out a first draft. That's a solid B minus. It's much, much better than an out of the box, throw it in chat, GPT. And we often can't do that sort of thing anyway for security reasons. And so it really started to yield good benefits there. Some of that is user dependent. Dan: errors is looking for factual errors is looking for logic errors is looking for whatever that you kind of put it in there. yeah, maybe that is definitely a way to start doing some of these things. I wonder what that would be for, for like design rationale, what you were talking about, like, why is this like this would be interesting questions that a Thomas Groendal: And ⁓ data lives in Elasticsearch. So lives in ⁓ a a database tool. And I a lot of documentation in there about how to use our API. A lot of documentation the product brain repo about what good trade craft looks like, what criminals are up to, what patterns to care about. And I started doing things ⁓ we have this German customer who. Dan: Art art director or design director agent could hover over someone's shoulder. Why is this like this? Why do you put that button there? did you do it that way? How come there's three screens instead of two? Nik: Yeah, we. Yeah, we've explored some of these ideas in our research, sort of asking questions. And it's interesting when you do that, you know, sometimes it really does push people to start thinking, you know, okay, yeah, this is why this is my reason. Sometimes it helps them, especially when something is maybe amiss ⁓ to themselves. And it can be, it could be great. I mean, it really can help people catch themselves. ⁓ We've this, some of our work is a little bit older, but we've done this in the space of like mechanical engineering design. But I think many of the ideas Thomas Groendal: Let's just say they're into model airplanes. It's the new threat is model airplanes. Like this is a made up absurd example. But if I that and we need to pitch them, I have methodology documents. You know, like I think April Dunford's method methodologies for pitching are awesome. Dan: Hehehehehe Nik: transfer over to interface design, product design, service design, just a lot of it is just thinking it through. But sometimes there's some really interesting things where people will even if they actually should be solid and like they've created good design, they should be solid. It'll cause them to lose confidence in themselves because they're like, ⁓ well, the machine thinks I've done something wrong. Thomas Groendal: And so, in fact, her book I think is called Obviously Awesome. But I have advice in there for Product Brain for how to pitch based on some of her thinking. And so it first was exposed by people saying, hey, I have to write this RFP. Can you give me a starting point? And Product Brain doing that recently, we have shifted that to a Slack app where they can just ask dark blue. Nik: you need to actually ⁓ the thoughtful for why you're doing things and be able to push back on any Because again, you can always make a critique bot that will just rip you to You can make it do that. So you kind of to be able to both listen openly and be able to defend. Thomas Groendal: it's doing great work. It's absolutely a well-trained, narrowly scoped version of that. And then the actual product brain itself ⁓ all of its possibilities tends to be with a kind of super power users, the people who know the repo really well, who know what all the canonical docs are, know which canonical docs to point. Nik: the decisions you're making. think like, oh, well, I'll just like, there's one simple trick and all of a sudden I'll not be cognitively surrendering. And it's like, well, it's not actually that simple. And so this is where people need to really think about like, how is it that you work with these things and what's helping you? and and you know, those methods and processes to support you in that. Thomas Groendal: the AI at, you know, if you're going to talk about this, you probably want to look at the pitch methodology and over here there's a document on, I don't know, the history of our data collection. So you definitely are going to want to check that now answer these questions. And so that sort of democratization of tooling works that way for everybody. So for instance, Jason Ogle has a that he the engineers ⁓ own, which our system. Dan: Yeah. Thomas Groendal: And that is not something I've spent a lot of time on. That's not my strong suit. But ⁓ benefit from it because if I am going to ⁓ something, I know the name of the repo, I know where to point it at and say, go make sure you are doing, ⁓ be Jason Ogle about this. Make sure you get it right. And that's really helpful. That saves us all a lot of time because again, that's valuable context that comes from human taste. Dan: do you deal with things are, I want say outside the organization, like, I'm sure you have a lot of like legal issues and stuff like that. How are those baked into this system? Do you have other and product brain that are all like, here's legal and compliance, ⁓ that kind of stuff. Thomas Groendal: Yeah, the vast majority of that for us is really around data collection. So there are competitors in this space who ⁓ maybe in countries with much more lax privacy laws that are indifferent to the terms and conditions for platforms like let's say Facebook or something like that. And we don't hack. We do not go violate the terms and conditions of use for major social media platforms. We will collect on a dark web platform that has no legal standing to begin with and is just doing bad things. so all of that is, not of it, but a great majority of it is documented in product brain. And that's really helpful. Dan: All right. our second article is Claude code is not making your product better by Ethan Ding, who is the co-founder of text QL. Ethan's argument here is a lot of people are telling you that coding agents have changed Whereas the actual Thomas Groendal: you're presenting on your data and you want to say like a good thing to say about dark blue is that our data is sourced by by people in the US doing legal things that's really important for what's called chain of evidence you go out and you catch a bad guy but you violated the law halfway through the process you might lose the bad guy Dan: story is a little more nuanced particularly engineers are telling you that hey it's a lot more complicated than that and i liked about this article ⁓ was about relationship between the organization and the quality of output ⁓ he is that if you have Nik: Tom, I'm curious, these systems that you've put in place, how has your relationship with your team changed? Like, do you feel like you're significantly differently than you were before ⁓ of these ⁓ systems, or has mostly ⁓ led to acceleration of work, but similar workflows? Thomas Groendal: to get to you. Dan: a really strong organization with people with he calls them taste makers. I'm going to call them people with good product and design judgment. That that is that's the real bottleneck that high product quality improvements. They're not bounded by Thomas Groendal: It absolutely has changed in a couple big ways. I don't know that they're always good either. ⁓ ⁓ just the amount of work is exploding. I don't know that that's true for everyone. In part, that's because I can prototype, I can go faster, I can go as fast as I want. So if there is urgency around a theme that we could not have even started exploring before, ⁓ Dan: how fast you can code, how fast you can put out prototypes. It's all about how fast you can come up with ideas that are that are good. that, ⁓ spending tons tons of tokens, ⁓ that's not really metric for tracking product quality. And that Thomas Groendal: I can go prototype now, so there's always more work to do and there was always more than enough work before, but the sense that I ought to have done it already. is definitely something I think we have to think about in terms of sustainability, quality of life. I cannot work at the pace that I've been working the last month for the next six months, which is fine. I mean, this kind of thing always happens when there's a big opportunity in the market, there's a new technological development, everybody sort of scrambles. But I think as a society writ large, there's a strain. ⁓ And can feel it in my non-work peers and loved ones who ⁓ are through kind of similar things. Dan: really product quality emerges from this idea of building less. hear about these big companies where it's like, well, TokenMax do as much as you possibly can. build as much stuff as you possibly can and that's how you're evaluated and ship stuff that's how you get promoted. Thomas Groendal: and suddenly they can solve all the problems that work with, know, Claude code or whatever. they're all working mad hours and going crazy fast. So that's kind of maybe the negative side of things. The, on the positive side, two big changes. One simply, again, that, that velocity means that because I can prototype, I'm tighter with the work. And I was already probably on the far end ⁓ of intimacy with development even though I'm not a coder. Nik: Yeah, this is something I've been thinking about incentive structures, right? Oftentimes within a company, ⁓ are promoted, ⁓ you know, rewarded for shipping, ideally great new product features that help to improve the customer experience, that help to gain more customers, that help to get customers upgrading and paying more. mean, you know, that's the reality of the business. And I, I think when you do that, well, ⁓ it's awesome. love when I get new... Thomas Groendal: just personality wise, I like being in the details and I've worked in software shops before where the product manager sits on an ivory tower somewhere and issues a PRD and I can't quite remember their name. They have POs to take care of that menial work and that's just not my preference. The other side is... Nik: features that solve problems because a team has figured out a need that I had when working with their tools and then they shipped a solution to that. The comes in though, whereas if the incentive is shipping and not the incentive being ⁓ shipping that has a demonstrable positive effect on the quality experience of the product, then that can get abused. Thomas Groendal: that I am doing a lot of prompt engineering, I'm building out an eval suite. And so there's something called eval driven development, which is essentially ⁓ code driven tests ⁓ and LLM as a judge tests. Nik: I like the use of term token maxing, this thing where companies are generate as much as can. And I think that that's going to just add fuel to this fire. You're going to get a lot of product features. Stuff's going to ship faster than ever. And unless you have something really checking on the quality side, I don't know if any of it's going to matter. If anything, it might make products worse. Thomas Groendal: Anchoring for each agent around known anti patterns building a data set where you know whether this should pass or should fail and then using that to judge the LLM as a judge. So you're judging the quality of your testing and then building out this whole suite where you can know whether a model change or a prompt change just broke your software rather than hoping. And then taking those anti-patterns, those bad behaviors that don't want your tool doing and then ⁓ removing them until your tool is great or good. And we're to mature into a full eval-driven pipeline ⁓ for lot of our AI products, is complicated because we do really complicated things across unstructured data with a ton of different diverse data, a ton of different types of Nik: And you know, and it actually may start to degrade product quality and then what you'll end up with is people saying well this product has gotten worse and so now I need to go and vibe code my own alternative to it someone's gonna do a startup and then they'll put that out there and interestingly enough it might be great because it's More streamlined it might align better with maybe your needs, you know, you know, it's funny. I think sometimes New products that aren't bloated feel great when they align well with you Thomas Groendal: and so answering questions like who is this bad guy, that's this messy stuff in in a pure human ⁓ world. Pulling all together just made me much more a of the team rather than ⁓ standing athwart or, aloof or of the team because I'm sort of driving the evals process. Nik: then more people get on, they have more users, and then you try to satisfy all your users, and then you end up building all these things, and eventually everyone starts to go, oh, hi, it's doing all this stuff that doesn't remind me of that first, hey, this streamlined, great experience. So yeah, I think that this is something that maybe the designers out there, you should be thinking about, because again, I love building, we love to build, and our goal is to build, is to create new things, but. Dan: we talked about last week on the show that there was article about how. ⁓ Nik: Yeah, at a certain time, now it's a matter of sort of thinking about, do you have those checks on quality? How do you regulate? Dan: this was fracturing a lot of teams because they were working very siloed where one person working on this one person one person working on that and then they they stopped actually like talking to each other and so the teams were some kind of strategic cohesion and morale was down and all those of things doesn't sound like that's the case where you're at though Thomas Groendal: I will definitely tap on one of those though, which is you look away, you look back in two minutes and it's like everything is different. What was the plan again? Where are we? What are we doing? You know, that is a definitely an impact of so much velocity. And I don't know that it's a positive one. I don't think it's always going to be that way, but it's definitely that way when. I am learning about evals at the same time I'm Vybe coding a dashboard for evals and the role of, in this case, not the designer, but the SME on the tradecraft side. So our, of my colleagues, data manager is ⁓ retired law and ⁓ role is often whether it did a good job and or it did a bad job and fell into one of those anti patterns and ⁓ sort of understanding the structure of that testing suite while the testing suite is evolving and all the tools that display it are evolving is very messy. And so every time he sort of has to go out and do another thing for a couple of days comes back into the office and he has to like start from zero. I think that can be a little alienating and frustrating for everybody. Dan: ⁓ one thing that I really liked about this article talking about people can push the frontier of what good software feels like. And for me, that is Thomas Groendal: we get more mature, think that'll go away. But anytime there's massive disruption like this and processes change, I that happens. Dan: Yeah. It's funny. I'm thinking back to my own user testing and prototype testing ⁓ yeah, you would spend a couple of weeks building this thing to test. You test it for a week and then it would take you ⁓ couple of weeks to refine it And now, man, you can be refining it and have, have the brand new one in an hour practically sort of about what we're talking about here where it's like, how can I reimagine this entire task or ⁓ a separate opportunity space ⁓ previously wasn't there or can combine things that normally wouldn't go together, but really work well together? And I think that is, that is what really certain apps apart because they do something in a way that feels fresh that feels interesting. Thomas Groendal: Yeah, there canonical plan documents that kind of help anchor it. But if every time you're going to touch something, you have to read a planning document and remember where everybody is, that can make you feel left out. It could make you feel isolated. of those would be reasonable responses. Nik: Just as a point of clarification, is your team primarily in person? Are you primarily remote? Are you a mixed team? Thomas Groendal: We are 100 % remote. that adds on top of it. know, on Friday I shipped three different updates to the prompts driving a feature that we're updating right now. And Monday brain kicks in and I don't remember where we are either. It's a lot. And when you're remote and you don't... Nik: okay, cool. Dan: ⁓ how that software worked changed how they felt about doing the task. And I think that is a really important thing. he, we as well mention it. He calls those like the ferraris of software rather than the Camrys of software. He's like, ⁓ Camrys, it'll be easy to make a Camry, but it's really hard to make a Ferrari. And so you really want to be aiming for Ferraris, not Camrys. I know you take umbrage with that. Thomas Groendal: overhear me saying, hey I want to ship this, can somebody approve my PR? You just sort of, it's lost in the the slack. Absolutely I can see that feeling very, very siloing, especially if you're not at the tip of the spear. Dan: Imagine going back in time and saying, I'm shipping prompts, three years ago, like, what does that even mean? What would that even mean? mean provocations? What is it? Yeah. Nik: Well, only because to make a Camry or any very high reliability vehicle is actually very hard and there's a ton of work and iterative work and actually in some ways, the making of a Camry is an exercise in iterative improvement in quality and restraint. Whereas, well, should say, arguably making a Ferrari is hard. The difference here maybe is that Thomas Groendal: Yeah, developed this core feature that I was updating on Friday, maybe 18 months ago. I don't know the exact timing on it, but almost everything we did then was fine then. And now in retrospect, I'm like, whoa, this isn't how you structure things. This is totally wrong. And just even keeping up on that sort of thing is a lot. Nik: Maybe if you look at the metaphor as like lots of people will drive a camera, like only a few people will drive a Ferrari, and honestly only a few people will drive a Ferrari to its full potential. But ⁓ do have sort of an opportunity here to develop software in more niche areas, and that's maybe where things could potentially go. Now this brings up a question though, which is when we talk about, actually they use the example of project management tools. Dan: Yeah, that is a common complaint with almost everyone we talk to is that it's so impossible to keep up with the tools, with prompting, with new things that are coming out that, on top of actually doing the actual job is keeping of tools to do the actual job now. Nik: And actually there's so many of them, which seems interesting. know, partly it's because, lots of features are actually project management is pretty complicated and everyone's different. And actually it's led to a diversification in the field of like, well, some people like Jira, some people like Notion, some people like Trello, some people like linear and people figure out what they've, I've actually been thinking about this. I'm looking at some of my own and I'm like, I don't like any of these. And the hilarious thing is that now I sit down going like, maybe I'll just build what I want that works for me and my project management in Claude. So maybe as a last question for you, if you had recommendations to teams that are out there now, what might you recommend to both the product managers and then also the designers that are working with them? but question that I have here is Does idea of saying I'm trying to create this really great feel and quality? Does that work only when you're creating something new and you're trying to break something open? versus We are to make you know Jira better. Jira is a big big machine now. Jira is ⁓ used by of people and you can't just be redesigning all of Jira Thomas Groendal: I would say everybody's context is different. So a couple of recommendations. One would be in agile software development, retrospectives are a keystone habit. so now more than ever having retrospectives where you ⁓ specific examples, ⁓ real examples of that is changing and you walk through it end to end and say, is this how this should work? How could this have gone wrong? Nik: because it works a certain way. So, ⁓ is mindset though something that has to be in ⁓ startup land as to in large company land? Thomas Groendal: that kind of eye toward risk would be a good thing to do. It's happened twice. in the last month or two as we have adopted Copilot aggressively and sort of vibe coding ⁓ or coding adjacent activities. mean, vibe coding means different things to different people, ⁓ but much faster, much less manually coded software development lifecycle, we have needed to touch base and we have needed to say like, this would be a bad behavior. Do we agree on that? Because I see a risk here. That would be a good Dan: I that often when you are doing a startup, you're building basically a Ferrari for a very tight audience, right? You are, you are at, I mean, ideally you were aimed at something very specific. And then more people find it. And then you start to expand, expand, until you have a Camry, let's say, I mean, until, yeah, you have something that many, many, many people can use and drive because that's what scale is. And think both of those things are hard. And I think that they're just hard at different times. it's hard get that like perfect market fit. Thomas Groendal: behavior do we agree on that because I see benefit here. been that's been really helpful. Another one I think another recommendation I would make is someone should be a context manager. ⁓ don't know if that's going to be a role eventually. Someone should be the librarian of context. They should know where everything they should be the routing table and they should maintain and update the routing table for you want to build a thing in the UI well ⁓ you to know about the design. Dan: And now it's easy some regards, because you can do exactly what you just said. I'm going to make this for me, for one or you can say, just going to, I'm making this for ⁓ one or for one team, for my team. That is just a tool for this extremely specific thing. But when you're trying to sell something, Thomas Groendal: system before you just vibe code some completely non-compliant garbage that maybe is interesting directionally but the trade-offs are not recognized, the trade-offs are not being respected that were made previously. I think a context manager who knows ⁓ where all the skeletons are where all the treasure is and can help Dan: You start by selling it or pitching it to a small group of people and then eventually you hope that that expands and you get a large group of people. Unless you're Ferrari, in which case, hey, we make cars for the top 0.01 % and to hell with mass market. Thomas Groendal: people who are prototyping to template will really make this much more efficient. Dan: It's so funny you say that that's almost exactly the title that Shane Johnson, who's a principal user researcher at Figma. That's exactly what he said two months ago. We were like, Hey, what's the future of user research? And he was like being a context manager. And it's, it's fascinating that there's a whole new We should have asked our special guests this question. since we mentioned product management and product management tools, we have a real live product manager ⁓ on the show today as our special guest. that is growing up that is this combo of product managing marketing ⁓ and research, ⁓ Thomas Groendal: This fits a pattern with AI where instead of AI replacing product managers, or designers or anything else, AI has made things that were too costly to do before. So would I have liked to have had a product brain before I had sort of a copilot, Claude code type tool. Sure, that would have been great. I did not have the time to do the the ingestion. The ingestion was what was amazing, right? It was. Robot goes out, reads your document and goes out and reads the entire Encyclopedia Britannica of dark blue and identifies where it thinks this information should go and makes a recommendation and then feeds that to me so I can just be the taste guy. Yes, no, you have that slightly wrong. Next, next, next, one at a time. That kind of context management is magic, but nobody had time for it before. It was just maybe one tastemaker on design, another tastemaker on engineer, another tastemaker on the product side, but it didn't really matter because ⁓ were still constrained by how much code you could write. Now that code is dirt cheap, bad code anyway, is dirt cheap, having tastemakers that or prevent that from turning into bad product is like, it's just a very different idea. Nik: Tom, thank you for being on. Actually, it ⁓ was super interesting to hear ⁓ from the product manager side. systems you're putting together, it's really cool to hear some of the similarities ⁓ that been hearing now across. different organizations and different people and what they're building, but also some of the context specific stuff that you're doing and how your team works. It's also really cool. Your perspective, I really appreciated it on your discussion of risk and how, I think, on our podcast, I mentioned before, ⁓ we're not fully AI is going to be everything, but also AI is not nothing. And I think that ⁓ you provided a really balanced here. So I really appreciate that. So thanks for joining us today. Thomas Groendal: Thanks for having me.