Dan: Welcome to AI and Design, the podcast where we explore how artificial intelligence is reshaping the world of design. I'm Dan Saffer.

Nik: And I'm Nik Martellero, and we're faculty at Carnegie Mellon University's Human-Computer Interaction Institute. Each week, we break down the latest AI developments, dive deep into topics that matter to designers, and talk with fascinating guests who are right at the intersection of these fields.

Dan: And whether you're designer working with AI or an AI practitioner interested in design, we're glad you're here. And this week, we are doing a special episode on the evolving design process in the age of AI. and I are joined this week by Mike Kuniavski co-author of Observing the User Experience and Smart Things, and wrote article called Design Practice Assumes a World That No Longer Exists. What's next? we'll link to that in the show notes along with several other articles that informed this conversation.

Nik: Mike, welcome to AI and Design. So today we're diving into a little bit of an existential crisis that's rippling through the product and UX world. AI is completely upending how we build software. And the hottest debate right now is whether the traditional design process, whatever that is, is obsolete. Mike, want to start off with the premise of your article and give us a quick overview?

Dan: Yeah

Mike Kuniavsky: Sure. So I've been essentially designing design tools for and with kind of ⁓ AI perspectives for, ⁓ know, good chunk of the last ⁓ couple of years. And as someone who's been doing AI research and also UX research and UX design for a long time, I essentially ⁓ something ⁓ ⁓ actually ⁓ as much about design practice, but ⁓ about the that design practice plays in organizations. Like ⁓ what it that designers are doing in an organization? And I realized that. ⁓ AI appends a lot of that. It essentially calls into question a lot of the assumptions that we have around what deliverables are for, what design processes are for, and what design processes and practices are relevant. And so ⁓ I have very clear ground to stand on to think about this because it's also new. decided to start writing down questions that I have and start writing down things that I've observed. And that was the origin of this article.

Nik: Actually, I like your first question here on a lot of what creating design concepts was about was fast iterative conceptual exploration. ⁓ and getting lots of ideas out there so that you can think through them and then eventually get to the point where you want to create higher fidelity, say prototypes, mock-ups, because it's actually expensive to do that. However, there's now a bit of an argument that AI is making high fidelity free. basically makes it almost effortless to create something that's high fidelity. And so it is not expensive anymore. So how does that change things?

Mike Kuniavsky: Yeah, so years ago, I saw ⁓ essentially wireframing toolkit that made functional wireframes or like functional UI designs, but everything looked like it had been done in Crayon. ⁓ And that explicitly created, as I remember, so that you could show people a working UI ⁓ design, but them know that it's actually not final. So they wouldn't treat it as final. So they wouldn't treat it as high fidelity. And so I think that the thing that was trying to do is that was essentially trying to say, ⁓ we want to the feedback at the fidelity of our ideas rather than at the fidelity of kind of the tool that we have. because our ideas are still low fidelity. And the reason that we want to do that is that we want to develop the ideas at this low fidelity place, this low fidelity ⁓ level, before we move them into higher fidelity. But now, getting to that functional through lovable or whatever is incredibly And so it's all high fidelity. So you go from sketchy half formed idea to high fidelity artifact very quickly. And so the entire reason for why we have low fidelity artifacts, why we go through that process essentially comes into question. Like, do we still need to do that? really, what is it that replaces that process now that we can make every single thing at this high level of fidelity and how does that change the responses that we get? How does that change we think about exploring ideas and essentially like how we evaluate ⁓ ⁓ of our ideas.

Dan: As I understood it, at least as I practiced it, the idea behind those kinds of sketches wasn't just about speed, although that certainly ⁓ the time was very valuable because you could crank out wire frames very quickly and just do them and show them off. But it was two things. was one so that you, the designer, didn't get fixated ⁓ thinking about the visual design at that point, the skin. two, so that the stakeholders didn't get wrapped in all that too, where they were like, well, I don't like blue. And you're like, well, this isn't about the blue. This is about figuring out whether this is the right UX flow through the product. And ⁓ seems those things are still there, but... Then again, I don't see anyone using these AI tools to crank out wireframes. I see people using them to crank out the full thing. I'm wondering now, it like my nostalgia or is that value not really there anymore?

Mike Kuniavsky: So I think that there's this interesting thing about getting really attached to design literacy. Like we as designers a kind of literacy about what good design is and why certain things are there and why they look a certain way and why they're arranged in a certain way through practice, essentially. Like we learn design literacy through ⁓ design practice, you know, we learned to read by learning to write design. And when show design artifacts to stakeholders, they don't necessarily have that design literacy. And so the entire process ⁓ of and showing or showing kind of three different ⁓ potential directions and letting people choose among them and then kind of guiding them from that. And all of these other practices that existed, personas get people outside, to get stakeholders outside of maybe their preconceived notions of who it's for and who this is going benefit and how they're going to use it. ⁓ All those practices existed because there was gap in essentially the literacy, design. couldn't show a final artifact even when we could make one because we didn't want people to misread it. And so a lot of those practices were born because of that. But now, essentially, the technology has done an end run around all of that. And I think that design discipline has to figure out how to deal with it, has to figure out what to do now that it's done ⁓ this run. And everything can be this high fidelity deliverable. I mean, the other thing that I think is happening is because everything is high fidelity it's high fidelity because it embeds all these ⁓ practices from before, because that's how the models have learned. They've learned from everything that's before. What they're doing is they're essentially generating kind of very generic design solutions that high fidelity. And that might actually be also constraining the end product. That might also be constraining what the stakeholders who don't necessarily have the same level of design literacy that experienced designers have, how they read artifacts. So I think that there's several things that are going on at the same time.

Nik: You know, one of the things that you made me think of too is the fact that when we work in certain media, this also shapes the way that we think. while it's amazing to create things in a super high fidelity so quickly, and you can do a lot that, I do wonder what that does ⁓ to ⁓ the way in which think. actually reminded there was a ⁓ thesis out of my former PhD at Stanford's Center for Design Research, was by Jonathan Edelman, which was exploring how teams make radical breaks. And one of the components of this was actually how teams work across different low and higher fidelity media, and the fact that they think differently with those media. This was looking at things like foam core models versus sketches, and also the translation of ideas between sort of representations, where now it's sort of like we kind of just have this translation from thought to prompt to then, mock-up, yeah, I'm actually wondering if that limits how we think as designers.

Dan: Does it limit or does it change? I know we've talked about this before where these things that are just generically that come out of the models for a lot of design, basic design problems, those are fine. You don't need super high. thinking through very complex work streams and stuff like that for a lot of times it is like, well, this is kind of a solved problem. so the models is fine. So, you know, if it, if it is ⁓ limiting cognitive ability there, not big deal, but it is that other, let's say to 40%, where you actually do have to really think through things that just accepting the output of the model, think is extremely limited.

Mike Kuniavsky: I think that essentially if what you are asking one of these tools to do is something that is essentially exactly like there are a million other things that already out there that are well known, it will produce something that is probably acceptable and reasonable. But if you are asking it to do something that deviates from those examples and increasingly with time things will deviate from all of the things that were carefully hand designed up to a year ago. As those new ideas and new projects and new questions start coming out, the design systems that these models have ingested will increasingly be less adequate well, at least that's my prediction, for designing those things. But I think to your point, There's a question, at least for a lot of stakeholders, they only care about the end product. And for something that is a very typical kind of output, a very familiar thing that they just want their own spin on, they probably don't even care about entire process. ⁓ They want the end product. They don't need to know. ⁓ And they don't want to pay for it. the entire process to make a new ⁓ to that. They just want to get the thing that they're already expecting from all the other things that they've seen, you know, maybe now in blue. ⁓

Dan: my friend, Andy bud cause this, ⁓ the designer theory where designers are often playing chess while our stakeholder business people are playing poker. They're just quickly, ⁓ they just want to get out and, ⁓ ⁓ show it in the ⁓ ⁓ just get it out there without the planning ⁓ strategizing that designers like to do.

Mike Kuniavsky: And I think that this is another part of what I was thinking about when I wrote that piece is that I ⁓ thinking like, we really have to stop essentially fetishizing the old practices. We don't need to do those things that we used to do anymore. ⁓ are the new things that we need to do? I don't know, necessarily. ⁓ have some ideas, but I don't

Dan: you do have an argument in the piece that all about accountability with AI artifacts that you used to be able to say, well, we've done a lot of thinking, here it is, we stand behind this because we've spent a lot of time putting together this case

Mike Kuniavsky: Yes.

Dan: now there's case because you haven't put any together. You're just pressing the button. I'm gonna take back. You may have put a case together. And in fact, I think when we talk more about process and what still needs to be done, I think that upfront thinking definitely helps ⁓ you write the prompts, helps you evaluate what the models are generating, all that kind of thing. but there's still gonna be these big gaps because you haven't done the thinking.

Mike Kuniavsky: All right. I think that like with a lot of products of current AI that were trained on of existing examples, we're really surfing ⁓ on this ⁓ of social that are baked in when we see an artifact that may no longer be true. And when we see a thick report, we're... still trained to expect that that thick report represents whole bunch of thinking by somebody, a whole bunch of work, a whole bunch of essentially ⁓ to understand various aspects a problem. that the thick report is essentially the documentation of somebody's thought process and understanding that they are sharing. as a demonstration and also proof that they have done all this work to understand it. We're still expecting that that's the case when we see the thick report, but there's no guarantee that any of that thinking has been done by the person that's giving you the report now because it can easily have been generated, which, and I make this point in the article, that doesn't necessarily mean that the things in the report are wrong. or untrue, it does mean that it's no longer an indicator. It's no longer a proof point that those things have been thoroughly examined.

Dan: Right, it's the McKinsey effect. The old McKinsey effect, here's the big report, and the new McKinsey effect. Well, here's a big report, but probably AI wrote a bunch of it.

Mike Kuniavsky: Yes. And to some extent had always been true. Like tangentially, if you remember the Pepsi logo refresh pitch that had gone around a while back that was, I suspect it was actually real, but it was fantastic BS. It was completely malarkey. that was designed to essentially look like they had done all of this thinking around the logo redesign. I've ⁓ believed it was true, maybe it's a hoax, I've always believed that it existed because they expected that no one in Pepsi would actually read it. They would just kind of flip through it on their desk in the executive boardroom and they'd go, yeah, okay, sure. And so that kind of thing had always existed, but at least it took effort. It's like somebody had to make that giant pilot BS justify that design. Now there's no guarantee that anyone's ⁓ like put that much effort into it. So we have to rethink kind of what the role of the artifact is that gets handed over and how people should respond to it or whether it should even exist at all.

Nik: one of the things about design processes as we often teach them and as we often talk about them is in the creation of intermediary artifacts that ideally are helping get us to good final design. At the end of the day, we're creating a product. That is what we're trying to do, a product or a service, but that's what we're creating and then that goes out to our users and customers. Everything in between is simply to hopefully help us get there, but do we need rethink the value of those artifacts ⁓ if we can just race towards things that look like products that can then go test and then can simply just get data on if it was good design or if it was not.

Mike Kuniavsky: mean, and you can test things with synthetic users too, right? And so then there's the question of like, are you actually testing when you're testing with synthetic users? And is that a realistic situation? And in some cases it will be, but in most cases it won't. I think the role that those intermediary artifacts played, they had several roles. ⁓ One of which a communication role, communicating to stakeholders. But the other one that I think is still absolutely relevant is for the designer or the person who's envisioning the product, for them to understand the design space, for them to understand where they're working. And so for that, the way that the older forced us to work, was that they forced us to break things down into pieces and explore individual pieces because we just didn't have the resources to explore kind of the full height of a set ⁓ ideas. Now we do, ⁓ the is how do we keep the valuable part of that practice where we learn things perhaps by isolating ideas into compartments and then individually testing compartments. And then we fuse those into larger units that we can then combine into ⁓ something that ends up looking like a product. And so there's the question of like, how do we do that now that we have all of these You know, I just actually, last week as a test, I had an idea for this product that I'm building. ⁓ And was like, I don't want to build it into the product itself, because I'm not sure it'll. I'm not really sure what I'm doing here. And so I just made a mock-up of it complete with like working AI models and complete with like all of the kind of cloud services. I made a mock-up of the idea in lovable in, an afternoon. but now have this thing and I'm like, ⁓ yeah, ⁓ I think actually, you know, a pretty good idea. Now I have the problem of like, okay. Do I want to now integrate it exactly in this form with this code into that lovable generated into the thing that I'm building, or do I rebuild it completely from scratch? do I go out and test it and ⁓ test just as one piece of it? and ⁓ I'm not sure. ⁓ not sure what the ⁓ practices in this situation, it's an interesting. proposition, but it's really different from the way that things done out of necessity earlier.

Dan: Well, I also want to dig into that because did you have upfront other than ⁓ idea? Did you have sketches? did you have task flows? you have a journey map? Did you have ⁓ anything? Did you have any that? Or was it just the idea? And then you're like, well, let's build this with just the idea and ⁓ see what happens.

Mike Kuniavsky: So for me, specifically for this, had an idea that ⁓ I wrote down ⁓ in paper notebook because I'm still old school that way. then ⁓ I kind of a classic design thing. ⁓ I and a friend of mine, I sat down for a couple of hours and I was like, let's work through this on paper. Like what happens first, what happens, let's sketch out what's going on. So I'm building design tools. So for me, there was this question of like, what is, what is going on in the designer's mind at this point? Okay. Now we do this. Okay. Now what do we think has changed? Now we do this. Now what do we think has changed? And so we kind of talked through it for a couple of hours and sketched out a bunch of stuff in my notebook. And based on that, I sat down the following day and I was like, okay, I'm going to, I have an idea of the technical components. Cause I've been working a lot with AI models. So was like, I think that I can get these models to do this. And I think that I can get them to produce ⁓ this kind of result. And then I sat down with the sketches that we had made and I was like, okay, I think that this is roughly the flow that we want. so I'm just gonna, essentially prompt it iteratively and get that thing that we were talking about yesterday to work. And so now it works. like I said, now I'm like, okay, what do I do now? And so, ⁓ and it technically works.

Dan: Right, but you did some design ⁓ You it through, you had flows, had sketches, you had all those things. and I've heard this story before. I've heard it other people before where they're like, yeah, I've done some stuff up front so that I don't spend forever.

Mike Kuniavsky: Yes.

Dan: Tweaking outcomes because I've already done some of the thinking so that when it does produce something I'm pretty sure I know how to Judge it because I know what my intent was and know kind of where I was going with it and ⁓ I can then tell like ⁓ is is quality is this is this good all of that required having that

Mike Kuniavsky: Yep.

Dan: up front thinking and not just the raw idea to stuff into the into the model

Mike Kuniavsky: But I want to say that of me wants to reject this idea that you can just prompt and test and prompt and test and prompt and test the that certain people are doing it. But I don't want to wholly reject it because a lot of people are doing that thing where they're like, OK, I have five customers for my four of which might be my relatives. I have five customers for my thing. And I have a discord and this person asked for this feature. So I'm just going to build it today. I'm going to use lovable and just, I'm going to build it. And then I'm going to see if they like it. And if they like it, then I keep it. If they don't like it, I change it. And then, yeah, this other person said that they're having this problem. Okay. I'm going to rebuild this thing. And people are genuinely spending that is the design philosophy that they have. And they design their products that way. And I don't want to reject that kind of wholly out of hand because there might be something there in terms of new design practices that are very user centered in this very way. But, but I think that that's the, that's kind of an alternate way of going from the maybe more traditional way that I did it.

Nik: And this is interesting, Mike, because this idea of this kind of constant feedback out in the open, building the open design process is something that I think we're seeing more of. But there have even been some examples of this, at least in smaller cases, that I've seen. like Tesla, remember, ⁓ I have to find the story for this, and I don't know, it could be apocryphal, but ⁓ was this really cool story about Tesla had... lowered their cars. have adjustable suspension and they did an over the air update and they lowered the cars because they got better aerodynamics and thus they improved the battery range basically for free. However, there was a user who was like my driveway is steep and all of a sudden my car scrapes when I try to get out of my driveway you know I think they tweeted about it or something and then someone was like whoa okay well here we'll just make it user configurable and then they put a interface within the center stack where you could ⁓ change height. And so the idea is that you could kind of choose, you know, you weren't set with always the car being so low. And this idea that like, you can just do that, you can make those changes, you can make them rapidly, you're listening to your customer. I mean, it's actually like, in some ways, it's almost this very oddly actually it's an almost very traditional form of design if you think all the way back to John Chris Jones his description of design practices that have changed.

Mike Kuniavsky: Okay.

Nik: So I'm speaking about the first part of design methods. And the first real discussion of design is this idea of craft evolution that you went in and you sat down with the cobbler to make your shoes and they would take your measurements. They make custom shoes for you. said, ⁓ know, something's rubbing wrong. They come in and adjust it. You know, you're making a wagon and you're getting wheels made. ⁓ hey, I, you know, my roads are worse. Can you try to do something? And they would basically evolve designs. And in some ways, I actually wonder if you're describing

Mike Kuniavsky: the Hmm.

Nik: bit might get us back to this craft evolution form of design because you can just do something where you can listen to your customers and your users and update things in extremely fast iterative successions.

Mike Kuniavsky: Okay.

Dan: But it might also get us to a world where there are 10,000 settings, one for each customer, then the product becomes completely unusable for everybody.

Nik: it has a built-in AI that is understanding your preferences and tuning everything to you. mean, but no, Dan, I think that you raise a really good point, right? Is that actually having... Customizing everything for everybody means that nobody has a shared experience, no one can potentially help each other, and then yes, if it's done poorly, everything is just going to be unusable. But I do wonder, right? It's interesting and there's a tension here, but I guess what's funny is that we talk about, actually, this whole episode's on the design process, and yet the design process has gone through major paradigm shifts before, right?

Mike Kuniavsky: Yes.

Nik: Craft evolution turned into design by drawing. And then design by drawing turned into, well, at least in John Kirstjong's perspective, sort of the design methods movement, which was all these different methods. And that's the other thing, is that I got a whole book of methods.

Mike Kuniavsky: Right. And think my question is for each one of those methods, there was a context in which it was a ⁓ relevant solution a set of problems. And so the question is, what were those methods and what are the design methods that we have inherited? What are they solutions to? And ⁓ we have better solutions or more effective solutions, ⁓ your point, that are available to us by using all of these AI tools? Because I think a lot of those methods ⁓ existed because was a set of implicit constraints that is no longer there. either financial constraints or time constraints or effort constraints or communication constraints. And this is, and I don't know if, like, I think this is where we have to re-examine our to figure out, why is it? You know, it's like the analogy that I have right now is that, you know, if you, I always think of like contracts as a form of organizational scar tissue. Like every single clause in a contract represents something that had gone wrong for somebody at some point. And then the contract is there to try to it or like at least ameliorate the impact of that, of injury going forward. And so like everything in a contract, there's a story behind every clause in every contract. And so ⁓ for it's like, what, is the equivalent of that in design methods? in design practices, you know, for every design method, there's a story about why that is necessary. And so what's that story and can we essentially address that in other ways today?

Dan: this is why I still teach ⁓ wireframing to because ⁓ I'm like, you may never use this, but it's a tool in your toolkit. And at very specific context, this might be valuable to you. Another example that do is task analysis. Task is really great for working out what an agent to do for you. Just having that. understanding of what the task is end to end, even if the agent does it differently, knowing of what the task was is helpful when you are designing and evaluating what the agent is doing. So I think at all different points, some of these design methods could be valuable pulled out, but then doing them all, all the time in the traditional double diamond, yeah, it feels like that. era is, it was ever really a thing, that era is ⁓ kind of away from us. So the question now is, yeah, I think it's less like, is the design process dead, but more like, how is it evolving? Is it just like compressing? Is it shattering into a number of different things?

Mike Kuniavsky: Exactly.

Dan: And then is it democratizing? Is it not being held just by designers anymore? That's my question is, we have all these tools now, should PMs be using them? Should engineers be using them? And ⁓ answer probably yes. ⁓ everyone is design, whether think about it that way or

Mike Kuniavsky: ⁓ I think that it's a question of like how, know, someone's on how do you define what is, right? ⁓ And, I think that in a sense, I agree with you that it is ⁓ democratizing and people can do ⁓ design as was practiced in 2024, much more effectively. And lots of people will continue to successfully do 20, 20 for design for a long time. But the question to me is, so does, learning all of those things for a designer is not about learning the, the specific practice. Like you do this when you're doing card sort which nobody has done in a decade. or you do this when you're, doing a ⁓ customer journey or you do It's by learning that to your point, you are learning a way of thinking. You're learning a way of thinking about how to deconstruct a problem and approach it in different ways. And then all of the methods are just a toolbox for how, what to do with that deconstructed problem. But the people who are doing 20, 20 for design, aren't necessarily going to have that tool, that set of cognitive tools in order to be able to deconstruct design. Like, just did So there's a set of Claude skills that somebody made called impeccable.style. ⁓

Dan: ⁓ we know Impeccable here. We're fans.

Mike Kuniavsky: Okay. So, I, so I just, built this, I, I, I vibe coded another experiment tool and I was like, okay, I just need like a reasonably okay, ⁓ UI for it. was like, great. This is what impeccable style is for. ⁓ And it came back with something that was I like, all right, this is And then, and then I had it, it actually wasn't as good at this other task. I kind of pointed it ⁓ at, ⁓ a specific, a specific style in the aesthetics Wiki. So, the, the aesthetics Wiki, which has every trendy style ever in history in it, well described in text with visual examples. And so I was like, Oh, you know, can you make it like this? And it went in and we read it and looked at it and did it and it did a mediocre job of it. And so was like, Okay, if what you want to do is something that looks like, an okay 2024 responsive website, it does great at that. If you wanted to go outside of those parameters, no, not so good. ⁓

Dan: We've talked about slash delight on here ⁓ quite bit. It's my new favorite catchphrase. we've talked the last couple of weeks about this woman, ⁓ MC Dean, who has been just making design skills that are on everything from ⁓ micro interactions to accessibility and all these things really level up game, but then the question is, will they have skills, design skills, or will they have like ⁓ skills? ⁓ really wish, I really wish they had come up with another term for them, but I get why they did it. But it's so confusing to talk about human skills ⁓ the one side and then skills for AI on the other, but that's a whole other episode.

Mike Kuniavsky: ⁓ we're ⁓ a shift in terms of everybody getting up now, but they're getting leveled up to the same level. And so the question is once everyone's leveled up to that level, which is what I just coined like five minutes ago, 2024 design, like, cause that's where everything was trained on. Once everything is leveled up to that, it all looks pretty good from that. Where do you go next? how do you use the tools to go and solve new problems and different problems? because the actually enable a ton ⁓ of possibility there, all the various AI models, you can just look at hugging face. There's a million different things you can do. I think of hugging face. as a big box of Legos and like, which Lego do I need today? but right now, all things that everyone's leveled up to kind of looks like a product from a of years ago, you know, maybe a slightly nicer Google product, ⁓ but, ⁓ or, know, slightly Bezier ⁓ Google product, but

Dan: Mm-hmm. It's actually slightly purpler. Yes.

Mike Kuniavsky: ⁓ yeah, no, you're right, you're right. ⁓ everything whatever color of the year from Pantone was couple of years ago, ⁓ like every week or whatever.

Dan: Right, it's kind of terrible purple gradient that makes me want to, yeah, get to the top of ⁓ a building with a sniper rifle. But we need to slash de- purple everything.

Mike Kuniavsky: Right. bringing it back kind of to the original topic of design practice, I think one of the things that as a UX researcher that I have questions about is one of the big philosophies in HCI, UX design, design in general, was essentially this very high touch user centered approach where You really wanted to understand the perspective of people who might be using your thing, even before they used it, even before you've, you've built it. And now you have some of that, like we described earlier, where you can just like make a thing and then show it to people right away. But the larger questions of. understanding people and understanding how to understand their experience still remain. And I think that there are a of questions about practice, about how you incorporate that today that are not answered well. And some extent, I've written about this elsewhere, I wrote ⁓ about in ⁓ the edition of the book, ⁓ is How do you use synthetic users well? Like where do you use that? Like that's a great AI tool potentially to give you like a first order cut at something. But is it giving you essentially 20, 24 users for 20, 24 designs? And if you want to do something that is not that, will it give you any kind of reasonable feedback at all?

Dan: only give you ⁓ users who still believe that Biden is president right now.

Mike Kuniavsky: Yes. And so, like, so I guess my question is what does that do to design practice? Like, what does it do for us? Like, how can we, maybe as a community, ⁓ perhaps, that problem where we're essentially getting essentially a reflection of where things were a couple of years ago, you know, even as useful as that is, ⁓ where do put that? And how do we do that? And moreover, how do we do it in a way that still makes sense for stakeholders? Because stakeholders aren't going to want to pay for it. And so ⁓ be like, If we have impeccable style, if they even know what that is, then ⁓ why ⁓ put prototypes in front of at You have the skill that just does the thing and then now it's all done. It's good.

Dan: Right. And I think the, the industry's answer to that right now is a, just to not use any user research, not even synthetic users, which is just like, well, I'm just going to intuition this and, we're going to make five different versions of this. And we'll just vibe our way to product market fit with each potential user and see, see, see who likes this thing. But unstructured intuition is often a trap for lots of biases and lots of issues where you're building for yourself often unthinkingly.

Mike Kuniavsky: And I think it biases towards a solution that's not going to scale. Like if you don't know why that product market fit worked, if you don't understand the context, you won't understand why it ⁓ work when the winds of people's ⁓ use patterns change.

Dan: And in a couple of years, lots of things can change. Styles can change, ⁓ levels of and privacy can change. I mean, I remember a couple of years ago when Venmo was showing you what people were paying for on a public feed. And I was like, my God, who would show their Venmo public money transactions? That's disgusting. And now I don't even think about it. And now most people don't even think about it. But...

Mike Kuniavsky: ⁓

Dan: That kind of stuff, if you were designing now and had old data from people years ago, they would be like, well, never show that or I never do that. And so those kinds of behavior patterns ⁓ ⁓ can shift pretty rapidly over a short period of time.

Nik: what are the costs though now to this because in some ways if it doesn't cost me if it doesn't ⁓ harm me or harm my organization to put out all these and to try things out also the idea of scaling right I think scaling is in many a certain viewpoint of the world but Arguably, you can have great products and services that work for a smaller group of people. And if anything, if you can reduce the cost of production develop business model that someone's okay making a certain amount of money than providing that product and service, does ⁓ scale And then maybe in those regards, does the design process change because of that?

Mike Kuniavsky: still in many of our practices, these set of kind of industrial revolution assumptions that what you eventually want is you eventually want whatever it is that you're making to be used by everybody on earth because, know, and why is that? And apart from kind of maximizing profit, the other thing is that you're minimizing the amount of effort. that is ⁓ necessary per unit of someone's interacting, which is straight up industrial revolution thinking. Like you want to make the most generic thing possible so that you can expend the least amount of effort per person. that's actually no longer, Nik to your point, that's no longer true. That's no longer how things get sold. we no longer have general foods, general mills, general motors. You no longer have these kind of single conglomerates that do everything. Instead, you have a million fragmented things where you have some limited edition drop designed by somebody who... it sells it through TikTok. And that drop can be a shoe or it can be a app.

Dan: that world of industrial revolution, ⁓ mass production, ⁓ way that worked was also that they just going to ⁓ cases where it's like, well,

Mike Kuniavsky: Yes.

Dan: we don't really care about this group of people or we don't care about these kinds of needs. And when you're working at scale now, that kind of stuff is important. Like I know from working at Twitter, there was a small fraction of people that caused a lot of problems, a lot of harm. And if you don't address that, if you were like, well, we're just putting Twitter out there, Well, I mean, you do, you get Twitter from whatever, 2005 to 2012, where it's just like, well, that's everybody else's problem. And then as it starts to become harmful, people start leaving the site. the site gets sued, people commit suicide, all kinds of terrible things. So hopefully we are beyond that now, but I think those edge cases,

Mike Kuniavsky: Yeah. Yes. Yeah.

Dan: are some of the things that the design process ⁓ light, ⁓ I'm worried that is also one of the things that's gonna get dropped here. ⁓

Mike Kuniavsky: Yes. And and synthetic are not going to catch those edge cases.

Dan: No, not at all. I think the answer to that seems to be like, well, we'll just make a whole other app for those people, which fine, you can do that. But what if your app is a social one and it relies on network effect? It's hard have this kind of splintering effect because ⁓ the app just becomes less valuable.

Nik: gonna actually disagree a little bit here to your point, Dan, that ⁓ users may never capture these edge cases, if anything.

Dan: ⁓

Nik: who is thoughtful can go out and figure out how to create synthetic users of many different kinds of peoples to then surface edge cases. I'll say full disclosure folks here, some of these ideas we're studying in our lab, thinking about this kind of way in which AI systems which have broad overview, ⁓ much than even the members of your team who...

Mike Kuniavsky: Yes.

Nik: can only come with their own context and backgrounds and their way they think about users and people, that you can actually leverage these things. But I do think it has to be thoughtfully created within the AI systems. I think if you just do like a blanket I mean, then you have the issues that we had when we started kind of turning personas into sort of data-based averages of people.

Mike Kuniavsky: Yes. Yes. Yeah.

Dan: ⁓ so, I'm supposed to know unpublished research now? Sure, okay. ⁓ see where we're going.

Mike Kuniavsky: Nik to ⁓ point that and don't what you don't know. And so therefore, like one of the major reasons for historically for doing ⁓ user research with real has been to understand what you don't even conceive of is out there. to know, I had not even had any idea that people would do with this tool or would use it for this. And so that is the thing that no matter how, like you can write good edge cases, ⁓ for for example, that are very, hard get, ⁓ surgeons, like ⁓ synthetic users, probably pretty good at, or better than you're going to get by just trying to recruit surgeons, because you'll never recruit more than a couple of them. But with synthetic users, you can probably get something that gets you a better perspective. But it's not going to get you the thing you don't even know.

Nik: a of the design process about managing and mitigating risk. Like we do these things to help mitigate risk to basically having a failure of a product, to doing bad design. And so a question is, Has any of the risk actually changed? We've got all this new tooling, we have all these new opportunities, AI is changing things, but ⁓ any of the same risk gone away? Or do these new technologies change the calculus on how we manage the risk?

Mike Kuniavsky: I think the risk surface has changed shape in ⁓ to that. So I think that there are certain risks that ⁓ been in the sense that there are a bunch of basic usability things that people just don't even worry about anymore. There used to be some really terrible UI out there

Dan: The past is still with us. It's also just unevenly distributed. You will find some really terrible usability things in the corners of like B2B ⁓

Mike Kuniavsky: That is true. Yes. a user of Google cloud, OMG is the Google console, a giant usability mess for somebody who doesn't do ⁓ a DevOps 24 seven for a living. Holy cow. and they've tried to ameliorate it by giving you. an agent that runs on top of their UI to tell you where things are in their UI, but then the agent itself has its own issues. ⁓

Dan: the software we use to record this podcast is an HCI nightmare. It just has incredibly bad usability, incredibly bad information architecture, incredibly bad interaction design. And yes, they've the solution. we'll just put some AI in here and you can ask the AI questions about how to do things. And the AI... can't do them either.

Mike Kuniavsky: several things have shifted or shifting at the same time. So it's hard to say whether it's a ⁓ UX slash UI ⁓ shift or it's like an entire transition and people use technology in general. But the, think that we have a differently shaped risk surface that ⁓ design practice. may not adequately address right now.

Nik: one of the largest risks you were saying the stuff don't know, you know, the concept of sort of unknown unknowns. And I guess this is something where I'm wondering in regards to practice today and even what we've been doing, right? Like how much are we focusing on that? I'm almost wondering is, is this something we should be focusing on more? that a lot of our design practice should be on figuring out what unknown unknowns are, such that we can bring them to the fore and make them at least a known unknown, and then hopefully help in addressing them and understanding how to wrangle and wrestle with them.

Mike Kuniavsky: Like one of the big, I'll just give you one of the big unknown unknowns and how to address it with UI and UX for me is cognitive offloading. Essentially ⁓ we give the responsibility to an AI agent to do something on our behalf, we lose our ability to know how that's done. And so ⁓ the question of Where is that appropriate? How is it appropriate? How should the agent interact with us so ⁓ understand what we need to understand about that? And there is not a good, like we've spent years, we spent decades working on information architecture as a way to organize information. I think cognitive offloading is a problem that's about the same size as information architecture. and that we haven't really addressed hardly at all right now. And so I think that that's a straight up user experience ⁓ ⁓ we have.

Dan: Yeah, there's a bunch of studies that just came out this week and we'll put a link to them that show exactly that, that there's just some real problems with people knowing ⁓ the AI is doing and not being able to fix it if wrong or incorrect because they don't know how it was made. They don't know what the constraints were. They don't know what was put into it. ⁓ it definitely something that we need to start. designing for and nobody's really thinking about it. Although the problem is at least there and I guess admitting the problem is the first step in solving it.

Mike Kuniavsky: I think that essentially ⁓ organizational dynamics and organizational power are being changed right now. I talk about this a lot in terms of design as a... career, but also in terms of what AI does to organizational power, in terms of the of the internal practices that historically design has enabled. Essentially among the communication between teams, decision making for stakeholders, ⁓ kinds of essentially organizational knowledge about their specific products or context or users or all of that. Like design has enabled all of this stuff that is essentially organizational knowledge and organizational practice that has nothing to do with making a AI is equally quickly changing all of that and changing the organizational dynamics behind it.

Dan: Mm-hmm. Yeah, no, this is something I've been giving some thought to and this idea of power and organizational power and the power to things like take something off the roadmap or kill a feature and Are the only things that are currently separating a product manager from a designer at this point? Is I don't even know if that's true, but If a designer has that power plus all their design judgment, that mean that product managers are irrelevant? Or does that mean that the design job is now also includes, or as would say, officially includes the product manager role too? So I think that is an interesting place where

Mike Kuniavsky: Yeah.

Dan: where the power to affect things like the roadmap and to say what gets built, what gets killed, if that shifts over to design, what happens then? I don't know.

Mike Kuniavsky: Mm-hmm. Yeah, I don't either. But again, 2024.

Dan: But I will say I'd rather that it shifts over to design than that all the aesthetic judgment goes from designers goes away and then PMs are like, well, I can make these prototypes too. And then we end up with the 2024 style purple gradients.

Mike Kuniavsky: think that that's our future in the near term. It's I guess, gradient ascent.

Dan: God. do a check on me later. That's ⁓ just We're entering my hell.

Nik: So this has been a fantastic conversation. I'm not sure we've come to any strong answers or conclusions on the design process is ⁓ and is going. But I think that there's a lot of food for thought here. Mike, I want to give you an opportunity. Is there any final things you'd like to say, you know, if there's something you want to share with our listeners for them to kind of think about or wrestle with?

Mike Kuniavsky: I think that... design practices fit a certain kind of way of working and a certain kind of organizational dynamics and a certain kind ⁓ set of practices and a certain set of technologies, at least the design of practice that we have today. And I think ⁓ of those things have shifted, partially because of AI and partially because essentially big shifts in the way that organizations make decisions now, especially compared to where they did a couple of years ago. So I think that everyone should both look very carefully at what AI tools are available, but also look at what do they do in terms of the organizational relationships that they create or change. And I think that that is going to be where design practice goes. Because whatever we call design practice, it's not going to be what it was in 2024 in a couple of years. But it will be still there in some form, having been deeply changed by AI.

Dan: Well, Mike, where could they find you and your books?

Mike Kuniavsky: people can find me on LinkedIn. And ⁓ that's where I will posting both incidental thoughts and links to ⁓ various things that I've built. They can find ⁓ my books, SmartThings, which is now-aged IoT ⁓ UX design book that I believe still actually is kind of interesting. And Observing the User Experience, which is going to be ⁓ entering its third edition. I am the second author on that now. I was the original author, but now Elizabeth Goodman is the first author, and her and my book will be coming out hopefully later this year. We're hoping for November.

Dan: Great. Well, thanks for coming on and talking about design practice and process. Obviously there's a lot for us to process about this conversation and we'll also have to do that online, but thanks for listening please be sure to like, subscribe ⁓ tell your friends about design and AI podcasts. We're closing in on 500. subscribers and it'd be great to see that number grow. But thank you all for listening and we'll see you next week.