Nik: Welcome to AI and Design, where we explore how artificial intelligence is reshaping the world of design. I'm Nik Martelaro Dan: And I'm Dan Saffer and we're faculty at Carnegie Mellon's Human Computer Interaction Institute. Each week we break down the latest AI developments, dive deep into topics that matter to designers, and talk with fascinating guests who are right at the intersection of these fields. Nik: Alright everyone, we're here now with Mark Boyes- Smith, head of AI Design at Miro. Mark, thanks for coming on the podcast. we're super excited to you because we know that Miro has just released a ton of new updates to the Miro platform, many of which include AI. I actually got to check out the YouTube video. We're gonna link to that in the show notes so that people can check out your demonstration of this. But for now, can you tell us a little bit about what you've been working on at Miro? Whether you're a designer working on AI or an AI practitioner interested in design, we're glad you're here. Dan: As we're recording this, Apple is kicking off its annual WWDC, so we'll get all the Apple news next week. week, their old Pal Microsoft unveiled their redesigned co-pilot. ⁓ Mark Boyes-Smith: Yeah, of course. And thank you for having me as well. so it's been a it's been a bit of a whirlwind at Miro. I rejoined them actually in February this year. And ever since we've been hot on AI and all of the different capabilities that we can ship. And ⁓ I think like everybody right now, we're trying to go abroad and have a look at all the different AI opportunities that we can put in front of our ⁓ million users now, which is insane to think of. so the things that we launched in San Francisco a couple of weeks ago, sidekicks and flows, two things that we introduced last year and we've taken a big step forward with. So sidekicks are really your agent when you're working in Miro itself. ⁓ they can create on the canvas with you all that wonderful stuff. And we gave them voice as a as a new opportunity, which is something that I've been personally very excited about. I'm a big fan of like open AI's voice models, ⁓ and using Chat GPT in the car when I'm trying to like list off a bunch of thoughts and ideas. So the the fact that I can now do that in Miro and see the things coming to life in front of me was a bit of a magical opportunity. And we have a long way to go there, but we're really excited about kind of the journey there. And then after that, we took a look at Sidekicks more broadly and we were like, hey, maybe there's an opportunity for us to help teams take on deeper work with them through things like a planning mode, connecting out to all of the different tools in your stack, no matter where that is. and so we've launched as well a bunch of connectors out to all the different platforms for very different teams. but also one that was really cool and and came in before the event: custom connectors, so you can reach out to basically any tool, even if you've built it yourself. And we think that's just a an awesome opportunity. and I think on the video, if you watch that, there's a really interesting bit towards the end. And we wanted to bookend with exciting stuff, right? And as you get to the very end, our CPTO Jeff Chow comes back on stage and presents a slew of really creative experiences that are AI led. so stickers, you can do custom stickers in there now. So you can kind of have that, you know, you you can have your own personal emojis in there while you're working. Nik: Another story that's maybe a little new for the podcast, but we actually think might become a bit more prevalent, is on Amazon's new Proteus robots and their ability to be directed by human employees using plain conversational language. Mark Boyes-Smith: That was pretty cool. And custom widgets as well. I think, you know, like like any great platform, there has to be a level of hackability in what you give your users. And we wanted to make sure that, our users weren't just wholly reliant on us shipping new components and new features to the board. ⁓ instead you can create stuff yourself. And and again, all of that underpinned by AI. So yeah, a whole slew of features. Dan: Done is dead, says Jeff Gothelf, when comes to AI products. Calibrated is the new hotness, apparently. Nik: Amber Buabdalla walks us through her Salesforce design team's process to learn new AI tools and processes. Dan: Finally, this week, Nik interviews Mark Boyes- Smith, head of AI design at Miro, to talk about how they're designing AI into Miro and how they're using AI on their design team. But hey, let's start with those radicals from Redmond, Microsoft. Microsoft this week. ⁓ Nik: I wanna start start on the last point that you made, which I think is really interesting from a product strategy, which is this idea of hackability. So right now, your team has put together a ton of ideas and basically shipped a bunch of ways in which AI can be integrated into Miro. But this idea that people can create custom connectors, that they can create their own things, how are you thinking about that in regards to Mark Boyes-Smith: Yeah. Dan: Gave its AI assistant, Microsoft 365 Copilot, a massive makeover. Which is good because users from the last year and a half have been really unhappy with the last version, which came out back in January of 2025. So an AI lifetime or two ago. So here's some of the highlights, according to their chief design officer, Jon Friedman. Nik: design strategy and giving ⁓ your this ability to create and add on to the platform. Mark Boyes-Smith: It's so tough, right? Like I think, you know, we we recognize that everybody's workflow is unique and different. And you put that time into ⁓ evolving it with your teams. You want to share it across your organization. It's actually part of your business IP in some ways. And we wanted to be able to facilitate that as best we could. From a design perspective, it's it's always incredibly important that Miro is at its heart very, very to use because we range from people who ⁓ Dan: So number one is that the text box, it's now a workspace. So instead of just being, you know, your little standard text box where you type a question, that little prompt area, it's been redesigned into a task-aware workspace and it expands or contracts to give you room to like paste text format writing and organize ideas like before you hit enter. It's actually Mark Boyes-Smith: engage with technology every now and then all the way up to the engineering product and design teams like ourselves who are like super super comfortable with tech and also oftentimes operating at the bleeding edge and we have to find a way to cascade that entire spectrum of user demographic. And so yeah we spend a lot of time figuring out the foundations and the languages that we want to put in place to make sure that we can help our sidekicks, all of our AI engine capabilities. Dan: really nice. The second thing is that it shows you only what you need. ⁓ really leaned into ye HCI progressive disclosure. So the screen pretty clean and focused, but as you work, Copilot starts revealing extra tools and new formatting options and suggesting prompts they actually make sense for what you're doing. Mark Boyes-Smith: to build really good stuff. So it it's interesting actually. Design has become probably about as impactful as I thought it could ever be. ⁓ also we're having to go much, much deeper into our tech architecture in order to exert direction and opinion as well. Nik: Can you tell me a little bit about the team behind all this? Like how many people do you have? Like what's the kind of makeup of folks and how are the designers on your team actually engaging in creating some of these new features? Dan: the third thing is that ⁓ it they have a thing called work IQ, and it's like the underlying brain that pulls info from emails and chats and your meetings Mark Boyes-Smith: Yeah, absolutely. So I mean, obviously it takes a village and in ⁓ Miro's case it's a pretty large village. We have about sixteen hundred members of staff overall, I think, roughly. and within that we have an AI stream, which is composed of product engineering design and product marketing partners. we have around five designers that that design pretty much everything that you see from an AI perspective. Dan: All those things to give you better answers, sort of like how Gemini is now personalizing your responses And is one of those things where you can choose between different models on how deep or complex your task is. And the last thing that I want to talk about yeah, it it kind of works across. Nik: ⁓ this is beginning very good if the mouth would have been too. Just I think we've got this. Mark Boyes-Smith: But actually overall in Miro, we have ⁓ somewhere closer to sixty, sixty-five, something like that. and everybody obviously benefits from all of the AI capabilities and then can build some of the amazing capabilities off the back of it. so like prototyping is a great example. Separate team, we've got designers there who do some fabulous work and they benefit from some of these foundations. But yeah, design is a core in AI, five of us, which is always really exciting and there's always a lot to do. Nik: Yeah. Dan: Office apps. So instead of forcing you to jump between different windows, it acts as an editing partner right inside Word, Excel, PowerPoint, and Outlook. ⁓ the other thing it's pretty funny. So it uses these specific agents, ⁓ and I was ⁓ you are so close. So one of the agents is a designer. Nik: I just find work powerful. It's a bit Mark Boyes-Smith: similarly ⁓ on the product side, probably a couple more there and then engineering I think we're probably around a hundred, hundred and fifty in that. Dan: One of the agents is a researcher, and I'm like, yeah, yeah, they finally figured out that it's not about what app you're in. But then the next agents are like Word, Excel, PowerPoint. And I'm like, ⁓ so close, Microsoft, so close. ⁓ but I have to say, I really like this update. It looks really nice, it looks really clean, it looks like it is. Nik: Okay, five. I don't know why that number felt somewhat surprising to me. It's partly because it seems Yeah, it it seems small, maybe because the number of features the number of ways in which you are pushing out AI integrations. Like you had voice, you've got flows, you've got new these new agent sidekicks. It seems like Mark Boyes-Smith: Like small? Nik: Yeah. Yeah. Dan: much more around this idea of activity centered workflows rather than being siloed or that AI i is a feature. ⁓ It more like AI is becoming a layer. I think it's pretty good update I'm I'll be curious to see how people responding to it. Nik: a larger team would do that, but at the same time, of course, I mean I I'm teaching a class on augmenting AI designers and I I now have single students who are producing full-on applications that only two years ago would be wild for a team of maybe 10 people to produce. Tell me a little bit about how some of the workflows are changing within your team. Yeah. Mark Boyes-Smith: Yeah. Yeah, yeah, yeah. I mean, it's it's worth saying, like we're in June now, and in March this looked different. In January this looked different, and last year this looked completely different. and I think in the next couple of months it we're gonna make another shift again. what we've seen obviously at a very high level is exactly what you're saying. ⁓ this kind of abundance of opportunity, and all of a sudden you can run much, much further as an individual than you've ever been able to do before. And that's that's exciting for many reasons. Nik: I think that they're doing a lot here that they probably learn from seeing how applications are ⁓ doing the ⁓ AI layer. I specifically out co-work by Anthropic. I think that co-work in the way that it brings together. a number of tools and has skills for all these tools. Actually, Cowork does a lot of stuff in Microsoft documents. Like when you ask it to do something, like write something up or do maybe a calculation, it will actually create Word files or Excel files and work with those. I will say one of the limitations that I found when doing that is the initial creation is actually pretty good, but then it can't really edit the documents very well. And I think that Microsoft is really poised to be able to have Mark Boyes-Smith: ⁓ what we've seen from a process perspective is first of all, our designers are in the code base with our engineers, with our product folks. We've seen this democratization of the creative process, which for us is is fantastic because someone like Miro, we we have to be able to run as many ideas as possible to find like the three out of the thousand that are really worth our time and effort. So it takes everyone's ideas to come together and and so being able to get everyone into the code base is a fantastic starting point for that. but within design itself, we've obviously seen the emergence of skill sets that are a little bit more technically focused. That's a challenge, I think, for all teams, as the design industry, especially in technology, we've done a great job of hiding from the tech complexity in design. Nik: AI as a layer, as you say, that can edit these documents where you can work much more iteratively. Because this is often the way that we are doing a lot of work, is in this iterative manner where you're setting up a problem to be solved, you're setting up a task, you're getting the output, and then you're sort of evaluating it. And then you're usually having to change it. I do this all the time. Arguably, maybe as you get better at setting up certain repeatable things, you can really set that like here's the goal, do this with this input, and just create. Mark Boyes-Smith: ⁓ and I think we're starting to pay some of that down now. We're getting back together and we're we're really figuring out how to build those competencies. ⁓ there's not a day that goes by that there isn't like an AI brown bag session where a few designers are teaching each other their latest tricks. and then finally, in terms of like the tooling, because I think that's really at the heart of this. You know, we were a big Figma shop. we still do use Figma, absolutely. It's a great design tool. Nik: you know, this spreadsheet, this PowerPoint deck based on these materials and it can act more autonomously. But I actually think that the the iterative nature of the work is really important. And that's one of the things that I think is nice about this update where they're really showing it inside of the tools themselves. So it isn't that, ⁓ you're in a chat window, it's created a doc, you have a viewer, but then you have to open up Word. And there's not really that bi-directional connection that you have with other tools that it's it's really integrated. And I think that that could be Mark Boyes-Smith: But I'm seeing and less of it. I'm seeing more designers. Well, actually, if to be completely honest, probably a hundred percent of our prototypes now are coded prototypes in Claude code. and we're we're oftentimes using the MiroBod kind of like share and trade ideas. And then we have this MCP that we launched, so you can like pull it in and use that as kind of the the ingredients list for your Claude code, which is pretty cool. now more than ever, what we're seeing is we're taking out Nik: Microsoft's big differentiator and selling point for co-pilot assistance over other companies. Dan: What do you think about picking different models? ⁓ I I'm of minds at this thing. I I mean I do like the choice of it, but for most people, ⁓ are they up with the different AI models and what's good for what? And it it it like nerdery that all these ⁓ Mark Boyes-Smith: a prototype that's got like a a finer degree of fidelity to it, like we've baked it little bit more. We're extracting that back out to Figma, doing any like scalpel cuts and fine tuning and pushing it back in and ⁓ that tool that way, which is again pretty novel and and quite an interesting way of doing it. So yeah, it's the landscape's changing fast. Dan: AI companies and I guess Microsoft's like, we're an AI company too, rather than software. like normal people always having to choose the model. I mean, I guess you don't have to do it, but Nik: Yeah, you mentioned the brown bags and people sharing what they're learning and how they're using the tools. I'm curious, is that involving engineering at all? Because you mentioned too that everyone's having to learn, or some people are having to learn some of the technical depth of what's going on so that they can work in the code base effectively. is there a lot of mixing of of teams here? And how are people learning some of those technical skills that help them work effectively? Dan: I don't know. It just feels like it is it's a piece of machinery that I always stumble on. and I never know if I'm picking right. Like, ⁓ you know, am I doing this right? Do I want more? Do I want less power? How many tokens am I burning with this one versus this one? ⁓ I never know. I try to keep up on that kind of stuff. Mark Boyes-Smith: Yeah, we we're really, really fortunate at Mira that our engineering teams are they're really leaning forward and into this and they're they're incredible coaching partners. I've been in businesses where like oftentimes you get this Nik: I won't I guess don't have like the way. You think this one report? Yeah. Mark Boyes-Smith: almost like this wall between design and engineering. I have not seen that at Miro. And I think that's actually why we've been able to move quite quickly. So a lot of the sessions that we have are either inspired by or led by engineering. They are the people who are always willing to put in an extra 30 minutes to check a PR. if it's one of mine, it's probably terrible. ⁓ and and and and we really value that because every single time we're committing something, Nik: Yeah, I think from a design of consumer or prosumer software, that is still not something that most people are going to learn and get into, picking what model they use and then what thinking level and effort level they're going to use. Programmers do this, but they're playing around with the models, they're building potentially their own harnesses, they're doing a lot of testing to see what these things can and can't do. I think it's been now. ⁓ Mark Boyes-Smith: It's a learning opportunity. And I think for those guys as well, they're able to test the safety nets they have in place, the harnesses, because hey, if there's if there's anything more chaotic, it's gonna be bringing designers into ⁓ your code base. Nik: To be able to do that because the models do have differentiated abilities in different tasks. So there definitely are things where I use anthropic for, you know, I use Claude Sonnet for something, and then I'll use GPT for something else because I just happen to know that it works better on this task for some reason. And who knows entirely why? It's one of the challenges where you look at the the benchmarks and it's like, well, they all look pretty good, but then on my real world tasks, why is there such a big difference in how they actually perform? I think that we're still in this sort of nascent stage where everyone's figuring out how to do more and more of these tasks. But over time I think that's going to matter less. And actually companies that are building the layer into the work and into the tooling that they I'm curious, how ⁓ are you all about working with diagrammatic representation, more visual laid-out representation of text? I feel like a lot of what we see today people producing markdown files. Everything is sort of a document, and that's kind of the common workflow. But I'm curious, arguably, you all probably should be the best at figuring out how to use what maybe is more of a designerly way of working on a board, on a canvas. but how is that? Like how are you figuring out how to set that up so that tools like Claude Code can actually work with that effectively? Mark Boyes-Smith: Yeah. Nik: Designed for, i.e., in this case Microsoft, which also came out with a lot of its own AI models recently that are not just open AI GPT models, but they're actually Microsoft models. I think they're going to figure out and build specific models that actually act within the tools that work really well for the tasks that people are trying to do, not just on the benchmarks. And at that point, I think it won't matter. It'll just be like, ⁓ does the tool I use that has an AI layer, does that work better than? Mark Boyes-Smith: Yeah, so a lot of this falls into the the bucket of like investments around our MCP and how we how we share all of the tools and all of the things that we do in Miro effectively with your agent wherever that work is is happening. You know, we acknowledge that more and more often now the work starts, continues and ends with maybe your Claude or maybe your Chat GPT or like, pick your flavor, right? Nik: another tool as opposed to I have my tool that is a wrapper and I can choose what AI I'm putting into it. I'm not sure how long that's going to last for. Mark Boyes-Smith: and so we want to make sure that the canvas is just as accessible to an agent as it is a human, which is a a complete mind shift for me. I think we've gone through three or four mental models for this, even over the past three months since I rejoined them. but I I I think this is one that's gonna stand the test of time. We know that people individually want to work with those agents, and so let's make sure it feels like a welcome place for everyone. Dan: we'll what does at WWDC, 'cause supposedly with the new Siri stuff, again it's one of these like choosers where you're like, you can choose which model that you'd like to have in here that's answering your Siri questions. ⁓ okay. I hope that this is a temporary thing ⁓ or that ⁓ Mark Boyes-Smith: ⁓ I think with regards to, markdown files and just the way that we represent information, data, and progress, we see that there's there's a challenge emerging where I don't know if you've noticed this, but with the more documents we generate with things like AI, the less we're actually reading them, the more we're like just scanning through them, or you're like, ⁓ two pages, really? ⁓ And know, we see that. we have to find better ways to be able to Dan: Maybe I need an agent that will help me pick out which is the right AI for me to be using for a particular task. Can't can't we figure this out somehow? Nik: Yeah, or you need some type of interface or user experience that helps a user do that if that is what you're gonna do. Cause I think one the challenges there is if you are designing your own or you've picked the model and you've designed the tasks and the inputs to work with whatever model you have, whether if it's your own or you're choosing someone else, a third party provider, you at have a little bit more control over the user experience of people when they actually interact with it. Mark Boyes-Smith: quickly transmit a thought, an idea, get to the point. And so we see the opportunity there as like, hey, this isn't just about markdown, this isn't just about text, right? We have to use visual, we have to use audio, we have to be able to use diagrams, flowcharts, anything that's gonna help communicate that in the fastest way possible. And critically, when you have an opinion that you need to exert or a direction change or something like that, you have to be able to do that easily as well. Nik: When they can choose their own models, right? Like you might have something where you've got application that is you can pick models, and if you're always if they just default to some sort of mediocre, more token efficient, maybe, but maybe not so intelligent quote unquote model, people might be like, This thing sucks, like I don't like using this. Whereas everyone who decided like, ⁓ let me put it on, Opus four eight or or GPT five five, and they're like, ⁓ this is awesome. This is so good, it's amazing. Mark Boyes-Smith: That may not be if you're prioritizing a bunch of items that that you and Dan are going to focus on next or episodes, you're probably not going to want to do that in a markdown file. Probably a table or maybe like a Kanban board where you can just quickly grab the cards and shuffle them is the easiest representation. And so that's that's the way we're thinking about it. How do we bring this back to something that like we as humans can just easily digest that information and then form a view? Nik: And it's a really differentiated user experience. so yeah, do you design for How do you, if you're gonna do this, how do you also get user feedback and separate the issues with the model with the issues your integration of the model into the application? So in interview, you had also mentioned voice interaction. I've been researching voice-based interaction for many years and actually have a lot of things where we do voice-based design support agents in our in our research. so I was really excited to see that that you've integrated that. But I'm really curious because voice is still kind of new in regards to people having good design patterns for interacting with it because we just haven't spoken with our computers forever. my current working hunch is that in the next year or two, we're gonna see more companies building in ⁓ an AI and leveraging either their own models or at least ⁓ really working get whatever model they use to be deeply integrated and to really focus on sort of the the tasks that people are trying to do and then How are you and the Miro team thinking about voice interaction and designing for it? Mark Boyes-Smith: Yeah, we're we're trying to bring this as close as possible to collaborating with another human. I think that's the starting point, right? Because I would agree with you. the patterns that we have today don't necessarily support like a fluid interaction with voice. It can feel a little bit staged at times. also if you're not used to it, I mean, I don't about you, but when you started did it feel a little bit like unnatural for a second? ⁓ I certainly felt like that with GPT, like took me a while. and my husband was like, Nik: making sure that that user experience works really well. And ideally they're going to be doing that one to make sure that they have a consistent experience for their product that hopefully is great. And two, they're going to be looking to make all of those AI calls much more efficient. Because right now, if you say I'm going to put Opus 4.8 on everything, you're spending a lot on tokens. Whereas if you can use a much smaller model or potentially, maybe someday ⁓ ⁓ yeah. Mark Boyes-Smith: You gotta stop talking to that phone. Like it's creeping me out. But I think after a while you just accept it. It's like it's on speed dial and you know, it feels really natural. but yeah, you hit the the nail on the head. We need to continue to figure out better ways to be able to kind of support those interactions. A few top of mind ideas that I'm kicking around with my team. I'm like, ⁓ I love this idea. when you come into the Miro Canvas, you're shown as a collaborator along there with me. You can follow me. I can Nik: In the future, local models that just run on your your laptop, which you know it's is definitely possible. And in the next maybe decade, we might see a lot more of that. That's kind of my hunch of where things are gonna be going as more people use this and you can learn more about your product. So again, I we talk about this a lot on the show. User researchers out there, we're here for you. Please start looking at how your customers are using this stuff because you're definitely gonna be there to figure out how to. Mark Boyes-Smith: point to things, I can draw attention to certain areas with you. you can comment on stuff, I can comment on stuff. We have all these human to human interactions. why wouldn't we start to extend some of that to agents as well and make sure that that feels like a somewhat of an equitable relationship? That way you get to benefit from all of those ⁓ more natural collaboration It's just that what's at the other side is agentic and not human. I don't know. We'll see how it goes. I'm curious to see how it evolves as well. ⁓ Nik: integrate this stuff in a in a deep effective And then you can work with your engineering teams to make that cost effective for the customer and for your operations. Dan: I mean ⁓ nice to see Microsoft back in game. Our next article is done means when you're shipping AI features. Jeff Gothelf argues that the traditional deterministic software development rules, where product behavior is fixed and predictable, he it to a vending machine. that doesn't apply to AI because as we all know, AI products Nik: sidekicks. And agents, I wanna talk a little bit about how your team is thinking about designing them. And then I hear a little bit about how or if we as users can design our own ⁓ agentic So I know you talked about customizability with sort of MCP and stuff, but I'm wondering, is there something within the Miro tool that I can define, hey, agent, act more like this or do something like this? Dan: are unpredictable. they are not deterministic systems. They are probabilistic systems. And so when we are designing, we to change our definition of done from a binary specification to instead a calibration of acceptable variance. unpacking what that means, he basically says need to start Mark Boyes-Smith: Yeah, there is. so I think I think the first thing to say is, we need sidekicks to be fit for the task at hand. There's no point having a sidekick that's behaving like a facilitator when you need a sparring partner. That's just gonna wind you up very quickly. so yes, today you can actually specify the behavior of your sidekick. You can build out custom sidekicks and give them instructions, prompts and and extra context as well. Nik: I think that's Dan: Writing ⁓ ⁓ criteria ⁓ as distributions, not assertions. So rather than input x ⁓ y, teams need to start writing criteria that acknowledges probabilistic outcomes. ⁓ Mark Boyes-Smith: and we see a future in that as well, right? we want to imbue your sidekick with the right instructions to have the right relationship with you as you're doing that work, or you and your team, whatever that looks like. and then imbue them with the right context. And we see that coming from the connectors that we have and that data, no matter where it lives. And then the final piece of it is like, okay, we need to be able to take on chunky work with them. So it's not you Nik: I just quick Dan: 80% of responses meet a specific quality bar, but the other 20%, those things fail gracefully. second thing he says is you need to start. Nik: Yeah. Mark Boyes-Smith: Directing an agent ⁓ step at a time and you're like, hang on a second, I could be doing this myself. This is not where the book is going. we you to be able to take on bolder tasks with them so that you can set your agent, your sidekick to work, dismiss that, and then focus on the work that you're doing, and then loop back with them once they've got something for you to review. So we're kind of thinking about this as a multi-dimensional problem, but what you were saying there about, Dan: Building failure triage capacity into the launch, not after. So, and this is something that Nik and I, I know we talk about this a lot with our students, you just can't wait for user complaints to pile up before you act problems. have to start thinking about them beforehand. And when you're with that may have Mark Boyes-Smith: Having a Sidekicks that's fit for purpose and customized to your liking, we think is incredibly important. There was actually one piece that we didn't get to share at Canvas, but it's that, it's that idea of like, when I'm writing a doc, I do not want to see an M-dash, even though, even though I was a lover of the M-dash before AI came around and I feel somewhat jaded that it's been taken from me, I can't bear to use them now. So we need even those like micro preferences in there. Nik: Yeah, Yeah, it's ⁓ Dan: UX or PR or other kinds of harm problems, you have to be ready for that kind of stuff before the feature goes live. The feature, isn't done until downstream teams know exactly how to handle it when it inevitably And that sometimes means getting your legal team on board or your policy team on board, your QA team on board, your PR team and marketing team on board. Mark Boyes-Smith: to to make sure that you get the the expected result for the investment that you're putting into working with them. Nik: You mentioned one thing that agents can work alongside you. And I imagine some things are pretty quick. Actually, in the demos, you showed things that were really quite interactive. But then there's also this idea of a longer running task. And we see this in software development, right? Actually, for all of those of us who are using things like Claude code now, what used to be kind of like near instant code gen is like, no, I'm gonna work for five or ten or sometimes even longer. How are you the design team thinking about Yeah, you can probably Dan: And then the last thing he talks about is Defining tripwires and rehearsing rollback. So we rolled out this thing. And is there a moment where we say, Whoops, it is misbehaving so bad, might have to roll this back? So things that could be the tripwire would be like high error rates or like severe hallucinations, user escalations to your customer service, all those kinds of things. So before anything rolls out, the product team. Nik: this long running task and the interaction people have. We've talked on the show before about how all of a sudden it feels like a lot of our interactions with AI have turned into waiting. Like we do something, we have a thought and then we wait. But Mark Boyes-Smith: ⁓ I I think it starts with confidence. if you know that the outcome is gonna be within ⁓ a mile of where you needed it to be or a hundred of where you needed it to be, you're probably a little bit more prepared to wait and go do something else. It's almost the nervous energy of waiting. ⁓ Cause you're like, hang on, I've been waiting 30 minutes for this I actually don't know if this is gonna get me anywhere close to where I need to be. Dan: We've got to agree what the threshold triggers are and then rehearse what happens when those triggers are met and they have to roll back. Again, this may involve legal, it may involve your PR, it may involve marketing, it's definitely going to involve engineering. So the idea here, just as in kind of summary of what Jeff's talking about, that in the AI era. Mark Boyes-Smith: So the first thing that we're doing is, I mean, I I I talked a little bit about plan mode and I I won't sweat the detail there because I know this is like a very common pattern these days, but we want to make sure that any work that you're doing with a sidekick, you're able to track with it and understand exactly what they're doing. and one of the really cool pieces is they're actually going to start to build that out on the canvas with you. So you don't just get like the output at the very end, but it's a little bit like when you're in a math class and they say like, no, the answer isn't the important bit, it's also showing the working. Dan: Shipping software is not a check the boxes kind of thing. That you gotta have a build culture ⁓ and learning culture, which he's Mr. Lean UX, so he's all about that build to learn kind of stuff that is really focused on calibration and deliberate planning. And not just okay, shipped it, it's done, go. Mark Boyes-Smith: We need you to be able to track with that work so that you build confidence in the output. Like that's a critical part of this. for us as well, obviously we're looking at how we how we allow you to begin queuing tasks and and then being able to go back to them at a later date or a later point in the day so that it feels disconnected from the the synchronous work that you're doing at that moment. ⁓ and we see this, we see this everywhere, right? I and I think it's gonna become more prevalent. The interesting bit for me is like what happens when ⁓ Dan: ⁓ Nik: I don't know if we've really had software that is done for quite some time because a lot of software has been ⁓ developed sort of an agile way. There's constant updates because it's over-the-air updatable or it's a web application. I mean, the famous sort of Google Gmail was in beta for forever, right? And it was just always in beta and they were always updating it Dan: Ha ha Mark Boyes-Smith: the tasks begin and end with your agents locally and how does that piece thread into it as well. personally I'm very curious to see what the the orchestration looks like when you bring many, many different types of agents together and where that lives. ⁓ The ⁓ I the jury's out there. Nik: Now of course there were features that were done and then they moved on and you know they haven't touched those. I like a lot of what's here. I'm just I maybe I'm not so sure that the ⁓ the done part is is to focus on. Yeah, this is something that we're seeing in our software engineering tools, right? You can have, tens of agents going out, and now actually people are trying to figure out orchestration tooling. I've been seeing new experimental like terminals. it's funny, so many of us have now migrated into using terminal based tools. For many designers, this is actually pretty new, like a Dan: I do wanna say that ⁓ while I definitely agree, having been on some of those teams, there's like the done in theory and the done ⁓ in practice. ⁓ And a lot of times it's like, well, we launched that, we're done, move on it did meet the checkboxes, like okay, is it? Nik: Yeah. Terminal interface is not even something actually many students in my class, I'm the first time that they open up terminal.app on their on their Mac. And I'm seeing these experiments of how people are going to manage these agents, but I'm not sure we have great design patterns, especially in more graphical tools. It's actually interesting Miro has an infinite canvas, I could totally imagine. Dan: Hitting the metrics, yes. there obviously might be some tweaks and stuff that happen in there, but it's also like, okay, we're finished with this, we never have to worry about it again. We know that X equals ⁓ Y. ⁓ Bye. ⁓ And now there's stuff that you could that from or months now starts to act up ⁓ Nik: these agents working in sort of their different spaces. Also you can imagine having different panels. It would be really cool to see where your team goes with experimenting with that and sort of seeing what this future of multi-agent orchestration looks like. Dan: And maybe it is a model change or ⁓ new data comes in that the feature to be calibrated again. I think that's what he's talking about here. Mark Boyes-Smith: this should be fairly obvious, but at Canvas we shared probably 20% of everything that we've behind the scenes and we're excited about. ⁓ But ⁓ I would agree with you. I think that's a really fascinating space for us. of the things that I'm excited about is seeing how this idea of like you and your agent coming to the party almost with the agent as like the plus guest, what does that actually feel like in the canvas if you and I Nik: I actually really like the aspects on building in the capacity to triage failures, being for it. ⁓ we try to teach students about this, but I think developing a professional practice around this is ⁓ still not fully out there in regards to ⁓ some software. Mark Boyes-Smith: There together. I brought my agent. You might be using Miro's native sidekick. You might have brought your own. Like, how do we start to blend that world so that it feels like a natural resting place? and then agent to agent as well, right? You know, if we've got both of our agents sat in the canvas working together, what does it feel like when we go in and join them to help decide where we go next or, take a directional move? Nik: I think also the idea of the tripwire where you're saying, okay, we're gonna roll this back. I mean, that that's also really interesting and I think important for people to consider. And it's more so not that, ⁓ you're thinking like, well, we're we're gonna fail and we're gonna have to roll it back. It's like how do you do that in a graceful manner? hopefully your customers are coming along for the ride of like, yeah, okay, you tried this, this didn't work. how is your team thinking about strategizing on ⁓ what to prototype and then what to potentially ship? There's so many people move so fast. ⁓ Put it back, give me the thing that I know and love. It's okay. Versus people getting really upset because the risk here if people get really upset with what you've done and how you've changed their software and you've ⁓ up an experience so poorly, do they switch? Now a lot of things are really sticky. And so you people will stick with things for a long, long time and can kind of deal with a lot of failure and a lot of potentially BS. What I actually focus on so that we can push it out and actually ⁓ see hopefully great user experience and a great return on our investment before they really do make a change. That being said, if you're not doing well, it is an opportunity for competitors to come in and say, hey, we, we're offering something that doesn't have these issues. Mark Boyes-Smith: I'm gonna give you a bit of a woolly answer, but it's it it's ultimately about like how we build conviction as a as an engineering product and design team. Like I think one of the things I've been really excited about over the last couple of months is how ⁓ Because we're we're spending a lot less time on the building, we spent we can spend a lot more time on the critical thinking. And we've managed to kind of like tip the skills a little bit for ourselves. So the the way we think about this is historically prototype in Figma, probably like a it's like a one or two-week thing if you're talking about something of like substantial complexity. And then somebody comes in for a review two weeks later, and you got to break the thing and start again just in order to like work the feedback in. Dan: I think that the concept of a tripwire can also happen pre-launch ⁓ I think seeing this a lot with anthropic right now, where they're like, ooh, hey, this could be very dangerous. We are not gonna release this at all until we are certain that the guardrails are good. And I think that is an important Mark Boyes-Smith: The working relationship that we've managed to develop quite quickly actually is like, hey, okay, somebody takes a stab at a prototype, maybe two people, they're kind of like jamming on different branches and merging things. That's become part of our kind of core lexicon. get to a point where they want to share it out. Well, now what we have is the ability to like branch and remix. So we see many more people start to come into the fold with new ideas. And so we find that we get this very Dan: kind of tripwire too, not just about after we've launched and bad stuff is happening, but if you do these kinds of upfront exercises to think about negative consequences and you're seeing a bunch of them, you had to be able to pull the emergency brake and like, yeah, we probably shouldn't release this. ⁓ Mark Boyes-Smith: snappy proliferation of ideas that help us diverge quicker. That has been absolutely amazing. Don't get me wrong, chaotic at times when you have like 30, 40 versions of something, but a really, really great opportunity for the team for everybody to feel like they're they've got involved. And then we figure out, back to that idea of building conviction. We take a a view on everything that we're seeing. We look for the ownable moments, the things that will offer Nik: I do sometimes wonder if Anthropic does this as a marketing ploy sometimes 'cause they say, Well, we've got something and we're not gonna release it, but we'll tell you all about it. Dan: Ha ha It's so ⁓ it's so amazing that yeah, it's it'll blow your minds, but it really it's just for us. Nik: Right, but or or that it's it is risky. Now, at the same time, I I think that there are a lot of people at Anthropic who are serious about this. I I I do think from what I've read that they're they're serious about the safety aspects of stuff. But it is interesting they tell us. Like I'm trying to think of like, you know, another product saying, Hey, we we were gonna release this project product. Mark Boyes-Smith: a degree of delight to our users and I think that's an overused term, but you know, we don't just mean like ⁓ sparkles that dance, but actually more the things that help you move your work forward and you just weren't expecting it. Like that's always a delightful moment. And when we can shave an hour off your day, we for those opportunities as we consolidate on what we think is the right idea. Nik: That was gonna be an AI enhancement and we thought it'd be really cool, but actually we realize it's pretty dangerous, so we're not, or we're only gonna release it to a select few people as we're testing it. Again, there's sort of a marketing, whereas I think a lot of other features just kind of come out. Are you together in a room like with maybe a big Miro board on a screen? Like actually I wanna kinda get a sense of like ⁓ how do you manage this chaos as a team? Mark Boyes-Smith: Yeah. so we're we're pretty remote as a as a design team. We have folks in Amsterdam, Copenhagen, Berlin, New York. And then I am the lonely guy in in the north of the UK. We've got a few folks dotted around the UK in London as well. so a lot of the that we have tend to be live calls just like this. We always have the Miro board to side. It tends to be like our Miro boards are insanely chaotic. we all know exactly how to use it and we all know the different tools that are in there. So we can we can benefit from a little bit more chaos than maybe what some of our customers would be happy with. so we build all of our ideas out there. We tend to bring them in. most recently through the prototyping application in there. So like the custom and being able to drop that into the canvas. That is ⁓ awesome because for the first time. we can actually collaborate on a prototype, which until now has been incredibly difficult because you're I mean, ⁓ we had that in Figma and that was wonderful, right? But the second we moved into code, it was like me, my Claude and no one else. And I could bake my ideas to the point where I did not want to change them. And that became problematic very quickly. So now we can we can use Miro to pull them all together and then be able to comment and vote on the things that we like and use that as a human merging strategy in a way. That's that's how we do it. Nik: So when it comes to actually everybody coming up with the ideas, is this ⁓ just a complete free-for-all? how is it that the team ⁓ is actually being inspired? Is it that they're looking at new technologies that are coming out and they're just coming into the office or the the virtual office and throwing something on the board, ⁓ does this sort of ideation come from? it be that teams are getting better, right? They're they're putting out features that are maybe less risky. They're putting out features that they're getting a sense of what works well, how do we actually some performance so that we know when we roll it out that it's gonna be taken up well by users, they're gonna enjoy it. Mark Boyes-Smith: we are very invested in looking out there and seeing what the market's doing right now. we have a very strong story that we tell ourselves around the role of collaboration, especially in the new era of AI. And I think both of those things help really guide an ideation session. we always bring things to the table, like that's always the starting point. ⁓ the other Dan: I think the foundation models now have more guardrails on them they ⁓ did a year and a half, two years ago. Nik: It's true. That's a good point. another thing to consider here is the area that you are working in and the risk levels that are apparent. Waymo has actually had a number of rollbacks on certain things because they've had incidents. And you know, one example is they were driving into flood waters, and so they've actually stopped, cars in Texas. Mark Boyes-Smith: Side of it is like, yes, people, anybody can get involved. We're we we really like to democratize access to certainly early ideation. Because I think one of the wonderful things that we've seen from having so many prototypes, ⁓ which I'll come on to like how we kind of surf through them in a second because I think it's quite important. is that we always find a little nugget of joy in each one. There's always something that we to take back. I mean, like voice mode, for instance, that we shared in San Francisco, where the the tool. Nik: ⁓ where flooding can occur during heavy rains they they have a problem, they recognize it, and so they're like, all right, we're not gonna drive anymore. ⁓ it's a big story, but that they are doing rollbacks, and actually Waymo of any company sort of has like a process, but Waymo also has a pretty serious safety culture because it's a lot of their safety culture within the company is ⁓ based in engineering, ⁓ analysis. Mark Boyes-Smith: Toolbar shrinks and you get that gorgeous, like yellow ripple. This was something that we'd prototyped with our creative studio team in an afternoon. We were like, ⁓ we kind of love this idea. We put it to one side for ages, we didn't bother with it. And then all of a sudden we were like, Do you know what? I think we've got to put that in there. That's gonna give it the like the shine we need. And we ported it over. And because it was already code, we could work with an engineer really quickly to get that in the mix. Nik: ⁓ this is something that I'm not entirely sure many ⁓ designers, if you work on say consumer software products, if you've ever about. I know many of our students haven't heard of this, although that's our goal is to help to help teach in this. But that really sort systematic exploration of failure modes and their potential effects, which comes from manufacturing from really, really risky. Mark Boyes-Smith: Yeah, and that's that's how we do it. But in terms of how we share prototypes, this is a really important one. Yes, they happen on a Miro board, but you know, you can't you can't bring a thousand prototypes onto a Miro board and look at them and make any sense of it, obviously. we have like the geeks in the design of which I would I aspire to be one. ⁓ the folks who come a little bit more from the engineering side. ⁓ And one of our designers, Mauricio, built this Nik: ⁓ energy production, nuclear, military systems, space systems, flight. I I don't know if it if it exists ⁓ broadly in products. But I wonder if they ⁓ people need think a little bit more about that. And I think that's actually one of the things is really exciting about Jeff's article is because you're kind of getting at like, ⁓ hey, this is a short piece, but maybe you should start to think about this more. And ⁓ what types systems would you put in place at your company to be ready to do this type of Mark Boyes-Smith: fantastic tool he called Viblab. ⁓ I don't know if the name is going to stand the test of time. Sorry Maurizio if you're listening to this, but like I what's amazing about it is You can upload the prototype. You can see all of them, who's built out prototypes. They're automatically served internally behind our VPN. And then you can branch off them, you can remix, you can comment on them. And that was a That was a a complete moment for us. All of a sudden we could start to work together again between that and the Nik: ⁓ thoughtful and responsible development. Jenny Wen from Anthropic has talked actually about some of the ⁓ really important things for designers today, maybe because they can create so much more, is actually being to find those moments and to create those types of delightful features. is this something that you feel the team just has naturally is this something that you look for when you hire designers? and talking about some riskier applications of AI. ⁓ Amazon has released some updates to their Proteus warehouse robots. you can talk to them in natural language and they'll go off and do tasks. This as I mentioned a little bit earlier, a bit new for us on the show, but we are interested in the applications of AI and designing user experience beyond just software interfaces. And actually, it's going to become more prevalent in real world Mark Boyes-Smith: having great product taste is obviously important for a designer, but there is a special type of designer who spends a lot of time looking out there at what everybody else is doing and trying to come up with their own take on it. ⁓ the last business I worked for, I hired a fantastic mid career designer who did exactly that. She would literally spend an hour a day surfing through Maubin. Nik: interactions, robots being a good one a good example of that. what typically was a highly orchestrated, pre-programmed operation of warehouse floor robots. So these are imagine big, big Roomba robots that can lift up super heavy ⁓ sets of crates and move them around. So crates of products and move them around the warehouse. They now have an ability to ⁓ listen to a human worker that says, hey, Mark Boyes-Smith: Picking out all the different little bits she liked. She'd be diving in, she'd be sending me ⁓ videos of interactions. And it was always so inspirational because it was the starting point for a conversation of like, wouldn't it be cool if we could do something like that? And I I I I often think of her when we're hiring because I'm like, that is exactly the type of mindset that I would like to instill in everybody. I would like to have more of that myself, if I'm honest. And I think it's absolutely critical. the other side of it is not everything in your design should feel ownable and unique. ⁓ I think a lot what we need to do today is be incredibly familiar to people coming in. Nik: take this thing over here and instead of programming it in, they can just talk to it like they would talk to any other human. And then it has the AI to sort of figure that out. Mark Boyes-Smith: You know, not all Miro users are in there every single day. Some folks are brought in via workshops. Some people might come in for certain moments in their team's calendar. And then obviously you get the power users like me. we need to make sure that when you come in, you can bring all of your experience that exists outside in those other platforms into us. And maybe there's one thing that we do different that you're like, ha, that was cool. Dan: when I worked in consumer social ten ago, this was a dream, like that we could be like, ⁓ hey, go over there and do this thing there was ⁓ some kind of rudimentary Alexa style voice stuff you could do, but not this kind of discussion that they're talking about. being able to talk to things and have them ⁓ you're trying to do with them, I think, is a really interesting Nik: is your team, and specifically when thinking about features, looking to users for inspiration? Dan: space to play in whether that is for industrial stuff or even can imagine things like lamps like ⁓ could you could you aim that light over here a little bit I mean this is full on Iron Man first stuff where he's like got the little helpers that are doing things and Mark Boyes-Smith: like any great design team, we're really deeply engaged with our users and putting concepts in front of them early, they're helping us to really guide the evolution of those capabilities. ⁓ And and what's really nice as well is like help us understand the the true value of them and whether something that they would adopt or not. Nik: This is Mark Boyes-Smith: And then the other side of it is it it's really about observing the usage patterns of AI. When are our users actually engaging with it? What is the length of those sessions? Like, can we kind of introspect there maybe what some of the success of the session is that they might have had? obviously, like any business, we're limited as to what we can see. and so we have to look at those behavioral patterns to understand like, okay, is the user actually picking up a next step here? Are they producing something on the canvas? Are they then Dan: I think it's I think it's really fun. I think it opens up a whole new venue in this physical AI space. Nik: I also think that this is something where we will see a lot more natural language ⁓ physical interaction ⁓ in semi-controlled like a warehouse and so There probably are gonna be more opportunities for designers who work in this space to start incorporating AI ⁓ within these interactions and moving away from everything being totally programmed. This is something that we Mark Boyes-Smith: interacting with that thing or is it being deleted? And so we look for a lot of those behavioral patterns as well to help us understand whether we're actually helping to move the work forward or there are areas where we can double down a bit further. What's wonderful as well is because we are own first customer. We dog food our product in a way that feels wonderful. ⁓ it means we get to observe our own metrics as well. whenever we release a feature, we always release it to Miro as first. I I don't think we are Nik: study a lot here at Carnegie Mellon and there's a lot of researchers out there who've been working in say the human robot interaction space for decades, but it's gonna be a interesting time, I think, to start seeing stuff move its way from the lab and into real world operations. Mark Boyes-Smith: ⁓ unique in that and and we make sure that when we release to mirrenias we have a really high bar for quality because we know we want them to give us great feedback first. Dan: Our last story is an article called Designing How Designers Master AI by Amber Buabdallah. And this is a really interesting article, and one I'd like to see more of is just showing how teams and individual designers are working to master new tools. And I really like what she's saying about mastering these tools isn't about doing the one thing correctly. It's not like learning, I know, auto layout or something ⁓ in Figma. It's more about learning to the tool to like what Nik: Yeah. Dan: you would already do so that mastery of it is personal and a little bit more fluid. And the other thing I really liked about this article was that it about how much peer to learning was going on Nik: so ⁓ the world is changing really, really fast and the AI integrations are happening at a rapid speed. Dan: with her team at at Salesforce and think that's been ⁓ as we've to a number of different people on the pod and outside, like that's been huge. because everything's new and everything's fresh, like learning from each is so And ⁓ Nik: if you were talking shop with designers who are there listening, what would you say to them ⁓ to help them start to make sense of the chaos of the moment? Mark Boyes-Smith: I think I think there are things really. the first one is we're all updating our skills right now, we're all expanding them. and I won't wax lyrical about that, but I think what's critically important there is like design has a very unique opportunity in this moment to be the glue for all of the other functions that make great products. Because we can be a little bit strategy, we can be a little bit technical. Dan: So essential, like learning just ⁓ like, ⁓ that's how you did that? what's the prompt for that? And you see this everywhere right now, not just among designers, among anyone who's playing with these tools. ⁓ And I really think that culture of reminds elder me. ⁓ Mark Boyes-Smith: And actually, if we ⁓ really into blurring the lines, we have an opportunity to help many more people bring their ideas to life. And I I truly believe that. I think an enormous opportunity for us there. It relies ⁓ on, ⁓ this where the change comes in, it relies on us relinquishing some degree of control about how those ideas actually come to life in a product. You end up catching quite a few more areas where you're like, hmm, I probably wouldn't have approached it that way. Nik: Yeah, Dan: So much of the like late 90s web stuff where it was like, ⁓ well, how did you how did you make that frame? Or what how did you build that with CSS? Or, you know, those kinds of like old tricks that were just kind of passed around from person to person. And it was less about like, ⁓ you're gonna read this book because In the late nineties, like the books would change so fast that stuff went out of style almost immediately, in the same way that ⁓ stuff is happening right now so that it's very hard to capture that ⁓ in anything ⁓ solid Mark Boyes-Smith: But actually we have a role then as stewards and we have to figure out better systems, better design languages to support and guide that development when we're not in the room. I think that's that's kind of one area. Nik: yeah, sweetly. be very difficult. Mark Boyes-Smith: The other area for me is is one of resilience. I think we're in a moment right now where businesses are running incredibly quickly. we're having many more conversations about how we prioritize our efforts, where we focus, where we think the next gaps are in the market, or some of the interesting collaboration patterns. And so I think resilience is a a really important part of the role for each of us. And I don't say that lightly. I don't think resilience is something you can just click your fingers and it appears, but finding the things that feel Nik: are doing a lot of this within my group and then also ⁓ my class, we're doing ⁓ sort of peer sharing of how we work with AI. I'm finding that a lot of students really value seeing how other people work, like almost like watching them go through something and actually take the steps. Mark Boyes-Smith: timeless and enduring, hold on to those and then find ways to engage with the things that feel a little bit more chaotic or frenetic. that's what I think is an incredibly important skill for us all to develop. I'm I'm doing that myself. I have to figure that out. ⁓ but I think that's an important part for designers looking forward. We have to be able to inspire and then navigate the the slightly rocky path to getting there. Nik: I'm actually likening it now to like almost like a cooking show. Like I actually love watching cooking shows. And part of it is cause I like to watch, food being made. I like to hear about how to prepare, say, recipe. But actually there's a lot of other knowledge that's being conveyed in like watching people how they cut something, watching people how they flip things in a pan, hearing about the little tricks of like, ⁓ season it in this order. Amazing. Mark, thank you so much for being on the show. This was awesome, inspiring. again, you so much for being here. Do it in this way. I mean, I it's goofy. I just watched a video on making smoothies and apparently I've been making smoothies wrong because I don't load my blender correctly. I'm always sitting there fighting the blender to actually get everything. And it's like, no, if you actually put it in this order, then it'll all blend up really nicely. And then remember to start on slow and go high. And I was like, wow, okay, cool. Thanks, six minute video for for helping me with that. But I That's our show for today, everybody. Thanks again for listening. Join us next week for another set of news, deep topics, and more guests here on AI and Design. probably wouldn't have figured it out myself because I wasn't experimenting with it. I was just, well, I want to make a smoothie. And I think this is one of the things here is that we're in this phase where everyone's experimenting and trying new things. They're learning from all these areas. There's so much going on that we are often better off from each other when we find these tricks. And so it's to see. And I think the article has some nice case studies. that you can read about of a variety of tools, people coming in showing off things like Notebook LM and then moving to showing how we use cursor and then how they used internal tools that Salesforce was developing. And then finally example of using Figma Make. So, yeah, take a look at the article. I think there's some good inspiration in here for you and your organization to think about how to potentially share what you're doing, learning how to use AI and work with AI tools within your own company so that you can do this sort of peer learning. Also, this actually flows really well into the interview that I had with Mark Boyes- Smith, and parts of that go over how his team is. thinking about using AI in their own workflows and how they're teaching each other and leveling up each other's skills as they figure out how to work with these tools.