Kevin Williams: All right. Check that out. A new intro. That's that's AI Kevin to like the nth degree there, Matt. I I might have to have a little conversation with with Gabriel, our our video editor about how little that looks like me. Yeah, it is it is like this weird uncanny valley when you're looking at an avatar of yourself like

Matt: Yeah, I like that. Well, everyone's always saying you're replaceable, right? So here we go.

Kevin Williams: Not to mention, I mean, you know, I I am I am like tipping towards fifty and it made me look like every bit of fifty. So that's that's a little awkward for me.

Matt: You know what's pretty cool actually? Our team, so I have a lot of content online, a lot of videos, specifically shorts. And sometimes when they need a like an action shot or a B roll, AI is actually really good at doing that because it's just a quick clip. Like if you stare at it too long, you can tell it's AI generated, right? But like when it's B roll and it's just flashing on the screen, sometimes you just need a filler. And AI's been really, really good for that. Very interesting.

Kevin Williams: Yeah, yeah. We use descript sometimes for for like audio burbals and things like that. but like wrap on a little bit on that for just a second, because I think I have actually struggled a lot with Hey Gen and Sinestia and the avatar companies that they actually are working pretty well when you use a stock avatar that's off the shelf. But as much video as I feed into these things, I can't really I I still don't feel the the Kevin Avatar or the fake Kevin Avatar is enough for anything more than B roll.

Matt: I I have so much yes, exactly. I've had the same struggle. And I've been doing this for like two or three years when it first came out because part of our like one of our tactics for getting calls booked is to do videos, send videos to prospects. And like so I was really obsessed with scaling this. Back in twenty twenty one, I was in the top point one percent of Loom video creators because I made it so many custom loom videos to get these calls booked. Then we tried to automate and scale it. And we've been using Hey Gen and other tools for years now, and it's still not quite there. So a couple of trip tips and tricks for folks if they are interested is if you use a small little segment, like I described, like a b-row, a flash here and there, just where you need some filler, it's pretty good at that. Like if you paused the screen and just stared at that one frame, you could probably notice, but they're going so fast nobody notices. Then the second thing is if you want to make a custom video, sometimes you can put your picture in the bottom as an avatar. Profile, like you know, little circle in the bottom corner while you're sharing your screen on something else. And so when your face is small, like people can't really tell because they're focused on what you're doing on your screen and not necessarily your face. So that's like another little tri trick tip. And then the third thing is the voice, there's a huge variance in the voice output. So sometimes when you generate from a script, the voice is really rock solid. But then other times

Kevin Williams: Mm.

Matt: If you generated your voice a hundred times for a hundred different scripts, there's gonna be like three where you're like, Whoa, that sounds like really bad. so that makes it actually kind of hard to catch. You need a human in the loop to kind of review like what was generated and make sure it didn't go off the rails on some of those educations.

Kevin Williams: Interesting. So a couple c two two quick points. one is, you know, we do custom thumbnails for every one of these episodes. And that's basically that reference image is really important. So the prompt is really important that it pulls from the transcript, it tries to figure out something clever, and then it has images of you and me that are reference images such that it can tie to it, but it drifts. The other is what you were just saying about human in the loop. I don't know if you've noticed, but so we're recording on Thursday, the 16th. The last couple of days, at least in Anthropic, the Opus models have been really dumb. And like I feel it because I live and breathe in them all day long. And then there's this cold realization that we have these operating platforms like my LeadWorks tool and customer-facing tools that are out there, and they're running on Opus as well. And although we do have Pretty balance prompts and we've tested a lot of that. if and we have feedback loops in place such that it can flag issues. People need to be aware that that quality can change like in a given hour or a given day in ways that is really pretty distressing. Like if you're using Opus 4.8, you sh you you most people would just assume, hey, it's gonna be great. Like, you know, it's gonna be doing the thing I want. And then on some random Wednesday, like it's dumb. And so your your your outputs are going out the door with that like slower and slower model. So it's it is kind of just a point in time right now. And I'm curious to sort of serve on your take on that in general. And when you're working with clients, how you do Prompt like quality outputs. Not necessarily getting like over t overly techie, but sort of philosophically. How do you guys think about that?

Matt: Yeah, it's a great point because the variability within a model, if you run it a thousand times, ten thousand times, there's gonna be some amount of variability on the output. And then like you said, when they're always making tweaks and everything, it could even vary depending on the day. So you have to be able to put guardrails and in place, what we do is we have multiple models check it. So we may have three to five models on on really particular outputs. now that you have to be careful of cost, right? Like we've talked about that. That can get outrageous. but Either having a human in the loop or having other models do double checking for you is very helpful. but really I think in most park people in the human in the loop is how most people are catching a lot of this variability. And it's it's kind of funny because like we've been talking about self-driving cars for like over 10 years now. There's clips of like Elon Musk and everyone talking like 2015, 2016, self-driving is here. But it reveals like how hard it is to get those edge cases and that variability. out of the, you know, like the the model, right? How the model can solve that. With a car, extremely bad if it's there's an edge case. It hits somebody or it goes off the road and kills somebody. For our models, they just deploy them and like say, yeah, yeah, like people can d deal with like the five or two or one percent variability issues that these models have. but it just it remo should remind our listeners like it it's not perfect yet and it's going to take a long time to squeeze out that last few percentage points of perf perfection.

Kevin Williams: It also matters what the application is. So if you're putting out, you say a blog, it doesn't really matter too much if the language is is a little bit off, but if you're doing some sort of financial analysis or you're putting pushing data around, the other thing that we we really look careful carefully at is how much of this process can we make deterministic? Like we want to strip out as much of the probabilistic like LLM layer as we can, particularly when you're using numbers such that those numbers stay intact. Up until points where that intelligence layer is is needed. But yeah, it you know, it's it is amazing, the tools are great, and then all of a sudden they're not. And if you put it occasionally it equals 4.1, it could do all kinds of interesting things there. So yeah. So for the listeners, just think about the type of things that you're putting out and the things that are on autopilot and whether or not you're keeping

Matt: Two plus two doesn't equal five sometimes. Yeah.

Kevin Williams: human checkpoints in the loop or at least you have like model fallbacks in the loop. And as Matt said, like that can get pretty expensive if you're running frontier models on top of frontier tier models with the tokens, etc.

Matt: So Kevin, quick question. Like with the I think the numbers area is something we see with clients a lot. There's a lot of interest in automation and agents work within marketing and sales departments, but also finance departments, especially creating dashboards. Now dashboards, of course, you want your numbers accurate, but it's not devastating if it's four point one versus four, maybe sometimes, right? Depending on the business. Still doesn't sound very good to me. But like if you're sending invoices or you're doing payroll or something like that. Of course, like every single penny matters on that stuff. So I like how do you do this deterministic how do you connect some deterministic model or process that can put some guardrails around a an inherently stochastic process or a probabilistic process?

Kevin Williams: So, you know, d d at the at the risk of getting too technical, I mean, there are coding solutions for for for dealing with it. So you know, OCR image recognition has been around for a long time, being able to pull out highly accurate OCR data and then using basically Python. to interpret what's in there. You're passing solid data. There's no AI in there. You may be, you may be using AI in your coding development, but you're developing deterministic code that's handling it in the same way that an invoice processing system would forever ago. And you know, I I call it the spaghetti bowl problem, where people look at wow, I have all of these invoices and I'm just going to like peer into all of the invoices with an AI and You know, first off, that's not gonna work because there's just too much information in there. So they say, okay, cool. What we're gonna do is we're gonna create a reg pipeline. So retrieval augmented generation, most of the audience has a pretty good idea of what that is, of turning it into data such that the LLMs can then look at all those invoices. That's still a mistake in general because you're looking at this giant bowl of data, and it's pretty easy for individual line items to get confused. even when you're dealing with vector databases. So we had this challenge a while ago where we were working with an insurance company and they had there was like an audit of of of of liability insurance. And it turned out that it was like 10,000 policies for white Ford Oconoline vans. And the only real difference between one painter van and another painter van would have been like the vehicle identification number. And even with a vector database, it can it can like bleed together. So what you want to do is use more deterministic coding to separate out like what the individual identifiers, policy number, VIN, et cetera. And then you're going to query that as a on an individual basis. And then when that comes out, now you can use the AI to interpret what it's looking at and inform whatever that next process is, but it's highly deterministic.

Matt: So th that's exactly what we do. Like try to like as part of your harness, try to like call your coding agent to code in the deterministic parts of any workflow. I think this is why coding was such a huge unlock for these models tool tool, because as soon as they were able to solve coding with AI, they were able to create this loop of like.

Kevin Williams: Mm-hmm. Mm.

Matt: This probabilistic gen text generator now has the ability to realize, like, we can this is somewhat deterministic. Let's go code a solution and it creates this loop. It goes there, then once that's done, it goes and starts this process again with the agent, and then it can go back to code. So I think that's why coding was such a huge unlock in the industry for making a gentic AI. You know, it's still again not perfect, but that's what allowed us to have more precision and accuracy with these agents that we're building.

Kevin Williams: So I'm not gonna be the guy who casts shade at vibe coders, but the the the only way you're going to get to a result like that is to understanding what you're asking for and why. And this is why there's been such a big appetite for like mid-career devs, like more senior devs out there. it's really still not a great time to be a junior, junior dev. But people who have enough experience with enough different approaches that they basically are asking the right questions. Like the LLM will happily cloud code, if you just tell it to solve this problem, it's just gonna do the spaghetti bowl thing. So you need to be able to ask it in terms of your source documentation, like your markdown approach or whatever it is, the right questions such that it leads it to that deterministic approach. And the good news is you probably don't have to write the Python. It's going to write that for you as part of its development approach, but it's not going to do it unless you tell it to.

Matt: Exactly. Yeah. it I met at this these I th I actually have known them for a long time. And so they're taking the approach of trying when you have a workflow at and we were we're talking in the c the start of this with numbers, like you need the numbers to be very specific. What they're trying to do is in c make those deterministic coding environments expand beyond just two plus two is always four. into like, hey, this step always happens in this way. And that's like a deterministic rule that we can put in code. And so when you run your agent, it has its probabilistic side, but it's able to call this kind of like governor that has like a set of deterministic rules that never can be broken. And so if you put those rules in your probabilistic agent, I'll say, yeah, I know that's the rule, but sometimes, you know, like the probability is we may not follow that rule. But Yeah, so it's an interesting concept of they're just trying to take more steps on the coding side to be able to make these more deterministic steps for like very sensitive work for f workflow projects that they're trying to build agents for.

Kevin Williams: Yeah, super smart that, you know, I I I'm I always encourage people just to get going and and experiment. But you know, we have a funny we have a funny thing going on where we're starting to get hired to like surper supervise the inmates, essentially, where where an organization has hired like you know, it's basically some smart young thing vibe coders and they're like running around doing things and nobody in the organization knows what they're doing. And it's not like the the no malfeasance here. It's not like like like the the the people who are doing those those roles like aren't like well incentivized and they really want to solve problems, but they don't know what they don't know. So they're doing all kinds of kind of crazy hackery stuff. And the organization itself woke up one day and is like, wow, we don't have a CTO. We have no idea what these people are doing. And yeah, they're contracting with us to basically create that adult supervision such that we could add a little bit more structure and rigor. And again, encouraging that rapid development, but encouraging that rapid development in a way that's that's wrapped in a repeatable process. I know, whoa, we can't just like do whatever Wild West thing we want, like between each individual project, but standardizing that such, you know, hey, if somebody leaves and somebody else comes in, like, you have your standard approach to the way that you're building things. or worse, what's happening in some organizations is that bad process is basically being institutionalized. That you have, again, well-meaning, very smart, vibe coder doing the thing. And then they hire another one and the first one trains the second one, but nobody has addressed like security vulnerabilities or standard data sets or like all of this other stuff. So There's a there's a lot of maturation that that organizations need.

Matt: Yeah, it's I these citizen developers have been around for like ten years since the dawn of like Zapier Airtable and these no code tools. So this is definitely a conflict that's happening. on our side, they just they want like blueprints. What is you know, like your governance blueprint or draft or template or how do we manage permissions to different types of databases? How do what's our framework for thinking about which data is accessible and not? Like you said, it's really easy to kill that wonderful entrepreneurial spirit that those people have in their in their business. With their employees. So cool. Kevin, what else is going on? You spent a lot of time this weekend, you were telling me. so I'm excited to hear what you what you find out.

Kevin Williams: Yeah, I mean that this this show is all about candor, but I I had I had a pretty rough weekend in the the AI trenches. the deal was that I la I'm building all kinds of things internally. This this is generally what I'm talking about here are what you and I have termed before like side quests, like l little process improvement things, experimentation, whatever it is. And what I had done. is I had built this behemoth of an individual database project. So you know, again, to serve calibrating a little bit, like you're gonna build something, whatever it is, has a user experience. That's okay. So there's the UX, right? And then there's should be a back end of it. And we tend to recommend Superbase as a Postgres database, but you can use Cloudflare. There are other ways of like hosting the data that that sits behind the app. And then you have a deployment environment. Again, internally we tend to use Versal for that, such that that your front end interacts with the back end, and the two of them can be deployed somewhere where people can access it. And I got myself belt bound all into a pretzel because I wanted to build something that's increasingly common in SMBs in particular, which is like an internal app framework for all of these different tools. that the organization is building. And usually that it's a little bit ragged where people are you know, they build this whatever little markup tool or whatever, and it has a login or it has no login and everybody just gets a link and maybe it's in there like pinned to a Slack message or whatever. But the right way to do that is to have like user authentication such that different people in your team have different access to different apps and it all lives in like an app of apps, essentially. And what I had done is sometimes my experimentation ends up kind of creeping into beta experimentation with clients and not like production environment type stuff, but This internal platform was all intertwined with things that had morphed into sort of external platforming. And it was just a complete disaster. And what I needed to do is strip apart the back-end database and reconstruct it in a new database. And it took like 15 hours to do. I burned entire lakes in the Midwest using. using a fable, which we've we've talked a little bit about fable in the past, which is you know, we actually it's a topic of conversation. Yeah, yeah, your nuclear scientist finding your taco truck, basically. But in this case, it was a pretty good use of it of you know, there are like millions of lines of code and like deconstructing the whole thing. But at the end of the day, it was like diffusing a bomb. So I had to do it like step by step.

Matt: Nuclear scientists finding a taco truck.

Kevin Williams: And I've known that I was gonna I I was doing this for a while and I just couldn't face it. And like because of Fable and because of the fact that the the Anthropic kept expanding the usage of Fable, I decided that this was gonna be a good use of my tokens to like try and and do it. And yeah, so it worked out. But the the the the pieces of it of of you know, do what I do, not what I say or do what I say, not what I do here is If you're building, you do need to keep your back-end projects separated unless you have a really good reason for doing it in terms of like cross-app usage. Like my rationale made sense because I wanted everybody to use it. The problem was once it crept beyond internal, now I have a problem. And the way to deal with that is you set up another project. And yes, a superbase project will cost about $10 per project per month. For most mer mere mortals, we're not talking about like like huge money, but you get a you set up that separate project. And then what you need to do is you can actually create like parallel paths in GitHub such that even if you have a customer facing version of something and an internal facing version of something, you you basically can update both. It adds complexity, it adds a little cost, but save yourself the pain that I went through and don't. Don't combine it into one monster project.

Matt: Okay, so a couple of things. This is your classic like tech debt, you know, which is becoming like you know, tech debt is becoming even more of a problem now in the age of citizen developers, vibe coders, all that. So like how how much to undo that fifteen hours of tech debt, like how many weeks had you been building? Like months, just a couple of weeks, a couple of weekends?

Kevin Williams: that was months. That was, that was, that was months. That was probably all of 2026 was was in there in some way, shape, or form. And I just knew it. Every time I'd go and do something new, I'd be like, my gosh, I'm just making this worse. I have to, I have to stop this. But and at the same token, the apps are functional. So they're doing things in the organization. So I had to do it over a weekend to make sure that I wasn't like

Matt: Months. Wow. Mm.

Kevin Williams: breaking things in the rest of the organization and sure enough, I broke plenty of things in the rest of the organization. But you know, now we're like we're band-aiding the rest of us together.

Matt: Gotcha. I think it's just helpful if so after like six months, you were able to solve a lot of the technical debt was 15 hours of work, which is pretty reasonable.

Kevin Williams: It's pretty reasonable. Like like don't get me wrong, like the sort of mess I created would have been like weeks of work for for a real like database architect to like figure this out in just a traditional super a SQL database. And fifteen hours is sucked, but it it it it was it was way better than it could have been.

Matt: Yeah, so I think it's good for anyone listening to this. Like you always have to put stuff relative, right? You're gonna hear people say technical debt is bad is bad. Other people say still vibe coding's awesome, but you gotta like it's all relative, right? So you wanna code fast for six months and then spend one weekend, fifteen hours fixing it all. That still seems like a good trade off from a business perspective to me.

Kevin Williams: Mm-hmm. So you'll go well well, I I I I want to interject because I think there's this like funny opportunity for for both of us or people like this. If if if I wanted to, there is totally this giant opportunity in AI plumbing. Like, you know, why do you call a plumber? Because your pipes are leaking or whatever it is. Like, absolutely everything that's been built in the last three years, all of it. I don't care if it was built by like really sophisticated devs or not, it all has leakage, it all has like token inefficiency, prompt inefficiency, security vulnerabilities, all of this stuff. And there's gonna be a really big need for AI plumbers to come in and fix this stuff.

Matt: definitely cleanups via vibe coding cleanup specialist. That's like the new c new job for everybody. so how do you think that changes if you have like I think a lot of people on the listening to us are maybe debating between like, hey, I can use my my kind of like tech savvy talent at the company to help me build, but I'm kind of worried about the tech debt that's gonna come from that or the security concerns. Like

Kevin Williams: Ha ha ha.

Matt: Who how much tech debt would have accumulated if you had a more senior engineer working on all this? You know, do you think you would have still faced that fifteen hour set of work to undo a lot of issues, or you think it might have got cut in half to like seven hours? Or you think you could've gone all of twenty twenty six without even needing to worry about it?

Kevin Williams: I don't think the engineer would have mattered. What would have mattered was a proper practice. So it's exactly what I was saying a few minutes ago about you know, internal teams not having a good rule-based set of approaches of when they do what and why. And we're very disciplined about it, of course, if we're doing, you know, production work, but you know, me or semi-technical members of the team who are sort of building things at at night on on the weekends, it's easy to get fast and loose with it. would an engineer right off the bat would have recognized the issue? Maybe, but the types of things you're talking about are high agility, high velocity, and and in a lot of cases sort of disposable. So you just wouldn't you just wouldn't have allocated the engineering hours to it in the first place. So It's probably my fault as a as a leader and the default like t technical leader of the organization, that, you know, I I have the privilege in the organization of being my own boss. So like nobody's just holding me accountable other than me. And yeah, I I did listen to our engineer. This is, you know, obviously this was getting flagged internally. And it it does come downstream of one of our full stack engineers saying, yeah. This is gonna be a problem at some point. And maybe we need to fix this. And then he, of course, would have would have spent a lot of time doing it from and I didn't really want to allocate those resources to expensive resources, so you know, off comes the band-aid.

Matt: An expensive resource to have on it. Yeah. There you go. Yeah. I love it. So you were using Fable. How did you get a chance to try soul at all? Because that that's what I thought you were gonna say this weekend.

Kevin Williams: Yeah, because I r so I ran I ran out of fable. I think a lot of people did. I completely tapped out my two hundred dollar max plan and it was just done. It couldn't do any more, which might actually explain given my usage in the last, you know

Matt: Was that after fifteen hours or like halfway through or something?

Kevin Williams: It made it to the end, but it was funny. It was like at 95%. And I just kind of screeched to a halt of like, uh-oh, okay. I need to re-reserve that last five percent just in case like something like big needs to be done. And then yeah, yeah. So I switched over to Seoul, which you know, we're we're not a business show, but like the switching cost is so low, and people should understand that that okay, a lot of people listening to the show use cloud code. Easy, approachable, great. Claude's my friend. Yada, yada, yada. But you open up Codex and you load your repo and it's the same. Like it's it doesn't really change your practice if you're not a more traditional dev who's using an IDE to do like file-based, you know, agentic type work. so yeah, the switching cost is simple. Just just open the repo in Codex. load up soul and continue with your day. and it was great. It was absolutely great. But I don't actually maintain the the like a full you know two hundred dollar plan on GPT so I burned through it pretty quickly.

Matt: Gotcha. Okay. So okay. Any any quick observations of differences or are we just like in my opinion, I don't get caught up too much in the weeds of which model's the best 'cause it's like, yeah, it's it's a point in time, but they're always switching. They're always like neck and neck. Maybe one's a little I mean, anthropic's been ahead for a while now, but I feel like open AI is is putting the fire and trying to catch up. Do you feel like soul is there or what?

Kevin Williams: I think it's there. Absolutely. It's great. I you know, I is it okay. I I you know, this is gonna be a little bit contrarian because because people are are just like gushing over fable and they're gushing over soul. And the fact is that I don't necessarily have the technical challenges that let them shine to a degree that the difference between opus and fable or fable and soul are immediately apparent. Like there's a there is just sort of a feel of, okay, well, you know, perhaps this is going a little bit faster and a and a little bit better. But we also have a lot of rigor in the way that we approach this in terms of you know, having proper skills that are built that and hooks that are built in the code such that it does quality control as it goes. So, you know, there is a level of discipline there. But the but that that obscures, I think, where these models Really, really shine. But from a business perspective, I think this is really interesting. I mean, Opus is great. It's a really good model. And now Opus is last week's news, and everybody talks about Fable. And here you have Anthropic continuing to expand their timeline where they they keep it pops up and it says, you know, 50% of your allocation can be used for Fable, and then you're going to have to pay on a per token basis. And I did run a little bit of Fable on just a per dollar basis and it was hideously expensive. It was just like a token, it was like a dollar vacuum. you know, I think I I think I burned like 50 bucks in, you know, 10 minutes or something like that. And I was I just retreated because the difference wasn't that important given the tasks that I was going, I was using versus Opus being more efficient, right? And

Matt: Ha ha ha.

Kevin Williams: This it's just interesting because at some point we don't I don't need fable seven or whatever. I think I've said this specifically here that I don't need fable seven if I have, you know, Opus five and it's cheaper and maybe it takes a little bit longer. you know, specific things. Look at all of my code base, figure out my data and my my the issues I created in superbase and help me resolve. That feels like a fable task and maybe that's worth paying two hundred dollars for. But I would probably be resistant internally using it unless I found a very clear avenue where it felt like it was gonna be more important.

Matt: Yeah, I think that's w we're gonna continually see that as new releases come out. There's gonna be a question of like what can I use this for? And we're starting to see some differentiation. Use this model for this, use this other model for this other aspect. But unless you're like an expert in the weeds, like generally as a business owner, I don't think you're gonna notice too much of the differences. You will in your bill at some points, right? Sol is supposed to be, I think, half the price of Fable. And that's why they're pushing it really hard competitively to get people to switch. But In general, I don't think a lot of business leaders are in the weeds, maybe technical leaders for sure, but you're gonna struggle to see the differences between models as they come out and more and more. Like the jump we saw from GPT two to three, huge, right? And those jumps are becoming less and less noticeable for folks like us.

Kevin Williams: Cool. so changing gears a little bit, you know, you and I were chatting earlier this morning about a lot of movements that are happening from a business perspective with how OpenAI and Anthropic are going to market. let's wrap on that for a second. So the the news that's been it's been reasonably well covered, but is definitely firming up is that Both Anthropic and OpenAI have built what they're calling forward deployed engineering organizations. Anthropics is is a funding partnership with Blackstone and Goldman Sachs, and OpenAI's is TPG Advent, and then they have like service partnerships going on with Bain and McKinsey. And the numbers that they're throwing at this are staggering. I think GPT's is $4 billion of cash that's been put in to build this thing. And anthropic is probably similar. And, you know, the, the, the business logic is is is really obvious that if you have a trillion dollars of infrastructure that's being built annually, that's what it is. It's going to be a trillion dollars a year of infrastructure. That is not going to be paid for by my $200 max plan. It's not going to be no way, no, no, no how. The only way that that is going to be paid for, and and another, this is we're not gonna go down this rabbit hole. I also contend it's not gonna be paid for by advertising. That the Google's entire global advertising business is 165 billion. And if you're talking trillions of need, you're not gonna have to like run ads to make all of this work. It only comes through token consumption. And the only way that token consumption happens is if organizations know how to use this. You you and I both exist because organizations are having a really hard time trying to figure out how to use this. So by creating these forward-deployed engineers that get embedded within business functions, they're amplifying the need for the tokens that are going to justify the business model and drive them forward. So this isn't just a four billion dollar investment in in a consulting firm, which would be completely nuts. It's a four billion dollar investment in essentially the future value of all of those tokens that make the economic all work.

Matt: Yeah. So when you when you think about the value chain of this market, you start with your chips, your data centers, your energy inputs, and the frontier models. All that is commoditized. What's beyond that layer is the application layer. That's where like Palentier has set for a long time. And they've always the CEO has always argued like the value in that value chain is not the chips, the energy, the compute, the data centers, or the frontier model. It's actually how you're going to leverage this in a in a business context. Palantir's obviously had tons of success with it. And that's what we're seeing. We're seeing all these other players, even AWS, you didn't mention, is also jumping into this game. And so people need help figuring out how to apply this technology to their business. That's what these forward deployed engineers are. They're sitting over the shoulder of the person in the department who's doing the actual workflow. And they're not just making an agent to do that exact workflow. They're rethinking how that workflow works and how. making like an AI native or AI first workflow to replace it. And that's where your junctive AI is. But I think that's where the massive amount of the value is in this whole, you know, multi-trillion dollar market is really at the front edge with the customer sitting at the desk. And they can't, these models actually need some sort of engineering help. That's why these forward deployed engineers are there, is because the models are really good at generating text. There's lots of text on the internet to train the models on. Right? Self driving cars. Lots of cars driving around with cameras to train that model on. But with workflows, there's no documentation. Right. There's not trillions and trillions of documents about workflows. And it's very hard to train the model on that. And so that's why we need actual humans, actual engineers working hand in hand with the people in these departments to understand their particular workflow to be able to install an agent in the right way. Because these models, I I personally don't think we'll just have enough training data to become super awesome employees or super awesome agents without that engineering help.

Kevin Williams: So that's I mean the the sideline there is that's the bet that like Mercor, the model training organization that you know you hire accountants and you pay them on a per dollar basis to to develop the training knowledge of how an accountant works. The eventual idea is that you can take an accounting trained agent that has the business logic to then look at the process and then figure out how to automate it. But but that's a ways out. So Internally, we operate these pods as well, where like we've found that the business logic, you need to combine the business logic, whoever's actually operating, and you need to have the technical know-how. So our own forward deployed engineers sit next to those people. But we also find that a third body is needed, at least briefly, and it's a it's a process engineer. you know, you and I were doing the rounds about Uber in particular, who dropped a press release about all the neat stuff they're doing with pods. And their attitude is that the engineer plays the process role because engineers tend to be process oriented, which okay, you know, you know, I buy that. But I find, particularly in mid-market and smaller organizations, that the process expertise is is actually missing. So We tend to take two people from our side, somebody who's who's technical, somebody who is process oriented, and then plant them next to the person who has the challenge and lean in to very rapidly going through the discovery, the solution identification, rapidly getting an MVP, getting the feedback loop in place, and being able to launch in just two or three weeks.

Matt: Well, I think that's so I agree with you. With Uber, the example of Uber, they had this huge it's a great announcement. Everyone should check it out, but they've had a lot of success deploying these their top engineers as a pod into these other departments to sit over the shoulders of those folks and help automate those workflows. The difference is those engineers have been at Uber for a long time. They're really good. They have all the contextual knowledge about how the organization works. They know all the lingo and

Kevin Williams: Mm-hmm.

Matt: Probably that's maybe why they don't need that process engineer. Cause for us, when we're going into a business, you know, we don't have those like five years of contact. So we need that process engineer who can think more what questions asked. Like they understand how departments work, they understand different stakeholders, they understand they've seen so many different processes. They may not be the person coding it, and that's fine, but they need that different kind of knowledge to be able to bridge the gap, like you're saying. I think internally, companies that have big engineering teams that are able to kind of take their top engineers and go on a little task force to different departments may not be needed, but for folks that are bringing in outsiders, I think you're spot on.

Kevin Williams: Yeah, I think that's a really good point. I hadn't thought about that, that there is just so much institutional knowledge tied around the existing engineering. They already get it. They they get how the organization works and how it thinks. So that's why they don't necessarily need that third body in in place. But most organizations aren't that sophisticated, even if they do have a dev function, and most of them are pretty lean. So they're they don't exactly have a bunch of people that they can cleave off to do this. So when they're thinking about how they make this happen, they they they end up at a bit of a loss. But these pods move really fast. Like that's the other part to keep in mind is that it's about getting to that that minimum viable solution as fast as you can, such that you can iterate on it while you have the attention span to solve it. And Keep an ROI focus such that you you you know if you're hitting the objectives that you're supposed to be hitting. And when you get there, yeah, it might require some polishing over time. Then you're gonna quickly pivot to the next one and the next one and the next one. I wouldn't be surprised if you start seeing like these embedded teams getting more vertical specific. So you you're not just a stormtrooper troubleshooter running around the organization, but you end up developing more expertise in like the marketing stack. And now you are the person that the marketing stack goes to to solve the marketing stack problems. It only makes sense. But you're not gonna be able to hire these people. So, you know, it's good for us.

Matt: Yeah, yeah, yeah. Well, I think I think finance specifically, because of the things we talked about before, they're so sensitive with where's the decimal place? Is it a one or a zero? Or you add an extra zero? Like those things I think are really sensitive and it's helpful to have someone who's just been doing those financial workflow automations. So yeah, I I can see it getting segregated like that. But in general, you made a joke that you can't hire for this. There is no one to hire. Like everyone's figuring this out. Like all this technology is new. The labor market doesn't exist. And people I think are, you know, like, I'll just go like hire this guy. And that's that's that's not what it is. That guy might be talented and capable of doing it, but like this this forward deployed engineer concept is just I mean, it's been around for a long time with Plant Palantir, but there's not a rich labor market of talented individuals. And so

Kevin Williams: Mm.

Matt: You know, even in our own businesses, we see that challenge of, you know, trying as we are getting more and more clients and building and scaling, like, you know, you have to work hard to find those people that are capable of doing this and then have the internal training to make sure they're doing it the right way, right?

Kevin Williams: Yeah. Yeah, yeah. absolutely. But and and noting that, okay, we started this with the conversation that, Anthropic and OpenAI are doing this. It makes all kinds of business sense, et cetera. But those services when you put McKinsey in front of it are not generally going to be for mere mortals. Like those are going to be, you know, they're they're putting a lot of money into it. They are going to abs The under my understanding is they they they are letting the Baines and the McKinseys of the world sort of handle the the cream at the very top in the Fortune five hundred type. And then thereafter the next like, you know, Fortune five thousand or whatever it is. But those are all really big companies. I don't know where the demarcation right now is at Fortune five five thousand, but it's gotta be in the billions somewhere. So

Matt: Mm.

Kevin Williams: There's this giant swath of mid-market and SMB that has absolutely no way of dealing with this, couldn't afford to go to McKinsey or Accenture anyway to do this, and is gonna be grappling with with trying to find their own solutions for forward deployment.

Matt: Definitely, definitely. it's great opportunity for us. I think business owners are like know that this is like a valuable, but they're lost on like how to think about it. So definitely.

Kevin Williams: Yeah. Yeah, we we do a lot of trying to teach people how to fish, as it were. I mean, our I my our businesses are are slightly different in in in that way, not you know, just a different flavor in that we do a lot of enablement type work and trying to build a lot of those capacities. but even so, we're finding that even with really sharp people that are at the keyboard and doing the right things, they really do need need support that pushes them into that next layer of discipline. And if they don't do it, you you can really create a mess later.

Matt: Yeah, rat's nest. And then then you get called as the vibe code specialist, clean cleanup specialist.

Kevin Williams: Yeah. I like where you were so you you framed it as a citizen developer. I've been I've been struggling. I feel like vibe coding has developed sort of a pejorative feel to it. like, you're a vibe coder. That means you're not doing anything good. Well, yeah, that's that's not really it. We're not like really most of this conversation has been about the facts that it's not about vibes. It's not about just like going with it and and building, it's about actually having some rigor in your approach such that Yeah. So maybe maybe where the this term might just start going away.

Matt: Yeah, we'll see. I don't know. I think I think citizen developer came because it was like democratization of access to the digital world. The digital world was always blocked off by people who had to spend years and years n understanding this foreign language called code. And now anyone can code, right? So yeah, I like that term citizen developer too.

Kevin Williams: So I guess, you know, I'm gonna move on to another topic because we we actually put out a resource this week that I'm pretty proud of for a couple of different reasons. one, we get asked a lot by companies that are just getting started how they actually need to be levering the teams and collaborative functions of the chat platforms themselves. A lot of this conversation has been sort of Post chat bot, but there's a ton of activity and interest in organizations that are, you know, they have maybe a CLOD subscription or a bunch of people have Claude subscriptions or a bunch of people have GPT subscriptions. And they're trying to figure out how they systematize that into the next generation where it becomes more of a true agentic harness. That allowed, and we've spent a lot of time in the past talking about this and the ability to connect a whole bunch of things into a central chat experience such that you have chiefs of staff and you can access your CRM and you can do all of these things. Well, just like a year ago, if you were to do a feature comparison between OpenAI and Anthropic, it would have been pretty straightforward. Like, okay, you can't do images in in anthropic, you can in GPT. you know, co-work was first to the table as far as like agentic work. So if you wanted to do that, you go that way. And now the questions have become much more sophisticated. And we took a really close look in the last couple of weeks at what this landscape looks like from a a a sort of GPT teams versus anthropic teams perspective. And the remote results really surprised me. There are about 85 different dynamics at this point that most people aren't even aware exist. So I I built a a page at resources.ascendlabs.ai that goes into a a lot of detail in this respect. But know that that that there's a lot more detail in this than there was before. And if you're trying to figure out where you're gonna place your bet, you need to be pretty aware of what the strengths and the limitations of each platform actually is. yeah. So yeah, go ahead.

Matt: Well, I was just gonna ask so for the audience, like what is the price points of some of these plans? You know, are we talking hundreds of thousands of dollars or tens of thousands?

Kevin Williams: So So is there's difference between Teams and Enterprise. So once you get over about a hundred users, you're talking enterprise and it's all over the map as far as what they're charging on a per user basis. It still feels like it's being negotiated on a per account. Really, what we're comparing here is the team level. So if you have I mean, honestly, if you have over five users on either of these platforms, you probably need to be on the teams level of the account. Because it enables account level control, account level visibility as far as use. It allows you to set access so people can't just like willy-nilly connect to random things and create data risk in your organization. technically, you only need two users in GPT to do that and five in anthropic. But at that point, on an annualized basis, you're paying about $20 a license, $20 to $25 a license per. for just that team's access. But what you do from there matters. Like does your organization need granular control over security IP controls, as in like IP addresses? you may need to upgrade to enterprise to have a lot of these controls. does your team really value collaboration within the platform? Like GPT, to be honest. is really great as far as in-stream collaboration. If you want to share chats, you want to share projects, you want to share skills, like it's all just baked into GPT to allow you to do that. But it doesn't have a great Microsoft integration yet. Like that will probably happen at some point. on the anthropic side, it has all kinds of neat like coming features like their tag feature is is is really interesting that they launched in the last couple of weeks. Yeah, so it's it's challenging. And guess what? There is going to be a high switching cost once you make this commitment. And it is starting to work its way up to the CFO's office in a way that that conversation needs to be had intelligently, as opposed to sort of, well, we got about 15 people who are using GPT and a bunch who are using Claude. And like we need to go in like one direction or the other. Because once you commit, A lot of your workflows, a lot of your training materials, your onboarding, all of this stuff is going to get locked into one of these platforms, which of course is what they're betting on, right? That that lock-in, that's why they're doing the forward deployed engineers. Yep. And they have, you know, they have the it's it's going to be a turf war basically for who's going to choose which. I'll say you probably couldn't go wrong with either.

Matt: That's why they're doing the forward deployed engineers. They want to make sure you're using their platform.

Kevin Williams: I I I am anthropic fanboy, I probably always will be, but it has limitations, it costs more. it can't do images, like there's a bunch of stuff that that are very frustrating there. So I don't want people to take away that, well, I have to switch over to anthropic. No. GPT is great, but you need to understand what makes it great and what the ramifications of the things that aren't so great mean to your organization, not just today. But over the next six months, 12 months, five years, whatever it is, because I don't even want to think about what it would take to to to migrate off. Like, yeah, they offer you can download your your chats and things like that, sort of, but it is not low friction to do this if you have twenty, fifty, a hundred employees.

Matt: So who in the organization is this document best suited for? Is it CEO, CTO, tech leadership team? Who's like use what in your experience like

Kevin Williams: I mean all of the above. It's it's who whoever's making the decision, right? of why are we doing this? And know, I I I I you know, tech leadership I think struggles a little bit with with this because the whole thing is so fast and loose as far as what having this agentic harness in the middle of your organization actually means. But you know, that really the could the constituency should be whoever the user advocate is, whether you have like a head of innovation or you have a CAIO type who is leading the charge, obviously they're the ones who are going to bring it to the table. And then having that CFO interface so that everybody understands what the costs are that are involved and what you're really signing up for. you know, there needs to be buy-in there. And then honestly, there needs to be some stakeholder buy-in. Like what we find, which is can be really frustrating for an organization, is they have that we have three people who are complete clawed phonetics and they would rather chew off their left foot than be forced to use GPT. they've have three years of enculturation, they have all of their workflows, they have all of these different pieces in place. And you need to decide what you're losing there. And in fairness, some exceptions can be made, right? Like like it's okay to have a smattering of GPT licenses or Claude licenses in an organization that's otherwise biased for the other other direction. You're just going to lose some of that centralized control and you need to be open eyes about what that means. But it is a little bit wild. I mean, I I think about my own chatbots. I have, you know, 30,000 chats in in both Claude and GPT at this point. There's a lot of personal IP there. That I'd be loath to lose. And starting over fresh with like a baby LLM. my goodness, that'd be daunting.

Matt: So when you're working with a client like 'cause we've talked in the past and even today a little bit about like having multiple models, check them check each other's work. Or maybe having to run things locally and how that can save on token costs because token costs can balloon like crazy. When you're going to into an engagement, are you thinking through like, you know, with them which model, single model they should adopt from a workflow perspective? So what you know, what we're talking about is a team plan and how teams can use these models. Are you separating that from like, okay, we're gonna go in and we're gonna create some agents, automate some workflows, and we're gonna use the same model that your teams use or we have some more freedom to kind of pick multiple models, run things locally, save on token costs where we can. Are these two separate conversations or does the the team plan influence what you're gonna use to build your agents with?

Kevin Williams: they tend to be separate as far as the the the conversation, but the overlap happens when, say, that citizen developer person on your staff is shoulders deep in Claude and like couldn't possibly imagine doing the switch. So even so, that person maintaining a Claude Max plan individually often makes more sense than forcing them into a a mold that doesn't really. Really, we're talking about not rank and file, but sort of independent contributor layers in the organization and standardizing that. it's really, really helpful to have a common language of you know, hey, w welcome to the marketing team. These are the skills that we use, these are the GPTs, these are the shared projects, et cetera, such that they're well enculturated. that has a ton of value, but the edge cases are legion basically of of.

Matt: Yeah.

Kevin Williams: having good reasons to to do it on a one but one off.

Matt: Yeah, so folks listening are thinking through like these team plans, like internally at our own company, we have the same thing. Like you don't want your HR instant your HR team sharing the same information with the rest of the organization. Please draft a separation letter, please draft this contract or whatever it is. Like you don't want that information kind of getting across. So the team aspect of these plans is really helpful. It helps build those those bridges where you need bridges, but also those silos where certain teams need access to only certain things. and so that's how we use it internally. But when we go to build an agent or something that's gonna cross multiple parts of the organization, that's where we have more freedom. We don't have to be narrowed in on a particular plan with one model. and we can kind of pick what's best for that use case and what's low cost and gonna get the same amount of quality out.

Kevin Williams: I I suppose sort of closing out here, the the the the takeaway is you do have to make a decision about how you're going to approach this in your organization. And from a business risk and just opportunity perspective, it is almost certainly the right decision to upgrade to at least a teams plan before too many more people get. get like sidetracked into their own environments. And yes, it it it it is a scary number if you have twenty or fifty people, but it also gives you that visibility to see what people are not necessarily on a granular level to see what they're doing, but how often they're using it. Who are your leaders in the organization who are really leaning into it? Right now, I promise you, if if everybody's just running their own their own account and they're just expensing it, that's Pretty much the most common approach that I see. You don't have any of that visibility. You don't have any of that control. Like you need that in the organization. So if you're in that situation where that's what you're doing, I would, I would highly recommend that you take a look at that that resources document and get a sense of which direction you want to go. there's it's also kind of a cool document because It's it's a living web page in that it is constantly going to be out and looking for these sort of granular changes. and not completely self-updating, but it's built on feedback loops within my Slack channels such that I will immediately see when there's a feature change and can basically approve whether or not that gets updated. So as opposed to being we're gonna update this month once a month and you're gonna send an intern off to do a bunch of research. Like the document is basically self healing. and that as far as a a web approach gets my like like marketing mojo definitely way up because it's it's really, really powerful. It means anytime anybody visits that that document, it's gonna be as up to date as it possibly can be.

Matt: I love that. That is a cool tip for anyone out there that's building similar resources for your own audience. That like have that refresh. That's amazing. One last little tip that if we recognized this two years ago, that this was going to be a problem because we have a team of over 200 people. We weren't going to put pay for a 200 person plan. These plans didn't even exist at that moment, right? and the same problem, like there was no team capability, everything's in silos. So two years ago, we actually built our own little like wrapper on top of all these models. We had Groc, Gemini. Open AI and anthropic. and so we actually put the entire company into this app that we built to keep all the teams separated, keep the projects separated where need where they need to be separated, then created the collaboration function between different departments. And the best part is is you don't pay per seat and you pay per token with whatever model is the best for the task you're working on. So if it's generating audio or visual or or text or whatever it is or code. Like you just plug in the model and you just pay the token cost, which is significantly lower than paying well, depending on how much you use it, than paying per seat for everybody. So yeah, we ho we recognized this two years ago and and used this instead so we can kind of avoid these expensive team plans. Little workaround for everybody.

Kevin Williams: I think that's one we wanted to explore a little different. It is getting my spidey senses going as far as there's some risk in there. as far as you know, like cost expense, which is the other other part that you know, CFOs get twitchy about. But Matt, this is great. you know, this again, there's so much exciting stuff happening around us. companies are really starting to lean into it. This is this is like two thousand twenty six is the year of activation, like, you know.

Matt: Yeah.

Kevin Williams: Twenty four was just about curiosity, twenty five, maybe some pilots. Now the best clients are super serious about making this real.

Matt: Yeah, people can sense it. Non technical people can see it's real and it's all b this agentic AI that emerged is creating enormous value. So exciting time to be in the biz. Yep, good seeing you, Kevin.

Kevin Williams: All right, man. Great seeing ya.