Taylor MacDonald: Coming in crisp and clear today. All right. Hey Sean, how are you doing?

Sean C Davis: Risp. I'm doing all right. How are you, Taylor?

Taylor MacDonald: I'm doing okay. so it's we're recording this on Thursday, July 9th. my name's Taylor McDonald. With me as always is Sean C. Davis, and this is Blood, Sweat and Tokens, the podcast that we're using to talk about agentic workflows, upskill more junior level engineers in this age of AI, build real products, primarily centered around things like natural language search or or natural language technology and and and other fun. interrelated technologies and stuff like that. So this is episode number eleven, which is nuts. We finally, or I finally made an investment in a microphone and a camera. So hopefully this is going to get better. Ironically, our last episode where we did in person, we had a few technical challenges. And I think we even started out the episode saying, our audio quality is going to be amazing. And I actually think it it suffered pretty significantly. So

Sean C Davis: Just a few.

Taylor MacDonald: apologies for that if you suffered through that one. I that that was hard for me to listen to, but I to Sean's point, there's good content and we're probably holding ourselves to a higher standard than than maybe we should. that said, hopefully we're gonna work all the technical kinks out and and we'll have a a real great production quality moving forward. So, Sean, today we're gonna talk a little bit in terms of the the three segments that we talked about last time. We'll do a little bit of a demo and then we have a discussion topic and then Kind of something fun you're working on towards the end of the episode. So as far as the demo goes, do you got anything to share with us today? What have you been working on over the last week?

Sean C Davis: I do, yes, I know it's been a it's been a few weeks coming and I am frozen. Bring me back. Wait, where'd I go?

Taylor MacDonald: Your video has stopped.

Sean C Davis: Now it's totally gone. Hey, there we are. Okay. All right. So yeah, we've been teasing this for a while, and I have and I have enough that we can dig into. So let's take a look and you know, hold me to some time here so we don't go we'll go too deep. All right, window.

Taylor MacDonald: He's back.

Sean C Davis: I'm not gonna spend a lot of time on this one, but I wanted to call out we talked last week, or maybe it's two it was two episodes ago. We talked about kinda the art of abstraction and where you you point that line. And Taylor, you mentioned the Ruby on Rails model and the whole, hey, I'm gonna give you the happy path, and then I and then and then you can opt into complexity. And this is a really, really horrible website that I I have not nitpicked in any way. But this was this was kind of fun to put together. We use this demo site, and I'm like, okay, let's follow that approach. There is this cool little like, you can change, you can change the theme of the thing. But what I wanted to call out here is the approach more than anything else, which is Here's this library that is going to ship with our suite of tools. And we have pre-baked components. But the concept with all of these components, and there will be much, much more, is that they're all based or they're all built on top of this set of primitives. And all of this will get way more fleshed out. But that idea being that there's this happy path where, hey, I want, you know, I have this button. And the button looks like this, or you know, maybe I've configured my theme so it looks like this. And okay, the button by default supports an icon on the left, but maybe I want two icons and the button component doesn't bring that. So we want it to be really easy for any agent or any human developer to to basically be like, okay, cool. I don't have to rebuild the entire button from scratch. I can just almost almost like You know, monkey patch it, so to speak. And then I have that component. So what I did was I basically went all the way back, stripped things down, and said, okay, we have this demo site, and we're gonna come back here to our friends at Lumen Industries and say, Okay, here's the demo site. Let's make it static. And then let's use that site. First we'll build out the site. So Step one was build rebuild it as a completely static site. Step two was turn those basically componentize it and then use the components to build out this very early version of the component library. So we have those pieces, but ultimately what someone using the application what what you know one of our users or clients would have is this would be their website. They would be consuming the components because the piece that I there's two pieces that I don't have wired up yet. Which will be our next two episodes, hopefully. one is the adjusting code on the fly, and I'll get into that later. And then two is the the actual an initial discovery and how sites are created from nothing. And I that's what I'm aiming to be able to to show in a week or two. But

Taylor MacDonald: Cool.

Sean C Davis: What we'll eventually have, and what we've shown in the past, is that on when you're when you're looking at your site, if you are authenticated through our system, then you'll be able to have an overlay right on this particular site. But what I wanted to do was say, let's let's build out our hub and all of the pieces that work together, and then we'll be able to extract that. and use s hopefully reuse some of those components. So I just started with this hub and here's this thing. I need to go all the way back to

Taylor MacDonald: Let's let's recap briefly what we're looking at here. So so we started the podcast by saying like CMSs are a pain in the ass. We think you should delete your CMS, but then what? You know, why are they they're a pain because it's you're tightly coupling your code directly to these fields. Then often in this day and age, I mean, unless you're on WordPress, which has its own separate suite of issues, if you're paying for a product, the vendor lock-in situation is significant.

Sean C Davis: Sure, yeah, yeah, yeah.

Taylor MacDonald: There's a lot of volatility on pricing. There's a lot of volatility on entitlements that exist for users of tools like, you know, contentful insanity and so forth. And so the idea was what if we just built our own? But maybe we thought about it a little bit differently. We're using AI for everything these days. What if you could just speak at it? You know, the minority report, homegrown open source CMS, if you will. So Sean built a couple of demos, right? The first was kind of a

Sean C Davis: Mm-hmm.

Taylor MacDonald: triad of, if you will, of different components that work together to make this work. And I forget which episode that was in. It was probably two, three, four, something like that. Well, that posed a few architectural challenges that we talked about in a subsequent episode. And I think where we're at now, Sean, is you're kind of thinking more about that implementation piece. You know, what does it actually look like if I was going to use this product and I was going to bolt my website into this framework? What are the steps necessary to get up and running and have it functional and what is the necessary impact to developer experience? Is that fair?

Sean C Davis: Yes, yes, exactly. And that's the step that I'm really eager to show, but you kind of need all these other pieces in place first. So you you know, where you can be like, yeah, I want to adopt this system. Here's my existing site. What does that initial flow look like? So this is more like after you've gone through that onboarding flow.

Taylor MacDonald: Okay, got it. All right, please continue.

Sean C Davis: All right, so this and one thing we talked about last episode was part of the development flow that I follow. And I did a lot of going back and forth here. and so it's like let's I I went through some design iterations and then was basically like, okay, here's the design, Claude. Now there's a lot of functionality we haven't talked about. Tell me what you don't understand about it and work through it. And there's more working than I anticipated. Like I thought this was just a placeholder and it doesn't look great, but it's kind of cool that it is, it is, it's pulling the actual site in. So what a user can do, and we've got some technical bits here and there, but we can say something like, What do we say,

Taylor MacDonald: Can you remove some of the items from the nav, for example? I see how your nav is kind of like running off the left and right edge. Is that something that the system could could touch or is it strictly tied to, you know, content blocks and stuff like that?

Sean C Davis: Let's let's try it. Let's try it. Let's say change the founder in the nav to founder. And so then we start it's go ahead, go ahead.

Taylor MacDonald: Okay, so if you're if you're listening, I was just gonna say if you're if you're listening to this episode, it kind of looks like almost like a lovable interface interface kinda or V Zero where you've got sort of this chat sidebar hanging off the left side of the screen, and then the website is sort of encapsulated in like an iframe style interface in the middle where you it looks like you could toggle between desktop and mobile screen sizes. let's see what happened.

Sean C Davis: Yeah, yep. And so this is one where it's there's something funky going on here. Just as a like if you I'll try another one here and then show you the difference here. change a message to all who enter to Everyone is welcome. I don't know. Something goofy. And we should get two here. I don't know if it will revise this or not. Let's see what happens. Okay. So we're gonna add this to our drafts. And immediately, look, it did work. Immediately we get that. And then everyone is welcome. Let's try this.

Taylor MacDonald: yeah. It did. that's interesting.

Sean C Davis: And where we go. And then we've got this. Yeah, go ahead.

Taylor MacDonald: It worked. Huh. Now wait a minute. So ooh. that's cool. Hold on. Sorry, I totally interrupted you, but sorry, you're creating in every change represents a draft. Meaning there's content objects in the database that make up all the requisite parts of this page. You've made changes to those. How are you managing the changes in the on the data layer? Like are is this a new

Sean C Davis: Yes.

Taylor MacDonald: record for every change and then each record is kind of an a a sort of atomic picture of the content in that moment. Are you using NoSQL technology to t take me down the rabbit hole?

Sean C Davis: Yes, it's it's kind of like a it's goofy because it's almost like a a no sequel approach in a sequel world, so to speak.

Taylor MacDonald: so you're doing like are you doing like JSON blobs or like a big document that you stuff into a table? Like ha tell

Sean C Davis: It's that every here, let's let's let's look. We're gonna we're gonna go deep for a moment. don't go too deep. Okay, great.

Taylor MacDonald: Don't go too deep. We're we're already at like ten minutes in here.

Sean C Davis: Here's our production database. Let's make a little bit bigger. All right. So we've got, so we've got pages. Are we gonna be able to, so here's our home page. we got two of them. block instances. Let me see if I can find. Yep, okay. So this is holding the content for that whole that whole section. You can see everyone is welcome right here. And then we've got the draft status. and then we're tying it to a particular environment ID. So we're not

Taylor MacDonald: Uh-huh.

Sean C Davis: That's not super obvious right now through the UI. We need to do a bit of that. But yeah, that idea is that you can create multiple environments that you can work in, but that every change in that experience is automic within that environment, yes.

Taylor MacDonald: Within that environment though. So it's almost like a nesting doll kind of strategy. Like you've got the outer layer,

Sean C Davis: yeah, I mean there's an inheritance. so you're you've got your everything is built on what exists in production. And then within a particular environment, you can change one particular object. And so when the page loads in some spit right now, I right now it's either o preview is off or it's on. But eventually you can imagine multiple environments. And so when you're within, for now we'll just say preview mode. We're basically looking at the way the page is getting queried, because here we know we're in preview mode up here. and this is something that every site will inherit, is that we say first, you know, basically give me everything for this that is on this particular page. Give me all the content for this particular page. but if there is something in a draft state or previous in part of attached to that preview environment, it's going to override what is in production. That's what we're going to show in preview mode. And we're going to ignore all that stuff when we're in prod.

Taylor MacDonald: Got it. So those block instances represent pictures of every content, well, every model or every sorry, component that lives on this page. And you create a different instance of that based on every adjustment that you've made throughout the course of your editing to that particular co component's content. Is that think about that right?

Sean C Davis: Yeah, and so this is what this this viewer was hopping over here to take a look at. And this is what's coming next, which is we're gonna actually be able to write some code automatically through this system. But we can see okay, we've got three different drafts here. And you can you can undo any one of them. but if I go back here

Taylor MacDonald: Yeah. That's awesome. So we're looking at basically Sean built a Kanban board that tracks every one of these changes. That's extremely cool way to solve it. Okay, so you can publish all the changes all at once. Now, does that move it on the Kanban board, you know, from left to right when you publish this? Or how like can you manage state? I'm assuming you can drag and drop them in between the different swim lanes here.

Sean C Davis: Yes, so This is this flow is going to be more about monitoring. For now, it's gonna be more about monitoring the things that need code changes because those have to go through several more steps. The content is more, it's either in draft mode or it's in publish mode. So for here, with the nav change, we could undo it and then it would drop off the board, or we could merge it. And so over here, let's say we're gonna publish, but we wanna change the nav.

Taylor MacDonald: Sure. Is this when you say merge okay. Okay.

Sean C Davis: I meant I meant publish. I meant publish. So let's say okay, we're not going to change that. Everyone is welcome. We don't want to publish that. We're just gonna publish, we're gonna change founder to founder in the nav, the founder to founder in the nav, and we're gonna change the work is sacred here, which is left over from a previous demo.

Taylor MacDonald: This is interesting, man. I like this. I I like how you visualized everything that's taking place. One thing I've really struggled with with Claude, you know, it moves so fast. Sometimes I miss a thing. You know, I try to read all the input and try to understand what it's working on. Whoops, but it would be really handy if I had like a list. Of everything, you know, visually, like I did this, I did this, I did that. And I guess it kind of gives you a tack, like often the plan will be like a task list, which is a very high overview, but it's that's a really nice UI, and it's interesting that you use the Kanban style interface to make that work. I don't know, I it's it's pretty cool. I'm impressed.

Sean C Davis: Yeah, yeah, thanks. So I've yeah, I a lot more to do there. But yeah, I mean in that particular case what what I had to work through originally with the design was I was explaining these two flows, right? And it was like it's either content and it's a really fast change, or it's code and it's it needs to change, but it needs to get checked, it needs to fit within the system. But a user doesn't care, right? Like a a user's like, I have a change. I want to see the change. And so this is trying to approach it in a way where we're like, Okay, these flows are different, but we're we want to speak in your language. So we're saying, yeah, it's a content change. Cool, we're gonna do that super quick. Or hey, hold up. So it's a design change. It's gonna take a minute, but we're gonna visualize everything together so you can see, you know, what's available to to publish to production. So my my goal for next week, next week's demo would be that we can see those things merged in action. I can do both those things and be like add a new component or add a new section or whatever.

Taylor MacDonald: That's awesome, man. That's super cool. I love that. That's you've made a lot of progress. My goodness. Are you actually you you still have a full time job, don't you?

Sean C Davis: Yeehaw. yeah, don't t don't tell anybody.

Taylor MacDonald: Okay, they're not listening anyway. all right. well let's move on to the next set. That's great. Appreciate the demo, Sean. I can't wait to see where you next week you said we were gonna be adjusting code on the fly, so I'm excited to see.

Sean C Davis: Woo, yeah. Pressure's on.

Taylor MacDonald: Yeah, right. okay, so let's I wanted to revisit a topic that came up on our show last week, which is this idea of like being addicted to AI. and and specifically corporations getting addicted to AI and how that necessarily impacts, you know, the way we're using it, the way we're thinking about it. You know, one of the things that we talked about a little bit had to do with like salary being coupled to your token consumption or or sorry, you having a token budget as a as another angle on compensation for your employees, which is an interesting thought, fairly provocative. We spent some time talking about how that was very similar to like the coal companies in Appalachia that would, you know, house you, feed you, clothe you, and give you non-American dollars sometimes in payday advances that you could only spend at the company store. And how that kind of ushers in a very uncomfortable reality. If you start thinking about my productivity and my, you know, confidence is all tied to one company and and and its budget that it's giving me, you know. So if we take a big step back, I want to think about this in like three three phases, you know. So companies have created this artificial urgency around adopting this, right? I'm to blame too. Like anybody in a C level position, any business owner is gonna be jumping at the opportunity to increase productivity and reduce costs. That's the game. And so that has resulted in things like prove you're an AI champion messaging in certain business contexts. You know, we talked a little bit last time about how Meta has a I understand they have a leaderboard in their office that shows the employees who are using the most amount of tokens in their day-to-day, i.e. meaning the token consumption and the volume of tokens that you're consuming is a direct correlation to your productivity and your worth at a company, which seems gross. and then lastly, companies have this fear of being left behind by their competitors, right? So they have to maintain this this urgency and and they're pushing people to adopt it quicker and faster. And a lot of the security protocols and the risk assessments and things are going straight out the window. Because of the massive productivity gains they're seeing. So then phase two, the employees got hooked, right? So you're now massively productive. You can open Cloud Code. I mean, look at how much work you've accomplished. How long, objectively, do you think it would take you to have built that application 10 years ago? I mean, we were building them 10 years ago, but in this is obviously way more sophisticated, mature technology. But just think about that. You're building a a Maybe even take the natural language piece out and you're just building like a C what was the wasn't there a CMS back in the day called Tina, which operated similar? Like, and that I think was writing to flat files. But let's say that I don't know, how long would you ballpark this level of polish and this level of feature set, you know, as closely as you could track to 10 years ago because it's not like, you know, the technology is not one-to-one.

Sean C Davis: Yeah, yeah, yeah, yeah, yeah. Yeah, I've been I've been thinking about that a lot lately. And I I I rolled my eyes for many months at the the idea of the 10x developer. And I think in a lot of cases it is not true. I've but I've been experiencing it a bit lately. And so if you say this took me I I I mean I I just think it's on that order of magnitude. I wasn't paying super close attention and and it's always like it's a thing in between many other things, but let's just say it laddered up to, you know, ts the eight eight to ten hours or something like that. Yeah, then it's probably like it probably would have taken eighty to a hundred hours to to build that by hand.

Taylor MacDonald: Yeah. If the if that, I mean, you'd be sub I mean, we know this, but we have this running joke at Ample where every employee or every engineer, like we don't trust their estimates. It's unbelievable. Wow as a yeah, as humans, we always overestimate our abilities and underestimate the amount of time it's gonna take to do something. And it's I mean, it is consistent. It's as consistent as gravity, man.

Sean C Davis: Yeah. What do you double it? Yep, always.

Taylor MacDonald: Like it impacts everybody. It's so funny. So anyway, so you give developers and and development teams gasoline to rocket fuel. We'll call it rocket fuel, right? Now we're super productive. We get instant gratification from the immediate response from these AI agents. We're reducing our cognitive load. you know There's a status signal that happens too amongst engineers where it's like, you know, I'm I'm killing it. I'm all AI all I've been doing. AI. I was at a a meetup a couple of weeks ago and somebody was like, I was doing this, you know, back in two thousand eight before anybody even knew machine learning was on the radar. And it's like, okay, man, like whatever, that's fine. At at the end of the day, that's a real problem, right? Like we're now

Sean C Davis: Yeah.

Taylor MacDonald: Now it's we're feeling good. We're super productive. We have all these ideas and we can do whatever we want. And then reality hits the budget specifically. So finance departments are starting to panic because of the unpredictable costs of this technology. companies are starting to realize that they can't even afford their own, you know, success metric on this because now they're beholden to this vendor lock-in. You know, if you're using clawed code to stand on the shoulders of giants.

Sean C Davis: Mm-hmm.

Taylor MacDonald: They close up one model, open up a new one on the, you know, frequently so that can disrupt all your workflows. Secondarily, like the cost just keeps going up. The the maturity of the technology keeps going up. So that's probably an appropriate correlation of, you know, cost to functionality. But that's a difficult thing to predict. I read recently that Uber has blown through their entire software budget. you know, in terms of AI tokens or whatever, by like April or March or May of this year. So we'll just say Q two of this year. Like the entire annual budget gone, liquidated. Now what? Right? And so I think CFOs are starting to freak out, man. And so then you start worrying about talent. You s you know, you're gonna start pulling back on this stuff as of the employee. What kind of anxiety problems are we instigating?

Sean C Davis: Mm-hmm. Yeah. Yeah, wild.

Taylor MacDonald: So let me ask you this. W w what happens when employees become dependent upon AI for tests that they used to do themselves? What it what happens to their skills and their confidence?

Sean C Davis: Yeah, so okay, it's fascinating. Cause I've been my my brain wants to answer this in like five different ways because I've been I've been thinking about this from so many different angles this week. And I think the and part of this is coming from I feel like I have been experiencing that that addiction pretty pretty strongly or acutely over the last couple of weeks where it's like You know, I think so many engineers are falling and and products and companies are falling into this trap. And so the byproduct of that is yes, things are moving fast, but the quality is going down. And then it has this reverse effect where I wanna build more because it's fun, but also because I have control over the quality. Like I don't know what I don't know what anthropic is gonna ship in Claude Desktop tomorrow. Do I really wanna put all my eggs in that basket and then have them, you know, pull the rug in something that I felt like was really important to me. So I'm building more, but then that makes me obsessive about what I'm building more. Anyways, I'm off on a tangent. I think that the the the the one thing I've really been trying to hone in on.

Taylor MacDonald: No. You're good.

Sean C Davis: is what it's a the the question I've asked about many different things in life over the course of life is like when is it enough? Like what is the where's the or maybe better, where's the balance? Because you're totally right. Like you can you can absolutely burn through thousand dollars or more of tokens every day. So how do we find w when is it enough? When is it impactful? When are you wasting money? When are you wasting resources? And it it it kind of sounds like do you think that's the question we're trying to answer and we just don't really have a good way to do that right now?

Taylor MacDonald: I don't know. I it's weird because corporations are now viewing w if a token represents either a form of currency or a form of like a value statement, like this represents productivity, then it would make sense to be like, well, I want you to be as productive as possible. So therefore you need to use more tokens. And we're starting to evaluate people based on the amount of tokens they're using, right, with the expectation that we're being more efficient.

Sean C Davis: Mm-hmm.

Taylor MacDonald: I think the problem is there's no guarantee that you're being efficient. You're just burning tokens because, you know, the leaderboard puts me at the top if I, you know, have a couple of Ralph Wiggum loops, you know, running twenty four hours a day. But you know, which is not to suggest that any of these companies are using this technology. Well, may I'm sure s a lot of them are, but like yellowing

Sean C Davis: Right. Right.

Taylor MacDonald: features to prod or stuff like that. I mean, I think the reputable companies are being mindful about the human in the loop thing. I just think it's devastating for human agency. Well, maybe not. Let me ask you this. If you have this apparatus that can answer questions and tell you anything, one of the things that has been really don't know, addictive for me is that I have a lot of ideas and I have a lot of things that I want to learn. I just I I always want to like tell me how

Sean C Davis: Mm-hmm.

Taylor MacDonald: How do you do this? How do you do that? How does air conditioning work? How does, you know, well, like what happens inside a fuel pump in a car? Like weird stuff goes through my head. And so it's given me this like forum to throw those questions out there and get honest, immediate responses that are fairly objectively accurate. and so I've really enjoyed that. In a way, that has given me more agency. However, when you think about this in the context of the office.

Sean C Davis: Yeah. Mm-hmm.

Taylor MacDonald: What happens to human agencies when companies are basically incentivizing constant delegation to AI?

Sean C Davis: Yes, yes. I think I think that piece is a huge problem. And I know there's a lot of arguments about factories AI factories on one side and humans in the loop on the other side. And again, I I constantly come back to there is a balance. There are things that you could farm out to factories because there are because they are objective. But the products that are going to win, the companies that are going to win, need human brains. And I don't think that's going to go away. You just have to find that right balance. And I had this great conversation yesterday. the something that w I I've started to see, and and many others have started to see as well, is so to To try to make that productivity visible, y you know, people are like, okay, cool, I've got access to the data and I've got access to this model. So I'm gonna be like, give me the next three things we should build, and then I'm gonna look like a genius. But I think what a lot of people are doing is like they do that, right? And then they're like, Cool, here's this 2000-word document that has all of my thinking, except

Taylor MacDonald: Mm-hmm.

Sean C Davis: That's not they didn't actually put their brain and and their agency to work in that process. And so that's one thing that we've been we've been talking about a bit at work is is like you've got this company, and a company is a group of people that are trying to move in the same direction to build something that people will find useful. If you want somebody to read something, then you need to apply some level of effort that is greater than it takes to read that thing. And it's more than just saying, hey, this is, you know, make it shorter. It's more like, are these actually your thoughts and you use the tool to codify them? Or are you trying to use something to do your work? You know, are you just trying to have this thing that's like pushing the button like, yes, I'm online.

Taylor MacDonald: Yeah. We've had a bunch of conversations at Ample about the volume. Like it's really great for doing structured tasks or things with like, you know, get the creative ideas going or whatever, but the volume of content that it can generate can be s more overwhelming than anything.

Sean C Davis: so how deeply they're using your like how how many tokens they're consuming and what they're using them for. Where's that balance?

Taylor MacDonald: okay, are we creating AI anxiety with this? I mean, I want my employees to be productive and I want them to have all the tools that they need to stay competitive. are we creating like a fear of being unproductive, I guess is what I'm asking. Without AI. Meaning i it's this self-reinforcing it's a it's almost like a I don't know, it's an encouragement to stay in your job because your productivity is now attached to those tokens, that budget, those workflows that don't belong to you. That seems weird.

Sean C Davis: Yeah, I I agree. I think it came up a couple episodes ago that I think it probably needs to come back to the impact that someone's having and and have to figure out how exactly to measure that. Cause I know it's something you and I were going through many years ago was he hey, that so the company makes money by billing at an hourly rate. That is, you know, something somewhere above what you pay employees all in. And so we need to make sure that we're being ethical and honest and that employees are covering that time. So then a lot of that work is you know, it's like, well we want the quality to be high, but they also have to spend this amount of time on it. And so i you know, I don't know, is this a diff different version of that and we need to shift a little bit more towards how do we find a better better way to measure impact when impact is usually just such a subjective thing.

Taylor MacDonald: Well, it's you know, companies get the write-off benefit, you know what I mean? Like there there's all sorts of they can also buy these things in bulk. They can barter and negotiate relationships and partnerships with the company the service providers. So it makes sense why they're footing the bill, but in a if you flip it, like if the employee was responsible for their tools, their productivity outputs, in a lot of ways, does that give them more

Sean C Davis: Yeah.

Taylor MacDonald: control and more you know flexibility in in their their ability to leave the job and and you know be more marketable for those services. That would necessarily require, you know, you would think that like so if Jensen wants a $500,000 employee to spend $250,000 in tokens, why not just give that employee $750,000 so that

Sean C Davis: Mmm, interesting.

Taylor MacDonald: with the expectation that they hit this productivity level. I think inherent in that, I mean, that makes sense for the employee, I would think, but it doesn't make sense for the business because the business needs to be able to track how the tokens are consumed. you know, maybe they're i they're correlating that employee's output or the number of tasks that move across the board and they're comparing that to the token consumption for that person. You know what I mean, they lose a ton a business would lose a ton of visibility, not to mention the more cynical interpretation that they're using this as a as a means to kind of lock you into your employment.

Sean C Davis: Yeah, there's the security aspect too, right?

Taylor MacDonald: That that's on paper that's not a bad that's not a bad thing. yeah, and the sec yes, you're right, the security is another big piece. So I don't know.

Sean C Davis: 'Cause I was gonna say, well it's not a it's that's not a need. It's yeah, it's a desire, it's a control thing. But I think you could probably argue that the security side of it is a need that hey, if you're going to give an LLM proprietary company information, then the company needs to control that account. That's not any different than email or or anything like that.

Taylor MacDonald: You're right. No, you're right. It's interesting. I I'm going through a risk It's my most substantial in a long time in the middle of all this AI stuff. And it's the first time that it's had significant sections spelled out. Like how do you use it? You know, what providers are you attached to? Like how do you ensure data privacy and security and all that stuff? And it is it is the wild west right now, man. I'll tell you. It is nuts.

Sean C Davis: Yes, yeah.

Taylor MacDonald: In fact, I'm actually kind of curious about this. We've got a lot of clients that are showing up with vibe coded apps that are simply not scalable. But they're like, Can you make this work? Like, can you make this production ready? and so the team is trying to figure out like, you know, how do we price this? How do we support it? What risks are inherent? Like what risks is the client taking on, you know. So it's kind of it's kind of wild that it's kind of wild that

Sean C Davis: Mm.

Taylor MacDonald: This is happening and and so we're looking at companies like Netlify to help us solve this problem. You know, if you have an enterprise Netlify account, you can use the gateway to like meter all the all the different providers and all the consumption. It can tie directly into some of your workflow tools like Claude Desktop, for example, you could just say build this report, Google now ship it. up to Netlify and it'll be available in a couple of minutes, which I think is is huge. I'm curious from your standpoint, is that how how do you guys deal with that? Because you've got employees that are vibe coding stuff all day too. How do you control that in the corporate environment?

Sean C Davis: Mm. Yes, that's a it's a huge question and it's a a part of our focus lately has been essentially that we need to solve that for ourselves so that we can help other companies and customers solve it for themselves as well. and and and I mean we we have some process in place. A lot of this is around It's it's around the security of data and how do we ensure that the right how do how do we ensure that the the the app itself is configured in a way that data is only visible to the right people. and that's you know within the organization and outside the organization. And then we kinda have an an added layer of complexity, right, where we're like Okay, so we've got we got our human resources team that is vibe coding stuff and they're building amazing tools and it's it's i inherently it's it's human resources. So like it needs to it needs to be private and confidential and at the same time anybody who has access to the production database can get that information. So how do we build some layer of of trust and

Taylor MacDonald: Yeah. That's so interesting.

Sean C Davis: Accountability there. So it's it's a fascinating world and I will say that the I think the fact that we are still working toward a solution is I I think it it tells me that yeah. It's a challenge, yes.

Taylor MacDonald: That's a testament to the challenge. Like it's not an easy thing to solve. well, that's refreshing and a little bit terrifying. do you guys have a do you do you have like a business clawed subscription that everybody's a member of? Or do you have a budget? How do how do you access the tools that you're using? I mean, are you paying for like the stuff you

Sean C Davis: A little bit.

Taylor MacDonald: did to build this demo, did you pay out of pocket for that or is that tied up in the corporate expense line?

Sean C Davis: Yes, I mean I have my own I have my own personal accounts for personal use, but then the the company we we it seems like our pattern is to step through a series. So it's like we have we have a budget so that if you want to experiment with something then you kind of go through you you still go through the security audit and all that and then you can kind of pull from this sparse budget to experiment. And then when it feels like a tool is going to be part of a workflow for one or a group of people, then you get you know you can get a virtual credit card that can only be spent with this one provider can be locked down with limits and things like that. And then when it becomes something that needs to get adjusted adopted more widely, that's when we look into SSO and enterprise level contracts. And so for us right now, we're kind of in this space where we've got to contract with anthropic. So everything that everyone is using Claude Code is using it through the company account. But then there are a few folks who are like, like me, who am like, well, I I need to pit different models against each other for my specific role. And so Codex comes through another layer of account because we don't have that with open AI, though I imagine we will at some point soon.

Taylor MacDonald: Spreads you're using open router.

Sean C Davis: I am using open router for a lot of I don't kinda like personal a little little personal projects. And and we're we're always talking about, you know, what's the what's the next step for AI gateway, what other models do we need to introduce there. And then I think we get into legal complexities and stuff like that.

Taylor MacDonald: Yeah, well that's really interesting. Thanks for indulging that. let's move on to the last part of our show here. We're getting close on time today. this is the experimentations. I don't know that I've been doing much coding in the last week or so. So what do you got going on? What what are you experimenting with right now, Sean?

Sean C Davis: So many things, so many things. okay, the one one thing I think was interesting because it was it it had been a lot of talk over the last week or so is Theo Brown put out a video about the skill he was using and I forget exactly what he called it, maybe called it orchestrator or something like that. And it's it's a it's so I took that idea Which I was talking to a coworker who had who has taken another approach with it. so we're working kind of similarly. But I think these these things are coming together. So that was an interesting idea, and I'll talk more about that in a second. And then at the same time, you know, we got this limited look into Fable without without spending that extra usage. So that'll change in by the time this episode comes out I think that'll Fable will be on extra usage. which means not part of the built in subscription.

Taylor MacDonald: You have to move to the API key, is that what you're referring to? Okay.

Sean C Davis: Yes, yeah, yeah. Or I think you c you can use the subscription, but it comes out of this extra pool of money. So

Taylor MacDonald: okay. Got it. Okay.

Sean C Davis: What I've what I've played around with a lot over the last couple of weeks is the effort level. So you can say effort and then you've got that scale. And I generally keep it on that extra high. But when I want it to like go bonkers and I'm like, I want to be out of the way, I go into that ultra code mode. And and I so I tried that with Fable earlier in the week and it just like woo, it was blowing through my my $200 a month. subscrip limit just like real quick. So okay, so there's that problem to solve. And then there's this other problem where I find myself you're talking about being good for repeatable AI being great for repeatable tasks. And I think repeatable workflows same sort of thing. And what I find myself doing is basically like, okay, I'm gonna go and plan this chunk of work.

Taylor MacDonald: The the

Sean C Davis: And then I'm going with with an agent, and I'm gonna give the agent the work to do. And then when the work's done, I'm gonna go get that's usually like, okay, I'm gonna have Claude do that work. And then when it's done, I'm gonna go over to Codex and I'm gonna have Codex read the plan and then audit the code. And I'm in the process this entire time. What I really want it to be was. I want to be there at the beginning to kind of spot check that like, yes, you're solving, it looks like you're going to try to solve this in the way that I want you to solve it. And then at the end, which is, hey, we've taken a number of different angles. The code is solid. We feel like it meets the original requirements. Go smoke test it however you want. So I want to be there at the beginning and the end, and I don't want to keep hitting yes, yes, yes throughout the entire time. So what I've what I'm working on is this series of skills and the concept here. is it's really it's really pretty cool and for me it's it's three different skills that come together and it's actually no it's it's mostly it's really just two there's pre flight which is the planning stage and then it it leads into this autopilot which is where it runs but the reason I'm bringing this up is because there's a concept baked into this which I think is fascinating and it's that the the same model never does the does multiple tasks, especially when reviewing is or auditing is involved. So you take the most capable model. So let's just say we're in Fable World. So you give the task to Fable and Fable becomes your orchestrator.

Taylor MacDonald: Hmm.

Sean C Davis: And the first thing it has to do is make sure that it's got the plan and it's checking everything out. But to do the planning, to do the research, Fable doesn't do that. It tells Sonnet or Haiku or Opus to do it, depending on what it is. Comes back with the plan and it's like, okay, nope, I don't have enough. Go back in. And so you've got that most capable model that's delegating. And then it does the same thing with the code. It sets up the coding tasks, sends it to usually Sonnet, maybe Opus for some of the more complex tasks. And then when it's done, there is a I learned about recently, there is a Codex for Claude code plugin. So you can have Claude delegate tasks to Codecs. So then it does that through the QA process.

Taylor MacDonald: Sure.

Sean C Davis: And so I have been able to, I'm getting close, I haven't merged it to my personal skill set yet, but I'm getting close to basically being able to say pre flight, let's plan. And then either Fable or Opus is like, okay, cool, ready to go. And then I can walk away and I come back to a pull request that is ready to review. And so it's like to to bring this kind of full circle.

Taylor MacDonald: Well

Sean C Davis: It feels really close to that balance between here's this factory and here's me watching over the shoulder and every step and more like I'm I'm in the loop only when I want to be in the loop and my level of annoyance over the last week has gone down significantly.

Taylor MacDonald: that's great. What is the quality output what you would expect? I mean, are you getting the results?

Sean C Davis: So far the it's so far it's positive. Yep. And And every every round is I I kinda build in a retro. I'm like, I I didn't like A, B, and C that you did. And why why do you think this happened? Because you can read this skill, what's wrong? And then it gives me that report. I take that back to the agent that helps me build the skill and we just kind of iterate from there. I've done that maybe three or four times and it's I'm gonna do it probably once more later today and then and then merge it.

Taylor MacDonald: Good idea. Cool. Yeah, that's a good idea. Did you ever play with Cloud Code? What was it? Like agent swarms or agent teams. I can't remember what they called it. I don't even know if it's still around. But it was a similar concept. It was it was I I stopped using it because it got so expensive. I mean, it was it was a lot. And so at some point I built my own using Hermes. and it it worked also a little bit inefficient, although I think token costs are coming down.

Sean C Davis: Yeah. Yeah, you and I were talking about it many, many months ago. Yeah. Yep.

Taylor MacDonald: at least for the non frontier models and maybe it might be time to go back and revisit that. But that's cool, man. That's yeah, that's that's a great wo maybe on one of these future episodes, once you get it kind of nailed down, you could share with us on, you know, the various skills and how they work and we could put it up on the website or something like that.

Sean C Davis: Yeah, I think you know what would be really cool I'm I'm thinking about on the fly now, 'cause the the it's the the delegation to the lower level models is it's saving an I it seems like it's saving an enormous amount of cost. I would love to I should take two identical repos and go through this go through the process say with Opus in ultra code mode and and then manually checking it and then go through the the skill process where it delegates to those lower level models and try to compare the cost between the two. That would be interesting.

Taylor MacDonald: Yeah. It'd be interesting, yeah. It'd be interesting if you could develop a heuristic on both that matched like the quality output, you know. Like if you had here are the stylistic standards I require, you know, I don't want duplicate methods. Like you'd have sometimes as the Claude does this to me all the time. It'll generate like the same utility method in every file as opposed to just moving those into a shared library and then calling it once.

Sean C Davis: Mm-hmm. Yeah.

Taylor MacDonald: You know, but that could be a heuristic, you know, and you could stack up if you did this, you did this, you did this, you did this. Now I'm gonna grade you on your output. It'd be interesting to compare those across the two the two outputs as well.

Sean C Davis: Yes, yes, absolutely. Absolutely. the one thing another thing I learned that I I didn't know about was I I was like, okay, this is cool. How do I check that this is happening? And so there is another there's another skill I attach to it called roster, just weird name. but after it's done

Taylor MacDonald: Yeah.

Sean C Davis: doing that end to end work, I can call this roster scale and it will print me out a table of all of the agents and what they did because it gets stored it's like you can't view it in real time, but it all gets stored somewhere on the machine and so you can look it back up. Yes. Yeah. Yep.

Taylor MacDonald: Mm. That's like an That's like an audit trail. That's that's awesome. Now that's interesting. If you could take that audit trail and you could apply it to that Kanban interface, that would be an interesting visualization of everything that happened or everything that exists.

Sean C Davis: Ooh, say more about that. What do you think the value of that? Like what what would a user wanna see from that?

Taylor MacDonald: Well, I I don't know. I think Kanban in general has become so commonplace that it helps us all I don't know, it's become like a like a standard way of organizing information that has multiple stages and and multiple dependencies and all that kind of stuff. And I I guess it might be interesting It might be interesting. Well, it I will go going back to that teams. Part of the reason it got so expensive for me when I would built my agent team was that I had it updating, manually updating the Kanban board. So for every feature it would create, it would go to this, you know, the backlog and it would create a new thing. And then that would get kicked off to the senior engineer to review or the

Sean C Davis: Mm.

Taylor MacDonald: lower level designer to like do thought work and align it with the design system and and I physically I I I forget how I did it, but I told the main orchestrator agent to to effectively like operate the board like they would a project manager. and it was awesome. You sit there and watch all the cards kind of like moving around all over the place. But like I said, at the time it was quite consumption heavy and it just didn't make a ton of sense. So I guess what I'm trying what I'm getting at is I enjoy it's completely superficial, but I enjoyed the visual awareness of what it was doing. And and Kanban makes sense to me in that regard because you have each swim lane represents a different part of the process. So it's always I I always I appreciate being able to glance at the board and seeing where the work is and like where the tasks are and who's working on what and all that kind of stuff.

Sean C Davis: Interesting, interesting. Mm-hmm, mm-hmm.

Taylor MacDonald: And it's and it's a fascinating study, you know. It's like one of the things that is both great in this era of AI coding agents is that it just does things and you don't have to worry about it. But I like to wa like I wanna see it. Like I wanna read it. I wanna see what that thought process, I wanna see what the delin deliberations yielded, you know, like and

Sean C Davis: Yeah. Yep.

Taylor MacDonald: it is a struggle just to even see what a single agent does. But if you have a single agent with like twelve sub agents that are all doing various and different specialized tasks, I would love to be able to just see it and like glance over and be like, that guy's working on this and she's working on that.

Sean C Davis: Yeah. Mmm, yes, yes. I will I was like, I'm playing around with so much. I will I'll have to demo my I'll have to demo the desktop app I've been building that's a agent harness next time. yeah. I'm nuts or I'm addicted.

Taylor MacDonald: Add that to the list. God, you you're nuts. All right, man. Well look, you you're a you're an addict. You're an addict. Get off the juice, bro. All right. thanks for the talk today, Sean. as always, I appreciate it. We went a little over what we intended to, but it was probably worth it. we'll catch you next week. I am as this episode is released, I am currently sitting on a beach in Michigan and I'm enjoying myself. So

Sean C Davis: Yeah. Yeah.

Taylor MacDonald: when I come back I'll be nice and tan and exhausted. So I'll look forward to catching up with you then. All right. Thanks, buddy.

Sean C Davis: Amazing. All right, we'll see ya.