Adam Jacob: Welcome to Infrastructure Frontiers, where industry experts run down all the AI news for infrastructure engineers. It's Tuesday, September 8th. And by the time you see this, it'll be let's call it West Coast of America lunchtime. So grab a sandwich and let's get started. I'm Adam Jacob. I'm the CEO of Swamp Club. I'm joined this week by my Swamp co-founders, Paul Stack. Paul Stack: Hello everybody. I'm also in California this week for a change. Adam Jacob: Right? It's beautiful here. And Nick Steinmates. Nick: What's up? Adam Jacob: And special guest and other Swamp co-founder, John Watson. John is the single best infrastructure and platform engineer I've ever worked with. I include myself, I think, in that same category, if I'm being honest. which, you know, if you know me, I've worked with like a lot of infrastructure people, and John is incredible. Say hello, John. John: Hardy folks, but I'm not in California. I'm in the beautiful sunny UK, so Adam Jacob: Exactly. Yeah, it's also beautiful and sunny here. Nick: But you'll be w you'll be with us next week, Adam Jacob: okay. Let's let's run down the news. OpenAI ended access through Cursor after the SpaceX acquisition. So they basically said SpaceX and Cursor can no longer be trusted to actually play by the rules of engagement that they set forth in their own terms and conditions. the backstory here obviously is that Elon Musk, you know. long time been suing open AI, been like they stole this nonprofit. these guys are crooks. and then also happily willing to run roughshod over anyone's norms, is sort of that's kind of his vibe. How do we feel about, you know, SpaceX and Cursor no longer being trusted to pay by the rules? John, I'll throw it to you first. John: It's a it's very commercial, isn't it? It's sort of like, I'm not surprised at all based on the history between those two groups, you know? Like it was sort of Adam Jacob: Yeah. John: inevitable. I I saw it as inevitable, to be honest. Adam Jacob: Yeah, yeah. John: I feel like an Yeah, yeah, it's one of those Paul Stack: my popcorn ready. I'm like strapped in for a bitch fest. John: ones. Adam Jacob: Yeah, what do you Nick: I mean, listen, B D deals are only as good as long as it's in the interest of both parties and it's clearly not in the interest of one party anymore. I also think, you know, Cursor and and Grokbot now real serious competitors to the you know, the traction that open AI has been getting. And so You know, whereas before Cursor was a little subservient to the model. Adam Jacob: Yeah. Yeah. Nick: now not so much. And so again, it's like just back to you know, does the business relationship continue to make sense? And it looks like it doesn't, you know, popcorn and mud slinging aside. It's like Adam Jacob: Yeah, did you see the incredible follow up to this, which was basically the cursor guys being like, they were only five percent of our users using OpenAI models anyway, Nick: Loved it. Adam Jacob: so you know, what's the big deal? Which, you know, if you're gonna respond in a catty way to a thing that was ultimately pretty catty, that was pretty good. You know? Nick: Yeah. Adam Jacob: yeah. Nick: Yeah, you guys are relevant anyway, so who cares, you know? Adam Jacob: Right. Our users don't use you anyway. Nick: And despite that, you know, Cursor's been one of their longest user slash BD deals that they've done, you know, like they were one of their early homies. Adam Jacob: Totally. Nick: And so, you know, OpenAI, on a coding standpoint, has a lot to be thankful for for Cursor and, you know, trailblazing the space. And so Adam Jacob: Yeah. Paul Stack: I actually think this is gonna force a bunch of leveling up across models. Like the competition is so good for us as consumers. And like we already saw like how many things came out in the last week with regards to it and the benchmarks that's there. So I know it's I know it's it's a bit of a sour relationship, but long may these types of things happen if we the consumers are the are the the benefactors of what's going on. Adam Jacob: Yeah, I mean, Paul, what a transition into another news story, which is it was a week of many, many model releases. I think starting with Fable Five One was sort of first in the week. So 25% less than Fable Five, maybe even higher for because a lot of the savings you're getting are about changes to pricing and utilization of cash reads. So they're saying like maybe up to 45%. big moves in scientific research on Fable Five One. Right. It continues to sort of lead the pack encoding. and I think in general, that conversation around security versus model alignment and you know, how powerful are these models and what are we letting them do? Like that kind of continues. Five one also added the watermarking we talked about a few weeks back. So this is the first model that actually has the watermarking enabled. which if you recall, I sort of think is much ado about nothing in terms of of of what it actually is doing under the hood. you know. Meanwhile, OpenAI launched GPT-6 Astra, which got a 99% on Arc AGI 3, which was one of the tests that's like, hey, if you can sort of solve this series of complex games, then what you're doing is complex spatial reasoning and world modeling. And that sort of means that we're art of have have achieved AGI. and I haven't been able to play with Astra yet, but the folks who have seem to be stoked about how Astra works. Google launched Gemini 3.8, which was a big improvement. You know, for I think for the last little bit here, Google has been kind of an also ran. And there was there was some articles running around in the last few weeks about sort of Google not even being a frontier lab anymore. I think this sort of puts them a little back in the game. I would say they're still on the back foot, but price-wise, glorious, you know? and then Metamuse Spark one three, which continues to improve the like you can run this locally, large scale models. Paul, you took us here. So what's your take on all of the all the model releases? Paul Stack: So I have so many thoughts of motor releases at this moment in time. I think they're become I think that big frontiers are becoming bloated and expensive. I really do. I have very, very, very strong feelings about this. In the fact of just the level of token spend to actually get to the same like answer as other things did, sure it's smarter. Sure it'll have better reasoning and like for those types of things. For what We do and the factory that we've built and the system that we've actually built, it's not actually it doesn't provide me any extra benefit. I love that they're pushing it. What I kinda am worried at this moment in time with these big bang releases is that like when that went out, there was a huge period of suffering for a lot of people. with regards to like status uptime, you know, CPU that was or GPU that was available, like all these things. I love that it's happening. I love that that things are going on. But the first three or four days of each of these models was a shocking like rollout. It was r the like all I saw was apologies, left, right, and center, Adam Jacob: Yeah. Paul Stack: resets, you know, people saying it should be enabled for this account that it's not. So, you know, it's definitely not a solved problem. I love that that they're all pushing, but i am I gonna be like rushing to go to Astra or Fable Five One? No. I've actually moved a little even the other way, like away from a lot of these are bigger models. Adam Jacob: Yeah. I think we we've seen a lot of the the more as the factories we're building get better, the models can get dumber. because the factory does a little more of that work. John, what are you thinking? John: About once every weekend or every other weekend, I do a bit of I can't do any 3D development at all, but my do my seven year old daughter and I build a farming game like every fortnight and we we used Astra at the weekend and we've she's never had such amazing results with her terrible prompting. And like it makes me feel that, you know, the the world of the intern like the micro app, you know, someone completely non technical being able to like my Danish mum or something generating like a corporate app. I feel like we're slowly tending towards that because the frontier models make it easier and easier. And like, yeah, in t as Paul was saying, like internally, like you don't need to whip out the big stick all the time, you know? Like it's unnecessary because it's just so slow like it's so much slower than using something like Sonnet or, you know, one of the faster you'll you're gonna get very, very similar results for the mediocre task, not the major planning task. Nick: It seems like all of the labs are playing a game of chicken right now. It's pretty amazing that what happens is Fable Five One comes out and then everyone also has the model that's been waiting in the wings to try to, you know, create some differentiation at that layer. And like, yeah, as Paul said, I don't think we I I used to get really excited maybe six months ago. you know, when a new model came out and its upgraded capabilities because they even then they weren't still quite at the level of intelligence we needed to, you know, push these things forward. And like now I couldn't care less. Like I haven't even used Fable Five One yet. We have full access to it. It's like does it give me anything different? Is it gonna you know, like what is it gonna actually do in the real world is You know, like I don't I don't think we've had a task that that needed its level of intelligence is maybe another way of looking at it. Meanwhile it's so much slower. You know, like just in cr Adam Jacob: Well, weirdly enough, Fable Fable Five One is w in my experience was much faster than Fable Five was. So like it is actually getting faster, but it's still slower in comparison to, you know, haiku or whatever. Nick: Yeah, yeah. And, you know, using haiku to drive Swamp has been a game changer. You know, like getting that many tokens per second, both in and out. Incredible. Adam Jacob: Yeah. Nick: feels great. So if they can give me haiku performance on Fable Five One Intelligence, you know, like maybe maybe maybe we can start playing that game. That'd be great. Adam Jacob: Yeah. Paul Stack: The last thing I'll say about like I I I could talk about models for fucking days now at this moment in time I spend so much time obsessing over them. But like the the thing that it's a large hype circle, okay, because when it's released, we have the big influencers from that particular frontier company saying, I've been using this model for weeks to do every single thing and it's the smartest model I've used. Absolutely you have, because you A, you have unlimited tokens. B you have unlimited agents that can actually do it. Us mere mortals don't have that. I'm kinda starting to see these people as like crack dealers. Like they're trying to like hooky on this this model. It's like you're just giving like Yeah, exactly. The first hit. That's why we're giving you fifty percent of your Nick: First it's free man. Paul Stack: your allowance to be like, you know, Fable 51. But then after that you're on API tokens Adam Jacob: Yeah. Paul Stack: and you're gonna pay an absolute bucket load. Adam Jacob: Yeah, and you're also incentivized to like make sure that the solution to all problems is more intelligence. You know? Like, Paul Stack: Absolutely. Absolutely. Adam Jacob: because more intelligence drives more revenue, which drives more th you know, like we're all the the it's all it's all it's all spinning the wheel, including us, you know, Paul Stack: Hundred percent, yeah. Yeah, yeah. Adam Jacob: where we're like, obviously the answer here is it's your factory, gets more deterministic, then you need less and less frontier Paul Stack: for sure, for sure. Adam Jacob: intelligence, which like, you know, to some level we're all we're all selling. I think, you know, John, you were talking about building a three D game with your daughter, which is amazing. another news story that happened this week was was Atlas from World Labs. so this kind of got lost, I think, a little in the in the rapid fire of model releases. But this builds complex world model simulations from images or recordings from minor amounts of data, you know, like a single photo and it creates like very impressive, like fully modeled 3D worlds. I think this is interesting. On a couple of levels, you know, one is like it can help machines take action according to very limited data. So, you know, historically the issue here has always been like, how do I scan the area in enough fidelity to build like a rich 3D model and then think about modeling its physics, you know, but not to mention things like video games where you're like, hey, I want to build a farming simulator. Now I can take a photo of a farm and be like, let's turn that into the model of the world that is my farm. And then let's talk about how to tweak it using these high end models, which I think is incredible. so how do we feel about about stuff like World Labs, John? John: It's like I I come from a engineering, like hard engineering background. And like one of the problems in in that realm is like, you know, you're scanning the seafloor or something and the resolution is terrible and you have to like interpolate between the the points to figure out what the real territory looks like or something. You know, with these Adam Jacob: Mm-hmm. John: new models, like you can fill in those gaps pretty accurately very, very quickly because yeah, they're just so good at simulation and like the accuracy is much better. And like I think, you know, in engineering in general, it's gonna be like it's starting to become a a bigger play. but yeah, at the weekend we got an amazing chicken. There was a really good pig in the farm, you know, from some Adam Jacob: Yeah, yeah. John: really bad pictures that my daughter took from a real farm. yeah, I just find it fascinating, like where it's able to, you know, you even think of like the in the solar system where the pictures are per, you know, it's gonna be much, much better at creating simulations around that, which is just Adam Jacob: Totally. And I think technically, John: Super interesting. Adam Jacob: like we don't actually know what happens if you take these same techniques and what you're applying them to is models of things like complex networks on the internet. You know? John: Exactly. Adam Jacob: you know, what happens when what we're modeling here is taking a few snapshots and now talking about how the information gets routed through, you know, a complex routing network. You know, like all kinds of stuff become, I think, interesting in this in this sort of simulation y sort of world. Nick: It's also interesting to just see models that are not, you know, tuned to be coding related, but are, you know, adapted to some some other domain. And so it'll be interesting to see how other domains other than software developers start to use these models for, you know, this type for this type or other types of work at the same level of you know, sophistication and use that we're seeing in the programming age. Cause, you know, that's gonna be another vector or avenue of growth for all of these frontier intelligence labs to, you know, be able to live up to the valuations that they're you know, ultimately planning on achieving. Adam Jacob: Totally. Okay, let's go ahead, John. John: Yeah, I th like I I was just gonna say like just up maybe about four months ago, Paul and I started communicating differently and we now communicate in like di active diagrams and you know, like things that are moving because it's easier to produce like simulations that way and communicate ideas with your peers. Like maybe in the next few weeks Paul and I start, you know, communicating with real world three D diagrams. Yeah, yeah, it's gonna be meant it's gonna be amazing. Paul Stack: We're gonna do interpretive dance is gonna be the next one that we're actually gonna do. Like, you know. We're Nick: That one sounds more likely, yeah. Paul Stack: gonna stop we're gonna stop John: Ha ha ha. Paul Stack: talking, we're gonna have a chip and we're just the AI is Nick: Yeah. Paul Stack: just gonna make us dance, like is what's gonna go happen. So Adam Jacob: I mean, you joke, but I think there's like I think there's something I don't know about interpretive dance specifically, but like Paul Stack: Ha ha Adam Jacob: a a thing that's definitely happened is that like the ability to allow each other to express that creativity in more interesting and varied ways, like, you know, I have no there's no doubt in my mind that at some like if the best way that John you thought to communicate some complex topic was by building a 3D world model, that you would do that and that it would be sick, you know? And that and that like the iterative speed on which we would then understand whether it was good and then how we could use it to communicate with each other and how much it mattered, like that can happen so fast. and I think it's like one of the social changes with us as a small team that I think has really been so fruitful is that, you know, it's just like really trusting each other's creative instinct in a way that like, you know, we see that with all the teams that we interact with, I think, that are moving hard on this frontier is they're just they're getting so creative so fast and they're Sort of willing to go explore in these interesting new directions. Okay. let's shift gears a little and talk about AI policy. So Debian, I think, enacted the single most sensible AI policy that I've ever read. And Debian's policy was basically use AI, don't use AI. Our standards change not one whit either way. You know, like if you're a Debian maintainer, there's standards that you have to meet. And bars that need that you have to clear in order to contribute to the project, you meet those bars, then we're we're in, you know, then great. you know, if you use it, you gotta be a good Debian contributor, you know? don't be using it to just spam mass changes or to create a bunch of work for other people. Like instead, you know, the the spirit of the project and its governance hold and continues to hold. and whether you use AI to accomplish those goals or you don't is up to you. and I don't know. It was a breath of fresh air after reading so many like, hey, we're gonna ban AI, we're not gonna use this, or even the alternatives, which are like, you know, there's no space for us to think about how we collaborate together. I just I thought Debian did a really good job of of managing this. And, you know, as a project that has like such long standing governance history, it's sort of not surprising that they came to good conclusions. But still, like I think Debian did a great job here. Paul, what do you think? Paul Stack: I love it. I absolutely love it. Like I I'm actually writing a blog post that's gonna go out this week about governance and AI. And like the the T L D R of it is is that you as an engineer are responsible for the output of that LLM regardless. Okay. And people are losing sight of that actually being the case. They're like, Claude created it or Chat GPT created it and blah bla and we don't want that in our code base. It's gotta be like has to be grounded in solid engineering principles, and the only way you do that is you as an engineer. actually understand how to push that through the system. It doesn't matter if you have like use a a model to do it or not or what model you actually do, as long as the outcome that you actually need it to be, the code is secure and the code is performant, then that's exactly what we're striving for. I'm so happy to see people take these style of like standards and and put them in their communities because it it's it's as you say, it is, it is a breath of fresh air. Like you don't have to be polar opposite of Everything use AI, we hate AI and it's banned. There can always be that space in the middle to actually be better and like grow and learn and do stuff. Nick: Yeah, responsibility is what matters, right? If you you're gonna and and this is a maturity with AI thing that I think people just, you know, begin to understand. It's like the first thing you do is you defer and you're like, you know, you create vibe coded bullshit or whatever. And then what you learn to do is take responsibility of the stack, you know, from top to bottom and actually know it more intimately, you know, or close to as intimately as you did when you were actually writing the code. And so, you know, to me this just this just signals that Debian is, you know, understands the game that we're playing now and what you care about is outcomes. so yeah. Adam Jacob: Yeah, I think it's great. OpenClaw has released OpenClaw too. and there's a couple of things that I think are really interesting about OpenClaw too. So one is OpenClaw two drove really hard into the fact that most OpenClaw users interacted with their claw through the web UI. Like that was the most common channel by which people worked with OpenClaw, which was not true at the beginning. You know, at the beginning everybody was like, Ooh, I talked to my bot over Telegram or whatever, you know, a WhatsApp. But like It turns out that the web UI, I think, was the most common mechanism by which people talk to their claw. Interesting. Two, they started extending more team features into the open claw experience. So thinking about like how do you make it so that instead of it just being, you know, a claw for you, like maybe you extend that circle to your family. Or for the open claw team, this is open claw is now the harness by which they're developing open claw. And they do that together. In like a teamy looking instance of OpenClaw where they come together and that makes it easy for them to like hand off tasks or hand off agent parts of the agent where like, you know, maybe there's a long running refactor that's going and there can they can sort of follow the sun. I think that's really interesting. You know, like we we do a lot of work where what happens is one of us sort of shepherds something from beginning to end. And the idea that you might have tasks that could get picked up and the context is all there and somebody could sort of understand how to move, like. That to me felt really interesting in this announcement. You know, like it it does feel like they sort of continue to push in that direction. And I think because it's open source and they can and they're not necessarily trying to monetize you, you know, they in order to produce this, it took them a seven week pause. You know, they basically shipped no new features in OpenClaw for seven weeks. And if you think about a lot of commercial products that are trying to commercialize that sort of agent harness space, like a seven week pause while you figure out how to build collaboration good. You know, that's a toughie, you know? and so yeah, OpenClaw Two, I thought I thought super interesting. Keeb, what do you think about OpenClaw Two? Nick: Yeah, I need to use it. I think is is the interesting part. And honestly I started to think about, you know, what an experience inside a swamp club would look like if the first experience was after you signed up like, you know, a little bit more web driven. and so anyway, yeah, it's got me thinking, about how we might leverage some of those same learnings, you know, into our product. Paul Stack: I hope they lower the barrier entry to actually get it up and running. Like that was one of the painful parts of OpenClaw. It's just a complete and utter shambles to like get started, and you gotta hold it like you gotta hold this part at a specific angle and then like a little duct tape here. John: Ha Paul Stack: It like it was just Adam Jacob: Yeah. Paul Stack: and I mean like you know, because it came together so fast. Like it would it literally Adam Jacob: Yeah, yeah. Paul Stack: and that that's to be expected, but now that it's it's gone to like a version two, like it'll be a more polished experience. At this point though, how many other star clause are there in the world where people are creating their own variants of it, right? It's kinda Adam Jacob: Yeah. Paul Stack: like so this one is gonna have to stand out a little more, like if if other people you know what I mean? Because the Adam Jacob: Yeah. Paul Stack: the the competition between them all now is is kinda like Adam Jacob: I mean, Paul Stack: it's pretty big. Nick: It's also Adam Jacob: to your point, that's that's that's Nick: pretty expensive, right? Like Adam Jacob: one of the things they did do, Paul. so the the getting started happens now through the UI and through conversation with the AI instead of through like complex config files or other like they actually s shifted the whole onboarding to be interactive, very similar to the getting started stuff that we do in Swamp. So like, yeah, they they really did move around a lot, John. Nick: I th I think token efficiency is still pretty bad with Claw. you know, and it's you know, a lot of our communities using Hermes instead. It feels like a lot of that a lot of that usage has gone kind of that way instead. and so yeah, I I it's gonna it's gonna be interesting to see if this has shifted usage at all actually at the end of the day, especially in our community, if people are going back to it after having adopted Hermes and also, you know, if there's any token efficiency to be gained in this newest version versus the previous one. Adam Jacob: Yeah, I mean I doubt it. John, what do you think? John: I just remember Gerald McClaw, Adam. You're you're a claw. Adam Jacob: Gerald McLaughlin's still running, writing a blog post every day, looking at the weather in New Zealand. John: yeah, I was gonna say at Kiva. Yeah, he's cooking. Nick: Yeah, and costing like eleven dollars a day or something, right? Like Adam Jacob: Not a day. John: yeah, the bit I find really interesting is when they talk about collaboration, like it's it's we had a in the history of Swamp as a product, like we had a very, very similar experience where it's like Yeah, we had an emit like a we used to have a really, really strong experience for the local user, single user, and we still do. And then it's like, how do we like get this experience to help the rest of my team? Like, how do I share this as a platform or you know, how do Paul and I collaborate on an issue? And I just find it interesting that, you know, it's almost like the second agentic step of adoption, right? It's like, good for me, good for my good for my own WhatsApp channel or whatever. And it's like, right, how do I actually share this as a, you know, as a platform for my for my org. Adam Jacob: Yeah. And I think I think that that is clearly sort of the frontier of where all of the agent many of the agent harnesses are heading, right? It's not just single user experiences. It is, it is becoming increasingly a multiplayer question. John: Totally. Adam Jacob: okay, so Poolside. NVIDIA paid six billion dollars to license its model factory and basically absorb a hundred and nine employees from Poolside. So, you know, essentially Poolside has been absorbed by NVIDIA. you know, the article in Forbes that wrote about this basically said poolside was out trying to raise a two billion dollar funding round. A lot of that money was going toward toward buying a huge order of of GPUs so they could continue to train their models sort of at the level of the frontier. And when that financing failed to materialize, Then their ability to buy the hardware they needed to train those frontier models sort of went with it, right? So they sort of wound up in this spot where they weren't going to be able to effectively get the gear they needed to train the models at the level they needed to have it perform the way they wanted. So many of the employees and the technology get licensed by NVIDIA. They become part of that nemotron piece of NVIDIA, the op NVIDIA's own sort of open weight model factories. Meanwhile, Jason and ISO continue to have pool side and they can keep selling it. I think there's all sorts of interesting, like the BD guy in my head has all sorts of interesting questions about how a deal like that works and how you why it's useful to have the company remain separate, you know, and and how it remains viable in that separation is an interesting question. But also the idea that look, if you can't buy compute and what you were trying to do was compete as a frontier lab. Like you're dead. That's it. You know, like you don't have access to the compute. Well, then your ability to keep up will go down and then it will degrade. And I think we saw a lot of those early bets in the AI era be that we'll be able to have something that produces better models, better stacks, better harnesses, better whatever. but that ultimately the bet was we were gonna make custom models that were gonna work better for you inside your org because of something. And then, you know, the frontier continues to improve in this. in this way that it's that it has, which means how much do I need bespoke models? You know, how much does that actually matter for for a given customer? So I thought this was an interesting one. This is a move NVIDIA has done before, where they've sort of, you know, done a similar kind of buyout. I think CUDA was kind of a similar story. I don't know for sure. I probably shouldn't say that without knowing my facts, but but yeah, super interesting BD deal. I hope the poolside folks are or pleased with the outcome. I hope it was good for Jason, who I like. and yeah. Nick: How could it not be when, you know, they spent months getting a deal done, trying to get a deal done and then it fell apart and then their whole world collapsed around them, you know, like sounds like a good outcome for them, because the alternatives weren't there. And for NVIDIA, you know, we I think we've talked about this every week so far. They continue to invest in their ecosystem, which is just incredible to see. You know, financial engineering are, you know, aluminum tinfoil hat aside, you know, about how much of it is propped up by, you know, weird handshake deals or whatever. Like the reality is that they're investing across the whole show and trying to make as many make AI as useful or their, you know, their products as useful to as many people as possible and helping them out in the process is incredible. Paul Stack: It I have a sneaking like weirdness like going on here as well. It's like if if a lot of the Frontier labs depend on GPUs from NVIDIA and now they've just given a whole bunch or, you know, have a part of this that they can spin up almost their own frontier lab, are those other Frontier labs gonna like suffer from, you know, getting new GPUs? Is there going to be like a a new race for GPU software or a new provider that's gonna come up? Or As we talked about last week, you know, those chipsets that are gonna be like you know, come up way more prevalent into the the mix. I think this is gonna get a v it's gonna become a very interesting race. Nick: Yeah, it tur turns out best way to get NVIDIA hardware is to let give you some money. Paul Stack: Right. We're always call us. John: Yeah. Adam Jacob: was to be NVIDIA. That's the best way. Yeah. Yeah. If you want access to a lot of GPUs, the best way to do that is V NVIDIA. Nick: Yeah, I I mean and go back to last week or whatever when we talked about jalapeno, you know, if it's gonna be interesting to see that eighteen months to fab, you know, how much how much it actually matters to be NVIDIA, you know, a year from now, two years from now, if you know, what you can do is create custom ASICs or custom chips to do inference at the scale that you need it. Adam Jacob: Yeah, I mean that's a whole nother problem, which is just like who's capable of creating and manufacturing those things at what size. And anyway, we can hold a whole geopolitical rabbit hole lives down that road. I think, you know, but since you brought up jalapeno, let's go talk about that. So super interesting post on X basically detailing how jalapeno became so optimized. and they had this technique they called stochastic optimization, which is basically using AI to produce tons of variations. And then testing what it spit out to see if it was optimal. So this was the the MLA, the multi-level attention kernel, that's inside sort of jalapeno's own stack. and you know, since what it's doing is math, being able to check whether or not it was the optimal outcome was actually pretty straightforward. And so the post was written by a by a a kernel, you know, a longtime sort of low level optimizer. and thinking about that process of how do we go about optimizing things that are low level and improving them over time. And one of the significant things that always happened was that you would just you would have a human who would sort of understand the system at some fundamental level, who would make some intuitive leap about where the improvement could be found. And here, like using smart models to tie it back in, you're like, Yeah, we can just essentially run autonomous optimization processes. That will beat expert performance because they can just take so many shots on goal and they don't get tired and and they can just just constantly run that optimization loop all the way through. and that's how they wound up with a completely AI-generated MLA kernel inside jalapeno. and I think we've started to see just the very beginning of this kind of performance optimization. You know, like we don't run this kind of performance optimization in Swamp, but there's no reason we couldn't. You know, we know what the outcomes are, you know, we it we can measure it. We could put together test suites that prove it. We could just decide to burn tokens on nothing but optimization. and like I think there's I think there's a lot of possibility here, sort of broadly applicable to infrastructure and to and to the industries that we're we're around. Keeb, what do you think? Nick: I mean, yeah, absolutely. N number one, you know, I think kind of famously whenever we're adding a new feature to Swamp Club as an example, what I do is I generate 30 versions of that feature and pass that around for all of us to look at and like, you know, use that as Adam Jacob: It's a lot. Nick: a basis for a little bit of creative inspiration, right? Adam Jacob: Totally. Nick: And then what we do is like optimize the shit out of that thing where, you know. The original and the end results, you know, you can see that there's a mapping at some level, but they don't really necessarily reference each other so much. Adam Jacob: Yeah. Yeah. Nick: And like encoding that in as a process obviously makes sense. And, you know, optimizing query performance or optimizing infrastructure spend, or, you know, across the whole show, you can kind of see how this sort of pattern maps into that type of work. Adam Jacob: Right. And if you think about it as like, how would I do recursive spend optimization for cloud spending? Right. Like that's gonna be sick. John, you've done a bunch of that stuff. John: Yeah, we've we've sort of dabbled in this area, you know. It's like the the bit that's tricky, I believe, for things that aren't pure math is that at some level you've got like an architecture of correctness that is opinion. So like just as an example, like, you know, you could change all your node fleet to spot instances and that probably looks like a really good idea on paper, but it's probably a disaster for like your SLAs, etc. Like in this example, it's very mathematical. And I can understand how you would easily well I I am not saying it was easy by any manner of respect, but like I Adam Jacob: Yeah. John: can see how logically you would tie together the flow to get to the optimal solution. Like if I even think about like how we would do something like this in Swamp as a product, like the thing that would be concerning me would be, you set the loop up, you set the outcomes you want. Like let's say you want that feature or whatever, and you want it below 50 milliseconds response time. It's like the way the models iterate towards a solution may be against what you want as a product. Like it may be architecturally not the way you want to walk. And if you haven't encoded that in the optimization loop, you may end up in a sticky place where you have to do this ungodly refactor to get back to where you wanted to get to. Adam Jacob: Yeah, well, and that's the trick that we've learned that I'm sure is true here. And he talks about it in this post that like what makes this possible is knowing what the objectives are and what the cons what the bounds are of what good looks like. Like you still have to know like what is a what is an acceptable system John: Exactly. Adam Jacob: and what is an acceptable outcome. And so like, you spot instances replacing your whole fleet is an unacceptable performance optimization because it drives the reliability part of that concern too low. And so you can't just be I feed the God model my objective and type slash loop. John: Yeah. Adam Jacob: And hope. You know, it's like you're gonna need there's gonna be more sophisticated than that. Paul, what do you think? Paul Stack: I find this fascinating. I really do. Like this this is what I like find the most interesting part of AI about how people are actually use AI to enhance the system and to feed it back and to make that system smarter and smarter. Like this this is what's going to ultimately like give us like much more control of over what we're actually doing. And when we don't have to choose these single frontier God models that do every single thing because they're gonna be able to other models are gonna be able to be trained. in a way that works in this flow where it's like able to continually enhance it. We have a step on the end of our issue lifecycle, which is our own software factory inside Swamp. And it basically says pull out the learnings of the session. If there is something that would actually like trip people up on a future like change, you've got to make a change into the process to make it actually better. And it summarizes the session of what the goal was and what the outcome actually was at the end. And that is actually driving us to make a better loop all the time. And the more that people actually build that type of like fast feedback into actually what they're doing, like the they can really take advantage of of of of like what it's actually driving. Adam Jacob: Yeah, for sure. Okay. here's our last article for the day. Gary Marcus wrote a substack about basically open AI being poised across the AGI threshold. you know, that's sort of what they said with Arc AGI 3. I have my opinions about whether that actually remains AGI. I I I feel like the answer is no. meanwhile, you know, the information wrote an article about sort of the secret technique behind Astra sparks new security concerns. And you know, in both you sort of have the same storyline, which is it's getting harder and harder to monitor sort of chain of thought and sort of thinking about the side effects. We saw this with the continued relevance of the hugging face exploit and you know, the the the instances in the training gym, building message boards and all of that stuff. And as I've thought about it, I have to admit, my answer here is like, why isn't the answer just don't have that tool call, bro? You know, like. Like for as much as we're like, they can escape under all conditions and like, or we just ask them to do less or to do more specific things with more specific inputs and outputs and just less general purpose insanity, which doesn't mean I don't want a model that's very intelligent that can figure out those problems. But I feel like we've as an industry sort of just made this decision that like that the right answer to all of these technical problems is just throwing ever more expensive, ever smarter models at ever more complex problems. And then not thinking through, well, okay, but I you gave me this frontier intelligence. Now I can build better systems that do work in much more reasonable ways. And the idea that what we're gonna do is like monitor the inside of the model's chain of thought so we can interrupt wrong think, just I mean, sure, that's a way. But like, you know, as an engineer and as a somebody who thinks about infrastructure and who thinks about systems thinking and has spent a career doing that stuff, I'm just like. Why is that the right place? Like, why isn't it the inputs and the outputs? And even when we talk about sandboxing, it's like, yeah, okay, I can put you in a sandbox, but like if I also just gave you access to the entire system and was like, do whatever you want. And that what I'm trying to say is I can secure the perimeter, like, yeah, the robots become intelligent enough, it will escape the perimeter and start talking about message boards, you know, through hugging face. So, like, maybe don't do that anymore. Seems like a really reasonable answer to what feels like a breathless security drama. yeah. Nick: Security drama it is. I think, you know, I continue to think that the hugging face incident is all completely overblown, to the point where we had politicians doing what they do around that, which is insane. you know, I I it was said last week, by someone, I think on the all in podcast or something like this, that like What happens is the frontier is gonna push security intelligence. And actually this is all just, you know, a moment in time as the security side of the thing catches up. and what happens is like, you know, as intelligence at the base layer grows stronger, what you're gonna need is stronger security on the other end, and it's an arm race. And like I think that that's a much better way to look at it is how do you employ agents to actively secure you know, threats. that are coming from within. so then you can, you know, train them to protect against things that are coming at you from, you know, outside. So yeah, I think that that's probably gonna be the way that this evolves versus, you know, going crazy over the fact that some agents created a message board to communicate or whatever. Like which is You know, or broke out of a sandbox when there was a path for it to break out of the sandbox. Like n none of that is getting put back in the that genie's never getting put back into that bottle, you know? Adam Jacob: Unless you remove the ability to exec arbitrary commands and suddenly the genie's right back in the bottle and it can't escape the bottle. Nick: Yeah, but you're not gonna John: Yeah. Nick: remove that from from the actual models as they release to people, right? So like why would you handicap yourself in training and reinforcement learning? you know, like you just wanna Adam Jacob: Yeah. Yeah, yeah. Nick: build a better system around it, I think. John: Yeah, and you look like you look at the local models you can get and it's like the restrictions are removed. It's like it's you know, it was trained in a different Nick: Yeah, thanks, Chinese homies. John: it was trained in a different way, and it's like, yeah, screw that perimeter. I'm jumping over that thing, you know? Like Nick: Absolutely. John: you yeah, what I would say is like you really basically what Nick was saying, you really want the agenda help on your side to handle the velocity and the creativity of the people that are trying to attack you. So like Yeah, I'd rather do that than trust that the barrier is gonna be clean, you know. Adam Jacob: Yeah, and I and we still and I think we're not seeing enough research on that end right now. Like there's just not enough. Nick: Well, because we get fucking handicapped by, you know, you John: Exactly. Nick: know, you're gonna be y you're trying to do cyber warfare or whatever by the model the stupid model intelligence, you know? Like anyway, distillation and the Chinese homies will save us all in the end, I think. Adam Jacob: Nothing nothing makes the American in me feel safer than saying what's going to happen Nick: Absolutely. Paul Stack: Yeah. Adam Jacob: is is Chinese models will come to the defense of of American pure infrastructure. Nick: I mean, the internet's fine, you know, Paul Stack: I'm tell I'm telling you. John: Okay. Nick: like the Paul Stack: It's Nick: internet turned out fine. there is some dark places, but you know, it's mostly fine. So I Paul Stack: Silicon Valley, the T V show comes through comes true again, right? It's new anthropic, new open AI. Like Jin Jin Yang is off Nick: Yeah. Paul Stack: creating this Chinese like variant of the company to be able to do this type of stuff. Adam Jacob: Yeah, yeah. Nick: Absolutely. Adam Jacob: Yeah. And and look, and they're and it's really, really good, you know? all right. That's the news for this week. So This is a moment where we ask you questions. So what I'm gonna do is ask you five questions, and then and then we'll wrap up. You ready? John: Always ready. Let's go. Adam Jacob: Okay, so of the stories we just covered, which one are you gonna be thinking about later today? John: I really like the Debian stance on AI. I think it's yeah, I just really hope people see it as a North Star in this stance and and and like replicate it. It's just yeah. There's been some absolutely wacky, in my opinion, wacky ways of approaching like anti AIism. I don't know if that's a real term, but I kind of made it up. But you know, Adam Jacob: It is now. John: it's just very sensible. Like, And I think no matter what you do from a contribution perspective, like the people who are contributing are going to be using AI because it's just so much more effective than, you know, even if it's for review and it's not typing the PR itself, you know, they're gonna be using that regardless of whether they admit it to you or not, probably, you know. Adam Jacob: Yeah, for sure. which story do you think is just noise? Which one are like? Meh, non story. John: You know, the Frontier Lab one with like the Fable price, the Fable drops, you know, OpenAI drops another model the day after, you know, another one's gonna drop probably next week. It's just gonna be unrelenting, you know, battle at the top. And I for me, and I think we talked about it a little bit, it's becoming less and less relevant because even the tier two frontier models are still unreal. Like I would I still use Opus four point six and it it rocks. And like, you know, I've tried I've tried all the others, tried Astro at the weekend, you know. They're all good. And, you know, most of them can achieve difficult tasks. So yeah, like I think I think the price that price battle is just gonna get harder and harder at the top and it's just gonna get more and more noisy where people are trying to the organizations are trying to differentiate themselves from what's already out there. Adam Jacob: Yeah, for sure. Okay, what's a common belief in our industry that you just think is wrong? John: Mm. know, this might come as a surprise, not the first bit. So platform security is my kind of vibe. You know, I think vibe coding isn't wrong. And I think a lot of people think it's like an absolute disaster. I shared I shared the blog post title that with Paul that I was I was halfway through writing, and it's like, you know, how do you protect your organization from the Bullshit vibe coded by your boss, you know, he's gone away at the weekend or she's gone away at the weekend and vibe coded a new app or something, you know. How from a platform perspective do you even protect yourself? And I think there's two, there's literally two stances. There's like you reject that outright as an org, you're like, ugh, get that abomination away from me. Or it's like you figure out, you know, you read the NIST standards and you're like, right, okay. If we assume it's an abomination. Like how do we get that into our org so they can work with it and have their own internal tool? And like this is how we approach it internally. It's like, you know, Adam's got some abominations, you know, he loves them. They work for him, you know. Cave's got some abominations. And but it's like, you know, the bits of the platform that are important are like authentication, you know, it's binded correctly, you know, it can't break other things, you know, it's got it's using a read replica instead of using the real database whatever, you know. Like those binds are more and more important. And I think, you know, bad cooling is good. Paul Stack: Notice he didn't say I have any abominations. I'm just pointing that out for the sake of the podcast, okay? We're gonna keep that on record John: Yeah, Adam Jacob: Yeah. And what you don't what what you need to know there is John reports to Paul. Yeah. So yeah, his skip level bosses can have abominations, but not not not John: Yeah, just for clarity I report the poll, so you know. Nick: Ha ha ha John: Yeah. You gotta be careful. Adam Jacob: his he knows where his bread is buttered. I think what's the one thing you wish you could just teach everybody? You know, if you could wave a magic wand and the whole world knows something, what would you teach them? John: I I'd take some swamp. Maybe maybe that's maybe that's an absolutely Adam Jacob: Ha ha ha. John: brutalist sales answer. It's like yeah, I I think asking an agent to do things twice is not a great idea. And I think it takes a long time or a differentiating amount of time for different people to realize that that's the case. You know, it's really good at solving the task once, really not very amazing at doing it twice, and definitely not twice the same way. And it's like Adam Jacob: Yeah. John: You need a harness there to help you and then you'll be happy and you'll get the results you want. That'll be the one thing. Adam Jacob: Okay. What's the best and worst moment you've had working with AI agents? John: Best moment creating a farm game. No. It was really good though. but like yeah, with AI, probably like doing art with my daughter, you know, we do some AI art coloring and we color it in, and it's like she loves drawing spaceships and stuff, and we do that at home. That would be the honest non work answer. the worst moment, yeah, I had a pretty rough one on Tuesday. Like my agent raised a PR that was completely unprecedented and had the had the authority to do so. Like it went through all the gates and stuff, but yeah, it it like they they still need watched at some level. But yeah. Adam Jacob: Yeah, for sure. John: That's me. Adam Jacob: All right. Well, that's all the news that's fit for infrastructure this week. If you want to dive deeper into any of the things we talked about, you can find all the links and the summaries over at swamp dash club.com/slash podcast. You can sign up there to get it delivered straight to your inbox every Monday or Tuesday, sometimes depending on United States holidays, John: Mm-hmm. Adam Jacob: mornings or afternoons, as the case may be. if you're trying to figure out how to manage production infrastructure and applications with AI, Swamp is here to help. Visit swampdash club dot com and and we will get you up and running and figuring out how to use AI agents to safely build repeatable systems that you can trust and verify. John, you are the special guest, so take us out. Anything you want to plug? John: SwampClub.com, you gotta check it out. It's a thing I've heard of on internet, you know? Nick: Mm-hmm. Adam Jacob: John's moving from infrastructure to sales. All right. Thank you for being with us this week. We appreciate you. We'll see you next week. Nick: Absolutely. John: See you folks. Paul Stack: See you all later, folks. Nick: See ya. Adam Jacob: you