Isaac: Welcome to Never Rewrite. I'm Isaac Askew. Dustin Rea: Dustin Ray. Jeffrey Sherman: And I'm Jeffrey Sherman. And today we're gonna give you crazy AI stories. No there's no giant overall thing, we're just here to make it today. It's just for the giggles. ⁓ so Dustin, you wanna kick us off? Let's give us a crazy AI story that's happened to you. Isaac: Mm-hmm. Mm-hmm. Dustin Rea: Yeah. ⁓ I'll kind of start off with I think one of the more common ones. ⁓ so like I have an an agency or a you know freelancing business. I work with several clients at the same time. ⁓ I have my own work ⁓ in linear as well. Of course I'm connected to Linear for different clients ⁓ as well. ⁓ there was a time, and this is I guess a case for why I think human oversight is still important, ⁓ even in in autonomous ⁓ agent workflows. ⁓ I caught my agent pulling context from my business ⁓ and including it like in a client's like context. like f to work on like a plan, like it was including things that I just talked to it about in a different project. ⁓ I don't know I can't remember exactly why in this case it had that access. ⁓ but that's what led me to the case of like it they all have to be like in isolated VNs now. ⁓ but yeah the the problem is the the leakage of data. Isaac: Mm. Jeffrey Sherman: Was that? Was what it was talking about at least relevant? Like you were solving the same problem for multiple customers and you had a Dustin Rea: Yeah, it was more or less relevant in that sense where like it was I can't remember exactly what it was, but it on like some of the frameworks that I'm working on or like my starter cater, it was something like that that I would I was working on. It was like pulling that context into the client, like to like use it already. And I was like, No, like this isn't these aren't connected things. Like these are there should be a wall between these two things. Like that's you know, and I hadn't I hadn't ran into it until I saw it happen, you know, personally. ⁓ and then I was like, Okay, this is something we gotta you know Isaac: Hmm. I haven't run into that. I haven't run into it yet, but I I feel like I'm in danger of that 'cause I've got multiple clients. Right. And then like you know everybody every company's got like a partner panel or like a support panel. Or like they're named very similar things. So if like I had two different clients and I was like logged into ⁓ you know linear locally or it could pull from Jira and I'm like, ⁓ fix the partner panel and there's like an ambiguity there and it kind of like looks to what I'm connected to. Seems like I would have to like log off and like log into a new ⁓ Mac user for each session almost to make it not pull from environment variables that might be related to a separate client or something. Dustin Rea: Right. Jeffrey Sherman: Mm-hmm. Dustin Rea: Yeah. Yeah. Yeah. I mean that's what I've been using VMs for. which again eats a lot of resources. You need even a decent machine to do that. But I think there's there's some middle ground too. I don't want to get too far into solutioning, but one thing I've seen that's effective is one, I mean obviously you have to not have your own environment variables, that's just a problem anyway. So having a separate Mac user does does help ⁓ when you're running those flows. But you can set like the MCP variables like in a specific like folder and then it only will connect to the correct like linear space. That was the trick for me, is that it needed to always when I'm working when it's working in that folder, it needs to always connect to the correct tools and only have access to those tools. That was the solution on the the tenancy leaking. ⁓ and then of course, like if you have global stuff, like you know, so we can talk about this one. I think this kind of gets into a more interesting one. ⁓ if you're logged in on your machine just as a person, whether that I think a very common one for all of us, even non-developers, is gonna be GitHub. So like imagine the permissions that your GitHub has. Like mine has like admin and ownership level permissions for several organizations. If my autonomous agent is acting as me. Isaac: Is this Dustin Rea: It has so much permission across organizations to cause havoc. So, like, one thing I had to do was make an agent have its own GitHub profile for the sake of I had to be guaranteed that it couldn't accidentally use my permissions. And so, like in the environment where my agents run, my keys are not in that environment. That's that was another one of the principles. ⁓ same thing for like your G Cloud. Yeah, sorry. Isaac: Yeah. Jeffrey Sherman: Right. This goes ⁓ well I can say this is again not a new problem, but it's exacerbated, right? Like it's if you if you're root or ad you know you're not supposed to run as root or administrator on your local box. ⁓ you know, even if it's your personal computer, you still shouldn't be root or admin level. You should have a second you should have a regular account and then an admin account. I mean, ⁓ not that anybody does, but in theory you should have that. Dustin Rea: Right. Yeah, a hundred percent. Isaac: Imagine imagine your AI is a lower level employee that has ⁓ temper problems and then like reflect on am I happy that they could delete the repo if they got mad at me for firing them? Treat it all as a security concern. And but like you said, Jeffrey, it's these are these are old problems anyway. Like the whole principles of like the the least amount of permissions needed to get your job done is like a security thing. Not even if the employee does something nefarious, but if they got compromised and somebody was able to take take over. Dustin Rea: Yeah. Jeffrey Sherman: Right, in the I have not had this AI problem myself, but I have in the past had, you know, a work computer and being remoted in in the big days before everybody gave you a laptop and committed code to the wrong repo because I was not where I thought I was. ⁓ and it's just like whoopsie. Isaac: This is just a good practice. Hm. Yeah. Dustin Rea: Mm-hmm. You ever been accidentally logged into the production database and thought you were in the dev database? I've I've heard that story. I haven't done it, but. Jeffrey Sherman: I have not, but almost every ⁓ I I I yes, almost every DBA I know has that story of dropping production, but I have not ⁓ done that myself. Dustin Rea: Yeah. And again it's the same it's the same problem, right? Like if you give yourself the keys to do some dangerous things, you may find yourself in a place you didn't mean to be. ⁓ and I think the same is you should have even of a stricter view of of of agents. And and I think it's not it's not just about even not trusting the LLMs themselves. But I think if you have if you watch any amount of security news, you see all of these ⁓ supply chain attacks, right? So like if your agent is pulling down NPM packages, and I've seen it's very common for especially Vibecoders to have ⁓ unpinned ⁓ Symvar in their package.json. So they're just pulling in like the latest package, like every time they install, which is every time they do anything on CI or do a build, they're pulling in that latest potentially compromised package because that's when the supply chain attacks happen. The package gets, you know. Isaac: Yeah. Jeffrey Sherman: Mm-hmm. Dustin Rea: released and then people download it through these autonomous workflows, even even if it's not an agent, like you just running NPM install, it's gonna put that compromise package right before it's patched. So if your agent is running against that and now you get injected into that prompt, it's gonna start X filling your data off of your machine. And if you don't have anything to catch that or you're not in a contained environment, that could be your GitHub keys, that could be anything on your keychain. Jeffrey Sherman: Right, there's a window before it gets noticed and revered. Dustin Rea: That could be whatever's stored in your passwords, like all of this data could be essentially just taken from your machine. ⁓ not to mention whatever else it could get the agent to do on your machine, especially if you're not savvy to like what's happening, you know. Isaac: Mm-hmm. Oof. You give me fears, new fears. But we didn't. Jeffrey Sherman: Yeah, this is not funny, Dustin. These were supposed to be funny. Dustin Rea: That's why I told you I started doing VMs. Hey, I saw I told you that's why I started doing VMs. You know, you gotta have some level of of containment from these things, right? 'Cause there's imagine the same that we're all now vibe coding all of these solutions and and working at breakneck speeds, you don't think the attackers are doing the same thing? Jeffrey Sherman: Yes. Isaac: Yeah. Yeah, this is kind of w you know, the idea of plugins getting out compromised is one thing, but plugins of plugins too. Like when you have nested dependencies and you were good about, you know, taking care of minimizing the plugins that you needed and making sure they were upgraded, but the plugins you installed had dependencies and those people didn't. And so now you're still pulling in, you know, nested dependencies that have the issue. And this has happened I forget which plugin it was, but this happened like, you know, two months ago or so. Wherever I like, okay, everyone we have to archive a repo, make a new one 'cause it could have been hook compromised, but we're not sure. Rotate all of our environment variables. Yeah. Dustin Rea: Yeah. Yeah, I've seen it across multiple repos in the last few months. ⁓ and it it's the same thing. It's it scares me too, right? I'm I'm working and I'm responsible for client workloads, so I feel responsible to Isaac: ⁓ Dustin Rea: take care of those workloads. But I again I think as long as you apply these principles, you can can t you can protect yourself as much as you can. It may sound scary, but I don't think that it means not using these things, right? Because you're still you're still exposed. Like it's not like AI itself is what's exposing you to these attacks. Like you're exposed anyway. Your developers could expose you. ⁓ I've been exposed to several hacks due to contractors that just didn't mean to, you know, so it can definitely happen. Isaac: Yeah. Yeah. Yeah. Jeffrey Sherman: I mean, there's a reason that the t that the thing has a name. It's it's 'cause it's not new. Supply chain attack. Right? It's Dustin Rea: Yeah, yeah, exactly. Isaac: Right. Dustin Rea: Yeah, exactly. So back to our I guess back to our topic. I think tenant isolation, I think we talked we kind of covered that one really well. connectors, I think, is kind of the same thing. Like be wary of the connectors that you have, whether it's your Figma, your linear, you know, it's very, very easy to accidentally pull in. And I know a lot of people have multiple Figmas that they have access to, not just the single client or the single work space that they're attached to. Like it's a multi-workspace account. So when you give your agent MCP access to that Figma and to your Isaac: Yeah. Dustin Rea: your point, Isaac, you say go fix the portal. Man, who knows what portal it's looking at. You know, you need some type of verification step in there or some type of isolation. That's what's drawn me to giving them their own accounts. So I can control that access. Like you really only have read only on this one workspace. And then I I know for sure it's them. And you can use like the email trick too. So you're just using like one email plus client name or plus tool or you know and that that makes like a systemized ⁓ isolation. Jeffrey Sherman: He Isaac: Yep. Yeah. Jeffrey Sherman: Mm. Isaac: Yeah. Jeffrey Sherman: Mm. Isaac: That's a a good example, actually. I I talked about it in a prior episode, but I've run into that problem specifically. And it did kill production. And unfortunately it was a side project that was like a hobby thing for me. And like two people saw it go down, you know. But it was a good thing to run into early on, so I don't cause that, you know, in a much, you know, worse case. But essentially I had ⁓ Vercell connectors set. ⁓ on two different computers and I had moved a pro I was trying to push up all the work that I'd done locally. ⁓ there was some stuff that was kinda like because I was tinkering around on my local machine, I had some environment variables set on my local machine and some other stuff and processes that were running on that computer and I was trying to make it just all contained so I could clone it down on a separate computer and it could still continue working. And so I cloned it down on the separate computer and then told it to, you know, boot itself up and run migrations and set things up. And On my old flow, I had the environment variable, the local one already populated with values, and on my new machine it didn't. And so it looked at him as like, ⁓ I could see it thinking through. And he's like, I've got, you know, no local variable set. ⁓ I see that this is a Vercel project. ⁓ I see I'm connected. ⁓ I see I can pull these values from production. And so it actually pulled down the value for the database, ran migrations, and I had a pinned version live. So Jeffrey Sherman: Mm. Isaac: the version I pulled down from Maine was not the same as the one that was live. It ran migrations and then columns were not there that needed to be there for Maine, which had not been released, this different version. So production goes down 'cause there's a schema mismatch. Well, I didn't roll back. I just pushed through and fixed it 'cause it was a side project. Yeah, fell forward. Because I mean but yeah, I could see if this happened in the professional world and I did that and brought down pride, that would be a huge issue. ⁓ Dustin Rea: Roll then you have to roll back. ⁓ you fell forward. Yeah. Yeah. Right. Yeah, you'd have an incident report to to fill out for that. Yeah. Isaac: Yeah, and a lot of ag on my face. So, you know, I'm I was glad I was glad I ran into that the first week of agentic programming. And I'm like, Ooh, okay, I really need to understand the power of you know, th i it will try and figure things out and even if you didn't tell it it could do some crazy stuff. Or at least then, you know. Jeffrey Sherman: Mm. Dustin Rea: Yeah, I've got a I've I've got a one that's on the environment variable one that I've seen that actually shocked me and I I was like, Wow, I can't believe you just did that. Like I I even I didn't think you would do that. ⁓ but it I don't remember what I was having it do one time, but it it for some reason it read the dot en v and then put the entire dot en v into the chat. And then like literally the next message it was like, ⁓ oops, I just exposed all of your keys. You'll need to roll all of those and I was like, Thanks. I'm Jeffrey Sherman: Yeah. That's a funny one. Yeah. Dustin Rea: I'm so glad that I got to do that right now. That's what I needed in the middle of this project was to just stop everything and roll the keys across the board. Right. Like that's exactly what I wanted to do right now. Jeffrey Sherman: Mm-hmm. Dustin Rea: ⁓ and it to be fair, like right, so to give you like context, like this is a platform that has like a legacy platform, a new platform that's out, they have shared environment between them. It's kind of like in a transition period. So that's why I have those credentials locally. ⁓ and it's just like, man, I can't believe you just did that. Like I that's just cost me so much work. ⁓ so rolling keys due to exposure, that's ⁓ I gotta imagine vibe coders and like everybody is experiencing being exposed to that, ⁓ which is dangerous. ⁓ Isaac: Mm. Right. Jeffrey Sherman: Mm-hmm. Dustin Rea: ⁓ then the other thing I've seen that worries me a lot, this happened in G Cloud, but I I could see the exact same thing happening in Azure or AWS, which is ⁓ as the developer, you may have and may not even realize this is what happened to me, may not realize that you're logged into the CLI of a specific tool. It could be even Stripe that you don't realize you're logged into. and the agent has access to that if you're not logged, you know, if it's not in a VM, it has your your accesses. ⁓ so what this one did is it went in and Isaac: Yes. Jeffrey Sherman: Mm. Dustin Rea: and started looking at in you know production infrastructure to like answer my question. Like what I was actually asking it was to like go look at the docs, like read the documentation on how we built the infrastructure and remind me, you know, what this how this part works. Isaac: Mm-hmm. Dustin Rea: And it's like, well, I'm just gonna go check the actual infrastructure. Like, you know, just scanning this account. And I'm like, wow, this is it's got full ride access. This is crazy. ⁓ so I stopped that process. And what it made me think about is like, imagine if you were giving it a problem and it just started spinning up infrastructure for it. Like you could essentially have it build out, especially if you don't know what you're doing to our last episode. ⁓ you know, it could spin up a bunch of scaling infrastructure. Jeffrey Sherman: Oof. Dustin Rea: For a platform with zero users, I actually saw something like this recently, not this bad, but a smaller case of this. ⁓ so you've got all of this scaling infrastructure and you have no users. And like the biggest the platform could ever be is a single box anyway. So you've got multi-scale, you know, multi-box load balancer, scaling infrastructure in dev to test it to make sure it works, then duplicated in prod. Isaac: Mm-hmm. Dustin Rea: So you're essentially running anywhere from eight to sixteen small boxes to do something you could do on one box in dev and one box in prod, right? Because the AI recommended the best practice for scaling infrastructure. Right. Isaac: Yep. Or or you did, maybe. Maybe you said to it, Hey, ⁓ there's an issue where this thing keeps running out of resources. I want you to fix it using best practices and you're really vague and it goes and spins up all that for you. Cause that that was the Vercell thing for me being logged in locally. 'Cause that if you're logged in, like even AWS, that's the scary one too, where you if you log in via the client, it saves those credentials locally. And sometimes you you'll the login if you're smart, you've set it to expire in a short amount of short window and will have to keep reva re Jeffrey Sherman: Right, to internet scale. Yeah. Dustin Rea: Right. Right. Right. And you just needed to like boost the box up, you know. Yeah. Mm-hmm. Isaac: validating every thirty minutes. But the damage it could do in thirty minutes if it saw those credentials and you put them in an area that was already accessible to it. ⁓ to spin up, that would be very scary to me. So again, thanks for all these new fears. Dustin Rea: Yeah. Significant. Yeah. You're welcome. Yeah. Jeffrey Sherman: Yeah. Well, but they're not new like ⁓ you reminded me ⁓ of a news article I read years and years ago where somebody had hired a company to move their car, right? ⁓ and they didn't take down their transponder. And like they paid a flat rate of, you know, come pick up this car and move it here and deliver it by such and such a date. And this car carrier went all over the country picking up cars and dropping them off and Isaac: Mm. Jeffrey Sherman: This guy got a four hundred dollar ⁓ transponder bill because his car just, you know, was going here, there, everywhere. And the car carrier was like, Yeah, you know, we we put the car on there. We charge you, we charge you a rate. We didn't say we'd go direct. We just said we'd pick it up and we'd drop it off by this date. And that's what they did. But you know, they the car it went here, there, and everywhere. And this transponder is just like racking it up because and it wasn't even for anything, because it was the the car carrier was paying the toll. Isaac: Yeah. That's wild. Yeah. Jeffrey Sherman: Right? It's things but it's to say like it's not a new problem, it's just making existing problems worse. Hey, you if y your car's inside another car and it can get transponder. Isaac: That's kind of the fear, I guess, is like imagine you on your worst day, like the worst thing you've done. And imagine be able to being able to do that fifty times faster. That's that's scary. Limits, we gotta put in those limits. Yeah. Mm-hmm. All the security people were like, hey hey hey. I told them. I told them. Jeffrey Sherman: Mm-hmm. Dustin Rea: Zero trust is the buzzword, right? Yeah. Jeffrey Sherman: I told him, man. It's all it's all we gotta Anyway. Alright, any more good good AI stories or should we put a pin in it? Dustin Rea: Yeah, ⁓ yeah. Yeah. Yeah. Isaac: Well, ⁓ for the sake of getting this one out there, 'cause ⁓ I said it again ⁓ in a prior episode, but it's worth saying for anybody worried now at this point about other things it can do. ⁓ but I had a case where ⁓ I forget what Google In ⁓ enterprise API I was using, but it was it was something to do with pooling like ⁓ ⁓ a location, like a like a venue and data about the venue, like the contact details and whatnot. ⁓ if you pull in basic information about it, you get billed at one like cheap rate. But if you pull in some other details, like I think it was the ⁓ like the image, like if somebody uploaded an image or any reviews about any reviews somebody left for the business, some of them are billed at a different rate and they're considered part of the enterprise API for that. Jeffrey Sherman: Hmm. Isaac: ⁓ and I discussed with ⁓ AI which one I wanted to use, and it told me what the rate would be. And I was like, ⁓ okay, this is like forty dollars to pull all the data that I need. But it actually included one of those columns, not realizing it was part of the enterprise ⁓ API. And all Google does is just bill you for the API, the enterprise rate, if you include that in your query. That's really all it is. ⁓ and so I suddenly I had this $200 bill on Google Cloud. Jeffrey Sherman: Mm-hmm. Isaac: Unfortunately it was on one of those accounts where I hadn't attached like a card yet. And it's just like they they they ⁓ granting you like two hundred dollars to get you using their API. Yeah. And so it was just like exhausted immediately, ⁓ from a loop it ran. And I was like, ⁓ whoa, what? And then I asked it, why did you do that? And it's like, ⁓ oops, sorry, ⁓ didn't realize that was part of the the enterprise one. But I just imagined if somebody had their, you know, their debit card on file and I went to bed and woke up with a ten grand bill, you know, that I could see that happening. So I'm Jeffrey Sherman: Mm-hmm. Trial account. Ha ha ha. Isaac: That's a that's again, I I accidentally had a limit, right? I accidentally had a two hundred dollar one. But I have also seen it try like I I got a random bill one day. It was like a two hundred dollar bill for Claude. And I'm like, Huh? And I went back and thought that you I'd already paid that monthly fee. I I'd do the twenty X plan or whatever, so I'd already paid that fee. But when I looked it was just an auto-renew. And it was like doing a dark factory pattern trying to Jeffrey Sherman: I'm sure that happens all the time. Isaac: you know, keep use keep building things in the background and I had it like auto renew and I'm like I ⁓ went in there immediately and just turned it off. And it was doing what I needed to do and I would have to pay for it anyway, so it was fine. But I thought, ⁓ I wanna make sure I don't let it think, ⁓ this is the best way I should build it. But it's gonna be expensive, but that's what the user wants. And then just starts like I just suddenly see like fifty separate two hundred dollar auto reload charges when I come home one day or back from vacation or something. So another limit, another security limit we should place on ourselves. Turn off auto reload. Dustin Rea: Right. Jeffrey Sherman: Yeah. Mm-hmm. Well and and again, this is ⁓ not a new thing of I have seen developers crush computers because they didn't put a limit on a SQL query and they pulled in sixteen million rows. And it's like, Well, did you you know, did you need sixteen million rows? May maybe you should put a limit there of even if you don't think you know, put a put a limit of ten thousand even if you don't think you need it. Isaac: Mm. Yeah. Literal limit, yeah, yeah, in that case. Dustin Rea: Right. Yeah. I imagine the amount of runaway cues that have been developed in the last six to twelve months, you know, there's gotta be several stories, you know. Isaac: ⁓ God. In the last two weeks. Agentic programming for runaway runaway cues. ⁓ god. Jeffrey Sherman: Yeah. Dustin Rea: That's a bill you don't want to wake up to. Imagine that plus you also have something that hits tokens, like an API key. On a c in a c and then in a queue. I mean, you could really rack up serious bills doing nothing. And it could just be a runaway infinite loop process of processing nothing over and over again. And then you add auto scaling to that, you can really rack it up. You know. Jeffrey Sherman: No. Isaac: Mm-hmm. I'm gonna be telling all my clients, do not give me access. If you do give me access, give me read-only. I don't wanna spin up infra. Yeah. Dustin Rea: Read only. Yeah, no billing access. Jeffrey Sherman: Well again that's not a new thing. Like always give the least amount of permissions necessary. Dustin Rea: Yeah. Isaac: No, but but the the amount of damage you can do that fast is a brand new thing. Absolutely. Jeffrey Sherman: Yes. Isaac: All right. Jeffrey Sherman: Well, thank all for listening. I'm Jeffrey Sherman. Dustin Rea: Dustin Ray? Isaac: And I'm Isaac Askew, and this is Never Rewrite.