Manav Gupta: Hello there, welcome to ShipAI. I know it's been a while since since I since I started talking about AI. This time I have a special guest, and I think somewhat of a unique individual, because I actually spent a bunch of time with his dad when his dad was at IBM. So please welcome Sunit Vadhya, who's the co-founder and CEO of Crafting, who spent a bunch of years at Mera and Uber and Discord. And now he's working on a on removing friction in AI first engineering organizations working at scale. So Samit, welcome to the show.

Sumeet Vaidya: Thanks so much for having me, Manov. Really appreciate it.

Manav Gupta: Thank you. Okay, so I know there is a lot to unpack with what's going on around AI-driven software developer lifecycle, building of test harnesses and so on and so forth. So we'll get into all of that, but why don't you tell us a little bit about your journey, tell us who you are, what are you building, right? Before we get into AI and how it fits into Enterprise Site.

Sumeet Vaidya: Yeah. So quick background on me. I've been an engineer all the way to a director at, as you mentioned, some great companies and had the opportunity to see a variety of companies that invested in developer tooling and infrastructure. And the majority of my career I was on the product engineering side at Facebook and Uber and Discord. when I was leaving Discord, one of the biggest frustrations I had is that The investment in engineering tooling and the developer experience has just been very inconsistent across the industry. And I've seen how great it can be at organizations that invest and the challenges at organizations that don't do a good job of that. My co-founder, whom I met, he actually was at VMware for over 10 years, spent time at Google and Microsoft working on cloud native infrastructure. And we asked ourselves, what would it take to build best in class tooling that the Googles and Facebooks of the world have and make it accessible to every organization? Specifically every enterprise organization. And something interesting happened over the past year where we saw what was happening with AI agents and their capabilities and realized that AI agents would eventually need the same infrastructure, tooling, capabilities, and access that we've given engineers over the decades in the industry. So we repurpose our company and have built a platform that powers essentially the overall developer and AI software engineering experience.

Manav Gupta: Got that's pretty awesome. Why don't you tell up tell me a little bit about tell tell me about your take on, you know, clot code versus codecs or cursor versus windsurf or whatever else.

Sumeet Vaidya: Yeah. So I it was interesting. I was actually a very early user of Windsurf, played around with Codex, and I remember this must have been three years ago, maybe a little bit more. all these tools were taking off, right? And it was really fascinating. I think everybody had their own approach on what should happen at each cycle, but then they would converge on something, right? I think Windsurf was actually the first one to talk about plan or spec driven development. And then everybody else started saying the exact same thing, and everybody's okay, now we need to use specs. And this was a year or two ago. same thing happened with are you using the IDE or the CLI? Everybody moved more towards the CLI, then they're adding better integrations within IDEs. And so to me, I think there's really two questions, right? One is what is the best set of tools I can use today? And even now we're seeing Codex is really great at execution and doing more backend tasks. Claude Code has better UI instincts, if you want to call it that, when building web front end apps. But that's a very tactical question of what should one of my engineers use for a specific task. I think that's a very different question from what will this look like in three to six months, or what should we really invest in? And I think if a year ago you went around to people and said, hey, we need to invest in call it a harness, call it AI infra or whatever. Every organization would say, that's completely insane. What are you talking about? And I know that because we did it. We we went to a bunch of companies saying, hey, we built this infrastructure. And they said, no, no, no. I'm I'm happy just using cursor IDE right now and essentially having better semantic analysis and autocomplete. It wasn't until what was it, Opus four six or whatever dropped at the end of last year when everyone said, I need to have dev environments and I need to crack my laptop open when I go to the gym so that the the loop doesn't die. And now everybody's talking about actually having loops. Ralph loops were a thing six months ago when everybody said that seems a little crazy. And now everyone says, okay, I need to figure out how to have this perpetually running agent. So to me, I think the debate between Claude versus cursor versus composer and all these other models, it's it's kind of a distraction, to be honest, depending on where you're at. If you are an IC engineer executing and just building, absolutely dig in and figure out what tools are best for you to use day to day. But if you're an engineering leader thinking about where do I invest my organization's time and money and where do I build infrastructure, I think that's the wrong question to ask in terms of models and providers and more around how do I build resilience and make sure that my organization can adapt as the next wave comes? And I have many opinions on what those waves might look like over the next six, nine months.

Manav Gupta: Okay. You know that we're gonna get into all of them. but you y you you said a number of interesting things in there and clearly clearly you're in the no because I think you managed to cram in pretty much everything that's going on, right? Loop engineering,

Sumeet Vaidya: Yeah.

Manav Gupta: harness harness development, harness engineering. But again, let let's start from the very beginning. So so I like your take that the discussion around which harness, which agentic harness for software engineering to use is almost a distraction. I think is is what you call it. So do you envision that most enterprises are going to provide are going to provide choices to their developers?

Sumeet Vaidya: I think so. I think that things will continue to get commoditized. I I know a few different companies have gotten traction recently talking about how they've saved a ton of money by switching over to DeepSeek, right? But even if you look at companies today, they say please don't use Opus when you're just testing something out. Like you don't need a Ferrari to go pick up the groceries, right? And I think that we will continue to see specialization of models and some preferences people have. And the reality is In a year or two, if all of these subscriptions are relatively the same and relatively affordable, why do you really care unless you're signing a massive deal with one of these providers? But even then, I think that enterprise engineering leaders likely want to avoid that. Because if this space is evolving so quickly, if there are going to be new tools that come out, you don't want to be locked into a certain way of doing things and then risk your team falling behind, or candidly, you getting pressure from the CEO or the board saying, hey, why are we falling behind? Why aren't we doing the things that everybody else is doing? Which has become a real challenge for a lot of engineering leaders as well. So I do believe that organizations will want to build resilience. So, you know, if Claude goes down, your entire engineering organization doesn't stop working, right? You need to have some fallback. We did the same thing with cloud providers. If you're on AWS and it fails over, you ideally you have GCP or Azure or somebody else as a backup. I think we'll see the same thing with AI providers too.

Manav Gupta: So you mentioned resilience a couple of times, so so let's let's get into that. I mean, clearly there's one case that if cloud was to go around, or the preferred DLC partner tool was to go around, with what happened with Fable 5 as an example. okay, so that's one impact. So so to your point, giving the develop developers an option or a choice is one way of looking at it. how do you define resilience then in this modern world?

Sumeet Vaidya: Yeah, I think it's a good question. And the reality is that every term these days is overloaded. Even the word harness that you used earlier, right, can mean 20 different things. So when I talk about resilience, it is really those two things. One is how do you make sure your business doesn't collapse today? I think that's the base definition of resilience. And that is in case of an outage, how do you make sure you're good? Right. Let's even talk about user-facing stuff. Right. If you're relying on Sierra or Decagon or somebody else for your customer support and you have no backup and it goes down, all of a sudden you have no customer support for your whole business. That is terrifying to any business leader. I think the same is true for engineering tooling and engineering organizations thinking through, well, if cloud code is down, and in five years, if most of my engineers don't know how to manually write a line of code or read it or test it, I need to have some kind of backup. Right, whether that is codecs, whether that is Composer 2.5, whether that's something else. So it's the same lesson we've learned in the past, whether it's with cloud providers or other infrastructure, or even login and auth providers, right? You want to make sure there's no single point of failure for your organization, and we just need to treat AI infrastructure in a similar manner. The

Manav Gupta: Yeah.

Sumeet Vaidya: second question I think is the more interesting one around what does the future of engineering look like? I'm I'm still someone who twiddles with his NeoVim configs and likes to hop in various files here and there, even though I know it's not the most efficient, and I think it'll be viewed as an archaic, more craftsman-like way of doing things.

Manav Gupta: Yeah.

Sumeet Vaidya: but you know, the models are very convenient, and I think in the midterm, we will rapidly see people touching the code less and less. And if we go to base principles, why is it that we want engineers to be able to look at the code, to touch it, to reason with it? And it's because if something were to go wrong, there is a deterministic way to evaluate the issue and resolve it. And I think what scares a lot of us is that models are non-deterministic, probabilistic things that are in charge of our infrastructure, our product, our APIs, and everything. So I think there's actually a different way to look at it around how do you make sure that models are correct? Right. And I think a lot of people are starting to talk about this validation problem. Where even if you look at harnesses, right, the average harness is a bunch of markdown files. There's no actual contract for what the agent can or cannot do. There's no true guardrails. And if you're talking about an enterprise organization, how do you deal with off credentials, dependencies, all of that? And so to me, I think the biggest thing is how do we essentially help agents act the same way engineers acted? And I think we can take this a step further, right? If you had an engineer a few years ago who sent out a PR for a major feature launch, and you said, Hey, how do you know it works? And they said, Well, I ran some unit tests and I looked at the API calls from other services, and it looks like the contract won't be broken. You say, What are you talking about? Like, go back, do a playtest with your team, make sure this stuff works, see it, feel it, right? We should have the same expectations for agents. The reason we don't is because it's really hard. That's the only reason. If we could snap our fingers and have that, we would do it. So I think the future is one where people actually are able to have software development agents work like engineers. And the engineers orchestrating them hold that same bar that we hold. Hey, agent, go actually validate that this thing works using this staging cluster or this external dependency or this third party off services staging tier. Do a run of it, record it, validate it, show me. Right. There's some proof that's there. And in that case, we move to a world that is less about do the tests pass and does the code look good? And more around outcome driven development. And I think that's what the future will look like.

Manav Gupta: Yeah, but but but surely the outcome is defined by or the the accu accuracy of the outcome is defined by the by the test and the completeness of the

Sumeet Vaidya: Absolutely.

Manav Gupta: test suite.

Sumeet Vaidya: Absolutely. And I think that today the way we define tests is not really ideal. It's either unit tests or brittle and very, very slow CI C D. Those two things were not designed to scale if you're running swarms of thousands of agents. And I think that that is the trip.

Manav Gupta: Yeah. I I I know I know GitHub is in the middle of trying to reinvent what GitHub is gonna look like, right? Because this current current swim lane model of or really a linear model, actually not in linear, sequential model of create a PR and have a human review of PR. It breaks down in the world of these swarm of agents. But let's get let's get get back into the determinism. So If I have an engineering agent that I'm validating, I mean really it's a it's a new contract I'm writing that clearly articulates what the outcome should be. Right. So how how does one write that? Is that just additional test cases? Is it just more test cases? Because you said something interesting that I've personally noticed as well, which I find people are writing a lot more test cases to your point to have the verifiability. Generally those tests end up being unit tests to your point. Or people are now having these elaborate CI and C D pipelines as well. But I think there's something like that.

Sumeet Vaidya: I agree. I think there's two parts here, right? In an ideal world, your test is as close to what's happening in production as possible. That is true regardless of agents, engineers, state of the industry. I think that's just a fact. You want to make sure that things don't break in production, so you want to mimic those sit those conditions as much as possible. And so what should happen is agents are being given tests that are close to production. So new user onboarding. Right. Don't just test out the front end. Make sure that you actually create that row in your USDB. Make sure the auth flow goes through. Let's say it's for a marketplace of some kind. Run the actual background check. Make sure you don't break that API call. So that could look like CI CD, but the reality is CI CD is just here is a battery of tests. I'm gonna guess what you've changed and just run the full suite, as opposed to unit tests, where it's like, okay, I've made these changes, just test this local thing. Really, it should be a combination of the two of I've made changes in these services, so I should test these golden workflows across the stack. Now, the reality is there has been a bit of a gap between the haves and have nots. And this goes back to what we saw from the developer experience side. The Googles and Facebooks and Ubers of the world have invested a ton of time and money and energy into tools that allow thousands of engineers to build in parallel. Agents need the same thing. And so what I see is an evolution of CI C D to these full stack tests, not even really tests, but actual workflows that agents are running through the entire stack, and doing so in parallel. And I think this is a very different problem if you're a 10 or 20 person startup versus a true enterprise organization. A true enterprise, you need to share all these resources. You're not going to spin up a full D B and a full queue for every single agent that's running.

Manav Gupta: Fair. I th I think that's fair. But interestingly you didn't mention integration tests. And what I'm noticing in a lot of companies is again back to my point, we can I think I I'm seeing engineers fooling themselves with very high unit test coverage. And things

Sumeet Vaidya: I agree.

Manav Gupta: are still breaking. And what I'm finding the weakness and the harness is more around integration test, especially between These multi hundred swarms of agents.

Sumeet Vaidya: So this is, I think this is something I believe. I I think that if you have the right infrastructure in place, there's no distinction between CI CD, a unit test, or an integration test. A test simply just makes sure that something works. And if that something spans five, 10, 100 different services, it can go do that. So in this instance, the thing that is lacking is not. what the ideal workflow looks like for these agents. It is how do you have infrastructure that allows you to do so? And I think that's a fundamentally different question that candidly, most of the industry is sidestepping because it's hard to do.

Manav Gupta: Yeah, that that I really with. Okay, so let's spend a minute on spec driven development just because you brought it up. And I think I think for a brief period of time, I was enamored with STD as well, although I've now gone back to however however I was prompting brief prior to STD because my conclusion with SDD is you're just prompting hardware. You're taking the can down the road. Right? You now have this elaborate document. Which is the spec. But there is no contract with the development agent, the engineering agent, that it's gonna have the fidelity over this much larger prompt and all the things that that that it that you that the spec's asking you to do. Do you agree with that?

Sumeet Vaidya: Kind of. I I actually think spec driven development serves a different purpose.

Manav Gupta: Yeah.

Sumeet Vaidya: I think it serves the same purpose that Architecture Docs did previously, where if you're doing something complicated and you just try to YOLO code it, you're going to get things wrong. You're going to forget edge cases. And it's very easy to get caught up in getting the thing done as opposed to building it the right way. And I think spec-driven development actually forces the orchestrator, the engineer, whatever you want to call us these days, to really refine what it is we're asking the agent to do. I do agree with you that we're not putting hard guardrails on the agent itself, right? Like you can still do things wrong and go off the side.

Manav Gupta: That that's my that's my point. You can you know you you can have a beautiful speck which

Sumeet Vaidya: Yeah.

Manav Gupta: passes human scrutiny and things still fall apart when the agent or a swarm of agent delivers it because of the d of the lack of determinism.

Sumeet Vaidya: I I think the value here though is I agree. I agree with that. But I do think that when things do go wrong, you can then still point back to that spec saying, Hey, this is busted. You broke this invariant. Go through and fix it. Right. It's the same contract we had on teams, but I it's overhyped.

Manav Gupta: Yeah, but but but but but i true, but then but then the but then you get the dreaded yes, you're absolutely right from Claude Code. Right?

Sumeet Vaidya: Yeah, look, I agree with you. It is not perfect. I don't think it's a silver bullet. I do think it is. Look, I mean, it's not even the hype anymore, right? Everyone's focused on loops right now, too. and so I think

Manav Gupta: That's right. Okay. Fair. Okay.

Sumeet Vaidya: this this is what I think happens with every hype cycle. People find the latest thing, they latch onto it, they assume it solves all of things. When in reality it solves some stuff really, really well and has some drawbacks. And I think the lack of nuance and discourse is causing us to say, like, well. Spec-driven development isn't all that. We're going to go with loops now. In six months, we're going to say, well, loops still have these issues. We need infrastructure access, right? It's going to keep happening. But

Manav Gupta: Yeah.

Sumeet Vaidya: I think at the end of the day, there is some value, but we need to acknowledge every single iteration of these tools is not perfect.

Manav Gupta: Okay, I think that that that I can agree with. Okay, so let's talk about development of these harnesses and loop engineering or I think we're into the third iteration of this, right? So there was the software engineering harness, then the harness engineering itself, then the loop engineering. What's

Sumeet Vaidya: Mm-hmm.

Manav Gupta: your take on this?

Sumeet Vaidya: You want my hot take on this?

Manav Gupta: Yes, of course.

Sumeet Vaidya: I think that sure there's value here. It's easy to make better progress and there's a better way of doing things. But if you look at loop engineering, it's always prompt the agent, agent write stuff, check if it works, if not loop. The problem is the check if it works part is doing so much heavy lifting and it's being completely glossed over. Your markdown files aren't going to make sure that your agent is thoroughly testing everything and making sure it works. We talked about this earlier. They're just probabilistic models, right? And I think selfishly here, this is why we're doing what we're doing at crafting. I think the reality is they need real access to infrastructures. They can do what you would expect Ninja to do to check if something works. And this goes back to my point earlier, where I think the industry is sidestepping the hard question because candidly, it's easy to build hype around, hey, we've improved how this model works, which is true, but it's not deterministic anymore. And you see. People chasing all these trends, there's tons of hype, whether it's raising money or speaking at a conference around how you're the AI leading infrastructure organization or whatever. But the reality is, I think the companies that will be most successful are the ones that take a more sober approach in the next year or two to invest in this infrastructure, make it accessible to agents. And that's how you actually scale, as opposed to band-aid on top of band-aid on top of band-aid. Like I'll be very honest with you, right? With all of these loops. We'll say we fixed the validation bottleneck, even though we really haven't. We've just reduced probability of errors getting through to production. In three to six months, we're gonna have way more code being shipped to production, way more regressions. Even if you cut regressions in half, but you 10x the amount of code, you got five X more outages, right? The next iteration, I think, is gonna be we need production agents that are proactively monitoring production, detecting errors and then resolving them. And I think there's already some companies trying to solve this. But again, that's yet another band-aid. Because they're going to do some semantic analysis, say, I think this is the root cause, let's fix it, and they'll they'll paper over it. And that's fine. But all of a sudden you have loops over here, you have a glut of unit tests coming through, your CICD pipeline is moving at breakneck speed with too many PRs coming through, with low confidence, and you're mitigating things instead of addressing them proactively. Every single step of this process would have been resolved if you invested in infrastructure for your agents. And by infrastructure, I mean Real tooling and access and dependencies and credentials, not a bunch of markdown files that we call them a hardness.

Manav Gupta: Okay, I think that that I would agree with. Okay, so let's talk about the tooling for the for the agents and you know, what to give it and so on. So So what happens when these agents are having to act on behalf of a of a user and then there is all this hot debate going on around human versus non human identity. Is is do you think that's all now?

Sumeet Vaidya: I mean, I think it it's not actually that big of a deal, to be honest, right? If you have an organization where like this is actually a people problem. Who is accountable and culpable when something bad happens? And if you say that whoever kicks off an agency workflow is responsible, great, you can inject credentials as needed. We we do this. We use Vault and we can replicate credentials and various agent sandboxes that are spun up. If instead you have an organization where PMs and designers can ship changes, that's kind of scary. To be honest, because they may break some front end not realizing that there's a state machine they've broken on the back end that's dependent on it. In that instance, obviously they're not going to be on call when things break. So the contract you actually need is which engineers are going to wake up at 2 a.m. when some onboarding flow for a car rental company is busted and Ladam, right? I think when that happens, it's more of a question of. The agents that are developing the code need certain credentials associated with the non engineers. The agents that are validating it should probably have similar credentials to the engineers responsible. They

Manav Gupta: Yeah.

Sumeet Vaidya: have the same access to make sure that things aren't completely busted. And if they are, check what's going on. But that honestly feels like more of an organizational question of, you know, single throat choke for any given issue. From

Manav Gupta: Mm.

Sumeet Vaidya: a technical standpoint, we can do it. And we've actually seen companies implement different models here, but it depends on their risk tolerance, who they're giving access to these agents to, what they're actually shipping. And so I don't think there's a one size fits all answer here. But I do think it is a solved problem in that we can build it for any company. But whether every company uses the same model, I don't think that's going to be the case.

Manav Gupta: Yeah, and I think you're you're also touching upon something really interesting that for these agents to work reliably during development time, you gotta provide I don't use the term infrastructure, which was a I think all encompassing infrastructure, which is a production like environment with real dependencies. And I know it's been the holy grail in many organizations. So just walk walk me through how do you deliver a production like environment without exposing P I I, P H I, et cetera. Like tell me tell me how you guys trying to solve this with crafting or crafting dot jet.

Sumeet Vaidya: Yeah. So with crafting, we actually do custom deployments in our customers' clouds. So no data ever leaves. We don't even have access and they can actually run all the software themselves they want to, but typically they ask us to take that load off their plate. That's a big part of the value here. now the big thing is we've had staging tiers for years, right? And so on the engineering side, everybody would argue over who can use a staging tier, which checkout are you gonna have for any given service and be a nightmare. The Facebooks and Googles and Ubers of the world have all built all of this, right? There's ways to share infrastructure with intelligent network interception by injecting headers and all that stuff. That's great. Solved problem, really hard to get with. The challenge is creating that in a way that is cloud agnostic. And so there's no secret here, right? We spent a long time integrating with the biggest cloud providers in the world, making sure that we can run on any cloud stack. And we work with our customers to say, hey, what does your stack look like? Let's get these things up and running and do a true partnership, as opposed to kind of a more easy PLG approach of hey, run this script and you're good to go. And the reality is if you're running one of the biggest banks in the world, You don't want that. You want to make sure that somebody is around if you're trusting them with your infrastructure. So we work with our partners, they share what their network topology looks like. We get things up and running in like we say two weeks or less, but typically is a couple days, to be honest. But that sounds too good

Manav Gupta: Mm. Right.

Sumeet Vaidya: to be true. So I'd rather underpromise and overdeliver. And we just partner with them. We say, What are the golden workflows you want up and running? Is it the validation layer? Is it giving coding agents access to All these databases, right? To your question around PII, that's on the companies where they say, Hey, we're willing to just replicate this DB, or if there is PII, sanitize it, drop it in. We support database snapshots, and now every single sandbox that needs it is up and running as part of our template. So it really is we built what we believe is a best in class developer experience when you're dealing with hundreds of different services and shared infrastructure. And just repurposed it to make it accessible for agents. And I think that is a pattern we're going to see across the industry where we've solved so many things for engineers. Why are we reinventing the wheel for agents as opposed to spending the time to change what access looks like to these products in this infrastructure?

Manav Gupta: Okay. And then so give give me the blossom radius story. So the typical problem that I'm running into that I hear from most companies is an agent made a made a change, the change paused validation in their environment, but still wrong and prompt. Right? We've we've never seen the problem happen ever. So just walk me through what's the containment model look like? How how are you guys approaching it and thinking about it?

Sumeet Vaidya: So the way we're thinking about it is usually when things have broken, it's because they haven't actually tested in a true production-like state. You might call it a production-like environment, but typically it's just in a micro VM, right? And you're running a few different services there. If you are Amazon and you are running AWS, you're not going to run the entire AWS stack for a single agent. It's just not gonna happen. The number of services that are there, the number of dependencies that are there. So, what we do is we have a pre-production cluster that mimics staging as much as possible. You could theoretically run crafting and production if you wanted to. We just don't recommend it. but the reality is it's just a Kubernetes cluster. We have orchestration on top, run all of your services there, run a smaller set of data if needed. But the reality is this scales nonlinearly. If you're able to share All of these Kafka cues, all of these DBs, all of these auth services and everything across thousands of agents, the scale is very, very manageable. And I think there's a very big difference between designing products that work for individual engineers. And I think a lot of these sandbox companies have done a great job of that. But it's fundamentally different if you're designing a product that is meant for large-scale enterprise organizations.

Manav Gupta: So just on that topic, have you guys cracked a genuinely regulated enterprise, a bank or an insurer? Yet? And if so, what broke the first time you guys tried?

Sumeet Vaidya: I wouldn't say we have a global bank or anything like that yet. I will say that you know, we've got BREX, we've got Fair, we've got Webflow and others too. I think every new engineer at BREX is on board using crafting, so that's nice. for true enterprise, there are some organizations that I can't name just yet that we are actively supporting and will likely become real customers in the coming months. But the conversation is fundamentally different. One, one of the largest real estate companies in the world. That has a pretty large engineering organization that we're working with, this is a massive problem for them. And it's consistency for both engineers and agents. But when you dig into how you get these things up and running, it's not just, hey, I need my agents to be the latest and greatest models doing all the coolest stuff. It's a question around security. What do they have access to? How do I make sure that they can access this data without breaking things? And how do I have visibility in everything that's going on? It's a question of cost, right? I think we've seen this where Uber's CTO was speaking a couple months ago and finally broke the silence of, yeah, well, token maxing is a problem. We're blowing through our budget. And every organization I've spoken with has said that. And so with a lot of them, it's hey, here's how we help mitigate these workloads. Here's how we help make sure that every agent has a minimal footprint with maximal capabilities. And there's an international government agency that we're actually talking to as well. And They've assessed some of these other sandbox providers too. And they said it's a great product, but it just doesn't work at our scale, given the number of services we have, given the number of engineers we have. And so I think the to crack these enterprises, it's really being able to speak the same language as them and understand what they care about. If you're a small team, it's speed, it's time to value. If you're an enterprise organization, security and compliance are non-negotiables, and cost now is two.

Manav Gupta: Okay, so so let's play it forward, right? So I think almost every company that I've now talked to, they are writing code via AI, despite all the protests and contrarian arguments being put put forward put forward online. But if you play it forward, what happens to a senior engineer's job? So then almost all of the code is machine written. So what do you think? What skills are gonna be important, what skills become scarce? Just give me a give me a thought on what what happens from this point onwards.

Sumeet Vaidya: Yeah, I I won't lie, I think it'll be a really hard time. I think it will be some major shifts in the industry. But we can also look at the industry historically. I don't know about you, but I remember writing Verilog when I was in college. I remember learning about compilers. I remember learning about the the guts of the Linux kernel. And the reality is most engineers don't need to care about that today. Right. At the same time, the word judgment gets thrown around a lot, but I think there's a reason for it. There's certain outcomes that we are responsible for as engineers, whether you're a VP of engineering or you're a fresh grad. At the end of the day, there are people who are responsible for building the things the business needs and doing so in a way that doesn't put the business at risk. That's why we have jobs. And I don't think that will change. I do think that we may see some collapsing of functions. I think there's a lot of conversation on the product side around. What's the role of a PM versus a designer versus a senior engineer on a team, which I think is fair. But when you're integrating with an author provider, when you're handling sensitive legal documents, when you're handling health records for folks, somebody's on the hook. And that somebody needs to have an airtight case about how that data is being handled appropriately. We're still gonna have SOC2 certification, FedRAMP's still going to exist, right? So there will still be people who are deeply technical who are accountable for the security, safety, and efficiency of these systems without blowing costs out of the water. And I think that the days of engineers who are just heads down crafting the perfect infrastructure or caching layer or routing layer, those might be numbered for sure. But those same engineers are still incredibly valuable as long as they're able to tie their technical judgment. To business outcomes and needs. So I think there'll be a little bit more fuzziness around the edges for all these roles, but at the end of the day, not every designer I know will be able to take on the responsibility of managing that front-end state machine for onboarding a new driver at Uber. Not every PM, even on the infrastructure I know, will be able to own the future caching layer to make sure that whatever is the most popular messaging app in the world is able to handle scale. These technical challenges don't really go away and models make it good enough to solve a lot of these things, but when they get it wrong, someone's on the hook. And that someone needs to be technical enough, have the right judgment, and be able to tie all that to what the business needs.

Manav Gupta: Do you how do you define productivity or this you know the golden pot at the end of the rainbow of

Sumeet Vaidya: Yeah.

Manav Gupta: developer productivity with AI, right? It's being baked into CIO targets and CTO targets now. What's what's your take on that? I mean clearly cats are the back, we can generate code nearly free. So maybe the jobs were code machine and PR creation. this arcane follow-the process and so on, maybe those jobs might be optimized. But how how far do you think this is going? And and how and what are the best organizations in your opinion? How are they booking capitalizing revenue, capitalizing productivity?

Sumeet Vaidya: Yeah, so first thing I'll say is I think anybody who's focused on PR count has lost the plot. I think that was true with engineers. I think it's true with agents. They're just throwing that out there. second, one thing that was really interesting is some of the best engineers I've ever worked with. They produced obscene amounts of code way before coding tools. Now with coding tools, their output has actually scaled much more dramatically than other engineers I know. And so I don't think this concept of this person is a coding machine is going to go away. If anything, I think it's going to be reinforced. It's just how they produce all of that output is going to change. At the same time, those people might not be the best people to tackle a more nebulous question around pulling from my background. How might we build the right set of moderation tools for moderators and admins in large community servers on Discord? It's not a coding question, but there's a lot of questions technically about how you make that happen. So I think we will still see archetypes and they will continue to evolve as time goes on. So, to your original question of how do you measure productivity, it's the same way we always have. And I might be a little more business oriented than the average engineer, but I think that's just it. What are the business outcomes you're driving? Is it usage of the platform? Is it resiliency and a lack of outages? Is it efficiency in terms of integrations? Is it just revenue generated? At the end of the day, that's going to be the metric. Ten years ago, if you had an engineering team that crafted the most well-architected product you've ever seen, zero flaws, zero issues, people loved it, but it cost the business money and did not help sentiment and did not drive any core business values. you would probably kill that project and move that team elsewhere. So whatever you define from an artistic standpoint about being an engineer doesn't actually matter when we're talking about the concept of what does the future of our industry look like from a job standpoint. At the end of the day, business success will always be king. And if we're not delivering on that, it doesn't, it doesn't really matter.

Manav Gupta: Yeah, but I think underpinning the the the the question is is the threat of restructuring, optimization, job loss, et cetera, if somebody's a developer. So do you foresee ten, twenty thousand developer shops that currently exist, sometimes in large organizations in regulated industries by the way? What happens there? Right? If CIOs are talking about hundred Two hundred percent, hundred fifteen percent developer productivity. Are you seeing that at all, Bowler?

Sumeet Vaidya: No. We're we're seeing we're seeing double digit productivity gains. I do think there will be

Manav Gupta: Okay. So then is it what fewer had come?

Sumeet Vaidya: It's the same, it's always going be. Can you do more with less or the same? And we're seeing a bit of a knee-jerk reaction, right? I th was it Microsoft that just rehired a bunch of their senior engineers after laying them off, right? Like we're

Manav Gupta: They did, yes, yeah.

Sumeet Vaidya: we're seeing the knee-jerk happen back and forth. We've seen this in the past where it's we don't need

Manav Gupta: Okay.

Sumeet Vaidya: junior engineers. Let's only focus on hiring senior talent. And then that led to ossification of the organization. This happened 10, 15 years ago, right? I I think we will see a winnowing of the middle layer of engineering. I think that at a lot of organizations that aren't in Silicon Valley, that might be mid-markets, that have reasonably sized engineering teams, they might realize, well, this isn't a core part of our business. We can probably do more with less. I think that part of the industry will probably see a big change. And I think things will be tough. At the same time, I think there are new opportunities

Manav Gupta: Yeah, I think yeah. So finish it up.

Sumeet Vaidya: though. I I think that like There's so many businesses that have no online presence or have a very mediocre website or whatever. I don't see a reason why every dental and doctor's office wouldn't just have a software engineer either on contract or full-time saying, This one person can run all this stuff for us. Why wouldn't we just do that? It'll help our business out. So I don't know if that's going to be a net negative, a net positive, or or neutral, but I do think there's gonna be a pretty dramatic shift in the the middle section of the software engineering industry.

Manav Gupta: Yeah, I think I think what you're articulating is mediocrity is is gone, right? What people are expecting is this this amazing first time accuracy, pleasant experience, things just work the first time at a unprecedented pace. I mean clearly the cost per unit of work continues to decline. okay, so final couple of questions for you, Sami. So, you know, something that I ask every guest. So if what advice would you give to people that are just turning out right fresh graduates in the university or those that are going in com side is com side dead?

Sumeet Vaidya: I don't think it's dead. It there's this interesting discourse that's happening side by side in Silicon Valley where every big c company is saying, I'm not hiring junior engineers. But then you look at all these startups that raise 15, 25, 50 million or whatever, and they I refuse to hire anybody over 23 because they move too slowly. Those two cannot be true universally at the same time. It's just

Manav Gupta: That's right.

Sumeet Vaidya: it's a fact, right? So from a philosophical standpoint, standpoint for new engineers, I would say Compsci is not dead, but it's changing. And they would likely have had to relearn how to use AI four times over the past six months with all the changes, which means that they're probably more advanced and innovative than the average senior engineer who might be very productive at top-tier companies. I would encourage them to find companies that are looking for people that have their skills, that are hungry, that are ambitious, that are willing to just throw the old ways of doing things out the window. Those might be startups, those might be larger organizations, but It is hard, it is challenging, but opportunity is there if if you are good enough and if you have the skills.

Manav Gupta: Yeah, I think I think I would agree with that. So, Sumit, listen, thank you for this. What I'm gonna take away from this is a reframe, right? We spent the last couple of years obsessing about how fast agents can write code. Whereas I think the actual constraint, as you quite are quite well articulated, is downstream. You know, can you trust what what has been shipped? So code generation is getting cheaper, validation is where the real engineering is now. And for those of you who are listening, running platforms in banking, insurance, healthcare. The part that sits with me, the part I want you all to sit with is control control autonomy. I think that's the storyline, tagline that Sumit was suggesting. Not letting the agents lose, not letting agents in just a toy sandbox, but agents operating in inside real systems, as Sumit said, with scoped access, attribution, and an auditor defensible answer to who approved this. One of the companies to watch out for there is crafting.dev, right? That's the bar. If the validation story of your agent is not as serious as your generation story, you don't have an engineering strategy. You have just a faster way to ship risk. So, this week, one question for everybody who's listening for your own organization. When an agent's change passes every check and is still wrong in production, what's your blast release? And who's accountable? And what if if you cannot answer that keenly yet, that's your roadmap. With that, Sunit, congrats on the launch. I'm gonna put crafting and your fortune piece in the show notes. To everybody that's listening, keep building, keep validating, and I'll see you in the next one. This is Ship AI.