Isaac: Welcome to Never Rewrites. I'm Isaac Askew. Jeffrey Sherman: And I'm Jeffrey Sherman, and today we're gonna revisit AI and the shiny object syndrome. in the first part, we talked about whether or not AI changes the fundamental problem of you start work, you get a new shiny object, and you put the old work aside without ever finishing anything. And we came to the conclusion that, well, it depends, and really, if you can finish it, then maybe AI gets you over the hump. But really, it it what we want to talk about today is. Does that even matter because if you pump out two or three or four times more shiny objects, can your customers use them? and I was reminded of this doubly so because I just was listening to a conference call where somebody was bragging, well we shipped four times more features this time this quarter than we did this time last year. and therefore you know our velocity was four times faster. And I Whoa, whoa, whoa, hold on. What are the usage numbers on these new features? Isaac: Right. Jeffrey Sherman: And they did not have those. Now to be fair, they didn't have usage numbers on the old features either. Isaac: That brings up an interesting problem though. Yeah, I I didn't really consider that because we're so used to focusing on the fact that we have like a big feature to deliver and the feature could take three months to deliver. Now, if the feature takes days to deliver, so we just queue up many features and we just start like inundating customers with all these features. How do we know? I mean, first of all, I imagine there's probably two things going on here. Either you're just building features that you think people will like. some of the Jeffrey Sherman: Mm-hmm. Isaac: ones that they'll like and a couple of ones that you think they like because now you have the bandwidth to do so. Or you've actually, and this is probably not true, but it should be true, or you've actually vetted every one of those and the customers want all of those features. Jeffrey Sherman: Well, I don't think you could really vet every one of them. Isaac: Mm-hmm. Jeffrey Sherman: Right. Even if you're delivering incrementally and iteratively, it's you make a small bet and then you see which ones get usage and then you double down on whatever gets used and you cut whatever doesn't get used. Isaac: Right. Jeffrey Sherman: and in my career, I have certainly spent months building many, many features that got zero adoption. We put it out there and customers didn't care. Isaac: Well, I'm wondering if we're just what if we move into this phase where we go, imagine you just deliver eight new features a week. Just pretend like we're moving that fast. Jeffrey Sherman: Mm-hmm. Mm-hmm. Isaac: How do customers even keep up with that? And how do they even like what if they want to use it? It's just like it's like you're overwhelming them with too many types of features to tinker with. Jeffrey Sherman: Right, it just becomes noise. Isaac: Right. Yeah. Jeffrey Sherman: If you if you if you're making well, start with the the fundamental theory of your customers use you and they like the product. Right? They're they're generally happy with the product that has the features they need, that's why they're paying you for it. And now you add eight new features, maybe they care, maybe they don't, maybe they you know, most likely they're not even gonna notice. you know, for most of the SASs that I use. as an example, let's talk about Riverside. I I know you 'cause you tell me that Riverside is adding features all the time. Isaac: Right. Jeffrey Sherman: But really, we show up, we record, we're done. And right, Isaac: good point. Yeah. Yeah. Jeffrey Sherman: you can send us an email and and that's how we learn about new features, but we don't discover new features that they they keep adding. Isaac: And in fact, it gets annoying. It's like whenever you log in and someone's changed everything. humans are creatures of habit. We don't like when all of our workflows change. And if you log in and you have a whole new code of paint and everything's moved around and the menus are redone or there's new features, or you get this tutorial overlay where it tries to walk you through. Jeffrey Sherman: Yes, I was about to say the tutorials. Isaac: imagine you get that every day because there's a new tutorial and a new thing. Look what we've just added. Like X out, X out. Come on, just let me get back to Jeffrey Sherman: Yeah. Isaac: the thing I wanted to do. Come on. Jeffrey Sherman: I opened my y I opened the app for a reason and Isaac: Yeah. Jeffrey Sherman: the reason was not to to see the new thing. Isaac: Yeah, so I wonder if we'll see a pivot towards tried and true products that are just they always do what they say they're gonna do, they don't change often, and they're just exceptionally great at doing a few things rather than like this throwing everything, you know, like trying to do everything for you. Jeffrey Sherman: Right, a reemergence of the Unix philosophy of have lots and lots of small things that do one thing really, really well and Isaac: Yeah. Jeffrey Sherman: work together and then you can compose them. Isaac: Yeah, actually that's a good point. Where we we we saw this this explosion of like the MCP connectors, right? Where you have if I need this particular type of data data, I can use this. Or a t Ticketmaster had one they created for Muse, right? Where if you wanted to buy a ticket, you could go through Muse and say, Hey, I buy this thing, or check prices for this ticket, and it would use their connector to find prices of tickets and help buy it. So everything becomes an API. Right? Every everyone is kind of wrapped in an MCP connector and they provide really good. email sending a really good event data, a really good what have you, rather Jeffrey Sherman: Mm. Isaac: than just having to try one thing do everything. 'cause even in my experience of just trying to spin up a vibe coded side project for event data, there's so many Jeffrey Sherman: Mm. Isaac: facets to it I have to play roles on. I have to be like the product owner for I made an iOS app, an Android app, and there's a website, and now there's an API and there's an MCP connector. And then I have to debug all of those things and just one thing is hard to manage everything. Then I have to upgrade all the packages Jeffrey Sherman: Isaac. Isaac: for everything. Jeffrey Sherman: As a developer are you saying product is useful? Yeah, of course they are. Yes. Isaac: Product's very useful. Yeah. Jeffrey Sherman: Mm-hmm. Isaac: What is just extra fluff? Because I I've also seen where Claude will give me all kinds of interesting new ways to like I've asked it, I've got all this data. What's a great way to monetize the data? we could do Jeffrey Sherman: Mm. Isaac: sponsored listings for this or promote certain events higher if people pay for the listing, or you could put ads on the website, which you shouldn't do because you know might make people go, because they don't like seeing ads, you know. all these different trade-offs and different features. And some of them are just like ridiculous suggestions. But I could I could tell in the pipe dream era of tech bro built some new stuff to like that's an exciting idea. Yeah, let's just build everything and then you just have a ton of features you have to manage all over the place. Right? With no proof that anyone even cares about them. You're just wasting and burning tokens. Jeffrey Sherman: Yes. And I think that that gets back to sort of the the shiny object syndrome of you you have this thing and even if AI gets it out the door, if you don't have a way of knowing about feature adoption, Isaac: Mm-hmm. Jeffrey Sherman: you don't know what features that you're throwing against the wall are actually sticking, right? Unless the features cost money. There's most pretty much anywhere I've ever worked, they did not have a you know, how many people are using this feature, how many people are using this feature? That's not a a thing that was ever tracked religiously Isaac: Yeah. Jeffrey Sherman: anywhere I've ever been, except where it costs money. And if AI now means, well I'm gonna get twice as many features, that gives you even that it totally dilutes any signal that you do get. Because if you get a customer in the door and you don't really have any idea what they're using, then you don't know which of these features are you should double down on and which of them you should let go. Isaac: Well, that actually ties in really well with recent analytics and logging work that I done at a prior company where I kinda pulled in like a ground cover, which is like a logging connector, and pulled in all the different log stats on routes people were hitting to kind of verify which Jeffrey Sherman: Mm-hmm. Isaac: pages people got to. And there were some pages that didn't have good analytics, so I just use that a way of I know they landed on this page, even if I didn't have better analytics that might show, they clicked on this button on this page with this drop-down, this is what they Jeffrey Sherman: Mm-hmm. Isaac: engaged with, which is a great thing to add to, especially in these cases. but it provided me with like a way to know what not to work on. Because there's definitely been Jeffrey Sherman: Mm-hmm. Isaac: pages that were broken. I'm like, should I fix this page? And then if you look at it and no one landed on it in a year, why spend time fixing it? Just cut it. Especially if it's like a menu item. You could simplify the menu. You you want to reduce noise for your users. You want a really easy-to-navigate menu. You don't want extra features that take away and confuse people or make things hard to navigate to. So these are really Jeffrey Sherman: Mm-hmm. Isaac: great methods, especially now. I I use analytics more than ever. I scatter it wherever I possibly can, just so I can see like. what pages people land on, what they click on, how long they stayed on certain pages. and then that that data is way more valuable to me now because AI can consume all of that and make decisions much faster. And then I can see, wow, Jeffrey Sherman: Mm-hmm. Isaac: nobody navigates to this. Let me gut that. Let me not spend any time. I'm wasting my time and money here. Jeffrey Sherman: Right, AI makes analysis really easy. Like Isaac: If Jeffrey Sherman: that Isaac: if you give it breadcrumbs, plenty of breadcrumbs. Jeffrey Sherman: Yeah, if you have a giant wad of data and you're like analyze this and show me something, Isaac: Right, yeah. Jeffrey Sherman: it's very good at finding signals. Isaac: I I log a lot more frequently now and I I add analytics to more like UI elements than I than I used to just to provide as much data as possible so it can pull in from Google Analytics and pull in from any access logs or and just make and just give me some interesting stats. Cause they can find anomalies too. Like before we had to log and use some kind of things, some kind of telemetry to understand anomalies and it can just look and say, you had a spike in traffic. And you're like, wow, didn't know that. because I didn't set that up yet or didn't think to set up in that particular area. I'd only set up Jeffrey Sherman: Yeah. Isaac: anomalies in this area over here. and it can find it f faster. Jeffrey Sherman: Okay, so for people who have a system and they don't have good tracking on or good observi observability around what features they have, how would you go about it? But you mentioned ground cover and and Google Analytics, but that's sort of well, Google Analytics will tell you what people are clicking on. but ground cover kind of will only work if you've got kind of a one to one ratio of endpoints to feature. And Isaac: Mm. Jeffrey Sherman: if you've got features that are multiple endpoints or there's subtleties there. So how would you go how how do you go about building that observability? Isaac: definitely add logging in to areas that make sense to add logging. ground cover is not just access logs about what you Jeffrey Sherman: Hmm. Isaac: it's just like cloud watch. You can log all kinds of things from the fact they landed and got a 200 on a page to actually like dot info dot debug dot whatever logs too, right? Jeffrey Sherman: Mm. Isaac: so scattering those where it makes sense to scatter them. like don't don't blow up too much noise. I guess AI is good at deciphering the noise too, so maybe just Go ahead and go wild with it to begin with and throw logs everywhere. That Jeffrey Sherman: Back to big data, man. Isaac: might actually it might actually work better. I'm just I'm just thinking about like the old days where you wouldn't want to do that, but maybe it's better at handling that. Jeffrey Sherman: Well it's not free. Isaac: No, I mean well it depends on your setup, I guess, and where you're where you're logging to and if it's a vibe coded project or you know, it depends on what you're doing. but it especially doing things like UI clicking. one really useful thing I found I think Sentry has this, is rage clicking. they can tell Jeffrey Sherman: Yes. Isaac: when someone just spams a button they're just frustrated, and then you can kinda go back and there's a recording of their screen. That kind of stuff is useful. You don't have to necessarily pay for something like Sentry to get that. but you could also use things like just any kind of Google Analytics and tying in click tracking for things that you care about. if there's an FAQ on on a developer page on that and then you can see someone engaged and they kept expanding this particular drop down and they all were the related to the same question. So you okay, I can tell people must be confused about this thing 'cause everyone's looking for the same FAQ or whatnot. Or just any kind of elements that you think would be useful metrics. So like for me, for a side project I'm working on. my developer API page, I want to know if people are landing on it and if they're signing up Jeffrey Sherman: Mm-hmm. Isaac: from that page versus if they're signing up from an event page. And then I want to know if people are navigating and trying to make API keys and making requests. And so I'm logging requests. I log the type of requests they make, where they make it. So that way I can go, well, they are looking for places in more rural areas versus city areas. Maybe I can reach out and expand coverage for them. Anything like imagine any kind of data that you could use to understand what the customer is enjoying about your product and how to make it better for them. And those are the areas you want to scatter logs in. Jeffrey Sherman: Awesome. anything else people should do or i I guess the the flip side there is if you're a developer and you're getting inundated with lots of shiny object requests, Isaac: Mm-hmm. Jeffrey Sherman: how would you go about pushing back? Isaac: Where where the where are the requests coming from? Jeffrey Sherman: Say product is giving you lots of shiny objects and now they're saying, well you've got AI now, so you can get all these shiny objects out into production because now we're gonna have four times more features. We're gonna push four times more features. Isaac: I guess I'd push back and just make sure that they knew how to measure success. you you mentioning you mentioned they didn't know the engagement. That's that's success. Jeffrey Sherman: Right. Isaac: So they need to have some kind of metric by which to say, people are enjoying this thing. And then also I would also push back if there are too many changes in one area to where there's signal conflation. Jeffrey Sherman: Mm. Isaac: So where you think they're landing on page for one reason, but it actually is because you You built two things on the same area and you can't because you didn't set up your data right, you couldn't tell which one they actually cared about more. So just don't cross contaminate experiments. Jeffrey Sherman: Right, that's a very good point. If you change five things on a page, who knows which one is the thing that they care about or is driving them nuts. But if you change one thing on almost every page, you at at least get more a cleaner signal. Isaac: Yeah. And I'd also ask product to have a plan for removing th failed experiments. make sure it doesn't just get s AI slot left in the code base, right? Jeffrey Sherman: Yes, I I think that's a very good call out is removing failed experiments. If you try something Isaac: Yeah. Because every I've noticed this too, every bit of code left over is context given to Claude next time. And they go, here's a feature Jeffrey Sherman: Mm. Isaac: here someone's using. And they think about it. And they give you suggestions on how to improve it. And you're like, well no, actually we killed that thing weeks ago. Why are you giving me advice on this? Just remove it. Remove dad code. You have all the reason now to like slim at your code base and only care about the things that you need and remove everything that could be distracting Jeffrey Sherman: Mm-hmm. Isaac: any kind of just like before whenever you threw a junior developer at your code base and they got they went down a rabbit hole with one feature actually you don't even use, that will happen to Claude. Don't let it happen. Just remove it. if we have Git history, we can add it back if it was an accident and to remove it. Clean it up. Jeffrey Sherman: Yeah, I like that. That's that's a great thing to leave it on is y you if we when you're especially when you're throwing stuff against the wall to see what sticks, remove the stuff that doesn't stick. It ju it's just noise. It's just clutter. Awesome. Well thank you all for listening. I'm Jeffrey Sherman. Isaac: And I'm Isaac Askew, and this is Never Rewrite.