Episode Transcript
[00:00:00] Speaker A: Foreign.
Whether you're a technical or non technical founder, I think the question of how you use AI in coding and development should be top of mind for all of us, right? I'm a solo non technical founder and I think all the time about how are developers using Claude, ChatGPT, Perplexity, Gemini, all these tools, right? Replit cursor to make themselves better, to be more productive, to ship higher quality code faster and with less bugs. In this episode, I chat with Aaron Francis, who is kind of the person I go to for kind of developer developer education things, to see his perspective on how he's using AI as a developer, as a developer educator, how he's seeing people in his space use it well and not and kind of for different levels of seniority of developers. What kind of tooling and approaches to using AI is he seeing working particularly well? And where is it not right? Where is AI falling down and causing folks more grief than benefit? This is a great conversation for me. Some of this was definitely over my head. But if you're a developer, you're totally going to get what he's talking about. And I hope you get a lot out of this conversation. I sure did and enjoyed the heck out of my chat with Aaron Francis.
It's funny, you know, I, so I'm like, I'm not a developer and I've, I've tried to learn to code a few times and it never really stuck.
So like I, I kind of went through the whole thing of like, let's try Laravel and let's try Rails and let's try Django and it kind of never really stuck. And like now I think, oh, I can just code without knowing how I can build shit without knowing how to code. Right? Is that, is that the promise?
[00:01:49] Speaker B: Do you think that is the promise? Yeah, that's what I was going to say. That is the prom.
Speak to whether or not that is the reality. I keep, I keep seeing people say on Twitter that you can do that. You, you don't have to know anything and you can just hit, you know, tab, tab, tab, tab, and you'll make a full app. And yet I feel like I don't see a lot of new apps. I mean, that's the thing that I'm a little bit confused by is all the, I don't know, all the AI talking heads on social media are like, you know, we're not going to have any more developers and yet it seems like we still need developers. And so I'm a little skeptical on like, I don't need to know anything. AI is going to do everything, at least at this point in time. But I do think AI is incredibly valuable and it would be silly as a developer or even slightly technical person, I think it would be silly to not use it at all.
[00:02:43] Speaker A: Yeah, yeah. So I think it's helpful in these kind of conversations to qualify your perspective a little bit, like frame your perspective. Because there's idiots like me who are like, I just use replit and you know, build shit. And there's like quite senior developers who I think when they talk about AI, their perspective should be quite different. And there's mid level and junior. Like, where would you say you fit on there as we're kind of like sharing your perspective?
[00:03:07] Speaker B: Yeah, I would say I'm a pretty senior developer. I've been a developer for a long time, I guess professionally maybe 15 years. And beyond that, you know, I started, I'm one of those that started when I was like 12, 13, so I've been doing this for, you know, 20 years now and feel pretty, pretty qualified all across the stack, you know, front end, back end, basically everything. And so I feel very confident using AI to kind of make me go faster. But I do worry a little bit about junior developers that are coming in cold and kind of just relying on AI to do stuff for them. That makes me a little bit nervous because, because I think they're so. I think once you get past like, once you get past like a demo app or like a to do app, there are a lot of, there are a lot of architectural decisions that need to be made right. There's a lot of keeping, keeping kind of the structure of the whole app in your head and slotting things in where it should belong and knowing what's going to work, what's going to scale up well and what's going to scale out well and stuff like that. And I think you can very easily get in a situation if you're just letting any of these tools write the code for you. I think you can get easily get into a situation where it focuses on, you know, one task at hand without consideration to like the broader structure or architecture, future, you know, future concerns. And then you end up kind of like down a rabbit hole that's really hard to come back out of.
And so designing software is different, I think, than just like writing a feature designing not, not like on the front end, but like designing the software on the back end or even on like the co of the front end. That is a very, like, that's a very important skill to have. And I'm not yet convinced that these AI models have really good and broad architectural design, like patterns.
[00:05:21] Speaker A: Yeah.
So maybe like getting specific to talk about, like, the tooling and where, like, it might fit for the right type of person. So I'll just share kind of what I know because I think I'm sort of representative of a lot of our audience. And then it kind of spans tech from the technical spectrum up to kind of like where you are.
I got started with Claude and just talking to Claude and building, you know, command line stuff, that, that was fine. I have used replet. I still don't know how to pronounce it. Replit is what I say. I don't. Yeah, it. It is much better at, like, I just want this simple thing. We're. We're using it to build some, like, tools, like free tools, like marketing stuff.
That's probably as far as I would expect it to go. And then I am using cursor a bit to try to build slightly more complicated things.
But I don't expect any of these to be able to build Cast us. I mean, it's a pretty complicated, you know, product.
Like, do you agree with, like, kind of like the use case for those? And then like, I'm sure I'm missing a whole bunch of shit. Like, I know people on our team use Super Maven. I hear of Windsurfer. Like, what. Tell me about, like, your perspective of like the tech here on February 7th.
[00:06:40] Speaker B: Yeah. I think the thing you said there at the end is the most telling. Like, it wouldn't. It couldn't build Kestos. And you know, I, I am not like a. I'm not like a thought leader or anything. So I won't say what it can and can't do in the future. I have no idea. But where we're standing right now, I agree it can't build like a full SaaS. Like, you can't go to it and say, build Castos, build Savvy Cal, build whatever. Like, it's just not going to work.
But I think on the other side, the other extreme of like, AI is so stupid. I'm not going to use it. Is is fooling yourself. Yeah. And so I think where we come down is somewhere in the middle, which is the AI can be incredibly powerful and incredibly productive for discrete tasks. And so the way that I like to use it, so I use it. And I'm not like a full on, you know, AI bro that just like never touches my keyboard and just hits accept. Next, next, next. But I don't want to be a Luddite either. And so the way that I use it is I don't have any, I don't have any models built into my, my ide. So like in my actual editor, I don't have anything that is like, hey, here's maybe the next 10 lines of code you were about to write. Maybe not, but maybe that drives me crazy because it's, you know, it's wrong 50% of the time. And even that, it's like the other 50% of the time it narrows down what could have been. Like, oh, I have this kind of thought and then it completes something and you're like, oh yeah, I guess that's good enough. And I just don't like that. And so the way that I work right now is, you know, I work in JetBrains PHP Storm, which is, you know, just my IDE. And I have the machine learning autocomplete on. And so that's all local. Local only doesn't do a lot of like writing logic for me. It's just kind of like a much better autocomplete. That's, that's as much as I like built into the editor. That being said, I pay for 01 Pro, which is $200 a month. I pay for Claude, I play, I pay for Gemini. Like I pay for all of the tools because they're so useful. And so where that leaves me is when I run into, when I run into like a discrete problem, I will take, I will take some files, some context over to a lot of times right now it's Claude and say like, here's what I'm trying to do. Here are the 4, 5, 6 relevant files.
And it works best if you say, here's the test that I'm trying to get past.
Can you write the implementation for me? It does that pretty well. It's like 90% of the time it comes back and it totally works. Then I take that from the cloud interface in the web and put it back in my editor and get back to work. I like that air gapped approach where I can walk over to the machine and say, do this really hard thing. Here's like, here's like a minuscule amount of context that you need so that you don't get distracted. Because if you distract the models, they go off and you're like, what the hell are you doing? Like I asked you a specific question. And so that's my preferred style right now for like discrete tasks. I also have some for like, like spiking out new stuff, but we can, we can pause there.
[00:10:13] Speaker A: Okay. Okay.
How difficult is it? Or limiting, I guess, is it to get that context and to input it, or are you always like, I know I need to reference this function and this model and this other thing?
Or do you think sometimes you struggle with that?
[00:10:35] Speaker B: Yeah, in the ways that I've been using it, it's not very much of a struggle. I have. There's a. There's a Chrome plugin that allows you to like, do file uploads to Claude. So with Claude you can upload files, but with this plugin you can upload folders. That's what I meant to say, you can upload folders. And so, like, a concrete example is, I'm working on this, this library. The library is called Fusion. And part of what the library does is it takes a block of PHP that the developer has written and it goes through a pipeline, like a build step.
It takes the block of code that the developer has written and then it turns it into an abstract syntax tree. It inspects all the code and turns it into all these tokens, and then it walks through the user's code and makes slight modifications.
This is a really nice controlled environment for CLAUDE to operate in. Because what I can do is I can say, here's the user's code before, here's what it should be transformed into. Like, once you're done parsing the ast, this is the output, here's the structure by which we're processing files. So I've written like a. A class called Conformer. And what it does is it conforms the PHP to the right style. And in the Conformer there are, you know, 10 or 15 different little classes that do one or two things at a time, and it just walks through them in like a pipeline. And at the end you should have something good come out the other side. So I can go to Claude, say like, hey, here's the whole folder of the all the conformers that I already have. Here's the before, here's the after. Write me a brand new one that makes what I just showed you work, and it just spits it out. And I take that class and I go paste it into my IDE and I run the tests and if they turn green, then I go back to the class and I'm like, yeah, this isn't totally my style. Let me like, change that, change this. Run the test again. Still green. Okay, back to work. And so that's a very, like, scoped down universe of what CLAUDE has to Worry about with a well defined task of before, after. Here's the structure. Write something to fill in the gap and it works like first shot 95% of the time.
[00:12:51] Speaker A: Yeah, yeah. So to kind of like go, go up a level, I guess. You know, you hear, you hear from a lot of folks like, we're, we're not hiring developers. Like everyone needs developers. Like, we're not, we're not hiring new developers. Benioff. I think so it's like we're not hiring new developers like until further notice. I, I feel like that, like I hear things from our team like, hey, I'm 30 to 50% more productive right now than, than I was. What do you, I mean this is getting a little like esoteric but like, what do you think that means for like developer jobs and the economy? Yeah. Are we just going to build a lot more or are there going to be fewer developers? If so, like what, like what aspect of that is at jeopardy?
[00:13:38] Speaker B: Um, yeah, I, I'm torn. So I think, I think the appetite for software is functionally infinite at this point. So I don't think, I don't think we're like running out of things to build.
That's not something I have seen yet. So I don't think that, I don't think that's on the horizon. Um, in terms of, do we need more developers?
I, So I have felt this a little bit as I'm working with AI. I do think I am maybe 5, 6, 7, sometimes 10 times as productive using AI as when I'm not. It's a lot times, times, times. Yes. So like one of these, like writing one of these AST parsers could take me solo, it could take me an entire day because they're incredibly finicky. It's like, enter this node, exit this node, recursive hell over and over and over, right? And I can get this thing done in literally five minutes. Talking to Claude and so it's like many like an order of magnitude faster.
So just my experience alone tells me, boy, I'm getting stuff done a lot faster.
If I were on like a big team, we would all be getting stuff done a lot faster and we wouldn't need other people.
So I do think that there is going to be some downward pressure on hiring and some because people are going to be AI enabled.
I don't think developers are going to go away. I don't think that, that. I just don't ever see that happening. Because at some point someone has to describe to the computer exactly what you want to happen. And that is called programming. Like describing to the computer what should happen is programming. And so there is, I think, a future where programming looks very different, where you're describing to this intermediate computer the AI, what you want to happen. But you still, like, you still have to be pretty good at describing the problem and pretty good at describing the proposed solution for it to get it right.
And so I do think there's going to be programmers in some way, shape or form in the future.
My, my plan for staying relevant and not broke is to see this coming wave that is supposedly going to crush us all and try to harness it and just ride it out. Like instead of just being crushed and tumbled, I'm going to try to surf it as much as I can. And so that's why, that's why even though I, even though I like chopping wood down with my bare hands, like I'm going to learn how to use the chainsaw because I would like to have some sort of income in the future. And so I feel like there's this certain set of programmers that are like AI stupid. I'm never going to use it. And I think that's going to be the first set to go. I think there is another set of programmers that's like, AI is amazing. I'm only going to use it and I think their potential will be capped. And of course I'm talking my own book here because it's what I'm doing. I think there is a middle where you continue to be a good AI enabled programmer that goes 10 times faster than everyone else and you might just be able to surf that wave out without getting crushed.
[00:17:05] Speaker A: Well, and it's everything, right? Like on the marketing side, I sure as hell use. Yeah, same. I pay for Claude and ChatGPT. I use all both of them. I don't know, a third of my day is spent in those tools and the other third is on Zoom, I think. So it's like. Yep, yeah, I mean I use it. So I'm writing a book, I haven't talked about it much. I'm writing a book on sales and I, my process was, it's very AI enabled. My process was a lot like you're talking about. I spent about a week working on kind of the premise and the outline and the character development and all this kind of stuff. And then it is fed that into Claude and it wrote the book in about an hour.
[00:17:44] Speaker B: Yep.
[00:17:45] Speaker A: And it's awesome. Like it's really, really, really good. Like it's, it's way better than I could have written. And that, that, that's, to me, that's the magic is like you take, just like you were describing with your pipeline thing is like you take the architecture, you get it right, you show it like what good looks like, and you let it kind of fill in the blanks. Yeah, I think the, I think that like, at the same time, I, it's funny. It's really funny. I am at the same time the most at peace with where Kastos is in AI because I'm like, yeah, ain't nobody like Andrew Wilkinson talks about his businesses being New Zealand. Right. Like, big enough to be relevant, but not so big that somebody's going to come after you. Right. I think we're kind of there. Podcasting generally is kind of there. It's not AI, it's not all this kind of stuff. So I think we're in the shadows a little bit. It's also pretty complicated.
So I feel at the same time like, hey, geez, knock on wood. We're not a content marketing agency or SEO that's about to get walloped. At the same time, I'm the most stressed I've maybe ever been because I have these talking about this wave that's coming and I'm like, what's going to happen? Like, not what's going to happen to Castos in this business. I think it's pretty secure. But what's going to happen to the world? What's going to happen to like, all of our 4,000 customers that are paying us money and all this kind of stuff? Like, I don't know. I mean, this is not a development conversation, but like, I don't know. Are you more or less stressed?
[00:19:20] Speaker B: Yes, I am.
[00:19:20] Speaker A: Because of AI.
[00:19:23] Speaker B: Yes, I am way more stressed. I am way more stressed.
[00:19:26] Speaker A: That's most people.
[00:19:27] Speaker B: I think I've got it way worse than you because we make our money through education. So, like, we make our money by teaching developers how to use databases well. Right. And so we've got a course on postgres, of course on SQLite, that those are our two main sources of income. And if we're saying that like the super intelligence is on the horizon, people aren't going to be like studying very much anymore. And so, yeah, that freaks me out big time.
I think, I think that is, I think that is short sighted to say, like, I don't need to learn, I just need to ask the AI.
But I think that's what people are doing. Like, I think, you know, just watching, watching people who are younger than us. So people maybe in their early twenties interact with development. They're like, yeah, I don't know anything. I just talk to Cursor all day. And I think that's a terrible idea. I think your career is going to be very painful. However, it is reality that that's what's happening. And so I look at that and I'm like, am I going to try to convince the whole world that, like, you should continue to watch video courses about how to use databases, or am I going to do something else that. That harnesses that AI wave instead of tries to, like, hold it back? Because I can't hold it back. And so, yeah, it's. It's really scary. Like, we picked the worst time in human history to start an education business on the cusp of superintelligence. And so we talk about that a lot. We're like, we're trying to figure out how to. How to harness it for our business. And we have a few ideas, but it's still. It's still like, pretty terrifying.
[00:21:07] Speaker A: Yeah. Yeah.
About. I got eight or 10 months ago, I started doing some coaching kind of on the side, and it's been really rewarding. And similarly, though, I look and say, like, why would you hire me to be your business coach or your sales coach if you can just go talk to Claude for a while? And I think the answer is like, one people. Right. I think that's the thing that is the big opportunity with all this. And like, if you're looking to do something, it. It should involve human beings looking each other in the eye and being in the same room.
I think the other thing is that that kind of like autocomplete, like, misguidance. Right? It is.
And I see this in most written content and you see it in a different way in development is it's not wrong, but it's not like the most. Right. A lot of times too. Yep.
And I think that's. I think that's the only limiting factor right now, to be honest, though, because you could sure as hell take all of my podcast, all of Rob Walling's podcast, all of, you know, whoever's podcast, load them into a ram rag.
Right. And just talk to it. Right.
[00:22:17] Speaker B: And.
[00:22:17] Speaker A: And like, you wouldn't need me anymore. I don't know that'll happen. But. But it's, it's. If that is as good as actually talking to a person or actually taking your course and actually leveling up to be able to get that architect level. Right perspective.
[00:22:36] Speaker B: And even, like, even, even in some of the AI work that I'm doing, I have still found it beneficial to read the documentation. Like, I'm. I'm big on reading documentation for tools that you use. And there was an instance just a few days ago where I was.
I was talking to, I think it was 01 Pro, and I was asking it how to do something and it like, lined out this really, like, complicated, not very like, developer experience friendly way to do it, using a certain pattern. And I said, wait, hang on, can't we just use a reactive? Because I knew that the VUE JS docs had this concept of a reactive instead of a ref. And it was like, oh, a reactive. That's a great idea. Here's how you could do it with a reactive. And if I had not known that the word reactive was a thing, that it was a concept in VUE js, that it was shaped similar to what we were talking about, I couldn't have turned around and said, hey, you gave me this solution that kind of sucks. Did you think about this other thing?
And that's one of the things that it's like.
I think the most valuable thing that you can do as like, a working developer is have a broad understanding of the landscape. Like, read the documentation front to back without any pressure on yourself to, like, remember or understand or, like, deeply grasp any of it. Just read it all the way through. And then in your head, you know, like, what the area of the thing you're working with is. And when you. When you get to a problem that is shaped like some vague thing you read in the documentation, then you can walk over to Claude and be like, hey, I'm having this problem. I remember this was in the docs, like, is this the right thing? And it can say yes or no, but at least you understand. Like, you have a map of the area that you're working in, whereas if you don't ever do that, you're. You're like, you can't see two feet in front of your face and you're just kind of like fumbling around in the dark. And so you as the human, taking a lot of context to Claude and guiding it and saying, like, you went down this rabbit hole. Try this thing instead. Start over. Literally, start over from scratch. Try this approach and see what happens. And you don't have to know what's going to happen, but you have to have enough knowledge to say, like, hey, there is another way that you didn't think about do.
[00:25:04] Speaker A: So I was playing around with Replit last night for, like, a Landing page. And I wanted to have it do something else. And I was like, ooh, this is where I would want to, like, fork. And I know that's not the right term. A branch maybe, Like, I want to take. This is okay, I want to take and go do something over here that I can maybe kill and come back to this thing. And I don't know if it does that, but do you. Do you ever, like, take something that it gives you and then feed it back in and say, this is what you just gave me. Like, shoot holes in this. What's wrong with this? How would you improve it? What other questions do I need to ask for you to give me a better response? I do that all the time.
[00:25:37] Speaker B: And I'll do that against models. So I'll take the Claude output and go over to O1 and be like, I wrote this. It's a lie, but I wrote this.
Is there anything that I could do better? Have I missed any edge cases as a great question? Like, hey, what are the things that are that I'm not considering in this code? And yeah, I'll do the opposite. I'll take it from 01 back to Claude and tell it I wrote it. And yeah, I do that kind of adversarially sometimes with Claude. If it does get, like. If it does get kind of, like, wonky, I will copy out the last bit of code that it wrote and just start a new chat and say like, hey, I wrote this just back to Claude itself. And then see, you know, what it says about, you know, changing it or something like that. But yeah, I'll absolutely, absolutely do that.
[00:26:24] Speaker A: So you've talked a lot about, like, oh, 1 and Claude Sonet, I guess, 3.5.
How do you decide when to use Which?
[00:26:35] Speaker B: Right. So Zero1 Pro is the big dog, right? So it's the one that is both expensive and also thinks for, like, several minutes at a time. And so 01 Pro is. 01 Pro is my guy when I need, like, I gotta write two paragraphs of what's on my mind and give it as much like I'm thinking about doing this. Here's the problems. Here's what I've done already. Here's why I think it could work. Here's why I think it couldn't work. What do you got? And then I just let it cook. I go back to work and I come back, you know, five minutes later, and it's got a page and a half or two of ideas. And so that's really, really helpful when I'm like kind of at the beginning and thinking, how do I want to architect this? How do I want to structure this? Or as is the case recently, I'm operating in an unfamiliar environment. So I'm like writing a vite plugin. I've never written a vite plugin in my life. And so I walk over to OOD Pro and say, I'm writing a vite plugin. I've never done that before. Here's exactly what I need it to do. Here are the considerations. Can you spike out the first implementation of that? And it normally does that just fine. So 01 Pro is for like, hey, here's what's on my mind. Help me figure out where we're going here. Works amazing for that.
O1 Mini is for like, hey, how do I do a. How do I do a regex replacement? And on a JavaScript string? And it's like thinks for half a second and spits out the thing back to me.
[00:28:04] Speaker A: Yeah, it's great.
[00:28:05] Speaker B: It's great for that. So that's kind of the O1. The O1 family I use. I'm basically using O1 mini instead of something like Google or Stack Overflow specifically for programming questions. I'll copy a piece of code, paste it into one mini and be like, what am I doing wrong here? And it comes back like, super duper fast.
And then Claude, I think it is 3.5 Claude. That's where I go. And I upload a bunch of files and I'm like, you've or I have done six of these. Here's the spec for the seventh one. Write it. Or I'll go paste in an entire PHP class that I've written. And I'll say something like, hey, here's this class.
I need you to clean this up. Like, I'm doing a couple of things twice. Can you like, refactor this for me? And it does. It does like a great refactor and spits it out. And I paste it back in and run the tests and it still works, which means I'm good.
[00:28:58] Speaker A: Interesting. So just to paraphrase Claude, give it kind of examples and ask it to fill in the blanks or repeat something with a new iteration of it.
O1 mini just kind of asking questions like, you would use Google or stack overflow. O1 Pro. Mega Pro. Yeah, so I pay the chump $20 version for both.
For like deep thinking. Strategic architect is the term I would use the level kind of stuff. Cool. Okay.
[00:29:28] Speaker B: Exactly.
[00:29:28] Speaker A: I mean, to me, that's like the whole point of this conversation, right, is like you as a super experienced developer, how are you using these tools today? And because I like to opine about the world is going to end or not and we're just going to universal basic income and not have to work anymore, I think both are very possible.
But between now and 18 months from now when that happens, we didn't know how to feed our families.
[00:29:50] Speaker B: Yeah, no, it is, it is a, it is a 100% selfish pursuit. I want to make sure that my family still has food. And so like maybe we get to, maybe we get to nirvana, we get ubi, everybody's happy, we do whatever we want. But, but what I, what I do know is that it's going to be a bumpy road to get there if we ever get there. And the people that are able to use the tools are going to have more opportunities than the people that are not able to use the tools. And so I don't know what that's going to look like. But in times of great transition, there's a great like reshuffling and I want to be shuffled out on top in the end. And so I'm going to learn how to use the tools. I'm going to get way more effective.
Is it good for the economy or junior developers? Boy, I don't know. But I have to look out for my family, you know?
[00:30:40] Speaker A: Yeah, yeah, I don't, I don't know. I think junior to me, like what I told, what I told Jesse, like our senior developer on our team is I said I would, I would hire a you again and I would hire a person straight out of Bootcamp. I wouldn't hire a mid level developer anymore.
And that's just like from my perspective, knowing what I know from conversations like this, I think a junior person can probably operate at about a mid level, but a mid level can't think like a senior level developer with cursor Claude or whatever.
[00:31:14] Speaker B: Yeah.
[00:31:15] Speaker A: Okay, so I want to play what would Aaron do? So our last, our last episode I had Jordan Gal on, he's building Rosie. It's a automated call answering service for home service businesses. Reason I wanted to have him on is like he's building in AI like a wrapper. Right. So Aaron is building a wrapper product.
What would you do different than 18 or 24 months ago when you're just building a Laravel app old school style?
[00:31:50] Speaker B: Okay. So is the situation I'm building some sort of AI wrapper to offer to the marketplace.
[00:31:57] Speaker A: Yep.
[00:31:58] Speaker B: Yeah, I think, let's see what would I do differently than two years ago.
[00:32:06] Speaker A: Sorry, let me, let me give a bit of additional context just for folks who didn't listen to the episode, because I think, I think like, what made Jordan's episode and his situation so interesting is the way he put it is like, in the past you could answer your own phone or have a call service, and now you have a third option which is AI. And I think that that's the thing is like, I don't want just a freaking like wrapper on ChatGPT to repurpose my podcast episodes. I want like an AI native implementation of a thing that I had to do manually before. So, so that's the. And that may or may not be slight, but, but that's what I'm thinking.
[00:32:44] Speaker B: Yeah, I think it would, it would definitely change the universe of possibilities for what types of businesses I would maybe go into or create. Right? Because it used to be taking fuzzy or ill defined user input was very difficult. Right? You had to write a lot of tests, a lot of heuristics, maybe a bunch of regex, maybe a bunch of whatever. And it's like at the end of the day, you're still just kind of hoping that it works. Now you have this, this very like, intelligent thing where you can receive input from a normie and turn it into some sort of structured thing upon which your program can then operate. Right. And so I have to imagine Jordan's doing the same thing. So he like gets all the, you know, the voice from the end user, the AI, turns it into something a little bit structured, they query the business's calendar and then turn it back into voice for the end user. That just wasn't possible, you know, two years ago. You're pushing two to hear, like, when our calendar is open, right? And it's like, we don't have to do that anymore. So it would definitely change, change the shape of like what I want to offer.
It would also be, I think it would also be prudent to have a, to have a buffer between like your code and the models, because the models are getting better every single day. Like Deep Seek came out of nowhere. Like, hey, it's, it's, you know, nine in the morning. Oh, the world is different. It's 901 and it's like, we didn't see that coming. And so I think having some sort of router or translator intermediate later layer where you can swap out new models is going to be very, very valuable. I would hate to be in the model game. I know they're raising you know, $500 billion or whatever, but every day some kid is trying to eat your lunch. Where I would want to play is, where Jordan is playing is like, all right, I'm going to harness these, like these, you know, megaton companies beating each other to death. And I'm just going to, yeah, great. Y'all have fun. Good luck. I'm just going to use your latest model with zero loyalty to whoever made the latest model. And so I think in, in, you know, in the past maybe eight to 12 months, the tides have turned on GPT rappers. It's like, oh, suddenly they're the smart ones. Because then, you know, it's like there's no loyalty whatsoever to any models. A new model comes out and everybody cancels. Claude goes back to 01 and vice versa every other week. And so that's probably like, it would, it would be interesting to me if I were like trying to start a business. The heuristic that I would try to do is like, so what's messy, what's unstructured inside of a business? What is painful, what is valuable? And what I would do is I would put some sort of interface to that, put an LLM in it, have my code behind that, and then I would like sell that as maybe not even AI enabled. I think that. Or like AI solution, I think that is. Another thing you could do is just say, like, I will solve the problem for you. It doesn't matter how I do it. Am I using AI? Maybe, who knows? We'll find out. But I think you could go to a lot of businesses and say, cast upon me all of your unstructured, horrible PDF bound data and I will give you something good out on the other side. I think that's totally viable. Again, the problem is like, what are the niches? How do you get in there? Who do you know? Are you a plumber? Are you an H Vac person? How do you get into those industries to find those meaty problems? That's the same problem that everybody's had since, you know, the dawn of software is like, how do I get into these cordoned off industries to find what is painful and fix it? Because that exists everywhere. It's just like, who do you know that knows somebody, you know?
[00:36:37] Speaker A: Yeah, yeah, I agree. I think like the term AI, I think has already kind of like had a turn. Right. Like a year ago, if you look at like YC or Tiny Seed or whatever, like everything was AI. I think now like the name AI probably showing up less in a lot of products because it's like duh, you know, like of course, like if you're a new product, you have AI even we have like quite a bit of AI going on and are planning for a lot more.
So in Reef on. On the Reforge site, Brian Balfour had a really interesting piece about how users expectations of software are changing a lot. I'll include a link in, in the description, but the short version is we as software providers, we're kind of like helping people take the next step. A lot of times you think about Canva right now, it's like, just do it for me, right? Like there's definitely like a whole set of, a whole set of products that are like don't help me create like an image, just like create the image for me. Right? And yeah, I, but I agree with your, your like analysis of like unstructured data.
I think that like still, if I was going into it, I still would lean on the thing that I think it does really well, which is like take data, synthesize it and give me something from that.
I think like, you know, if you're, if you're in our world, like that's like AI show notes and titles and all, like understanding previous, you know, episodes and like coming up with title suggestions or topics or something like that. And we're building all that stuff. But what I wouldn't do is like automated video editing or something like that. Like, it's just like, it's that gap, right? It's that gap from like go build me, you know, savvy cow. Like, I just don't, I just don't see it there.
[00:38:27] Speaker B: No, what I would not. On, on that note, what I would not try to do is count on the AI to have good taste. And so like automated video editing I think is a terrible idea because in, in the same way that like creating music requires good taste, creating video requires good taste. And so as the human, I'm going to lean into my human ability to have good taste. And not every human has good taste, but it's something you can learn. But I don't think any AI has good taste because it's such like a, it's such like an undefinable human thing to be like, this is good and that is bad. Can I explain it? No, not really. I can try but like I just know what is good and what is bad. And so like do I want AI enabled video editing that does certain things? Sure. But I don't want to give like a raw video to the AI and be like edit this because I don't think it's gonna, it's gonna turn out right. Yeah, I'm fine if a bunch of other people do that because it makes it easier for me to stand out. Right. And I think that's another thing that, like, is going to be pretty important is where is there no human touch? And where can you be a human? And I think YouTube is a great, like, YouTube is a great avenue for that because we're seeing lots of like, faceless channels pop up that's basically like either stock photos or AI generated photos or AI clips stitched together with an AI voiceover. And it's like, yeah, maybe there's a gold rush there and it's kind of like drop shipping and, you know, the top 1% can make a million dollars and then everybody else makes nothing.
But I, Regardless of, however that, you know, grift is working, I think humans are going to want to connect with humans. And behind the written text, you don't know who you're connecting with. But behind video, you can still connect with a human. And yes, there are like, you know, there are AI avatars of real humans. I still don't think those are very good, but I still think video is one of the last video. And actually podcasts, because it's such a longitudinal format, is one. Those are one of the two of the last. Like, I'm going to leverage my humanness for, like, for an unfair advantage here.
[00:40:49] Speaker A: Yeah, Yeah. I told the team this week in our meeting. I was like, it's the most excited I've been about podcasting as a business model in a very long time. Because for a while I was kind of like, gosh, I don't know, like, post Covid and all this and the number of podcasts are going down and stuff. And now you look at it and you're like, I mean, it's really, it's hard for a lot of businesses right now, right? Like blogging and SEO, as a lot of us have tried to do for a very long time. Okay, I won't say dead, not doing good, right? Really, really, really hard, right? Cold email, LinkedIn, all this kind of stuff, all because of the AI enablement. I think just the volume of shit has made the signal noise ratio such that nobody cares, right? And nobody uses it. You mentioned, like, you don't go to Google anymore. Like, my 12 year old has chatgpt on his phone and like, it's all he uses. You know, he's like, why would I go to Safari?
[00:41:45] Speaker B: Like, I think, I think distribution is Becoming even more important somehow. Like back in, back in 2010, you could, you know, at that time, spin up a Rails app, build a little nothing burger, and end up with, you know, 50K. Mrr. Those days are just gone, right? Because almost everyone, almost every developer can spin up some app and have something out there, and there are 50 competitors for every single thing, and nobody cares. Nobody cares about your little app. Nobody cares about anything. And so having, having distribution is becoming ever more important. And I think it's one of those things that you might not know exactly what the outcome is, but you know it's going to be positive, right? So you can, like, build up distribution without a definite goal in mind. And as you're working towards building up distribution, you can find what it is you're going to distribute, right? And that is more important now than ever because there are solutions for everything. And. But if you have people that know like, and trust you, you can find the thing along the way. You can try a few different things and you have, like, distribution. That's why I think, frankly, it's more important than ever to have a podcast, an email list, a Twitter following, a YouTube following, all of that stuff, because it is getting harder to reach people. And I think those are the last, like, as organic or as inorganic as they are, those are the last, like, organic ways to reach people, is person to person.
[00:43:18] Speaker A: Yeah, yeah.
Last question. Who do you look to for information about how to use AI better at work?
[00:43:27] Speaker B: Yeah, that's a tough question. There are so many grifters and charlatans.
There are a few people. There are a few people I follow on Twitter that I think are doing it well.
Swyx Swix is one. He's. He's a good guy to follow. And then I think there are other. There are other reasonable lists that I follow for, like, just generally good takes. Like, Andreas Kling is building a browser called Lady Bird. I feel like he always has good takes. Thorsten Ball wrote a few books on programming. I feel like he's a reasonable list.
I feel like Taylor Atwell of Laravel hasn't talked about AI hardly at all, but I've always found him to be kind of right down the middle on, like, trends and tradition. And he just kind of cuts. He kind of cuts right down the middle on that. And so I try to follow. I don't think I follow anyone just for AI takes. I try to follow, like, well rounded, pragmatic, practical senior engineers and use them kind of as my bellwether for, like, what are the reasonableists Thinking these days about AI and every, like, breathless warp is going to change your life. I just mute. I just mute those people, frankly, because I don't want 15 tips to use, you know, bolt to, like, start a new SaaS every day.
So I kind of just like, filter all of my advice through people that I deem as. As reasonable. And there are many. There are many reasonable lists, and those are the. Those are the people that I tend to follow.
[00:45:02] Speaker A: Yeah, that's cool. That's cool. I mean, that. That's the goal for this podcast, right? Like, this podcast has had several iterations, but. But really? Yeah, we don't. I don't want to be like. And we might even rebrand the podcast at some point, which will be crazy. I don't want to. I don't want to be like an AI influencer, but I just want to be a vehicle for folks who actually know what they're talking about to share a few things to help everybody else out. Like, and I think that's. I think that's good. I think it's hard, right, because you have to bring people on who are genuinely helpful and not kind of pushing their own agenda. But sure enough, like, we all need as much of this as we can get. Oh, yeah. To filter the crap and take the good stuff and implement it to, like, make ourselves better. So.
[00:45:46] Speaker B: Yep. Yeah. I think on that note, something else that is true about the people that I tend to trust on these topics is they are working developers. They are not. They're not like, primarily YouTubers or influencers or anything. They are working developers. Like, they are building towards an end goal. And usually that end goal is some sort of capitalistic pursuit. Those are the people that I think, like, if something is able to filter all the way down to them, and they're like, my goal is to build and ship product. Turns out it's easier to do with this thing. Then I'm like, great, tell me about that thing. But if your whole business is like, sign up to my newsletter and buy this course on AI, I'm like, I don't know if I can trust you on that. So that's what I. That those are the people that I try to follow is somebody who's building a tool, a product, a company. And they're like, guys, I'm just going faster. I'm like, great, I trust you.
[00:46:48] Speaker A: Sorry, super. Last question. Does it matter if that's a kind of indie maker or they work in a large organization?
[00:46:55] Speaker B: Not to me, it doesn't.
I am, you know, I Have a business partner, but we're, you know, solo. It doesn't matter to me if they are indie or work in a big corporation. I think if you work in a big corporation, you probably want to listen to people that also work in a big corporation. But in terms of like discrete productivity improvements, I don't care. I don't care where you work.
[00:47:17] Speaker A: Yeah, yeah, yeah. I asked just because I think the adoption and perspective on adoption between us and somebody that works at, you know, freaking Pfizer or whatever is, is quite different.
[00:47:29] Speaker B: Yeah.
[00:47:30] Speaker A: And, and like their context maybe should be too. I don't know. Yeah, yeah. Still, still very much trying to figure it out. Yeah. I'll link to in the show notes. I don't have them all in front of me. A couple of folks that I listen to as well, mostly on like, just business stuff and like they're all YouTube channels as you said. So. Yeah, it's a good, good place to be bullish. Cool.
[00:47:49] Speaker B: I have one, I have one more charge.
[00:47:50] Speaker A: Yeah.
[00:47:50] Speaker B: That I think is incredibly important in the coming age. I have found that tests are 2 to 10 times as valuable as they were before because before testing was to like offload stuff out of my head. Like I can kind of keep the whole program in my head up to a certain point, but, you know, it's better if I like concrete or I like formalize some of these like assumptions I'm making or things I'm holding in my head into tests. Right. And then I feel warm and fuzzy when I run the tests. Everything passes and I, you know, I do something and it breaks something somewhere else. I'm like, oh, thank goodness I have a test. Right. I think it's like 10 times more important now because the AI is non deterministic and it's going to do something different every single time. Right. And in terms of like, let's go back to that example where I'm writing all of these, these AST parsers, these conformers, sometimes I didn't write a single character in the actual implementation code. And so having a test or having, in fact, you know, 30 tests that all test, this single file, this one class, I've got 30 different test cases. Then I feel super confident when it's like, oh, I need to add one thing to that, to that transformer. I'm just going to copy the whole implementation and go to Cloud and say, I need you to augment it such that it does this. And I can take that back and know that the 30 test cases that were passing that I'M super relying on, I can know that those still work. And I wouldn't be able to know that because I don't have enough context to look at the old code and the new code and really see what the behavior difference is. Because I didn't write the old code. Yeah, I didn't write the old code, I didn't write the new code. And so I'm really relying on the test that I wrote and then also the tests that I had Claude write for me. Because what I want to do is I want to say, hey, hey, Claude, here's. Here is. Here's a class, write 30 tests for it, and then I can look at all the tests and say, yes, those are correct now forevermore, I know that that is going to stay exactly the way that it should. And I can churn that class 50 different times. And as long as the test pass, I still feel pretty good. And so I do think that, like, if the, if the machine is going to write code for you, you better have some way to prove that it's still doing everything it used to be doing. Because, boy, it will just forget about stuff you told it to do. It'll just forget that, like, hey, when you encounter this, make sure you do this. And then you ask it to do something else, it totally forgets the first thing you asked it to do. And so having those tests like that is how you ensure that your software works in the long run.
[00:50:28] Speaker A: I'm glad you mentioned that. I wanted to get to qa, but want to respect our time. Yeah, it's a thing that we think about a lot. Right. It's like, how can we extend our AI usage to qa? I think it's harder because the Kelly, the gal on our team that does most of our qa, she's thinking, how can I break this? How can I have this edge case? How can I? And I think that tying. Maybe, maybe it's that architect level thing again. It's like as we're thinking of these tests and the coverage of these tests, it needs to be somebody with that mindset which isn't always a developer. Sometimes if you have QA on your team.
[00:51:05] Speaker B: No, it's not. And I think before. So like, if you have an existing code base and you're about to embark on letting Claude rewrite some stuff for you, I think the first thing you do is every time you're about to like rewrite a section or a chunk or a class or a method or whatever, you take it to Claude and say, hey, can you write 10 tests for this method as it exists right now.
[00:51:26] Speaker A: Yeah.
[00:51:27] Speaker B: And then you commit those tests to your code base, make sure that they pass, and then you're free. Have fun. Like, go muck it all up and make sure the tests still work. And you're kind of ensured that everything is still going to work the way that it used to.
[00:51:41] Speaker A: Tab, tab, tab, ship. Right?
[00:51:43] Speaker B: Tab, tab, tab, test, ship.
[00:51:45] Speaker A: Yes, exactly.
Love it. Awesome, buddy. Best place for folks to reach out and connect from here.
[00:51:53] Speaker B: Twitter, probably. Twitter Aaron D. D as in Daniel Aaron D. Francis.
[00:51:58] Speaker A: Awesome. Awesome. I appreciate it, man. Thank you very much.
[00:52:01] Speaker B: Yep. Thanks for having me.
