I Made AI 6 Agents to Be My Dev Team (Kinda Legit TBH)

hey there I'm Parker Rex I led product and Technology at a startup that did well over $50 million in sales and on this channel I'm building a $100,000 a month AI services company in public and I cover a bunch of different things every single morning this video goes up we cover AI news things that I'm using in the field some commentary on it from someone who actually knows how to develop these tools we cover some strategy and then we do part of a build if you want wanted to see the full build there's a different Channel called Park re builds but we've got a lot of stuff to cover today there wasn't that much going on yesterday but wow today's today's a different one so exciting first of all open AI voice came out yesterday with a huge update if you're unfamiliar open AI voice it's uh two things one is the text to voice so you type in something and it starts talking and then another is voice to text a common use case for this is if I had a YouTube video that I wanted to turn into transcription so let's say I wanted to take this video of Alex Heros about building brand and I wanted to have it turn into a transcript so that I could make a blog post maybe I want to do whatever it is with that you would use I have a workflow so I click on workflow I would do a vid to text and you can see I have whisper down here and then I type in the name of the file and then the language that I want it written in and then the output format F format as a flag and then obviously the extension which would be text so that's an example of what a model could do on the video to text and then there's the other way of doing it they came out with the new one yesterday that's the same price let see it's the same exact price as whisper except it's mini and let's look at it so it reduce the transcription error let me just make sure the camera's going good yeah so reduce the transcription error and latest speech to models on flours what does that mean well it's basically that it's not a huge difference if you're only using English but you can see in other languages like Bengali and morati all these other ones they got like insane benchmarks um that's one part but the actual sort of features in it are really interesting I believe I bookmarked them or liked them so I can just pull those up but the ability for you to quickly build voice agents now that's the main thing they're making it a lot easier for developers to make voice agents you can also see they have custom turn detection it uses the content of the speech to tell if the users is done that's very different than how it's been done historically I had a client that we were building a multi-agent real estate platform and so what it would do was you'd have a bunch of jobs around what a realtor has to do they're pretty low leverage but they're really important meaning you don't have as a realtor a lot of Leverage on doing the outbound or dealing with the inbound across phone with text MMS email or actual voice calls so what we were working on was building different agents that had different tools that could handle all these things and they'd kind of work work in Orchestra and it was an engine behind the scenes that had to listen and look for pauses to tell if the person was done talking and it was kind of tough and you're still using libraries but this this is amazing and I can't wait I think I'm going to build something with this for the builds Channel which we're going to be doing basically right after this I'll once I finish filming this video I'll go straight into a second video where I'm building in public on the things we talk about this video but yeah it makes it a lot easier and you can see all the documentation here I'm G to pull that up and then I'm G pull up this tweet that I made which you can follow me on on Twitter or X I post a lot of this stuff as I find it so this is crazy so AI or sorry the agents SDK came out about a week ago and it made it a lot easier for you to sort of bundle up what agents are capable of doing with file search web search the file search would could spin up a vector DB behind the scenes it's all like in there right it's this nice package it didn't have voice until yesterday so if you have a websocket server set up you're defining what the we socket endpoint is that opens up the direct connection between you the client and then the server the thing running it and then it'll add the trace this will go through and add some more logic about the text based messages then if you wanted to add audio you don't have to go and use a third party you don't have to do any sort of calls to anywhere instead it's just this these eight lines of code then give You full access to the new models and so let's take a look at the API docs because they also just made them look really good if it will load okay so if we go into here and we scroll down to where would audio be well probably would be an assistance because let's see audio we want to just see all actually let's take a look at this models what it be in docs sorry about this guys yeah it's in docks this is what I was looking at yesterday and it just looks really good so obviously we have GPT 45 that's what they say is the largest and most capable model but then when we go down past the reasoning models past the flagship chat models we get into the text to speech so we've gone from whisper one whisper two I don't know if there was a whisper three but to the GPT 40 transcribe so not text to speech sorry uh GTP GPT 40 transcribe try to say that three times fast that's hard can you say it gp40 transcribe that's first time that's pretty good so anyways there's two of them and they are really powerful and really cheap so the new one is the same price as whisper but better lower lower air detection but I think really the sauce is in the agents update so I'm excited about that I haven't seen any demos but what you could do if you wanted to see if anyone's built a demo is you just grab a snippet so what I could do is I will open up this is just a little trick of the trade that I've learned from having built a lot of stuff is I'll open up the website repo and all the resources for what I talk about in these videos are on the website park.com blog and can click the little daily updates button and so if I go in here and I wanted to take this block of code because someone's just going to take that and I just typed that into GPT using companion and I say give me this text formatted in Python so I'll say that it's going to run an OCR on it and I'll get back that block of text and what I'm doing is I'm just taking that and maybe there's like one line in it that I know is exactly what someone would paste because you don't want to do the whole thing but I take this and then I want to have oop I don't need that I want to have some nice stuff for syntax so I'll just change the language mode to python cool and I'll grab this evate yeah I'll put equals a weight voice Pipeline and then I'll go over we're done here now I'll go into GitHub and this is you can do this for anything when you're trying to see an example of how someone built something with a tool you want just type this in so I can see there's 91 examples of people in it's been less than 24 hours and now I see all the stuff that has to do with this so class voice pipeline looks like no wow no one's really opened up the floodgates yet this is cool what about this when was this published maybe yesterday yeah cool so that's how you do it you can see that they made updates probably based around this stuff no not so you would go and you just continue kind of pick out a piece that you think would be in there so this is probably be better I'll grab this because now I want to prove a point and I'll go back into GitHub if you're not using hot keys on Chrome what are you even doing command shift day gets you everywhere is audio complete oh it's because I have this repo let's do a search there man no one's posted stuff this is crazy so no one okay because it came out less 24 hours ago I guess no one yeah there we go I'll be the first one to put something out so that is the open AI stuff same price now I want to talk about mcps they're is a bunch of hype around these and they've been out since November but they're getting a lot more attention now so let me see if I can't find my bookmarks I have no clue how to find bookmarks in Reddit I should know how to do that Reddit saved okay oh I wasn't going to mention this but it's here this is cool someone built they trained flux the image generation model on claude's aesthetic so if you know Claude and how everyone so hype about it they just took all that stuff all their brand imagery and then fed it to flux the way that you would do that let's just show you is they have it hosted on on hugging face so if you wanted to you could just go grab this you could try here but you train a Lura with 15 images using this app called glyph pretty crazy so if I go back now where I guess I didn't have it saved huh let me think it was basically a really good explainer for how mcps work so naturally I wanted to to show you it because a lot of people get confused oh I found it okay so mCP architecture in short mCP standing for model context protocol allows you to have the puzzle pieces in place that you can connect your llms to other tools now a classic use case would be I want to send an email so I'm going to type in to either Claude desktop app or cursor those would be the clients I'm going to type in hey send an email to myself about a lesson I learned and you'd put in the content instead of going and actually doing that it would just reach out using an mCP um connector essentially or an mCP server that would be running to Gmail and it is that kind of bridge between the two so that's in short how it works but I wanted to talk about it a little bit because it is insane how it's going to like essentially speed up the ability for for non um technical pieces of software or nonone it allow it basically speeds up the gap of connecting Ai and getting good AI experiences for tools that won't develop them as quickly as all the tools that we use as developers because we know that the pace for new tools for developers is absolutely insane so you see a new cursor you see a new when surf you see a new warp you see every like every every week right we get a new version of all of these then you have the hypers shippers for example Klein where it's every 3 days and there's this scale right as we go down it we get to GitHub and Microsoft ship something new call it once a month and these are all based around co-pilots or agents something to that tune and as we go even further so these are all Dev tools right but then if we cross that Chasm to non-developer tools things start to get slower and if if you have blender which we're going to set up together in a second blender is 3D modeling software really complex lots and lots and lots of menus so I have blender here and I wanted to make a cool logo for a 3D logo logo for trouble-free AI a paid Community resarch lab that I wanted to make something cool for that but guess what I don't know how to do this I can click that it's not moving I I don't know what's going on and then look at all these crazy menus isn't that insane this is a whole profession and what I'm predicting is mCP will speed up the learning curve for anyone that wants to use the tool so that me not a prosumer just a normal consumer can come in and make something simple not going to make the new scene for Minecraft or whatever but I'll be able to get something basic and that's with an mCP so my prediction is that all those menus if we look at another example maybe I don't have Premier Pro open but that would be another really good example of a really complex pro tool that will be augmented with an mCP and in the case of blender we're not going to see a co-pilot for three to five years that's my belief same thing goes in the music industry Ableton Live a lovely piece of software I've used I've probably put 40,000 hours of my life into it you probably won't see something for four to seven years now that's partially because it's just not in the same industry it's partially maybe because it's not a priority maybe because it's culturally not aligned with what musicians want there could be some backlash but it allows you to make some things and so if I go and I type in if I go to my Parker Rex builds Channel which right now it's called explains but by the time you watch this it will be builds I released a video this morning and it is talking about using the mCP for Ableton so you download a file you put it in a folder Moonlight Sonata 16 bars 140 BPM Tempo 909 drums and I don't need to play play the whole video but I go into the fact that this is totally doable and that someone will most likely develop an extension something that augments it from a third party you saw this with xcode and apple I was a swift developer building apps for my company map for about six months and cursor was so much better than using xcode and apple still hasn't shipped any sort of AI features so everyone's moved over to cursor but before I did that I just used a third party that was called co-pilot for xcode that someone made that happened to be an iOS Developer and wanted it so there's probably going to be some musician that's really talented developer as well and they'll build something the blender mCP I'm not going to scratch the idea of doing that because it's going to take a lot of time and I'll just put that on the builds Channel but I'll just show you what it does so you can bring in high quality assets to blender just through prompts because they integrated with poly Haven which is a 3D asset Library so you get access to 1500 assets and he's just jumping in saying make me a beachy scene and you can see every time that it has this grayish bold text it's calling a tool and that's how it works is each one of the menus when you are going in and let's say in blender if you're going in and you're clicking on ADD and then you go to mesh and we want to add a cube and that was a tool that we did and behind the scenes Claude is given a set of tools they're well defined so if I go into cursor and I click into mCP I can see I have four different servers that are all using different um sorry I just paused now I have to rotate the password on my database because I just noticed that that is there so hey Siri remind me to rotate my password on a database in 45 minutes cool but it allows you to have access shut up okay gives you access all these tools they're well defined they go and they do things cool that is mCP and if you want to learn more about it you can also jump in their Discord they have a watcher that shows you when new C mcps come out and there's just so many of them it's absolutely insane I'm going to be going way deeper on this stuff because every industry can benefit from mcps based on the quality of them and so that could be a service offering that you have I should should write that down to introduce mcps to non-technical Industries as a way to save time money or generate more leads cool so next if we get back to this co- pallets everywhere I wanted to talk about structured vers non-structured offers we're getting into strategy as a reminder green equals strategy so when I started doing AI Services when I pivoted away from doing SAS full-time because it was a very expensive Hobby and I needed to fund that to continue development I decided to do very specialty like specialist Dev work and it can pay well right you can make 20 upwards of 25 almost $25,000 a month or my case was about 20 a month doing that but it's not a great High leverage thing so then I was trying to think okay well how do I get more leverage on my time and in order to get the most leverage on your time you have to find things that you don't start from zero when you do another piece of work for a client so that's one part and then the second part is actually the delivery of it so coming in and having structure around the thing that you're selling it makes it a lot easier both for you and the client because if I come in and I have this really like messy kind of all over the place offer where I do this I do that I do this I do that if you don't deliver that well if you don't explain this it's almost like the the cartoon character that has the trench cat on and they open it up and they have all these things in there well if all the things which are your skills your hard skills that you're selling aren't well articulated and don't have clear price points on them and clear expectations deliverables it's going to be really confusing because you can have all those skills which is great that's how I feel I have a lot of different skills I can do pretty much everything on a computer but if you don't say okay this is the skill we're going to leverage if you don't leverage that and then package it where it's hey we do content automation for SEO we make sure that the structured data is in place that you have a Google Knowledge Graph if you don't we do these couple of bullets within that and this is how much we typically charge clients that is much better packaging than saying we're going to do SEO and not having a price and then Reinventing the wheel because if it's structured and in the case of SEO so let's say we do AI SEO optimization slash content creation like even this as I'm doing it like these are two separate things so you could maybe split those maybe you sell them as a bundle but you're going to be doing both because if you're making content and you're posting on a Blog if you're not SEO optimizing it then you're an idiot so then you'd say okay we're packaging this and we charge four 4K or 4500 for this we do an exploratory Milestone before we're after our kickoff call we're going to go we're going to give you access to this stuff so that you can go and do it I charge $500 for that but if we land on the proposal I'll refund you that we'll use it as a discount towards the project delivery so then it ends up being four right but they like they've won because you gave them the discount and you're not doing free work so it plays into a lot of that stuff but that' be an example of a structured offer versus non-structured where I just thought of talking about this because I've done it non-structured it's a pain for me it's a pain for the client you have change request expectations aren't set right ends up being annoying I also had another client that I'm going be working with and it's not clear what the structure is because the maybe the client relation is really good so it's just hey I trust you go do stuff I want you to explore what's possible well with something squishy like that where it's I want you to explore what's possible with AI you could still put structure to it so if you had the conversation they're saying I want you to explore what's possible okay we're going to introduce a $500 exploratory Milestone and during that I'm going to go and I'm going to sift through all the different assets that you have so maybe you have a website maybe you have an email funnel you have all these things and they're chained together to work in unison as a conveyor belt for generating cache once I go through all those and do the audit at that point then I can come up with a strategy and then I put the structure together so then you come back and you say hey these are the things that I think we should do I have a list of 40 of them and this is the one I think we should do first this is how much it's going to cost and you name it having name is important then it is all set and everyone feels good about it this is really exciting this might be this will be the most important part of this video so six code agent what does that mean well think about when you're on a team Who's involved with delivering the product who does what what's their job description what are their key expectations what are their deliverables well that's what I'm doing when i'm mimicking and building out systems for multi-agent coding workflows I built them before they were okay I think this is the best one I've ever built now how did I do it well it's just what I said where I think of what are the different roles on the team this was actually about the last part this outcome Focus there's a channel called automations by Jack he runs a paid Community it's mainly no no code stuff so very different but in the case of a team when I have a new idea and I was a technical product manager for six years typically when you have a new idea it's starts like this you have a TPM technical product manager they figure out the what and the why of the product so each rle has a box around it it's number one now that TPM they go and they think about it they're outlining the what and the why inclusive in the what and the why is the problem statement why are they doing this it'll look the same if you're going from 0o to one or one to 1. n they're figuring that out they have the problem statement they have the solution statement maybe some breadboard drawings where they're just sketching some things out with a fat marker and then after they do that they'll meet with an architect so yeah the solutions architect is typically the best engineer who's not too caught up in the day-to-day probably managing some people but Nosa systems like the back of their hand then they figure out the refined version so let's call the what 2.0 the why 2.0 and they figure out the how 1.0 so they're going through this and why it becomes what 2.0 is because they may know stuff that the technical product manager doesn't they will if they don't they're bad or the TPM is really really good but I knew that when I met with my architect that I was going to learn something and that what we were building was going to be an expansion on what I had initially thought and why we were building it typically the same but it gets refined and and more context is at it so it's stronger after that you'd go to an engineer or maybe a team in our case because we're mimicking we're mirroring real life roles with agents well you're just going to treat as one because you can clone those if they have different stocks you can modify it slightly but then you figure out how to point out so we've gone from here to here here to here and this is typically what it would look like and this is just for new features we're not talking about testing them we're not talking about doing a workflow for fixing something that is the second one that I did so let's call it new bug we're going to go through this so when you have a new bug come up you would typically have an architect or somewhat it's basically like an incident response engineer it's like a fancy way of saying it but I'm only saying that because that's what I Define it as when I'm making the agent but it's really just who's the guy who's the most qualified the guy or girl that's most qualified to go through they're tracing logs they're doing a root cause analysis what happened how do we prevent this they doing sequence diagramming this ideally is already stored somewhere ahead of time but if they're not they're going to want to do this to help kind of problem solve a really good engineer May skip over some of these steps because it's just internal and they want to get it out but the beauty of leveraging AI for these is they probably maintain the same level of speed and get more output and more documentation no one likes writing docs so in a lot of cases in a startup it's just going to be one engineer doing all these but in the case of Agents you can specialize and you should specialize so then we'd have the I'm going to use IR as instant response so I'd have an IR architect and it's doing something similar to the architect above except it's taking in this and this so these will end up being out puts and I'm putting little tags around them because when we come and talk to them well they're in real life as a human taking in these inputs and then reading them whether or not it's a different person they're still thinking okay I have this in context I just made this root cause analysis in my head I I traced through where things are going how this data flows what's Ingress what's ESS and they're doing that right so in this case the architect takes in the sequence diagram and the RCA or root cause analysis they take in the sequence diagram probably using like mermaid and then they come up with the solution right so it will call it like a solution and then third you'd actually have the IR engineer and they're going to take in the solution they're going to go and do the step-by-step instructions so that solution looks like a step-by-step set of atomic steps of instructions and I'm going to be using this it's not perfect yet but it's really good so in the case of the new features we have step one the technical product manager going to be binding these to hot keys so I can fly through as I'm doing things you're a senior product manager cool your task is to create this comprehensive product requirement doc using these tags and I mentioned that I should use voice here because I want to Yap I want to say these are all the things this is how we're going to do it or sorry this is what we're doing this is why we're doing it I can touch on how because I know and most PMS if they're technical they'll try to figure out how Ahad of time but you're going to want to fill that suck up then it has all the steps on how to do it then it takes the output of the of the PRD and then brings it into the solution architecture role so step two and I have examples of what good steps look like versus bad steps bad steps if there's any sort of infrastructure set up would just be like hey go like do that and do that oh no I need step by step you put the PRD in there technical product requirement Doc and then also code base because you need that if it's contextual so certain area then you just put that in there maybe with if you're using AER you could do a wild card but you bring those files in and then third would be the implementation engineer I started using this yesterday and it worked so so well I was also really tired so I I haven't completed more work on it then when it comes to debugging stuff it's same sort of setup we're working in threes here if I ran into something I do exclamation point B1 I'm going to get that RCA from them that RCA is going to be handed to the solutions architect I do B2 paste that in there paste in the sequence diagram it has its key responsibilities and its output which is the step-by-step set of in or sorry the solutions architect document and then they take that in and I think that's step three yeah there we go and then they go and they execute what I think is cool here is this set of flows could be literally a text box and then it could be an extension in in GitHub I really think that's where it's going and I think that node-based code editing is going to be a real real thing I'm going to be trying to build this in NN over the weekend you can check out par X builds for details on it and I'll also be posting a bunch of updates in the school cool um yeah so V1 of that would just be using the text expanders but I do think in node base makes sense the question is is can you have an n8n can you have something that's like this where it's if I have node one which would be step one node two which is step three node or sorry step one step two step three for the different agents in the case of the new feature thing can I have the code base actually run the diffs meaning like can this be the code base I have no clue that's what I want to figure out so that I can go back and forth between these I think that that's a magical idea let's see what else I think that's about it for today I'm splitting these out because the builds would just take too long and it's not really the right place to do it on this channel this is about strategy what it's it's going to be more focused on but yeah a little bit just to close you would have these kind of trees of logic where if it's a feature then you'd run F1 you'd have human in the loop so I'm looking at it you'd have F2 human in the loop F3 straight to GitHub afterwards this would actually be a new flow so you you see entire multi-million dollar companies that VCS are shoveling cash into that just literally do this one flow this is probably four or five agents working in Orchestra to deliver that so that would be something that you could go and You' reverse engineer what are they doing how are they prompting and um this was based off of some info that I pulled off of the I want to say the Klein Discord someone made this really cool loop basically for debugging where identify the bug describe it in detail maximize observations you have an analysis agent and a validation Loop that's what agents are by the way this is not my definition this is anthropics definition but agents are tools and looping it's pretty much it tools are the things that you saw with mCP they're functions that are basically really well defined trying to think of the word right now but it's escaping me that was why I paused if you enjoyed this video if you found it helpful make sure you like the video drop a comment below I'd love to get some questions because then we can start the video off with questions if you want to go deeper on this stuff you can join our research lab over at trouble free it's going paid today you we're going to want to lock in the price this is just how it works right I can't make awesome content for that without funding it and uh it's going to be around for a long time we're following dunbar's law which states that you can't have more than 150 human connections that you're close with so we cap it at 150 and uh it's $39 today it's going to be about 150 soon so yeah I'll see you in the next video AKA tomorrow on Friday March 22nd see you then