Gemini Preview of Live Coding Agents using Voice (Vibe Code Future)

Hey there, I'm Parker Rex back with another video. And uh shout out to Splash, one of the guys in our community who's like, "Yo, your intro, you got to jazz it up." But I want to test something. And you have to promise that you're not going to take my secret keys. Just kidding. You don't have to promise that cuz I'm not an idiot. But we're going to test something today, which is can we use multimodal? fancy word for this thing watches my screen. Can we use Google to debug? Because I'm sitting here and I'm like, okay, I'm I'm adding a bunch of stuff to my YouTube upload pipeline and I really need it because I'm going out of town tomorrow and it was working, but I keep adding stuff to it, right? That's just how it works. You get it working and then you're like, oo, what other stuff can I do? because I didn't realize that there's other things I can do like I don't know these custom thumbnails. I didn't really think about the scheduling aspect and passing that. I didn't think about tags which I've never even had in a single YouTube video. So I'm like, all right, let's add that. But instead of doing it the normal boring way, I want to instruct Gemini Live. So let's give it a whirl and see what it does. So, what I have here is this session for Gemini Live, and we're going to give it a whirl with just the only model they have. So, yeah, I'm going to prompt it with something like this. And then I want to plus up the prompt and I want to give it the context of my project. So, let's start by going into Vertex. I always tell people go and make the prompt better using the stuff that they have. So let's just duplicate this tab because we're already here and we'll go into it's already streaming. How about that? Hello on every visit. Yeah. So this is the live stream of what the AI can see which is augment for debugging. It is cursor and that is it. So we're going to go we have two of these. Now, this tab is recording. We're going to go into our create a prompt tab. And I guess I have to switch out of multimodal. Or is this in it? This is extract. Okay. So, we're going to switch models because this actually changes what quality of prompt will get generated. So, if I kept it on the lower end one, I mean, why would you want a Honda when you can get a Ferrari? So, let's pick the Ferrari. Open this up. I'm going to put this here. And let's see. Then I need to get some context. So, in order to get some context, I'll make this a little bit larger. And let's go into dark mode. It's better. Uh, let's see what would be good for this to have. Probably some bit of this. I have a bunch of stuff that I'm trying to get better at documenting as I go. So we have the testing architecture, the usage, I don't know the context window size on the model. So let's see rate limits. Yeah, I'm just trying to figure out the context window will dictate how much stuff I should give it. So let's see if we can't find out some info. context token. No, that's not it. So, let's see. Let's grab this the model providing a response 18 months as a GA. So, we want this and let's put this in context window size 1 million is it? 32,000. Is that what I saw? Default session length 10 minutes. So, I'm not sure if I want to burn tokens on this because 32,000 I'm going to get up 1/8 of the way through it just off the rip. Not great. So, let's close this. Oops. Let's close that out. Let's take a look at the parts that would be helpful. Probably this guide. So this would be helpful. This is how the product works. We have main, it initializes. And then it handles these requests. So we end up with all the assets and I just added the tags. And now I want to add the uploads, all that. But let's just try something. Let's see what would have happened because if it can read everything on the screen, then maybe I can just do that and just show it. So, we have Ida. Let's get Puck. Puck seems right. What I did find interesting here is this grounding your data because what I could do is first flip that. Oh, only Iowa. Lame. What I could do is if I flipped on grounding your data, then if I selected a rag engine and I had a corpus of my own project, that could be cool. problem is is why would you want fast rag on that? Like you're basically just doing the same thing. But if you had like a bunch of code, I I could see there's probably a use case where that makes sense. So that that is a cool thing. So let's go ahead. Yoda rip. Hey Puck, I need help debugging something. It's actually just going to be starting by moving all of our credentials that are spread across the project into one place. And I'm about to show you the different relative paths. If you can see at the bottom left, we have a credentials for/service_account.json. We have another one secrets. And then we have a third one that's in our docs folder. So one's in docs, one's in credentials, and one's in secrets. And I'd like to have all of them in credentials. What should we do? Okay. To consolidate all credentials into the credentials directory, you would need to one move the credential file from secrets slash to credentials slash2 move the credential file from docs/tocredentials/.3. Update any code that uses these credential files to point to the new location. What about is there a way to tell if that credential is being referenced from within the file? Similar to how in VS Code you can have a setting where I can see the references in line the number of times that that function is called. What is the equivalent for JSON? To find references to the credential files within your project, you can use GP command. For example, if you want to find all references to credentials/service_account.json, you can run grep- r credentials/service_account.json in your projects directory. So useless command. Okay, sorry. No. Cool. That was a bar trick. That's that's not it at all. Um, yeah. I don't It seems like it needs to do both, right? Like that would that'd be kind of cool. But yeah, I don't that sucked. So none of that. What would you even use that for? I guess. Yeah, I don't I don't quite know. Like could I have it? Let's see about this. Let's Let's change the session. And I'm going to do this for learning how to make thumbnails with pillow. So let's disable the camera. Let's enable the camera. Let's put it onto this. And that should be the full window when I go away. Now I have it here. Cool. And then when I come back, we're going to hit start. And then we're going to immediately go into pillow. Hey, so I'm going to be using Pillow to construct thumbnails in a three-step process for YouTube. Where on this page would I go to start learning about that? Well, what do you think? Okay. Well, you know, they said it's in beta. It's definitely in beta. Well, no, it's in preview, huh? So, that's that's not that that cool. Let's go to do something else. Google Assistance. I talked about this in our community, but I just want to highlight how bananas it is. So, let's compare a couple. Pyantic well known in the space. I've been building an uh like different agents for quite a while now. And before if you wanted to make them, you basically have to go and pick up one of these libraries. And then once you've picked up one of these libraries, you pick up another and another and another and another because Pyanic alone doesn't have a database. It doesn't have rag. You have to go get pine cone and then you need to store your sessions and you got to roll that and then you got to containerize and you got to do that. And so you're seeing Google coming out the gates with just some absurd stuff. So, I attended someone's call last night and took some notes on it from what I thought was interesting, which is basically that they're built to be deployed. And having looked at the code, it's very simple. And there's scripts for it. It's still required to be technical in order to make these agents, which I don't think is ridiculous, but soon there's a guey coming and you'll be able to click around and deploy an agent. And they do the whole process for 11 cents per hour. Think about that. 11 cents. Nuts. And it's like a lambda function. Meaning that when it's not running, it's free. It's idle. When it's on, it's 11 cents an hour. Whereas in this stuff, the other world, you have to go and piece all these pieces together. So having rolled basic rag in the early days when these vector DBs were just coming out, I mean it was a pain just to be able to ask a question about a PDF. Now you have nice stuff such as PG Vector that out of the box on Superbase you can just go ahead and get going and now it's even easier. One thing that's missing from the ADK so far is just custom code, but it's built to support adding something like that real easily. So, it's goated. Absolutely ridiculous. And I'm excited to see what's to come. I'm excited to start building some of these. And I wish so much that I think this is where this will go is I always say that I teach people how to talk to their computer, but we just saw me actually talking to my computer and it's not controlling anything. It's context window. I don't know how that works because if it's 32,000, but is that on top of the way that it's constantly picking up new frames as I move the mouse? Is it getting the XYaxis and then running it against this? Because if you go and you see Gemini plays Pokemon on Twitch, we can watch the AI try to play Pokemon. And I would own this thing in Pokemon. Like it has to burn 122,000 tokens every time to like figure out where it is. And you can see it's stuck trying to move the scientist because you can't move down. Like he's stuck against a door right now. This guy's getting mad. We got a fight going on over watching people. I'm watching people watch AI try to play Pokemon. That's That's my Wednesday night. Where are you at? But yeah, I guess maybe I should look and see if there's any questions because this video is a daily channel and I'm supposed to do that. So, let's see if there's any questions. Oh, we hit 800 subscribers. Hear that? That's pretty good. That's a real live girl over there, everybody. Perance Basil. And uh you should go watch this video that I put out. It's probably one of my best videos on product strategy, but like we got a bunch of jabronis that are disliking it. Well, just one jaboni. But other stuff seems good. It's going up and right. Let's get over to the community. By the way, how do you h how do you maintain the mental stamina needed to keep piping stuff? Well, you should have see me today. I leave town tomorrow. Baze. Was I tired today? Yeah, it's You hit a wall. I hit a wall today, but guess who's here? Guess who's back. you hit these little walls and a lot of it's just mental, but for me, I had to produce a significant amount of content and then just to keep the the ball rolling. But I could do it while I'm out of town, but that's not fun. So, I just reframe these things as challenges and I don't negotiate my goals. So, my goals matter a lot. And I also recognize that I'm only six months behind the biggest creators slashinfluential, not influencer, the influential people in this space. So why would I negotiate my goals? I just wouldn't. Now, you definitely as you push, you find like limits. So let's say in coding, you're trying to get good at coding and I'm 18 years old and I'm trying to learn to code and I'm like, I just don't have it in me. And then I blame the code. I'm like, "Coding's stupid." Like, I'm more creative anyways. You come back three years later and you're like, "Wow, that was wrong." Like, "I can figure this out." And then you hit the another wall and then you quit again. This is my story where every few years I would try and learn how to code. And it was always this insecurity that I had where I'm like, I can't do it. And then you can you can learn literally anything. And how it plays back into the mental stamina thing is cuz it is mental. And when you realize that you just keep going. Now you do have the diminishing returns on your sleep. So I know that very well. I actually published an essay every single week on mapthemap.com. So it's really ironic. You know, it's like I'm not Brian Johnson, who I also think is controversial because he claims, "Oh, you can sleep all you want, have perfect scores, and then you see a talk of him when he was actually printing the money and he's like talking about the Diet Coke and the cigarettes, and you're like, dude, what?" But I definitely know about this, and it's like basically, cool, my brain is a three and a half pound supercomput that requires charging. Charge or suffer deathly consequences. Cool. Clickbait. But yeah, point is brain's a supercomput. You got to recharge it. And currently battery is low, but I've prioritized again the goals. And I know that this is like a short sprint. It's getting stuff off the ground and pushing me to then have the confidence to do more and more and more. And when you have a constraint, my constraint is 10 pieces of content across the week and I can read them off. But it's like at least work work on at least one high ticket contract of a minimum of 30k one main channel piece of work per day to output three across a week seven across a week for daily which literally is just framing like what are things that I need to do myself and then I just record it and then some school assets or vibe with AI which is becoming bigger than just school. So there's a product there engaging with the people and then OSS contributions. None of them are actually open source yet, but most of what I build will be open source and they all a lot of stuff that's fun for me. So it's like I don't sit here and like cringe. These are things I enjoy. So that's that's a really long way of saying that work on stuff that you enjoy and it it won't be a pain in the ass. Back to the questions. Turbo starter kit. Turbo. Turbo. Turbo. I should release [Music] that. This is the perfectionist in me. I've made this already and I just don't like it. And I don't want to give people something I don't like. So, let me reply. Oops. Let me reply to this guy. Yo, new yapping about this now. Bid goes out tomorrow, I think, because I've been on a content I don't even know what's going out tomorrow. Uh yeah, that goes out tomorrow. Cool. Other channel I'm sure is going to be just fantastic. We're about to hit 6,500. That's cool. Let's go to community. Bring the haters. Just nice stuff. Hooray. I'd really love a tutorial on how to use Taskmaster for debugging SLR RCA if that's even possible. Every time I try, I end up buried in subtasks inside subtasks and it quickly becomes a mess. I really love the videos and really appreciate them. Thank you, sir. Seven likes. That's a lot. Seven. One of them is Basil. Yeah. But I think this is what I'll be putting out on the main channel more is it's funny. If I make a video about Taskmaster Pops, if I make a video about how to think before you use Taskmaster, no one knows that that's actually more important than Taskmaster. That's the thing is maybe I just frame it like half clickbait people because they go and I yap about this a lot, but I'm going to yap about it again where you think that you're going to get handed this like let's use the rubber duck per us. We're going to use the rubber duck. I give you this and you're like, "Oh my gosh, I need the duck. I need the duck. I need the duck. If I just have the duck, I'm going to be able to build the next Facebook. I'm going to be so good. The duck is representative of a product requirement document, which is incredibly ironic because they were not popular. Like I used to write these every single day and it wasn't in vogue, but now that you need it to make something, everyone wants it, but they don't know how to write them, and you have to put some real thought ahead of time. So I think I just need to make more videos and then frame it within Taskmaster that would probably ultimately like teach people about product management, product strategy, which is a mixture of if you had three vertices on a triangle, one of them would be design, one of them would be business, and one of them would be development. And so business is kind of all-enccapsulating. That's why people called me literally the dev whisperer because it's like the dog whisperer because I could go and speak the tongue of the nerd, you know, and then flip into, okay, cool. How's this apply to the business? Meet with the ops guys. Boom, boom, boom, cool. This is how roll out's going to go. Meet with the sales team. Okay, this is what you guys need. Good, good, good. How much is it like? How much how much uh business is it going to bring in? Blah, blah, blah. Go to the design team. Design team's like, we don't have the drop shadows. and you're like, "Chill out and get you your drop shadow and meet with the devs again." So, it's doing all those things, but just being like a person of one and trying to learn that and trying to map it to the thing you're trying to build and trying to ask the questions of like how do I make this thing awesome? That's really the skill and uh it's hard to hard to explain. I literally made a 27 video course on it after we sold the company that I was a part of. And I think everyone in the community, if you're watching this and you haven't watched that and you're complaining about it's not good, please just go watch that. I worked really hard on it and you will learn a lot. Like I talked to someone the other day and they're like yesterday I talked to someone yesterday and they're like, "Yeah, I pretty much figured it all out." um about to push it out, but I just don't know a couple things. Like I just figured out what GitHub is and I don't know like how do you do payments and also like how do I get it off the internet like onto the internet? I'm like dude you're not even what do you mean you figured it all out? How do you you can't get it on the internet? And I don't say that but that's just a lot of people just don't know what they don't know. And so it's kind of coming to me now that a lot of the stuff that I find easy, it's because it's just sets and reps. So I need to just write out all these things and make the content. But I'm yapping. I'm also Parker. You should like the video. You should subscribe to the video. You should join the school before I double the price and roll out a school competitor. All right. I'll see you in the next video. Like