You NEED Perfect Memory Remote Coding Agents ASAP (Build your fleet of AI agents)
June 5, 2025
Parker dives into the first principles and practical setup for building a fleet of Perfect Memory Remote Coding Agents (Augment) to research, spec, and execute work across a codebase. He shares hands-on setup tips, workflow patterns, and a concrete use-case to show how to scale with an army of agents.
What perfect memory remote coding agents are and why they matter#
- Augment remote agents combine a strong context engine with proactive and reactive task execution.
- Conceptually: treat a fleet of agents as "an army of interns" that can research, spec, plan, and execute work across your project.
- Two main flavors:
- Cloud remote agents (remote workspace, ongoing tasks)
- Auto agents (in-product avatars that act inside your environment)
- The goal is to reduce back-and-forth, surface context automatically, and enable scalable AI-powered work across the software development lifecycle (AI SDLC).
Setup, environment, and integration#
- GitHub integration: Augment works directly with GitHub (no MCPS middleman), which makes setup more reliable.
- Environment bootstrap (per-agent workspace):
- Choose a Debian/Ubuntu VM setup.
- Install system packages, Python, and virtual environment tooling.
- Install project dependencies and wire up pytest for tests.
- The bootstrap process uses a streamlined – and customizable – TL;DR-like flow, and you should create a dedicated directory for these environment setups.
- Branching and mapping:
- Start with a branch pattern (e.g., Auggie as the chief, Auggie-QA, Auggie-UIUX, etc.) to map agents to responsibilities.
- Each agent can be tied to a branch for isolated work; later you can orchestrate across agents.
- UI and workspace management:
- The right-side panel holds threads, with the blue remote agent indicator for cloud agents and the user avatar for auto agents.
- Drag the chat panel to the right to create a dedicated remote workspace you own.
Workflow tips, prompts, and snippets#
- Combine snippets and 3-letter commands to speed up interactions:
- Alt + 6 ties to a prompt snippet; you can chain prompts using keyboard shortcuts (e.g., L, I, N).
- Raycast can host these snippets, but you can also use native text expanders.
- Prompts and shell-patterns:
- Use shell-script patterns to load context and run conditional logic inside the remote agent (e.g., loading context, handling tests, and performing database interactions).
- Think in terms of patterns: load context, set guardrails, execute steps, verify outcomes.
- Example prompt pattern (ticket workflow):
- After copying a ticket from Linear, press Alt+6 + L to paste a structured prompt, then run:
- Move the ticket to In Progress
- Create a new branch:
feature-{ticket} - Plan the work step-by-step
- Load libraries or context as needed
- Update the ticket with your plan
- Do the actual work using the provided tools
- If needed, interact with the database
- Create a PR to merge the ticket
- Ultra Think concept:
- Use a prompt rhythm that pushes the model to "Ultra Think" through the decision and plan steps before acting.
- Enhanced prompts:
- After you’ve validated patterns, use enhanced prompts to reduce ambiguity and improve outputs.
- Practical note:
- Treat each remote agent as a reusable pattern: extract the workflow, then implement it as a prompt template so future tasks can reuse the same flow.
A practical use case: YouTube memberships spike#
- The spike use-case is a research-and-execute spike to evaluate adding YouTube memberships and tying it back to a ticketing workflow.
- Spikes to production flow:
- Define a lightweight YAML/Markdown board (title, description, status, backlog, type) to track tasks.
- Map to a real ticketing system; designate Auggie as the chief and spawn sub-agents for different areas (QA, UI/UX, frontend, etc.).
- Use an AI SDLC approach: one agent per step; orchestration later as the pattern matures.
- Workflow from spike to sprint:
- Start with a spike in Auggie; identify relevant files; surface context from the codebase.
- Create tasks, estimate impact, and outline a plan using an enhanced prompt.
- Spin up a remote workspace (German Debian VPS example mentioned) to implement and test.
- As work completes, generate a PR to merge the changes.
- Realistic orchestration goals:
- In the long term, consider a centralized orchestrator that handles PRs, conflicts, and dependencies across agents.
- Move from manual prompts to a fully automated, context-aware pipeline that surfaces outputs per agent.
- How this maps to the AI SDLC:
- Each agent covers a stage: research, planning, implementation, testing, and deployment steps.
- Branch-per-agent helps keep work isolated and auditable.
Thoughts on future directions and where this could win#
- Separate agents by codebase responsibility (product review, prioritization, architecture decisions) so they can “debate” and optimize at scale.
- UI/UX for multi-agent orchestration:
- A node-like, YOLO-style orchestration UX could show each agent’s outputs and reasoning, enabling quick checks and adjustments.
- Mobile and surface-area expansion:
- Mobile access (e.g., Termius-style SSH in the field) and deeper GitHub integration (beyond tickets) to trigger actions automatically.
- Proactive vs reactive agents:
- Proactive agents could self-heal or self-optimize based on observability data (Prometheus/Loki/Grafana context).
- Observability and governance:
- The more capable the context engine, the more important it becomes to manage prompts, fallbacks, and safety rails to avoid “drift” or brittle outputs.
- Non-fork adoption path:
- Focus on orchestration, modular agent prompts, and robust patterns rather than forking the codebase to avoid forks and lock-in.
- Market-ready patterns:
- Agent templates and a marketplace of proven prompts/patterns accelerate adoption and reduce ramp time.
Actionable takeaways#
- Start small, then scale:
- Pick a concrete spike (e.g., evaluate a YouTube membership feature) and map it into a lightweight AI SDLC with a few agents.
- Use GitHub-native setup:
- Connect GitHub first; avoid MCPS dependencies to keep the integration reliable.
- Establish a clear agent taxonomy:
- Create a chief agent (Auggie) and specialized agents (Auggie-QA, Auggie-UIUX, etc.) with branch-based alignment.
- Build repeatable prompt patterns:
- Extract and codify patterns into prompts; use enhanced prompts to ensure consistent, actionable outcomes.
- Leverage snippets and shortcuts:
- Use Alt+6-style prompts and 3-letter commands to accelerate repetitive tasks; tie them to a single context or project.
- Structure your outputs for actionability:
- Use ticket-like prompts (Linear) to generate task plans, then translate into branches, PRs, and tests.
- Plan for orchestration early:
- Consider how multiple agents could be orchestrated (visual UI, YOLO-like prompting, automated PRs) as you scale.
- Gather feedback and iterate:
- Engage with the Augment team and the community to refine prompts, guardrails, and integration points.
Links#
Transcript
What if I told you there was a coding agent that had the best context engine in the game that could go out and do the research on the things that you're trying to get done to fill out the spec and then fill out the tasks and then also go execute on them. What if I said that you could multiply those and have a bunch of them running? That is what we're covering today. It is the augment remote agent. And I'm really excited to talk about this today because it's actually really sick and I've been waiting. If you've watched this channel, I've covered agents for a while and I always say you need aic workflows and then you go to agents. So, we're going to talk about some of the quotes from the team. We're going to talk about the setup, how to get started. We're going to talk about some workflow tips, some prompts, and some snippets, and then a use case and my thoughts on it, as well as how I think they'd win long term. Lots to cover. Let's jump right into it. So, first of all, this is from the Augment team where I went into their Discord and they had an ephemeral channel on their Discord when they were doing beta testing and I was keeping an eye on it and I was curious. I was like, how are you guys using it? And I talked to a couple different people. Some of them said, "Oh, it's just for these small things." Some said, "Oh, it's the kitchen sink." Like, we're throwing whatever we want at it just to see. And we use remote agents for all sorts of tasks. I use them for small issues that I know I'm not going to get to. But the power users who get the most out of it spend time up front setting up specific prompts, detailed specs, and guard rails. That includes how the workspace environment is set up, and we'll cover that later in the video. So, some of the features when you're comping this against something like cursor or cloud code, this is the stuff that they cover in their blog article, but I'm going to go in depth because I've just been pushing this thing. Little things like kill these flaky tests, get rid of some of this doc debt. Those are nice use cases. Basically, the bottom cortile of tasks is what they're saying to go after and to think of it as an army of eager interns. I think it's more than that. I think they're underelling, which is good. You don't want to overpromise and underdel. But in the background, for instance, I'll just show you one that I have running where I'm treating this as I would a normal agent, right? It's literally the same thing, but you need to make sure that you're mapping it to the typical software development life cycle stuff. So when you do any sort of software development, you know that you need to do the research up front and then once you've completed the research, you're trying to validate this is actually a problem. If you're building something for yourself, it's a lot easier because you're just dog fooding. But there's all these steps that go into it. And so that's what I'm doing here where I'm figuring out a pattern that works with remote agents with some sort of syntax on the actual file names. So you can see I have spike here. So this is me trying to do the research to figure out is this something we should even do? What would that look like? What are the files that it would touch to figure out the level of effort that it would take? And you make a basic ICE score. If you're not familiar with ICE, it's a methodology a lot of product managers use. You should definitely implement it as a developer. It's 1 through 10. How high of an impact do I think that this piece of work is going to have on what we're what our goal is? So 1 through 10, what's the impact towards the goal? What is 1 through 10? my level of confidence that this is going to make an impact on the goal and then one through 10, 10 being the easiest. What's the level of ease? So, it's directionally helpful and that's what we're doing here is we're doing the research to figure it out. So, in my use case, I run a private network of builders and AI. We have people from Microsoft and Google in it and it's awesome and we learn and we share and we build stuff and try to make money with things that we build. And so for that, I'm trying to figure out would it make sense to do a YouTube membership and tie that back in. That's my use case. You can see I set up a branch called Auggie and we're just going through it. So I wrote a really simple spec prompt for it and I'm having it do the research. Like I said, beyond the use cases that they mentioned, that's what I want you guys. encourage you guys to go and whether it's cursor augment I think augment's better at the context engine side of things go and just push these for the use cases because this is where this is going this is so clear meta is already doing it where they have their own internal tools and all the things are trained to do the jobs that the you know mid level a junior engineer was doing a year ago so it says what are you going to try to do now remote agent ticks all these boxes are you just going to ask questions about your code I wouldn't be using a remote agent for this. I guess you could if you didn't want to block it if it was some complex question. If you want to get advice on refactors, sure. So, you just spin these up and I'm treating them and I'm mapping them to different branches. Then you have more use cases. So, quick edicts, paper cuts, refactor again, bottom cortile stuff. I'm really excited about this flow that I learned from somebody in their Discord around combining snippets with threeletter commands. And I will show you. So to get set up, you need to connect your GitHub account. And that's one of the best parts that I like about Augment is it doesn't rely on MCPS to work with GitHub. And I just think that a lot of there's just a lot of bad MCPS. So, by them having direct integrations, it just always works, which is really nice. So, once you get that set up, this is what it looks like. I think I took screenshots of it before. I'll just show you so you're not reading all the docs, but you have this tab on the right, and that's what I'm looking at here. So, you have threads, and then the blue one is the one that is the remote agent. You can see it's idle cuz it's waiting for me. And I have a bell notification on. So, I'm going to get a little ding when I want to go and see where it's at. You can see that I was doing a chat mode here and running. And then I had a remote agent on the cloud. So those are the different icons. Cloud equals remote agent. The user avatar equals auto agent. So it's the one that you would expect when you're working with cursor augment. It's just ripping yolo mode. And then the chat icon is chat. So those are the three different ones. And when you go to set it up, you basically just point whichever remote agent you want to whichever branch you want it to work on. So, I'm looking at this and I'm thinking, "Wow, I should configure the absolute crap out of these such that I have a whole fleet of them." And this is like a developer's dream come true because you don't have to run your own infra. You don't have to do all this stuff. I had made videos about this in the past where I was like, "Oh my gosh, this is going to be awesome. I'm going to have these things and they're going to be trained on the context of the codebase. We can do so many things with them. They can be proactive. They could be reactive. They can be self-healing. All these things, but they're building out a lot of the infra for it. There's areas that I think they need improvement on, but it's early. So, I'll talk about those in a bit. And I'm also just going to be adamant with the team and just cold email them and say, "Hey, you should make this because they got a lot of talent there working on this problem." So, this is what it looks like when you go and set one up. Right? So, we've pointed at there. You get this little spot to make the environment. So if you're not familiar when you set up a new environment on Linux or Debian doesn't matter not Linux or Debian but Debian or Ubuntu it is Ubuntu then you get different ways to set up the actual virtual environment or the virtual machine. So it's running pseudoapp get if you're not familiar that's in JavaScript world that' be like npm and in python world that'd be like pip. So it's setting up all the stuff for the package management on the actual virtual machine and then here's the tlddr. So update system package list installs Python and the virtual environment tools. So in my case, I love that it started using UV cuz that's what I use. It's goated. It's a lot faster. And then it adds it the path, installs the main project dependencies, and you're off to the races. It uses pi test the box, which is nice. But I can see that I should make a directory that's just for these setups. And I'm going to now I'm going to write that down. I'm going to make a directory and I'll share all this with you guys on how I'm setting up my remote agents because again, Mega Corpse at Fang, Mag 7, whatever you want to call them. They have all this stuff internally and we need to glue together these systems and have other companies help us with the infra. So, next up is workflow tips. So, I've been using VS Code a bunch because of some of the APIs that released and I was excited about their use of Copilot and being tied in and open source, but I'm also realizing cursor is getting a lot better. So, I want to combine the best of both worlds where I can tab tab if I need to. And so, you can move it to the right. There's a video on here, but the TLDDR is you just drag and drop it from the chat header, which is not that intuitive. You wouldn't guess that, but that's how you move it over to the right. And then now let's talk about snippets and actual usage and we'll end with my thoughts on it and an actual rundown of my use case. So in this case this guy on there I was like whoa I don't even know what that is but apparently when you hit alt 6 that right so alt and any number should be tied to commands aka prompts. And so this is a snippet hosted inside of Raycast. I've been using the native text expanders for a lot of these, but I'm quickly realizing it's nice to have it in there, but I'll just show you some of these little nugget inside of my keyboard. I have text replacements. And then I just tie these to different prompts. So I have a workflow for debugging B1, B2, B3. This is an incident expert analyst, engineer, whatever. And I just chain them together. I'll be using these as the basis for prompts that I then give to remote agents that then are baked into the VM so they can go and have ability to use those as tools. So in this case, he's tied it to that thing and then L I N which stands for linear and let's take a look at the actual prompt. So let's see. Is it this one? Yeah. Okay. So ticket, here's the story. Wait a second. Here we go. It takes when you run that snippet, it will paste in whatever it is that's in your clipboard. So this guy's workflow was I go and I copy the thing from linear whatever it is that I'm working on and then I do alt 6 l and then it gives you all of this. So please can you work on the ticket and then name of ticket firstly please move the ticket into in progress. Firstly, create a new branch called feature ticket and then what it is and then use the following workflow. Check linear for details on the issue because there's a linear integration. Plan the work stepping through step by step. Search the web for any details you need on libraries or use context 7 MCP. Update the ticket with a comment containing your plan of action. Cool. Do the actual work making use of the tools you have. And if you need to interact with the database, use boom. That is great. So this is very similar to what we've seen in aer where you have read only or if you watch indie dev dan great channel. He has a primer which basically loads in your context your file tree. So very similar for analytics use this. So having these shell scripts is pretty pretty cool because you're basically combining this with if you wanted to do oneoff remote agent work where like oh boom this ticket doom done and you would give it all the context and have it in here. So this would be obviously customized to your need but think of these as patterns right that's the thing with prompts is you need to extract the pattern and match it to your use case. So in his use case, he is loading in the context, giving it conditional shell scripts to run. I imagine these shell scripts are basically loading in whatever the context is that he wants to load, providing additional information about runtime stuff versus error handling, yada yada. And so it gives you these basically if statements where if it's this, then look at that. If it's this, then look at that. And he ties in tests as well. So that's awesome with playright. and he has the MPX thing there in case because it might not be on the remote agent. And then after you've finished, please create a PR to merge the ticket. You're doing a great job. Keep at it. Ultraink. I saw that and I was like, whoa. Ultraink. There's some words that you give to LLM that just are like the it's like the opposite of a pain point. It's I don't even want to say it, but you're you know what I'm saying. So, Ultra Think hits that that seventh letter of the alphabet spot. And so what it looks like is you basically could go and snag one of then it'll paste in everything and then it's off to the races, right? So other things on a use case. Let's jump into a use case. So I covered this briefly as a teaser at the beginning, but in my case, I'm exploring what YouTube memberships would look like. It's more of a spike. It's research stuff, but it's like low priority for me because I don't even know if it's worth it. So I go and what I do is I mentioned I'm trying to make my own little flavor of markdown with how I'm going to use these agents moving forward. So you'd have the title, the description, status backlog type. I'd probably map this to whatever ticketing system I'm using. Because I'm working solo on this, I don't really care. But in my case, the assigne, I would want to have a team file that explains right now it's just Auggie, but I'm looking forward and I'm saying, hey, it's going to look a lot more like this where you have different ones. So the top one would be the chief, right? He's the chief Auggie. But then below that, you'd have all 14 of the different ones that map to the AI SDLC or artificial intelligence software development life cycle, which is just literally how to use AI to use the traditional development life cycle. If you want to check this out, this is a hotkey driven AISDLC that you can just see all the different parts of the life cycle. I've mentioned this before in previous videos and I've covered it in our VI community, but it's going to be one agent per step and then you can probably break them down even further. So within front end, you might have different things that you want to go and have set up for a particular remote agent. But this is the way that I see it where I think tying these to each branch would be really nice and then having it go up to one branch where in the future I'd like if Auggie could actually act as an orchestrator where it could know okay here's all these PRs. Let's figure out are there merge conflicts? What's going on here? This one's 17 commits behind that one. All that stuff. And so let's back to here. You can see that I started here and then I went straight into a normal chat with just basically building out this spike. So I go from spike to let's find all the relevant files. And I just wanted to rip through that. Now I imagine that each one of these things that I'm doing manually and it's so funny now the new manual is using agent auto which is awesome. Our new baseline is just fantastic. But then I'd go and after the fact, once I've done this a few times and I see what's working, then I can go and build out the prompt knowing that it gives me the expected outputs and the expected behavior, then I can go full remote. So do it in agent auto first and then go remote. And then after that, you jump over and I'm going to use that as context for the remote agent. And so this is just me going vanilla raw dog with it, but I wanted to not have a full custom config. I wanted to see what's the generalist going to do, how's it going to handle it. And so I just say, hey, I need to go through and make task list out of this so we can actually get a sprint put together, get the tickets going. Always use enhanced prompt. So I just tagged this backlog item, which was with the YouTube membership spike. Hit enhance prompt. You also have the ability once it is enhanced, once you create it, it starts to ask you other questions like, hey, do you want to autogenerate a script? Do you want to write this script by hand? That's that environment thing that I mentioned earlier with the TLDDR. And then once you've completed that, I always have the notification bell on. And then it starts to spin up the environment. You can see that I selected Auggie as this, but in the future, it's going to be Auggie- QA, Auggie- UIUX, all these different ones, which is awesome. Low key, it allowed me to get a German Debian VPS set up with all the bells and whistles in I don't know couple hours, which was great because I had to translate it from German boot time or boot language to English. There's all these little things. But it goes and it spins it up and then you have the ability just to open that as a remote workspace which is really cool. So you own the whole process. So again, do it manually, then create the prompts around it. That way when you kick off a new remote agent, you can provide a superdetailed ultra system prompt. Example research. MD. So some of my closing thoughts on it actually lastly is before I go to the next part is you can see that now I'm in the cloud little cloud icon. So that represents the remote agent and you can see that it has some questions for confirmation. Really nice, right? because it's actually listening, understanding, and seeing that there's holes in the story. So, do you have access to this? Yeah, I do. Should the membership reflect that membership? Because it's already thinking about the cut that it's going to take. So, it's actually clarifying some of the thought. And then how should we handle users with both? And then what roles should we have? And then do you have these templates? And it's funny because these questions are really just some of them are, hey, we already know this because the contact engine sees that I don't have an email template set up. So then I would go back and I would actually answer those and kick it off. So very exciting. And I've used these, just to be totally honest, I've used these for those use cases that they mentioned in the docs and they work great for that. But I want to push harder and I want to chain these things together. And I'd love to hear your guys' thoughts on how you think this should work, but that's where my head is at. So, some thoughts on it is we have Dan Man and Ian. Dan man is Googler and VI. And he said, "I think the future will be distinct agents, each responsibility for different parts of the codebase. They'll have product review meetings, debate priorities, etc., just like humans, but at a thousandx speed." And is that an M dash? Cuz I think this is written by AI. Huh? Is that AI there? I'm sniffing a little bit of AI. No, but yeah, I totally agree and that's why I think you will have them split out to different areas of the codebase. I think the branching strategy is something that needs to be improved and also what happens when you need multiple ones within that same area of responsibility. How do you shard it? I just wanted to drop shard. And then Ian says his cursor wish list and how I tie this to augment is he says that I he hopes the cursor will eventually become the everything app and not just an IDE something akin to the desktop app where I can chat and plan maybe multiple models side by side. So not just for code but for business content creation life news whatever. I found myself already doing this where I end up doing all of my notes and based on what I'm trying to do, I'll use augment when there's a bunch of stuff and then I go and I clean and I tap tap. But totally agree. And I think that augment in order for them to win win-win, they won't do a fork, right? They just won't. And I'll talk about that in a second. But the more specific the better. You need to map it to specific steps. This is what I'm going to be following. I think it's going to look something like this. And then I think they'll win by focusing on a couple of these things. So mobile adoption, I see people using Termius to jump in and cloud code on their phone or their iPad. Who would do that on an iPad? I'm just trying to replace doom scrolling or when I'm on a walk with this anxiety that I have when I'm not running agents because I want them running all the time. And having it on my phone would be fantastic. Now, you can do this if you set up coder, I believe it's called, which is it lets you like tunnel in and have your extensions, but it's just like brutal. What they did hint on the Discord, don't know if Yeah, obviously I could say it was on Discord, but the messes got deleted, is they're exploring other areas beyond the panel, right? They understand that they're more than this little surface area that is the panel. So maybe a world where they have API where I can hook it into Twilio. I'd love to build that integration where I can just open Twilio, boom, out to a browser right in there. Here's what's going on and I'm just chatting with it. That'd be awesome. But some sort of mobile adoption. And then another way to increase the surface area is some sort of GitHub integration beyond just like ticket creation and whatnot but actually in GitHub is action workflow or something that does like what task code rabbit does where it's like hey here's this issue cursor just launched some version of this but it's just still means you have to go back to cursor I don't want to go back I want it to do it there. So, I think that would be great where it's, well, we identified this. We ran it against context engine. Here's the things we needed to fix. Are you feeling good about it? If you're on YOLO, it just does it and then it fixes it. So, you're not back and forth ping-ponging of, ooh, what failed? Let me go paste this in and then paste it back and just basically window hop. That sucks. I think they'll win big if they figure out multi- aent orchestration UX. I've thought about this a lot where it's almost like a node based thing like an N8 and a lot of people hate on it, but nodes aren't bad. And so you could see how you just basically chain these together and some UI like this where it's I want to hover over the research agent and then see the outputs. I want to hover over the UIUX1 and see what it was thinking. And then you could have YOLO mode for multi-aggentic orchestration where it's like boom calls home. home says, "Yep, good. Next." And you're just ticking them and you could literally just see the software get built in front of you. And that's so good. Agent templates, I think, makes a lot of sense. So, taking the things that are tried and true and then having a marketplace for those for adoption, having proactive remote agents. So things like, hey, we didn't need you to ask about this refactor, but I promise you based on this research, based on we know about your codebase, based on what we know about your codebase, you should go and do this because you're going to see performance benefits or you're going to see, I don't know, better load times. That's the same, less errors. I think reactive remote agents, so self-healing where if you had an observability, if you had an observability remote agent that's just running forever on prod and it knows the second something goes off, instead of there being on call devs, it's just no like we have Prometheus and Loki and Graphfana, we have all this information, we need to put it to use and then loop back in and that is self-healing. That's awesome. And then just more context engine improvements. It's already awesome, but I know it's going to get better. So, those are my thoughts on it. Like I said, I've been using it for the small things, but I wanted to do something a little more saucy, a little more forward thinking in an actual codebase. So, if you learned one thing, make sure you like the video. If you learned two things, subscribe. If you learn more than that, then give yourself a pat on the back cuz you're a good guy or girl. Probably a guy, though. All right, I'll see you in the next video. Check out VI. If you want to get real nerdy with people that are building real stuff, learn how to use AI better. Join the Discord. We got a new launch coming out June 13th. Oh my gosh. And uh I got to go work on that. See you.