Updated on
hello. imagine you’re working on a project, like how cohesive it is like how cohesive it is like how cohesive it is when starting a new project as a OLO engineer robably you will create the something well structure and organise with a cohesive style design and style. but overtime you will emort the new engineers to help you, and maybe, things will start to you know maybe. get slightly off track. But at the same time you’ll still be able to keep you know a good structure and this has always been and will always been, but maybe you go past this initial build hase and the project moved to move to a different team that will now take over. You know the day to day maintenance of the project and smaller new features because the core has already been built, and now this theme will start to onboard their different coding style, a different way of doing things. And maybe you know they’re going to start to when adding new features or fixing things start to go a bit off track of their style, and start to onboard the new style, but they’re not going to rewrite the whole thing, O. It’s going to get messy, and you know this always happen, so now imagine a where on this old project you’re onboarding AI and it’s going to be the same thing. It’s going to have his own stylish way of things since he doesn’t understand, like the new team taking over the day to day doesn’t fully understand many things. O every change will otentially introduce side effects and roblems that are going to be impossible to understand for the AI and maybe even for that team that you know it would have been the same with the changes time by the team but now you essentially introducing a 3rd team that does the thing and the 2nd team is reviewing it and when actually what you needed is you know who fully understood that thing, the original author, to sort it out, with this metaphor, what I want to get, well, it’s not really a metahor. This is reality. But you know, what I want to get is that, you know, you maybe read and hear a lot about good things has been made thanks to AI. But then when you try to apply it to your project it doesn’t really workout. You’re like oh well I asked you to do a thing you create a lot of side effects that R was garbage and Ohio it wasn’t following my style I had to fix it. Everything took me forever to make it look like I wanted or make it understand my side effect and then yeah fine whatever maybe you know this was a one off. So I going to try again and I going to have the same issue over and over again and yes of course you can have a bad experience and justice opening arentheses have to say that maybe you know, um Github Copilot does a bit of a bad service here. He’s doing a disservice to AI, which is you know a bit ironic, even. How much Microsoft and Github are pushing for it. But the limited context window especially in these scenarios, when you’re entering into a project that is not being, made for AI on double cost it will not work well, anyway, that’s a completely different topic. Maybe we can get into it. Another day, O going back to our code base well, so we have this big problem that we are trying to drop an AI agent into it and it will not be what we want. It will not behave as we expect. It will not be able to fully understand how to do things correctly without creating side effects, without creating breaking things, and so on. Yes, of course, if he runs the test. But you know, we all know what test cover is me. Even a project with good test coverage is not going to be really, you know, testing for all kind of side effects and weird alication. Of course it depends what we’re talking about. If you’re talking about the simple API, yeah that’s not really a big deal. But as soon as we’re going to more complex stuff, maybe you know, into like low level or into like concurrency like you know going to have a test most of the time for all these possible edge cases and scenarios. So what’s happened? Well, things are not going to work well OK so how do we actually on board AI, in a code base because you know if you look at the example, mostly they are about oh let’s vibe code this new tool and so on. But that’s not what you know you want to do in a real company software and at the same time you’re not going to have many Greenfield projects to work on. So you know, yes you can say 0 let’s wait for the next time we have a Greenfield project to start to do it. From D0 with AI and try to you know, build the roject that works well with AI, but I mean, this has always been like, you know, you test the new, coding language, the new coding framework. That’s what happened back in the days when we what dot net core first came out when it was still called core and people are like Oh yes, and I’m going to test these things that is not compatible much with my existing code base and convert my code base of course it’s like 00 finally now we have a nice opportunity to create a new project. So let’s start using dot net core and that’s you know, but then the system evolve and you know anyway, that’s an aside going back to what we were talking about. Oh yes, so where I was so yeah, we’re not going to wait for the new Greenfield roject to work on. So how do we, transform, how do we and reach our code base to enable for an AE ad AI agent to properly work, keep in mind we probably have that limitation on the context I briefly mentioned as an aside, like yes. If we onboard I can tell you if you onboard like tools that’s like you know latest and greatest Frontier model, with the 1 million context which probably means paying quite a bit of money, ‘cause probably you know you need AI an expensive plan, and probably that will not really umm be enough for any. proper work. O. Then you robably need to go over quota, start to pay extra and you know you easily get from oh I need a $200 month plan and then oh I have to pay £2000 a month extra. So how we can make our code base work of course, you know if it really works and Spain 2000 a month, maybe it’s not such a big deal but anyway, let’s go back. Maybe we have just a $20 a month such copil subscription with only a tiny tiny one 100, 20K context. But can we make it work? Well, to make it work, you need to make sure that you know your model is able to know how the code base work, is able to reason around the code base without having to read all the code base. Because if he has to read your code base and is more than you know, simple API projects that just call two other Apis and maps the response as soon as. It’s a bit more complex than that. Just reading all the code it will not fit the context. Yes, of course the context can be compressed, which means after a while it will, you know, do something to reduce the amount of data in the context and yes, there are different techniques to that, but the outcome is always like any loose I will say losing compression is is not going to be accurate and that will drift and Causeway problem. So what it needs to do is also takes time and of like what you need to do is to be kind of able to index your code base. You need to be able to describe how it works. Also like you know, imagine even yourself like if every time you have to read the code base from scratch, trying to understand how does it work, how does interact with that. And maybe the code base is also a bit messy. It’s a nightmare. It will take you forever. It will give you an headache every time. And the say we could same we could see with the model. Of course everything is million times faster than you but, it will have the same problem, it will make mistakes and maybe every time different mistakes and so on. S. What you need is something to describe how the code work, which is essentially what you would have in your mind, what the original authorised in your mind when it initially created that code base. And hoefully overtime is kind of California. And you have this kind of things in your head, which unfortunately nobody really write down. Yeah sure there are teams that are better not with documentation, but you know, so if you already have that documentation, actually you are robably well position to leverage AI. What you have to do is probably to you know to make things easier. Yes you can use MCS and stuff and make. To make things easier just convert them all in markdown files in text. Then you just put inside your repository. Now starting with this documentation, maybe starting with nothing. You can leverage an AI model create or improve this documentation. Start to you know, make sure it’s up to date, make sure it reflects the code so it can start to read the code, and so on. O. Essentially you need an initial phase when you first what I will call identify your code base which means creating an initial agents MD file or you know whatever other file name you want to use that is more specific to your code base but to sorry to the tooling you’re going to use. But I will say the agents MD is kind of the standard at the moment every tool support correct me if I’m wrong so maybe that’s your starting point, but actually you don’t really need that what you start is like, you know, a docs folder, a specs folder, whatever to define how your code base works. Well, it’s not even how it works, it’s more about, you know, what’s the stuff that matter in this product, what’s the goal of this project, what these features are supposed to achieve, what this functionality do and what are the constraints it works in. You know, maybe you have some architectural constraint, write the files that define the architecture of the system, and again, you don’t need to write Rayan. Maybe you already have it. Awesome. You’re a really good starting point. Maybe you have diagrams. Awesome. Even better. But otherwise, you can leverage a model to create this document. Or you can leverage to say 0, import VMC from Confluence and write it down in markdown so you don’t have to do the conversion yourself or you know, I have these diagrams and let’s leverage the model to convert into A to mermaid format so it’s easier to maintain in markdown and for AI model to quickly read and so on and also then to you to the simple plugin or builtin on Github you will see the diagram and so you start to document all this stuff. You start to describe how everything works and then is important when you start onboard AI that you keep this stuff in the loop what’s the other bit we talked about test. Test will not cover everything. Test will not really care about side effects. Well why not? Because of writing good test is time consuming. But now you have a tool that will you know could help you do it in a fraction of the time, so start to use it to improve your test. So I would say you know, before saying 0 I try the I doesn’t work. Yes. But you know, instead of trying I and using AI implement new features, fix box in your codebase, stuff like that. Try to do it to make your codebase in a better state to start with and not necessarily touching the code at all. like. Not your work in production code. Just you leverage it to improve your test imrove your test coverage or your test quality. Or you know, the extra bits of handling constraint and edge cases like oh, maybe I, I need to make sure that you know how it works with concurrency and so on. Like this could be problem that you could touch the code that unless you have your fully you know logical mind engage 100 percent when writing and review with this code are easy. It is going to be easy to make this kind of subtle box this tiny mistakes that will break things. So what you have to do is leverage it to improve this stuff. And you can kind of drive it and you know, start to use that. And of course now I’m like, oh, I’m using the I to imrove my roductivity. Now I’m using AI. Yes, it could, you know, be productive to write new test. But the same time I didn’t need those tests in the to begin with. So now I’m spending time on something doesn’t give me value. And I think that’s true. But it’s also true that every time you try to abort, you know, new tooling, new techniques, new way of working, there are always some cost. There is always some cost an investment you’re investing to be able to enable yourself later on to leverage a tool that has the potential to improve things. And I understand this is R. Because until it’s proven that then you’re going to get the benefits out of it until you’ve seen it with your eyes, you’re like, oh is it really worth this investment? But sometimes you know you need to take a bet everything in life and in work is about taking some bets with some, you know, controlled risks and so on anyway so you finally do this. You start to improve things a bit. But most important you start to write those talks. You start to write those markdown files that describe how your software is supposed to work, what is a constraint it has, why it has been designed a certain way. And you probably don’t know if you don’t really have it. And maybe you don’t even remember. But you put what you have at the moment, what you can come up with and you know, maybe you just do some ramblings. Or you can even like, you know, use a dictation tool, just fry it. Your flow of thought and then give it to AI and say oh make this look like a nice rojects documentation that can be you can even ask prepare it in a form that you can later on use it yourself to understand the rojects. When you’re going to need to work on a future and a new bug, make sure that you have indexes like it’s really important. Again, we were talking about working with the limited context. That’s always true. You always want to limit the user’s context to what is really needed. But the same time, if you use a tool like it app compiler with limited context, this is even more true and more important so you’re going to have to create kind of index files and again you don’t really have to do it. You can ask AI to do it for you and this index file are like you know, describing where to go and find the documentation, but could also be index files of code. And these are not secial files. I’m not talking about anything like crazy like that. This is just like markdown file. Like you know, maybe you have a docs folder or a specs folder that define each feature as a spec. You have a rhythmic file that kind of indexes say OK. So we have this feature here and it is described described in this file. This other feature XY in this other file, this other aspect of week, this extra constraint we have, the architecture is there and maybe not in the architecture file then you you have an initial rhythm that describe high level architecture and then say oh more details are in this other file, more detail are in this other file. Like breaking down these files is a lot better because the problem is like if you create a long file it will have either to ingest it all or it will try to ingest you know first hundred lines may mean that 100 lines you have the level description and then you have a table of content of the other thing. But of course it’s going to be hard. Yes, you can kind of figure out, you know. Then you potentially could use oh, I find the adding that I need in the table of context. Then I can use a regex to go find the right place and so on. But, for some reason at the moment tooling doesn’t really work like that. They end U most of the time, just scanning in more and more lines until they get to the part they need. So ending up you know, putting more stuff that most of the time’s gonna end up in the context. So if you split down keep small files that are kind of linking to other files so it will be able to see OK 0. I actually care about this stuff, I go have to read it. You know it’s a slightly different way compared to our human World War that you can, you know, just click on a link and go to the other part. It doesn’t really matter if it’s in the same page or not. But the same time even a human, you will get confused a bit scrolling up and down on your screen so anyway, separating files works well for the agent, work well for you O win win now that we have all this thing, we have took the opportunity to also improve our test and using this thing. And when we ask the eye to do something and that’s something we can put in the agents MD or equivalent file. We can always say oh every time you, improve something. Make sure to maintain the documentation. Make sure to maintain your test and we can ask it like have these things kind of reflect each other. Any new constraint documented should have corresponding text, test for it, and every new test that you know is validating a constraint, a potential problem and so on, should kind of capture in documentation that this is something you care about, you can put it in the agents, MD. You can put it in your romts. Yeah, that’s another topic of how you could automate and embed all these things in what you do, now that we finally got to this point, we can try to now finally, you know, implements a new feature that maybe starts with something small, but implement a new thing and we can first of all that’s again different topic to discuss is like use the plan mode. That’s really important when you need to implement something. Plan mode is when you know the model will think and improperly start to ingest all the stuff. It will start to look at the code base and figure out what is needed and use leverage these new documents file we created. Go and cheque out well code it could be. Then in our documents file we do either manually or again with the first pass with the I When we create the we also add reference to files. It’s like 0 this document, how this thing work and actually you know in that file maybe that you know there is going to be this method that does this and so on and of course you would have not really done this in the past because maintaining these documents U to date will there be a Ain. But now you can leverage a tool and tell it to automatically Kee this file U to date so every time it does the change you can tell it to go an update this thing and you tell it once in the agents MD and it will happen automatically most of the time now we start imlementing a new feature. We LAN a lot, and when we’re planning it we’re like, oh, we realise, oh, actually we want to change this thing, that actually this thing is not going to work because O, we go on the plan, still in the planning phase and we say actually you have to do it differently. And maybe we can also tell it Causeway of this reason, and tell it well. In this plan also include adding this new information, this new constraint to the documentation or the agents MD. You know it does depends what kind of constraint we’re talking about but let’s say you know is something related to the application. We say go and add it to the documentation adding to the diagram this important information O, then we keep going, we keep going. And then we got you know, to this nice plan that now we believe is going to work. And as part of the plan that is also to all these new things we thought about while thinking about this feature with all this thing we have considered that the model was not able to to figure out with the limited amount of information it was looking for. Or with this non existent thinking capabilities let’s not start to talk about this. Models are thinking or no, so yeah it was unable. You were able to but you told OK, so you you driven out the implementation should be and you also told it like what you need to store in this documentation so that the next time you will probably be able to know this and do a better job at that. So you know two birds with one stone, whatever is the metaphor. Cool. Now we finally go to the implementation of phase. We like to implement and then maybe, you know, we have to drive things during the implementation or maybe, you know, it’s later on it goes to code review and again we find some problems in that we’re not gonna need to do, you know, careful review. I will say it’s not really different from humans doing review. And that’s again another topic on its own but it’s important to know, that is not different from viewing code from a human. The problem here is that the amount of things if this really work and you can speed up things and they seem kind of OK you could get overwhelming reviews and that’s where the problems could arise. But anyway, that’s topic for another day, OK so now we have what have we done OK. So yeah I was saying we got it implemented then maybe during the implementation phase, the review phase doesn’t really matter. We find out new things that will not work, mistakes that he has made seeing as ever really consider and what we can do we can simply ask it, you know maybe it’s like you know, you spin up A new agent, a new conversation let’s not get into what’s the best way to for doing it at the moment, but essentially like you will leverage it to go or you can do it manually yourself to be honest, but what’s important is the outcome. You need to udate the documentation with this new information O essentially every time we do sounds and we need to build a feedback loop. This is kind of the basic of what that hysics systems what’s called a Informational no close the loop system, close the loo system. Maybe it’s the metaphor I’m looking for, but essentially it’s like every time we do something, no matter if we are analysing, how to imlement a feature. So like in the LAN phase or we are implementing it actually or we are reviewing, we doing things we find problems like. We take that as an opportunity to improve our knowledge base built alongside the code which is much more powerful than code and comments like. That’s another thing I will add in the agents if I like don’t write comments in the code to describe obvious behaviour of the code. Like comments should be minimal. I feel like agents tend to write too many comments. Maybe this is another topic on his own, but it’s like having this situation where you want to just write the minimum amount of comments that are really needed to explain the important constraints. The important reason behind that Not obvious Iece of Code and of course, again, here’s the extra mile instruction. Could be when you are in a situation that you need to add the comment to explain, the code takes that as an opportunity to refuse that code and understand if that code could be made more simple and readable and obvious so that the comment will not be needed in the first place by. Again, this is a separate topic, so going back to our thing, we finally got we finally got to this point, of working and keep improving things and then you know it up to your workflow. You want to include this documentation and changes and as part of the PR when I introduce the feature, maybe you want that split it down in separate PR or whatever. That’s up to you, an an important bit is also then we always add test for it and again maybe you do as part of this, maybe you tells agents and them, but it could even be you know a background job. It could be AA, schedule, workflow in CI that every day or after every PR looks at what was changed in terms of documentation or in terms of test and mirror these things you know from one side to the other. So it’s like new documentation. Others that explain these problems is constraint to consider. Try to figure out these that are tested to be other that could like fix these problems. Because what you want to do is try to be as much safeguard you can for this and to you know try to catch this thing which is not really important just because you know 0 at some point you’re going to forget noticing you’re not going to notice in the review. And it’s gonna slip through, it’s not really that. It’s that you need to you know the so called shift left. You need to review your you need to move your important review, your logical thinking and analysis as late as possible in the process. You want that when the PR reached you, when the code change like you know you’re trying to get to the point where you don’t have to provide all these insight during the plan and then you don’t want to have to provide the insight during the imlementation you want to get, you only need to cheque it as a review. And most of the time everything is already fine. Like you want to make this process as autonomous as possible, which means you have to empower the tool to to get all this problem. That’s why it’s important that you add more and more of this stuff into the documentation. You translate all your logical thinking, into actual instruction. Actual things you need to consider and actual things that need to be embedded in the test. Things that need to be checked for. And if it is an automatic test, you know you’re not going to have this problem by even and then you don’t even going to try to do a mistake that then it will lead to a test failure and so on. By knowing the documentation. Because you know even if it’s just capturing the test, it could have already lead to its creating a mess implementation well if this is caturing the plan is much better. Actually this is a good technique. Like if you got to the point that doing the review phase you capture all these things and you improve. Then the test that you add the new test you improve in the docs to make sure this will not happen again. Actually you can get to the point that you merging these changes to test and docs, but you discard or keep aside the implementation, the agent that came up with. And then you ask again to implement again, because now he has much more information to avoid doing the same mistake and actually wall design and implementation of the feature with this constraint embedded from day zero could probably lead to a much better, cleaner, and better outcome. Yes, of course you can always ask 0 clean up this PR, improve these redesign, whatever. But sometimes it’s actually a lot quicker starting from scratch like we probably even need lead to less cost in a consumption of AI tools, AI, request so sometimes, you know, you can use this opportunity like this could be just an indication actually that, you know, my code base is still not at the point that I could fully, empower the LLMS to work on it but I just use it. Like maybe it’s like oh, in the end you imlement it yourself. Or you know, ‘cause maybe you provided so much Fitbit in the PR that you ended up saying, well, it’s a mess. Easier to fix it myself, and maybe you have fixed it myself, your hands. There’s no, no problem with that. That’s perfectly fine if you have to took the time to do it. And it’s actually easier to just do it yourself instead of writing lots of feedback to the agent and ask it to change things. No, actually that’s not what I meant back and forward. Just write the code yourself. No problem with that at all. You do the code yourself just fixing the PRI creator or maybe recreating the PR from scratch to you because you know you don’t really know how to drive it to do the right things. You’re still learning. Or maybe you know he actually is too dumb and is not going to understand well, it’s quicker. I’m so fast and good and writing code compared to explaining thing fine, not a big deal. You can still work with this tools. You try to code yourself and then ask it to you know, review, your code change and understand why you’re doing things in a certain way. You can ask the model to try to do understand what is now and create itself. The documentation changes the test, including extra tests, whatever is needed so that the next time maybe you know, even provide what you would have asked it to do to say the next time. I want you to be do whatever is needed so that the next time given this prompt with given this ask for this feature request, you would end U with close, so much close to the implementation I created, like if your best skill is not describing things, leverage for it. This is important. It as a tool is like you know when use it in your more interactive way use it to you know, complement your weakness. That’s the best thing to do. I don’t know. Maybe this, you know, is a different topic for another day. And maybe I’m saying too much about this. And maybe what’s more important is that you, you drive what is more important is that you drive. You get to a place where, is. You know the focus is on automating as much as possible that’s you know that’s end goal is automating as much as possible. There is a different things you know what to do when you’re using as an interactive tool to complement yourself versus when you want to get to a point that you can fully autonomously imlement the feature and get it to a really good state so that you can then be happy with it. Like you will be, you know where, you onboard new members into a team. When you create a new team, when you are doing the endover of your projects to a different team and is always like that. You know the first time this bit of bit of carry handling and the bet get in the situation that the other person understand all and you know it’s not that different. Like when you you know you finish the initial implementation of this big new project and sync and you pass it to somebody else for the day to day maintenance. A small improvement over time passing another team, what’s the first thing everybody has to do is like, oh, our documentation is not great. We need to improve the documentation. We need to make it so you know we need to do this knowledge transfer and that’s what you’re doing. You’re doing knowledge transfer to AI, you’re doing the markdown is actually really good as like 0 you talk it to it a bit and is actually write it down in nice structured way probably is going to do a better job than everybody else. Maybe just because of, you know, it’s so fast that everybody else will be constrained by time and you’re gonna create this superstructure things and this is important to be in Markdown and is there everybody can read it, not only the eye is good for everybody. And the eye can help you and you can instruct it to make sure that it never drift. Is always kept In Sync. And the important bit is not even like building something to make sure it doesn’t drift. Now you build something, you just tell it too, keep it up to date. So again you get into this habit of, and again maybe different topic, but you get into this habit of not leveraging tools to catch problems, but you leverage tool to automatically fix problems sure. You’re still going to be in the loop. You’re still going to validate the. What it’s doing is make sense and it’s no nonsense but you know it’s done a job for you. So there is no excuse anymore to do to not do certain things because they’re too time consuming anyway hoe you enjoyed it. Yeah, hope it was helpful and yeah, let me know what you think. Let me know why you disagree. with this down there in the comments. Cool. have a great day lay with LLMS, have fun and most of all I hope you’re gonna have a delightful day. Thank you. Bye
self.about()
