AI & Machine Learning,  Software Development

How a Shared AI Memory Repository Focuses Teams

Author

Tyler James

Date Published

When using AI it can be easy to build up a large internal set of memories that we rely on for our day-to-day operations. The trouble starts to surface when we are on teams and each person on the team has their own internal set of memories which can conflict with one another in terms of output and consistency. Even if all team members are generally on the same page, these memories can start to stray and cause issues down the line. 

In this post, I’ll share what we did to reduce memory drift issues and create a more consistent and streamlined development process using a shared memory repository. This solution is easy to implement and quick to adopt with just a few additions to a project and one created repository. 

Examples

Scenario One

For one of the clients we worked with, our team received a new set of requirements regarding database default values that had to be used going forward or the synchronization with their legacy system would break and lead to data not being properly created for the new system we were building. There were many different field defaults given to us and prior to us coming up with a solution the best course of action was to either set up local memory to try and track these defaults or manually implement them which could be tedious and time consuming. Having local memory could get troublesome if defaults changed and not everyone kept their memory up to date. In our case, we had a few instances where an improperly defaulted database field caused the legacy system to not properly create records.

Scenario Two

For the same client as scenario one, I did a lot of work rebuilding the application forms and input validation to be more cohesive and reusable. This included new input components, and an entirely new validation system from front to back which was to be used going forward. My local memory was set up to recognize forms and to implement this new process automatically when scaffolding out new forms. The issue was when other team members would occasionally do work with forms or input validation their local memory could fall behind and not have all of the newest information such as new components to use and new processes. They knew about the new system and were kept in the loop, but with the rate at which it changed it was difficult to stay perfectly up to date especially when it wasn’t something they worked with that often.

Scenario Synopsis

Now take the same scenarios and scale the number of developers to ten and add in a year's worth of work.  Each developer is quietly maintaining their own local memory. In the best case scenario every developer is diligent about keeping their memory up to date - and they pay for this in time. In the more likely scenario, somebody’s local memory falls behind quietly which also leads to paying in time and inconsistent output. After implementing a shared memory solution for this project we reduced the amount of legacy system synchronization issues and fixed any issues we had with inconsistent forms or validation.

Our Alternative

The alternative that I came up with to help circumvent some of the outlined issues that occur when relying on local memory was to create an organized git repository (or folder within the project) containing memory around shared functionality or pieces of the project. This repository is shared with the team, and as the team finds things specific to the project that should continually be implemented or remembered, they are added. It’s helpful to organize this repository so that it's easy to track what exists without having one or two large files containing hundreds of memories. Here is an example of how our repository looks:

The setup here is down to personal preference and can be built as granular or as loose as you would like it to be. I’ve set it up so that we have a section for the major parts of the application. Inside of each of these folders, I’ll have a few markdown files to go over certain pieces of information.

Once the repository is made and shared, there are various ways to actually use it. We just need to point whatever AI we are using in the direction of the repository. This could be a skill where it dynamically detects when to use it, inside of an instruction file (like CLAUDE.md), or my preferred method: a slash command. I prefer the slash command so I have explicit control of when that repository context gets fed into my session. In our case over at Seven Hills Technology, we also use a wide variety of internal commands and I have pointed our CLAUDE.md file to read the repository prior to executing any of those commands. Here is what my slash command looks like:

It’s nice knowing that prior to running any of these company level commands, our project level preferences and memories will be fed into it. Prompt contexts usually do a good job of only loading in the repository context once, so you shouldn’t have any issues with loading the repository multiple times in the same session.

There are other benefits to having this setup beyond just having memories in a shared and accessible location. This information can serve as mock documentation. If a new developer is onboarded or you need to read up on a certain piece of information relating to the project, there is now a repository of information for you to reference. Need to find out if there is a standard for select inputs? Shared repository. This can even be taken a step further and this repository can serve as a documentation markdown site if you need a live website to actually display this information. Personally, I prefer to use markdown files anyway so that at the very least the files can be easily readable.

The Downsides

While my solution works pretty well for our team and has certainly had a positive impact on our development process, it’s not perfect. Here are some issues that I find with this solution:

  1. It can be easy to fall into the trap of not keeping this repository up to date. As much as we can try our best to keep this up to date, sometimes it's easy to forget or put off updating the repository.
  2. Although small, this process does add an amount of effort and time to our regular development process. It pays off in the long run, but in the short-term it does have a cost.
  3. Prompt context is generally pretty good about reading the repository and using the information within but sometimes things are missed. What this means is that we cannot blindly trust output when using a shared memory system like this and assume that all standards are being followed in their entirety. A shared memory setup like this should be used as a tool to speed up work and to make tedious tasks less tedious.

Conclusion

This solution may not be the answer for everybody, but I hope that at the very least this brings attention to the fact that we should be careful when building up a large internal set of memories. Try opening your local memories folder and look through it to see if there is anything in there that may need to be updated or maybe something important that should be shared with your team. I’m sure that you will find something that is outdated or something that may be useful to share.


blog hero 68cac8c9c3104eb54617d545_Blog Post Header
AI & Machine Learning,  Product & Business,  Software Development

What Every Dev Should Know About Integrating with AI

This blog primarily draws on Conversational AI for examples, but there are many other ways to integrate Generative AI into a project, and I hope there is helpful information for those pursuits as well...