Real time RAG with context and meta

Nov 19 20247 min read

All LLMs share a common limitation: they are trained on data that may be outdated. As a result, when a user asks for information such as the current pricing of XYZ company, LLMs may either generate inaccurate (hallucinated) responses or state that this information is not available in their training data.

Langbase AI Memory agent is designed to overcome this limitation by providing real-time data to LLMs, enabling them to answer user questions accurately. Together with AI agent pipes, Langbase allows users to seamlessly build real-time Retrieval-Augmented Generation (RAG) systems, complete with contextual data and metadata.

In this guide, let's take a look at how we can build a real-time AI memory that will use your data to answer user queries.

We will be building an AI memory agent using Langbase. So please go ahead and create an account on Langbase.

Step #1Setup a Node project

Now let's set up a Node project. To do it, run the following command in your project terminal:

1npm init -y

This will create a package.json file with basic information. We will use the dotenv package to read environment variables in our code. So let's install it.

1npm install dotenv

Lastly, let's create an index.js file in our project directory. This will contain all the necessary code we will write to build an AI memory agent.

Step #2Generate a user API key

The next step is to generate a user API key which you can do here. We will use this key to authenticate our Langbase API requests.

Now go ahead and create a .env file in your project directory and add your API key there.

LANGBASE_API_KEY=<REPLACE_WITH_LANGBASE_API_KEY>

Step #3Create an AI memory agent

Now we will create an AI memory agent on Langbase using API. Let's copy the Node.js code from the docs into our index.js file.

1import 'dotenv/config';
2
3async function createNewMemory() {
4	const url = 'https://api.langbase.com/v1/memory';
5
6	const memory = {
7		name: 'ai-memory-agent',
8		description: 'This is an AI memory agent created by the Langbase API.',
9	};
10
11	const response = await fetch(url, {
12		method: 'POST',
13		headers: {
14			'Content-Type': 'application/json',
15			Authorization: `Bearer ${process.env.LANGBASE_API_KEY}`,
16		},
17		body: JSON.stringify(memory),
18	});
19
20	const newMemory = await response.json();
21	return newMemory;
22}

Step #4Upload data to memory

AI memory agents can contain a wide range of data from text files, pdfs, code files, markdowns, and more. I will be uploading a markdown file to the AI memory we just created.

Let's upload two docs in our AI memory. We can use Gemini documentation around structured outputs and fine tuning. I have already downloaded these pages as markdown. You can download them from here.

I have placed the documents inside the docs directory of my project.

Generate signed URL

To upload a document in memory, let's first generate a signed URL. We will later request this URL to upload the document.

1async function getSignedUploadUrl({ name, href }) {
2	const url = 'https://api.langbase.com/v1/memory/documents';
3
4	const newDoc = {
5		memoryName: 'ai-memory-agent',
6		ownerLogin: 'saadirfan',
7		fileName: name,
8		meta: {
9			href,
10			source: 'Google',
11		},
12	};
13
14	const response = await fetch(url, {
15		method: 'POST',
16		headers: {
17			'Content-Type': 'application/json',
18			Authorization: `Bearer ${process.env.LANGBASE_API_KEY}`,
19		},
20		body: JSON.stringify(newDoc),
21	});
22
23	const res = await response.json();
24
25	return res.signedUrl;
26}

Upload docs

Now let's write a function to request this generated signed URL to upload documents in AI memory.

1async function uploadDocument(signedUrl, filePath) {
2	const file = fs.readFileSync(filePath);
3
4	const response = await fetch(signedUrl, {
5		method: 'PUT',
6		headers: {
7			'Content-Type': 'text/markdown',
8		},
9		body: file,
10	});
11
12	return response;
13}

Step #5Retrieve relevant data

Lastly, let's ask a question from our AI memory to retrieve chunks that can contain the answer.

1
2const structuredOutputPrompt = `What kind of data gemini generates by default?`;
3const fineTuningPrompt = `How does fine-tuning work in gemini?`;
4
5async function retrieveSimilarChunks(query) {
6	const url = 'https://api.langbase.com/v1/memory/retrieve';
7
8	const response = await fetch(url, {
9		method: 'POST',
10		headers: {
11			'Content-Type': 'application/json',
12			Authorization: `Bearer ${process.env.LANGBASE_API_KEY}`,
13		},
14		body: JSON.stringify({
15			query,
16			memory: [{ name: 'ai-memory-agent' }],
17			topK: 2
18		}),
19	});
20
21	const result = await response.json();
22	return result;
23}

Here is what the final code will look like:

1const fs = require('fs');
2const path = require('path');
3require('dotenv/config');
4
5async function createNewMemory() {
6	const url = 'https://api.langbase.com/v1/memory';
7
8	const memory = {
9		name: 'ai-memory-agent',
10		description: 'This is an AI memory agent created by the Langbase API.',
11	};
12
13	const response = await fetch(url, {
14		method: 'POST',
15		headers: {
16			'Content-Type': 'application/json',
17			Authorization: `Bearer ${process.env.LANGBASE_API_KEY}`,
18		},
19		body: JSON.stringify(memory),
20	});
21
22	const newMemory = await response.json();
23	return newMemory;
24}
25
26async function getSignedUploadUrl({ name, href }) {
27	const url = 'https://api.langbase.com/v1/memory/documents';
28
29	const newDoc = {
30		memoryName: 'ai-memory-agent',
31		ownerLogin: 'saadirfan',
32		fileName: name,
33		meta: {
34			href,
35			source: 'Google',
36		},
37	};
38
39	const response = await fetch(url, {
40		method: 'POST',
41		headers: {
42			'Content-Type': 'application/json',
43			Authorization: `Bearer ${process.env.LANGBASE_API_KEY}`,
44		},
45		body: JSON.stringify(newDoc),
46	});
47
48	const res = await response.json();
49
50	return res.signedUrl;
51}
52
53async function uploadDocument(signedUrl, filePath) {
54	const file = fs.readFileSync(filePath);
55
56	const response = await fetch(signedUrl, {
57		method: 'PUT',
58		headers: {
59			'Content-Type': 'text/markdown',
60		},
61		body: file,
62	});
63
64	return response;
65}
66
67async function retrieveSimilarChunks(query) {
68	const url = 'https://api.langbase.com/v1/memory/retrieve';
69
70	const response = await fetch(url, {
71		method: 'POST',
72		headers: {
73			'Content-Type': 'application/json',
74			Authorization: `Bearer ${process.env.LANGBASE_API_KEY}`,
75		},
76		body: JSON.stringify({
77			query,
78			memory: [{ name: 'ai-memory-agent' }],
79			topK: 2
80		}),
81	});
82
83	const result = await response.json();
84	return result;
85}
86
87const structuredOutputPrompt = `What kind of data gemini generates by default?`;
88const fineTuningPrompt = `How does fine-tuning work in gemini?`;
89
90(async function () {
91	const newMemory = await createNewMemory();
92
93	const src = path.join(__dirname, 'docs');
94	const files = fs.readdirSync(src);
95
96	for (const file of files) {
97		const filePath = path.join(src, file);
98
99		let href = '';
100		if (file === 'gemini-structured-outputs.md') {
101			href = 'https://ai.google.dev/gemini-api/docs/structured-output';
102		} else {
103			href = 'https://ai.google.dev/gemini-api/docs/model-tuning';
104		}
105
106		const signedUrl = await getSignedUploadUrl({ name: file, href });
107		await uploadDocument(signedUrl, filePath);
108	}
109
110	const structuredOutputResult = await retrieveSimilarChunks(structuredOutputPrompt);
111	console.log(JSON.stringify(structuredOutputResult, null, 2));
112
113	const fineTuningResult = await retrieveSimilarChunks(fineTuningPrompt);
114	console.log(JSON.stringify(fineTuningResult, null, 2));
115})();

Step #6Run AI memory agent

Lastly, we will run our index.js file. It will create a memory, upload markdown documents inside it along with their metadata and then finally retrieve chunks from the memory for different user queries.

1node index.js

Here is what the response will look like for the user query: What kind of data gemini generates by default?:

1[
2  {
3    "text": "Title: Generate structured output with the Gemini API\n\nURL Source: https://ai.google.dev/gemini-api/docs/structured-output?lang=node\n\nMarkdown Content:\nGemini generates unstructured text by default, but some applications require structured text. For these use cases, you can constrain Gemini to respond with JSON, a structured data format suitable for automated processing. You can also constrain the model to respond with one of the options specified in an enum.\n\nHere are a few use cases that might require structured output from the model:\n\n*   Build a database of companies by pulling company information out of newspaper articles.\n*   Pull standardized information out of resumes.\n*   Extract ingredients from recipes and display a link to a grocery website for each ingredient.",
4    "similarity": 0.5605214238166809,
5    "meta": {
6      "docName": "gemini-structured-outputs.md",
7      "href": "https://ai.google.dev/gemini-api/docs/structured-output",
8      "source": "Google"
9    }
10  },
11  {
12    "text": "*   Build a database of companies by pulling company information out of newspaper articles.\n*   Pull standardized information out of resumes.\n*   Extract ingredients from recipes and display a link to a grocery website for each ingredient.\n\nIn your prompt, you can ask Gemini to produce JSON-formatted output, but note that the model is not guaranteed to produce JSON and nothing but JSON. For a more deterministic response, you can pass a specific JSON schema in a [`responseSchema`](https://ai.google.dev/api/rest/v1beta/GenerationConfig#FIELDS.response_schema) field so that Gemini always responds with an expected structure.",
13    "similarity": 0.5243381261825562,
14    "meta": {
15      "docName": "gemini-structured-outputs.md",
16      "href": "https://ai.google.dev/gemini-api/docs/structured-output",
17      "source": "Google"
18    }
19  }
20]

As you can see that we have text chunks along with their metadata and similarity score. This data can then be sent to the LLM along with the user query. LLM will use this information to generate the response.

Wrap up

That's all from this guide. Now you can use Langbase AI agent memory to build AI apps.

We have also written a guide on how you can build multi-agent AI support that you can find here.

⌘Langbase

Guides