How to build multi-agent human-in-the-loop AI support with Langbase?
Multi-agent infrastructure is an effective approach for building AI apps. Just as it's best practice to organize components into separate files rather than crowding everything into one, the same applies when leveraging LLMs for various tasks. Expecting a single LLM request to handle multiple tasks simultaneously can lead to hallucinations and errors. Here multi-agent AI infra can be quite useful.
By using a multi-agent AI setup, each agent can focus on a specific task, enhancing performance and reliability.
In this guide, we will
- build a multi-agent AI support that will
- leverage AI pipes, tools, and memory to
- answer support query from docs
- connect to a live agent
We will be working with the stream responses in this guide. So make sure you keep the stream on while following through this guide.
Step #0Langbase sign up
We will be using Langbase for the complete AI infrastructure. So please go ahead and sign up if you haven’t already.
Step #1Create AI pipes
We will use two agentic pipes in our AI support agent. One is a decision maker and the other is to generate an AI response from company documentation using RAG.
The decision maker AI pipe will analyze the user query. If it is around the company, it will call the getInfoFromDocs tool to get relevant information. However, If the user wants to talk to customer support, the AI agent pipe will call the connectToLiveAgent tool instead.
In case neither conditions are met, it will respond with that it cannot answer questions other than the ones around the company.
You can create a pipe by navigating to pipe.new. Go ahead and give your pipe a name, description, and select create. By default, the pipe will be public.
I have already created the pipes we will use in this AI support agent. These pipes are public so you can just fork them and get started.
Step #2Add tools to pipe
In the previous step, I mentioned something about tool calling. LLM tool calling allows a language model (like GPT) to use external tools (functions inside your codebase) to perform tasks it can't handle alone.
Instead of just generating text, the model can respond with a tool call (name of the function to call with parameters) to trigger a function in your code.
In our case, we will use tool calling to fetch relevant information via AI from the company docs and to connect with a live support agent.
We need to add tool definition schema in our decision maker pipe for both tools. There is a tools section in the pipe playground to add tools in a pipe. Let’s add getInfoFromDocs and connectToLiveAgent tools to our decision maker pipe or just fork the pipes that I have already created for this guide.
Step #3Create memory
Let’s create an AI memory that will contain all the company documentation. Navigate to rag.new. Give the memory a name, description and select Create memory.
After memory is created, please upload all your company docs into it.
Step #4Attach memory to pipe
Open support answer agent pipe that will generate answers from company docs based on user query. Click on the Memory button and then select the memory we created in the last step.
Step #5Create a Node project
Let’s get to the fun part of writing code. Create a directory in your local machine and open it in a code editor. Run the following command in the terminal:
1npm init -y
2npm install dotenv @baseai/coreThis command will create a package.json file in your project directory with default values. It will also install the dotenv package to read environment variables from the .env file. We will also be using the @baseai/core package for handling the stream, getting tools from the stream, and more.
Step #6Code
Let’s write the code for our AI support agent.
Step 6.1: Create a .env file
Go ahead and first create a .env file in your project. Add the following API keys to this file:
# https://langbase.com/examples/support-agent-decision-maker
LANGBASE_SUPPORT_AGENT_DECISION_MAKER_PIPE=""
# https://langbase.com/examples/support-answer-agent
LANGBASE_SUPPORT_ANSWER_AGENT_PIPE=""
You can find pipe secret API keys either by clicking on the Code button on the upper right corner or by navigating to the API page.
Step 6.2: callLangbasePipe()
Let’s now create an index.ts file in the root of our project. We will write a callLangbasePipe() function inside it where we will call our agentic AI pipes.
1import 'dotenv/config';
2
3const ENDPOINT = `https://api.langbase.com/beta/pipes/run`;
4
5async function callLangbasePipe({ apiKey, prompt }) {
6 return await fetch(ENDPOINT, {
7 method: 'POST',
8 headers: {
9 'Content-Type': 'application/json',
10 Authorization: `Bearer ${apiKey}`,
11 },
12 body: JSON.stringify({
13 messages: [{ role: 'user', content: prompt }],
14 }),
15 });
16}Step 6.3: getInfoFromDocs()
In step 2, we added a getInfoFromDocs tool to our decision maker pipe. Since tools are essentially functions in your code, let’s create a getInfoFromDocs() function that will call another pipe to generate a user query response from the company documentation.
1const userPrompt = `What is the monthly pricing of XYZ company?`;
2
3async function getInfoFromDocs() {
4 return await callLangbasePipe({
5 prompt: userPrompt,
6 apiKey: process.env.LANGBASE_SUPPORT_ANSWER_AGENT_PIPE,
7 });
8}Step 6.4: connectToLiveAgent()
Another tool that we added in step 2 was for connecting to a live agent. LLM will result with a connectToLiveAgent tool call in case the user wants to talk directly to a customer support agent. So let’s create a function to handle this tool call.
1// const userPrompt = `I want to connect to a live agent`;
2
3async function connectToLiveAgent() {
4 // connect to live agent
5 return `Hello! How can I help you today?`;
6}Step 6.5: logStream
Let’s create one last function that will log the stream we will receive from our AI pipe into the terminal.
1import { getRunner } from '@baseai/core';
2
3async function logStream(stream: ReadableStream<Uint8Array>) {
4 const runner = getRunner(stream);
5
6 runner.on('connect', () => {
7 console.log('======= Stream started. ======= \n');
8 });
9
10 runner.on('content', content => {
11 process.stdout.write(content);
12 });
13
14 runner.on('end', () => {
15 console.log('\n\n ======= Stream ended.======= ');
16 });
17
18 runner.on('error', error => {
19 console.error('Error:', error);
20 });
21}I have used getRunner() function from @baseai/core to iterate over the stream to log it on the console.
Step 6.6: main()
Now let’s connect all these pieces together in a single main() function. We will first call our decision maker pipe that will decide whether to:
- Get information from company docs against user query
- Connect with a live agent
- Send a generic response to user
Here is how we will do it.
1import { handleResponseStream } from '@baseai/core';
2
3const userPrompt = `Hey`;
4// const userPrompt = `What is the monthly pricing of XYZ company?`;
5// const userPrompt = `I want to connect to a live agent`;
6
7async function main() {
8 const response = await callLangbasePipe({
9 prompt: userPrompt,
10 apiKey: process.env.LANGBASE_SUPPORT_AGENT_DECISION_MAKER_PIPE,
11 });
12
13 const { stream } = await handleResponseStream({ response });
14}Here I have used handleResponseStream from @baseai/core to handle AI pipe response. The function will return back with an object that will contain the stream.
1import { getToolsFromStream, handleResponseStream } from '@baseai/core';
2
3const userPrompt = `Hey`;
4// const userPrompt = `What is the monthly pricing of XYZ company?`;
5// const userPrompt = `I want to connect to a live agent`;
6
7async function main() {
8 const response = await callLangbasePipe({
9 prompt: userPrompt,
10 apiKey: process.env.LANGBASE_SUPPORT_AGENT_DECISION_MAKER_PIPE,
11 });
12
13 const { stream } = await handleResponseStream({ response });
14
15 const [streamForTool, streamForReturn] = stream.tee();
16
17 const toolCalls = await getToolsFromStream(streamForTool);
18
19 if (toolCalls.length) {
20 }
21
22 // NO TOOL CALLS
23 return await logStream(streamForReturn);
24}Next, I have created two streams using the .tee() method. One stream is for checking for tool calls. The other one will be used in case there is no tool call.
I have then used the getToolsFromStream() function from @baseai/core. This function will return an array. If the array is empty, it means the decision maker didn’t call any tool. This in turn means that the user asked an irrelevant question.
If the toolCalls array is not empty then we need to call a tool in our code. Since tools are essentially functions that we already wrote earlier, we can check for the called tool and correspondingly call that function.
1import { getToolsFromStream, handleResponseStream } from '@baseai/core';
2
3const userPrompt = `Hey`;
4// const userPrompt = `What is the monthly pricing of XYZ company?`;
5// const userPrompt = `I want to connect to a live agent`;
6
7async function main() {
8 const response = await callLangbasePipe({
9 prompt: userPrompt,
10 apiKey: process.env.LANGBASE_SUPPORT_AGENT_DECISION_MAKER_PIPE,
11 });
12
13 const { stream } = await handleResponseStream({ response });
14 const [streamForReturn, streamForTool] = stream.tee();
15
16 const toolCalls = await getToolsFromStream(streamForTool);
17
18 if (toolCalls.length) {
19 toolCalls.forEach(async toolCall => {
20 const toolName = toolCall.function.name;
21
22 if (toolName === 'getInfoFromDocs') {
23 const response = await getInfoFromDocs();
24 const { stream } = handleResponseStream({ response });
25 return await logStream(stream);
26 }
27
28 if (toolName === 'connectToLiveAgent') {
29 const message = await connectToLiveAgent();
30 console.log(message);
31 return;
32 }
33 });
34 return;
35 }
36
37 // NO TOOL CALLS
38 return await logStream(streamForReturn);
39}I have iterated over the toolCalls array, checked for the tool name and called the appropriate function against it inside the code.
Here is what the complete code will look like:
1import 'dotenv/config';
2import { getToolsFromStream, handleResponseStream, getRunner } from '@baseai/core';
3
4const ENDPOINT = `https://api.langbase.com/beta/pipes/run`;
5const userPrompt = `Hey`;
6// const userPrompt = `What is the monthly pricing of XYZ company?`;
7// const userPrompt = `I want to connect to a live agent`;
8
9async function main() {
10 const response = await callLangbasePipe({
11 prompt: userPrompt,
12 apiKey: process.env.LANGBASE_SUPPORT_AGENT_DECISION_MAKER_PIPE,
13 });
14
15 const { stream } = await handleResponseStream({ response });
16 const [streamForReturn, streamForTool] = stream.tee();
17
18 const toolCalls = await getToolsFromStream(streamForTool);
19
20 if (toolCalls.length) {
21 toolCalls.forEach(async toolCall => {
22 const toolName = toolCall.function.name;
23
24 if (toolName === 'getInfoFromDocs') {
25 const response = await getInfoFromDocs();
26 const { stream } = handleResponseStream({ response });
27 return await logStream(stream);
28 }
29
30 if (toolName === 'connectToLiveAgent') {
31 const message = await connectToLiveAgent();
32 console.log(message);
33 return;
34 }
35 });
36 return;
37 }
38
39 // NO TOOL CALLS
40 return await logStream(streamForReturn);
41}
42
43main();
44
45async function callLangbasePipe({ apiKey, prompt }) {
46 return await fetch(ENDPOINT, {
47 method: 'POST',
48 headers: {
49 'Content-Type': 'application/json',
50 Authorization: `Bearer ${apiKey}`,
51 },
52 body: JSON.stringify({
53 messages: [{ role: 'user', content: prompt }],
54 }),
55 });
56}
57
58async function getInfoFromDocs() {
59 return await callLangbasePipe({
60 prompt: userPrompt,
61 apiKey: process.env.LANGBASE_SUPPORT_ANSWER_AGENT_PIPE,
62 });
63}
64
65async function connectToLiveAgent() {
66 // connect to live agent
67 return `Hello! How can I help you today?`;
68}
69
70async function logStream(stream: ReadableStream<Uint8Array>) {
71 const runner = getRunner(stream);
72
73 runner.on('connect', () => {
74 console.log('======= Stream started. ======= \n');
75 });
76
77 runner.on('content', content => {
78 process.stdout.write(content);
79 });
80
81 runner.on('end', () => {
82 console.log('\n\n ======= Stream ended.======= ');
83 });
84
85 runner.on('error', error => {
86 console.error('Error:', error);
87 });
88}Step #7. Run AI support agent
All you need to do now is run the multi-agent AI support that we have just built. Go ahead and open your project terminal and run the following command:
1npx tsx index.tsThis will generate an AI response in your terminal.
======= Stream started. =======
I can only answer questions that are around XYZ company. How can I assist you with that?
======= Stream ended.=======
Here is what the reference diagram of this AI support agent looks like that uses multi-agent architecture to make a decision in order to respond to a user query.
Wrap up
We have successfully created an AI support agent that uses multi-agent architecture. You can also build all this AI infra locally using BaseAI – the web AI framework.
BaseAI – The first Web AI Framework.
The easiest way to build serverless autonomous AI agents with memory. Start building local-first, agentic pipes, tools, and memory. Deploy serverless with one command.
