Guide

API Layer in LobeChat

[Architecture: API Layer in LobeChat] [C: Introduction]

In this guide, you will learn the API layer used in Lobe Chat.

🤯 Lobe Chat is an open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / Azure / DeepSeek), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Vision/TTS) and plugin system. One-click FREE deployment of your private ChatGPT/ Claude application.

What's API Layer?

In the context of web applications, API layer means communicating with your server to get/mutate data. Inspired by Bulletproof React's API Layer We analyze the API Layer in the Lobechat.

Lobe Chat API Layer

To study the API Layer, we pick a page on https://chat-preview.lobehub.com/ and look at the source code except we are narrowing down our expedition to focus on the API Layer.

Concepts you will learn:

You will be learning the API layer used in the Lobe Chat. Read through this guide to understand how Lobe Chat's API layer works.

Data fetched in Discover Home Page

[Discover Home Page] (https://github.com/lobehub/lobe-chat/blob/main/src/app/(main)/discover/(list)/(home)/page.tsx)
DiscoverService

In this guide, we will find out how the API layer is implemented in Discover page. You will learn how the data is fetched in the Discover page.

[Insert a screenshot of Discover page - https://lobechat.com/discover]

Discover page has Home, Assistants, Plugins etc., tabs. For this guide, let's pick assistants tab and find out how the data is fetched.

In the Discover page.tsx, You will find the below code snippet.

const discoverService = new DiscoverService();
const assistantList = await discoverService.getAssistantList(locale);
const pluginList = await discoverService.getPluginList(locale);
const modelList = await discoverService.getModelList(locale);

DiscoverService

DiscoverService is class that is instantiated using new DiscoverService(). You could also just export the functions from a service file but using a class in a service helps you organize your functions better as your functions are encapsulated in a class. This approach also make it easy to mock your service for testing purposes.

discoverService.getAssistantList(locale)

getAssistantList = async (locale: Locales): Promise<DiscoverAssistantItem[]> => {
    let res = await fetch(this.assistantStore.getAgentIndexUrl(locale), {
      next: { revalidate },
    });

    if (!res.ok) {
      res = await fetch(this.assistantStore.getAgentIndexUrl(DEFAULT_LANG), {
        next: { revalidate },
      });
    }

    if (!res.ok) return [];

    const json = await res.json();

    return json.agents;
};

This above code snippet is picked from getAssistantList

This function uses fetch. Next.js extends the native Web fetch() API to allow each request on the server to set its own persistent caching semantics.

Let's understand what this second param in the above fetch does.

{
  next: { revalidate },
}

options.next.revalidate from the Next.js documentation.

Set the cache lifetime of a resource (in seconds).

false - Cache the resource indefinitely. Semantically equivalent to revalidate: Infinity. The HTTP cache may evict older resources over time.
0 - Prevent the resource from being cached.
number - (in seconds) Specify the resource should have a cache lifetime of at most n seconds.

so where's revalidate value coming from in Discover service? revalidate is initialized at L25:

const revalidate: number = 3600;

This 3600 is in seconds which is 1 hour. getAssistantList has a cache lifetime of at most 1 hour.

assistantStore.getAgentIndexUrl returns the URL used in the fetch API.

[Insert a screenshot of https://github.com/lobehub/lobe-chat/blob/main/src/server/modules/AssistantStore/index.ts#L14]

References:

Data fetched in Chat Page

Data fetched in the Chat page is different to what you have seen in getAssistantList in discover service. getAssistantList uses fetch, where as Chat uses useSWR. Let's find out how this is done.

[insert screenshot of lobechat chat session page]

You will find useFetchSessions hook in SessionListContent/DefaultMode.tsx useFetchSession is in useSessionStore. API layer is tightly coupled with state management in Lobechat. We want to focus only on API layer, check out the [state management in lobechat](link to lobechat state management).

[Insert screenshot of https://github.com/lobehub/lobe-chat/blob/main/src/app/(main)/chat/%40session/features/SessionListContent/DefaultMode.tsx#L29]

useFetchSessions in session store:

You will find the below code for the useFetchSessions in session/action.ts

useFetchSessions: (isLogin) =>
    useClientDataSWR<ChatSessionList>(
      [FETCH_SESSIONS_KEY, isLogin],
      () => sessionService.getGroupedSessions(),
      {
        fallbackData: {
          sessionGroups: [],
          sessions: [],
        },
        onSuccess: (data) => {
          if (
            get().isSessionsFirstFetchFinished &&
            isEqual(get().sessions, data.sessions) &&
            isEqual(get().sessionGroups, data.sessionGroups)
          )
            return;

          get().internal_processSessions(
            data.sessions,
            data.sessionGroups,
            n('useFetchSessions/updateData') as any,
          );
          set({ isSessionsFirstFetchFinished: true }, false, n('useFetchSessions/onSuccess', data));
        },
        suspense: true,
      },
    ),

Looks compicated, let's break it down. It uses:

useClientDataSWR

useClientDataSWR is imported from libs/swr/index.ts

export const useClientDataSWR: SWRHook = (key, fetch, config) =>
  useSWR(key, fetch, {
    // default is 2000ms ,it makes the user's quick switch don't work correctly.
    // Cause issue like this: https://github.com/lobehub/lobe-chat/issues/532
    // we need to set it to 0.
    dedupingInterval: 0,
    focusThrottleInterval: 5 * 60 * 1000,
    refreshWhenOffline: false,
    revalidateOnFocus: true,
    revalidateOnReconnect: true,
    ...config,
  });

useSWR is a React hook for data fetching provided by Vercel's SWR API. The name “SWR” is derived from stale-while-revalidate, a HTTP cache invalidation strategy popularized by HTTP RFC 5861(opens in a new tab). SWR is a strategy to first return the data from cache (stale), then send the fetch request (revalidate), and finally come with the up-to-date data.

Let's now understand what each of these options do by referring to the documentation: a. dedupingInterval

SWR Docs mentions this - dedupingInterval = 2000: dedupe requests with the same key in this time span in milliseconds

but Lobechat has this configured to 0 and it explains why with a nice comment.

// default is 2000ms ,it makes the user's quick switch don't work correctly.
// Cause issue like this: https://github.com/lobehub/lobe-chat/issues/532
// we need to set it to 0.
dedupingInterval: 0,

just in case, if you were wondering what dedupe means, it is about removing duplicates.

b. focusThrottleInterval:

SWR docs provides this information about focusThrottleInterval - focusThrottleInterval = 5000: only revalidate once during a time span in milliseconds

In the context of useSWR, the focusThrottleInterval option controls how frequently the data revalidation happens when the window or tab regains focus.

ChatGPT's explanation:

Explanation in the useSWR context: SWR's behavior with focus: By default, SWR will revalidate (i.e., refetch) the data whenever the user switches back to the app or tab (when the window or tab regains focus). This is useful because it ensures that the data displayed is up-to-date when the user returns to the app.

focusThrottleInterval = 5000: This option tells SWR to throttle the revalidation when focus changes. Specifically, it will revalidate data at most once every 5000 milliseconds (or 5 seconds).

If a user quickly switches between tabs or focuses on the app multiple times in rapid succession, SWR will ensure that revalidation only happens once within that 5-second window, preventing excessive or redundant requests.

What's the value that lobechat uses?

focusThrottleInterval: 5 * 60 * 1000

Converting 5 * 60 * 1000 milliseconds, it is 5 minutes. What this means is that when you focus by revisiting lobechat application, data is not refreshed until 5 minutes but by then you would visit different pages, differen chats and latest data might load already but that depends on the actions that you perform.

c. refreshWhenOffline:

SWR documentation explanation:

refreshWhenOffline = false: polling when the browser is offline (determined by navigator.onLine), Read more about refreshWhenOffline

ChatGPT explanation:

SWR offers an option to periodically refetch data in the background (polling), but when the browser goes offline, trying to fetch data may be pointless. To prevent unnecessary requests, setting refreshWhenOffline = false ensures polling is paused when the browser is offline, saving resources and avoiding failed fetch attempts.

What's the value that lobechat uses?

refreshWhenOffline: false

d. revalidateOnFocus:

SWR documentation explanation:

revalidateOnFocus = true: automatically revalidate when window gets focused (details)

ChatGPT explanation:

Revalidate: In SWR, "revalidate" refers to refetching data from the source (e.g., an API) to ensure the data is still up-to-date.
Focus events: In web applications, a "focus" event occurs when the user returns to the browser window or tab, either after switching tabs, windows, or after minimizing and reopening the app.
revalidateOnFocus = true: This setting enables the app to automatically revalidate (refetch data) every time the user switches back to the tab. By default, this is set to true in SWR, meaning the data will be refreshed every time the tab regains focus.

What's the value that lobechat uses?

revalidateOnFocus: true,

d. revalidateOnReconnect:

SWR documentation explanation:

revalidateOnReconnect = true: automatically revalidate when the browser regains a network connection (via navigator.onLine) (details)

ChatGPT explanation:

Revalidate: In SWR, revalidation refers to refetching data from the server to ensure it's up-to-date.
Reconnect: This event happens when the browser loses and then regains internet connectivity. The browser detects this change using the navigator.onLine property: true: Browser is online (connected to the internet). false: Browser is offline (disconnected from the internet).
revalidateOnReconnect = true: This setting ensures that whenever the browser switches from an offline state back to an online state, SWR will automatically refetch the data.

What's the value that lobechat uses?

revalidateOnReconnect: true

Params:

Now that we understand useClientDataSWR and its options, let's look at the params used. The first param is the key

useClientDataSWR<ChatSessionList>(
      [FETCH_SESSIONS_KEY, isLogin],

The second param is the fetch function written in sessionService.getGroupedSessions()

Third params is the additional config passed to useClientDataSWR function.

At this point, we are introduced to the new unknown sessionService. Let's analyze sessionService.

References:

[C: sessionService]

In this guide, we analyze the sessionService, that gets called in a store.

The way we approach this is that we pick a function, understand it and see where it leads us.

getGroupedSessions()

getGroupedSessions() has the below code:

getGroupedSessions(): Promise<ChatSessionList> {
  return lambdaClient.session.getGroupedSessions.query();
}

The unknown here is lambdaClient. What could this be about? to find that out, you have to see where lambdaClient is imported from.

import { lambdaClient } from '@/libs/trpc/client';

lambdaClient is imported from @libs/trpc/client. This tells us that Lobechat uses tRPC. What's TRPC?

tRPC

Move fast and break nothing. tRPC provides end-to-end typesafe APIs for your fullstack application.

Since Lobechat uses app router, you may want to read this set up with React components guide to understand how the tRPC is configured in Lobechat codebase.

We refer to the steps provided in tRPC Setup with React Components and find out how Lobechat configured tRPC.

Useful links:

Dependencies required

tRPC docs tells us to install the following dependencies in your project:

@trpc/server@next
@trpc/client@next
@trpc/react-query@next
@tanstack/react-query@latest
zod
client-only
server-only

Lobechat has the following packages installed, available in package.json

"@tanstack/react-query": "^5.59.15",
"@trpc/client": "next",
"@trpc/next": "next",
"@trpc/react-query": "next",
"@trpc/server": "next",
"zod": "^3.23.8",

There is no 'client-only', 'server-only' packages in Lobechat package.json and has a package named @trpc/next installed.

Create a tRPC router

tRPC Documentation summary

You would need create a tRPC router as explained in tRPC docs

You create the following files:

trpc/init.ts
trpc/routers/_app.ts
app/api/trpc/[trpc]/route.ts

Lobechat implementation:

init.ts

You will find init.ts in src/libs/trpc/init.ts

export const trpc = initTRPC.context<Context>().create({
  /**
   * @link https://trpc.io/docs/v11/error-formatting
   */
  errorFormatter({ shape }) {
    return shape;
  },
  /**
   * @link https://trpc.io/docs/v11/data-transformers
   */
  transformer: superjson,
});

Only tRPC is exported, but if you were to look at init.ts provided in documentation, it has the below code:

const t = initTRPC.create({
  /**
   * @see https://trpc.io/docs/server/data-transformers
   */
  // transformer: superjson,
});
// Base router and procedure helpers
export const createTRPCRouter = t.router;
export const baseProcedure = t.procedure;

Examples provided in documentation exports router and procedure but Lobechat only exports trpc which is t in tRPC documentation example.

routers

In the tRPC documentation, routers are defined in trpc/routers/_app.ts.

import { z } from 'zod';
import { baseProcedure, createTRPCRouter } from '../init';
export const appRouter = createTRPCRouter({
  hello: baseProcedure
    .input(
      z.object({
        text: z.string(),
      }),
    )
    .query((opts) => {
      return {
        greeting: `hello ${opts.input.text}`,
      };
    }),
});
// export type definition of API
export type AppRouter = typeof appRouter;

Lobechat has a folder named routers in server folder, We chose lambda/session.ts inside server folder for explanation in this guide.

For you to register a route, you need trpc.router. This is imported from @/lib/trpc.

import { authedProcedure, publicProcedure, router } from '@/libs/trpc';

For example, getSessions is registered with the router as shown below:

getSessions: sessionProcedure
    .input(
      z.object({
        current: z.number().optional(),
        pageSize: z.number().optional(),
      }),
    )
    .query(async ({ input, ctx }) => {
      const { current, pageSize } = input;

      return ctx.sessionModel.query({ current, pageSize });
    }),

we have now been introduced to sessionModel that makes the database calls, we will learn more about sessionModel in the next article.

sessionProcedure is defined after the imports in lambda/session.ts as shown below:

const sessionProcedure = authedProcedure.use(async (opts) => {
  const { ctx } = opts;

  return opts.next({
    ctx: {
      sessionGroupModel: new SessionGroupModel(ctx.userId),
      sessionModel: new SessionModel(ctx.userId),
    },
  });
});

authedProcedure is from @libs/trpc.

// procedure that asserts that the user is logged in
export const authedProcedure = trpc.procedure.use(userAuth);

[C: SessionModel]

Based on the example, getSessions, we picked in the previous article, sessionModel.query is used. query function in sessionModel has the below code:

async query({ current = 0, pageSize = 9999 } = {}) {
    const offset = current * pageSize;

    return serverDB.query.sessions.findMany({
      limit: pageSize,
      offset,
      orderBy: [desc(sessions.updatedAt)],
      where: and(eq(sessions.userId, this.userId), not(eq(sessions.slug, INBOX_SESSION_ID))),
      with: { agentsToSessions: { columns: {}, with: { agent: true } }, group: true },
    });
}

Unknown here is serverDB. serverDB is imported from database/server/core/db;

import { serverDB } from '@/database/server/core/db';

In Lobechat's tRPC procedures, models are used as an abstraction instead of directly dealing with database calls.

[C: Database Core API]

In the previous chapter, we learnt that model uses serverDB imported from @/database/server/core/db. In this chapter, we analyze the Lobechat's database core API.

The below imports are picked from database/server/core/db.ts

import { Pool as NeonPool, neonConfig } from '@neondatabase/serverless';
import { NeonDatabase, drizzle as neonDrizzle } from 'drizzle-orm/neon-serverless';
import { drizzle as nodeDrizzle } from 'drizzle-orm/node-postgres';
import { Pool as NodePool } from 'pg';
import ws from 'ws';

import { serverDBEnv } from '@/config/db';
import { isServerMode } from '@/const/version';

import * as schema from '../schemas/lobechat';

You can tell from the imports that Lobechat uses Drizzle ORM and Neon database. Check out this Drizzle with Neon tutorial to find out more. This tutorial shows how to connect to Neon database using Drizzle.

import { drizzle } from "drizzle-orm/neon-http";
import { neon } from "@neondatabase/serverless";
import { config } from "dotenv";
config({ path: ".env" }); // or .env.local
const sql = neon(process.env.DATABASE_URL!);
export const db = drizzle({ client: sql });

Lobechat has additional checks as to which drizzle ORM is used. For example, in the below code:

if (serverDBEnv.DATABASE_DRIVER === 'node') {
    const client = new NodePool({ connectionString });
    return nodeDrizzle(client, { schema });
}

nodeDrizzle here is imported from drizzle-orm/node-postgres.

serverDBenv is imported from config/db.ts

import { createEnv } from '@t3-oss/env-nextjs';
import { z } from 'zod';

export const getServerDBConfig = () => {
  return createEnv({
    client: {
      NEXT_PUBLIC_ENABLED_SERVER_SERVICE: z.boolean(),
    },
    runtimeEnv: {
      DATABASE_DRIVER: process.env.DATABASE_DRIVER || 'neon',
      DATABASE_TEST_URL: process.env.DATABASE_TEST_URL,
      DATABASE_URL: process.env.DATABASE_URL,

      DISABLE_REMOVE_GLOBAL_FILE: process.env.DISABLE_REMOVE_GLOBAL_FILE === '1',

      KEY_VAULTS_SECRET: process.env.KEY_VAULTS_SECRET,

      NEXT_PUBLIC_ENABLED_SERVER_SERVICE: process.env.NEXT_PUBLIC_SERVICE_MODE === 'server',
    },
    server: {
      DATABASE_DRIVER: z.enum(['neon', 'node']),
      DATABASE_TEST_URL: z.string().optional(),
      DATABASE_URL: z.string().optional(),

      DISABLE_REMOVE_GLOBAL_FILE: z.boolean().optional(),

      KEY_VAULTS_SECRET: z.string().optional(),
    },
  });
};

export const serverDBEnv = getServerDBConfig();

getServerDBConfig uses createEnv from @t3-oss/env-nextjs.

Finally, neonDrizzle is returned from this function - getDBInstance().

const client = new NeonPool({ connectionString });
return neonDrizzle(client, { schema });

schema in the above code snippet is imported from schemas/lobechat file.

session.ts Schema

Since Lobechat uses Drizzle, we picked schema/lobechat/session.ts as an example.

imports used are:

import { boolean, integer, pgTable, text, unique, uniqueIndex, varchar } from 'drizzle-orm/pg-core';
import { createInsertSchema } from 'drizzle-zod';

Session table defined:

export const sessions = pgTable(
  'sessions',
  {
    id: text('id')
      .$defaultFn(() => idGenerator('sessions'))
      .primaryKey(),
    slug: varchar('slug', { length: 100 })
      .notNull()
      .$defaultFn(() => randomSlug()),
    title: text('title'),
    description: text('description'),
    avatar: text('avatar'),
    backgroundColor: text('background_color'),

    type: text('type', { enum: ['agent', 'group'] }).default('agent'),

    userId: text('user_id')
      .references(() => users.id, { onDelete: 'cascade' })
      .notNull(),
    groupId: text('group_id').references(() => sessionGroups.id, { onDelete: 'set null' }),
    clientId: text('client_id'),
    pinned: boolean('pinned').default(false),

    createdAt: createdAt(),
    updatedAt: updatedAt(),
  },
  (t) => ({
    slugUserIdUnique: uniqueIndex('slug_user_id_unique').on(t.slug, t.userId),

    clientIdUnique: unique('sessions_client_id_user_id_unique').on(t.clientId, t.userId),
  }),
);

You also want to make sure to define your schema path in the drizzle.config.ts

Introduction

Design. Build. Ship.

We are experts at Next.js, React and Node.js

We analyse large open-source projects and bring experience and best practices to ship quality software products.