V4 (#311)

Sourcebot V4 introduces authentication, performance improvements and code navigation. Checkout the [migration guide](https://docs.sourcebot.dev/self-hosting/upgrade/v3-to-v4-guide) for information on upgrading your instance to v4.

### Changed
- [**Breaking Change**] Authentication is now required by default. Notes:
  - When setting up your instance, email / password login will be the default authentication provider.
  - The first user that logs into the instance is given the `owner` role. ([docs](https://docs.sourcebot.dev/docs/more/roles-and-permissions)).
  - Subsequent users can request to join the instance. The `owner` can approve / deny requests to join the instance via `Settings` > `Members` > `Pending Requests`.
  - If a user is approved to join the instance, they are given the `member` role.
  - Additional login providers, including email links and SSO, can be configured with additional environment variables. ([docs](https://docs.sourcebot.dev/self-hosting/configuration/authentication)).
- Clicking on a search result now takes you to the `/browse` view. Files can still be previewed by clicking the "Preview" button or holding `Cmd` / `Ctrl` when clicking on a search result. [#315](#315)

### Added
- [Sourcebot EE] Added search-based code navigation, allowing you to jump between symbol definition and references when viewing source files. [Read the documentation](https://docs.sourcebot.dev/docs/search/code-navigation). [#315](#315)
- Added collapsible filter panel. [#315](#315)

### Fixed
- Improved scroll performance for large numbers of search results. [#315](#315)

Author: msukkari

.cursor/rules/style.mdc

@@ -0,0 +1,7 @@
+---
+description: 
+globs: 
+alwaysApply: true
+---
+- Always use 4 spaces for indentation
+- Filenames should always be camelCase. Exception: if there are filenames in the same directory with a format other than camelCase, use that format to keep things consistent.

.env.development

@@ -18,10 +18,10 @@ SRC_TENANT_ENFORCEMENT_MODE=strict
 AUTH_SECRET="00000000000000000000000000000000000000000000"
 AUTH_URL="http://localhost:3000"
 # AUTH_CREDENTIALS_LOGIN_ENABLED=true
-# AUTH_GITHUB_CLIENT_ID=""
-# AUTH_GITHUB_CLIENT_SECRET=""
-# AUTH_GOOGLE_CLIENT_ID=""
-# AUTH_GOOGLE_CLIENT_SECRET=""
+# AUTH_EE_GITHUB_CLIENT_ID=""
+# AUTH_EE_GITHUB_CLIENT_SECRET=""
+# AUTH_EE_GOOGLE_CLIENT_ID=""
+# AUTH_EE_GOOGLE_CLIENT_SECRET=""
 
 DATA_CACHE_DIR=${PWD}/.sourcebot  # Path to the sourcebot cache dir (ex. ~/sourcebot/.sourcebot)
 # CONFIG_PATH=${PWD}/config.json # Path to the sourcebot config file (if one exists)

CHANGELOG.md

@@ -7,6 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+Sourcebot V4 introduces authentication, performance improvements and code navigation. Checkout the [migration guide](https://docs.sourcebot.dev/self-hosting/upgrade/v3-to-v4-guide) for information on upgrading your instance to v4.
+
+### Changed
+- [**Breaking Change**] Authentication is now required by default. Notes:
+  - When setting up your instance, email / password login will be the default authentication provider.
+  - The first user that logs into the instance is given the `owner` role. ([docs](https://docs.sourcebot.dev/docs/more/roles-and-permissions)).
+  - Subsequent users can request to join the instance. The `owner` can approve / deny requests to join the instance via `Settings` > `Members` > `Pending Requests`.
+  - If a user is approved to join the instance, they are given the `member` role.
+  - Additional login providers, including email links and SSO, can be configured with additional environment variables. ([docs](https://docs.sourcebot.dev/self-hosting/configuration/authentication)).
+- Clicking on a search result now takes you to the `/browse` view. Files can still be previewed by clicking the "Preview" button or holding `Cmd` / `Ctrl` when clicking on a search result. [#315](https://github.com/sourcebot-dev/sourcebot/pull/315)
+
+### Added
+- [Sourcebot EE] Added search-based code navigation, allowing you to jump between symbol definition and references when viewing source files. [Read the documentation](https://docs.sourcebot.dev/docs/search/code-navigation). [#315](https://github.com/sourcebot-dev/sourcebot/pull/315)
+- Added collapsible filter panel. [#315](https://github.com/sourcebot-dev/sourcebot/pull/315)
+- Added Sourcebot API key management for external clients. [#311](https://github.com/sourcebot-dev/sourcebot/pull/311)
+
+### Fixed
+- Improved scroll performance for large numbers of search results. [#315](https://github.com/sourcebot-dev/sourcebot/pull/315)
+
 ## [3.2.1] - 2025-05-15
 
 ### Added
@@ -93,8 +112,8 @@ Sourcebot v3 is here and brings a number of structural changes to the tool's fou
 ### Added
 - Added parallelized repo indexing and connection syncing via Redis & BullMQ. See the [architecture overview](https://docs.sourcebot.dev/self-hosting/overview#architecture).
 - Added repo indexing progress indicators in the navbar.
-- Added authentication support via OAuth or email/password. For instructions on enabling, see [this doc](https://docs.sourcebot.dev/self-hosting/more/authentication).
-- Added the following UI for managing your deployment when **[auth is enabled](https://docs.sourcebot.dev/self-hosting/more/authentication)**:
+- Added authentication support via OAuth or email/password. For instructions on enabling, see [this doc](https://docs.sourcebot.dev/self-hosting/configuration/authentication).
+- Added the following UI for managing your deployment when **[auth is enabled](https://docs.sourcebot.dev/self-hosting/configuration/authentication)**:
   - connection management: create and manage your JSON configs via a integrated web-editor.
   - secrets: import personal access tokens (PAT) into Sourcebot (AES-256 encrypted). Reference secrets in your connection config by name.
   - team & invite management: invite users to your instance to give them access. Configure team [roles & permissions](https://docs.sourcebot.dev/docs/more/roles-and-permissions).

docs/docs.json

@@ -50,6 +50,7 @@
                         "pages": [
                             "docs/search/syntax-reference",
                             "docs/search/multi-branch-indexing",
+                            "docs/search/code-navigation",
                             "docs/search/search-contexts"
                         ]
                     },
@@ -63,6 +64,7 @@
                     {
                         "group": "More",
                         "pages": [
+                            "docs/more/api-keys",
                             "docs/more/roles-and-permissions",
                             "docs/more/mcp-server"
                         ]
@@ -77,17 +79,16 @@
                         "group": "Getting Started",
                         "pages": [
                             "self-hosting/overview",
-                            "self-hosting/configuration",
                             "self-hosting/license-key"
                         ]
                     },
                     {
-                        "group": "More",
+                        "group": "Configuration",
                         "pages": [
-                            "self-hosting/more/authentication",
-                            "self-hosting/more/tenancy",
-                            "self-hosting/more/transactional-emails",
-                            "self-hosting/more/declarative-config"
+                            "self-hosting/configuration/environment-variables",
+                            "self-hosting/configuration/authentication",
+                            "self-hosting/configuration/transactional-emails",
+                            "self-hosting/configuration/declarative-config"
                         ]
                     },
                     {
@@ -98,6 +99,7 @@
                     {
                         "group": "Upgrade",
                         "pages": [
+                            "self-hosting/upgrade/v3-to-v4-guide",
                             "self-hosting/upgrade/v2-to-v3-guide"
                         ]
                     }

docs/docs/agents/review-agent.mdx

@@ -53,6 +53,7 @@ Before you get started, make sure you have an OpenAPI account that you can creat
         directory that you mount to Sourcebot
         ![GitHub App Private Key](/images/github_app_private_key.png)
         - `OPENAI_API_KEY`: Your OpenAI API key
+        - `REVIEW_AGENT_API_KEY`: The Sourcebot API key that the review agent uses to hit the Sourcebot API to fetch code context
         - `REVIEW_AGENT_AUTO_REVIEW_ENABLED` (default: `false`): If enabled, the review agent will automatically review any new or updated PR. If disabled, you must invoke it using the command defined by `REVIEW_AGENT_REVIEW_COMMAND`
         - `REVIEW_AGENT_REVIEW_COMMAND` (default: `review`): The command that invokes the review agent (ex. `/review`) when a user comments on the PR. Don't include the slash character in this value.
 
@@ -76,6 +77,7 @@ Before you get started, make sure you have an OpenAPI account that you can creat
                     GITHUB_APP_ID: "my-github-app-id"
                     GITHUB_APP_WEBHOOK_SECRET: "my-github-app-webhook-secret"
                     GITHUB_APP_PRIVATE_KEY_PATH: "/data/review-agent-key.pem"
+                    REVIEW_AGENT_API_KEY: "sourcebot-my-key"
                     OPENAI_API_KEY: "sk-proj-my-open-api-key"
         ```
     </Step>

docs/docs/connections/gitea.mdx

@@ -82,7 +82,7 @@ Next, provide the access token via the `token` property, either as an environmen
 
 <Tabs>
     <Tab title="Environment Variable">
-        <Note>Environment variables are only supported in a [declarative config](/self-hosting/more/declarative-config) and cannot be used in the web UI.</Note>
+        <Note>Environment variables are only supported in a [declarative config](/self-hosting/configuration/declarative-config) and cannot be used in the web UI.</Note>
 
         1. Add the `token` property to your connection config:
         ```json
@@ -107,7 +107,7 @@ Next, provide the access token via the `token` property, either as an environmen
     </Tab>
 
     <Tab title="Secret">
-        <Note>Secrets are only supported when [authentication](/self-hosting/more/authentication) is enabled.</Note>
+        <Note>Secrets are only supported when [authentication](/self-hosting/configuration/authentication) is enabled.</Note>
 
         1. Navigate to **Secrets** in settings and create a new secret with your PAT:
 

docs/docs/connections/github.mdx

@@ -111,7 +111,7 @@ Next, provide the PAT via the `token` property, either as an environment variabl
 
 <Tabs>
     <Tab title="Environment Variable">
-        <Note>Environment variables are only supported in a [declarative config](/self-hosting/more/declarative-config) and cannot be used in the web UI.</Note>
+        <Note>Environment variables are only supported in a [declarative config](/self-hosting/configuration/declarative-config) and cannot be used in the web UI.</Note>
 
         1. Add the `token` property to your connection config:
         ```json
@@ -136,7 +136,7 @@ Next, provide the PAT via the `token` property, either as an environment variabl
     </Tab>
 
     <Tab title="Secret">
-        <Note>Secrets are only supported when [authentication](/self-hosting/more/authentication) is enabled.</Note>
+        <Note>Secrets are only supported when [authentication](/self-hosting/configuration/authentication) is enabled.</Note>
 
         1. Navigate to **Secrets** in settings and create a new secret with your PAT:
 

docs/docs/connections/gitlab.mdx

@@ -116,7 +116,7 @@ Next, provide the PAT via the `token` property, either as an environment variabl
 
 <Tabs>
     <Tab title="Environment Variable">
-        <Note>Environment variables are only supported in a [declarative config](/self-hosting/more/declarative-config) and cannot be used in the web UI.</Note>
+        <Note>Environment variables are only supported in a [declarative config](/self-hosting/configuration/declarative-config) and cannot be used in the web UI.</Note>
 
         1. Add the `token` property to your connection config:
         ```json
@@ -141,7 +141,7 @@ Next, provide the PAT via the `token` property, either as an environment variabl
     </Tab>
 
     <Tab title="Secret">
-        <Note>Secrets are only supported when [authentication](/self-hosting/more/authentication) is enabled.</Note>
+        <Note>Secrets are only supported when [authentication](/self-hosting/configuration/authentication) is enabled.</Note>
 
         1. Navigate to **Secrets** in settings and create a new secret with your PAT:
 

docs/docs/connections/overview.mdx

@@ -11,11 +11,11 @@ There are two ways to define connections:
 
 <AccordionGroup>
     <Accordion title="Declarative configuration file">
-        This is only supported when self-hosting, and is the default mechanism to define connections. Connections are defined in a [JSON file](/self-hosting/more/declarative-config)
+        This is only supported when self-hosting, and is the default mechanism to define connections. Connections are defined in a [JSON file](/self-hosting/configuration/declarative-config)
         and the path to the file is provided through the `CONFIG_PATH` environment variable
     </Accordion>
     <Accordion title="UI connection management">
-        This is the only way to define connections when using Sourcebot Cloud, and can be configured when self-hosting by enabling [authentication](/self-hosting/more/authentications).
+        This is the only way to define connections when using Sourcebot Cloud, and can be configured when self-hosting by enabling [authentication](/self-hosting/configuration/authentications).
 
         In this method, connections are defined and managed within the webapp:
 

docs/docs/more/api-keys.mdx

@@ -0,0 +1,8 @@
+---
+title: API Keys
+---
+
+An API Key is required when querying Sourcebot outside the context of the web app client (ex. MCP server, review agent). To create an API key, login to your Sourcebot instance and navigate to
+**Settings -> API Keys**:
+
+![API Keys UI](/images/api_key.png)

docs/docs/more/mcp-server.mdx

@@ -4,7 +4,7 @@ sidebarTitle: Sourcebot MCP server
 ---
 
 <Note>
-This feature is only available when [self-hosting](/self-hosting) with [authentication](/self-hosting/more/authentication) disabled.
+This feature is only available when [self-hosting](/self-hosting)
 </Note>
 
 The [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) is an open standard for providing context to LLMs. The [@sourcebot/mcp](https://www.npmjs.com/package/@sourcebot/mcp) package is a MCP server that enables LLMs to interface with your Sourcebot instance, enabling MCP clients like Cursor, Vscode, and others to have context over your entire codebase.
@@ -176,6 +176,7 @@ Parameters:
 | Name                     | Default                | Description                                       |
 |:-------------------------|:-----------------------|:--------------------------------------------------|
 | `SOURCEBOT_HOST`         | http://localhost:3000  | URL of your Sourcebot instance.                   |
+| `SOURCEBOT_API_KEY`      | -                      | Sourcebot API key.                                |
 | `DEFAULT_MINIMUM_TOKENS` | 10000                  | Minimum number of tokens to return in responses.  |
 | `DEFAULT_MATCHES`        | 10000                  | Number of code matches to fetch per search.       |
 | `DEFAULT_CONTEXT_LINES`  | 5                      | Lines of context to include above/below matches.  |

docs/docs/more/roles-and-permissions.mdx

@@ -4,8 +4,7 @@ title: Roles and Permissions
 
 <Note>Looking to sync permissions with your identify provider? We're working on it - [reach out](https://www.sourcebot.dev/contact) to us to learn more</Note>
 
-If you're using Sourcebot Cloud, or are self-hosting with [authentication](/self-hosting/more/authentication) enabled, you may have multiple members in your organization. Each
-member has a role which defines their permissions:
+Each member has a role which defines their permissions within an organization:
 
 | Role | Permission |
 | :--- | :--------- |

docs/docs/search/code-navigation.mdx

@@ -0,0 +1,44 @@
+---
+title: Code navigation
+sidebarTitle: Code navigation
+---
+
+import SearchContextSchema from '/snippets/schemas/v3/searchContext.schema.mdx'
+
+<Note>
+This feature is only available in [Sourcebot cloud](app.sourcebot.dev) or with an active Enterprise license when [self-hosting](/self-hosting). Please add your [license key](/self-hosting/license-key) to activate it.
+</Note>
+
+**Code navigation** allows you to jump between symbol definition and references when viewing source files in Sourcebot. This feature is enabled **automatically** when a valid license key is present and works with all popular programming languages.
+
+
+<video src="https://framerusercontent.com/assets/B9ZxrlsUeO9NJyzkKyvVV2KSU4.mp4" className="w-full aspect-video" controls></video>
+
+## Features
+
+| Feature | Description |
+|:--------|:------------|
+| **Hover popover** | Hovering over a symbol reveals the symbol's definition signature as a inline preview. |
+| **Go to definition** | Clicking the "go to definition" button in the popover or clicking the symbol name navigates to the symbol's definition. |
+| **Find references** | Clicking the "find all references" button in the popover lists all references in the explore panel. |
+| **Explore panel** | Lists all references and definitions for the symbol selected in the popover. |
+
+## How does it work?
+
+Code navigation is **search-based**, meaning it uses the same code search engine and [query language](/docs/search/syntax-reference) to estimate a symbol's references and definitions. We refer to these estimations as "search heuristics". We have two search heuristics to enable the following operations:
+
+### Find references
+Given a `symbolName`, along with information about the file the symbol is contained within (`git_revision`, and `language`), runs the following search:
+
+```bash
+\\b{symbolName}\\b rev:{git_revision} lang:{language} case:yes
+```
+
+### Find definitions
+Given a `symbolName`, along with information about the file the symbol is contained within (`git_revision`, and `language`), runs the following search:
+
+```bash
+sym:\\b{symbolName}\\b rev:{git_revision} lang:{language}
+```
+
+Note that the `sym:` prefix is used to filter the search by symbol definitions. These are created at index time by [universal ctags](https://ctags.io/).

docs/docs/search/search-contexts.mdx

@@ -1,6 +1,6 @@
 ---
 title: Search contexts
-sidebarTitle: Search contexts (EE)
+sidebarTitle: Search contexts
 ---
 
 import SearchContextSchema from '/snippets/schemas/v3/searchContext.schema.mdx'
@@ -16,7 +16,7 @@ A **search context** is a user-defined grouping of repositories that helps focus
 - `( context:project1 or context:project2 ) logger\.debug` - search for debug log calls in project1 and project2 
 
 
-Search contexts are defined in the `context` object inside of a [declarative config](/self-hosting/more/declarative-config). Repositories can be included / excluded from a search context by specifying the repo's URL in either the `include` array or `exclude` array. Glob patterns are supported.
+Search contexts are defined in the `context` object inside of a [declarative config](/self-hosting/configuration/declarative-config). Repositories can be included / excluded from a search context by specifying the repo's URL in either the `include` array or `exclude` array. Glob patterns are supported.
 
 ## Example
 
@@ -41,7 +41,7 @@ shared/
 ├─ ...
 ```
 
-To make searching easier, we can create three search contexts in our [config.json](/self-hosting/more/declarative-config):
+To make searching easier, we can create three search contexts in our [config.json](/self-hosting/configuration/declarative-config):
 - `web`: For all frontend-related code
 - `backend`: For backend services and shared APIs
 - `pipelines`: For all CI/CD configurations