HTTP 500 on explore/code search after repo delete (with enabled code indexer via elasticsearch)

[IT-AlexKor]

Gitea Version

1.16.3

Git Version

1.8.3.1

Operating System

CentOS Linux 7 (Core)

How are you running Gitea?

Self-hosted instance in ProxMox

configuration:

APP_NAME = <ERASED>
RUN_USER = git
RUN_MODE = prod

[security]
INTERNAL_TOKEN     = <ERASED>
INSTALL_LOCK       = true
SECRET_KEY         = <ERASED>
PASSWORD_HASH_ALGO = pbkdf2
DISABLE_GIT_HOOKS  = false

[database]
DB_TYPE  = postgres
HOST     = <ERASED>
NAME     = <ERASED>
USER     = <ERASED>
PASSWD   = <ERASED>
SCHEMA   =
SSL_MODE = disable
CHARSET  = utf8
PATH     = /var/lib/gitea/data/gitea.db
LOG_SQL  = false

[repository]
ROOT = /var/lib/gitea/data/gitea-repositories

[server]
SSH_DOMAIN       = <ERASED>
DOMAIN           = <ERASED>
HTTP_PORT        = 3000
ROOT_URL         = https://<ERASED>/
DISABLE_SSH      = false
SSH_PORT         = 22
LFS_START_SERVER = true
LFS_CONTENT_PATH = /var/lib/gitea/data/lfs
LFS_JWT_SECRET   = <ERASED>
OFFLINE_MODE     = false

[mailer]
ENABLED = true
HOST    = <ERASED>
FROM    = <ERASED>
USER    =
PASSWD  =

[service]
REGISTER_EMAIL_CONFIRM            = false
ENABLE_NOTIFY_MAIL                = false
DISABLE_REGISTRATION              = false
ALLOW_ONLY_EXTERNAL_REGISTRATION  = false
ENABLE_CAPTCHA                    = false
REQUIRE_SIGNIN_VIEW               = true
DEFAULT_KEEP_EMAIL_PRIVATE        = false
DEFAULT_ALLOW_CREATE_ORGANIZATION = true
DEFAULT_ENABLE_TIMETRACKING       = true
NO_REPLY_ADDRESS                  = noreply.localhost

[picture]
DISABLE_GRAVATAR        = false
ENABLE_FEDERATED_AVATAR = true

[openid]
ENABLE_OPENID_SIGNIN = true
ENABLE_OPENID_SIGNUP = true

[session]
PROVIDER = file

[log]
MODE      = console
LEVEL     = info
ROOT_PATH = /var/lib/gitea/log
ROUTER    = console

[metrics]
ENABLED = true

[mirror]
MIN_INTERVAL = 1m

[indexer]
ISSUE_INDEXER_TYPE = elasticsearch
ISSUE_INDEXER_CONN_STR = http://<ERASED>:9200
ISSUE_INDEXER_NAME = gitea_issues
ISSUE_INDEXER_PATH = indexers/issues.bleve
REPO_INDEXER_ENABLED = true
REPO_INDEXER_TYPE = elasticsearch
REPO_INDEXER_CONN_STR = http://<ERASED>:9200
REPO_INDEXER_NAME = gitea_codes
REPO_INDEXER_PATH = indexers/repos.bleve
UPDATE_BUFFER_LEN = 20
MAX_FILE_SIZE = 1048576
REPO_INDEXER_INCLUDE =
REPO_INDEXER_EXCLUDE = resources/bin/**
STARTUP_TIMEOUT = 30s

Database

PostgreSQL

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Description

Steps to reproduce:

1. (maybe can be skipped) Migrate repo with pulling enabled
2. (maybe can be skipped) Convert repo to regular
3. Force repo code indexing by pressins button "Add to Reindex Queue" (in repo settings), wait until it ends
4. Search from some code string in this repo via Explore/Code (type "match"). You should see your code in this repo
5. Delete repo via "Delete This Repository" button (in repo settings)
6. Repeat you search from step 4. You will receive HTTP 500 from https://.../explore/code?tab=&q=addSbtPlugin&t=match

Error from logs

Mar 14 12:37:07 ci-gitea gitea: #033[36m2022/03/14 12:37:07 #033[0mStarted #033[34mGET#033[0m #033[1m/explore/code?tab=&q=addSbtPlugin&t=match#033[0m for #033[1m127.0.0.1:55048#033[0m

Mar 14 12:37:08 ci-gitea gitea: #033[36m2022/03/14 12:37:08 #033[0m#033[32m...v7@v7.0.25/client.go:1500:#033[32mPerformRequest()#033[0m #033[1;32m[I]#033[0m #033[1mPOST#033[0m #033[1mhttp://<ERASED>:9200/gitea_codes/_search#033[0m [status:#033[1m200#033[0m, request:#033[1m0.842#033[0ms]

Mar 14 12:37:08 ci-gitea gitea: #033[36m2022/03/14 12:37:08 #033[0m#033[32m...s/context/context.go:208:#033[32mHTML()#033[0m #033[1;31m[E]#033[0m #033[1mRender failed#033[0m: #033[1mtemplate: explore/code:41:40: executing "explore/code" at <$repo.HTMLURL>: error calling HTMLURL: runtime error: invalid memory address or nil pointer dereference#033[0m

Mar 14 12:37:08 ci-gitea gitea: #033[36m2022/03/14 12:37:08 #033[0mCompleted #033[34mGET#033[0m #033[1m/explore/code?tab=&q=addSbtPlugin&t=match#033[0m #033[1;41m500#033[0m #033[1;41mInternal Server Error#033[0m in #033[33m875.460841ms#033[0m

If you need more information - please tell

Screenshots

No response

[IT-AlexKor]

Debug log

2022/03/14 13:44:42 ...v7@v7.0.25/client.go:860:dumpRequest() [D] POST /gitea_codes/_search HTTP/1.1\015
        Host: <ERASED>:9200\015
        User-Agent: elastic/7.0.25 (linux-amd64)\015
        Transfer-Encoding: chunked\015
        Accept: application/json\015
        Content-Type: application/json\015
        Accept-Encoding: gzip\015
        \015
        154\015
        {"aggregations":{"language":{"terms":{"field":"language","order":[{"_count":"desc"}],"size":10}}},"from":0,"highlight":{"fields":{"content":{}},"number_of_fragments":0,"type":"fvh"},"query":{"bool":{"must":{"multi_match":{"fields":["content"],"query":"addSbtPlugin","type":"phrase_prefix"}}}},"size":10,"sort":[{"repo_id":{"order":"asc"}}]}\015
        0\015
        \015

2022/03/14 13:44:43 ...v7@v7.0.25/client.go:870:dumpResponse() [D] HTTP/1.1 200 OK\015
        Content-Type: application/json; charset=UTF-8\015
        \015
        {"took":1018,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":null,"hits":[{"_index":"gitea_codes.v1","_type":"_doc","_id":"1l_project/assembly.sbt","_score":null,"_source":{"commit_id":"e453fcba028617ed8bba32f670139dafe2d722e3","content":"addSbtPlugin(\"com.eed3si9n\" % \"sbt-assembly\" % \"0.12.0\")\n","language":"Scala","repo_id":57,"updated_at":1647247491},"highlight":{"content":["<em>addSbtPlugin</em>(\"com.eed3si9n\" % \"sbt-assembly\" % \"0.12.0\")\n"]},"sort":[57]}]},"aggregations":{"language":{"doc_count_error_upper_bound":0,"sum_other_doc_count":0,"buckets":[{"key":"Scala","doc_count":1}]}}}
2022/03/14 13:44:43 ...s/context/context.go:198:HTML() [D] Template: explore/code
2022/03/14 13:44:43 ...s/context/context.go:208:HTML() [E] Render failed: template: explore/code:41:40: executing "explore/code" at <$repo.HTMLURL>: error calling HTMLURL: runtime error: invalid memory address or nil pointer dereference
2022/03/14 13:44:43 ...s/context/context.go:198:HTML() [D] Template: status/500

[IT-AlexKor]

As system fix - you should call code search index clean/remove (only for deleted repo) after repo delete, for example like that

As workaround manual remove from ES index will fix this problem.
Find potential bad rows (see example code in logs above), get project id's, validate if project exists (vi API), call delete for each unexisted repo

POST http://<elastic>:9200/<index>/_delete_by_query
{
  "query": {
    "bool": {
      "must": [],
      "filter": [
        {
          "match_all": {}
        },
        {
          "match_phrase": {
            "repo_id": 57
          }
        }
      ],
      "should": [],
      "must_not": []
    }
  }
}

[RaymondKroon]

I submitted a fix. Additionally the Elasticsearch index should be cleaned, however without a fix in the explore/code code, a 500 response could stil happen because of eventual consistency / delays in index purging.

Furthermore a workaround could also be adding a custom template for explore/code.tmpl which checks if $repo exists: https://github.com/go-gitea/gitea/blob/release/v1.17/templates/explore/code.tmpl#L40 (use if, not with)