insights: prototype in-line links to insights

[Joelkw]

On relevant files, LOC, or repos.

Basic example:

Let's scope, in order of importance:

1. LOC view
2. File level
3. Repo level

[sourcegraph-bot-2]

Heads up @Joelkw @felixfbecker @vovakulikov @unclejustin - the "team/code-insights" label was applied to this issue.

[Joelkw]

Notes from sync:

• partial implementations that would require backend work
• in general, the closest approximation would be to take the search idea we worked on (search any part of an insight/query/series labels)
• No estimate/not a clear solution
• You could imagine ripping search queries out.
  First issue is to explore solutions prototypes.
  Split it out frontend and backend.

[CristinaBirkel]

As a first pass, here's what I'm thinking for the line-of-code use case.

Basically, we just try and match on symbols in the files. I believe this will get us a lot of useful matches without trying to do something totally exhaustive like string matching words or something like that. So we'd create a new endpoint that would take in:

filename
repo
revision
firstLine
lastLine

So the webapp would make one call to this new endpoint per file match, with these input arguments. (A couple of other endpoints work this way on the search page, like GitBlame and HighlightedFile.)

Then the backend would use the Symbols query endpoint internally to get all of the relevant symbols for the file, try and match those symbols against insight titles, series titles, and series queries, and return an array with any matches so that the webapp can display them.

Risks:

• What if the Symbols endpoint doesn't work the way I expect? I think it should, but I won't know for sure until trying it.
• Will it be super slow?
• Does this solution work for the frontend? Am I missing something there?
• General risk of unknown unknowns.

@Joelkw Questions:

• Is this useful? Does this meet expectations for something we want to prototype here?
• This gets at the line-of-code use case, which was first on the list. I'm not sure I understand what is meant by repo or file level, though. Can you give an example of where these would show up in sourcegraph search, and what the expected result would be?

[felixfbecker]

@CristinaBirkel I'm struggling to understand how querying symbols fits into this/provides a benefit. This would basically mean that we would only show this decoration if a symbol definition (like class MySymbol) was to be matched by an insight, but not any non-symbol query (like className="class-not-to-be-used" or deprecatedMethodCall()), correct?

Is there a reason we could not do the following?

• For the current repo and revision, check with a DB query if there is any insight series match recorded with count > 0 (basically an early-abort perf optimization)
• If yes, take those insights series and for each
  • Run the series query on the file and highlight the lines with results, linking to the respective insight

This obviously wouldn't scale if there are a lot of insights that all have matches on the current repo and revision, but could be enough for a feature-flagged prototype that we can show to users to find out if it's useful.

[CristinaBirkel]

@felixfbecker Okay, I think I see what you're saying! Yeah this is almost the opposite approach I was thinking of. I was attempting to narrow down the number of things we have to search our insights for (by narrowing the problem space to symbols,) and I think you're suggesting we narrow down the number of insights we need to search a file for.

The reason I narrowed it down to symbols was really just to make the problem easier for a v1 prototype, and I thought we'd still get a lot of benefit from it. Even though you're right it would miss certain queries!

For the current repo and revision, check if there is any insight series match recorded with count > 0 (basically an early-abort perf optimization)

I don't believe we have this information.. we record series_points that have a timestamp and a repo, but we don't store the revision anywhere. If we're just looking for counts in the repo, I'm not sure how much this would narrow things down. Surely some. But as a worst case scenario we can just assume we're running a query against the file for every insight series in the db. Is that scalable? I'm really not sure.

And then yes, we could run a query against the file for each insight series and then use the results to return matching lines. Is that what you're thinking. It would definitely be accurate and be sure to match all of the results. I like it for those reasons. It also doesn't sound that hard to implement.. maybe on-par with what I was assuming for the symbols approach.

Is this preferable? My only concern is it's potentially a lot of queries, like you said. As an example.. if a customer has 30 insight series, then we're running up to 30 additional search queries per matching file every time they do a search. (Can do it lazily as they scroll down the page, but still.) That seems like potentially a lot of overhead. But sure, I also take your point that we can put it behind a feature flag and just turn it on for certain customers to demo it to them or something.

Did I understand your approach correctly?

[Joelkw]

@CristinaBirkel to answer your direct questions (thanks for asking them!):

Is this useful? Does this meet expectations for something we want to prototype here?

I'm not sure your proposal matches the use cases we expect to be most useful (agree with Felix's points). For example, just looking through both our known use cases and specific examples of how customers actually use insights, we'd want this "line decoration" indicator to show on:

1. Migrations from one thing to another (not always a symbol)
2. Deprecated classes, functions, imports, libraries
3. Inner source code adoption / code reuse

For examples of literally what those insights might be, I recommend checking out our demo dashboards like code reuse or code health or popular use cases and then asking yourself "okay, if I went to edit this insight and went 'preview results' and then was on the file for one of those search results, how do I show that this result is being tracked in this insight'". Does that help clarify here? I think most of these targets are going to be strings or regular expressions and not plain symbols.

This gets at the line-of-code use case, which was first on the list. I'm not sure I understand what is meant by repo or file level, though. Can you give an example of where these would show up in sourcegraph search, and what the expected result would be?

Great question! Yes, the "Css modules migration" is one of those – query is select:file lang:SCSS -file:module patterntype:regexp and I would expect that if you were viewing a file like this, you're told "this is tracked in this migration insight" because the file's existence/not being a module file is the search target. CI adoption is another example, at the repo level. @AlicjaSuska is still working on designs for this here.

[CristinaBirkel]

Thanks @Joelkw ! Taking a look at the demo dashboards I totally see what you mean. I was trying to narrow down the problem, but I didn't realize it would miss so many use cases.

Okay so when the query has select:file or select:repo that's what we're looking for with the second two use cases. That makes sense!

So here's what I'm thinking now:

We do 3 new endpoints, something like:

getRelatedInsightsInline(repo, file, revision)
getRelatedInsightsForFile(repo, file, revision)
getRelatedInsightsForRepo(repo, revision)

I think these can all work similarly, how Felix described.

1. We narrow down the possible insight matches as much as possible. For getRelatedInsightsInline (line-of-code matches) we can only go based on matches in the series_points and series_points_snapshots tables for that particular repo. For the other two, we can also narrow down to queries that have select:file or select:repo in them.
2. We run the queries from any matching insights against the file (or repo) in question. We'll need to add new decoders (I will suggest starting with just the basic search one though,) that gather more information than just the counts. This is so getRelatedInsightsInline can return line numbers with the matches.
3. We figure out how we want to package the results. For repo/file results I think this can just be an array of insight ids, but for the line-of-code results we'll need to figure out the right format for that with the frontend.

Risks:

• I am concerned about rate limiting or just generally making too many requests with this. Might be worth running by the search team.