[pull] main from sourcegraph:main #3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
sourcegraph/zoekt@ebb3ca2...764fe4f - sourcegraph/zoekt@c01b6c7778 remove SRC_EXPERIMENT_ITERATE_NGRAM_LOOKUP_LIMIT - sourcegraph/zoekt@bbd1fedfcd feat(Search): Add support for all Apex language extensions - sourcegraph/zoekt@764fe4f9de index: enable shard merging by default Relates to SPLF-175 Test plan: CI
#64244) This reverts commit sourcegraph/sourcegraph@81585cb. ## Test plan Tested in dotcom mode locally after clearing cookies and no redirect happens now.
ResolveRevision additionally adds a "^0" to the spec to make sure we actually check if it exists. This is important, since most uses of the ChangedFiles API pass in commit shas which do not get resolved by git unless the "^0" is present. I noticed this since hybrid search in searcher started to fail if the commit was missing. In particular Zoekt for empty repositories uses a fake SHA of 404aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa so when a repo started getting cloned hybrid search would fail until it was indexed. Hybrid search explicitly checks for the NotFound error, so hybrid search failing was a a regression from moving away from the SymbolDiff API.
… is not FAILED (#64243) With SRCH-802 we saw error messages when batch changes may still retry and resolve the problem. To give a better distinction between resolvable and non-resolvable errors, I'm changing the colour variation from error to warning if the changeset has not been marked as `FAILED` yet. Before: <img width="913" alt="Screenshot 2024-08-02 at 12 44 27" src="https://github.com/user-attachments/assets/b192c5e9-d23b-460c-82a7-a039edbca3f5"> After: <img width="1355" alt="Screenshot 2024-08-02 at 12 36 23" src="https://github.com/user-attachments/assets/02e231a7-168a-4fe9-bd3b-014d810fd236"> ## Test plan Manual testing ## Changelog - Batch changes that are still retrying now show a warning instead of an error.
Closes https://linear.app/sourcegraph/issue/GRAPH-727/make-range-for-usagerange-non-optional ## Test plan Tests continue to pass
In Go it's possible to have URLs without a valid host portion 🤷 . This has unintended side-effects when filtering hostnames. ## Test plan Tested that URLs that have no hostname are now blocked. <!-- REQUIRED; info at https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> ## Changelog <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
Resurrection of https://github.com/sourcegraph/sourcegraph/pull/64080 -- full details are there
…64204) Automatically generated PR to update package lockfiles for Sourcegraph base images. Built from Buildkite run [#285676](https://buildkite.com/sourcegraph/sourcegraph/builds/285676). ## Test Plan - CI build verifies image functionality Co-authored-by: Buildkite <[email protected]>
The URL path for streaming blame is ambiguous when the file path contains `/stream/`. The pattern looks like `/blame/<repo_name>@<revision>/stream/file/path.txt`. However, if the file contains `/stream/`, the revision is matched greedily up until the last stream, so we end up with a revision that looks like `81af3g/stream/my/path`, which is in invalid revision that results in a 404. This makes the URL pattern unambiguous by adding a `/-/` element after the revision, which is not allowed in a revision name and acts as a path separator. So now, the pattern looks like `/blame/<repo_name>@<revision>/-/stream/file/path.txt`. Note, this is a public-facing breaking change, but I don't think it really matters since I don't expect any customers use our streaming blame API
Followup from #64236
Closes https://linear.app/sourcegraph/issue/SRCH-720/new-chat-button-in-side-panel-view Closes https://linear.app/sourcegraph/issue/SRCH-808/chat-history-in-side-panel-view This PR does a few things - Updates Cody Web to 0.3.6 (this includes improvements around mentions UI, web url mention support, etc) - Adds "create new chat" button to the sidebar cody chat UI - Adds history chats UI to the sidebar cody chat UI (note that in GA we will rely on the new Tabs UI, this history UI is just a temporal solution for the Sourcegraph Aug release, in sep GA release it will be improved) ## Test plan - General manual checks on Cody Web. - Check that you can create a new chat from the sidebar chat UI - Check that you can select chats from the history panel from the sidebar chat UI <!-- REQUIRED; info at https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> ## Changelog <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
Introduces basic (and incomplete) UI support for perforce, including displaying changelist ids, listing changelists, and removing references to commits and branches.
The `getSourceRange` function can have a range translation failure, in which case it would be wrong to propagate the range directly without the commit information. So skip the ranges in that case.
Searcher doesn't speak to the database nor has it for a long time. See https://github.com/sourcegraph/sourcegraph/pull/61463 Test Plan: The following command is empty go run ./dev/depgraph/ summary internal/database | grep 'cmd/searcher'
The cursor state can be "" when the data is exhausted.
This only contains one commit which reduces how often we call scan in indexserver on dotcom. - acacc5eda1 shards: only trigger rescan on .zoekt files changing Test Plan: tested in zoekt CI
… limit hits (#64133) <!-- PR description tips: https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e --> Closes https://linear.app/sourcegraph/issue/CODY-2758/[autocomplete-latency]-add-circuit-breaker-in-cody-gateway-to-handle This PR introduces an in-memory model availability tracker to ensure we do not send consequent requests to the currently unavailable LLM providers. The tracker maintains a history of error records for each model. It evaluates these records to determine whether the model is available for new requests. The evaluation follows these steps: 1. For every request to an upstream provider, the tracker records any errors that occur. Specifically, it logs timeout errors (when a request exceeds its deadline) and responses with a 429 status code (Too Many Requests). 2. These error records are stored in a circular buffer for each model. This buffer holds a fixed number of records, ensuring efficient memory usage. 3. The tracker calculates the failure ratio by analyzing the stored records. It checks the percentage of errors within a specified evaluation window and compares this against the total number of recent requests. 4. Based on the calculated failure ratio, the tracker decides whether the model is available: - Model Unavailable: If the ratio of failures (timeouts or 429 status codes) exceeds a predefined threshold (X%), the model is marked as unavailable. In this state, the system does not send new requests to the upstream provider. When a model is unavailable, the system immediately returns an error status code, typically a 503 Service Unavailable, to the client. This informs the client that the service is temporarily unavailable due to upstream issues. - Model Available: If the failure ratio is within acceptable limits, the system proceeds with sending the request to the upstream provider. This PR suggests considering a model unavailable if **95% of the last 100 requests within the past minute** either time out or return a 429 status code. I am not sure about these exact values and suggest them as a starting point for discussion ## Test plan - Added unit tests - CI <!-- REQUIRED; info at https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> ## Changelog <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
…#64235) The problem was that we were incorrectly not doing a "stack push" operation when encountering a comment, but we were doing a pop which triggered a change in the source range for the next source range (this is done to avoid overlap between the current occurrence and the subsequent occurrence that will be emitted). This is fixed by not ignoring comment markers. As a consequence, we will emit a few more occurrences than earlier, but I don't think it should be a big problem in practice. We can also fuse these if needed.
To make things more explicit and remove the global variable, this is now passed down to where it's needed. It is a bit messy right now, since it's used deep in the serve-handler but that just highlights better where it's actually used IMO. As a next step, I want to get rid of the requirement to indicate server-restart required, so we should be able to drop a bunch of the prop drilling here. Test plan: Still compiles, E2E test works.
This PR simplifies and consolidates the remaining parts of the various registry packages, and moves everything under cmd/frontend/internal for better encapusation. This also removes the need to _ import packages which feels very brittle. Test plan: Go compiler doesn't complain about moved code.
Nothing outside frontend needs to import this package, to properly signify that we move it to internal where 90% of the other packages reside, and avoid ambiguity on "what service is running what code exactly". Test plan: Just moved a package, Go compiler doesn't complain.
In response to [this request](https://sourcegraph.slack.com/archives/C05MW2TMYAV/p1722633022111559), I modified the welcome banner to: - Add a close button in the top right - Close on click outside the banner - Make the button primary rather than secondary
There is no need to have a separate image file for dark mode. The SVG file can handle color switching itself. ## Test plan Switched theme in the user menu and in the browser dev tools.
) This updates the client (React and Svelte) to show the query examples based on the default pattern type instead of the currently selected (navbar) pattern type. Before, toggling the regex button changed the example queries which was very confusing. Another bug was that we used the current pattern type for the links of the query examples. --------- Co-authored-by: Julie Tibshirani <[email protected]>
This is currently only used in the connectivity check, but could be a massive footgun if someone ever used it - the `ps` is NOT passed to `send`, it's assigned to the `Values` property of the struct that is sent there, so `ps` will always be empty.a Test plan: CI passes.
**chore(appliance): name otel-agent things clearly** Disambiguate agent from collector. **chore(appliance): compare-helm can take multiple golden files** Needed for otel, which is split into agent and collector in the appliance but is represented by one component label in Helm. **feat(appliance): deploy otel-collector**
#64488) Part of sourcegraph/devx-support#1164 Also updated: - precommit as we were using a 2019 version - updated buildifier hook as we were 2 major versions behind ## Test plan Modified a Bazel file locally and the Buildifier hook kicked in ## Changelog <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
All the how-tos have been ported to the [Notion How-tos Database](https://www.notion.so/sourcegraph/c884a407e41a443991180bf1ad77e162?v=1337863cc83a4a939994fedf9e14c805&pvs=4). A few of them that were completely outdated have been removed. The how-tos were imported through github.com/soucregraph/notionreposync on Wednesday, so they reflect content at that point in time, which is _after_ the last known commit on that particular tree of files (so we have all the recent updates). ## Test plan n/a this is just docs being moved, and if there was something depending on the how tos, the CI will complain.
This PR adds a seperate check how ChatIntent api should be enabled. For now the chatIntent api is enabled for all the enterprise instances. ToDo: 1. Check if we should only enable the end point on `dotcom` and `S2` instance. ## Test plan tested locally, need to add test cases here ## Changelog <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
See DINF-194 ## Test plan n/a
This is to use the new p4-fusion version that does not fetch labels when `noConvertLabels` is set. <!-- PR description tips: https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e --> ## Test plan Builds pass <!-- REQUIRED; info at https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> ## Changelog <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
This adds the ability to filter cody context by a file pattern. We build on top of this to support [directory mentions](sourcegraph/cody#5210 (comment)) in Cody Chat.
It seems the execution ID must be a UUID - our invalid-ID check-ins were being silently dropped from Sentry without any logs or errors 😬 ## Test plan ``` export TestJobExecutionCheckIn_SENTRY_DSN=... go test -timeout 30s -run ^TestJobExecutionCheckIn$ github.com/sourcegraph/sourcegraph/lib/managedservicesplatform/runtime/contract ```
This PR defines an Enterprise Portal version of the "license check" API that is currently bundled in Souregraph as a separate gRPC service that will be implemented in Enterprise Portal (in a PR I will stack on top of this one). The API schema is pretty much the same, except: - removed the `error` field, since I'm pretty sure we can return a real error instead in a connectRPC implementation for the same effect. - made `license_key` a parameter, as we already propagate the license key all over the place (pings, telemetry) - this is a fairly low-frequency check so there shouldn't be any bandwidth concerns or the like. For now, this parameter will support the existing format so we can forward checks in dotcom to EP, but I've set a target removal release Existing API: https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/cmd/frontend/internal/dotcom/productsubscription/license_check_handler.go?L133 Part of https://linear.app/sourcegraph/issue/CORE-227 ## Test plan CI
PR adds `otherCompletionProviders` as an allowlisted key for `privateMetadata`. <!-- PR description tips: https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e --> ## Test plan Updated test to validate multiple allowlisted keys for one feature <!-- REQUIRED; info at https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> ## Changelog <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
…4400) Implements the RPC defined in https://github.com/sourcegraph/sourcegraph/pull/64396. Follow-up PRs will implement migrations for in-instance checks and for through-dotcom checks. One major change is that we now bypass the check for subscriptions that are denoted as associated with `INTERNAL` instances. Most of the diff is generated mocks. Part of https://linear.app/sourcegraph/issue/CORE-227 ## Test plan - [x] Unit tests - [x] E2E tests (`sg test enterprise-portal-e2e`) 
The existing handler has a similar escape hatch - when enabled, all checks return healthy. ## Test plan unit tests
See https://linear.app/sourcegraph/issue/CORE-256 and https://sourcegraph.slack.com/archives/C05GJPTSZCZ/p1723835604698979?thread_ts=1723834983.348549&cid=C05GJPTSZCZ There is no backend validation for this. ## Test plan n/a
Adds a new `GetCodyContextAlternatives` GraphQL resolver that returns a list of `ContextList`, each of which is a distinct list of results that match the user query, along with a name that describes how that result list was computed. The idea here is to be able to show alternative context fetched with different methods/rankers, which can be displayed by the client to help us iterate on context quality and ordering. The existing context fetching mechanism is preserved and given the `keyword($query)` name. We add a new experimental context list, which modifies the user query to do term expansion using a table of keywords. This keyword table is computed in the `RepositoryReindex` GraphQL endpoint and stored in Redis. We do not worry about clean up, because this feature is experimental. All of this is protected by the feature flag `enhanced-index`. Existing production behavior should not be modified. --------- Co-authored-by: Rishabh Mehrotra <[email protected]>
This is the result of updating the relevant callsites from ```shell git grep '<-.*Done' | grep -v case ``` to use context.AfterFunc instead of directly spinning up a goroutine to wait for Done. Test Plan: close code review and CI
Relates to https://linear.app/sourcegraph/issue/REL-322/appliance-helm-chart-should-be-versioned but does not close it. See test plan in sourcegraph/deploy-sourcegraph-helm#534
<!-- PR description tips: https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e --> An error occurred in the last action run https://github.com/sourcegraph/sourcegraph/actions/runs/10444798914/job/28919786852.  ## Test plan <!-- REQUIRED; info at https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> Next Update. Should be alright ## Changelog <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
I keep typing "sg migrate up" instead of "sg migration up" since that just make more sense to my mind. Test Plan: yolo
This adds a new command to "sg migration" which will enable RLS on the tenant_id column. It relies on the introspection features already present in our migration store to automatically update any table which has the "tenant_id" column. Additionally we provide functionality to disable the enforcement via the "--disable" flag. Test Plan: "go run ./dev/sg migration enforce-tenant-id" with and without "--disable" specified. Then inspected the DB state to confirm the migrations run correctly. Fixes https://linear.app/sourcegraph/issue/SPLF-211/easy-way-to-enable-rls-enforcement-for-dev
Fixes CODY-3194 Previously, using `/chat/completions` with OpenAI models always returned an empty completion because we were reading a non-existent `"text"` property instead of the nested `"message": { "content": ...}` property. This PR fixes the bug and adds a test case to demonstrate how we parse a real-world OpenAI response. <!-- PR description tips: https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e --> ## Test plan See new test cases. I'm failing to get a locally running setup to manually test this e2e, see https://sourcegraph.slack.com/archives/C04MYFW01NV/p1723664513193939 <!-- REQUIRED; info at https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> ## Changelog * Fix bug where requests to `/.api/completions/stream` for OpenAI models returned an empty completion when using `stream: false`. <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
Previously, the `/.api/llm/chat/completions` API returned an error message saying a model was unsupported if it used the wrong syntax. Now, we only validate that the request is using the new modelref syntax and otherwise delegate to the underlying `/.api/completions/stream` endpoint to return the error message. This error message at least mentions the default model, which I've found very helpful when debugging why things are not working as expected. ``` unsupported chat model "openai::2024-02-01::gpt-4o" (default "anthropic::unknown::claude-3-sonnet-20240229" ``` <!-- PR description tips: https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e --> ## Test plan See updated unit test. Also manually tested locally ``` ❯ curl 'https://sourcegraph.test:3443/.api/llm/chat/completions' \ -H 'Content-Type: application/json' \ -H "Authorization: token $HURL_token" \ --data-raw '{ "maxTokensToSample": 4000, "messages": [ { "role": "user", "content": "Respond with \"no\" and nothing else." } ], "model": "openai::2024-02-01::gpt-4o", "temperature": 0, "topK": -1, "topP": -1, "stream": false }' failed to forward request to apiHandler: handler returned unexpected status code: got 400 want 200, response body: unsupported chat model "openai::2024-02-01::gpt-4o" (default "anthropic::unknown::claude-3-sonnet-20240229") ``` <!-- REQUIRED; info at https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> ## Changelog <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
If they are configured, there is no point warning about the defaults being different. In my case I configure these flags via envvars, so I don't need to be warned about it all the time. Test Plan: I ran an sg command with and without envvars configuring and it didn't and did warn respectively.
Following up on code review comment https://github.com/sourcegraph/sourcegraph/pull/64473#discussion_r1721604646 <!-- PR description tips: https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e --> ## Test plan Green CI. <!-- REQUIRED; info at https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> ## Changelog <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
This PR attempts to handle GHAS check to be non-zero exit code in semgrep scan script ## Test plan - CI 🟢 ## Changelog - chore(security): Fix GHAS check as non-zero exit code <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
I noticed src-prof-services.json still existed even though we inline it in sg.config.yaml. I then did a quick scan of the dir and removed other stuff that was unreferenced or referenced things that no longer exist. Test Plan: just affect dev, if I am wrong someone can always revert.
<!-- PR description tips: https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e --> Closes DINF-175 Update changelog files, with no broken links. ## Test plan <!-- REQUIRED; info at https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> ## Changelog <!-- OPTIONAL; info at https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c -->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )