-
-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Invalid output in batch_reader.go
#29101
Comments
Stacktrace:
|
Some related codes: gitea/modules/git/repo_base_nogogit.go Lines 81 to 86 in f04e71f
if repo.batchReader.Buffered() is 0, as what mentioned above, reader buffer will randomly be empty after read one line,CatFileBatch will return the existing reader buffer , and in for loop in func SubTree , in the next time, func getTree will read <content> as the first line.
|
Fixes the reason why #29101 is hard to replicate. Related #29297 Create a repo with a file with minimum size 4097 bytes (I use 10000) and execute the following code: ```go gitRepo, err := gitrepo.OpenRepository(db.DefaultContext, <repo>) assert.NoError(t, err) commit, err := gitRepo.GetCommit(<sha>) assert.NoError(t, err) entry, err := commit.GetTreeEntryByPath(<file>) assert.NoError(t, err) b := entry.Blob() // Create a reader r, err := b.DataAsync() assert.NoError(t, err) defer r.Close() // Create a second reader r2, err := b.DataAsync() assert.NoError(t, err) // Should be no error but is ErrNotExist defer r2.Close() ``` The problem is the check in `CatFileBatch`: https://github.com/go-gitea/gitea/blob/79217ea63c1f77de7ca79813ae45950724e63d02/modules/git/repo_base_nogogit.go#L81-L87 `Buffered() > 0` is used to check if there is a "operation" in progress at the moment. This is a problem because we can't control the internal buffer in the `bufio.Reader`. The code above demonstrates a sequence which initiates an operation for which the code thinks there is no active processing. The second call to `DataAsync()` therefore reuses the existing instances instead of creating a new batch reader.
Fixes the reason why go-gitea#29101 is hard to replicate. Related go-gitea#29297 Create a repo with a file with minimum size 4097 bytes (I use 10000) and execute the following code: ```go gitRepo, err := gitrepo.OpenRepository(db.DefaultContext, <repo>) assert.NoError(t, err) commit, err := gitRepo.GetCommit(<sha>) assert.NoError(t, err) entry, err := commit.GetTreeEntryByPath(<file>) assert.NoError(t, err) b := entry.Blob() // Create a reader r, err := b.DataAsync() assert.NoError(t, err) defer r.Close() // Create a second reader r2, err := b.DataAsync() assert.NoError(t, err) // Should be no error but is ErrNotExist defer r2.Close() ``` The problem is the check in `CatFileBatch`: https://github.com/go-gitea/gitea/blob/79217ea63c1f77de7ca79813ae45950724e63d02/modules/git/repo_base_nogogit.go#L81-L87 `Buffered() > 0` is used to check if there is a "operation" in progress at the moment. This is a problem because we can't control the internal buffer in the `bufio.Reader`. The code above demonstrates a sequence which initiates an operation for which the code thinks there is no active processing. The second call to `DataAsync()` therefore reuses the existing instances instead of creating a new batch reader.
Backport #29298 Fixes the reason why #29101 is hard to replicate. Related #29297 Create a repo with a file with minimum size 4097 bytes (I use 10000) and execute the following code: ```go gitRepo, err := gitrepo.OpenRepository(db.DefaultContext, <repo>) assert.NoError(t, err) commit, err := gitRepo.GetCommit(<sha>) assert.NoError(t, err) entry, err := commit.GetTreeEntryByPath(<file>) assert.NoError(t, err) b := entry.Blob() // Create a reader r, err := b.DataAsync() assert.NoError(t, err) defer r.Close() // Create a second reader r2, err := b.DataAsync() assert.NoError(t, err) // Should be no error but is ErrNotExist defer r2.Close() ``` The problem is the check in `CatFileBatch`: https://github.com/go-gitea/gitea/blob/79217ea63c1f77de7ca79813ae45950724e63d02/modules/git/repo_base_nogogit.go#L81-L87 `Buffered() > 0` is used to check if there is a "operation" in progress at the moment. This is a problem because we can't control the internal buffer in the `bufio.Reader`. The code above demonstrates a sequence which initiates an operation for which the code thinks there is no active processing. The second call to `DataAsync()` therefore reuses the existing instances instead of creating a new batch reader.
Fixes #29101 Related #29298 Discard all read data to prevent misinterpreting existing data. Some discard calls were missing in error cases. --------- Co-authored-by: yp05327 <[email protected]>
Fixes go-gitea#29101 Related go-gitea#29298 Discard all read data to prevent misinterpreting existing data. Some discard calls were missing in error cases. --------- Co-authored-by: yp05327 <[email protected]>
Backport #29297 by @KN4CK3R Fixes #29101 Related #29298 Discard all read data to prevent misinterpreting existing data. Some discard calls were missing in error cases. Co-authored-by: KN4CK3R <[email protected]> Co-authored-by: yp05327 <[email protected]>
strconv.ParseInt: parsing "number": invalid syntax
Steps to reproduce:
/.gitea/workflows
with the contentnot a number
/.github/workflows
with the contentnot a number
/<owner>/<repo>/actions
The reason is this code
gitea/modules/actions/workflows.go
Lines 47 to 50 in ce9978b
which fails in line 179
gitea/modules/git/batch_reader.go
Lines 152 to 181 in ce9978b
If the error occurs the code tries to parse
not a number
as output of the batch reader. A normal output looks like8b2bfd3a117fcb7ae93f93616cb2f7a9706c513a tree 104
. I don't know how the content of a file can be present here. And even worse is that it does not fail every time. If I create only oneworkflows
file it does not fail but that may just be a timing issue.Git Version
2.43.0, non-gogit
The text was updated successfully, but these errors were encountered: