Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix USFM parsing issues #228

Open
Enkidu93 opened this issue Jul 15, 2024 · 0 comments
Open

Fix USFM parsing issues #228

Enkidu93 opened this issue Jul 15, 2024 · 0 comments
Assignees
@Enkidu93

Comments

@Enkidu93
Copy link
Collaborator

After running the manual tests on all SF projects, here are the 5 that failed, @ddaspit, @johnml1135:


Multiple failures or warnings in test:
  1)   Failed to parse project aSno:
  Expected: No Exception to be thrown
  But was:  <System.InvalidOperationException: Stack empty.
   at System.Collections.Generic.Stack`1.ThrowForEmptyStack()
   at System.Collections.Generic.Stack`1.Pop()
   at SIL.Machine.Corpora.UsfmTextUpdater.PopNewTokens() in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmTextUpdater.cs:line 396
   at SIL.Machine.Corpora.UsfmTextUpdater.EndVerseText(UsfmParserState state, IReadOnlyList`1 scriptureRefs) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmTextUpdater.cs:line 253
   at SIL.Machine.Corpora.ScriptureRefUsfmParserHandlerBase.Verse(UsfmParserState state, String number, String marker, String altNumber, String pubNumber) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/ScriptureRefUsfmParserHandlerBase.cs:line 58
   at SIL.Machine.Corpora.UsfmTextUpdater.Verse(UsfmParserState state, String number, String marker, String altNumber, String pubNumber) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmTextUpdater.cs:line 148
   at SIL.Machine.Corpora.UsfmParser.ProcessToken() in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 396
   at SIL.Machine.Corpora.UsfmParser.ProcessTokens() in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 132
   at SIL.Machine.Corpora.UsfmParser.Parse(String usfm, IUsfmParserHandler handler, UsfmStylesheet stylesheet, ScrVers versification, Boolean preserveWhitespace) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 44
   at SIL.Machine.Corpora.UsfmManualTests.<CreateUsfmFile>g__GetUsfmAsync|4_0(String projectPath) in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 135
   at SIL.Machine.Corpora.UsfmManualTests.<>c__DisplayClass4_1.<<CreateUsfmFile>b__7>d.MoveNext() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 145
--- End of stack trace from previous location ---
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
   at NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter awaiter)
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
   at NUnit.Framework.Internal.ExceptionHelper.RecordException(Delegate parameterlessDelegate, String parameterName)>

  2)   Failed to parse project RVEES95:
  Expected: No Exception to be thrown
  But was:  <System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')
   at System.Collections.Generic.List`1.get_Item(Int32 index)
   at SIL.Machine.Corpora.UsfmTextUpdater.ReplaceWithNewTokens(UsfmParserState state) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmTextUpdater.cs:line 368
   at SIL.Machine.Corpora.UsfmTextUpdater.OptBreak(UsfmParserState state) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmTextUpdater.cs:line 226
   at SIL.Machine.Corpora.UsfmParser.ProcessToken() in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 559
   at SIL.Machine.Corpora.UsfmParser.ProcessTokens() in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 132
   at SIL.Machine.Corpora.UsfmParser.Parse(String usfm, IUsfmParserHandler handler, UsfmStylesheet stylesheet, ScrVers versification, Boolean preserveWhitespace) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 44
   at SIL.Machine.Corpora.UsfmManualTests.<CreateUsfmFile>g__GetUsfmAsync|4_0(String projectPath) in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 135
   at SIL.Machine.Corpora.UsfmManualTests.<>c__DisplayClass4_1.<<CreateUsfmFile>b__7>d.MoveNext() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 145
--- End of stack trace from previous location ---
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
   at NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter awaiter)
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
   at NUnit.Framework.Internal.ExceptionHelper.RecordException(Delegate parameterlessDelegate, String parameterName)>

  3)   Failed to parse project JraLM:
  Expected: No Exception to be thrown
  But was:  <System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')
   at System.Collections.Generic.List`1.get_Item(Int32 index)
   at SIL.Machine.Corpora.UsfmTextUpdater.ReplaceWithNewTokens(UsfmParserState state) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmTextUpdater.cs:line 368
   at SIL.Machine.Corpora.UsfmTextUpdater.OptBreak(UsfmParserState state) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmTextUpdater.cs:line 226
   at SIL.Machine.Corpora.UsfmParser.ProcessToken() in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 559
   at SIL.Machine.Corpora.UsfmParser.ProcessTokens() in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 132
   at SIL.Machine.Corpora.UsfmParser.Parse(String usfm, IUsfmParserHandler handler, UsfmStylesheet stylesheet, ScrVers versification, Boolean preserveWhitespace) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 44
   at SIL.Machine.Corpora.UsfmManualTests.<CreateUsfmFile>g__GetUsfmAsync|4_0(String projectPath) in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 135
   at SIL.Machine.Corpora.UsfmManualTests.<>c__DisplayClass4_1.<<CreateUsfmFile>b__7>d.MoveNext() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 145
--- End of stack trace from previous location ---
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
   at NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter awaiter)
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
   at NUnit.Framework.Internal.ExceptionHelper.RecordException(Delegate parameterlessDelegate, String parameterName)>

  4)   Failed to parse project ABP:
  Expected: No Exception to be thrown
  But was:  <System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')
   at System.Collections.Generic.List`1.get_Item(Int32 index)
   at SIL.Machine.Corpora.UsfmTextUpdater.ReplaceWithNewTokens(UsfmParserState state) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmTextUpdater.cs:line 368
   at SIL.Machine.Corpora.UsfmTextUpdater.OptBreak(UsfmParserState state) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmTextUpdater.cs:line 226
   at SIL.Machine.Corpora.UsfmParser.ProcessToken() in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 559
   at SIL.Machine.Corpora.UsfmParser.ProcessTokens() in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 132
   at SIL.Machine.Corpora.UsfmParser.Parse(String usfm, IUsfmParserHandler handler, UsfmStylesheet stylesheet, ScrVers versification, Boolean preserveWhitespace) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 44
   at SIL.Machine.Corpora.UsfmManualTests.<CreateUsfmFile>g__GetUsfmAsync|4_0(String projectPath) in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 135
   at SIL.Machine.Corpora.UsfmManualTests.<>c__DisplayClass4_1.<<CreateUsfmFile>b__7>d.MoveNext() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 145
--- End of stack trace from previous location ---
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
   at NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter awaiter)
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
   at NUnit.Framework.Internal.ExceptionHelper.RecordException(Delegate parameterlessDelegate, String parameterName)>

  5)   Failed to parse project SekoP:
  Expected: No Exception to be thrown
  But was:  <System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')
   at System.Collections.Generic.List`1.get_Item(Int32 index)
   at SIL.Machine.Corpora.UsfmTextUpdater.ReplaceWithNewTokens(UsfmParserState state) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmTextUpdater.cs:line 368
   at SIL.Machine.Corpora.UsfmTextUpdater.OptBreak(UsfmParserState state) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmTextUpdater.cs:line 226
   at SIL.Machine.Corpora.UsfmParser.ProcessToken() in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 559
   at SIL.Machine.Corpora.UsfmParser.ProcessTokens() in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 132
   at SIL.Machine.Corpora.UsfmParser.Parse(String usfm, IUsfmParserHandler handler, UsfmStylesheet stylesheet, ScrVers versification, Boolean preserveWhitespace) in /home/enkidu/SILdev/machine/src/SIL.Machine/Corpora/UsfmParser.cs:line 44
   at SIL.Machine.Corpora.UsfmManualTests.<CreateUsfmFile>g__GetUsfmAsync|4_0(String projectPath) in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 135
   at SIL.Machine.Corpora.UsfmManualTests.<>c__DisplayClass4_1.<<CreateUsfmFile>b__7>d.MoveNext() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 145
--- End of stack trace from previous location ---
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
   at NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter awaiter)
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
   at NUnit.Framework.Internal.ExceptionHelper.RecordException(Delegate parameterlessDelegate, String parameterName)>


   at SIL.Machine.Corpora.UsfmManualTests.CreateUsfmFile() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 142
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.GetResult()
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext context)
   at NUnit.Framework.Internal.Execution.SimpleWorkItem.<>c__DisplayClass4_0.<PerformWork>b__0()
   at NUnit.Framework.Internal.ContextUtils.<>c__DisplayClass1_0`1.<DoIsolated>b__0(Object _)

1)    at SIL.Machine.Corpora.UsfmManualTests.<>c.<CreateUsfmFile>b__4_1() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 145
   at NUnit.Framework.Assert.Multiple(TestDelegate testDelegate)
   at SIL.Machine.Corpora.UsfmManualTests.CreateUsfmFile() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 142
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at SIL.Machine.Corpora.UsfmManualTests.CreateUsfmFile()

2)    at SIL.Machine.Corpora.UsfmManualTests.<>c.<CreateUsfmFile>b__4_1() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 145
   at NUnit.Framework.Assert.Multiple(TestDelegate testDelegate)
   at SIL.Machine.Corpora.UsfmManualTests.CreateUsfmFile() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 142
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at SIL.Machine.Corpora.UsfmManualTests.CreateUsfmFile()

3)    at SIL.Machine.Corpora.UsfmManualTests.<>c.<CreateUsfmFile>b__4_1() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 145
   at NUnit.Framework.Assert.Multiple(TestDelegate testDelegate)
   at SIL.Machine.Corpora.UsfmManualTests.CreateUsfmFile() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 142
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at SIL.Machine.Corpora.UsfmManualTests.CreateUsfmFile()

4)    at SIL.Machine.Corpora.UsfmManualTests.<>c.<CreateUsfmFile>b__4_1() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 145
   at NUnit.Framework.Assert.Multiple(TestDelegate testDelegate)
   at SIL.Machine.Corpora.UsfmManualTests.CreateUsfmFile() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 142
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at SIL.Machine.Corpora.UsfmManualTests.CreateUsfmFile()

5)    at SIL.Machine.Corpora.UsfmManualTests.<>c.<CreateUsfmFile>b__4_1() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 145
   at NUnit.Framework.Assert.Multiple(TestDelegate testDelegate)
   at SIL.Machine.Corpora.UsfmManualTests.CreateUsfmFile() in /home/enkidu/SILdev/machine/tests/SIL.Machine.Tests/Corpora/UsfmManualTests.cs:line 142
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at SIL.Machine.Corpora.UsfmManualTests.CreateUsfmFile()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Enkidu93 Enkidu93

Labels
None yet
Projects
Serval
Status: 🆕 New
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Enkidu93