OverflowException: Arithmetic operation resulted in an overflow

As far as I know, it was version 25.2.

Target platform: .Net 9

I cannot recall the exact number of documents in the index, sorry. Either, there were 1k to 50k or there were few, because the exception prevented them getting added to the index.

1 Like

@jamsharp

Thanks for the details. We’ll notify you in case of any update.

This issue is still under investigation. As a temporary solution, try adding the following code immediately after creating the index instance.

Index index = new Index(indexFolder, settings, true);
int[] range = Enumerable.Range(0x3300, 0x33FF)
    .Concat(Enumerable.Range(0xFE30, 0xFE4F))
    .Concat(Enumerable.Range(0xF900, 0xFAFF))
    .Concat(Enumerable.Range(0x2F800, 0x2FA1F))
    .Concat(Enumerable.Range(0x2E80, 0x2EFF))
    .Concat(Enumerable.Range(0x31C0, 0x31EF))
    .Concat(Enumerable.Range(0x3000, 0x303F))
    .Concat(Enumerable.Range(0x4E00, 0x9FFF))
    .Concat(Enumerable.Range(0x3400, 0x4DBF))
    .Concat(Enumerable.Range(0x20000, 0x2A6DF))
    .Concat(Enumerable.Range(0x2A700, 0x2B73F))
    .Concat(Enumerable.Range(0x2B740, 0x2B81F))
    .Concat(Enumerable.Range(0x2B820, 0x2CEAF))
    .Concat(Enumerable.Range(0x2CEB0, 0x2EBEF))
    .Concat(Enumerable.Range(0x30000, 0x3134F))
    .Concat(Enumerable.Range(0x31350, 0x323AF))
    .ToArray();
index.Dictionaries.Alphabet.SetRange(range, CharacterType.Separator);

Hello @atir.tahir, unfortunately, this workaround does not solve (or prevent) this problem. You already gave me this workaround for a different problem → Indexing corrupted docx file leads to OutOfMemoryException - #14 by atir.tahir, so we had it activated when the problem occurred.

@jamsharp

We’ll further look into this issue and notify you about the outcomes.

Hi @jamsharp!

Can you please check if this problem occurs on the latest version 25.9?
If this issue is still present, can you please share as much information as possible for we can reproduce the issue?

Hallo,

Unfortunately, this can still happen in 25.11:

GroupDocs reported an error while adding content to index (we got it via: pIndex.Events.ErrorOccurred):

“System.OverflowException: Arithmetic operation resulted in an overflow.\r\n at \u000F\u0010\u001A.\u0008(Int32 \u0002)\r\n at \u000F\u0010\u001A.\u0002(String \u0002, Int32 \u0008)\r\n at \u0002\u0018\u001B.\u0002()\r\n at \u0008\u0010\u0002.\u0002(\u0002\u0002\u001B \u0002, \u0006\u0002\u001B \u0008)\r\n at \u0003\u0017\u001B.\u000E\u0010\u001B\u0003\u0016\u0002(List1 \u0002, List1 \u0008)\r\n at \u0002\u0017\u001B.\u000E\u0010\u001B\u0003\u0016\u0002(List1 \u0002, List1 \u0008)\r\n at \u0008\u001B\u001B.\u0002(List1 \u0002, List1 \u0008)\r\n at \u0003\u0010\u001B.\u0002\u001B\u001B\u0003\u0016\u0008()\r\n at \u0008\u0010\u0002.\u0002(\u000E\u0015\u0003 \u0002, \u000F\u0016\u0003 \u0008, \u0008\u0017\u0002 \u0005, \u0005\u0019\u0017 \u0006)\r\n at \u0008\u0018\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u000E\u0015\u0003 \u0005, \u000F\u0016\u0003 \u0006, \u0008\u0017\u0002 \u0003, OperationType \u000E)”

Hello @jamsharp!

Let us investigate this issue again and get back to you once we have figured it out.
Please share if you were able to reproduce this issue on the same files provided before, or were there some other files?

Please share if you were able to reproduce this issue on the same files provided before
As far as I know, we were not able to reproduce this again with our files.

or were there some other files?
Yes. this happened with other files on a PC of our customers. Currently, we don’t know which files exactly caused that and are not sure whether we will get them. We hope that the callstack can help finding the source of the problem.

Hi, @jamsharp!

So far, it seems the problem we faced is related to the 2 GB memory limit for the array variable. This leads su to internal framework reworking.
Potentially, there are a few possibilities for avoiding this on your side. The specific solution depends on the target framework and application type you are using.

Can you please try to set up the configuration **gcAllowVeryLargeObjects**
For the .Core project you may try in scproj file

<PropertyGroup>
  <RuntimeHostConfigurationOption Include="System.GC.AllowVeryLargeObjects" Value="true" />
</PropertyGroup>

After build with .net NNN, check bin/Debug/netNNN/YourApp.runtimeconfig.json and you should see:

{
  "runtimeOptions": {
    "configProperties": {
      "System.GC.AllowVeryLargeObjects": true
    }
  }
}

Alternative — runtimeconfig.template.json next to your .csproj:

{
  "configProperties": {
    "System.GC.AllowVeryLargeObjects": true
  }
}

Hope this could help you until we resolve the issue on our side and deploy the new version.
Thank you!

Thanks. I will try out the workaround.

Hello,

We evaluated your quickfix/workaround suggestion, but concluded that that will probably not work.

These are AI thoughs about this topic:

  • gcAllowVeryLargeObjects only affects array length limits, not integer arithmetic boundaries.
  • A System.OverflowException thrown inside arithmetic code cannot be fixed by changing GC settings.
  • The stack trace shows obfuscated method names, but the exception is thrown during an Int32 arithmetic operation, not during memory allocation.
  • If an overflow happens during array indexing (index * elementSize, offset calculations, etc.), enabling very large arrays may change internal code paths and could indirectly avoid the overflow, but only if the library authors accessed arrays incorrectly.

Also:

  • The 2‑GB LOH array limit is real in .NET Framework, but no longer applies in .NET Core / .NET 5+, where arrays can exceed 2 GB even without AllowVeryLargeObjects = true.

We are using .Net 10, so setting the runtime setting should not change anything, unfortunately.


Can you estimate when this will be fixed? This is important for us, because a different customer of us experienced this, too.
When the arithmetic overflow happens during index.Add, future index.Add operations tend to not work because of a System.ObjectDisposedException:

System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u000F\u0010\u001A.\u0002(String \u0002, Int32 \u0008)\r\n   at \u0002\u0018\u001B.\u0002()\r\n   at \u0008\u0010\u0002.\u0002(\u0002\u0002\u001B \u0002, \u0006\u0002\u001B \u0008)\r\n   at \u0003\u0017\u001B.\u0005()\r\n   at \u0008\u001B\u001B.\u0002(\u0003\u0015\u0003 \u0002)\r\n   at \u0003\u0010\u001B.\u0002(Document \u0002, IFieldExtractor \u0008, String \u0005, IWordSplitter \u0006)\r\n   at \u0003\u0010\u001B.\u0002(Document \u0002)\r\n   at \u0003\u0010\u001B.\u0002()

System.ObjectDisposedException: Cannot access a closed file.\r\n   at \u0008\u0015\u0002.\u0002\u0015\u0002\u0003\u0016\u0002(Byte[] \u0002, Int32 \u0008, Int32 \u0005)\r\n   at \u0003\u0017\u001B.\u0002()\r\n   at \u0003\u0017\u001B.\u0005()\r\n   at \u0008\u001B\u001B.\u0002(\u0003\u0015\u0003 \u0002)\r\n   at \u0003\u0010\u001B.\u0002(Document \u0002, IFieldExtractor \u0008, String \u0005, IWordSplitter \u0006)\r\n   at \u0003\u0010\u001B.\u0002(Document \u0002)\r\n   at \u0003\u0010\u001B.\u0002()

Hi @jamsharp!

We reworked internal index storage system to support larger indexed arrays.
Since there are multiple other errors highlighted in other topics with GroupDocs.Search we postponed release last week.
During this week we will try to find out if other possible fixes could be delivered and publish release in the next few days.

1 Like

Hi @jamsharp!

Please be aware that GroupDocs.Search for .NET 26.6 was published.
This release contains the following fixes

  • Implement support for indexes larger than 1 TB
  • Fix indexing PDF files in separate threads
  • Fixed Throw OverflowException when indexing
  • Update library dependencies to the latest version

In the meantime, we will continue to investigate and analyse other issues posted on the forum.

Thank you. We will deliver it to our customers and await whether it works for them.

Hi @jamsharp !

Just want to double check if you are no longer experiencing any issues with the latest version of the GroupDocs.Search 26.6.1.
Thank you!

Hello,

Unfortunately, we can still observe the exception.

During an index operation of around 2 million files, I observed around 400 occurrences.

I removed all the duplicated callstacks. After that, there remained 15 different call stacks with that exception :confused:

"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u000E\u001A\u001B.\u0008()\r\n   at \u000E\u001A\u001B.\u0002(Int32 \u0002, Int32 \u0008)\r\n   at \u000E\u001A\u001B.\u0002(\u0002\u0002\u0017 \u0002)\r\n   at \u000E\u0010\u0002.\u0008(Cancellation \u0002)\r\n   at \u000E\u0010\u0002.\u0002(IndexRepository \u0002, Cancellation \u0008)\r\n   at \u0002\u0003\u001B.\u0002(IndexRepository \u0002, MergeOptions \u0008, \u000F\u001A\u0017 \u0005, OperationType \u0006)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"
"System.OverflowException: Arithmetic operation resulted in an overflow.\r\n   at \u0008\u0010\u001A.\u0006()\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002(Int32 \u0002, Int32 \u0008, \u0008\u0010\u001A \u0005)\r\n   at \u0002\u001B\u0002.\u0002()\r\n   at \u0005\u0005\u001B.\u0002(\u0006\u001A\u0002 \u0002)\r\n   at \u0008\u0002\u001B.\u0002(\u0005\u0005\u001B \u0002, \u0005\u0005\u001B \u0008)\r\n   at \u0003\u0010\u0002.\u0002()\r\n   at \u0003\u0010\u0002.\u0002(\u0008\u0019\u0003 \u0002, \u0008\u0015\u0003 \u0008, \u0003\u0017\u0002 \u0005, \u000F\u0019\u0017 \u0006)\r\n   at \u0002\u0003\u001B.\u0002(Boolean \u0002, Int32 \u0008, \u0008\u0019\u0003 \u0005, \u0008\u0015\u0003 \u0006, \u0003\u0017\u0002 \u0003, OperationType \u000E)"

Hi @jamsharp !

Thank you for the feedback - let me investigate it and get back to you shortly if we will be able to reproduce it. Just to double check - does it happen while adding new documents to the index right?
Can you please share as more details as possible?

Yes. It happens while adding (or maybe while calling the “optimize” - I can’t say that for sure - maybe the callstacks give you that information) stuff to the index.

For performance reasons, we’ve always added the bytes of multiple files at once (usually around 100-250MB each time) to the index:

var options = new IndexingOptions { Cancellation = groupDocsCancellation };
m_Index.Add(extractedDataChunk.Select(c => c.ExtractedData).ToArray(), options);

The index optimize call does not look special:

m_Index.Optimize();

I hope/assume, the callstacks could help finding out where it happens / how to prevent it, even if we don’t find a repro. This happens on the machine of our customer again.

Update:

We tried creating an index based on all supported files (over 700k) of the “Corpora” dataset and were able to reproduce it with that ourselves: Govdocs1 – Digital Corpora

After adding around 57k of them to the index, we first observed it.

We let it run until the end (exception occurred over 800 more times), but according to the GroupDocs API, at the end, there are only 57k documents in the index. In our case, this means: Getting this exception once means: “The index is destroyed forever”.

Can you please try to reproduce it with that? This is currently our highest priority problem.

Hi @jamsharp !

Let me check it today and get back ASAP with some update