Hello,
We found a problem: Based on 1000 files, we found out that GroupDocs finds 4 of them when you search for “alex” in the index, but also when you execute a “NOT alex” search in it. That does not seem to make sense.
Would it be possible by the way to increase the upload limit for this account on this forum?
I had to remove 000559.ppt, because it was too large.
The 3 (4th was to large to upload) files are in the zip file, together with a repro project.
repro.zip (2.7 MB)
By the way: You told us that you are currently indexing the Corpora dataset. This is a good way to verify that it’s solved in general (Create an index with 100k of its files and search for “test” and for “NOT test” and make sure that the number of results from both searches add up to exactly 100k).
Best regards
Jam
Hello!
This is because 4 documents and several more nested items are added.
If you display indexed documents:
var docs = index.GetIndexedDocuments();
int i = 0;
foreach (var doc in docs)
{
Console.WriteLine((i++).ToString() + " - " + doc);
var items = index.GetIndexedDocumentItems(doc);
foreach (var item in items)
{
Console.WriteLine("\t" + (i++).ToString() + " - " + item);
}
}
The result will be:
0 - E:\Issues\SEARCHNET-3616\repro\input_files\000133.ppt
1 - E:\Issues\SEARCHNET-3616\repro\input_files\000133.ppt\Object 4.doc
2 - E:\Issues\SEARCHNET-3616\repro\input_files\000133.ppt\Object 4 (1).doc
3 - E:\Issues\SEARCHNET-3616\repro\input_files\001020.ppt
4 - E:\Issues\SEARCHNET-3616\repro\input_files\001020.ppt\Object 8.bin
5 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps
6 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2.jpg
7 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (1).jpg
8 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (2).jpg
9 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (3).jpg
10 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3.jpg
11 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4.jpg
12 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 5.jpg
13 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (4).jpg
14 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (1).jpg
15 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (2).jpg
16 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (3).jpg
17 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4 (1).jpg
18 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4.bin
19 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 7.bin
20 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (4).jpg
21 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (5).jpg
22 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4 (2).jpg
23 - E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 5 (1).jpg
Hi, we will take a look at that, but what I don’t understand:
Let’s assume, the documents and their nested stuff lands in the index…
index.Search(NOT alex);
index.Search(alex);
Why do both calls find the same result. My view: Either the word “alex” is in a document or its nested children or it is not.
Hello
This code:
Console.WriteLine("Creating index...");
using var index = new Index(indexFolder, true);
index.Events.ErrorOccurred += (s, e) => Console.WriteLine(e.Message);
index.Add(documentsPath);
var fileCount = Directory.GetFiles(documentsPath).Length;
Console.WriteLine($"\n=== 1. Wrong index creation: {fileCount} physical files ===");
var wildcardPattern = new WordPattern();
wildcardPattern.AppendZeroOrMoreCharactersWildcard();
var queryAll =
SearchQuery.CreateFieldQuery(CommonFieldNames.Content,
SearchQuery.CreateWordPatternQuery(wildcardPattern));
var resultsAll = index.Search(queryAll).ToList();
Console.WriteLine($"content == * returned {resultsAll.Count} documents:");
foreach (var pair in resultsAll.Select((d, i) => new { I = i, D = d }))
Console.WriteLine($" {pair.I} {pair.D.DocumentInfo}: {pair.D.OccurrenceCount} hit(s)");
var queryNotAll =
SearchQuery.CreateFieldQuery(CommonFieldNames.Content,
SearchQuery.CreateNotQuery(
SearchQuery.CreateWordPatternQuery(wildcardPattern)));
var resultsNotAll = index.Search(queryNotAll).ToList();
Console.WriteLine($"content != * returned {resultsNotAll.Count} documents");
foreach (var pair in resultsNotAll.Select((d, i) => new { I = i, D = d }))
Console.WriteLine($" {pair.I} {pair.D.DocumentInfo}: {pair.D.OccurrenceCount} hit(s)");
Console.WriteLine("\n=== 2. CreateNotQuery duplicate/hollow hits ===");
var word2 = "interface";
var wordPattern = new WordPattern();
wordPattern.AppendString(word2);
var queryEqual =
SearchQuery.CreateFieldQuery(CommonFieldNames.Content,
SearchQuery.CreateWordPatternQuery(wordPattern));
var resultsEqual = index.Search(queryEqual).ToList();
Console.WriteLine($"content == {word2} returned {resultsEqual.Count} documents");
foreach (var pair in resultsEqual.Select((d, i) => new { I = i, D = d }))
Console.WriteLine($" {pair.I} {pair.D.DocumentInfo}: {pair.D.OccurrenceCount} hit(s)");
var queryNotEqual =
SearchQuery.CreateFieldQuery(CommonFieldNames.Content,
SearchQuery.CreateNotQuery(
SearchQuery.CreateWordPatternQuery(wordPattern)));
var resultsNotEqual = index.Search(queryNotEqual).ToList();
Console.WriteLine($"content != {word2} returned {resultsNotEqual.Count} documents");
foreach (var pair in resultsNotEqual.Select((d, i) => new { I = i, D = d }))
Console.WriteLine($" {pair.I} {pair.D.DocumentInfo}: {pair.D.OccurrenceCount} hit(s)");
Console.WriteLine("\n=== 3. Like/NotLike (substring match) ===");
var word3 = "alex";
var likePattern = new WordPattern();
likePattern.AppendZeroOrMoreCharactersWildcard();
likePattern.AppendString(word3);
likePattern.AppendZeroOrMoreCharactersWildcard();
var queryLike =
SearchQuery.CreateFieldQuery(CommonFieldNames.Content,
SearchQuery.CreateWordPatternQuery(likePattern));
var resultsLike = index.Search(queryLike).ToList();
Console.WriteLine($"content == *{word3}* returned {resultsLike.Count} documents");
foreach (var pair in resultsLike.Select((d, i) => new { I = i, D = d }))
Console.WriteLine($" {pair.I} {pair.D.DocumentInfo}: {pair.D.OccurrenceCount} hit(s)");
var queryNotLike =
SearchQuery.CreateFieldQuery(CommonFieldNames.Content,
SearchQuery.CreateNotQuery(
SearchQuery.CreateWordPatternQuery(likePattern)));
var resultsNotLike = index.Search(queryNotLike).ToList();
Console.WriteLine($"content != *{word3}* returned {resultsNotLike.Count} documents");
foreach (var pair in resultsNotLike.Select((d, i) => new { I = i, D = d }))
Console.WriteLine($" {pair.I} {pair.D.DocumentInfo}: {pair.D.OccurrenceCount} hit(s)");
Gives us the output:
Creating index...
=== 1. Wrong index creation: 3 physical files ===
content == * returned 4 documents:
0 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps: 3645 hit(s)
1 E:\Issues\SEARCHNET-3616\repro\input_files\001020.ppt: 2501 hit(s)
2 E:\Issues\SEARCHNET-3616\repro\input_files\001020.ppt\Object 8.bin: 1201 hit(s)
3 E:\Issues\SEARCHNET-3616\repro\input_files\000133.ppt: 594 hit(s)
content != * returned 20 documents
0 E:\Issues\SEARCHNET-3616\repro\input_files\000133.ppt\Object 4.doc: 0 hit(s)
1 E:\Issues\SEARCHNET-3616\repro\input_files\000133.ppt\Object 4 (1).doc: 0 hit(s)
2 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2.jpg: 0 hit(s)
3 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (1).jpg: 0 hit(s)
4 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (2).jpg: 0 hit(s)
5 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (3).jpg: 0 hit(s)
6 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3.jpg: 0 hit(s)
7 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4.jpg: 0 hit(s)
8 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 5.jpg: 0 hit(s)
9 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (4).jpg: 0 hit(s)
10 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (1).jpg: 0 hit(s)
11 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (2).jpg: 0 hit(s)
12 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (3).jpg: 0 hit(s)
13 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4 (1).jpg: 0 hit(s)
14 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4.bin: 0 hit(s)
15 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 7.bin: 0 hit(s)
16 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (4).jpg: 0 hit(s)
17 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (5).jpg: 0 hit(s)
18 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4 (2).jpg: 0 hit(s)
19 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 5 (1).jpg: 0 hit(s)
=== 2. CreateNotQuery duplicate/hollow hits ===
content == interface returned 1 documents
0 E:\Issues\SEARCHNET-3616\repro\input_files\001020.ppt: 1 hit(s)
content != interface returned 23 documents
0 E:\Issues\SEARCHNET-3616\repro\input_files\000133.ppt: 0 hit(s)
1 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps: 0 hit(s)
2 E:\Issues\SEARCHNET-3616\repro\input_files\000133.ppt\Object 4.doc: 0 hit(s)
3 E:\Issues\SEARCHNET-3616\repro\input_files\000133.ppt\Object 4 (1).doc: 0 hit(s)
4 E:\Issues\SEARCHNET-3616\repro\input_files\001020.ppt\Object 8.bin: 0 hit(s)
5 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2.jpg: 0 hit(s)
6 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (1).jpg: 0 hit(s)
7 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (2).jpg: 0 hit(s)
8 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (3).jpg: 0 hit(s)
9 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3.jpg: 0 hit(s)
10 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4.jpg: 0 hit(s)
11 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 5.jpg: 0 hit(s)
12 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (4).jpg: 0 hit(s)
13 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (1).jpg: 0 hit(s)
14 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (2).jpg: 0 hit(s)
15 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (3).jpg: 0 hit(s)
16 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4 (1).jpg: 0 hit(s)
17 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4.bin: 0 hit(s)
18 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 7.bin: 0 hit(s)
19 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (4).jpg: 0 hit(s)
20 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (5).jpg: 0 hit(s)
21 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4 (2).jpg: 0 hit(s)
22 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 5 (1).jpg: 0 hit(s)
=== 3. Like/NotLike (substring match) ===
content == *alex* returned 3 documents
0 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps: 1 hit(s)
1 E:\Issues\SEARCHNET-3616\repro\input_files\001020.ppt: 1 hit(s)
2 E:\Issues\SEARCHNET-3616\repro\input_files\000133.ppt: 1 hit(s)
content != *alex* returned 21 documents
0 E:\Issues\SEARCHNET-3616\repro\input_files\000133.ppt\Object 4.doc: 0 hit(s)
1 E:\Issues\SEARCHNET-3616\repro\input_files\000133.ppt\Object 4 (1).doc: 0 hit(s)
2 E:\Issues\SEARCHNET-3616\repro\input_files\001020.ppt\Object 8.bin: 0 hit(s)
3 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2.jpg: 0 hit(s)
4 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (1).jpg: 0 hit(s)
5 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (2).jpg: 0 hit(s)
6 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (3).jpg: 0 hit(s)
7 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3.jpg: 0 hit(s)
8 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4.jpg: 0 hit(s)
9 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 5.jpg: 0 hit(s)
10 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 2 (4).jpg: 0 hit(s)
11 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (1).jpg: 0 hit(s)
12 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (2).jpg: 0 hit(s)
13 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (3).jpg: 0 hit(s)
14 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4 (1).jpg: 0 hit(s)
15 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4.bin: 0 hit(s)
16 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 7.bin: 0 hit(s)
17 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (4).jpg: 0 hit(s)
18 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 3 (5).jpg: 0 hit(s)
19 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 4 (2).jpg: 0 hit(s)
20 E:\Issues\SEARCHNET-3616\repro\input_files\001052.pps\Object 5 (1).jpg: 0 hit(s)
And in all 3 cases for mutually exclusive queries, it seems as if the total number of documents is the same.
Ok, thank you, please feel free to close this.
1 Like