Text search API, index update is not working in .NET

//path settings
string indexFolder = “c:\IndexStore\”;
string documentsFolder = “c:\DocumentStore\”;

        if (Directory.Exists(indexFolder) == false)
        {
            Directory.CreateDirectory(indexFolder);
        }
        
        //index settings AutoDetectEncoding
        IndexSettings settings = new IndexSettings();
        settings.AutoDetectEncoding = true;

        //index class with async option
        Index index = new Index(indexFolder, settings);
        IndexingOptions options = new IndexingOptions();
        options.IsAsync = true;

        
        //Creating an Indexing
        string[] files = System.IO.Directory.GetFiles(indexFolder, "*.body");
        if (files.Length == 0)
        {
            index.Add(documentsFolder);
        }
        else
        {
            UpdateOptions updateOptions = new UpdateOptions();
            updateOptions.Threads = 1;
            index.Update(updateOptions);

            MergeOptions mergeOptions = new MergeOptions();
            mergeOptions.Cancellation = new Cancellation();
            mergeOptions.Cancellation.CancelAfter(1200000); // Setting maximum duration of the operation to 30 seconds

            index.Optimize(mergeOptions);
        }
       
        //Search Results
        SearchResult searchResult = index.Search(findText);

The above code not updating the index properly.can i get the support.
Dynamically adding files into the documentsFolder .

i reffered console app for reference.

public static void UpdateIndexedDocuments()
{
string indexFolder = @“C:\Indexstore”;
string documentFolder = @“C:\DocumentStorage1”;
string query = “son”;

        // Prepare data
        //Utils.CleanDirectory(documentFolder);
        Utils.CopyFiles(Utils.DocumentsPath, documentFolder);


        // Creating an index in the specified folder
        Index index = new Index(indexFolder);
        

        // Indexing documents from the specified folder
        index.Add(documentFolder);

        SearchResult searchResult = index.Search(query);
        Utils.TraceResult(query, searchResult);


        // Change, delete, add documents in the document folder
        // ...
        // Adding documents to indexed folder
        Utils.CopyFiles(Utils.DocumentsPath4, documentFolder);


        UpdateOptions options = new UpdateOptions();
        //options.Threads = 2; // Setting the number of indexing threads
        index.Update(options); // Updating the index

        SearchResult searchResult2 = index.Search(query);
        Utils.TraceResult(query, searchResult2);

        Utils.CopyFiles(Utils.DocumentsPath4, documentFolder);


        UpdateOptions options2 = new UpdateOptions();
        options2.Threads = 2; // Setting the number of indexing threads
        index.Update(options2); // Updating the index

        SearchResult searchResult3 = index.Search(query);

        Utils.TraceResult(query, searchResult3);
        Utils.CopyFiles(Utils.DocumentsPath4, documentFolder);


        UpdateOptions options3 = new UpdateOptions();
        options3.Threads = 2; // Setting the number of indexing threads
        index.Update(options3); // Updating the index

        SearchResult searchResult4 = index.Search(query);
        Utils.TraceResult(query, searchResult4);
    }

even this is also not working properly.

image.png (279.2 KB)

@bharathiGK

We are not able to reproduce this issue at our end using API version 20.4. Please take a look at this screenshot.JPG (145.2 KB) and code below:

        static void Main(string[] args)
        {
            License lic = new License();
            lic.SetLicense("License path here");
            string indexFolder = @"D:\\IndexStore";
            string documentsFolder = @"D:\\storage1\";

            Index index = new Index(indexFolder);
            index.Add(documentsFolder); 
            string query = "test"; // Specify a search query
            SearchResult result = index.Search(query); // Searching in the index 
            for (int i = 0; i < result.DocumentCount; i++)
            {
                FoundDocument document = result.GetFoundDocument(i);
                Console.WriteLine("\tDocument: " + document.DocumentInfo.FilePath);
                Console.WriteLine("\tOccurrences: " + document.OccurrenceCount);
            }
            Console.WriteLine("\n_________Below are the results after updating the index__________");
            CopyFiles(@"D:\storage2", documentsFolder);
            UpdateOptions options = new UpdateOptions();
            index.Update(options); // Updating the index
            SearchResult result2 = index.Search(query); // Searching in the index

            for (int i = 0; i < result2.DocumentCount; i++)
            {
                FoundDocument document = result2.GetFoundDocument(i);
                Console.WriteLine("\tDocument: " + document.DocumentInfo.FilePath);
                Console.WriteLine("\tOccurrences: " + document.OccurrenceCount);
            }
            Console.WriteLine("\n_________Below are the results after updating the index__________");
            CopyFiles(@"D:\storage3", documentsFolder);
            UpdateOptions options3 = new UpdateOptions();
            index.Update(options3); // Updating the index
            SearchResult result3 = index.Search(query); // Searching in the index

            for (int i = 0; i < result3.DocumentCount; i++)
            {
                FoundDocument document = result3.GetFoundDocument(i);
                Console.WriteLine("\tDocument: " + document.DocumentInfo.FilePath);
                Console.WriteLine("\tOccurrences: " + document.OccurrenceCount);
            }
            Console.WriteLine("\n_________Below are the results after updating the index__________");
            CopyFiles(@"D:\storage4", documentsFolder);
            UpdateOptions options4 = new UpdateOptions();
            index.Update(options4); // Updating the index
            SearchResult result4 = index.Search(query); // Searching in the index

            for (int i = 0; i < result4.DocumentCount; i++)
            {
                FoundDocument document = result4.GetFoundDocument(i);
                Console.WriteLine("\tDocument: " + document.DocumentInfo.FilePath);
                Console.WriteLine("\tOccurrences: " + document.OccurrenceCount);
            }
            Console.WriteLine("\n___________End____________");
        }
        public static void CopyFiles(string sourcePath, string destinationPath)
        {
            DirectoryInfo source = new DirectoryInfo(sourcePath);
            DirectoryInfo target = new DirectoryInfo(destinationPath);
            target.Create();

            foreach (DirectoryInfo dir in source.GetDirectories())
            {
                CopyFiles(dir.FullName, target.CreateSubdirectory(dir.FullName).FullName);
            }

            foreach (FileInfo file in source.GetFiles())
            {
                file.CopyTo(Path.Combine(target.FullName, file.Name), true);
            }
        }

Can you please tell which API version you are using and can you please share a simple console application with us using that issue could be reproduced.

HI i have used 20.1 version
image.png (319.8 KB)

i checked with the console application of groupdcos.search
public static void UpdateIndexedDocuments()
{
string indexFolder = @“C:\Indexstore”;
string documentFolder = @“C:\DocumentStorage1”;
string query = “son”;

        // Prepare data
        Utils.CleanDirectory(documentFolder);
        Utils.CopyFiles(Utils.DocumentsPath, documentFolder);


        // Creating an index in the specified folder
        Index index = new Index(indexFolder);
        

        // Indexing documents from the specified folder
        index.Add(documentFolder);

        SearchResult searchResult = index.Search(query);
        Utils.TraceResult(query, searchResult);


        // Change, delete, add documents in the document folder
        // ...
        // Adding documents to indexed folder
        Utils.CopyFiles(Utils.DocumentsPath4, documentFolder);


        UpdateOptions options = new UpdateOptions();
        //options.Threads = 2; // Setting the number of indexing threads
        index.Update(options); // Updating the index

        SearchResult searchResult2 = index.Search(query);
        Utils.TraceResult(query, searchResult2);

        Utils.CopyFiles(Utils.DocumentsPath4, documentFolder);


        UpdateOptions options2 = new UpdateOptions();
        options2.Threads = 2; // Setting the number of indexing threads
        index.Update(options2); // Updating the index

        SearchResult searchResult3 = index.Search(query);

        Utils.TraceResult(query, searchResult3);
        Utils.CopyFiles(Utils.DocumentsPath4, documentFolder);


        UpdateOptions options3 = new UpdateOptions();
        options3.Threads = 2; // Setting the number of indexing threads
        index.Update(options3); // Updating the index

        SearchResult searchResult4 = index.Search(query);
        Utils.TraceResult(query, searchResult4);
    }

this is the code i followed.if possible can u check this in evaluation version and see whether can able to reproduce or not

1 Like

@bharathiGK

Please run this application SearchTester.zip (1.0 MB) at your end. We tested it with both 20.4 and 20.1 versions of the API. We get right/expected results in both cases (with and without license).
All you have to do is to put these Storage.zip (34.7 KB) files in some directory and then set their path in the application. We are getting this output.JPG (162.5 KB).

Okay.Thank you.i will check and update

Hi for me its not updating properly.

Actually my scenario is,
i will have common storage folder like c:\DocumentStorage .For example, this folder has file1,file2 and file3,file4.

file1 and file2 will be belongs to Project1 and file3 and 4 belongs to project2.

i will do filter on documents and create a index like,
c:\indexstore\project1
so this index will has file1 and file2 only.

I have a common function to create and update index.

First time the following code will make an index @“c:\indexstore\project1” and do search
function search()
{
string indexFolder = @“c:\indexstore\project1”;
string documentFolder = @“C:\DocumentStorage ”;
string query = “son”;
Index index = new Index(indexFolder);
index.Add(documentFolder);
SearchResult searchResult = index.Search(query);
}
this result is correct and coming properly.

Again i’m adding a new file(file5) into string documentFolder = @“C:\DocumentStorage
consider this file5 is also belongs to project1

Second time I’m calling the same function .so this time i should not create an index.
here i need to check if index already there need to update it.
in this case how can i make update is possible?

function search()
{
string indexFolder = @“c:\indexstore\project1”;
string documentFolder = @“C:\DocumentStorage ”;
string query = “son”;
Index index = new Index(indexFolder);
index.Add(documentFolder);
SearchResult searchResult = index.Search(query);
}

i hope you understand my expectation.i will share my complete code for your reference.

//path settings
string indexFolder = “c:\IndexStore\”;
string documentsFolder = “c:\DocumentStore\”;

        if (Directory.Exists(indexFolder) == false)
        {
            Directory.CreateDirectory(indexFolder);
        }
        
        //index settings AutoDetectEncoding
        IndexSettings settings = new IndexSettings();
        settings.AutoDetectEncoding = true;

        //index class with async option
        Index index = new Index(indexFolder, settings);
        IndexingOptions options = new IndexingOptions();
        options.IsAsync = true;

        
        //Creating an Indexing
        string[] files = System.IO.Directory.GetFiles(indexFolder, "*.body");
        if (files.Length == 0)
        {
            index.Add(documentsFolder);
        }
        else
        {
            UpdateOptions updateOptions = new UpdateOptions();
            updateOptions.Threads = 1;
            index.Update(updateOptions);

            MergeOptions mergeOptions = new MergeOptions();
            mergeOptions.Cancellation = new Cancellation();
            mergeOptions.Cancellation.CancelAfter(1200000); // Setting maximum duration of the operation to 30 seconds

            index.Optimize(mergeOptions);
        }
       
        //Search Results
        SearchResult searchResult = index.Search(findText);

kindly check and correct me if anything wrong.

deleting directory and re- creating directory for storing index will work and provides result with all newly added documents.
but this will not make sense when i deal with large volume of documents.

The above query is one possibility i'm looking for.
and the second possibility i'm looking for is,
when i upload document into documentstorage path,that time need to check whether index is available or not.
if index available need to update else create new index .

when i do search by typing keyword in another end,need to refer the indexed path and search keyword from that.

These 2 cases not a sequential work as like console app..Index creation and update needs to happen in one end and search will happen in another end
1 Like

@bharathiGK

We executed your code at our end. Please have a look at this screencast.

Firstly, we added 4 files in the storage and executed application, what happened:

  • Index was created
  • Search results were generated

Then we added another (5th) file in the storage and executed the application, following steps were invoked:

  • Index was only updated
  • Search results were generated

So, when we ran the application second time, index was only updated it wasn’t created.
If you are still facing the issue, we’d recommend you to share your sample application along-with the concerned folders/files and share a screencast (to show program/application execution) as well.

We shared a work-around with you here. However, we’d recommend you to always create a new thread for a new issue/scenario.

Just a general note,

Just call Directory.CreateDirectory(indexFolder);

If the directory exists then it will do nothing, else it will create the directory. There is no need to check if it exists.

1 Like