Feature request: Allow specifying FILE_FLAG_BACKUP_SEMANTICS optionally when opening files

Hello there,

We have a feature request:

We’d like to be able to extract files you usually don’t have access to, via the backup privilege, which seems to not be possible with GroupDocs.Search at the moment.

With Win32, it’s possible to call AdjustTokenPrivileges to acquire the backup privilege for the current process.

After that, it’s possible to read files you don’t usually have access to when specifying the flag “FILE_FLAG_BACKUP_SEMANTICS” (Win32).

The file-reading happens in GroupDocs code, so we want to ask whether it might be possible to implement something that allows us to specify that the flag “FILE_FLAG_BACKUP_SEMANTICS” shall be used whenever a file is opened during the extraction.

If you have any questions about this, feel free to ask!

Best regards
jamsharp

@jamsharp
We have opened the following new ticket(s) in our internal issue tracking system and will deliver their fixes according to the terms mentioned in Free Support Policies.

Issue ID(s): SEARCHNET-3477

You can obtain Paid Support Services if you need support on a priority basis, along with the direct access to our Paid Support management team.

Hello,

I saw this in your change logs:

SEARCHNET-3486 | Implement indexing of files with backup privileges | Feature

Is that what we asked for? Is it also usable when extracting manually?

@jamsharp

We are currently investigating your specific issue under ticket SEARCHNET-3477.

The issues you have found earlier (filed as SEARCHNET-3477) have been fixed in this update. This message was posted using Bugs notification tool by atir.tahir

Hello,
I finally got to try the feature and unfortunately encountered the following Exception:

Error during text extraction from \path\to\file.pdf
System.UnauthorizedAccessException: Access to the path ‘\path\to\file.pdf’ is denied.

I implemented the feature using manual text extraction like so:

private readonly Extractor m_Extractor = new();

var extractionOptions = new ExtractionOptions { UseRawTextExtraction = false, UseBackupPrivilege = true };
var extractedData = m_Extractor.Extract(document, extractionOptions);

Am I doing something wrong here or is this maybe a problem with the manual text extraction?
If there is any additional information I can provide, please let me know.

Best regards,
jamsharp

@jamsharp

Could you please share the complete code (how you are loading the file etc) and the problematic/sample source file?

Attached you may find the (simplified) code:

using System;
using GroupDocs.Search;
using GroupDocs.Search.Common;
using GroupDocs.Search.Options;

public class ExampleClass
{
    private readonly GroupDocs.Search.Extractor m_Extractor = new();

    public ExampleClass()
	{
        //...
	}

    public bool TryExtract(string pFilePath)
    {
        var extractionOptions = new GroupDocs.Search.Options.ExtractionOptions { UseRawTextExtraction = false, UseBackupPrivilege = true };
        var document = Document.CreateFromFile(pFilePath);

        GroupDocs.Search.Common.ExtractedData? extractedData;

        try
        {
            extractedData = m_Extractor.Extract(document, extractionOptions);
            // ...
        }
        catch (UnauthorizedAccessException)
        {
            // ...
            return false;
        }
        return true;
    }
}

Regarding problematic files, it does not seem to happen with a specific file, rather the exception occurs with any file where access without “FILE_FLAG_BACKUP_SEMANTICS” would not be possible.

Thanks in advance,
jamsharp

@jamsharp
We have opened the following new ticket(s) in our internal issue tracking system and will deliver their fixes according to the terms mentioned in Free Support Policies.

Issue ID(s): SEARCHNET-3522

You can obtain Paid Support Services if you need support on a priority basis, along with the direct access to our Paid Support management team.

Hello, are there any updates on this by any chance?

Best regards,
jamsharp

@jamsharp

This ticket SEARCHNET-3522 is still under investigation

Hello,

The last days we spent some more time trying to work out the described problem and could not find any solution.

Using Procmon, we could not locate any file operations other than the creation of the index files.

Could you maybe clarify, if this is expected behavior?

Additionally, it would be really helpful to have some kind of logging during the extraction or have the exception mentioned above enriched with additional information, so we could try to find out why it’s not working. The current exception does not mention the backup privilege and there is no logging, so it’s hard for us to analyze what’s not working.

As some further info: we are working on files on a network share.

Thanks in advance

jamsharp

@jamsharp

We’re still investigating this issue on our side and will continue to look into the behavior you’ve described, especially with the network share scenario. We’ll update you as soon as we have more information.

Hello @jamsharp
FILE_FLAG_BACKUP_SEMANTICS will only work with administrator privileges.
Are you running your application with administrator privileges?

Hello @Andrey.Golubkov, @atir.tahir

FILE_FLAG_BACKUP_SEMANTICS will only work with administrator privileges.

We verified that the user had administrator privileges on the file server we want to create an index for, although this does not seem to be strictly necessary. The user only needs to have the privilege SeBackupPrivilege on Windows.

What makes us think that the problem itself is not on our side / related to the user:

  • We have 2 Windows services with the same service user.

  • Service 1 is a Delphi process that uses the backup privilege to read a file that can only be read with it. It works perfectly fine.

  • Service 2 is a DotNet process with GroupDocs.Search with the same user, but we get “System.UnauthorizedAccessException: Access to the path … is denied” for that file.


2 Ideas:

  • We get the exception via the event extractor.ErrorOccurred by the way. I think, we haven’t mentioned that yet. Maybe, this helps reproducing it.

  • Let me give you the code we used to get the backup privilege with, so you can compare it to yours:

This is how we activated the backup privilege in our code (with CsWin32):

    private bool EnableBackupPrivilegeIfPossible()
    {
        // This along is not enough. When reading a file, the FILE_FLAG_BACKUP_SEMANTICS flag needs to be passed, too!!


        // open token for current process
        var currentProcessHandle = new SafeProcessHandle(Process.GetCurrentProcess().Handle, ownsHandle: false);
        bool success = PInvoke.OpenProcessToken(
            currentProcessHandle,
            TOKEN_ACCESS_MASK.TOKEN_ADJUST_PRIVILEGES | TOKEN_ACCESS_MASK.TOKEN_QUERY,
            out var tokenHandle);

        if (!success)
        {
            var errorCode = Marshal.GetLastPInvokeError();
            var errorMessage = Marshal.GetPInvokeErrorMessage(errorCode);
            m_Logger.LogOpenProcessTokenFailed(errorCode, errorMessage);
            return false;
        }

        using (tokenHandle)
        {
            // Lookup für das BackupPrivilege
            success = PInvoke.LookupPrivilegeValue(
                null, // local PC
                "SeBackupPrivilege", // name of privilege
                out LUID privilegeLuid);

            if (!success)
            {
                var errorCode = Marshal.GetLastPInvokeError();
                var errorMessage = Marshal.GetPInvokeErrorMessage(errorCode);
                m_Logger.LogLookupPrivilegeValueFailed(errorCode, errorMessage);
                return false;
            }

            // structure for privileges change
            var tokenPrivileges = new TOKEN_PRIVILEGES
            {
                PrivilegeCount = 1,
                Privileges = default
            };

            tokenPrivileges.Privileges[0].Luid = privilegeLuid;
            tokenPrivileges.Privileges[0].Attributes = TOKEN_PRIVILEGES_ATTRIBUTES.SE_PRIVILEGE_ENABLED;

            // activate privilege
            unsafe
            {
                TOKEN_PRIVILEGES previousState;
                uint returnLength = 0;
                success = PInvoke.AdjustTokenPrivileges(tokenHandle,
                                                    new BOOL(false),
                                                    &tokenPrivileges,
                                                    (uint)Marshal.SizeOf<TOKEN_PRIVILEGES>(),
                                                    &previousState,
                                                    &returnLength);
            }

            if (!success || Marshal.GetLastPInvokeError() != 0)
            {
                var errorCode = Marshal.GetLastPInvokeError();
                var errorMessage = Marshal.GetPInvokeErrorMessage(errorCode);
                m_Logger.LogAdjustTokenPrivilegesFailed(errorCode, errorMessage);
                return false;
            }

            return true;
        }
    }

This is the code we used to actually read the file (first 48 bytes in this case):

 // Win32 constants
    private const uint GENERIC_READ = 0x80000000;
    private const uint FILE_SHARE_READ = 0x00000001;
    private const uint FILE_SHARE_WRITE = 0x00000002;
    private const uint OPEN_EXISTING = 3;
    private const uint FILE_FLAG_BACKUP_SEMANTICS = 0x02000000;

    // Win32 API imports
    [DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)]
    private static extern SafeFileHandle CreateFile(
        string lpFileName,
        uint dwDesiredAccess,
        uint dwShareMode,
        IntPtr lpSecurityAttributes,
        uint dwCreationDisposition,
        uint dwFlagsAndAttributes,
        IntPtr hTemplateFile);
        
        
        
   if (pFilePath.Contains("secret"))
        {
            try
            {
                // Open the file with FILE_FLAG_BACKUP_SEMANTICS
                using (SafeFileHandle fileHandle = CreateFile(
                           pFilePath,
                           GENERIC_READ,
                           FILE_SHARE_READ | FILE_SHARE_WRITE,
                           IntPtr.Zero,
                           OPEN_EXISTING,
                           FILE_FLAG_BACKUP_SEMANTICS,
                           IntPtr.Zero))
                {
                    if (fileHandle.IsInvalid)
                    {
                        int error = Marshal.GetLastWin32Error();
                        throw new IOException($"Failed to open file. Error code: {error}");
                    }

                    // Read the file content using the file handle
                    using (FileStream fileStream = new FileStream(fileHandle, FileAccess.Read))
                    {
                        // Create a buffer to read the file
                        byte[] buffer = new byte[48];
                        StringBuilder fileContent = new StringBuilder();
                        int bytesRead;

                        // Read the file in chunks
                        if ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) > 0)
                        {
                            fileContent.Append(Encoding.UTF8.GetString(buffer, 0, bytesRead));
                        }

                        // Display the file content
                        m_Logger.LogFileContent(fileContent.ToString());
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Error: {ex.Message}");
                m_Logger.LogFileReadingError(ex);
            }
        }

We hope, this helps.

1 Like

@jamsharp

Thanks for the details. Please allow us sometime to further investigate this scenario. You’ll be notified as there’s any update.