iOS unable to read/open file from the "Files" app when registered as a handler

I have registered my App to open .ply files using "document types" and "imported type identifiers". My app gets the open URL callback and can open the file when shared using airdrop (200mb file).

My app is also listed when the user wants to open the file from the Files app. However, when I get the URL to the file selected in the "Files" app, I get an error saying I can not read the file. Can not copy or get a file handle.

I have the following keys in my plist:

<key>UIFileSharingEnabled</key>
    <true/>
<key>LSSupportsOpeningDocumentsInPlace</key>
    <true/>

How can my app open a file that it has registered as a handler, if the file resides in an arbitrary "Files" directory, and the user tapped on the file and selected my app as a handler?

func application(_ app: UIApplication,
                 open url: URL,
                 options: [UIApplication.OpenURLOptionsKey : Any] = [:]) -> Bool {
    //start loading file }

  // inside file loader:
  func prepare(toView file: URL) {
   do {
        let destination = self.scan.folderURL.appendingPathComponent("temporary_copy.ply")
        if FileManager.default.fileExists(atPath: destination.path) {
            try FileManager.default.removeItem(at: destination)
        }
        
        try FileManager.default.copyItem(at: file, to: destination)
    } catch {
        print("Error copying file: \(error)")
        return
    }
 }