IO.copy_stream is not copying file which is read from S3

Viewed 18

I have a model Report to export all the Pdf attachments done for the bills. I have other models Bill, Upload.

Bill has many uploads

Tech Stack used

Ruby 2.5.8 Rails 5.0.7

Dummy Code

  require 'fileutils'
  require 'zip'
  require 'open-uri'

  class Report < ApplicationRecord
    main_dir = "tmp/Attachment-#{self.id}"

    FileUtils::mkdir_p main_dir

    bills.each do |bill|
      bill_directory_name = 'Bill_#{bill.id}'

      bill.uploads.each do |bill_upload|
        begin
          FileUtils::cd main_dir do
            FileUtils::mkdir_p "#{bill_directory_name}"

            file = open(bill_upload.file.url, 'rb')

            upload_name = "Invoice-#{bill_upload.file_file_name}-#{bill_upload.id}"
            FileUtils::cd "#{bill_directory_name}" do
              IO.copy_stream(file, "#{upload_name}")
            end
          end
        rescue => exp
          Airbrake.notify(exp)
        end
      end
    end

    path = "tmp/Attachment-#{self.id}"
    archive = File.join(path, "Attachment-#{self.id}" + '.zip')
    FileUtils.rm archive, force: true

    Zip::File::open(archive, 'w') do |zipfile|
      Dir["#{path}/**/**"].reject{|f| f==archive }.each do |file|
        zipfile.add(file.sub(path + '/', ''), file)
      end
    end
  end

Zip folder is created successfully. When I extract zip folder, I can see there are many sub folders created for bills. Each bill folder have uploaded PDF files as well and it's working.

Problem

Most of the time exported bill sub-directory contains PDF uploaded file. But, rarely sometime (once in a 100's of iteration), when I export report, I cannot see PDF file inside some of the bill sub directory. I have added error handling but there is no error generated. So, it's hard to reproduce this issue at my end.

How to confirm that this file copy command is successful IO.copy_stream()?

Please check dummy code and provide your feedback if I need to improve or modify any line of code?

file = open(bill_upload.file.url, 'rb') # File is read from S3

upload_name = "Invoice-#{bill_upload.file_file_name}-#{bill_upload.id}"

FileUtils::cd "#{bill_directory_name}" do
  IO.copy_stream(file, "#{upload_name}")
end

Any help would be appreciated.

0 Answers
Related