How to copy S3 object with special character in key

Question

I have objects in an S3 bucket, and I do not have control over the names of the keys. Some of these keys have special characters and AWS SDK does not like them.

For example, one object key is: folder/‍Johnson, Scott to JKL-Discovery.pdf, it might look fine at first glance, but if I URL encode it: folder%2F%E2%80%8DJohnson%2C+Scott+to+JKL-Discovery.pdf, you can see that after folder/ (or folder%2F when encoded) there is a random sequence of characters %E2%80%8D before Johnson.

It is unclear where these characters come from, however, I need to be able to handle this use case. When I try to make a copy of this object using the Node.js AWS SDK,

const copyParams = {
    Bucket,
    CopySource,
    Key : `folder/‍Johnson, Scott to JKL-Discovery.pdf`
  };
  let metadata = await s3.copyObject(copyParams).promise();

It fails and can't find the object, if I encodeURI() the key, it also fails.

How can I deal with this?

DO NOT SUGGEST I CHANGE THE ALLOWED CHARACTERS IN THE KEY NAME. I DO NOT HAVE CONTROL OVER THIS

user_2899889 · Answer

Faced the same problem but with PHP. copyObject() method is automatically encoding destination parameters (Bucket and Key) parameters, but not source parameter (CopySource) so it has to be encoded manually. In php it looks like this:

  $s3->copyObject([
    'Bucket' => $targetBucket,
    'Key' => $targetFilePath,
    'CopySource' => $s3::encodeKey("{$sourceBucket}/{$sourceFilePath}"),
  ]);

I'm not familiar with node.js but there should also exist that encodeKey() method that can be used?

user_2915973 · Answer

trying your string, there's a tricky 'zero width space' unicode char... http://www.ltg.ed.ac.uk/~richard/utf-8.cgi?input=342+200+213&mode=obytes

I would sanitize the string removing unicode chars and then proceding with url encoding as requested by official docs.

encodeURI('folder/‍johnson, Scott to JKL-Discovery.pdf'.replace(/[^\x00-\x7F]/g, ""))

How to copy S3 object with special character in key

2 Answers