-
Notifications
You must be signed in to change notification settings - Fork 5
Open
Description
I got this error trying to upload_file under Python 3:
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import s3po
>>> conn = s3po.Connection.s3(aws_access_key_id='************', aws_secret_access_key='*********')
>>> conn.upload_file('*********', 'sbox/test', 'setup.py')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/vagrant/s3po/connection.py", line 72, in upload_file
headers=headers, extra=extra, retries=retries)
File "/vagrant/s3po/connection.py", line 63, in upload
bucket, key, obj_or_data, retries=retries, **opts)
File "/vagrant/s3po/backends/s3.py", line 54, in upload
key.upload_fileobj(source, Config=config, ExtraArgs=extra)
File "/home/ubuntu/.local/lib/python3.5/site-packages/boto3/s3/inject.py", line 513, in object_upload_fileobj
ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
File "/home/ubuntu/.local/lib/python3.5/site-packages/boto3/s3/inject.py", line 431, in upload_fileobj
return future.result()
File "/home/ubuntu/.local/lib/python3.5/site-packages/s3transfer/futures.py", line 73, in result
return self._coordinator.result()
File "/home/ubuntu/.local/lib/python3.5/site-packages/s3transfer/futures.py", line 233, in result
raise self._exception
File "/home/ubuntu/.local/lib/python3.5/site-packages/s3transfer/tasks.py", line 126, in __call__
return self._execute_main(kwargs)
File "/home/ubuntu/.local/lib/python3.5/site-packages/s3transfer/tasks.py", line 150, in _execute_main
return_value = self._main(**kwargs)
File "/home/ubuntu/.local/lib/python3.5/site-packages/s3transfer/upload.py", line 692, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
File "/home/ubuntu/.local/lib/python3.5/site-packages/botocore/client.py", line 324, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/home/ubuntu/.local/lib/python3.5/site-packages/botocore/client.py", line 603, in _make_api_call
request_signer=self._request_signer, context=request_context)
File "/home/ubuntu/.local/lib/python3.5/site-packages/botocore/hooks.py", line 242, in emit_until_response
responses = self._emit(event_name, kwargs, stop_on_response=True)
File "/home/ubuntu/.local/lib/python3.5/site-packages/botocore/hooks.py", line 210, in _emit
response = handler(**kwargs)
File "/home/ubuntu/.local/lib/python3.5/site-packages/botocore/handlers.py", line 210, in conditionally_calculate_md5
calculate_md5(params, **kwargs)
File "/home/ubuntu/.local/lib/python3.5/site-packages/botocore/handlers.py", line 188, in calculate_md5
binary_md5 = _calculate_md5_from_file(body)
File "/home/ubuntu/.local/lib/python3.5/site-packages/botocore/handlers.py", line 202, in _calculate_md5_from_file
md5.update(chunk)
TypeError: Unicode-objects must be encoded before hashing
>>>
The problem is that https://github.com/seomoz/s3po/blob/master/s3po/connection.py#L69 opens the file in unicode mode so that the file object that trickles down to boto is giving out Unicode objects instead of bytes.
The obvious fix is to use rb when we open that file, since we want to upload a byte-exact copy of the file.
Metadata
Metadata
Assignees
Labels
No labels