Get Time
Search | Watch Thread  |  My Post History  |  My Watches  |  User Settings
View: Flat (newest first)  | Threaded  | Tree
Previous Thread  |  Next Thread
Download Imgs | Reply
I am not able to download images from Amazon S3.
I can successfully list resources with command
aws s3 ls s3://spacenet-dataset/ --request-payer requester

But when I issue
aws s3api get-object --bucket spacenet-dataset --key SpaceNet_Roads_Competition/SpaceNet_Roads_Sample.tar.gz --request-payer SpaceNet_
Roads_Sample.tar.gz SpaceNet_Roads_Sample.tar.gz

I get "An error occurred (AccessDenied) when calling the GetObject operation: Access Denied"
Re: Download Imgs (response to post by paolog) | Reply
You have to use "--request-payer requester" also in the get-object call, not your account name.
Re: Download Imgs (response to post by walrus71) | Reply
OK sounds good, but I suggest to correct the RoadDetector Data Download Guide, which has a wrong example at page n.3
Re: Download Imgs (response to post by paolog) | Reply
Indeed, 'requester' was missing from the example, thanks for noting. It's corrected now.
Re: Download Imgs (response to post by walrus71) | Reply
Because the dataset is quite large and not everyone is on fiber, here's a trick

import boto3
s3 = boto3.client('s3')
url = s3.generate_presigned_url(ClientMethod='get_object', Params={'Bucket': bucket, 'Key': key, 'RequestPayer':'requester'}, ExpiresIn=3600*360)

This will print out a signed URL that you can download using your preferred method and supports multiple connections and resuming. Change "key" as needed.