dutchcoders/transfer.sh
1
0
Fork 0
mirror of https://github.com/dutchcoders/transfer.sh.git synced 2026-02-06 07:22:17 +00:00
Code Issues 48 Actions Packages Projects Releases Wiki Activity

Support for AWS Signature Version 4 #247

New Issue
Closed
opened 2026-01-19 18:29:35 +00:00 by michael · 11 comments
michael commented 2026-01-19 18:29:35 +00:00
Owner
Copy Link

Originally created by @computeralex92 on GitHub.

It would be great if a new version of transfer.sh support the AWS Signature Version 4 to host the S3 Bucket in a region where only this signature is available.

Currently this error appears when trying to upload a file to such a S3 Bucket:

[transfer.sh]2019/06/21 06:07:31 The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.
Originally created by @computeralex92 on GitHub. It would be great if a new version of transfer.sh support the AWS Signature Version 4 to host the S3 Bucket in a region where only this signature is available. Currently this error appears when trying to upload a file to such a S3 Bucket: ``` [transfer.sh]2019/06/21 06:07:31 The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256. ```
michael commented 2026-01-19 18:29:36 +00:00
Author
Owner
Copy Link

@paolafrancesca commented on GitHub:

please @computeralex92 can you test #243 ?

@paolafrancesca commented on GitHub: please @computeralex92 can you test #243 ?
michael commented 2026-01-19 18:29:36 +00:00
Author
Owner
Copy Link

@computeralex92 commented on GitHub:

The issue is related to the switch from Version 2 of the signature to Version 4 happen step by step from AWS.
Every newer region (e.g. eu-central-1 or ap-east-1) is only accessable with version 4, all other "older" regions of AWS supports both versions.

As far as I can see in the code the problem is that the S3 Lib transfer.sh is using does not support V4 and since the lib is unmaintained, from my perspective only a switch of the lib is solving this issue.

I found a old pull request which was never merged to master, maybe it is helpful in this case:
https://github.com/dutchcoders/transfer.sh/pull/49

@computeralex92 commented on GitHub: The issue is related to the switch from Version 2 of the signature to Version 4 happen step by step from AWS. Every newer region (e.g. eu-central-1 or ap-east-1) is only accessable with version 4, all other "older" regions of AWS supports both versions. As far as I can see in the code the problem is that the S3 Lib transfer.sh is using does not support V4 and since the lib is unmaintained, from my perspective only a switch of the lib is solving this issue. I found a old pull request which was never merged to master, maybe it is helpful in this case: https://github.com/dutchcoders/transfer.sh/pull/49
michael commented 2026-01-19 18:29:36 +00:00
Author
Owner
Copy Link

@paolafrancesca commented on GitHub:

@computeralex92 can you point to some documentation about the topic?
the S3 storage backend was done by @nl5887 and I never worked on it (neither I have the possibility to test against it at the moment).
I can check if I can just override the AWS Signature to use from the AWS SDK, is Version 4 supported in every region?

@paolafrancesca commented on GitHub: @computeralex92 can you point to some documentation about the topic? the S3 storage backend was done by @nl5887 and I never worked on it (neither I have the possibility to test against it at the moment). I can check if I can just override the AWS Signature to use from the AWS SDK, is Version 4 supported in every region?
michael commented 2026-01-19 18:29:36 +00:00
Author
Owner
Copy Link

@paolafrancesca commented on GitHub:

I see @computeralex92. as I said I don't use S3 and I didn't developed that part.
I think the whole code has to be moved to use official AWS SDK, unlikely I don't know when I'll have time for that and I cannot test.
in case I will create a branch for are you available to test it?

@paolafrancesca commented on GitHub: I see @computeralex92. as I said I don't use S3 and I didn't developed that part. I think the whole code has to be moved to use official AWS SDK, unlikely I don't know when I'll have time for that and I cannot test. in case I will create a branch for are you available to test it?
michael commented 2026-01-19 18:29:36 +00:00
Author
Owner
Copy Link

@computeralex92 commented on GitHub:

please @computeralex92 can you test #243 ?

I will try it and give you a result as soon as possible.

@computeralex92 commented on GitHub: > please @computeralex92 can you test #243 ? I will try it and give you a result as soon as possible.
michael commented 2026-01-19 18:29:36 +00:00
Author
Owner
Copy Link

@paolafrancesca commented on GitHub:

I already fixed this bugs locally and it works now with a bucket located in Frankfurt (eu-central-1).

how do you fixed these?
for the region, as long as you pass the endpoint it seems to be used only for signing, and I assumed you could use a signing region different than the one the bucket is. evidently my assumption was wrong :)

for 0 byte file size and the runtime error: I used the wrong method in Head and Get, can you please test now?

@paolafrancesca commented on GitHub: > I already fixed this bugs locally and it works now with a bucket located in Frankfurt (eu-central-1). how do you fixed these? for the region, as long as you pass the endpoint it seems to be used only for signing, and I assumed you could use a signing region different than the one the bucket is. evidently my assumption was wrong :) for 0 byte file size and the runtime error: I used the wrong method in `Head` and `Get`, can you please test now?
michael commented 2026-01-19 18:29:36 +00:00
Author
Owner
Copy Link

@computeralex92 commented on GitHub:

OK, looks good so far, but I need to fix some issues to get it work:

  • A S3 Bucket with SSL Support is not accessable since the CA Certificates are not copied to the docker image
  • The region is hard coded to "eu-west-1", so every other region is failing.

I already fixed this bugs locally and it works now with a bucket located in Frankfurt (eu-central-1).
To things are on my ToDo list for now:

  • Test the new code against Minio and some other S3 compatible providers
  • Find out why the download page of a file shows 0 KB as a file size.
@computeralex92 commented on GitHub: OK, looks good so far, but I need to fix some issues to get it work: - A S3 Bucket with SSL Support is not accessable since the CA Certificates are not copied to the docker image - The region is hard coded to "eu-west-1", so every other region is failing. I already fixed this bugs locally and it works now with a bucket located in Frankfurt (eu-central-1). To things are on my ToDo list for now: - [x] Test the new code against Minio and some other S3 compatible providers - [ ] Find out why the download page of a file shows 0 KB as a file size.
michael commented 2026-01-19 18:29:36 +00:00
Author
Owner
Copy Link

@paolafrancesca commented on GitHub:

#244 merged to the branch, just refactored the order of the param to be a little cleaner.
if you give me the go that all tests are fine I will merge this to master

@paolafrancesca commented on GitHub: #244 merged to the branch, just refactored the order of the param to be a little cleaner. if you give me the go that all tests are fine I will merge this to master
michael commented 2026-01-19 18:29:36 +00:00
Author
Owner
Copy Link

@computeralex92 commented on GitHub:

Good message: It is working as expected with Minio and other providers like Digitalocean and Scaleway.
Bad message:
When testing the download of a file, the following error appears in the browser:

runtime error: invalid memory address or nil pointer dereference

I think it's related to the 0 byte file size bug in the download page.
@aspacca Maybe you can see the issue with that; I will also try to find the problem.

@computeralex92 commented on GitHub: Good message: It is working as expected with Minio and other providers like Digitalocean and Scaleway. Bad message: When testing the download of a file, the following error appears in the browser: ``` runtime error: invalid memory address or nil pointer dereference ``` I think it's related to the 0 byte file size bug in the download page. @aspacca Maybe you can see the issue with that; I will also try to find the problem.
michael commented 2026-01-19 18:29:36 +00:00
Author
Owner
Copy Link

@computeralex92 commented on GitHub:

for 0 byte file size and the runtime error: I used the wrong method in Head and Get, can you please test now?

Now it's working perfectly.

how do you fixed these?

I add the pull request #244 to this branch with the fixed I added.

@computeralex92 commented on GitHub: > for 0 byte file size and the runtime error: I used the wrong method in `Head` and `Get`, can you please test now? Now it's working perfectly. > how do you fixed these? I add the pull request #244 to this branch with the fixed I added.
michael commented 2026-01-19 18:29:36 +00:00
Author
Owner
Copy Link

@computeralex92 commented on GitHub:

From my side everything is fine; upload and download is working as expected

@computeralex92 commented on GitHub: From my side everything is fine; upload and download is working as expected
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
docker
enhancement
hacktoberfest
help wanted
invalid
question
storage provider
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
michael
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dutchcoders/transfer.sh#247
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?