Max-Days not working #218

Closed
opened 2026-01-19 18:29:28 +00:00 by michael · 5 comments
Owner

Originally created by @Carton32 on GitHub.

Hi,

I'm running my own instance with local storage on AWS EC2. I need several scripts during about two months. So I used the parameters -H "Max-Days: 60" with curl to upload my files.

After 10 days (so less than 14 days, the default value), I got an HTTP 404 error while trying to download my files. It happened a second time after about 10 days after reuploading my files.

I uploaded again my files and checked the metadata and the Maxdate is correctly set (so Max-Days is working) but my files are still being deleted before this.

I searched in the code (not a Go expert at all) in hope to find a parameters I could change to increase the delay but without success.

So I think :

  • Max-Days is not enforced for Max-Days = 50 or 60 (what I tested) and files are deleted before.
Originally created by @Carton32 on GitHub. Hi, I'm running my own instance with local storage on AWS EC2. I need several scripts during about two months. So I used the parameters `-H "Max-Days: 60"` with curl to upload my files. After 10 days (so less than 14 days, the default value), I got an HTTP 404 error while trying to download my files. It happened a second time after about 10 days after reuploading my files. I uploaded again my files and checked the metadata and the Maxdate is correctly set (so Max-Days is working) but my files are still being deleted before this. I searched in the code (not a Go expert at all) in hope to find a parameters I could change to increase the delay but without success. So I think : - Max-Days is not enforced for Max-Days = 50 or 60 (what I tested) and files are deleted before.
Author
Owner

@Carton32 commented on GitHub:

[ec2-user@ip-111-111-111-111 tmp]$ sudo cat JV2kW/logrotate.conf.metadata
{"ContentType":"","Downloads":11,"MaxDownloads":-1,"MaxDate":"2020-05-16T11:51:02.809973937Z","DeletionToken":"11rWBsryVAN"}

This file is in this folder :

drwx------ 2 root root 4096 27. Mär 11:51 JV2kW

So it was uploaded on 27 March but the file is already gone :

[ec2-user@ip-111-111-111-111 tmp]$ sudo ls JV2kW/
logrotate.conf.metadata

@Carton32 commented on GitHub: > [ec2-user@ip-111-111-111-111 tmp]$ sudo cat JV2kW/logrotate.conf.metadata > {"ContentType":"","Downloads":11,"MaxDownloads":-1,"MaxDate":"2020-05-16T11:51:02.809973937Z","DeletionToken":"11rWBsryVAN"} This file is in this folder : > drwx------ 2 root root 4096 27. Mär 11:51 JV2kW So it was uploaded on 27 March but the file is already gone : > [ec2-user@ip-111-111-111-111 tmp]$ sudo ls JV2kW/ > logrotate.conf.metadata
Author
Owner

@paolafrancesca commented on GitHub:

@Carton32 sorry, it was not clear that the file were deleted: actually transfer.sh service doesn't delete any files. if they are deleted from your ec2 instances there must be something else not related to transfer.sh that deletes them
what kind of volume does the ec2 instance has? did you restart the ec2 instance at any time?

i guess the two commands you pasted were run at different time, are they?

@paolafrancesca commented on GitHub: @Carton32 sorry, it was not clear that the file were deleted: actually transfer.sh service doesn't delete any files. if they are deleted from your ec2 instances there must be something else not related to transfer.sh that deletes them what kind of volume does the ec2 instance has? did you restart the ec2 instance at any time? i guess the two commands you pasted were run at different time, are they?
Author
Owner

@Carton32 commented on GitHub:

So I've found what was the problem. You are absolutely right.

I have this cronjob running each day :
#! /bin/sh flags=-umc /usr/sbin/tmpwatch "$flags" -x /tmp/.X11-unix -x /tmp/.XIM-unix \ -x /tmp/.font-unix -x /tmp/.ICE-unix -x /tmp/.Test-unix \ -X '/tmp/hsperfdata_*' 10d /tmp /usr/sbin/tmpwatch "$flags" 30d /var/tmp for d in /var/{cache/man,catman}/{cat?,X11R6/cat?,local/cat?}; do if [ -d "$d" ]; then /usr/sbin/tmpwatch "$flags" -f 30d "$d" fi done

I will modify it accordingly to my need.

Thank you for your help!

@Carton32 commented on GitHub: So I've found what was the problem. You are absolutely right. I have this cronjob running each day : `#! /bin/sh flags=-umc /usr/sbin/tmpwatch "$flags" -x /tmp/.X11-unix -x /tmp/.XIM-unix \ -x /tmp/.font-unix -x /tmp/.ICE-unix -x /tmp/.Test-unix \ -X '/tmp/hsperfdata_*' 10d /tmp /usr/sbin/tmpwatch "$flags" 30d /var/tmp for d in /var/{cache/man,catman}/{cat?,X11R6/cat?,local/cat?}; do if [ -d "$d" ]; then /usr/sbin/tmpwatch "$flags" -f 30d "$d" fi done ` I will modify it accordingly to my need. Thank you for your help!
Author
Owner

@paolafrancesca commented on GitHub:

@Carton32 can you paste full metadata content?
currently MaxDownloads has precedence on MaxDays if set

@paolafrancesca commented on GitHub: @Carton32 can you paste full metadata content? currently `MaxDownloads` has precedence on `MaxDays` if set
Author
Owner

@Carton32 commented on GitHub:

What's strange is that my ec2 instance is fresh and I'm made it just to try AWS and transfer.sh. I did not install anything else than transfer.sh. So I wonder what specifically delete my files after ~10 days if it's not transfer.sh.

I ran all the commands in my last message one after the other and while writing it.

I did not restart my instance :

[ec2-user@ip-111-111-111-111 ~]$ uptime
12:18:15 up 24 days, 2:25, 1 user, load average: 0.00, 0.00, 0.00

what kind of volume does the ec2 instance has?

I don't understand this question.

EDIT: Ok. I've found some information about data being deleted from /tmp. I will check this and come back.

@Carton32 commented on GitHub: What's strange is that my ec2 instance is fresh and I'm made it just to try AWS and transfer.sh. I did not install anything else than transfer.sh. So I wonder what specifically delete my files after ~10 days if it's not transfer.sh. I ran all the commands in my last message one after the other and while writing it. I did not restart my instance : > [ec2-user@ip-111-111-111-111 ~]$ uptime > 12:18:15 up 24 days, 2:25, 1 user, load average: 0.00, 0.00, 0.00 > what kind of volume does the ec2 instance has? I don't understand this question. EDIT: Ok. I've found some information about data being deleted from /tmp. I will check this and come back.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dutchcoders/transfer.sh#218