I have tried wget --content-disposition --header="API-Key: myapi" -i download.txt . Also tried it manually. I was able to download the files until yesterday without any problems. Could you please help?
Thank you!
Dylan Nelson
5 Dec '23
What file(s) are you trying to download, i.e. what is in download.txt?
I am having a similar issue. The specific error comes from downloading TNG50-1 snapshot-13 data using the provided wget path. It reads as follows:
2024-02-18 08:53:38 URL:https://data-eu.tng-project.org/file/snapshot/L35n2160TNG/13/57/?token=277aeac3261d5bd79c16 [4430584244/4430584244] -> "snap_013.57.hdf5" [1]
GnuTLS: Error in the pull function.
Unable to establish SSL connection.
https://data-eu.tng-project.org/file/snapshot/L35n2160TNG/13/330/?token=989df3f47a133c0dcdb0:
2024-02-18 08:59:02 ERROR 403: Forbidden.
2024-02-18 09:02:56 URL:https://data-eu.tng-project.org/file/snapshot/L35n2160TNG/13/331/?token=fc24eeebb6f875a094da [4452653448/4452653448] -> "snap_013.331.hdf5" [1]
GnuTLS: The TLS connection was non-properly terminated.
Unable to establish SSL connection.
FINISHED --2024-02-18 09:08:45--
I get the same error at other snapshots such as 50 and 17. Sometimes, when some data is downloaded, after I check on the quality of the download with the also provided file, I get:
sha256sum: WARNING: 47 listed files could not be read
sha256sum: WARNING: 26 computed checksums did NOT match
Which makes me try to download files individually but that has been some work trying to do so for that many files.
Note, I am using screen to run some of these.
Dylan Nelson
20 Feb
If you repeat this command now, are you still having trouble?
For TNG50-1 snapshot-13, I get:
sha256sum: WARNING: 43 computed checksums did NOT match
Do I have to delete all 43 snapshots to run the code again?
I am still getting some errors when I deleted 10 of the erroneous files and ran the command to download files:
In addition, for TNG50-1 snapshot 50, I also get this error when I try to download some individual files:
URL transformed to HTTPS due to an HSTS policy
--2024-02-21 16:41:25-- https://www.tng-project.org/api/TNG50-1/files/snapshot-50.105.hdf5
Connecting to www.tng-project.org (www.tng-project.org)|130.183.17.94|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://data-eu.tng-project.org/file/snapshot/L35n2160TNG/50/105/?token=3799c33d7a232b378411 [following]
--2024-02-21 16:41:26-- https://data-eu.tng-project.org/file/snapshot/L35n2160TNG/50/105/?token=3799c33d7a232b378411
Connecting to data-eu.tng-project.org (data-eu.tng-project.org)|130.183.17.94|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2024-02-21 16:42:05 ERROR 403: Forbidden.
URL transformed to HTTPS due to an HSTS policy
--2024-02-21 16:42:05-- https://www.tng-project.org/api/TNG50-1/files/snapshot-50.138.hdf5
Connecting to www.tng-project.org (www.tng-project.org)|130.183.17.94|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://data-eu.tng-project.org/file/snapshot/L35n2160TNG/50/138/?token=1660891bd9c967f2f054 [following]
--2024-02-21 16:42:06-- https://data-eu.tng-project.org/file/snapshot/L35n2160TNG/50/138/?token=1660891bd9c967f2f054
Connecting to data-eu.tng-project.org (data-eu.tng-project.org)|130.183.17.94|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2024-02-21 16:43:00 ERROR 403: Forbidden.
Ashley Francis
22 Feb
Hi,
I am having the same issue when trying to download the offset files for TNG50-1-Dark snap 99. I get the error ERROR 403: Forbidden each time I try to download it. Perhaps there is something going on with the site such as updates?
Dylan Nelson
22 Feb
All working now?
Ashley Francis
22 Feb
It is working for me now! Thank you.
XIHAN DENG
22 Feb
Hi,
I also met similar issues. I want to download the full snap 33 of TNG100-1, and I kept enconutering error 403 and error 503. And when I tried to download the offset (to check my internet), I got error 503. I wonder if someone is doing maintenance or updates to the server?
Dylan Nelson
24 Feb
This appears to just be that the server is busy, because some other people are using it too much.
I will keep an eye on it. Please try again, and let me know if you have further problems,
Antonio Porras
27 Feb
Some of the downloads seem to be running ok but I still get the following message for TNG300-1 snapshot-99 download:
sha256sum: WARNING: 207 listed files could not be read
sha256sum: WARNING: 33 computed checksums did NOT match
Not sure what to make out of the 207 listed files. In this case, do I just run the command to download the data again?
Also, for reference to everyone: for the 33 computer checksums that did NOT match, I delete those 33 files manually (the ones where the check says FAILED) and then I redownload them. You can do so individually or using the command to download all the data. Since the -nc is in the command, it will only download the files that are missing.
Antonio Porras
1 Mar
checksums for TNG100-1 LHalomerger trees is not working! it says files are not available when they are in the current directory
I have tried wget --content-disposition --header="API-Key: myapi" -i download.txt . Also tried it manually. I was able to download the files until yesterday without any problems. Could you please help?
Thank you!
What file(s) are you trying to download, i.e. what is in download.txt?
Hi Dylan,
I am having a similar issue. The specific error comes from downloading TNG50-1 snapshot-13 data using the provided wget path. It reads as follows:
I get the same error at other snapshots such as 50 and 17. Sometimes, when some data is downloaded, after I check on the quality of the download with the also provided file, I get:
Which makes me try to download files individually but that has been some work trying to do so for that many files.
Note, I am using screen to run some of these.
If you repeat this command now, are you still having trouble?
For TNG50-1 snapshot-13, I get:
sha256sum: WARNING: 43 computed checksums did NOT match
Do I have to delete all 43 snapshots to run the code again?
I am still getting some errors when I deleted 10 of the erroneous files and ran the command to download files:
In addition, for TNG50-1 snapshot 50, I also get this error when I try to download some individual files:
Hi,
I am having the same issue when trying to download the offset files for TNG50-1-Dark snap 99. I get the error ERROR 403: Forbidden each time I try to download it. Perhaps there is something going on with the site such as updates?
All working now?
It is working for me now! Thank you.
Hi,
I also met similar issues. I want to download the full snap 33 of TNG100-1, and I kept enconutering error 403 and error 503. And when I tried to download the offset (to check my internet), I got error 503. I wonder if someone is doing maintenance or updates to the server?
This appears to just be that the server is busy, because some other people are using it too much.
I will keep an eye on it. Please try again, and let me know if you have further problems,
Some of the downloads seem to be running ok but I still get the following message for TNG300-1 snapshot-99 download:
sha256sum: WARNING: 207 listed files could not be read
sha256sum: WARNING: 33 computed checksums did NOT match
Not sure what to make out of the 207 listed files. In this case, do I just run the command to download the data again?
Also, for reference to everyone: for the 33 computer checksums that did NOT match, I delete those 33 files manually (the ones where the check says FAILED) and then I redownload them. You can do so individually or using the command to download all the data. Since the -nc is in the command, it will only download the files that are missing.
checksums for TNG100-1 LHalomerger trees is not working! it says files are not available when they are in the current directory