Public Data Access Overview / Discussion Forum

Hi all,

Is there a way to find the DM halo associated with a galaxy? I found that the maximum ID in TNG50 at snapshot 99 is ~5 million, while the length of array containing major to minor axis ratio of halos (from the Supplementary Data Catalogs) is greater than 10 million. I originally thought there's a one to one correspondence between halos and galaxies, now I'm a bit more confused...

1678167840908.png

Yes there are two types of objects in the group catalogs, "halos" and "subhalos".

In TNG50-1 at snapshot 99 there are 10247012 groups, and 5688113 subhalos.

You can map between the two with the SubhaloGrNr and GroupFirstSub fields (see documentation).

thanks so much!!

@Dylan Nelson said:
Yes there are two types of objects in the group catalogs, "halos" and "subhalos".

In TNG50-1 at snapshot 99 there are 10247012 groups, and 5688113 subhalos.

You can map between the two with the SubhaloGrNr and GroupFirstSub fields (see documentation).

When I tried to read the group catalog file with the field GroupFirstSub with the following code:

f = h5py.File('./Group_Catalog/fof_subhalo_tab_099.0.hdf5', 'r')
f['GroupFirstSub']

I got a key error: KeyError: "Unable to open object (object 'GroupFirstSub' doesn't exist)"
So it seems there's no such field? I was thinking which part could go wrong?

Dear Feiyu,

I think you should explore the data more and get a sense of its structure (e.g. with h5ls). This dataset is inside a group, called Group, I believe.

@Dylan Nelson said:
Dear Feiyu,

I think you should explore the data more and get a sense of its structure (e.g. with h5ls). This dataset is inside a group, called Group, I believe.

Dear Dylan,

The group catalogfof_subhalo_tab_099.hdf5 at snapshot 99 seems to be split into hundreds of smaller files, with names goes from fof_subhalo_tab_099.0.hdf5 to fof_subhalo_tab_099.679.hdf5. I tried loading some individual files, and some of them have 0 members in the "Subhalo" group. I read all these files and combined them to form a single and complete SubhaloGrNr dataset.

The problem is that the length of combined array is 5634505, which is a bit off from the number of subhalos 5688113. As I mentioned earlier, I suppose some parent halos have no subhalo since their "Subhalo" field is empty, but I also encountered some missing file errors (for example, the file fof_subhalo_tab_099.438 doesn't seem to exist in the group catalog). Could this be the cause of incomplete dataset?

If you downloaded all of the files for this snapshot:

https://www.tng-project.org/api/TNG50-1/files/groupcat-99/

and load the SubhaloGrNr dataset (yourself, or using the provided scripts), then the size of that dataset will be 5688113.

It sounds like, however you downloaded the files, you missed one (or more).

@Dylan Nelson said:
If you downloaded all of the files for this snapshot:

https://www.tng-project.org/api/TNG50-1/files/groupcat-99/

and load the SubhaloGrNr dataset (yourself, or using the provided scripts), then the size of that dataset will be 5688113.

It sounds like, however you downloaded the files, you missed one (or more).

Thanks for the reply! Now I have the complete SubhaloGrNr dataset. Suppose I have subhalo ID 300905 at snapshot 99, do I just use SubhaloGrNr[300905] to get the ID of its parent halo, and then use halo_structure_099[SubhaloGrNr[300905]] to get the various halo structure dataset for that parent halo?

Yes that's right, and you probably also want to check GroupFlag == 1 as the documentation suggests.

@Dylan Nelson said:
Yes that's right, and you probably also want to check GroupFlag == 1 as the documentation suggests.

Sorry but I was just wondering if there's a bash command that downloads all the available files in a catalog? Such as all the halo structure file on this site? Thanks in advance!

Sorry this is a deleted reply

Yes an example wget command is shown when you download either a snapshot or group catalog. In this case

wget -nd -nc -nv -e robots=off -l 1 -r -A hdf5 --content-disposition --header="API-Key: HERE" "https://www.tng-project.org/api/TNG50-1/files/halo_structure/?format=api"