Is is possible to use the FoF catalogs from the full simulation runs on the corresponding subboxes?

Christoph Behrens

20 Oct '17

I do understand that the FoF catalogs have not been created for the subboxes. Is it however possible to use the FoF catalogs from the full box when making sure that the snapshot ID is the same for the subbox as for the FoF catalog of the full simulation volume? In other words; are the subbox snapshots just extracted from the full simulation run at runtime? I basically want to get around running FoF on one of the subboxes.

Christoph Behrens.

Dylan Nelson

20 Oct '17

Hi Christoph,

It's a good point, I think this should be technically possible. The subboxes are written once per timestep (say), and the full snapshots at only specific timesteps, those which are the closest to a fixed list of desired output redshifts.

So I think it should be true (for Illustris-1) that each full snapshot has a corresponding subbox which was written at the exact same time, and here you can take the spatial subset of the group catalog.

For example I see snapshot 110 has z=0.360688 and subbox 3296 matches.

You could verify by re-doing one of the easier computations in the group catalog (i.e. stellar mass enclosed in the half mass radius), and verifying the result. I'd be curious.

Christoph Behrens

20 Oct '17

Okay thanks. I will do what you suggest!

Christoph Behrens

23 Oct '17

Hi Dylan,
I am a bit confused. I took PartType4/Coordinates and PartType4/Masses within Subhalo/SubhaloHalfmassRadType (4), and sum these masses for all subhalos that lie completely in the subbox. I do not get the same numbers if I compare with Subhalo/SubhaloMassInHalfRadType (4). Am I missing something here? The Snaps I use are 659 for Subbox 1 and the catalog for snap 38 for the big box, both Illustris 1. They both have redshift 8.012172.

Dylan Nelson

23 Oct '17

You mean within a distance of SubhaloHalfmassRadType(4) from SubhaloPos?

If you just do this calculation for the actual subhalo in the full snapshot 38, does it work?

Christoph Behrens

24 Oct '17

Yes, I take all the Type4 particles with distance(Coordinates,SubHaloPos)< SubhaloHalfmassRadType(4) and sum them up for each subhalo within the subvolume. I compare the mass I get for each subhalo with the values in SubhaloMassInHalfRadType (4).
I do not have the full snapshot on disk, and haven't used the WebAPI so much yet that I can do the latter calculation quickly. I was hoping I misunderstood something in the data specifications. The specifications for SubhaloMassInHalfRadType says: "Sum of masses of all particles/cells (split by type) within the stellar half mass radius.", which I thought to be a typo. Is it? If not, what is the stellar half mass radius (it should not be the same as SubhaloMassInHalfRadType(4), because that includes wind particles, right?)?

Dylan Nelson

26 Oct '17

Hi Christoph,

I checked and this seems fine. Most likely, you are getting messed up by including wind particles in your calculation for the stellar mass. These are not included in e.g. SubhaloMassInRadType[4], and can be filtered out as having negative GFM_StellarFormationTime.

Specifically, subhalo 13391 has 56 real stars and total stellar mass 0.0047439896 as in the group catalog for snapshot 38, and also based on the particle information in subbox1 number 659.

In [1]: sP = simParams(res=1820,run='illustris',snap=38)
In [2]: x=cosmo.load.groupCat(sP, fieldsSubhalos=['SubhaloPos','SubhaloMassInRadType','SubhaloHalfmassRadType'])
In [3]: s=4000.0
In [4]: cen=[43100,53600,60800]
In [5]: pos=x['subhalos']['SubhaloPos']
In [6]: pos.shape
Out[6]: (3436820, 3)
In [7]: w = np.where( (np.abs(pos[:,0]-cen[0]) < s) & (np.abs(pos[:,1]-cen[1]) < s) & (np.abs(pos[:,2]-cen[2]) < s) )
In [8]: w[0].size
Out[8]: 3161
In [9]: w[0][0:10]
Out[9]: array([13391, 15344, 19352, 19353, 19354, 19355, 19356, 19357, 19358, 19359])
In [10]: sP_sub = simParams(res=1820,run='illustris',snap=659,variant='subbox1')
In [11]: stars_pos = cosmo.load.snapshotSubset(sP_sub, 'stars', 'pos')
In [12]: stars_mass = cosmo.load.snapshotSubset(sP_sub, 'stars', 'mass')
In [16]: dist = cosmo.util.periodicDists( pos[w[0][0],:], stars_pos, sP )
In [17]: dist.shape
Out[17]: (7950,)
In [20]: mass=x['subhalos']['SubhaloMassInRadType'][w,4]
In [22]: mass[0:100]
Out[22]:
array([[ 0.00474399, 0.00235048, 0.00046931, ..., 0. ,
0. , 0. ]], dtype=float32)
In [23]: w_dist = np.where(dist <= 2*x['subhalos']['SubhaloHalfmassRadType'][w[0][0],4])
In [25]: np.sum(stars_mass[w_dist])
Out[25]: 0.010526301
In [26]: ages = cosmo.load.snapshotSubset(sP_sub, 'stars', 'sftime')
In [27]: w_dist = np.where( (dist <= 2*x['subhalos']['SubhaloHalfmassRadType'][w[0][0],4]) & (ages>0.0) )
In [28]: w_dist
Out[28]:
(array([816, 819, 827, 828, 855, 856, 857, 859, 869, 870, 871, 874, 875,
876, 878, 882, 883, 885, 886, 887, 888, 890, 895, 896, 897, 898,
899, 900, 902, 903, 904, 905, 906, 907, 909, 910, 911, 913, 916,
922, 923, 925, 927, 929, 930, 931, 932, 933, 936, 938, 939, 941,
942, 945, 956, 983]),)
In [29]: np.sum(stars_mass[w_dist])
Out[29]: 0.0047439896

Christoph Behrens

2 Nov '17

HI Dylan,
Thanks... I was indeed missing the negative age. Sorry for that!

I do understand that the FoF catalogs have not been created for the subboxes. Is it however possible to use the FoF catalogs from the full box when making sure that the snapshot ID is the same for the subbox as for the FoF catalog of the full simulation volume? In other words; are the subbox snapshots just extracted from the full simulation run at runtime? I basically want to get around running FoF on one of the subboxes.

Hi Christoph,

It's a good point, I think this should be technically possible. The subboxes are written once per timestep (say), and the full snapshots at only specific timesteps, those which are the closest to a fixed list of desired output redshifts.

So I think it should be true (for Illustris-1) that each full snapshot has a corresponding subbox which was written at the exact same time, and here you can take the spatial subset of the group catalog.

For example I see snapshot 110 has

`z=0.360688`

and subbox 3296 matches.You could verify by re-doing one of the easier computations in the group catalog (i.e. stellar mass enclosed in the half mass radius), and verifying the result. I'd be curious.

Okay thanks. I will do what you suggest!

Hi Dylan, I am a bit confused. I took PartType4/Coordinates and PartType4/Masses within Subhalo/SubhaloHalfmassRadType (4), and sum these masses for all subhalos that lie completely in the subbox. I do not get the same numbers if I compare with Subhalo/SubhaloMassInHalfRadType (4). Am I missing something here? The Snaps I use are 659 for Subbox 1 and the catalog for snap 38 for the big box, both Illustris 1. They both have redshift 8.012172.

You mean within a distance of SubhaloHalfmassRadType(4) from SubhaloPos?

If you just do this calculation for the actual subhalo in the full snapshot 38, does it work?

Yes, I take all the Type4 particles with distance(Coordinates,SubHaloPos)< SubhaloHalfmassRadType(4) and sum them up for each subhalo within the subvolume. I compare the mass I get for each subhalo with the values in SubhaloMassInHalfRadType (4). I do not have the full snapshot on disk, and haven't used the WebAPI so much yet that I can do the latter calculation quickly. I was hoping I misunderstood something in the data specifications. The specifications for SubhaloMassInHalfRadType says: "Sum of masses of all particles/cells (split by type) within the stellar half mass radius.", which I thought to be a typo. Is it? If not, what is the stellar half mass radius (it should not be the same as SubhaloMassInHalfRadType(4), because that includes wind particles, right?)?

Hi Christoph,

I checked and this seems fine. Most likely, you are getting messed up by including wind particles in your calculation for the stellar mass. These are not included in e.g.

`SubhaloMassInRadType[4]`

, and can be filtered out as having negative`GFM_StellarFormationTime`

.Specifically, subhalo 13391 has 56 real stars and total stellar mass 0.0047439896 as in the group catalog for snapshot 38, and also based on the particle information in subbox1 number 659.

HI Dylan, Thanks... I was indeed missing the negative age. Sorry for that!