Public Data Access Overview / Discussion Forum

Hi Dylan, hope you are well,

In the second table here, it says that the TNG100-3 run contains 8 chunks per snapshot.

I assume this to mean that the simulation box is split into 8 different sub-boxes, each which may be loaded separately. Is this correct?

When on the Max Planck virgo machine I look at where the data to the simulation is stored for a snapshot (e.g. in the dirrectory /snapdir_099/) I see the following:

snap_099.0.hdf5 snap_099.1.hdf5 snap_099.2.hdf5 snap_099.3.hdf5 snap_099.4.hdf5 snap_099.5.hdf5 snap_099.6.hdf5

This is only 7 chunks of data rather than 8. Please could you let me know where the final chunk is?

Furthermore, when I load, e.g., chunk 0 (snap_099.0.hdf5) and examine, e.g., the sum of all PartType0 masses in the data (e.g. using np.sum(ad['PartType0','Masses'])) and then I do the same for chunk 3 (snap_099.3.hdf5) I get the exact same result, all the way down to 8 decimal figures. I would expect to see slightly different values for different regions of the simulation, even if they are at the same redshift.

Please could you help me understand what is going on here?

Many thanks,

Charlie

Hi,

Typo in the table! Thanks for pointing it out, will fix it.

You can verify there are 7 files per snapshot by looking at the NumFilesPerSnapshot attribute of the Header.

Hi Dylan,

Thanks again forr the fast response. That clears up the table issue!

Regarding my second point, if i do:

    import yt
    data='snap_099.0.hdf5'
    ds=yt.load(data)
    ds.index
    ad = ds.all_data()
    tot_PT0 = np.sum(ad['PartType0','Masses'])
    print(tot_PT0)

and also do:

    import yt
    data='snap_099.3.hdf5'
    ds=yt.load(data)
    ds.index
    ad = ds.all_data()
    tot_PT0 = np.sum(ad['PartType0','Masses'])
    print(tot_PT0)

Should I really expect to see the same result for each chunk, even though they are separate areas in the simulation?

I ask this because I go on to mask this data for each chunk in order to sort matter into regions of large scale structure. However I am finding that as with total mass, the amount of matter in each type of structure is identical across chunks in a snapshot, which makes me suspect I am doing something wrong.

Cheers!

Charlie

Hi Charlie,

I suspect yt.load() is doing something you don't expect (i.e. loading all chunks no matter if you pass the filename of one chunk), but I cannot say what, not knowing much about yt. In this case:

for i in range(7):
   with h5py.File('snap_099.%d.hdf5' % i,'r') as f:
       print(i, f['PartType0/Masses'][()].sum())

0 80875.64
1 80654.95
2 78902.46
3 79220.86
4 78015.27
5 78685.61
6 81959.23

Hi again Dylan,

The sum of your output is:

558314.02

The result of either of my blocks of code yields:

558314.07068934

Which is extremely close. So your suspicion sound like it is on point. Did you round off your values slightly?

Cheers,

Charlie