Illustris Web API - performance in accessing a single field per subhalo

Connor Bottrell
  • 20 May '15

Is there a fast way to get a single field about each subhalo for a long list of subhalos?

For example, when I do:

import requests,string

halfmassrad_stars = {}

for subhalo in subhalo_list:
    halfmassrad_stars[str(subhalo)] = get(urlbase_for_Illustris1_135s + str(subhalo) + '/')['halfmassrad_stars']

This takes a considerable amount of time since it is parsing all of the field info for each subhalo. Or is it because of the url grabs?


Dylan Nelson
  • 20 May '15

Hi Connor,

As you suspect, the time here (probably 0.1-0.5 sec or so per subhalo) is mainly just the roundtrip latency for the page retrieval, and the time for the DB to grab the values for the particular subhalo. It wouldn't be any faster if there was some shortcut to get just a single value (there isn't).

If you want to do this for many subhalos (>1000) or many times, you should just download the group catalog (this is only 4.5 GB for snapshot 135). Then these same values will be found in e.g. SubhaloHalfmassRadType[:,4].

Connor Bottrell
  • 20 May '15

Alright. Thanks for your response, Dylan.

  • Page 1 of 1