README
*******************************************************************************
INTRODUCTION
This is a public void catalog of the following populations:
+ SDSS DR7 in redshift space and comoving space ("sdss_dr7")
+ SDSS DR9 in redshift space and comoving space ("sdss_dr9")
+ SDSS DR10 in redshift space and comoving space ("sdss_dr10")
+ 1024^3 particle, 1 Gpc/h Dark Matter Simulation ("mock1024")
+ Dark Matter Halos in the above simulation
+ HOD Mocks drawn from the above simulation
+ HOD Mocks drawn from a 4096^3 particle, 4 Gpc/h simulation ("lanl")
+ HOD Mocks from 98 2LPT runs matching SDSS DR9 (Manera et al. 2012)*
+ masked SDSS (un)constrained realizations ("masked3D")
For information regarding the construction of this catalog, please see Sutter
et al. "A public void catalog from the SDSS DR7 galaxy redshift surveys based
on the watershed transform" (arXiv: 1207.2524), Sutter et al. "VIDE:
The Void IDentification and Examination toolkit" (arXiv:1406.1191),
and the News & Updates section of http://www.cosmicvoids.net for corrections
and clarifications.
If you use the DR7 voids, please cite arXiv: 1207.2524
If you use the DR9 voids, please cite arXiv: 1310.7155
If you use the DR10 voids, please cite arXiv: 1404.5618
If you use the "mock1024" voids, please cite arXiv: 1309.5087
If you use the "lanl" voids, please cite arXiv: 1309.5087
If you use the "2lpt" voids, please cite arXiv: 1310.7155
If you use the "masked3D" voids, please contact florent.leclercq@polytechnique.org for more information
For questions, please contact Paul Sutter (sutter@iap.fr). Note that
requests for *additional* processing and/or analysis will be treated
as invitations for collaboration and joint authorship!
* Due to space constraints on our current server this is only available
by request. We are trying to resolve the situation. Sorry!
*******************************************************************************
VERSION HISTORY
2015.03.31: Added masked3D
2014.06.18 : Fixed bug in report of mean separation in sample_info.txt
Added sdss_dr10
Put void centers (x,y,z) for observations in correct coordinates
2013.10.25 : New layout: sdss_dr7 and sdss_dr9 now contain two subdirectories,
"redshift" and "comoving", corresponding to the coordinate
system in which we identified the respective voids.
Removal of alternate catalogs that trim on hierarchy level;
we now just provide two catalogs: a "default" catalog with
only parent voids with central density < 0.2 mean, and a
fully untrimmed version.
Guide to trimming the catalog added to README.
Number of particles, number of boundary particles, survey volume,
estimated mean particle separation added to sample_info.txt
Inclusion of 2LPT mocks and masked N-body mocks
2013.09.19 : "_nocut_" files now replaced by "untrimmed_" files and contain
full catalog information (barycenters, sky positions, shapes, etc.)
Voids that contain subvoids are now removed by default
(n.b. this really only affects high-resolution mocks)
All voids, including "parent" voids, are in the "untrimmed_" files
centers_ file outputs parent ID and number of sub-voids
Improved shape estimation
Addition of mock catalogs
Addition of DR9 voids
Inclusion of particle volume file
Inclusion of source catalogs
2013.02.20 : More accurate calculation of mean galaxy separation and hence
minimum acceptable void radius, but please USE CAUTION when
using the smallest voids!
Fix so that central density cut is actually applied
Correct units in centers_ file header
Inclusion of PRELIMINARY ellipticity information
2012.11.17 : Removal of NetCDF file
Inclusion of member galaxy RA, Dec, redshift, and catalog ID
in the zobov binary particle file.
New layout - no more "all" and "central" directories
List of void centers now includes density contrast
Inclusion of dumpVoidParticles.c in release package
Improvements to dumpVoidParticle to report RA, Dec, etc.
sample_info.txt file now contains all sample information
Corrected "central" sample voids
Added additional small voids to "all" sample
More consistent handling of redshift boundaries
2012.08.16 : new layout - "lcdm" now part of sdss/dr7 subdirectory
added void ID column to centers.txt and sky_positions.txt
added NetCDF file with void galaxy RA, Dec, and redshift
added sample_info.txt files which provide basic sample info
improved dumpVoidParticles.c
2012.07.04 : Initial release.
*******************************************************************************
LAYOUT OF THIS CATALOG
The void catalog for each sample is given its own top-level directory in the catalog. The directories associated with each catalog are:
+ sdss_dr7 : SDSS DR7 Main Sample and LRGs
There are two subdirectories: redshift and comoving,
corresponding to the coordinate system in which we
identified the voids.
We divide this catalog into subdirectories based on the
volume-limited subsamples of the source catalog, labeled as
sample_lss.dr72[sample_name].dat.
+ sdss_dr9 : SDSS DR9 CMASS
There is a single subdirectory: comoving.
We divide this catalog into subdirectories based on the
volume-limited subsamples of the source catalog, labeled as
sample_lss.dr9cmass[sample_name].dat.
+ mock1024 : Dark Matter Simulation of 1024^3 particles in 1 Gpc/h box
All particles are kept in real space at redshift 0
and peculiar velocities are not included.
We divide this catalog into subdirectories based on different
ways of tracing the dark matter population:
mt_ssNNNNN : Random subsampling of the dark matter particles.
NNNNN is the mean number density in h^3 Mpc^-3.
mt_halos_minNNNNN : Just taking halo centers, only including
halos with minimum mass NNNNN.
"none" means all halos.
mt_hod_XXXXX : Mock galaxy population generated with Jeremy
Tinker's HOD code. Tuned to the observational
HOD fits to galaxy population XXXXX.
The "_d00" suffix is used to indicate any subvolumes, which
we are not using for this catalog.
+ lanl : Dark Matter Simulation of 4096^3 particles in 4 Gpc/h box
This was used to generate a single HOD mock at z=0.5.
Peculiar velocities are included.
Note that despite the names below, these are HOD mocks! They were
processed slightly differently and ended up with a different name.
We divide this catalog into subdirectories as follows:
lanl_halos_minnone_pv : HOD Mock of dr9mid in full cubic volume
masked/masked_lanl_halos_minnone_pv_z0.5_rNN : Masked HOD mock
with 6 different rotations of mask within volume. Galaxies
placed on lightcone measured from center of box.
+ 2lpt : HOD 2LPT Mocks with the same survey geometry as SDSS DR9.
See Manera et al. (2012) for details.
We divide this catalog into subdirectories numbered 3001-3099.
+ masked3D : Centers and sky positions ONLY for constrained and
unconstrained simulations of arXiv:1410.0355, due to the
difficulty in constructing the source simulations.
WARNING: NOT COMPATIBLE WITH VIDE TOOLKIT!
Contact florent.leclercq@polytechnique.org for more information.
In addition, there are the following top-level directories:
+ source_catalogs: ASCII files of data populations used for void finding
(at this time we cannot include simulation populations
due to size contraints).
+ figures: contains data files necessary to reproduce the figures found in the 2012 paper
+ tools: routines and useful tools for reading the catalog. The program dumpVoidParticles has an example input file and can be used to extract void galaxies.
-------------------------------------------------------------------------------
The following files describe the sample, ZOBOV catalog, and void zone and galaxy information:
+ sample_info.txt: Small text file which lists some basic info of the sample
+ zobov_slice_lss.dr72[sample_name].dat: Binary file with the galaxy positions, RA, Dec, redshift, and unique catalog ID. Note that this file contains *all* particles in the particular sample, not just void galaxies. See the dumpVoidParticles.c routine for parsing this file and extracting void galaxies.
+ voidDesc_[all,central]_[sample_name].out: ASCII file with void indices and
basic void properties in ZOBOV-normalized units.
+ voidZone_[sample_name].out: Binary file which links voids to zones.
+ voidPart_[sample_name].out: Binary file which links zones to particles.
+ vol_[sample_name].out: Binary file with particle local volumes
To parse these files, see the dumpVoidParticles.c routine in the tools/ directory, or visit the ZOBOV help page at
http://skysrv.pha.jhu.edu/~neyrinck/voboz/zobovhelp.html
-------------------------------------------------------------------------------
In addition, we provide the following derived void information:
+ centers_[all, central]_[sample_name].txt: ASCII file with void centers,
volume, effective radius, void ID, density contrast, etc.
+ sky_positions_[all, central]_[sample_name].txt: ASCII file with void RA,
dec, effective radius, and void ID
+ shapes_[all, central]_[sample_name].txt: ASCII file with best-fit
eigenvalues and eigenvectors for each void, in same coordinate system
as x,y,z values in centers_ file
-------------------------------------------------------------------------------
In addition to the [all,central] version of each file above, there are
two versions of each catalog:
+ no prefix: only parent voids, density cut applied (default catalog)
+ untrimmed: all voids, density cut not applied
------------------------------------------------------------------------------
The figures/ directory contains data files which can be used to reproduce
various figures in the paper. The file names indicate the figure and whether
they are from the "central" or "all" catalog:
fig5_[central,all].dat: Size distributions.
ASCII file with 7 columns: radius bin, and one column of number counts
for each sample
fig6_[central,all].dat: Redshift distributions.
ASCII file with 7 columns: redshift bin, and one column of number counts
for each sample
fig7_[10-15,20-25,50-55,90-95]_[central,all].dat: Radial profiles.
One file for each stack (the number indicates the radius range of that
stack). Variable number of columns depending on which samples have voids
in that stack. The first column is the radius in normalized units, and
subsequent columns are normalized number densities. The header indicates
which samples are in this file.
fig8_[10-15,20-25,50-55,90-95]_central.npy: Projections.
One file for each stack (the number indicates the radius range of that
stack). We supply projections for only the central catalog. NumPy array
of the output of matplotlib's hexbin function, which was used to generate
the plots.
*******************************************************************************
FILTERING THE CATALOG
We have developed a philosophy about this void catalog in which we attempt to report as many voids as possible and let you, the user, apply cuts and filters as you see appropriate. We have a set of filters that we apply for our own analysis, and for historical reasons this has become the "default" catalog. We also provide the full unfiltered catalog (except for a hard minimum size cut, to be discussed below) so you can create your own version of the catalog.
Some cuts you may consider applying:
- Size: We only provide voids with effective radii greater than the mean particle separation. However, the smallest voids are likely highly unreliable. Thus, you may want to only take voids with two times the minimum radius. You can find the radius in the "centers_" files.
- Minimum density: The watershed algorithm reports all basins, even if they have densities higher than you may like. You can, for example, only take voids with minimum Voronoi cell density < 0.2 or 0.3 times the mean density (as in Nadathur et al. 2013). Since minimum density scales inversely with void size, this also effectively removes most small voids. However, this quantity is very noisy, so your cuts may be more/less aggressive than you think. The minimum density can be found in the "voidDesc_" files. Since this is a raw ZOBOV output, it is expressed in units normalized to the total number of particles, including boundary particles. So you can either a) correct for the presence of boundary particles by rescaling by a factor of (n_g/n_tot)^3, or b) just look at the distribution of minimum densities and make a cut as you see fit. You can find n_g and t_tot in the sample_info.txt file.
- Central density: This is similar in spirit to the minimum density above, but is typically computed in spheres about the barycenter. For the default catalog, we take spheres of 1/4 the void effective radius. This tends to affect larger voids, and becomes less effective for smaller voids (since we start to probe below the mean particle separation). If you want to calculate this on your own, then you have to use the dumpVoidParticles tool to extract the member particles.
- Density contrast: This is the ratio between the density on the void ridgeline and the minimum density. You can compare this to, say, the probability of getting that density ratio in a Poisson distribution (Neyrinck 2008), but this procedure is not necessarily appropriate for sparse, biased populations. You can find this quantity in both the voidDesc_ and centers_ files.
- Tree level: There is a full hierarchy of voids, subvoids, sub-subvoids, etc.. You may wish to take only top-level "parent" voids, or only leaf-level "children" voids. The default catalog contains only parent voids. You can find the tree level in the centers_ files.
Our default catalog contains only top-level parent voids with central densities < 0.2 times the mean density. If you are happy with these cuts, then you can just use the un-prefixed catalog.
If you want to make your own cuts, then start with the "untrimmed_" catalogs and help yourself.
In addition, we also have the "all" and "central" catalog division.
We would like to thank Seshadri Nadathur; our conversations with him prompted the creation of this post.
*******************************************************************************
SPECIAL NOTES:
*******************************************************************************