README

*******************************************************************************

INTRODUCTION

This is a public void catalog of the following populations:

+ SDSS DR7 in redshift space and comoving space ("sdss_dr7")

+ SDSS DR9 in redshift space and comoving space ("sdss_dr9")

+ SDSS DR10 in redshift space and comoving space ("sdss_dr10")

+ 1024^3 particle, 1 Gpc/h Dark Matter Simulation ("mock1024")

+ Dark Matter Halos in the above simulation

+ HOD Mocks drawn from the above simulation

+ HOD Mocks drawn from a 4096^3 particle, 4 Gpc/h simulation ("lanl")

+ HOD Mocks from 98 2LPT runs matching SDSS DR9 (Manera et al. 2012)*

+ masked SDSS (un)constrained realizations ("masked3D")

For information regarding the construction of this catalog, please see Sutter

et al. "A public void catalog from the SDSS DR7 galaxy redshift surveys based

on the watershed transform" (arXiv: 1207.2524), Sutter et al. "VIDE:

The Void IDentification and Examination toolkit" (arXiv:1406.1191),

and the News & Updates section of http://www.cosmicvoids.net for corrections

and clarifications.

If you use the DR7 voids, please cite arXiv: 1207.2524

If you use the DR9 voids, please cite arXiv: 1310.7155

If you use the DR10 voids, please cite arXiv: 1404.5618

If you use the "mock1024" voids, please cite arXiv: 1309.5087

If you use the "lanl" voids, please cite arXiv: 1309.5087

If you use the "2lpt" voids, please cite arXiv: 1310.7155

If you use the "masked3D" voids, please contact florent.leclercq@polytechnique.org for more information

For questions, please contact Paul Sutter (sutter@iap.fr). Note that

requests for *additional* processing and/or analysis will be treated

as invitations for collaboration and joint authorship!

* Due to space constraints on our current server this is only available

by request. We are trying to resolve the situation. Sorry!

*******************************************************************************

VERSION HISTORY

2015.03.31: Added masked3D

2014.06.18 : Fixed bug in report of mean separation in sample_info.txt

Added sdss_dr10

Put void centers (x,y,z) for observations in correct coordinates

2013.10.25 : New layout: sdss_dr7 and sdss_dr9 now contain two subdirectories,

"redshift" and "comoving", corresponding to the coordinate

system in which we identified the respective voids.

Removal of alternate catalogs that trim on hierarchy level;

we now just provide two catalogs: a "default" catalog with

only parent voids with central density < 0.2 mean, and a

fully untrimmed version.

Guide to trimming the catalog added to README.

Number of particles, number of boundary particles, survey volume,

estimated mean particle separation added to sample_info.txt

Inclusion of 2LPT mocks and masked N-body mocks

2013.09.19 : "_nocut_" files now replaced by "untrimmed_" files and contain

full catalog information (barycenters, sky positions, shapes, etc.)

Voids that contain subvoids are now removed by default

(n.b. this really only affects high-resolution mocks)

All voids, including "parent" voids, are in the "untrimmed_" files

centers_ file outputs parent ID and number of sub-voids

Improved shape estimation

Addition of mock catalogs

Addition of DR9 voids

Inclusion of particle volume file

Inclusion of source catalogs

2013.02.20 : More accurate calculation of mean galaxy separation and hence

minimum acceptable void radius, but please USE CAUTION when

using the smallest voids!

Fix so that central density cut is actually applied

Correct units in centers_ file header

Inclusion of PRELIMINARY ellipticity information

2012.11.17 : Removal of NetCDF file

Inclusion of member galaxy RA, Dec, redshift, and catalog ID

in the zobov binary particle file.

New layout - no more "all" and "central" directories

List of void centers now includes density contrast

Inclusion of dumpVoidParticles.c in release package

Improvements to dumpVoidParticle to report RA, Dec, etc.

sample_info.txt file now contains all sample information

Corrected "central" sample voids

Added additional small voids to "all" sample

More consistent handling of redshift boundaries

2012.08.16 : new layout - "lcdm" now part of sdss/dr7 subdirectory

added void ID column to centers.txt and sky_positions.txt

added NetCDF file with void galaxy RA, Dec, and redshift

added sample_info.txt files which provide basic sample info

improved dumpVoidParticles.c

2012.07.04 : Initial release.

*******************************************************************************

LAYOUT OF THIS CATALOG

The void catalog for each sample is given its own top-level directory in the catalog. The directories associated with each catalog are:

+ sdss_dr7 : SDSS DR7 Main Sample and LRGs

There are two subdirectories: redshift and comoving,

corresponding to the coordinate system in which we

identified the voids.

We divide this catalog into subdirectories based on the

volume-limited subsamples of the source catalog, labeled as

sample_lss.dr72[sample_name].dat.

+ sdss_dr9 : SDSS DR9 CMASS

There is a single subdirectory: comoving.

We divide this catalog into subdirectories based on the

volume-limited subsamples of the source catalog, labeled as

sample_lss.dr9cmass[sample_name].dat.

+ mock1024 : Dark Matter Simulation of 1024^3 particles in 1 Gpc/h box

All particles are kept in real space at redshift 0

and peculiar velocities are not included.

We divide this catalog into subdirectories based on different

ways of tracing the dark matter population:

mt_ssNNNNN : Random subsampling of the dark matter particles.

NNNNN is the mean number density in h^3 Mpc^-3.

mt_halos_minNNNNN : Just taking halo centers, only including

halos with minimum mass NNNNN.

"none" means all halos.

mt_hod_XXXXX : Mock galaxy population generated with Jeremy

Tinker's HOD code. Tuned to the observational

HOD fits to galaxy population XXXXX.

The "_d00" suffix is used to indicate any subvolumes, which

we are not using for this catalog.

+ lanl : Dark Matter Simulation of 4096^3 particles in 4 Gpc/h box

This was used to generate a single HOD mock at z=0.5.

Peculiar velocities are included.

Note that despite the names below, these are HOD mocks! They were

processed slightly differently and ended up with a different name.

We divide this catalog into subdirectories as follows:

lanl_halos_minnone_pv : HOD Mock of dr9mid in full cubic volume

masked/masked_lanl_halos_minnone_pv_z0.5_rNN : Masked HOD mock

with 6 different rotations of mask within volume. Galaxies

placed on lightcone measured from center of box.

+ 2lpt : HOD 2LPT Mocks with the same survey geometry as SDSS DR9.

See Manera et al. (2012) for details.

We divide this catalog into subdirectories numbered 3001-3099.

+ masked3D : Centers and sky positions ONLY for constrained and

unconstrained simulations of arXiv:1410.0355, due to the

difficulty in constructing the source simulations.

WARNING: NOT COMPATIBLE WITH VIDE TOOLKIT!

Contact florent.leclercq@polytechnique.org for more information.

In addition, there are the following top-level directories:

+ source_catalogs: ASCII files of data populations used for void finding

(at this time we cannot include simulation populations

due to size contraints).

+ figures: contains data files necessary to reproduce the figures found in the 2012 paper

+ tools: routines and useful tools for reading the catalog. The program dumpVoidParticles has an example input file and can be used to extract void galaxies.

-------------------------------------------------------------------------------

The following files describe the sample, ZOBOV catalog, and void zone and galaxy information:

+ sample_info.txt: Small text file which lists some basic info of the sample

+ zobov_slice_lss.dr72[sample_name].dat: Binary file with the galaxy positions, RA, Dec, redshift, and unique catalog ID. Note that this file contains *all* particles in the particular sample, not just void galaxies. See the dumpVoidParticles.c routine for parsing this file and extracting void galaxies.

+ voidDesc_[all,central]_[sample_name].out: ASCII file with void indices and

basic void properties in ZOBOV-normalized units.

+ voidZone_[sample_name].out: Binary file which links voids to zones.

+ voidPart_[sample_name].out: Binary file which links zones to particles.

+ vol_[sample_name].out: Binary file with particle local volumes

To parse these files, see the dumpVoidParticles.c routine in the tools/ directory, or visit the ZOBOV help page at

http://skysrv.pha.jhu.edu/~neyrinck/voboz/zobovhelp.html

-------------------------------------------------------------------------------

In addition, we provide the following derived void information:

+ centers_[all, central]_[sample_name].txt: ASCII file with void centers,

volume, effective radius, void ID, density contrast, etc.

+ sky_positions_[all, central]_[sample_name].txt: ASCII file with void RA,

dec, effective radius, and void ID

+ shapes_[all, central]_[sample_name].txt: ASCII file with best-fit

eigenvalues and eigenvectors for each void, in same coordinate system

as x,y,z values in centers_ file

-------------------------------------------------------------------------------

In addition to the [all,central] version of each file above, there are

two versions of each catalog:

+ no prefix: only parent voids, density cut applied (default catalog)

+ untrimmed: all voids, density cut not applied

------------------------------------------------------------------------------

The figures/ directory contains data files which can be used to reproduce

various figures in the paper. The file names indicate the figure and whether

they are from the "central" or "all" catalog:

fig5_[central,all].dat: Size distributions.

ASCII file with 7 columns: radius bin, and one column of number counts

for each sample

fig6_[central,all].dat: Redshift distributions.

ASCII file with 7 columns: redshift bin, and one column of number counts

for each sample

fig7_[10-15,20-25,50-55,90-95]_[central,all].dat: Radial profiles.

One file for each stack (the number indicates the radius range of that

stack). Variable number of columns depending on which samples have voids

in that stack. The first column is the radius in normalized units, and

subsequent columns are normalized number densities. The header indicates

which samples are in this file.

fig8_[10-15,20-25,50-55,90-95]_central.npy: Projections.

One file for each stack (the number indicates the radius range of that

stack). We supply projections for only the central catalog. NumPy array

of the output of matplotlib's hexbin function, which was used to generate

the plots.

*******************************************************************************

FILTERING THE CATALOG

We have developed a philosophy about this void catalog in which we attempt to report as many voids as possible and let you, the user, apply cuts and filters as you see appropriate. We have a set of filters that we apply for our own analysis, and for historical reasons this has become the "default" catalog. We also provide the full unfiltered catalog (except for a hard minimum size cut, to be discussed below) so you can create your own version of the catalog.

Some cuts you may consider applying:

- Size: We only provide voids with effective radii greater than the mean particle separation. However, the smallest voids are likely highly unreliable. Thus, you may want to only take voids with two times the minimum radius. You can find the radius in the "centers_" files.

- Minimum density: The watershed algorithm reports all basins, even if they have densities higher than you may like. You can, for example, only take voids with minimum Voronoi cell density < 0.2 or 0.3 times the mean density (as in Nadathur et al. 2013). Since minimum density scales inversely with void size, this also effectively removes most small voids. However, this quantity is very noisy, so your cuts may be more/less aggressive than you think. The minimum density can be found in the "voidDesc_" files. Since this is a raw ZOBOV output, it is expressed in units normalized to the total number of particles, including boundary particles. So you can either a) correct for the presence of boundary particles by rescaling by a factor of (n_g/n_tot)^3, or b) just look at the distribution of minimum densities and make a cut as you see fit. You can find n_g and t_tot in the sample_info.txt file.

- Central density: This is similar in spirit to the minimum density above, but is typically computed in spheres about the barycenter. For the default catalog, we take spheres of 1/4 the void effective radius. This tends to affect larger voids, and becomes less effective for smaller voids (since we start to probe below the mean particle separation). If you want to calculate this on your own, then you have to use the dumpVoidParticles tool to extract the member particles.

- Density contrast: This is the ratio between the density on the void ridgeline and the minimum density. You can compare this to, say, the probability of getting that density ratio in a Poisson distribution (Neyrinck 2008), but this procedure is not necessarily appropriate for sparse, biased populations. You can find this quantity in both the voidDesc_ and centers_ files.

- Tree level: There is a full hierarchy of voids, subvoids, sub-subvoids, etc.. You may wish to take only top-level "parent" voids, or only leaf-level "children" voids. The default catalog contains only parent voids. You can find the tree level in the centers_ files.

Our default catalog contains only top-level parent voids with central densities < 0.2 times the mean density. If you are happy with these cuts, then you can just use the un-prefixed catalog.

If you want to make your own cuts, then start with the "untrimmed_" catalogs and help yourself.

In addition, we also have the "all" and "central" catalog division.

We would like to thank Seshadri Nadathur; our conversations with him prompted the creation of this post.

*******************************************************************************

SPECIAL NOTES:

*******************************************************************************