diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..2451551a --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,48 @@ +# TreeCorr Codex Notes + +## Python Environment +Use the conda environment at: +- `/Users/Mike/miniforge3/envs/py3.12` + +Conda activation from Codex shells should source: +- `/Users/Mike/lsst/lsstsw/miniconda/etc/profile.d/conda.sh` + +Then activate by full path: +```bash +source /Users/Mike/lsst/lsstsw/miniconda/etc/profile.d/conda.sh +conda activate /Users/Mike/miniforge3/envs/py3.12 +``` + +## Sphinx Docs Build +Use this sequence to build docs in Codex: +```bash +source /Users/Mike/lsst/lsstsw/miniconda/etc/profile.d/conda.sh +conda activate /Users/Mike/miniforge3/envs/py3.12 +export LC_ALL=C LANG=C +cd /Users/Mike/rmjarvis/TreeCorr/docs +make html +``` + +Rationale: +- `py3.12` is not always discoverable by name from the conda install on PATH, so full-path + activation is more reliable. +- Setting `LC_ALL`/`LANG` avoids locale errors when invoking Sphinx. + +## TreeCorr Documentation Conventions +- Prefer "data are" (treat "data" as plural) except when clearly used as a mass noun. +- Target a 100-character line length for prose/code comments where practical. +- Do not rewrite or shorten long URLs only to satisfy line-length limits. +- Keep existing documentation images unless there is a concrete issue requiring a change. +- Preserve intentional diction choices (e.g. keep "Contrariwise" where used intentionally). + +## Documentation Scope Priorities +- Prioritize user-facing `.rst` pages and Python docstrings in `treecorr/*.py`. +- Config-only docs are lower priority unless explicitly requested. + +## Routine Ignore Rules +- Ignore `/Users/Mike/rmjarvis/TreeCorr/docs/_build/` during normal edits/reviews. +- Do not edit `docs/Makefile` just for style/line-length cleanup. + +## Typo Sweep Rule +- When a typo is found, grep the whole repo for that same typo pattern (excluding build/data + artifacts) and fix all clear occurrences in the same pass. diff --git a/README.rst b/README.rst index 0ad3b805..86798d84 100644 --- a/README.rst +++ b/README.rst @@ -1,5 +1,5 @@ -.. image:: https://travis-ci.org/rmjarvis/TreeCorr.svg?branch=main - :target: https://travis-ci.org/rmjarvis/TreeCorr +.. image:: https://github.com/rmjarvis/TreeCorr/actions/workflows/ci.yml/badge.svg + :target: https://github.com/rmjarvis/TreeCorr/actions/workflows/ci.yml .. image:: https://codecov.io/gh/rmjarvis/TreeCorr/branch/main/graph/badge.svg :target: https://codecov.io/gh/rmjarvis/TreeCorr @@ -8,13 +8,17 @@ functions. - The code is hosted at https://github.com/rmjarvis/TreeCorr - It can compute correlations of regular number counts, weak lensing shears, or - scalar quantities such as convergence or CMB temperature fluctutations. + scalar quantities such as convergence or CMB temperature fluctuations. - 2-point correlations may be auto-correlations or cross-correlations. This - includes shear-shear, count-shear, count-count, kappa-kappa, etc. (Any - combination of shear, kappa, and counts.) -- 3-point correlations currently can only be auto-correlations. This includes - shear-shear-shear, count-count-count, and kappa-kappa-kappa. The cross - varieties are planned to be added in the near future. + includes shear-shear (aka GG), count-shear (NG), count-count (NN), kappa-kappa (KK), etc. + (Any combination of shear, kappa, and counts.) +- 3-point correlations include both auto-correlations (e.g. NNN, KKK, GGG) + and mixed cross-correlations (e.g. NNG, NKK, KGG and permutations). +- Additional field types with non-zero spin are supported in two-point + correlations, including vectors (spin-1, V), shear (spin-2, G), trefoil + (spin-3, T), and quatrefoil (spin-4, Q). + Three-point correlations currently support N, K, and G fields (including mixed + combinations such as NNG, NKK, KGG and permutations). - Both 2- and 3-point functions can be done with the correct curved-sky calculation using RA, Dec coordinates, on a Euclidean tangent plane, or in 3D using either (RA,Dec,r) or (x,y,z) positions. @@ -28,7 +32,7 @@ functions. for a bin size b=0.1 in log(r). It scales as b^(-2). This is the slowest of the various kinds of 2-point correlations, so others will be a bit faster, but with the same scaling with N and b. -- The running time for 3-point functions are highly variable depending on the +- The running time for 3-point functions is highly variable depending on the range of triangle geometries you are calculating. They are significantly slower than the 2-point functions, but many orders of magnitude faster than brute force algorithms. @@ -38,8 +42,8 @@ functions. I've made since then, but this will suffice as a reference for now.) - If you use the three-point multipole functionality of TreeCorr, please also reference **Porth et al, 2023, arXiv:2309.08601** -- Record on the Astrophyics Source Code Library: http://ascl.net/1508.007 -- Developed by Mike Jarvis. Fee free to contact me with questions or comments +- Record on the Astrophysics Source Code Library: https://ascl.net/1508.007 +- Developed by Mike Jarvis. Feel free to contact me with questions or comments at mikejarvis17 at gmail. Or post an issue (see below) if you have any problems with the code. @@ -68,7 +72,7 @@ or:: conda update -c conda-forge treecorr -Depending on the write permissions of the python distribution for your specific +Depending on the write permissions of the Python distribution for your specific system, you might need to use one of the following variants for pip installation:: sudo pip install treecorr @@ -131,7 +135,7 @@ that is also relatively straightforward: potentially useful. - fitsio is required for reading FITS catalogs or writing to FITS output files. - - pandas will signficantly speed up reading from ASCII catalogs. + - pandas will significantly speed up reading from ASCII catalogs. - pandas and pyarrow are required for reading Parquet files. - h5py is required for reading HDF5 catalogs. - mpi4py is required for running TreeCorr across multiple machines using MPI. @@ -154,12 +158,11 @@ that is also relatively straightforward: 3. Install ^^^^^^^^^^ - You can then install TreeCorr from the local distribution. Typically this would be the - command:: + You can then install TreeCorr from the local distribution. Typically this would be:: pip install . - If you don't have write permission in your python distribution, you might need + If you don't have write permission in your Python distribution, you might need to use:: pip install . --user @@ -171,7 +174,7 @@ that is also relatively straightforward: Installing corr2 script to /anaconda3/bin or similar in the output to see where the scripts are installed. If the - directory is not in your path, you will also get a warning message at the + directory is not in your PATH, you will also get a warning message at the end letting you know which directory you should add to your path if you want to run these scripts. @@ -189,7 +192,7 @@ that is also relatively straightforward: Two-point Correlations ---------------------- -This software is able to compute a variety of two-point correlations: +TreeCorr can compute a variety of two-point correlations: :NN: The normal two-point correlation function of number counts (typically galaxy counts). @@ -207,7 +210,7 @@ This software is able to compute a variety of two-point correlations: quantity. :KG: Cross-correlation of convergence with shear. Like the NG calculation, but - weighting the pairs by the kappa values the foreground points. + weighting the pairs by the kappa values of the foreground points. There are also additional combinations involving complex fields with different spin than 2 (shear is a spin-2 field). See `Two-point Correlation Functions @@ -216,13 +219,13 @@ than 2 (shear is a spin-2 field). See `Two-point Correlation Functions Three-point Correlations ------------------------ -Three point correlation functions are significantly more complicated, being functions +Three-point correlation functions are significantly more complicated, being functions of three parameters defining the triangle size and shape, rather than just a single separation. For cross-correlations, there are also issues related to whether one wants -to allow the different catalogs to take all possible vertices in the triangles are be +to allow the different catalogs to take all possible vertices in the triangles or be fixed to a particular vertex. -This software is able to compute the following three-point auto-correlations: +TreeCorr can compute the following three-point auto-correlations: :NNN: Three-point correlation function of number counts. @@ -233,11 +236,11 @@ This software is able to compute the following three-point auto-correlations: :KKK: Three-point kappa correlation function. Again, "kappa" here can be any scalar quantity. -It is also possible to compute cross correlations combining two of these types, such -as NNG, NKK, KGK, etc.. The ordering of the letters indicates which type is placed -at which numbered vertex in the triangles where the first vertex is opposite d1, the -second opposite d2, and the third opposite d3. The meaning of the three side lengths -is particular to the choice of binning. +It is also possible to compute cross-correlations combining two of these types, such +as NNG, NKK, KGK, etc. The ordering of the letters indicates which type is placed +at each numbered vertex in the triangle: the first vertex is opposite d1, the +second is opposite d2, and the third is opposite d3. The meaning of the three +side lengths is particular to the choice of binning. See `Three-point Correlation Functions `_ for more details. @@ -251,18 +254,18 @@ which is the name of a configuration file:: corr2 config_file corr3 config_file -A sample configuration file for corr2 is provided, called sample.params. +A sample configuration file for corr2 is provided, called sample_config.yaml. See `Configuration Parameters `_ for the complete documentation about the allowed parameters. You can also specify parameters on the command line after the name of -the configuration file. e.g.:: +the configuration file. For example:: corr2 config_file file_name=file1.dat gg_file_name=file1.out corr2 config_file file_name=file2.dat gg_file_name=file2.out ... -This can be useful when running the program from a script for lots of input +This can be useful when running the program from a script for many input files. See `Using configuration files `_ @@ -271,15 +274,15 @@ for more details. Using the Python module ----------------------- -The typical usage in python is in three stages: +The typical usage in Python has three stages: 1. Define one or more Catalogs with the input data to be correlated. 2. Define the correlation function that you want to perform on those data. 3. Run the correlation by calling ``process``. 4. Maybe write the results to a file or use them in some way. -For instance, computing a shear-shear correlation from an input file stored -in a fits file would look something like the following:: +For instance, computing a shear-shear correlation from an input catalog stored +in a FITS file would look something like the following:: >>> import treecorr >>> cat = treecorr.Catalog('cat.fits', ra_col='RA', dec_col='DEC', @@ -299,7 +302,7 @@ Or for a more involved worked example, see our `Jupyter notebook tutorial `_. And for the complete details about all aspects of the code, see the `Sphinx-generated -documentation `_. +documentation `_. Reporting bugs @@ -321,3 +324,22 @@ issue and fill in the details of the feature you would like added to TreeCorr. Or if there is already an issue for your desired feature, please add to the discussion, describing your use case. The more people who say they want a feature, the more likely I am to get around to it sooner than later. + + +Since Jarvis, Bernstein, and Jain (2004) +---------------------------------------- + +The 2004 paper remains the foundational reference for TreeCorr's tree-based +correlation algorithms. Current TreeCorr includes additional capabilities that +were introduced later, including: + +* mixed-type three-point cross-correlations +* patch-based covariance estimates and cross-patch weighting options +* multipole three-point algorithm support (``bin_type='LogMultipole'`` and + fast ``'multipole'`` algorithm for ``'LogSAS'`` binning) +* expanded field support in two-point calculations (``Z``, ``V``, ``T``, ``Q``) + +References: + +* `Jarvis, Bernstein, and Jain (2004) `_ +* `Porth et al. (2023) `_ diff --git a/TreeCorr_LICENSE b/TreeCorr_LICENSE index 518a7672..aaf1c87c 100644 --- a/TreeCorr_LICENSE +++ b/TreeCorr_LICENSE @@ -10,7 +10,7 @@ modification, are permitted provided that the following conditions are met: this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. -This software is based in part upon work supported by the the University of +This software is based in part upon work supported by the University of Pennsylvania. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the University of Pennsylvania. @@ -25,4 +25,3 @@ software or any products related to or derived from the software, or for lost profits, business interruption, or indirect special or consequential damages of any kind. - diff --git a/docs/binning.rst b/docs/binning.rst index 4fa03f88..3ac55464 100644 --- a/docs/binning.rst +++ b/docs/binning.rst @@ -229,7 +229,7 @@ angle_slop For some calculations, the angular orientation of the line between two points is relevant to the correlation. For most complex quantities (e.g. shear), the value used in the correlation -has to be projected onto the the line between the two points (for two-point correlations) or +has to be projected onto the line between the two points (for two-point correlations) or the centroid of the triangle (for three-point correlations). Even for scalar quantities, three-point correlations using the multipole method also use the angular direction of the lines between points. diff --git a/docs/binning3pt.rst b/docs/binning3pt.rst index 4bc3cee3..169a2274 100644 --- a/docs/binning3pt.rst +++ b/docs/binning3pt.rst @@ -3,7 +3,7 @@ Binning for three-point correlations The binning in the three-point case is somewhat more complicated than for two-point functions, since we need to characterize the geometry of triangles. -There are currently three different binnings available, which +There are currently three different binning schemes available, which may be specified using the ``bin_type`` parameter in `Corr3`. .. note:: @@ -16,18 +16,18 @@ may be specified using the ``bin_type`` parameter in `Corr3`. this definition of vertices has the right field in the corresponding vertex. E.g. NNG only keeps triangles that have the G field in vertex 3. For triangles with the G field in vertex 1 or 2, you would need to use GNN and NGN respectively. - To fully characterize the full set of 3-point correlation information of the - three fields with mixed type, you need all three of these. + To fully characterize the 3-point correlation information for three mixed-type fields, + you need all three of these. See also `Other options for binning` for additional parameters that are relevant to the binning. These all work the same way for three-point functions as for -two-point function. +two-point functions. "LogRUV" -------- This binning option uses a Side-Side-Side (SSS) characterization of the triangle. -Thre three side lengths of the triangle are measured (using whatever `Metric ` +The three side lengths of the triangle are measured (using whatever `Metric ` is being used). Then we sort their lengths so that :math:`d_1 \ge d_2 \ge d_3`. If we just binned directly in these three side lengths, then the range of valid @@ -43,13 +43,13 @@ quantities which have better-behaved ranges: With this reparametrization, :math:`u` and :math:`v` are each limited to the range :math:`[0,1]`, independent of the values of the other parameters. The :math:`r` -parameter defines the overall size of the triangle, and that can range of whatever -set of values the user wants. +parameter defines the overall size of the triangle, and that can range over +whatever set of values the user wants. This provides a unique definition for any triangle, except for a mirror reflection. -Two congruent triangles (that are not isoceles or equilateral) are not necessarily -equivalent for 3-point correlations. The orienation of the sides matters, at least -in many use cases. So we need to keep track of that. We choose to do so in the +Two congruent triangles (that are not isosceles or equilateral) are not necessarily +equivalent for 3-point correlations. The orientation of the sides matters, at least +in many use cases. So we need to keep track of that. We choose to do so in the sign of :math:`v`, where positive values mean that the sides :math:`d_1`, :math:`d_2` and :math:`d_3` are oriented in counter-clockwise order. Negative values of :math:`v` mean they are oriented in clockwise order. @@ -58,11 +58,11 @@ Negative values of :math:`v` mean they are oriented in clockwise order. This binning can only use the 'triangle' algorithm, which is generally much slower than the 'multipole' algorithm. For most purposes, we recommend using - `"LogSAS"` instead, which can use the 'multpole' algorithm to calculate the + `"LogSAS"` instead, which can use the 'multipole' algorithm to calculate the correlation function. See `Three-point Algorithm` below for more discussion about this. -The binning of :math:`r` works the same was as `"Log"` for two-point correlations. +The binning of :math:`r` works the same way as `"Log"` for two-point correlations. That is, the binning is specified using any 3 of the following 4 parameters: - ``nbins`` How many bins to use. @@ -98,7 +98,7 @@ being used for the binning are called :math:`d_2` and :math:`d_3`. The angle between these two sides is called :math:`\phi`, and the side opposite it (not used for binning) is :math:`d_1`. -The two sides, :math:`d_2` and :math:`d_3` are each binned the same was as +The two sides, :math:`d_2` and :math:`d_3` are each binned the same way as `"Log"` binning for two-point correlations. That is, the binning is specified using any 3 of the following 4 parameters: @@ -176,7 +176,7 @@ section 4.2. (We no longer implement the algorithm described in section 4.3 due considerations.) This algorithm is much faster than a brute-force calculation, but it is still quite slow compared to the new multipole algorithm. -Starting in version 5.0, we now also implement the algorthm developed by +Starting in version 5.0, we now also implement the algorithm developed by `Porth et al (2024, A&A, 689, 224) `_, called 'multipole' in TreeCorr, which is much faster for typical data sets. diff --git a/docs/catalog.rst b/docs/catalog.rst index 7d5c0bb2..dd0da448 100644 --- a/docs/catalog.rst +++ b/docs/catalog.rst @@ -26,7 +26,8 @@ Other utilities related to catalogs treecorr.calculateVarQ .. automodule:: treecorr.catalog :members: - :exclude-members: Catalog, read_catalogs, calculateVarK, calculateVarZ, calculateVarV, calculateVarG, calculateVarT, calculateVatQ + :exclude-members: Catalog, read_catalogs, calculateVarK, calculateVarZ, calculateVarV, + calculateVarG, calculateVarT, calculateVarQ File Readers ------------ diff --git a/docs/conf.py b/docs/conf.py index f06a3485..7d75f9d3 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -39,8 +39,16 @@ 'sphinx.ext.viewcode', 'sphinx.ext.autosectionlabel', 'sphinx.ext.napoleon', + 'sphinx.ext.intersphinx', ] +intersphinx_mapping = { + 'coord': ('https://lsstdesc.org/Coord/_build/html/', None), + 'numpy': ('https://numpy.org/doc/stable/', None), + 'python': ('https://docs.python.org/3', None), + 'mpi4py': ('https://mpi4py.readthedocs.io/en/stable/', None), +} + # cf. http://stackoverflow.com/questions/2701998/sphinx-autodoc-is-not-automatic-enough #autosummay_generate = True @@ -130,7 +138,7 @@ def parse_class_attributes_section(self, section): return self._format_fields('Class Attributes', self._consume_fields()) GoogleDocstring._parse_class_attributes_section = parse_class_attributes_section -# we now patch the parse method to guarantee that the the above methods are +# we now patch the parse method to guarantee that the above methods are # assigned to the _section dict def patched_parse(self): self._sections['keys'] = self._parse_keys_section @@ -321,7 +329,7 @@ def patched_parse(self): # The scheme of the identifier. Typical schemes are ISBN or URL. #epub_scheme = '' -# The unique identifier of the text. This can be a ISBN number +# The unique identifier of the text. This can be an ISBN number # or the project homepage. #epub_identifier = '' @@ -338,7 +346,7 @@ def patched_parse(self): # The format is a list of tuples containing the path and title. #epub_pre_files = [] -# HTML files shat should be inserted after the pages created by sphinx. +# HTML files that should be inserted after the pages created by sphinx. # The format is a list of tuples containing the path and title. #epub_post_files = [] diff --git a/docs/correlation2.rst b/docs/correlation2.rst index 9e9dd5d2..e90f5d7f 100644 --- a/docs/correlation2.rst +++ b/docs/correlation2.rst @@ -31,8 +31,8 @@ to represent each kind of field as follows: Q is short for quatrefoil, a shape with spin-4 rotational properties. Not all possible pairings of two fields are currently implemented. The following lists -all the currently implemented classes for computing two-point correlations. If you have need -of a pairing not listed here, please file in issue asking for it. It's not hard to add more, +all the currently implemented classes for computing two-point correlations. If you need +a pairing not listed here, please file an issue asking for it. It's not hard to add more, but I didn't want to implement a bunch of classes that no one will use. .. toctree:: @@ -57,7 +57,7 @@ but I didn't want to implement a bunch of classes that no one will use. kq qq -Each of the above classes is a sub-class of the base class Corr2, so they have a number of +Each of the above classes is a subclass of the base class Corr2, so they have a number of features in common about how they are constructed. The common features are documented here. .. autoclass:: treecorr.Corr2 diff --git a/docs/correlation3.rst b/docs/correlation3.rst index 6ae2adfb..58bcf5d3 100644 --- a/docs/correlation3.rst +++ b/docs/correlation3.rst @@ -38,9 +38,8 @@ which field is on each vertex of the triangle. kkg kgg -Each of the above classes is a sub-class of the base class Corr3, so they have a number of +Each of the above classes is a subclass of the base class Corr3, so they have a number of features in common about how they are constructed. The common features are documented here. .. autoclass:: treecorr.Corr3 :members: - diff --git a/docs/cov.rst b/docs/cov.rst index 3a904733..cf7673f1 100644 --- a/docs/cov.rst +++ b/docs/cov.rst @@ -24,11 +24,24 @@ patches to estimate the overall sample variance. See `Patches` for information on defining the patches to use for your input `Catalog`. +.. admonition:: Recommendation + + For most use cases, the most accurate covariance estimate seems to be + jackknife covariances with matched cross-patch weights, so we recommend + this option for most users who want empirical covariance estimates from + the data. I.e. use ``method='jackknife', cross_patch_weight='match'``, + as described below. + + For the 5.x version series, the matched cross-patch weights are not the default + in order to maintain backwards-compatibility, but this may change at the + next major version (6.0). + + Variance Methods ---------------- To get one of the patch-based variance estimates for the ``varxi`` or similar -attribute, you can set the ``var_method`` parameter in the constructor. e.g.:: +attribute, you can set the ``var_method`` parameter in the constructor. For example:: >>> ng = treecorr.NGCorrelation(nbins=10, min_sep=1, max_sep=100, var_method='jackknife') @@ -60,7 +73,7 @@ from the sample. Then the covariance matrix is estimated as "sample" ^^^^^^^^ -This is the simplest patch-based covariance estimate estimate. It computes the +This is the simplest patch-based covariance estimate. It computes the correlation function for each patch, where at least one point falls in that patch. Then the estimated covariance matrix is simply the sample covariance of these vectors, scaled by the relative total weight in each patch. @@ -84,7 +97,7 @@ This estimate implements a bootstrap resampling of the patches as follows: will appear more than once, and some will be missing. 2. Calculate the total correlation function that would have been computed - from these patches rather than the original patches. + from these patches rather than from the original patches. 3. The auto-correlations are included at the selected repetition for the bootstrap samples. So if a patch number is repeated, its auto-correlation is included that @@ -137,7 +150,7 @@ Cross-patch Weights ------------------- There is some ambiguity as to the exact calculation of :math:`\xi` in each of the above -formulae, specifically with respect to the treatment of pairs (or triples for 3 point +formulae, specifically with respect to the treatment of pairs (or triples for 3-point statistics) that cross between a selected patch and an unselected patch. `Mohammad and Percival (2022; MP22 hereafter) `_ explored several different options @@ -152,8 +165,8 @@ which can be provided in the `Corr2` or `Corr3` constructor or in the call to which means the weight is the product of the two patch weights. For jackknife, the weights are all 1 or 0, so this means the pair is used only if both points are not in the excluded patch. For bootstrap, the weights are some - integer corresponding the multiplicity of that patch in the bootstrap selection. - Cross patch pairs are included at the product of the multipilicity of the two patches. + integer corresponding to the multiplicity of that patch in the bootstrap selection. + Cross patch pairs are included at the product of the multiplicity of the two patches. For sample and marked_bootstrap, a pair is included if the first point is the selected sample. * 'mean' involves weighting pairs by the mean of the patch weights. For jackknife, this @@ -172,7 +185,7 @@ which can be provided in the `Corr2` or `Corr3` constructor or in the call to (what they call mult) or 'mean'. The default value of ``cross_patch_weight`` is 'simple' for all variance methods. -MP22 recommends to instead use 'match' for jackknife covariances and 'geom' for +MP22 instead recommends using 'match' for jackknife covariances and 'geom' for bootstrap covariances. In order to maintain API consistency, we haven't made this the default yet, but we may in a future version of TreeCorr. @@ -180,10 +193,37 @@ For now, we recommend explicitly setting ``cross_patch_weight`` to either 'match' or 'geom' as appropriate, especially if your field has significant sample variance, but not much super-sample variance, where these options seem to be more optimal than the default weighting. +In many practical analyses, we find jackknife with ``cross_patch_weight='match'`` to be +the most accurate default choice, with bootstrap/``'geom'`` as a useful comparison. For 'sample' and 'marked_bootstrap', we don't see much difference between 'simple' and 'mean', -althought we welcome feedback from users whether 'mean' might be a better +although we welcome feedback from users whether 'mean' might be a better choice for these methods. +Recommended Baseline Workflow +----------------------------- + +For most analyses, a good baseline is jackknife covariance with ``'match'`` weighting: + +.. code-block:: python + + cat1 = treecorr.Catalog(cat1_file, npatch=50) + cat2 = treecorr.Catalog(cat2_file, patch_centers=cat1.patch_centers) + ng = treecorr.NGCorrelation(config) + ng.process(cat1, cat2) + cov = ng.estimate_cov(method='jackknife', cross_patch_weight='match') + +Patch consistency checklist for cross-correlations: + +1. Use a single shared patch definition across all relevant catalogs. +2. In particular, do not use ``npatch`` on multiple catalogs, since that will create different + patch definitions for the different data sets. +3. Typically use ``patch_centers`` to copy the patch definitions from one catalog to any other + catalogs being used in the correlation calculation. +4. This applies to random catalogs as well. If you are using random catalogs, use the same + patch definitions as the data. +5. Ensure any correlations combined with ``estimate_multi_cov`` were processed with compatible + patches. + Covariance Matrix ----------------- @@ -194,7 +234,7 @@ is complete. However, if the processing was done using patches, then you can also compute the covariance matrix for any of the above methods without redoing the processing -using `Corr2.estimate_cov` or `Corr3.estimate_cov`. E.g.:: +using `Corr2.estimate_cov` or `Corr3.estimate_cov`. For example:: >>> ng = treecorr.NGCorrelation(nbins=10, min_sep=1, max_sep=100) >>> ng.process(lens_cat, source_cat) # At least one of these needs to have patches set. @@ -202,7 +242,7 @@ using `Corr2.estimate_cov` or `Corr3.estimate_cov`. E.g.:: >>> cov_boot = ng.estimate_cov('bootstrap') Additionally, you can compute the joint covariance matrix for a number of statistics -that were processed using the same patches with `estimate_multi_cov`. E.g.:: +that were processed using the same patches with `estimate_multi_cov`. For example:: >>> ng = treecorr.NGCorrelation(nbins=10, min_sep=1, max_sep=100) >>> ng.process(lens_cat, source_cat) @@ -218,7 +258,7 @@ Covariance of Derived Quantities Sometimes your data vector of interest might not be just the raw correlation function, or even a list of several correlation functions. Rather, it might be some derived -quantity. E.g. +quantity. For example: * The ratio or difference of two correlation functions such as ``nk1.xi / nk2.xi``. * The aperture mass variance computed by `GGCorrelation.calculateMapSq`. @@ -226,8 +266,8 @@ quantity. E.g. * A reordering of the data vector, such as putting several ``gg.xip`` first for multiple tomographic bins and then the ``gg.xim`` for each after that. -These are just examples of what kind of thing you might want. In fact, we enable -any kind of post-processing you want to do on either a single correlation object +These are just examples of the kinds of things you might want. In fact, we enable +any kind of post-processing you might want to do on either a single correlation object (using `Corr2.estimate_cov` or `Corr3.estimate_cov`) or a list of correlation objects (using `estimate_multi_cov`). @@ -251,8 +291,8 @@ data vector, ``ratio``. Random Catalogs --------------- -There are a few adjustements to the above prescription when using random -catalogs, which of course are required when doing an NN correlation. +There are a few adjustments to the above prescription when using random +catalogs, which are of course required when doing an NN correlation. 1. It is not necessarily required to use patches for the random catalog. The random is supposed to be dense enough that it doesn't materially contribute @@ -273,7 +313,7 @@ catalogs, which of course are required when doing an NN correlation. will have ``varxi`` and ``cov`` attributes calculated according to whatever ``var_method`` you specified. 5. It also allows you to call `dd.estimate_cov ` - with any different method you want. + with any method you want. And you can include ``dd`` in a list of correlation objects passed to `estimate_multi_cov`. @@ -293,11 +333,12 @@ Here is a worked example:: >>> tx_cov = treecorr.estimate_multi_cov([ng,gg,dd], 'bootstrap') # Or include in multi_cov As mentioned above, using ``patch_centers`` is optional for ``rand``, but probably recommended. -In the last line, it would be required that ``ng`` and ``gg`` were also made using catalogs +In the last line, it is required that ``ng`` and ``gg`` were also made using catalogs with the same patch centers that ``dd`` used. -The use pattern for `NNNCorrelation` is analogous, where `calculateZeta ` -needs to be run to get the covariance estimate, after which it may be used in a list +The usage pattern for `NNNCorrelation` is analogous, where +`calculateZeta ` needs to be run to get the covariance estimate, +after which it may be used in a list passed to `estimate_multi_cov`. Design Matrix @@ -309,7 +350,7 @@ described `above `. This matrix is available using a parallel pair of functions to `estimate_cov ` and `estimate_multi_cov`. Namely `build_cov_design_matrix ` -and `build_multi_cov_design_matrix`. E.g.:: +and `build_multi_cov_design_matrix`. For example:: >>> A_ng_jk, w = ng.build_cov_design_matrix(method='jackknife') >>> A_tx_bs, w = treecorr.build_multi_cov_design_matrix([ng,gg,dd], method='bootstrap') diff --git a/docs/field.rst b/docs/field.rst index f1acd6a6..f1a433c6 100644 --- a/docs/field.rst +++ b/docs/field.rst @@ -2,13 +2,13 @@ Fields ====== The `Field` class and its subclasses repackage the information from a `Catalog` -into a ball tree data structure, allowing for fast calcaulation of the correlation +into a ball tree data structure, allowing for fast calculation of the correlation functions. There are several kinds of `Field` classes. - - `Field` itself is an abstract base class of the other kinds of fields, and has a - few methods that are available for all `Field` types. + - `Field` itself is an abstract base class for the other field types and has a + few methods available for all `Field` types. - `NField` holds counts of objects and is used for correlations with an N in the name, including `NNCorrelation`, `NGCorrelation`, `NKCorrelation`, and `NNNCorrelation`. @@ -25,15 +25,16 @@ There are several kinds of `Field` classes. It is used for correlations with a G in the name, including `GGCorrelation`, `NGCorrelation`, `KGCorrelation`, and `GGGCorrelation`. - `TField` holds both counts of objects and the mean trefoil field of those objects. - It is used for correlations with a V in the name, including - `VVCorrelation`, `NVCorrelation`, and `KVCorrelation`. + It is used for correlations with a T in the name, including + `TTCorrelation`, `NTCorrelation`, and `KTCorrelation`. - `QField` holds both counts of objects and the mean quatrefoil field of those objects. - It is used for correlations with a V in the name, including - `VVCorrelation`, `NVCorrelation`, and `KVCorrelation`. + It is used for correlations with a Q in the name, including + `QQCorrelation`, `NQCorrelation`, and `KQCorrelation`. Typically, one would not create any of these objects directly, but would instead -use Catalog methods `getNField`, `getKField`, `getGField`, `getVField`. -Or indeed, usually, one does not even do that, and just lets the relevant ``process`` +use Catalog methods such as `getNField`, `getKField`, `getZField`, `getVField`, +`getGField`, `getTField`, or `getQField`. +Usually, one does not even do that, and instead lets the relevant ``process`` command do so for you. .. autoclass:: treecorr.Field diff --git a/docs/guide.rst b/docs/guide.rst index c0e8628a..ed67c2b3 100644 --- a/docs/guide.rst +++ b/docs/guide.rst @@ -4,17 +4,64 @@ Getting Started Guide Jupyter Tutorial ^^^^^^^^^^^^^^^^ -The below page covers many of the same points as the +This page covers many of the same points as the `Jupyter notebook tutorial `_ available in the TreeCorr repo. You may find it useful to work through that as well as, or instead of, reading this guide. + +Choosing a two-point correlation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Choose based on the field types: + +* ``N`` = counts (discrete objects) +* ``K`` = real scalar field +* ``G`` = shear (spin-2 complex field) +* ``Z`` = complex spin-0 field +* ``V`` = vector (complex spin-1 field) +* ``T`` = trefoil (complex spin-3 field) +* ``Q`` = quatrefoil (complex spin-4 field) + +Common cases: + +* ``NN``: count-count clustering (`NNCorrelation`) +* ``NG``: count-shear (galaxy-galaxy lensing) (`NGCorrelation`) +* ``GG``: shear-shear (`GGCorrelation`) +* ``NK`` / ``KK`` / ``KG``: scalar-based correlations + +See :doc:`correlation2` for all supported two-point classes. + +Choosing a three-point correlation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Three-point classes are available for: + +* Auto-correlations: ``NNN``, ``KKK``, ``GGG`` +* Mixed cross-correlations: ``NNK``, ``NNG``, ``NKK``, ``NGG``, ``KKG``, ``KGG`` + +Each mixed family also includes the corresponding permutations (e.g. +``NNG``, ``NGN``, ``GNN``). + +See :doc:`correlation3` for details and class-level docs. + +When random catalogs are required +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +* ``NN`` and ``NNN`` require random catalogs for unbiased estimators. +* ``NG`` and ``NK`` can also use random catalogs for compensated estimators. + +See :doc:`params` for the relevant random-catalog configuration parameters. + Shear-shear auto-correlation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Let's start with how to calculate a shear-shear two-point auto-correlation. +Let's start with how to calculate a shear-shear two-point auto-correlation as a +typical concrete example. It's not necessarily the simplest choice of correlation, but this specific calculation was the original reason I wrote TreeCorr, so it's close to my heart. +The usage pattern is similar for other combinations of fields. + The basic pattern is as follows:: cat = treecorr.Catalog(file_name, config) @@ -33,7 +80,7 @@ You can do a cross-correlation between two sets of galaxies very similarly:: cat2 = treecorr.Catalog(file_name2, config2) gg.process(cat1, cat2) -If you would rather not write the results to an output file, but maybe plot them up or do some +If you would rather not write the results to an output file, but instead plot them or do some further calculation with them, you can access the resulting fields directly as numpy arrays:: xip = gg.xip # The real part of xi+ @@ -43,34 +90,9 @@ further calculation with them, you can access the resulting fields directly as n varxi = gg.varxi # The variance of each xi+ or xi- value # taking into account shape noise only -See the doc string for `GGCorrelation` for other available attributes. - -Other Two-point Correlation Classes -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The other kinds of correlations each have their own class: - - - `NNCorrelation` = count-count (normal LSS correlation) - - `NKCorrelation` = count-scalar (i.e. (R), where kappa is any scalar field) - - `KKCorrelation` = scalar-scalar - - `NGCorrelation` = count-shear (i.e. (R)) - - `KGCorrelation` = scalar-shear - - `GGCorrelation` = shear-shear (e.g. cosmic shear) - - `NVCorrelation` = count-vector - - `KVCorrelation` = scalar-vector - - `VVCorrelation` = vector-vector - -You should see their doc strings for details, but they all work similarly. -For the cross-type classes (e.g. NK, KG, etc.), there is no auto-correlation option, -of course, just the cross-correlation. - -The other main difference between these other correlation classes from GG is that there is only a -single correlation function, so it is called ``xi`` rather than ``xip`` and ``xim``. - -Also, NN does not have any kind of ``xi`` attribute. You need to perform an additional -calculation involving random catalogs for that. -See `Using random catalogs` below for more details. - +See the docstring for `GGCorrelation` for other available attributes. +If you want to run this same workflow from a config file via ``corr2``, +or compare executable and Python interfaces, see :doc:`scripts`. Loading a Catalog ^^^^^^^^^^^^^^^^^ @@ -95,7 +117,7 @@ in the FITS table. For ASCII input files, you specify the column number instead cat = treecorr.Catalog(file_name='input_cat.dat', x_col=2, y_col=3, g1_col=5, g2_col=6) -where the first column in numbered 1, not 0. +where the first column is numbered 1, not 0. When the positions are given as right ascension and declination on the celestial sphere, rather than x and y on a flat projection (like an image), you also need @@ -105,8 +127,9 @@ to specify what units the angles use:: ra_col='RA', dec_col='DEC', g1_col='E1', g2_col='E2', ra_units='hours', dec_units='degrees') -For the catalog of the N part of a calculation, you can skip the ``g1_col`` and ``g2_col``. -Those only need positions. For a K correlation, you should specify ``k_col`` instead:: +For catalogs corresponding to the N part of a calculation, you can skip ``g1_col`` and +``g2_col``; those catalogs only need positions. For a K correlation, specify ``k_col`` +instead:: cat = treecorr.Catalog(file_name='input_cat.fits', ra_col='RA', dec_col='DEC', k_col='KAPPA', @@ -120,9 +143,9 @@ skip objects with specific flags, and more. Building a Catalog from numpy arrays ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -If the provided tools for reading in the data from an input file are insufficient, or if -the data you want to use are being generated in Python natively, so there is no file -to read, then you can instead build the `Catalog` directly from numpy arrays:: +If the provided tools for reading data from an input file are insufficient, or if +the data are generated directly in Python so there is no file to read, then you can +instead build the `Catalog` directly from numpy arrays:: x = numpy.array(x_values) # These might be the output of y = numpy.array(y_values) # some calculation... @@ -139,7 +162,7 @@ from the data. For GG, you need ``g1`` and ``g2``, but for K correlations, you You can optionally provide a weight column as well with ``w`` if desired. This will then perform a weighted correlation using those weights. -Again, see the doc string for `Catalog` for more information. +Again, see the docstring for `Catalog` for more information. Defining the binning @@ -175,38 +198,41 @@ Finally, the default way of calculating separations is a normal Euclidean metric However, TreeCorr implements a number of other metrics as well, which are useful in various situations. See `Metrics` for details. +Other Two-point Correlation Classes +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The other kinds of correlations each have their own class. For example: + + - `NNCorrelation` = count-count (normal LSS correlation) + - `NKCorrelation` = count-scalar (i.e. (R), where kappa is any scalar field) + - `KKCorrelation` = scalar-scalar + - `NGCorrelation` = count-shear (i.e. (R)) + - `NVCorrelation` = count-vector + - `VVCorrelation` = vector-vector + +See `Two-point Correlation Functions` for the complete list including other spin varieties. + +You should see their docstrings for details, but they all work similarly. +For the cross-type classes (e.g. NK, KG, etc.), there is no auto-correlation option, +of course, just the cross-correlation. + +The other main difference between these other correlation classes from GG is that there is only a +single correlation function, so it is called ``xi`` rather than ``xip`` and ``xim``. + +Also, NN does not have any kind of ``xi`` attribute. You need to perform an additional +calculation involving random catalogs for that. +See `Using random catalogs` below for more details. + + Three-point Correlation Classes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TreeCorr can also do three-point correlations, to measure how the product of three fields depends on the size and shape of the triangle connecting three points. -So far, we have implemented the following combination: - - - `NNNCorrelation` # count-count-count - - `KKKCorrelation` # scalar-scalar-scalar - - `GGGCorrelation` # shear-shear-shear - - `NNKCorrelation` # count-count-scalar - - `NKNCorrelation` # count-scalar-count - - `KNNCorrelation` # scalar-count-count - - `NNGCorrelation` # count-count-shear - - `NGNCorrelation` # count-shear-count - - `GNNCorrelation` # shear-count-count - - `NKKCorrelation` # count-scalar-scalar - - `KNKCorrelation` # scalar-count-scalar - - `KKNCorrelation` # scalar-scalar-count - - `NGGCorrelation` # count-shear-shear - - `GNGCorrelation` # shear-count-shear - - `GGNCorrelation` # shear-shear-count - - `KKGCorrelation` # scalar-scalar-shear - - `KGKCorrelation` # scalar-shear-scalar - - `GKKCorrelation` # shear-scalar-scalar - - `KGGCorrelation` # scalar-shear-shear - - `GKGCorrelation` # shear-scalar-shear - - `GGKCorrelation` # shear-shear-scalar These classes are significantly more complicated than the two-point ones, since they have to deal with the geometry of the triangles being binned. -See their doc strings for more details. +See `Three-point Correlation Functions` for details. Using random catalogs @@ -214,7 +240,7 @@ Using random catalogs For the NN and NNN correlations, the raw calculation is not sufficient to produce the real correlation function. You also need to account for the survey geometry (edges, mask, etc.) -by running the same calculation with a random catalog (or several) that have a uniform density, +by running the same calculation with one or more random catalogs that have a uniform density, but the same geometry:: data = treecorr.Catalog(data_file, config) @@ -225,7 +251,7 @@ but the same geometry:: dd.process(data) dr.process(data,rand) rr.process(rand) - xi, varxi = dd.calculateXi(rr,dr) + xi, varxi = dd.calculateXi(rr=rr, dr=dr) This calculates xi = (DD-2DR+RR)/RR for each bin. This is the Landy-Szalay estimator, @@ -247,7 +273,8 @@ of the correlation function if you are able to use a random catalog. Furthermore, the `process ` functions can take lists of Catalogs if desired, in which case it will do all the possible combinations. This is especially relevant for doing randoms, -since the statistics get better if you generate several randoms and do all the correlations to beat down the noise:: +since the statistics get better if you generate several randoms and do all the correlations to +beat down the noise:: rand_list = [ treecorr.Catalog(f,config) for f in rand_files ] dr.process(data, rand_list) @@ -258,6 +285,49 @@ combinations that need to be computed: zeta = (DDD-DDR-DRD-RDD+DRR+RDR+RRD-RRR)/ Because of the triangle geometry, we don't have DRR = DRD = RDD, so all 8 need to be computed. See the docstring for `calculateZeta ` for more details. +Performance and accuracy tips +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Some practical recommendations that are often useful in production analyses: + +* Start with the default ``bin_slop``. If you need to verify numerical stability, + reduce it (e.g. by a factor of 2) and check whether your science vector shifts. +* For three-point calculations, prefer ``bin_type='LogSAS'`` with the default + multipole algorithm for speed, unless you specifically need ``LogRUV``. +* Use patches early if you will need covariance estimates. This avoids re-running + the full correlation later to get jackknife/bootstrap covariances. +* For patch-based covariance, use jackknife with ``cross_patch_weight='match'`` for the + best accuracy in most cases; use bootstrap/``'geom'`` mainly as a cross-check. + + .. important:: + + This is not currently the default. To avoid backwards incompatibility, + you must set ``cross_patch_weight='match'`` explicitly. The default value, + ``'simple'``, is probably less accurate in most cases, but matches the behavior + of TreeCorr prior to version 5.1. + +* For large jobs, use ``low_mem=True`` in ``process`` calls and use patches + to reduce peak memory usage. +* For OpenMP runs, set ``num_threads`` explicitly in configs when running on shared + systems, so results are reproducible and resource usage is controlled. + +See `Binning`, `Binning for three-point correlations`, `Patches`, and +`Covariance Estimates` for full details. + +Common pitfalls +^^^^^^^^^^^^^^^ + +Some common issues that can silently cause errors in the correlations: + +* Mixed coordinate conventions: ensure ``ra_units``/``dec_units`` are set correctly, + and check sign conventions (``flip_g1``/``flip_g2`` when needed). +* Missing or mismatched random catalogs for ``NN``/``NNN``: random catalogs should + follow the same survey geometry/masks as the corresponding data catalogs. +* Inconsistent patch definitions across cross-correlated catalogs: use shared + ``patch_centers`` rather than separate ``npatch`` runs for each catalog. +* Interpreting ``varxi`` as total uncertainty by default: shot-noise-only variances + can underestimate errors at large scales; prefer patch-based covariances when possible. + Manually accumulating the correlation function ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -265,7 +335,8 @@ For even more control over the calculation, you can break up the steps in the `process ` functions. There are typically three steps: 1. Calculate the variance of the field as needed (i.e. for anything but NN correlations). -2. Accumulate the correlations into the bins for each auto-correlation and cross-correlation desired. +2. Accumulate the correlations into the bins for each auto-correlation and cross-correlation + desired. 3. Finalize the calculation. If you have several pairs of catalogs that you want to accumulate into a single correlation @@ -282,7 +353,7 @@ function, you could write the following:: In addition to `process_cross `, classes that allow auto-correlations have a `process_auto ` method for manually processing -auto-correlations. See the doc strings for these methods for more information. +auto-correlations. See the docstrings for these methods for more information. Breaking up the calculation manually like this is probably not often necessary anymore. It used to be useful for dividing a calculation among several machines, which would diff --git a/docs/history.rst b/docs/history.rst index 058bee52..9cff3352 100644 --- a/docs/history.rst +++ b/docs/history.rst @@ -2,29 +2,125 @@ Previous History ================ -`Changes from version 4.3 to 5.0 -`_ +Changes from version 5.0 to 5.1 +-------------------------------- -`Changes from version 4.2 to 4.3 -`_ +User-impact highlights: -`Changes from version 4.1 to 4.2 -`_ +* Expanded three-point support to include mixed field-type classes + (e.g. NNG, NKK, KGG and permutations). +* Added three-point support for ``Rlens``/``Rperp`` and ``min_rpar``/``max_rpar``. +* Added improved covariance weighting options via ``cross_patch_weight``; ``'match'`` + (jackknife) and ``'geom'`` (bootstrap) are now recommended, with ``'simple'`` retained + for backward compatibility. -`Changes from version 4.0 to 4.1 -`_ +`Full changelog `__ -`Changes from version 3.3 to 4.0 -`_ +Changes from version 4.3 to 5.0 +-------------------------------- -`Changes from version 3.2 to 3.3 -`_ +User-impact highlights: -`Changes from version 3.1 to 3.2 -`_ +* Major three-point workflow update: default ``bin_type='LogSAS'`` and + ``algo='multipole'`` for large speed gains. +* Redesigned three-point cross-correlation handling around ``ordered`` in ``Corr3.process``, + replacing older CrossCorrelation classes. +* Added new two-point spin-field families (Z, V, T, Q). +* Switched C++ bindings from cffi to pybind11 and removed accumulated 4.x deprecations. -`Changes from version 3.0 to 3.1 -`_ +`Full changelog `__ -`Changes from version 2.6 to 3.0 -`_ +Changes from version 4.2 to 4.3 +-------------------------------- + +User-impact highlights: + +* Added covariance design-matrix helpers and improved patch-result I/O. +* Began transition of many function parameters to keyword-only usage + (positional forms still worked but were deprecated). + +`Full changelog `__ + +Changes from version 4.1 to 4.2 +-------------------------------- + +User-impact highlights: + +* Extended patch-based workflows to three-point correlations. +* Improved three-point cross-correlation behavior for NNN/KKK/GGG. +* Added covariance estimation of arbitrary derived data vectors via the ``func`` argument. +* Added additional catalog I/O options, including HDF5 and Parquet. + +`Full changelog `__ + +Changes from version 4.0 to 4.1 +-------------------------------- + +User-impact highlights: + +* Introduced modern patch-based covariance estimation + (jackknife/sample/bootstrap/marked bootstrap). +* Added practical patch tooling: k-means patching, patch-center files, and ``patch_col``. +* Added large-scale workflow features including ``low_mem``, ``save_patch_dir``, and MPI. + +`Full changelog `__ + +Changes from version 3.3 to 4.0 +-------------------------------- + +User-impact highlights: + +* Major API shift with new linear and TwoD binning options. +* Added/updated metrics (including Periodic and Fisher-style ``Rperp`` behavior). +* Improved performance in difficult ``bin_slop`` regimes. +* Updated output naming conventions and variance attributes, and moved coordinate internals + to LSSTDESC.Coord. + +`Full changelog `__ + +Changes from version 3.2 to 3.3 +-------------------------------- + +User-impact highlights: + +* Added YAML/JSON config support and new Arc/Rlens metric options. +* Updated separation-unit behavior to be less confusing in outputs. +* Changed C++ wrapping from ctypes to cffi and updated FITS I/O expectations around fitsio. + +`Full changelog `__ + +Changes from version 3.1 to 3.2 +-------------------------------- + +User-impact highlights: + +* Introduced three-point correlations (NNN/KKK/GGG). +* Added ``Rperp`` support. +* Added FITS I/O support for correlation objects. +* Improved split-and-accumulate workflows by allowing objects to be read, written, and combined. + +`Full changelog `__ + +Changes from version 3.0 to 3.1 +-------------------------------- + +User-impact highlights: + +* Renamed G2/NN/K2-style class and config names to GG/NN/KK naming. +* Transitioned documentation to Sphinx-hosted docs. +* Added stability fixes around file lists, logging verbosity, and compiler/OpenMP detection. + +`Full changelog `__ + +Changes from version 2.6 to 3.0 +-------------------------------- + +User-impact highlights: + +* Major architectural overhaul that introduced a first-class Python-module interface. +* Enabled direct Catalog construction from arrays and in-memory access to computed vectors. +* Enabled more flexible custom workflows than executable-only usage. +* Simplified legacy corr2 options and clarified ``file_name``/``file_name2`` cross-correlation + semantics. + +`Full changelog `__ diff --git a/docs/index.rst b/docs/index.rst index d253405e..e2b5d904 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -7,6 +7,7 @@ TreeCorr Documentation :maxdepth: 2 overview + guide catalog correlation2 correlation3 @@ -18,11 +19,9 @@ TreeCorr Documentation cov field scripts - guide changes history * :ref:`genindex` * :ref:`modindex` * :ref:`search` - diff --git a/docs/metric.rst b/docs/metric.rst index 00f780d3..800fadd0 100644 --- a/docs/metric.rst +++ b/docs/metric.rst @@ -5,12 +5,12 @@ Metrics The correlation functions need to know how to calculate distances between the points, that is, the metric defining the space. -In most cases, you will probably want to use the default Metric, called "Euclidean", -which just uses the normal Euclidean distance between two points. However, there are a few +In most cases, you will probably want to use the default metric, called "Euclidean", +which uses the normal Euclidean distance between two points. However, there are a few other options, which are useful for various applications. -Both `Corr2` and `Corr3` take an optional -``metric`` parameter, which should be one of the following string values: +Both `Corr2` and `Corr3` take an optional ``metric`` parameter, which should be one +of the following string values: "Euclidean" @@ -83,7 +83,7 @@ The distance in this metric is defined as :math:`d_{\rm Rperp} = \sqrt{d_{\rm Euclidean}^2 - r_\parallel^2}` -where :math:`r_\parallel` follows the defintion in Fisher et al, 1994 (MNRAS, 267, 927). +where :math:`r_\parallel` follows the definition in Fisher et al, 1994 (MNRAS, 267, 927). Namely, if :math:`p_1` and :math:`p_2` are the vector positions from Earth for the two points, and @@ -113,7 +113,7 @@ In this limit, the formula for :math:`d` reduces to Prior to version 4.0, the "Rperp" name meant what is now called "OldRperp". The difference can be significant for some use cases, so if consistency across - versions is importatnt to you, you should either switch to using "OldRperp" + versions is important to you, you should either switch to using "OldRperp" or investigate whether the change to "FisherRperp" is important for your particular science case. @@ -129,7 +129,7 @@ used by Fisher et al, 1994 (MNRAS, 267, 927). The difference turns out to be non-trivial in some realistic use cases, so we preserve the ability to use the old version with this metric. -Specifically, if :math:`r_1` and :math:`r_2` are the two distance from Earth, +Specifically, if :math:`r_1` and :math:`r_2` are the two distances from Earth, then this metric uses :math:`r_\parallel \equiv r_2-r_1`. The distance is then defined as @@ -196,7 +196,7 @@ The distance is defined as Of course, for 2-dimensional coordinate systems, :math:`dz = 0`. -This metric is particularly relevant for data generated from N-body simuluations, which +This metric is particularly relevant for data generated from N-body simulations, which often use periodic boundary conditions. diff --git a/docs/params.rst b/docs/params.rst index ca7bff3a..a7fa5382 100644 --- a/docs/params.rst +++ b/docs/params.rst @@ -69,8 +69,9 @@ Parameters about the input file(s) and ``file_name`` (or any of the other corresponding pairs). :file_type: (ASCII, FITS, HDF5, or Parquet) The file type of the input files. -:delimiter: (str, default = '\0') The delimeter between input values in an ASCII catalog. -:comment_marker: (str, default = '#') The first (non-whitespace) character of comment lines in an input ASCII catalog. +:delimiter: (str, default = '\0') The delimiter between input values in an ASCII catalog. +:comment_marker: (str, default = '#') The first (non-whitespace) character of comment lines in an + input ASCII catalog. The default file type is normally ASCII. However, if the file name includes ".fit" in it, then a fits binary table is assumed. @@ -81,9 +82,10 @@ Parameters about the input file(s) comment lines usually begin with '#', but you may specify something different if necessary. -:ext: (int/str, default=1 for FITS or root for HDF5) The extension (fits) or group (hdf) to read from +:ext: (int/str, default=1 for FITS or root for HDF5) The extension (fits) or group (hdf) to read + from - Normally if you are using a fits file, the binary fits table is + Normally if you are using a FITS file, the binary FITS table is taken from the first extension, HDU 1. If you want to read from a different HDU, you can specify which one to use here. For HDF files, the default is to read from the root of the file, but you can also @@ -96,7 +98,7 @@ Parameters about the input file(s) You can optionally not use all the rows in the input file. You may specify ``first_row``, ``last_row``, or both to limit the rows being used. The rows are numbered starting with 1. If ``last_row`` is not positive, it - means to use to the end of the file. If ``every_nth`` is set, it will skip + means to read through the end of the file. If ``every_nth`` is set, it will skip rows, selecting only 1 out of every n rows. :npatch: (int, default=1) @@ -123,6 +125,12 @@ Parameters about the input file(s) may instead give patch_centers either as a file name or an array from which the patches will be determined. + For cross-correlations where both catalogs use patches, this is often the + preferred way to ensure consistent patch geometry between the two catalogs. + Running separate ``npatch`` calculations for each catalog is incorrect for + patch-based covariance estimation, since it produces inconsistent patch + definitions between the catalogs. + :x_col: (int/str) Which column to use for x. :y_col: (int/str) Which column to use for y. :ra_col: (int/str) Which column to use for ra. @@ -177,8 +185,8 @@ Parameters about the input file(s) If you are doing one of the shear correlation functions (i.e. NG, KG, GG), then you need to specify the shear estimates of the corresponding galaxies. The g1,g2 values are taken to be reduced shear values. They should be - unbiases estimators of g1,g2, so they are allowed to exceed :math:`|g| = 1`. - (This is required for some methods to produce unbiased estimates. + unbiased estimators of g1,g2, so they are allowed to exceed :math:`|g| = 1`. + (This is required for some methods to produce unbiased estimates.) :v1_col: (int/str) Which column to use for v1 (the real component of the vectors). :v2_col: (int/str) Which column to use for v2 (the imaginary component of the vectors). @@ -263,7 +271,7 @@ Parameters about the input file(s) set any of the above items from ``file_type`` to ``flip_g2`` as a two element list (i.e. two values separated by a space). In this case, the first item refers to the file(s) in ``file_name``, and the second item refers - to the file(s) in files_name2. + to the file(s) in ``file_name2``. - You may not mix (x,y) columns with (ra,dec) columns, since its meaning would be ambiguous. @@ -273,7 +281,7 @@ Parameters about the input file(s) any format of input catalog. - Also, if the given column only applies to one of the two input files - (e.g. k_col for an count-scalar cross-correlation) then you may specify just + (e.g. k_col for a count-scalar cross-correlation) then you may specify just the column name or number for the file to which it does apply. @@ -297,6 +305,13 @@ Parameters about the binned correlation function to be calculated See `Binning` for details about how these parameters are used for the different choice of ``bin_type``. +:max_n: (int) The maximum multipole index to store for three-point multipole calculations. + + This parameter is required when ``bin_type='LogMultipole'``. + It may also be provided with ``bin_type='LogSAS'`` when using the multipole + algorithm path, in which case it controls the internal multipole expansion + before conversion back to SAS bins. + :sep_units: (str, default=None) The units to use for ``min_sep`` and ``max_sep``. ``sep_units`` is also the units of R in the output file. For ra, dec values, @@ -307,7 +322,8 @@ Parameters about the binned correlation function to be calculated See `sep_units` for more discussion about this parameter. -:bin_slop: (float, default=1) The fraction of a bin width by which it is ok to let the pairs miss the correct bin. +:bin_slop: (float, default=1) The fraction of a bin width by which it is ok to let the pairs miss + the correct bin. The code normally determines when to stop traversing the tree when all of the distance pairs for the two nodes have a spread in distance that is less than the @@ -378,13 +394,16 @@ about the output columns. - ``sigma_xi`` = The 1-sigma error bar for xi. - ``DD``, ``RR`` = The raw numbers of pairs for the data and randoms - ``DR`` (if ``nn_statistic=compensated``) = The cross terms between data and random. - - ``RD`` (if ``nn_statistic=compensated`` cross-correlation) = The cross term between random and data, which for a cross-correlation is not equivalent to ``DR``. + - ``RD`` (if ``nn_statistic=compensated`` cross-correlation) = The cross term between random + and data, which for a cross-correlation is not equivalent to ``DR``. -:nn_statistic: (str, default='compensated') Which statistic to use for estimator of the NN correlation function. +:nn_statistic: (str, default='compensated') Which statistic to use for estimator of the NN + correlation function. Options are (D = data catalog, R = random catalog) - - 'compensated' is the now-normal Landy-Szalay statistic: xi = (DD-2DR+RR)/RR, or for cross-correlations, xi = (DD-DR-RD+RR)/RR + - 'compensated' is the now-normal Landy-Szalay statistic: xi = (DD-2DR+RR)/RR, or for + cross-correlations, xi = (DD-DR-RD+RR)/RR - 'simple' is the older version: xi = (DD/RR - 1) :nk_file_name: (str) The output filename for count-scalar correlation function. @@ -402,7 +421,8 @@ about the output columns. - ``weight`` = The total weight of the pairs in each bin. - ``npairs`` = The total number of pairs in each bin. -:nk_statistic: (str, default='compensated' if ``rand_files`` is given, otherwise 'simple') Which statistic to use for the estimator of the NK correlation function. +:nk_statistic: (str, default='compensated' if ``rand_files`` is given, otherwise 'simple') Which + statistic to use for the estimator of the NK correlation function. Options are: @@ -410,7 +430,8 @@ about the output columns. Define: - NK = Sum(kappa around data points) - - RK = Sum(kappa around random points), scaled to be equivalent in effective number as the number of pairs in NK. + - RK = Sum(kappa around random points), scaled to be equivalent in effective number as the + number of pairs in NK. - npairs = number of pairs in NK. Then this statistic is ```` = (NK-RK)/npairs @@ -441,7 +462,8 @@ about the output columns. - ``weight`` = The total weight of the pairs in each bin. - ``npairs`` = The total number of pairs in each bin. -:nz_statistic: (str, default='compensated' if ``rand_files`` is given, otherwise 'simple') Which statistic to use for the estimator of the NZ correlation function. +:nz_statistic: (str, default='compensated' if ``rand_files`` is given, otherwise 'simple') Which + statistic to use for the estimator of the NZ correlation function. Options are: @@ -449,7 +471,8 @@ about the output columns. Define: - NZ = Sum(z around data points) - - RZ = Sum(z around random points), scaled to be equivalent in effective number as the number of pairs in NZ. + - RZ = Sum(z around random points), scaled to be equivalent in effective number as the number + of pairs in NZ. - npairs = number of pairs in NZ. Then this statistic is = (NZ-RZ)/npairs @@ -503,7 +526,8 @@ about the output columns. - ``weight`` = The total weight of the pairs in each bin. - ``npairs`` = The total number of pairs in each bin. -:nv_statistic: (str, default='compensated' if ``rand_files`` is given, otherwise 'simple') Which statistic to use for the estimator of the NV correlation function. +:nv_statistic: (str, default='compensated' if ``rand_files`` is given, otherwise 'simple') Which + statistic to use for the estimator of the NV correlation function. Options are: @@ -511,7 +535,8 @@ about the output columns. Define: - NV = Sum(v around data points) - - RV = Sum(v around random points), scaled to be equivalent in effective number as the number of pairs in NG. + - RV = Sum(v around random points), scaled to be equivalent in effective number as the number + of pairs in NG. - npairs = number of pairs in NV. Then this statistic is vR = (NV-RV)/npairs @@ -537,8 +562,10 @@ about the output columns. - ``R_nom`` = The center of the bin - ``meanR`` = The mean separation of the points that went into the bin. - ``meanlogR`` = The mean log(R) of the points that went into the bin. - - ``xip`` = where v1 and v2 are measured with respect to the line joining the two points, where p1 is on the left and p2 is on the right. - - ``xim`` = where v1 and v2 are measured with respect to the line joining the two points, where p1 is on the left and p2 is on the right. + - ``xip`` = where v1 and v2 are measured with respect to the line joining the + two points, where p1 is on the left and p2 is on the right. + - ``xim`` = where v1 and v2 are measured with respect to the line joining the + two points, where p1 is on the left and p2 is on the right. - ``xip_im`` = . In the formulation of xi+ using complex numbers, this is the imaginary component. @@ -573,7 +600,8 @@ about the output columns. - ``weight`` = The total weight of the pairs in each bin. - ``npairs`` = The total number of pairs in each bin. -:ng_statistic: (str, default='compensated' if ``rand_files`` is given, otherwise 'simple') Which statistic to use for the estimator of the NG correlation function. +:ng_statistic: (str, default='compensated' if ``rand_files`` is given, otherwise 'simple') Which + statistic to use for the estimator of the NG correlation function. Options are: @@ -581,7 +609,8 @@ about the output columns. Define: - NG = Sum(gamma around data points) - - RG = Sum(gamma around random points), scaled to be equivalent in effective number as the number of pairs in NG. + - RG = Sum(gamma around random points), scaled to be equivalent in effective number as the + number of pairs in NG. - npairs = number of pairs in NG. Then this statistic is gamT = (NG-RG)/npairs @@ -613,8 +642,10 @@ about the output columns. - ``R_nom`` = The center of the bin - ``meanR`` = The mean separation of the points that went into the bin. - ``meanlogR`` = The mean log(R) of the points that went into the bin. - - ``xip`` = where g1 and g2 are measured with respect to the line joining the two galaxies. - - ``xim`` = where g1 and g2 are measured with respect to the line joining the two galaxies. + - ``xip`` = where g1 and g2 are measured with respect to the line joining the + two galaxies. + - ``xim`` = where g1 and g2 are measured with respect to the line joining the + two galaxies. - ``xip_im`` = . In the formulation of xi+ using complex numbers, this is the imaginary component. @@ -646,7 +677,8 @@ about the output columns. - ``weight`` = The total weight of the pairs in each bin. - ``npairs`` = The total number of pairs in each bin. -:nt_statistic: (str, default='compensated' if ``rand_files`` is given, otherwise 'simple') Which statistic to use for the estimator of the NT correlation function. +:nt_statistic: (str, default='compensated' if ``rand_files`` is given, otherwise 'simple') Which + statistic to use for the estimator of the NT correlation function. Options are: @@ -654,7 +686,8 @@ about the output columns. Define: - NT = Sum(t around data points) - - RT = Sum(t around random points), scaled to be equivalent in effective number as the number of pairs in NG. + - RT = Sum(t around random points), scaled to be equivalent in effective number as the number + of pairs in NG. - npairs = number of pairs in NT. Then this statistic is tR = (NT-RT)/npairs @@ -680,8 +713,10 @@ about the output columns. - ``R_nom`` = The center of the bin - ``meanR`` = The mean separation of the points that went into the bin. - ``meanlogR`` = The mean log(R) of the points that went into the bin. - - ``xip`` = where t1 and t2 are measured with respect to the line joining the two points, where p1 is on the left and p2 is on the right. - - ``xim`` = where t1 and t2 are measured with respect to the line joining the two points, where p1 is on the left and p2 is on the right. + - ``xip`` = where t1 and t2 are measured with respect to the line joining the + two points, where p1 is on the left and p2 is on the right. + - ``xim`` = where t1 and t2 are measured with respect to the line joining the + two points, where p1 is on the left and p2 is on the right. - ``xip_im`` = . In the formulation of xi+ using complex numbers, this is the imaginary component. @@ -708,12 +743,14 @@ about the output columns. - ``meanR`` = The mean separation of the points that went into the bin. - ``meanlogR`` = The mean log(R) of the points that went into the bin. - ``qR`` = The mean real component of the quatrefoil field relative to the center points. - - ``qR_im`` = The mean imaginary component of the quatrefoil field relative to the center points. + - ``qR_im`` = The mean imaginary component of the quatrefoil field relative to the center + points. - ``sigma`` = The 1-sigma error bar for ``qR`` and ``qR_im``. - ``weight`` = The total weight of the pairs in each bin. - ``npairs`` = The total number of pairs in each bin. -:nq_statistic: (str, default='compensated' if ``rand_files`` is given, otherwise 'simple') Which statistic to use for the estimator of the NQ correlation function. +:nq_statistic: (str, default='compensated' if ``rand_files`` is given, otherwise 'simple') Which + statistic to use for the estimator of the NQ correlation function. Options are: @@ -721,7 +758,8 @@ about the output columns. Define: - NQ = Sum(q around data points) - - RQ = Sum(q around random points), scaled to be equivalent in effective number as the number of pairs in NG. + - RQ = Sum(q around random points), scaled to be equivalent in effective number as the number + of pairs in NG. - npairs = number of pairs in NQ. Then this statistic is qR = (NQ-RQ)/npairs @@ -747,8 +785,10 @@ about the output columns. - ``R_nom`` = The center of the bin - ``meanR`` = The mean separation of the points that went into the bin. - ``meanlogR`` = The mean log(R) of the points that went into the bin. - - ``xip`` = where q1 and q2 are measured with respect to the line joining the two points. - - ``xim`` = where q1 and q2 are measured with respect to the line joining the two points. + - ``xip`` = where q1 and q2 are measured with respect to the line joining the + two points. + - ``xim`` = where q1 and q2 are measured with respect to the line joining the + two points. - ``xip_im`` = . In the formulation of xi+ using complex numbers, this is the imaginary component. @@ -785,9 +825,11 @@ about the output columns. - ``zeta`` = The correlation function. - ``sigma_zeta`` = The 1-sigma error bar for zeta. - ``DDD``, ``RRR`` = The raw numbers of triangles for the data and randoms - - ``DDR``, ``DRD``, ``RDD``, ``DRR``, ``RDR``, ``RRD`` (if ``nn_statistic=compensated``) = The cross terms between data and random. + - ``DDR``, ``DRD``, ``RDD``, ``DRR``, ``RDR``, ``RRD`` (if ``nnn_statistic=compensated``) = The + cross terms between data and random. -:nnn_statistic: (str, default='compensated') Which statistic to use for the estimator of the NNN correlation function. +:nnn_statistic: (str, default='compensated') Which statistic to use for the estimator of the NNN + correlation function. Options are: @@ -796,6 +838,9 @@ about the output columns. - 'simple' is the older version: zeta = (DDD/RRR - 1), although this is not actually an estimator of zeta. Rather, it estimates zeta(d1,d2,d3) + xi(d1) + xi(d2) + xi(d3). + For most science analyses, ``'compensated'`` is recommended when random + catalogs are available. + :ggg_file_name: (str) The output filename for shear-shear-shear correlation function. This is the shear three-point correlation function. We use the "natural components" @@ -867,12 +912,15 @@ functions. The output columns are: - - ``R`` = The radius of the aperture. (Spaced the same way as ``R_nom`` is in the correlation function output files. + - ``R`` = The radius of the aperture. (Spaced the same way as ``R_nom`` is in the correlation + function output files. - ``Mapsq`` = The E-mode aperture mass variance for each radius R. - ``Mxsq`` = The B-mode aperture mass variance. - - ``MMxa``, ``MMxb`` = Two semi-independent estimate for the E-B cross term. (Both should be consistent with zero for parity invariance shear fields.) + - ``MMxa``, ``MMxb`` = Two semi-independent estimate for the E-B cross term. (Both should be + consistent with zero for parity invariance shear fields.) - ``sig_map`` = The 1-sigma error bar for these values. - - ``Gamsq`` = The variance of the top-hat weighted mean shear in apertures of the given radius R. + - ``Gamsq`` = The variance of the top-hat weighted mean shear in apertures of the given radius + R. - ``sig_gam`` = The 1-sigma error bar for ``Gamsq``. :m2_uform: (str, default='Crittenden') The function form of the aperture @@ -903,8 +951,10 @@ functions. The output columns are: - - ``R`` = The radius of the aperture. (Spaced the same way as ``R_nom`` is in the correlation function output files. - - ``NMap`` = The E-mode aperture mass correlated with the density smoothed with the same aperture profile as the aperture mass statistic uses. + - ``R`` = The radius of the aperture. (Spaced the same way as ``R_nom`` is in the correlation + function output files. + - ``NMap`` = The E-mode aperture mass correlated with the density smoothed with the same + aperture profile as the aperture mass statistic uses. - ``NMx`` = The corresponding B-mode statistic. - ``sig_nmap`` = The 1-sigma error bar for these values. @@ -915,8 +965,10 @@ functions. The output columns are: - - ``R`` = The radius of the aperture. (Spaced the same way as ``R_nom`` is in the correlation function output files. - - ``NMap`` = The E-mode aperture mass correlated with the density smoothed with the same aperture profile as the aperture mass statistic uses. + - ``R`` = The radius of the aperture. (Spaced the same way as ``R_nom`` is in the correlation + function output files. + - ``NMap`` = The E-mode aperture mass correlated with the density smoothed with the same + aperture profile as the aperture mass statistic uses. - ``NMx`` = The corresponding B-mode statistic. - ``sig_nmap`` = The 1-sigma error bar for these values. - ``Napsq`` = The variance of the aperture-weighted galaxy density. @@ -979,4 +1031,5 @@ Miscellaneous parameters The default is to try to determine the number of cpu cores your system has and use that many threads. - + For reproducibility and predictable resource usage on shared systems, it is + often better to set this explicitly. diff --git a/docs/patches.rst b/docs/patches.rst index 3d81b0c2..89dd0405 100644 --- a/docs/patches.rst +++ b/docs/patches.rst @@ -26,7 +26,7 @@ correlation function: 4. To run k-means on some data set for non-correlation reasons. TreeCorr happens to have an extremely efficient implementation of the k-means algorithm. So if you want to perform k-means clustering on - some data that can be represnted in a TreeCorr `Catalog` (i.e. + some data that can be represented in a TreeCorr `Catalog` (i.e. only 2 or 3 spatial dimensions), then using TreeCorr may be a particularly efficient way to do the clustering. See `Running K-Means` below. @@ -34,6 +34,7 @@ correlation function: Below we describe how to split up an input `Catalog` into patches and a few things you can do with it once you have done so. + Defining Patches on Input ------------------------- @@ -43,8 +44,8 @@ is to just tell TreeCorr the patch number for each object explicitly. If passing in numpy arrays for everything, then just pass in a ``patch`` parameter with integer values indicating the patch number. -If reading in data from a file, then set a ``patch_col`` to use which -should have these values. +If reading data from a file, set ``patch_col`` to the column that contains +these values. The next simplest way to define the patches is to tell TreeCorr how many patches you want using ``npatch``. @@ -57,12 +58,22 @@ where patches are on the sky, you would probably want to have a single set of patch centers and have all of your catalogs use that via the ``patch_centers`` option. See `Using Patch Centers` below for details. +.. warning:: + + If you are using patches for cross-correlations of multiple catalogs, + make sure you only use ``npatch`` on (at most) one of them. + If you use ``npatch`` on two catalogs being cross-correlated, then the patch + definitions will be different between the two catalogs and the resulting + covariance estimates will be wrong. Rather you should use ``npatch`` on one + of the catalogs and copy those definitions to the other catalogs using + ``patch_centers``. + Running K-Means --------------- -One standard way to split up a set of objects into roughly equal area -patches is an algorithm called +One standard way to split up a set of objects into roughly equal-area +patches is to use an algorithm called `k-means clustering `_. The basic idea of the algorithm is to divide the points :math:`\vec x_j` into @@ -89,7 +100,7 @@ sky that has objects, so this algorithm is a good choice for dividing up a catalog of astronomical objects into fairly uniform patches. To use the TreeCorr implementation of k-means, simply -set the ``npatch`` parameter in the `Catalog` constructor to specifiy +set the ``npatch`` parameter in the `Catalog` constructor to specify how many patches you want TreeCorr to split the data into. .. note:: @@ -100,7 +111,7 @@ how many patches you want TreeCorr to split the data into. astronomical survey area. If you really want to make patches according to 3-D clustering of points, then you should input x,y,z values instead. -There are also two additional options which can affect how the k-means +There are also two additional options that can affect how the k-means algorithm runs: * ``kmeans_init`` specifies what procedure to use for the initialization @@ -124,13 +135,13 @@ algorithm runs: of the inertia rather than the mean inertia, so it tends to lead to patches that have a smaller final size variation than the regular k-means algorithm. - This is not the default algorithm because it is not provably (at least by - me) stable. It is possible that the iteration can get into a failure mode + This is not the default algorithm because it is not provably stable + (at least as far as I know). It is possible that the iteration can get into a failure mode where one patch will end up with zero objects. The regular k-means provably cannot fail in this way. So if you care especially about having very uniform patch sizes, you might - want to try this option, but be careful about inspecting the results that + want to try this option, but be careful when inspecting the results so that they don't look crazy. See also `Field.run_kmeans`, which has more information about these options, @@ -142,12 +153,17 @@ where these parameters are called simply ``init`` and ``alt`` respectively. Before implementing k-means in TreeCorr, I investigated what other options there were in the Python landscape. I found the following implementations: - * `scipy.cluster.vq.kmeans `_ - * `scipy.cluster.vq.kmeans2 `_ + * `scipy.cluster.vq.kmeans + `_ + * `scipy.cluster.vq.kmeans2 + `_ * `kmeans_radec `_ - * `pyclustering.cluster.kmeans `_ - * `sklearn.cluster.KMeans `_ - * `sklearn.cluster.MiniBatchKMeans `_ + * `pyclustering.cluster.kmeans + `_ + * `sklearn.cluster.KMeans + `_ + * `sklearn.cluster.MiniBatchKMeans + `_ I made a `notebook `_ comparing the different algorithms using a random million galaxies from the DES SV @@ -161,7 +177,7 @@ where these parameters are called simply ``init`` and ``alt`` respectively. However, we don't really care about the total inertia being minimized. For most purposes here, we really want the patches to be all close to the *same* size. So rather than - the total inertia, my metric for quality was the rms variation of the intertia + the total inertia, my metric for quality was the rms variation of the inertia (aka the standard deviation). Fortunately, the process of minimizing the total inertia does tend to select patches with @@ -173,9 +189,9 @@ where these parameters are called simply ``init`` and ``alt`` respectively. Comparing the results of the various k-means implementations, I found that they all tend to be either fairly slow, taking a minute or more for just 1 million objects, or they have very high rms variation in the inertia. - I reran each code multiple times using a different random million objects selected from the original - catalog (of around 16 million objects). Here is a scatter plot of the time vs rms variation - in the inertia for the various codes. + I reran each code multiple times using a different random million objects selected from the + original catalog (of around 16 million objects). Here is a scatter plot of the time vs rms + variation in the inertia for the various codes. .. image:: https://user-images.githubusercontent.com/623887/57647337-ac6bd800-7590-11e9-80bc-900bda3bf66b.png @@ -198,7 +214,7 @@ where these parameters are called simply ``init`` and ``alt`` respectively. implementation according to any of these metrics. In addition, you can see some slightly smaller orange dots, which have even lower rms - variation but take very slightly longer to run. These are the alternate algorithm I mentioned + variation but take very slightly longer to run. These are the alternate algorithms I mentioned above. This alternate algorithm is similar to k-means, but it penalizes patches with a larger-than-average inertia, so they give up some of their outer points to patches with smaller inertia. In other words, it explicitly targets making the rms variation as small as @@ -265,12 +281,12 @@ The overall procedure for doing this is as follows: galaxies or clusters or even stars). Or it could be the large catalog you want to use, but sampled using the ``every_nth`` option to read in only a fraction of the rows. Run k-means on the smaller catalog - and write the patch_centers to a file, as describe `above `. + and write the patch_centers to a file, as described `above `. 2. Set up a directory somewhere that TreeCorr can use as temporary space for writing the individual patch files. -3. Define the full `Catalog`, specifying to use the above centers file for the +3. Define the full `Catalog`, specifying the above centers file as ``patch_centers`` and the temp directory as ``save_patch_dir``. -4. Make sure not to do anything that requires the catalog be loaded from disk. +4. Make sure not to do anything that requires the catalog to be loaded from disk. TreeCorr will delay doing the actual load until it needs to do so. Here, we want to make sure it never loads the full data. 5. Run the `process ` function (for whichever correlation @@ -300,7 +316,7 @@ be a problem, but the source catalog is too large to hold in memory:: In both cases, the result should be equivalent to what you would get if you could hold the catalogs fully in memory, but the peak memory will be much lower. The downside is that this usage will generally take somewhat longer -- -probably something like a factor of 2 for typical scenarios, but this of course +probably something like a factor of 2 for typical scenarios, but this depends heavily on the nature of your calculation, how fast your disk I/O is compared to your CPUs, and how many cores you are using. @@ -339,7 +355,7 @@ over multiple machines with MPI using `mpi4py ` functions take an optional ``comm`` parameter. When running in an MPI job, you can pass in ``comm=MPI.COMM_WORLD``, and TreeCorr will divide up the work among however many nodes you are using. -The results will be sent back the the rank 0 node and combined to produce the +The results will be sent back to the rank 0 node and combined to produce the complete answer: .. code-block:: python diff --git a/docs/scripts.rst b/docs/scripts.rst index ce76085e..7490b5a8 100644 --- a/docs/scripts.rst +++ b/docs/scripts.rst @@ -7,7 +7,7 @@ useful when driving the code from Python; however, it enables running the code from some executable scripts, described below. Specifically, the parameters defined in the configuration file are -loaded into a Python dict, which is passed to each of the classes +loaded into a Python dict, which is passed to each class as needed. The advantage of this is that TreeCorr will only use the parameters it actually needs when initializing each object. Any additional parameters (e.g. those @@ -18,7 +18,7 @@ The corr2 and corr3 executables Along with the installed Python library, TreeCorr also includes two executable scripts, called ``corr2`` and ``corr3``. -The scripts takes one required command-line argument, which +Each script takes one required command-line argument, which is the name of a configuration file:: corr2 config.yaml @@ -37,7 +37,7 @@ also allow JSON files if you prefer, or a legacy format, which is like an .ini file, but without the section headings, consisting of key = value lines. The three formats are normally distinguished by their extensions (.yaml, .json, or .params respectively), but -you can also give the file type explicitly with the -f option. E.g.:: +you can also give the file type explicitly with the -f option. For example:: corr2 my_config_file.txt -f params @@ -45,19 +45,48 @@ would specify that the configuration file ``my_config_file.txt`` uses the legacy "params" format. You can also specify parameters on the command line after the name of -the configuration file. e.g.:: +the configuration file. For example:: corr2 config.yaml file_name=file1.dat gg_file_name=file1.out corr2 config.yaml file_name=file2.dat gg_file_name=file2.out ... -This can be useful when running the program from a script for lots of input -files. +This can be useful when running the program from a script for many input files. -The corr2 function from python +Python API and corr2/corr3 parity +--------------------------------- + +The ``corr2`` and ``corr3`` executables and the Python ``treecorr.corr2`` / +``treecorr.corr3`` functions use the same configuration logic and core processing +pipeline, so they produce matching results when given equivalent configs and inputs. + +What is equivalent: + +* Same parameter names and meanings from :doc:`params`. +* Same correlation calculations and estimators. +* Same output products when the same output file options are set. + +What differs: + +* The executables are file-driven, one-shot runs from the command line. +* Direct class usage in Python (`Catalog`, `Corr2`, `Corr3`) provides lower-level + control, such as: + + - splitting processing into ``process_auto`` / ``process_cross`` / ``finalize`` + - custom post-processing with ``estimate_cov(..., func=...)`` and + ``estimate_multi_cov(..., func=...)`` + - explicit in-memory workflows without writing intermediate files + +When to choose which interface: + +* Use ``corr2``/``corr3`` for reproducible, config-driven batch jobs. +* Use direct Python classes for iterative analysis, custom data vectors, and + non-standard control flow. + +The corr2 function from Python ------------------------------ -The same functionality that you have from the ``corr2`` executable is available in python via the +The same functionality that you have from the ``corr2`` executable is available in Python via the `corr2` function:: import treecorr @@ -68,7 +97,7 @@ The same functionality that you have from the ``corr2`` executable is available .. autofunction:: treecorr.corr2 -The corr3 function from python +The corr3 function from Python ------------------------------ .. autofunction:: treecorr.corr3 diff --git a/docs/shear.rst b/docs/shear.rst index 96467166..7a3264b0 100644 --- a/docs/shear.rst +++ b/docs/shear.rst @@ -49,7 +49,7 @@ local coordinates in the neighborhood of the galaxy such that north is oriented vertically, and then measuring the ellipticity in the normal way. With this convention, :math:`g` is positive real when the shear is oriented -in the local E-W direction. It is negative real when oritented in the N-S direction. +in the local E-W direction. It is negative real when oriented in the N-S direction. It is positive imaginary when oriented along NW-SE. And it is negative imaginary when oriented along NE-SW. diff --git a/include/BinType.h b/include/BinType.h index 85e9b2f1..96953584 100644 --- a/include/BinType.h +++ b/include/BinType.h @@ -965,7 +965,7 @@ struct BinTypeHelper if (O > 1 && !metric.CCW(p1, p3, p2)) { // For skinny triangles, be careful that the points can't flip to the other side. - // This is similar to the calculation below. We effecively check that cosphi can't + // This is similar to the calculation below. We effectively check that cosphi can't // increase to 1. // First check if either side on its own can cause a flip of orientation. double sindphi2=0., sindphi3=0.; @@ -1644,4 +1644,3 @@ struct BinTypeHelper #endif - diff --git a/include/Corr2.h b/include/Corr2.h index 4f89942a..a1e1e8bc 100644 --- a/include/Corr2.h +++ b/include/Corr2.h @@ -90,7 +90,7 @@ class BaseCorr2 template struct R1 {}; - // This bit is a workaround for the the fact that virtual functions cannot be templates. + // This bit is a workaround for the fact that virtual functions cannot be templates. virtual void doFinishProcess(const BaseCell& c1, const BaseCell& c2, double rsq, double r, double logr, int k, int k2, R1<1,0>)=0; virtual void doFinishProcess(const BaseCell& c1, const BaseCell& c2, diff --git a/include/Corr3.h b/include/Corr3.h index 45af46b2..42a6dfe1 100644 --- a/include/Corr3.h +++ b/include/Corr3.h @@ -192,7 +192,7 @@ class BaseCorr3 virtual std::unique_ptr getMP2(bool use_ww) =0; virtual std::unique_ptr getMP3(bool use_ww) =0; - // This bit is a workaround for the the fact that virtual functions cannot be templates. + // This bit is a workaround for the fact that virtual functions cannot be templates. virtual void doFinishProcess( const BaseCell& c1, const BaseCell& c2, const BaseCell& c3, const double d1, const double d2, const double d3, const double u, const double v, diff --git a/include/Metric.h b/include/Metric.h index 1ca48b4f..592c046f 100644 --- a/include/Metric.h +++ b/include/Metric.h @@ -629,7 +629,7 @@ struct MetricHelper // In this case, d2, d3 are normal, but d1 needs both points projected to the plane // of p1 do get the right separation. // If we do it the normal way then d1 is too big by a factor r2/r1. - // So just do that, then then scale down by that factor. + // So just do that, then scale down by that factor. double s; if (d1sq == 0.) d1sq = DistSq(p2, p3, s, s); @@ -963,4 +963,3 @@ struct ValidMC { enum { _M = MetricHelper::_ThreeD == int(ThreeD) ? M : Euclidean }; }; #endif - diff --git a/include/Process3.h b/include/Process3.h index 2a7cd65a..32b403c6 100644 --- a/include/Process3.h +++ b/include/Process3.h @@ -36,7 +36,8 @@ void ProcessV( ++recursen; xdbg<= "<= "<= "<= "<= "<= "<= 0) Assert(x); } while (false) #else #define dbg if(false) (std::cerr) diff --git a/src/Corr3.cpp b/src/Corr3.cpp index 0ebb48b8..d85bbaff 100644 --- a/src/Corr3.cpp +++ b/src/Corr3.cpp @@ -606,7 +606,8 @@ void BaseCorr3::process12(const BaseCell& c1, const BaseCell& c2, const MetricHelper& metric, bool quick) { // Does all triangles with one point in c1 and the other two points in c2 - xdbg<& c1, const BaseCell& c2, const MetricHelper& metric, bool quick) { // Does all triangles with two points in c1 and the other point in c2 - xdbg<& c1, const BaseCell& c2, const BaseCell& c3, const MetricHelper& metric, double d1sq, double d2sq, double d3sq) { - xdbg<& c1, const std::vector*>& c2list, const MetricHelper& metric, bool quick, BaseMultipoleScratch& mp) { - xdbg<(c1, c2list, metric, newc2list, anysplit1, mp2, maxr2); std::vector*> newc3list; maxr3 = splitC2CellsOrCalculateGn(c1, c3list, metric, newc3list, anysplit1, mp3, maxr3); - xdbg<<"newsize = "< "<::calculateZeta( const BaseCell& c1, int ordered, BaseMultipoleScratch& mp2, BaseMultipoleScratch& mp3, int kstart, int mink_zeta) { - xdbg< std::complex gam0 = wg1 * mp3.Gn(ig2,n-1) * mp2.Gn(ig3,-n-1); std::complex gam1 = std::conj(wg1) * mp3.Gn(ig2,n+1) * mp2.Gn(ig3,-n+1); std::complex gam2 = wg1 * mp3.Gn(ig2,n-1) * std::conj(mp2.Gn(ig3,n+1)); - std::complex gam3 = wg1 * std::conj(mp3.Gn(ig2,-n+1)) * mp2.Gn(ig3,-n-1); + std::complex gam3 = + wg1 * std::conj(mp3.Gn(ig2,-n+1)) * mp2.Gn(ig3,-n-1); if (swap23) { gam0 += wg1 * mp2.Gn(ig2,n-1) * mp3.Gn(ig3,-n-1); gam1 += std::conj(wg1) * mp2.Gn(ig2,n+1) * mp3.Gn(ig3,-n+1); @@ -3043,10 +3055,12 @@ struct MultipoleHelper<6> // Now +-n for the rest for (int n=1; n<=maxn; ++n) { std::complex gam0p = wg1 * mp3.Gn(ig2,n-1) * mp2.Gn(ig3,-n-1); - std::complex gam1p = std::conj(wg1) * mp3.Gn(ig2,n+1) * mp2.Gn(ig3,-n+1); + std::complex gam1p = + std::conj(wg1) * mp3.Gn(ig2,n+1) * mp2.Gn(ig3,-n+1); std::complex gam0m = wg1 * mp3.Gn(ig2,-n-1) * mp2.Gn(ig3,n-1); - std::complex gam1m = std::conj(wg1) * mp3.Gn(ig2,-n+1) * mp2.Gn(ig3,n+1); + std::complex gam1m = + std::conj(wg1) * mp3.Gn(ig2,-n+1) * mp2.Gn(ig3,n+1); zeta.gam0r[iz+n] += gam0p.real(); zeta.gam0i[iz+n] += gam0p.imag(); diff --git a/src/KMeans.cpp b/src/KMeans.cpp index 2bcb541b..7e8fbd20 100644 --- a/src/KMeans.cpp +++ b/src/KMeans.cpp @@ -694,7 +694,8 @@ double CalculateShiftSq(const std::vector >& centers, double shiftsq=0.; for (size_t i=0; i "< "<>> cat = treecorr.Catalog(x=x, y=y, k=k, w=w) - Each of these input paramters should be a numpy array, where each corresponding element + Each of these input parameters should be a numpy array, where each corresponding element is the value for that object. Of course, all the arrays should be the same size. In some cases, there are additional required parameters. For instance, with RA and Dec @@ -247,47 +247,70 @@ class Catalog(object): catalog for the N has num=0, the one for G has num=1. This is only necessary if you are using a config dict where things like ``x_col`` have multiple values. (default: 0) - logger: If desired, a Logger object for logging. (default: None, in which case - one will be built according to the config dict's verbose level.) + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. + (default: None, in which case one will be built according to the + config dict's verbose level.) is_rand (bool): If this is a random file, then setting is_rand to True will let them skip k_col, g1_col, and g2_col if they were set for the main catalog. (default: False) - x (array): The x values. (default: None; When providing values directly, either + x (:class:`numpy.ndarray`): + The x values. (default: None; When providing values directly, either x,y are required or ra,dec are required.) - y (array): The y values. (default: None; When providing values directly, either + y (:class:`numpy.ndarray`): + The y values. (default: None; When providing values directly, either x,y are required or ra,dec are required.) - z (array): The z values, if doing 3d positions. (default: None; invalid in + z (:class:`numpy.ndarray`): + The z values, if doing 3d positions. (default: None; invalid in conjunction with ra, dec.) - ra (array): The RA values. (default: None; When providing values directly, either + ra (:class:`numpy.ndarray`): + The RA values. (default: None; When providing values directly, either x,y are required or ra,dec are required.) - dec (array): The Dec values. (default: None; When providing values directly, either + dec (:class:`numpy.ndarray`): + The Dec values. (default: None; When providing values directly, either x,y are required or ra,dec are required.) - r (array): The r values (the distances of each source from Earth). (default: None; + r (:class:`numpy.ndarray`): + The r values (the distances of each source from Earth). (default: None; invalid in conjunction with x, y.) - w (array): The weights to apply when computing the correlations. (default: None) - wpos (array): The weights to use for position centroiding. (default: None, which + w (:class:`numpy.ndarray`): + The weights to apply when computing the correlations. (default: None) + wpos (:class:`numpy.ndarray`): + The weights to use for position centroiding. (default: None, which means to use the value weights, w, to weight the positions as well.) - flag (array): An optional array of flags, indicating objects to skip. Rows with + flag (:class:`numpy.ndarray`): + An optional array of flags, indicating objects to skip. Rows with flag != 0 (or technically flag & ~ok_flag != 0) will be given a weight of 0. (default: None) - k (array): The kappa values to use for scalar correlations. (This may represent + k (:class:`numpy.ndarray`): + The kappa values to use for scalar correlations. (This may represent any scalar field.) (default: None) - z1 (array): The z1 values to use for complex scalar correlations. (default: None) - z2 (array): The z2 values to use for complex scalar correlations. (default: None) - v1 (array): The v1 values to use for vector correlations. (default: None) - v2 (array): The v2 values to use for vector correlations. (default: None) - g1 (array): The g1 values to use for shear correlations. (g1,g2 may represent any + z1 (:class:`numpy.ndarray`): + The z1 values to use for complex scalar correlations. (default: None) + z2 (:class:`numpy.ndarray`): + The z2 values to use for complex scalar correlations. (default: None) + v1 (:class:`numpy.ndarray`): + The v1 values to use for vector correlations. (default: None) + v2 (:class:`numpy.ndarray`): + The v2 values to use for vector correlations. (default: None) + g1 (:class:`numpy.ndarray`): + The g1 values to use for shear correlations. (g1,g2 may represent any spin-2 field.) (default: None) - g2 (array): The g2 values to use for shear correlations. (g1,g2 may represent any + g2 (:class:`numpy.ndarray`): + The g2 values to use for shear correlations. (g1,g2 may represent any spin-2 field.) (default: None) - t1 (array): The t1 values to use for trefoil (spin-3) correlations. (default: None) - t2 (array): The t2 values to use for trefoil (spin-3) correlations. (default: None) - q1 (array): The q1 values to use for quatrefoil (spin-4) correlations. + t1 (:class:`numpy.ndarray`): + The t1 values to use for trefoil (spin-3) correlations. (default: None) + t2 (:class:`numpy.ndarray`): + The t2 values to use for trefoil (spin-3) correlations. (default: None) + q1 (:class:`numpy.ndarray`): + The q1 values to use for quatrefoil (spin-4) correlations. (default: None) - q2 (array): The q2 values to use for quatrefoil (spin-4) correlations. + q2 (:class:`numpy.ndarray`): + The q2 values to use for quatrefoil (spin-4) correlations. (default: None) - patch (array or int): Optionally, patch numbers to use for each object. (default: None) + patch (:class:`numpy.ndarray` or int): + Optionally, patch numbers to use for each object. (default: None) .. note:: @@ -300,10 +323,15 @@ class Catalog(object): given patch number, and ``npatch`` is required to set the total number of patches, which this catalog is a part of. - patch_centers (array or str): Alternative to setting patch by hand or using kmeans, you + patch_centers (:class:`numpy.ndarray` or str): + Alternative to setting patch by hand or using kmeans, you may instead give patch_centers either as a file name or an array from which the patches will be determined. (default: None) + For cross-correlations that use patch-based covariance estimates, + all catalogs should use consistent patch definitions, typically by + sharing the same ``patch_centers``. + file_type (str): What kind of file is the input file. Valid options are 'ASCII', 'FITS' 'HDF', or 'Parquet' (default: if the file_name extension starts with .fit, then use 'FITS', or with .hdf, then use 'HDF', or with '.par', @@ -329,6 +357,11 @@ class Catalog(object): that are relevant for the area that was split into patches, which may include more catalogs than just this one. + If two catalogs in a cross-correlation both use patches, avoid + running separate ``npatch`` assignments for each one. Use shared + ``patch_centers`` instead so corresponding patch IDs refer to the + same sky regions. + kmeans_init (str): If using kmeans to make patches, which init method to use. cf. `Field.run_kmeans` (default: 'tree') kmeans_alt (bool): If using kmeans to make patches, whether to use the alternate kmeans @@ -508,9 +541,10 @@ class Catalog(object): should be an integer, which specifies how many digits to write. (default: 16) - rng (np.Generator): If desired, a numpy.random.Generator or numpy.random.RandomState - instance to use for any random number generation (e.g. kmeans patches). - (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for any random number + generation (e.g. kmeans patches). Legacy ``RandomState`` instances are + also accepted. (default: None) num_threads (int): How many OpenMP threads to use during the catalog load steps. (default: use the number of cpu cores) @@ -1651,7 +1685,8 @@ def checkForNaN(self, col, col_str): """Check if the column has any NaNs. If so, set those rows to have w[k]=0. Parameters: - col (array): The input column to check. + col (:class:`numpy.ndarray`): + The input column to check. col_str (str): The name of the column. Used only as information in logging output. """ if col is not None and np.any(np.isnan(col)): @@ -2400,7 +2435,8 @@ def getNField(self, *, min_size=0, max_size=None, split_method=None, brute=False max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default: self.coords) - logger: A Logger object if desired (default: self.logger) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired (default: self.logger) Returns: An `NField` object @@ -2434,7 +2470,8 @@ def getKField(self, *, min_size=0, max_size=None, split_method=None, brute=False max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default self.coords) - logger: A Logger object if desired (default: self.logger) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired (default: self.logger) Returns: A `KField` object @@ -2469,7 +2506,8 @@ def getZField(self, *, min_size=0, max_size=None, split_method=None, brute=False max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default self.coords) - logger: A Logger object if desired (default: self.logger) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired (default: self.logger) Returns: A `ZField` object @@ -2504,7 +2542,8 @@ def getVField(self, *, min_size=0, max_size=None, split_method=None, brute=False max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default self.coords) - logger: A Logger object if desired (default: self.logger) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired (default: self.logger) Returns: A `VField` object @@ -2539,7 +2578,8 @@ def getGField(self, *, min_size=0, max_size=None, split_method=None, brute=False max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default self.coords) - logger: A Logger object if desired (default: self.logger) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired (default: self.logger) Returns: A `GField` object @@ -2574,7 +2614,8 @@ def getTField(self, *, min_size=0, max_size=None, split_method=None, brute=False max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default self.coords) - logger: A Logger object if desired (default: self.logger) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired (default: self.logger) Returns: A `TField` object @@ -2609,7 +2650,8 @@ def getQField(self, *, min_size=0, max_size=None, split_method=None, brute=False max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default self.coords) - logger: A Logger object if desired (default: self.logger) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired (default: self.logger) Returns: A `QField` object @@ -2728,7 +2770,7 @@ def read_patch_centers(self, file_name): string, being the file name. The patch centers are read from the file and returned. Parameters: - file_name (str): The name of the file to write to. + file_name (str): The name of the file to read from. Returns: The centers, as an array, which can be used to determine the patches. @@ -2750,7 +2792,7 @@ def load(self): set of patches, since you may not be able to fit all the patches in memory at once. One does not normally need to call this method explicitly. It will run automatically - whenever the data is needed. However, if you want to directly control when the disk + whenever the data are needed. However, if you want to directly control when the disk access happens, you can use this function. """ if not self.loaded: @@ -2954,7 +2996,7 @@ def get_patches(self, *, low_mem=False): return self._patches if low_mem and self.file_name is not None: - # This is a litle tricky, since we don't want to trigger a load if the catalog + # This is a little tricky, since we don't want to trigger a load if the catalog # isn't loaded yet. So try to get the patches from centers or single_patch first. if self._centers is not None: patch_set = range(len(self._centers)) @@ -3223,7 +3265,7 @@ def read_catalogs(config, key=None, list_key=None, *, num=0, logger=None, is_ran """Read in a list of catalogs for the given key. key should be the file_name parameter or similar key word. - list_key should be be corresponging file_list parameter, if appropriate. + list_key should be the corresponding file_list parameter, if appropriate. At least one of key or list_key must be provided. If both are provided, then only one of these should be in the config dict. @@ -3240,8 +3282,10 @@ def read_catalogs(config, key=None, list_key=None, *, num=0, logger=None, is_ran Either key or list_key is required. (default: None) num (int): Which number catalog does this correspond to. e.g. file_name should use num=0, file_name2 should use num=1. (default: 0) - logger: If desired, a Logger object for logging. (default: None, in which case - one will be built according to the config dict's verbose level.) + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. + (default: None, in which case one will be built according to the config + dict's verbose level.) is_rand (bool): If this is a random file, then setting is_rand to True will let them skip k_col, g1_col, and g2_col if they were set for the main catalog. (default: False) diff --git a/treecorr/config.py b/treecorr/config.py index cadaf843..f5c53145 100644 --- a/treecorr/config.py +++ b/treecorr/config.py @@ -27,11 +27,11 @@ def parse_variable(config, v): """Parse a configuration variable from a string that should look like 'key = value' and write that value to config[key]. - :param config: The configuration dict to wich to write the key,value pair + :param config: The configuration dict to which to write the key,value pair :param v: A string of the form 'key = value' """ if '=' not in v: - raise ValueError('Improper variable specificationi: %s. Use syntax: key = value.'%v) + raise ValueError('Improper variable specification: %s. Use syntax: key = value.'%v) key, value = v.split('=',1) key = key.strip() # Cut off any trailing comment @@ -58,11 +58,12 @@ def parse_bool(value): Valid string values for False are: 'false', 'no', 'f', 'n', 'none' Capitalization is ignored. - If value is a number, it is converted to a bool in the usual way. + If value is an integer (or a string that parses as an integer), the integer value is + returned. This preserves special integer semantics used by some parameters. :param value: The value to parse. - :returns: The value converted to a bool. + :returns: The parsed boolean (or integer, as described above). """ if isinstance(value,str): if value.strip().upper() in [ 'TRUE', 'YES', 'T', 'Y' ]: @@ -229,7 +230,7 @@ def check_config(config, params, aliases=None, logger=None): :param params: A dict of valid parameters with information about each one. :param aliases: A dict of deprecated parameters that are still aliases for new names. (default: None) - :param logger: If desired, a logger object for logging any warnings here. (default: None) + :param logger: If desired, a ``Logger`` object for logging any warnings here. (default: None) :returns: The updated config dict. """ @@ -262,7 +263,7 @@ def check_config(config, params, aliases=None, logger=None): if value is None: continue - # If limited allowed value, check that this is one of them. + # If there are limited allowed values, check that this is one of them. if valid_values is not None and value is not None: if value_type is str: matches = [ v for v in valid_values if value == v ] diff --git a/treecorr/corr2base.py b/treecorr/corr2base.py index 3ea262fe..2665ee33 100644 --- a/treecorr/corr2base.py +++ b/treecorr/corr2base.py @@ -91,7 +91,7 @@ class Corr2(object): See `Metrics` for more information about these various metric options. - There are also a few different possibile binning prescriptions to define the range of + There are also a few different possible binning prescriptions to define the range of distances, which should be placed into each bin. - 'Log' - logarithmic binning in the distance. The bin steps will be uniform in @@ -143,10 +143,12 @@ class Corr2(object): Parameters: config (dict): A configuration dict that can be used to pass in the below kwargs if - desired. This dict is allowed to have addition entries in addition - to those listed below, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case - one will be built according to the config dict's verbose level.) + desired. This dict is allowed to have additional entries beyond + those listed below, which are ignored here. (default: None) + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in + which case one will be built according to the config dict's + verbose level.) Keyword Arguments: @@ -177,7 +179,7 @@ class Corr2(object): If bin_slop = 1, then the bin into which a particular pair is placed may be incorrect by at most 1.0 bin widths. (default: None, which means to use a bin_slop that gives a maximum error of 10% on any bin, - which has been found to yield good results for most application. + which has been found to yield good results for most applications. angle_slop (float): How much slop to allow in the angular direction. This works very similarly to bin_slop, but applies to the projection angle of a pair of cells. The projection angle for any two objects in a pair of cells @@ -204,7 +206,7 @@ class Corr2(object): log_file (str): If no logger is provided, this will specify a file to write the logging output. (default: None; i.e. output to standard output) - output_dots (bool): Whether to output progress dots during the calcualtion of the + output_dots (bool): Whether to output progress dots during the calculation of the correlation function. (default: False unless verbose is given and >= 2, in which case True) @@ -255,8 +257,9 @@ class Corr2(object): cross_patch_weight (str): How to weight pairs that cross between two patches when one patch is deselected (e.g. in a jackknife sense) and the other is selected. (default None) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for bootstrap - random number generation. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + bootstrap random number generation. (default: None) num_threads (int): How many OpenMP threads to use during the calculation. (default: use the number of cpu cores) @@ -830,7 +833,8 @@ def _make_expanded_patch(self, cat1, cat2, metric, low_mem): cat2e.name = cat1.name + " (expanded)" return cat2e - def _single_process12(self, c1, c2, ij, metric, num_threads, corr_only, temp, force_write=False): + def _single_process12(self, c1, c2, ij, metric, num_threads, corr_only, temp, + force_write=False): # Helper function for _process_all_auto and _process_cross for doing cross pairs. temp._clear() if c2 is not None and not self._trivially_zero(c1, c2): @@ -860,7 +864,7 @@ def is_my_job(my_indices, i, j, n): # Now the tricky part. If using MPI, we need to divide up the jobs smartly. # The first point is to divvy up the auto jobs evenly. This is where most of the - # work is done, so we want those to be spreads as evenly as possibly across procs. + # work is done, so we want those to be spread as evenly as possible across procs. # Therefore, if both indices are mine, then do the job. # This reduces the number of catalogs this machine needs to load up. # If the auto i,i and j,j are both my job, then i and j are already being loaded @@ -1168,9 +1172,9 @@ def process(self, cat1, cat2=None, metric=None, num_threads=None, comm=None, low In addition to computing the correlation function, this function also computes a few ancillary quantities that are useful for interpreting the resulting correlation function, including the attributes ``meanr``, ``meanlogr`` and ``npairs``. For most - use cases these calculation impart negligible overhead to the calculation time. - The exception is `NNCorrelation`, where they can result in somthing like 20-30% - overhead in the compute time. So if you want to optimize the efficiency of the + use cases these calculations impart negligible overhead. + The exception is `NNCorrelation`, where they can result in something like 20-30% + higher computation time. So if you want to optimize the efficiency of the calculation, we provide the option ``corr_only=True``, which skips these computations. In this case the resulting ``meanr`` and ``meanlogr`` attributes are the nominal centers of the bins, not the actual mean values. And ``npairs`` is estimated from @@ -1186,7 +1190,8 @@ def process(self, cat1, cat2=None, metric=None, num_threads=None, comm=None, low num_threads (int): How many OpenMP threads to use during the calculation. (default: use the number of cpu cores; this value can also be given in the constructor in the config dict.) - comm (mpi4py.Comm): If running MPI, an mpi4py Comm object to communicate between + comm (:class:`mpi4py.MPI.Comm`): + If running MPI, a ``Comm`` object to communicate between processes. If used, the rank=0 process will have the final computation. This only works if using patches. (default: None) low_mem (bool): Whether to sacrifice a little speed to try to reduce memory usage. @@ -1222,7 +1227,8 @@ def process(self, cat1, cat2=None, metric=None, num_threads=None, comm=None, low if cat2 is None: self._process_all_auto(cat1, metric, num_threads, corr_only, comm, low_mem, local) else: - self._process_all_cross(cat1, cat2, metric, num_threads, corr_only, comm, low_mem, local) + self._process_all_cross(cat1, cat2, metric, num_threads, corr_only, comm, low_mem, + local) self._processed_cats1.extend(cat1) if cat2 is not None: @@ -1404,14 +1410,14 @@ def estimate_cov(self, method, *, func=None, comm=None, num_bootstrap=None, leads to a different weight for pairs that cross between two selected patches. This option is only valid for bootstrap. - 'match' = Use the "optimal" weight that matches the effect of auto- and cross-pairs - in the estimate of the jackknife covariance. MP22 calculate this for the to + in the estimate of the jackknife covariance. MP22 calculate this to be w = 1 - n_patch / (2 + sqrt(2) (n_patch-1)). This option is only valid for jackknife. .. note:: For most classes, there is only a single statistic, so this calculates a covariance - matrix for that vector. `GGCorrelation` other complex auto-correaltions have two: + matrix for that vector. `GGCorrelation` and other complex auto-correlations have two: ``xip`` and ``xim``, so in this case the full data vector is ``xip`` followed by ``xim``, and this calculates the covariance matrix for that full vector including both statistics. The helper function `getStat` returns the relevant statistic in all @@ -1450,7 +1456,8 @@ def estimate_cov(self, method, *, func=None, comm=None, num_bootstrap=None, func (function): A unary function that acts on the current correlation object and returns the desired data vector. (default: None, which is equivalent to ``lambda corr: corr.getStat()``.) - comm (mpi4py.Comm): If using MPI, an mpi4py Comm object to communicate between + comm (:class:`mpi4py.MPI.Comm`): + If using MPI, a ``Comm`` object to communicate between processes. (default: None) num_bootstrap (int): How many bootstrap samples to use for the 'bootstrap' and 'marked_bootstrap' var_methods. (default: 500; this value @@ -1482,7 +1489,7 @@ def build_cov_design_matrix(self, method, *, func=None, comm=None, corresponds to a different estimate of the data vector, :math:`\\xi_i` (or :math:`f(\\xi_i)` if using the optional ``func`` parameter). - The different of rows in the matrix for each valid ``method`` are: + The different rows in the matrix for each valid ``method`` are: - 'shot': This method is not valid here. - 'jackknife': The data vector when excluding a single patch. @@ -1503,10 +1510,11 @@ def build_cov_design_matrix(self, method, *, func=None, comm=None, Parameters: method (str): Which method to use to estimate the covariance matrix. - func (function): A unary function that takes the list ``corrs`` and returns the - desired full data vector. (default: None, which is equivalent to - ``lambda corrs: np.concatenate([c.getStat() for c in corrs])``) - comm (mpi4py.Comm): If using MPI, an mpi4py Comm object to communicate between + func (function): A unary function that acts on the current correlation object and + returns the desired data vector. (default: None, which is + equivalent to ``lambda corr: corr.getStat()``) + comm (:class:`mpi4py.MPI.Comm`): + If using MPI, a ``Comm`` object to communicate between processes. (default: None) num_bootstrap (int): How many bootstrap samples to use for the 'bootstrap' and 'marked_bootstrap' var_methods. (default: 500; this value @@ -1665,9 +1673,9 @@ def sample_pairs(self, n, cat1, cat2, *, min_sep, max_sep, metric=None): Returns: Tuple containing - - i1 (array): indices of objects from cat1 - - i2 (array): indices of objects from cat2 - - sep (array): separations of the pairs of objects (i1,i2) + - i1 (:class:`numpy.ndarray`): indices of objects from cat1 + - i2 (:class:`numpy.ndarray`): indices of objects from cat2 + - sep (:class:`numpy.ndarray`): separations of the pairs of objects (i1,i2) """ if metric is None: metric = self.config.get('metric', 'Euclidean') @@ -2083,9 +2091,11 @@ def from_file(cls, file_name, *, file_type=None, logger=None, rng=None): file_name (str): The name of the file to read in. file_type (str): The type of file ('ASCII', 'FITS', or 'HDF'). (default: determine the type automatically from the extension of file_name.) - logger (Logger): If desired, a logger object to use for logging. (default: None) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for bootstrap - random number generation. (default: None) + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object to use for logging. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + bootstrap random number generation. (default: None) Returns: A Correlation object, constructed from the information in the file. @@ -2122,7 +2132,7 @@ def estimate_multi_cov(corrs, method, *, func=None, comm=None, num_bootstrap=Non cross_patch_weight=None): """Estimate the covariance matrix of multiple statistics. - This is like the method `Corr2.estimate_cov`, except that it will acoommodate + This is like the method `Corr2.estimate_cov`, except that it will accommodate multiple statistics from a list ``corrs`` of `Corr2` objects. Options for ``method`` include: @@ -2170,11 +2180,11 @@ def estimate_multi_cov(corrs, method, *, func=None, comm=None, num_bootstrap=Non leads to a different weight for pairs that cross between two selected patches. This option is only valid for bootstrap. - 'match' = Use the "optimal" weight that matches the effect of auto- and cross-pairs - in the estimate of the jackknife covariance. MP22 calculate this for the to + in the estimate of the jackknife covariance. MP22 calculate this to be w = 1 - n_patch / (2 + sqrt(2) (n_patch-1)). This option is only valid for jackknife. - For example, to find the combined covariance matrix for an NG tangential shear statistc, + For example, to find the combined covariance matrix for an NG tangential shear statistic, along with the GG xi+ and xi- from the same area, using jackknife covariance estimation, you would write:: @@ -2216,7 +2226,8 @@ def estimate_multi_cov(corrs, method, *, func=None, comm=None, num_bootstrap=Non func (function): A unary function that takes the list ``corrs`` and returns the desired full data vector. (default: None, which is equivalent to ``lambda corrs: np.concatenate([c.getStat() for c in corrs])``) - comm (mpi4py.Comm): If using MPI, an mpi4py Comm object to communicate between + comm (:class:`mpi4py.MPI.Comm`): + If using MPI, a ``Comm`` object to communicate between processes. (default: None) num_bootstrap (int): How many bootstrap samples to use for the 'bootstrap' and 'marked_bootstrap' var_methods. (default: 500) @@ -2254,7 +2265,7 @@ def build_multi_cov_design_matrix(corrs, method, *, func=None, comm=None, num_bo corresponds to a different estimate of the data vector, :math:`\\xi_i` (or :math:`f(\\xi_i)` if using the optional ``func`` parameter). - The different of rows in the matrix for each valid ``method`` are: + The different rows in the matrix for each valid ``method`` are: - 'shot': This method is not valid here. - 'jackknife': The data vector when excluding a single patch. @@ -2278,7 +2289,8 @@ def build_multi_cov_design_matrix(corrs, method, *, func=None, comm=None, num_bo func (function): A unary function that takes the list ``corrs`` and returns the desired full data vector. (default: None, which is equivalent to ``lambda corrs: np.concatenate([c.getStat() for c in corrs])``) - comm (mpi4py.Comm): If using MPI, an mpi4py Comm object to communicate between + comm (:class:`mpi4py.MPI.Comm`): + If using MPI, a ``Comm`` object to communicate between processes. (default: None) num_bootstrap (int): How many bootstrap samples to use for the 'bootstrap' and 'marked_bootstrap' var_methods. (default: 500) @@ -2313,7 +2325,7 @@ def _make_cov_design_matrix_core(corrs, plist, func, name, rank=0, size=1): # We aggregate and finalize each correlation function based on those pairs, and then call # the function func on that list of correlation objects. This is the data vector for # each row in the design matrix. - # We also make a parallel array of the total weight in each row in case the calling routing + # We also make a parallel array of the total weight in each row in case the calling routine # needs it. So far, only sample uses the returned w, but it's very little overhead to compute # it, and only a small memory overhead to build that array and return it. diff --git a/treecorr/corr3base.py b/treecorr/corr3base.py index 5266c149..123f78e5 100644 --- a/treecorr/corr3base.py +++ b/treecorr/corr3base.py @@ -47,7 +47,7 @@ class Corr3(object): Three-point correlations are a bit more complicated than two-point, since the data need to be binned in triangles, not just the separation between two points. - There are currenlty three different ways to quantify the triangle shapes. + There are currently three different ways to quantify the triangle shapes. 1. The triangle can be defined by its three side lengths (i.e. SSS congruence). In this case, we characterize the triangles according to the following three parameters @@ -141,7 +141,7 @@ class Corr3(object): See `Metrics` for more information about these various metric options. - There are three allowed value for the ``bin_type`` for three-point correlations. + There are three allowed values for the ``bin_type`` for three-point correlations. - 'LogRUV' uses the SSS description given above converted to r,u,v. The bin steps will be uniform in log(r) from log(min_sep) .. log(max_sep). The u and v values are binned @@ -250,15 +250,16 @@ class Corr3(object): .. note:: If you separate out the steps of the `process` command and use `process_auto` - and/or `process_cross`, then the units will not be applied to ``meanr`` or - ``meanlogr`` until the ``finalize`` function is called. + and/or `process_cross`, then the units will not be applied to the ``meand*`` and + ``meanlogd*`` attributes until the ``finalize`` function is called. Parameters: config (dict): A configuration dict that can be used to pass in the below kwargs if - desired. This dict is allowed to have addition entries in addition - to those listed below, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case - one will be built according to the config dict's verbose level.) + desired. This dict is allowed to have additional entries beyond + those listed below, which are ignored here. (default: None) + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which + case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -286,10 +287,10 @@ class Corr3(object): 3-d or flat 2-d positions, the default will just match the units of x,y[,z] coordinates) bin_slop (float): How much slop to allow in the placement of triangles in the bins. - If bin_slop = 1, then the bin into which a particular pair is placed + If bin_slop = 1, then the bin into which a particular triangle is placed may be incorrect by at most 1.0 bin widths. (default: None, which means to use a bin_slop that gives a maximum error of 10% on any bin, - which has been found to yield good results for most application.) + which has been found to yield good results for most applications.) angle_slop (float): How much slop to allow in the angular direction. This works very similarly to bin_slop, but applies to the projection angle of a pair of cells. The projection angle for any two objects in a pair of cells @@ -309,8 +310,8 @@ class Corr3(object): nphi_bins (int): Analogous to nbins for the phi values when bin_type=LogSAS. (The default is to calculate from phi_bin_size = bin_size, min_phi = 0, - max_u = np.pi, but this can be overridden by specifying up to 3 of - these four parametes.) + max_phi = np.pi, but this can be overridden by specifying up to 3 of + these four parameters.) phi_bin_size (float): Analogous to bin_size for the phi values. (default: bin_size) min_phi (float): Analogous to min_sep for the phi values. (default: 0) max_phi (float): Analogous to max_sep for the phi values. (default: np.pi) @@ -324,15 +325,15 @@ class Corr3(object): nubins (int): Analogous to nbins for the u values when bin_type=LogRUV. (The default is to calculate from ubin_size = bin_size, min_u = 0, max_u = 1, but - this can be overridden by specifying up to 3 of these four parametes.) + this can be overridden by specifying up to 3 of these four parameters.) ubin_size (float): Analogous to bin_size for the u values. (default: bin_size) min_u (float): Analogous to min_sep for the u values. (default: 0) max_u (float): Analogous to max_sep for the u values. (default: 1) - nvbins (int): Analogous to nbins for the positive v values when bin__type=LogRUV. + nvbins (int): Analogous to nbins for the positive v values when bin_type=LogRUV. (The default is to calculate from vbin_size = bin_size, min_v = 0, max_v = 1, but this can be overridden by specifying up to 3 of these - four parametes.) + four parameters.) vbin_size (float): Analogous to bin_size for the v values. (default: bin_size) min_v (float): Analogous to min_sep for the positive v values. (default: 0) max_v (float): Analogous to max_sep for the positive v values. (default: 1) @@ -347,7 +348,7 @@ class Corr3(object): log_file (str): If no logger is provided, this will specify a file to write the logging output. (default: None; i.e. output to standard output) - output_dots (bool): Whether to output progress dots during the calcualtion of the + output_dots (bool): Whether to output progress dots during the calculation of the correlation function. (default: False unless verbose is given and >= 2, in which case True) @@ -393,11 +394,12 @@ class Corr3(object): (default: 'shot') num_bootstrap (int): How many bootstrap samples to use for the 'bootstrap' and 'marked_bootstrap' var_methods. (default: 500) - cross_patch_weight (str): How to weight pairs that cross between two patches when one patch - is deselected (e.g. in a jackknife sense) and the other is selected. - (default None) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for bootstrap - random number generation. (default: None) + cross_patch_weight (str): How to weight triangles that cross between two patches when + one patch is deselected (e.g. in a jackknife sense) and the other + is selected. (default None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + bootstrap random number generation. (default: None) num_threads (int): How many OpenMP threads to use during the calculation. (default: use the number of cpu cores; this value can also be given in @@ -504,7 +506,7 @@ class Corr3(object): 'How many bootstrap samples to use for the var_method=bootstrap and ' 'marked_bootstrap'), 'cross_patch_weight': (str, False, None, ['simple', 'mean', 'match', 'geom'], - 'How to weight pairs that cross between a selected and unselected patch'), + 'How to weight triangles that cross between selected and unselected patches'), 'num_threads' : (int, False, None, None, 'How many threads should be used. num_threads <= 0 means auto based on num cores.'), } @@ -719,7 +721,8 @@ def __init__(self, config=None, *, logger=None, rng=None, **kwargs): self._ro.bin_slop = 1.0 self._ro.b = self.bin_size else: - self._ro.bin_slop = 0.1/self.bin_size # The stored bin_slop corresponds to lnr bins. + # The stored bin_slop corresponds to lnr bins. + self._ro.bin_slop = 0.1/self.bin_size self._ro.b = 0.1 if self.bin_type == 'LogRUV': if self.ubin_size <= 0.1: @@ -1359,7 +1362,7 @@ def is_my_job(my_indices, i, j, k, n): # Now the tricky part. If using MPI, we need to divide up the jobs smartly. # The first point is to divvy up the auto jobs evenly. This is where most of the - # work is done, so we want those to be spreads as evenly as possibly across procs. + # work is done, so we want those to be spread as evenly as possible across procs. # Therefore, if all indices are mine, then do the job. # This reduces the number of catalogs this machine needs to load up. n1 = np.sum([i in my_indices, j in my_indices, k in my_indices]) @@ -1553,7 +1556,8 @@ def is_my_job(my_indices, i, j, k, n1, n2): and self._letter1 == self._letter2): c1e = self._make_expanded_patch(c2, cat1, metric, low_mem) c2e = self._make_expanded_patch(c2, cat2, metric, low_mem) - self.logger.info('Process patch %d from cat2 with surrounding local patches',i) + self.logger.info( + 'Process patch %d from cat2 with surrounding local patches', i) self._single_process123(c2, c1e, c2e, (i,i,i), metric, 1, num_threads, corr_only, temp, True) if low_mem: @@ -1790,7 +1794,8 @@ def is_my_job(my_indices, i, j, k, n1, n2, n3): and self._letter1 == self._letter2): c1e = self._make_expanded_patch(c2, cat1, metric, low_mem) c3e = self._make_expanded_patch(c2, cat3, metric, low_mem) - self.logger.info('Process patch %d from cat2 with surrounding local patches',i) + self.logger.info( + 'Process patch %d from cat2 with surrounding local patches', i) self._single_process123(c2, c1e, c3e, (i,i,i), metric, 1 if ordered == 0 else 4, num_threads, corr_only, temp, True) @@ -1803,7 +1808,8 @@ def is_my_job(my_indices, i, j, k, n1, n2, n3): and self._letter1 == self._letter3): c1e = self._make_expanded_patch(c3, cat1, metric, low_mem) c2e = self._make_expanded_patch(c3, cat2, metric, low_mem) - self.logger.info('Process patch %d from cat3 with surrounding local patches',i) + self.logger.info( + 'Process patch %d from cat3 with surrounding local patches', i) self._single_process123(c3, c2e, c1e, (i,i,i), metric, 1 if ordered == 0 else 4, num_threads, corr_only, temp, True) @@ -1911,7 +1917,7 @@ def process_cross12(self, cat1, cat2, *, metric=None, ordered=True, num_threads= This accumulates the cross-correlation for the given catalogs as part of a larger auto- or cross-correlation calculation. E.g. when splitting up a large catalog into - patches, this is appropriate to use for the cross correlation between different patches + patches, this is appropriate to use for the cross-correlation between different patches as part of the complete auto-correlation of the full catalog. This method is only valid for classes that have the same type of value in vertices @@ -1970,7 +1976,7 @@ def process_cross21(self, cat1, cat2, *, metric=None, ordered=True, num_threads= This accumulates the cross-correlation for the given catalogs as part of a larger auto- or cross-correlation calculation. E.g. when splitting up a large catalog into - patches, this is appropriate to use for the cross correlation between different patches + patches, this is appropriate to use for the cross-correlation between different patches as part of the complete auto-correlation of the full catalog. This method is only valid for classes that have the same type of value in vertices @@ -2028,7 +2034,7 @@ def process_cross(self, cat1, cat2, cat3, *, metric=None, ordered=True, num_thre This accumulates the cross-correlation for the given catalogs as part of a larger auto- or cross-correlation calculation. E.g. when splitting up a large catalog into - patches, this is appropriate to use for the cross correlation between different patches + patches, this is appropriate to use for the cross-correlation between different patches as part of the complete auto-correlation of the full catalog. Parameters: @@ -2093,7 +2099,7 @@ def process(self, cat1, cat2=None, cat3=None, *, metric=None, ordered=True, num_ .. note:: - For cross correlations where the third field type is different from the other two + For cross-correlations where the third field type is different from the other two (e.g. KKG, NNG, etc.) then the 2 argument version will use the first catalog for first two vertices and the second for the third vertex, since that's the only valid combination for those correlation types. @@ -2102,10 +2108,10 @@ def process(self, cat1, cat2=None, cat3=None, *, metric=None, ordered=True, num_ except it will be slightly more efficient, since it knows the first two vertices are from a single field. - For cross correlations, the default behavior is to use cat1 for the first vertex (P1), + For cross-correlations, the default behavior is to use cat1 for the first vertex (P1), cat2 for the second vertex (P2), and cat3 for the third vertex (P3). If only two catalogs are given, vertices P2 and P3 both come from cat2. The sides d1, d2, d3, - used to define the binning, are taken to be opposte P1, P2, P3 respectively. + used to define the binning, are taken to be opposite P1, P2, P3 respectively. However, if you want to accumulate triangles where objects from each catalog can take any position in the triangles, you can set ``ordered=False``. In this case, triangles @@ -2147,7 +2153,8 @@ def process(self, cat1, cat2=None, cat3=None, *, metric=None, ordered=True, num_ num_threads (int): How many OpenMP threads to use during the calculation. (default: use the number of cpu cores; this value can also be given in the constructor in the config dict.) - comm (mpi4py.Comm): If running MPI, an mpi4py Comm object to communicate between + comm (:class:`mpi4py.MPI.Comm`): + If running MPI, a ``Comm`` object to communicate between processes. If used, the rank=0 process will have the final computation. This only works if using patches. (default: None) low_mem (bool): Whether to sacrifice a little speed to try to reduce memory usage. @@ -2161,7 +2168,7 @@ def process(self, cat1, cat2=None, cat3=None, *, metric=None, ordered=True, num_ algo (str): Which accumulation algorithm to use. (options are 'triangle' or 'multipole'; default is 'multipole' unless bin_type is 'LogRUV', which can only use 'triangle') cf. `Three-point Algorithm`. - max_n (int): If using the multpole algorithm, and this is not directly using + max_n (int): If using the multipole algorithm, and this is not directly using bin_type='LogMultipole', then this is the value of max_n to use for the multipole part of the calculation. (default is to use 2pi/phi_bin_size; this value can also be given in the constructor @@ -2222,7 +2229,8 @@ def process(self, cat1, cat2=None, cat3=None, *, metric=None, ordered=True, num_ if cat2 is None: if not self._letter1 == self._letter2 == self._letter3: - raise ValueError("{} cannot use one catalog version of process".format(self._letters)) + raise ValueError( + "{} cannot use one catalog version of process".format(self._letters)) if cat3 is not None: raise ValueError("For two catalog case, use cat1,cat2, not cat1,cat3") if not (ordered is True or ordered is False): @@ -2238,7 +2246,8 @@ def process(self, cat1, cat2=None, cat3=None, *, metric=None, ordered=True, num_ self._process_all_cross21(cat1, cat2, metric, ordered, num_threads, corr_only, comm, low_mem, local) else: - raise ValueError("{} cannot use two catalog version of process".format(self._letters)) + raise ValueError( + "{} cannot use two catalog version of process".format(self._letters)) else: self._process_all_cross(cat1, cat2, cat3, metric, ordered, num_threads, corr_only, comm, low_mem, local) @@ -2549,7 +2558,7 @@ def toSAS(self, *, target=None, **kwargs): Keyword Arguments: target: A target Correlation object with LogSAS binning to write to. If this is not given, a new object will be created based on the - configuration paramters of the current object. (default: None) + configuration parameters of the current object. (default: None) **kwargs: Any kwargs that you want to use to configure the returned object. Typically, might include min_phi, max_phi, nphi_bins, phi_bin_size. The default phi binning is [0,pi] with nphi_bins = self.max_n. @@ -2558,7 +2567,7 @@ def toSAS(self, *, target=None, **kwargs): An object with bin_type=LogSAS containing the same information as this object, but with the SAS binning. """ - # Each class will add a bit to this. The implemenation here is the common code + # Each class will add a bit to this. The implementation here is the common code # that applies to all the different classes. if self.bin_type != 'LogMultipole': @@ -2699,8 +2708,8 @@ def estimate_cov(self, method, *, func=None, comm=None, num_bootstrap=None, triangles that cross between two or three patches are weighted when some, but not all of the patches are selected. See Mohammad and Percival (2021) (https://arxiv.org/abs/2109.07071) for an in-depth discussion of these options for - two-point statistics. We use a similar definitions for three-point statistics. - Briefly the options are: (TODO! This is aspirational so far.) + two-point statistics. We use a similar set of definitions for three-point statistics. + Briefly the options are: - 'simple' = Don't use any triangles where any object is in a deselected patch. This is currently the default for all methods. @@ -2756,15 +2765,15 @@ def estimate_cov(self, method, *, func=None, comm=None, num_bootstrap=None, func (function): A unary function that acts on the current correlation object and returns the desired data vector. (default: None, which is equivalent to ``lambda corr: corr.getStat()``) - comm (mpi4py.Comm): If using MPI, an mpi4py Comm object to communicate between + comm (:class:`mpi4py.MPI.Comm`): + If using MPI, a ``Comm`` object to communicate between processes. (default: None) num_bootstrap (int): How many bootstrap samples to use for the 'bootstrap' and 'marked_bootstrap' var_methods. (default: 500; this value can also be given in the constructor.) - cross_patch_weight (str): How to weight pairs that cross between two patches when one - patch is deselected (e.g. in a jackknife sense) and the other is - selected. (default 'simple'; this value can also be given in - the constructor.) + cross_patch_weight (str): How to weight triangles that span selected and + deselected patches. (default 'simple'; this value can also be + given in the constructor.) Returns: A numpy array with the estimated covariance matrix. @@ -2788,7 +2797,7 @@ def build_cov_design_matrix(self, method, *, func=None, comm=None, num_bootstrap corresponds to a different estimate of the data vector, :math:`\zeta_i` (or :math:`f(\zeta_i)` if using the optional ``func`` parameter). - The different of rows in the matrix for each valid ``method`` are: + The different rows in the matrix for each valid ``method`` are: - 'shot': This method is not valid here. - 'jackknife': The data vector when excluding a single patch. @@ -2809,18 +2818,18 @@ def build_cov_design_matrix(self, method, *, func=None, comm=None, num_bootstrap Parameters: method (str): Which method to use to estimate the covariance matrix. - func (function): A unary function that takes the list ``corrs`` and returns the - desired full data vector. (default: None, which is equivalent to - ``lambda corrs: np.concatenate([c.getStat() for c in corrs])``) - comm (mpi4py.Comm): If using MPI, an mpi4py Comm object to communicate between + func (function): A unary function that acts on the current correlation object and + returns the desired data vector. (default: None, which is + equivalent to ``lambda corr: corr.getStat()``) + comm (:class:`mpi4py.MPI.Comm`): + If using MPI, a ``Comm`` object to communicate between processes. (default: None) num_bootstrap (int): How many bootstrap samples to use for the 'bootstrap' and 'marked_bootstrap' var_methods. (default: 500; this value can also be given in the constructor.) - cross_patch_weight (str): How to weight pairs that cross between two patches when one - patch is deselected (e.g. in a jackknife sense) and the other is - selected. (default 'simple'; this value can also be given in the - constructor.) + cross_patch_weight (str): How to weight triangles that span selected and + deselected patches. (default 'simple'; this value can also be + given in the constructor.) Returns: A, w: numpy arrays with the design matrix and weights respectively. @@ -3509,7 +3518,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result file_name (str): The name of the file to write to. file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) @@ -3534,7 +3543,8 @@ def _read(self, reader, name=None, params=None): # Version 5.0 used weight_re, weight_im. These are now weightr, weighti. # Fix in place to keep backwards compatibility. if 'weight_re' in data.dtype.names: - dt = np.dtype([(n.replace('_re','r').replace('_im','i'),t) for (n,t) in data.dtype.descr]) + dt = np.dtype([(n.replace('_re','r').replace('_im','i'), t) + for (n, t) in data.dtype.descr]) data = data.astype(dt) # This helper function defines how to set the attributes for each class @@ -3629,9 +3639,11 @@ def from_file(cls, file_name, *, file_type=None, logger=None, rng=None): file_name (str): The name of the file to read in. file_type (str): The type of file ('ASCII', 'FITS', or 'HDF'). (default: determine the type automatically from the extension of file_name.) - logger (Logger): If desired, a logger object to use for logging. (default: None) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for bootstrap - random number generation. (default: None) + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object to use for logging. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + bootstrap random number generation. (default: None) Returns: A Correlation object, constructed from the information in the file. diff --git a/treecorr/exec_corr2.py b/treecorr/exec_corr2.py index f197c74d..00945130 100644 --- a/treecorr/exec_corr2.py +++ b/treecorr/exec_corr2.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: corr2ex +.. module:: exec_corr2 """ from .catalog import Catalog, read_catalogs @@ -71,7 +71,7 @@ 'nn_file_name' : (str, False, None, None, 'The output filename for point-point correlation function.'), 'nn_statistic' : (str, False, 'compensated', ['compensated','simple'], - 'Which statistic to use for omega as the estimator fo the NN correlation function. '), + 'Which statistic to use for omega as the estimator for the NN correlation function. '), 'nk_file_name' : (str, False, None, None, 'The output filename for count-scalar correlation function.'), 'nk_statistic' : (str, False, None, ['compensated', 'simple'], @@ -154,11 +154,11 @@ def corr2(config, logger=None): The function `print_corr2_params` will output information about the valid parameters that are expected to be in the config dict. - Optionally a logger parameter maybe given, in which case it is used for logging. + Optionally a logger parameter may be given, in which case it is used for logging. If not given, the logging will be based on the verbose and log_file parameters. :param config: The configuration dict which defines what to do. - :param logger: If desired, a logger object for logging. (default: None, in which case + :param logger: If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) """ # Setup logger based on config verbose value diff --git a/treecorr/exec_corr3.py b/treecorr/exec_corr3.py index 09a72b30..75158559 100644 --- a/treecorr/exec_corr3.py +++ b/treecorr/exec_corr3.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: corr3 +.. module:: exec_corr3 """ from .catalog import Catalog, read_catalogs @@ -39,7 +39,7 @@ # description corr3_valid_params = { - # Parameters about the input catlogs + # Parameters about the input catalogs 'file_name' : (str, True, None, None, 'The file(s) with the galaxy data.'), @@ -57,9 +57,9 @@ # Parameters about the output file(s) 'nnn_file_name' : (str, False, None, None, - 'The output filename for point-point correlation function.'), + 'The output filename for count-count-count correlation function.'), 'nnn_statistic' : (str, False, 'compensated', ['compensated','simple'], - 'Which statistic to use for omega as the estimator fo the NN correlation function. '), + 'Which statistic to use for omega as the estimator for the NNN correlation function. '), 'kkk_file_name' : (str, False, None, None, 'The output filename for scalar-scalar-scalar correlation function.'), 'ggg_file_name' : (str, False, None, None, @@ -128,11 +128,11 @@ def corr3(config, logger=None): The function `print_corr3_params` will output information about the valid parameters that are expected to be in the config dict. - Optionally a logger parameter maybe given, in which case it is used for logging. + Optionally a logger parameter may be given, in which case it is used for logging. If not given, the logging will be based on the verbose and log_file parameters. :param config: The configuration dict which defines what to do. - :param logger: If desired, a logger object for logging. (default: None, in which case + :param logger: If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) """ # Setup logger based on config verbose value @@ -197,7 +197,8 @@ def corr3(config, logger=None): logger.warning("No random catalogs given. Only doing ntri calculation.") rrr = None else: - # TODO: I haven't don't all the options correctly with cat2, cat3 yet. + # Note: random handling here is currently only implemented for the primary + # data catalog in the NNN workflow. logger.warning("Performing RRR calculations...") rrr = NNNCorrelation(config, logger=logger) rrr.process(rand1) @@ -225,7 +226,8 @@ def corr3(config, logger=None): logger.warning("Wrote KKK correlation to %s",config['kkk_file_name']) # Do NNK correlation function if necessary - # TODO: randoms on all cross correlations that include N + # Note: random-catalog support is not implemented here for all cross-correlations + # that include N. if 'nnk_file_name' in config: logger.warning("Performing NNK calculations...") nnk = NNKCorrelation(config, logger=logger) diff --git a/treecorr/field.py b/treecorr/field.py index 6946beaa..5d9f1790 100644 --- a/treecorr/field.py +++ b/treecorr/field.py @@ -146,10 +146,14 @@ def count_near(self, *args, **kwargs): 3. For spherical coordinates: Parameters: - ra (float or Angle): The right ascension of the target location - dec (float or Angle): The declination of the target location - c (CelestialCorod): A ``coord.CelestialCoord`` object in lieu of (ra, dec) - sep (float or Angle): The separation distance + ra (float or :class:`coord.Angle`): + The right ascension of the target location + dec (float or :class:`coord.Angle`): + The declination of the target location + c (:class:`coord.CelestialCoord`): + A ``CelestialCoord`` object in lieu of (ra, dec) + sep (float or :class:`coord.Angle`): + The separation distance ra_units (str): The units of ra if given as a float dec_units (str): The units of dec if given as a float sep_units (str): The units of sep if given as a float @@ -157,9 +161,12 @@ def count_near(self, *args, **kwargs): 4. For spherical coordinates with distances: Parameters: - ra (float or Angle): The right ascension of the target location - dec (float or Angle): The declination of the target location - c (CelestialCorod): A ``coord.CelestialCoord`` object in lieu of (ra, dec) + ra (float or :class:`coord.Angle`): + The right ascension of the target location + dec (float or :class:`coord.Angle`): + The declination of the target location + c (:class:`coord.CelestialCoord`): + A ``CelestialCoord`` object in lieu of (ra, dec) r (float): The distance to the target location sep (float): The separation distance ra_units (str): The units of ra if given as a float @@ -170,7 +177,7 @@ def count_near(self, *args, **kwargs): or sep_units respectively to specify which angular units are providing. Finally, in cases where ra, dec are allowed, you may instead provide a - ``coord.CelestialCoord`` instance as the first argument to specify both RA and Dec. + :class:`coord.CelestialCoord` instance as the first argument to specify both RA and Dec. """ if self.min_size == 0: # If min_size = 0, then regular method is already exact. @@ -214,10 +221,14 @@ def get_near(self, *args, **kwargs): 3. For spherical coordinates: Parameters: - ra (float or Angle): The right ascension of the target location - dec (float or Angle): The declination of the target location - c (CelestialCorod): A ``coord.CelestialCoord`` object in lieu of (ra, dec) - sep (float or Angle): The separation distance + ra (float or :class:`coord.Angle`): + The right ascension of the target location + dec (float or :class:`coord.Angle`): + The declination of the target location + c (:class:`coord.CelestialCoord`): + A ``CelestialCoord`` object in lieu of (ra, dec) + sep (float or :class:`coord.Angle`): + The separation distance ra_units (str): The units of ra if given as a float dec_units (str): The units of dec if given as a float sep_units (str): The units of sep if given as a float @@ -225,9 +236,12 @@ def get_near(self, *args, **kwargs): 4. For spherical coordinates with distances: Parameters: - ra (float or Angle): The right ascension of the target location - dec (float or Angle): The declination of the target location - c (CelestialCorod): A ``coord.CelestialCoord`` object in lieu of (ra, dec) + ra (float or :class:`coord.Angle`): + The right ascension of the target location + dec (float or :class:`coord.Angle`): + The declination of the target location + c (:class:`coord.CelestialCoord`): + A ``CelestialCoord`` object in lieu of (ra, dec) r (float): The distance to the target location sep (float): The separation distance ra_units (str): The units of ra if given as a float @@ -238,7 +252,7 @@ def get_near(self, *args, **kwargs): or sep_units respectively to specify which angular units are providing. Finally, in cases where ra, dec are allowed, you may instead provide a - ``coord.CelestialCoord`` instance as the first argument to specify both RA and Dec. + :class:`coord.CelestialCoord` instance as the first argument to specify both RA and Dec. """ x,y,z,sep = parse_xyzsep(args, kwargs, self._coords) if self.min_size == 0: @@ -309,7 +323,7 @@ def run_kmeans(self, npatch, *, max_iter=200, tol=1.e-5, init='tree', alt=False, can produce slightly better patches for the purpose of patch-based covariance estimation. The ideal patch definition for such use would be to minimize the standard deviation (std) of the inertia of each patch, not the total (or mean) inertia. It turns out that it is - difficult to devise an algorithm that literally does this, since it has a tendancy to + difficult to devise an algorithm that literally does this, since it has a tendency to become unstable and not converge. However, adding a penalty term to the patch assignment step of the normal k-means @@ -360,22 +374,25 @@ def run_kmeans(self, npatch, *, max_iter=200, tol=1.e-5, init='tree', alt=False, field, traversing down to a level where there are npatch cells, and use the centroids of these cells as the initial centers. This is almost always the best choice. - - 'random' = Use npatch random points as the intial centers. + - 'random' = Use npatch random points as the initial centers. - 'kmeans++' = Use the k-means++ algorithm. cf. https://en.wikipedia.org/wiki/K-means%2B%2B alt (bool): Use the alternate assignment algorithm to minimize the standard deviation of the inertia rather than the total inertia (aka WCSS). (default: False) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for random - number generation. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + random number generation. (default: None) Returns: Tuple containing - - patches (array): An array of patch labels, all integers from 0..npatch-1. + - patches (:class:`numpy.ndarray`): An array of patch labels, + all integers from 0..npatch-1. Size is self.ntot. - - centers (array): An array of center coordinates used to make the patches. + - centers (:class:`numpy.ndarray`): An array of center coordinates + used to make the patches. Shape is (npatch, 2) for flat geometries or (npatch, 3) for 3d or spherical geometries. In the latter case, the centers represent (x,y,z) coordinates on the unit sphere. @@ -407,12 +424,13 @@ def kmeans_initialize_centers(self, npatch, init='tree', *, rng=None): field, traversing down to a level where there are npatch cells, and use the centroids of these cells as the initial centers. This is almost always the best choice. - - 'random' = Use npatch random points as the intial centers. + - 'random' = Use npatch random points as the initial centers. - 'kmeans++' = Use the k-means++ algorithm. cf. https://en.wikipedia.org/wiki/K-means%2B%2B - rng (RandomState): If desired, a numpy.random.RandomState instance to use for random - number generation. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + random number generation. (default: None) Returns: An array of center coordinates. @@ -473,7 +491,8 @@ def kmeans_refine_centers(self, centers, *, max_iter=200, tol=1.e-5, alt=False): the sizes at each step. Parameters: - centers (array): An array of center coordinates. (modified by this function) + centers (:class:`numpy.ndarray`): + An array of center coordinates. (modified by this function) Shape is (npatch, 2) for flat geometries or (npatch, 3) for 3d or spherical geometries. In the latter case, the centers represent (x,y,z) coordinates on the unit sphere. @@ -490,11 +509,12 @@ def kmeans_refine_centers(self, centers, *, max_iter=200, tol=1.e-5, alt=False): def kmeans_assign_patches(self, centers): """Assign patch numbers to each point according to the given centers. - This is final step in the full K-Means algorithm. It assignes patch numbers to each + This is the final step in the full K-Means algorithm. It assigns patch numbers to each point in the field according to which center is closest. Parameters: - centers (array): An array of center coordinates. + centers (:class:`numpy.ndarray`): + An array of center coordinates. Shape is (npatch, 2) for flat geometries or (npatch, 3) for 3d or spherical geometries. In the latter case, the centers represent (x,y,z) coordinates on the unit sphere. @@ -530,9 +550,11 @@ class NField(Field): max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default: cat.coords) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for random - number generation. (default: None) - logger (Logger): A logger file if desired. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + random number generation. (default: None) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired. (default: None) """ def __init__(self, cat, *, min_size=0, max_size=None, split_method='mean', brute=False, min_top=None, max_top=10, coords=None, rng=None, logger=None): @@ -593,9 +615,11 @@ class KField(Field): max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default: cat.coords) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for random - number generation. (default: None) - logger (Logger): A logger file if desired. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + random number generation. (default: None) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired. (default: None) """ def __init__(self, cat, *, min_size=0, max_size=None, split_method='mean', brute=False, min_top=None, max_top=10, coords=None, rng=None, logger=None): @@ -656,9 +680,11 @@ class ZField(Field): max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default: cat.coords) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for random - number generation. (default: None) - logger (Logger): A logger file if desired. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + random number generation. (default: None) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired. (default: None) """ def __init__(self, cat, *, min_size=0, max_size=None, split_method='mean', brute=False, min_top=None, max_top=10, coords=None, rng=None, logger=None): @@ -719,9 +745,11 @@ class VField(Field): max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default: cat.coords) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for random - number generation. (default: None) - logger (Logger): A logger file if desired. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + random number generation. (default: None) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired. (default: None) """ def __init__(self, cat, *, min_size=0, max_size=None, split_method='mean', brute=False, min_top=None, max_top=10, coords=None, rng=None, logger=None): @@ -782,9 +810,11 @@ class GField(Field): max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default: cat.coords) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for random - number generation. (default: None) - logger (Logger): A logger file if desired. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + random number generation. (default: None) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired. (default: None) """ def __init__(self, cat, *, min_size=0, max_size=None, split_method='mean', brute=False, min_top=None, max_top=10, coords=None, rng=None, logger=None): @@ -845,9 +875,11 @@ class TField(Field): max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default: cat.coords) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for random - number generation. (default: None) - logger (Logger): A logger file if desired. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + random number generation. (default: None) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired. (default: None) """ def __init__(self, cat, *, min_size=0, max_size=None, split_method='mean', brute=False, min_top=None, max_top=10, coords=None, rng=None, logger=None): @@ -907,9 +939,11 @@ class QField(Field): max_top (int): The maximum number of top layers to use when setting up the field. (default: 10) coords (str): The kind of coordinate system to use. (default: cat.coords) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for random - number generation. (default: None) - logger (Logger): A logger file if desired. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + random number generation. (default: None) + logger (:class:`logging.Logger`): + A ``Logger`` object if desired. (default: None) """ def __init__(self, cat, *, min_size=0, max_size=None, split_method='mean', brute=False, min_top=None, max_top=10, coords=None, rng=None, logger=None): diff --git a/treecorr/ggcorrelation.py b/treecorr/ggcorrelation.py index eefd4165..fb9acd44 100644 --- a/treecorr/ggcorrelation.py +++ b/treecorr/ggcorrelation.py @@ -27,10 +27,10 @@ class GGCorrelation(BaseZZCorrelation): r"""This class handles the calculation and storage of a 2-point shear-shear correlation function. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. - In addition to the attributes common to all `Corr3` subclasses, objects of this class + In addition to the attributes common to all `Corr2` subclasses, objects of this class hold the following attributes: Attributes: @@ -56,16 +56,19 @@ class GGCorrelation(BaseZZCorrelation): The typical usage pattern is as follows: >>> gg = treecorr.GGCorrelation(config) - >>> gg.process(cat) # For auto-correlation. - >>> gg.process(cat1,cat2) # For cross-correlation. - >>> gg.write(file_name) # Write out to a file. - >>> xip = gg.xip # Or access the correlation function directly. + >>> gg.process(cat) # Compute the auto-correlation. + >>> # gg.process(cat1, cat2) # ... or the cross-correlation. + >>> gg.write(file_name) # Write out to a file. + >>> xip, xim = gg.xip, gg.xim # Or access the correlation functions directly. + + See also: `NGCorrelation`, `NNCorrelation`, `GGGCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -139,8 +142,9 @@ def calculateMapSq(self, *, R=None, m2_uform=None): Parameters: - R (array): The R values at which to calculate the aperture mass statistics. - (default: None, which means use self.rnom) + R (:class:`numpy.ndarray`): + The R values at which to calculate the aperture mass statistics. + (default: None, which means to use self.rnom) m2_uform (str): Which form to use for the aperture mass, as described above. (default: 'Crittenden'; this value can also be given in the constructor in the config dict.) @@ -232,8 +236,9 @@ def calculateGamSq(self, *, R=None, eb=False): Parameters: - R (array): The R values at which to calculate the shear variance. - (default: None, which means use self.rnom) + R (:class:`numpy.ndarray`): + The R values at which to calculate the shear variance. + (default: None, which means to use self.rnom) eb (bool): Whether to include the E/B decomposition as well as the total :math:`\langle \gamma^2\rangle`. (default: False) @@ -297,9 +302,9 @@ def writeMapSq(self, file_name, *, R=None, m2_uform=None, file_type=None, precis Mapsq The real part of :math:`\langle M_{ap}^2\rangle` (cf. `calculateMapSq`) Mxsq The real part of :math:`\langle M_\times^2\rangle` - MMxa The imag part of :math:`\langle M_{ap}^2\rangle`: + MMxa The imaginary part of :math:`\langle M_{ap}^2\rangle`: an estimator of :math:`\langle M_{ap} M_\times\rangle` - MMxa The imag part of :math:`\langle M_\times^2\rangle`: + MMxb The imaginary part of :math:`\langle M_\times^2\rangle`: an estimator of :math:`\langle M_{ap} M_\times\rangle` sig_map The sqrt of the variance estimate of :math:`\langle M_{ap}^2\rangle` @@ -311,13 +316,14 @@ def writeMapSq(self, file_name, *, R=None, m2_uform=None, file_type=None, precis Parameters: file_name (str): The name of the file to write to. - R (array): The R values at which to calculate the statistics. - (default: None, which means use self.rnom) + R (:class:`numpy.ndarray`): + The R values at which to calculate the statistics. + (default: None, which means to use self.rnom) m2_uform (str): Which form to use for the aperture mass. (default: 'Crittenden'; this value can also be given in the constructor in the config dict.) file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) """ self.logger.info('Writing Map^2 from GG correlations to %s',file_name) diff --git a/treecorr/gggcorrelation.py b/treecorr/gggcorrelation.py index e5bc33a1..531a60b3 100644 --- a/treecorr/gggcorrelation.py +++ b/treecorr/gggcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: nnncorrelation +.. module:: gggcorrelation """ import numpy as np @@ -35,8 +35,8 @@ class GGGCorrelation(Corr3): this definition. It is also a fairly simple point to calculate in the code compared to some of the other options they offer, so projections relative to it are fairly efficient. - There are 4 complex-valued 3-point shear corrletion functions defined for triples of shear - values projected relative to the line joining the location of the shear to the cenroid of + There are 4 complex-valued 3-point shear correlation functions defined for triples of shear + values projected relative to the line joining the location of the shear to the centroid of the triangle: .. math:: @@ -46,11 +46,11 @@ class GGGCorrelation(Corr3): \Gamma_2 &= \langle \gamma(\mathbf{x1}) \gamma(\mathbf{x2})^* \gamma(\mathbf{x3}) \rangle \\ \Gamma_3 &= \langle \gamma(\mathbf{x1}) \gamma(\mathbf{x2}) \gamma(\mathbf{x3})^* \rangle \\ - where :math:`\mathbf{x1}, \mathbf{x2}, \mathbf{x3}` are the corners of the triange opposite + where :math:`\mathbf{x1}, \mathbf{x2}, \mathbf{x3}` are the corners of the triangle opposite sides d1, d2, d3 respectively, where d1 > d2 > d3, and :math:`{}^*` indicates complex conjugation. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -61,29 +61,30 @@ class GGGCorrelation(Corr3): gam1: The 1st "natural" correlation function, :math:`\Gamma_1`. gam2: The 2nd "natural" correlation function, :math:`\Gamma_2`. gam3: The 3rd "natural" correlation function, :math:`\Gamma_3`. - vargam0: The variance of :math:`\Gamma_0`, only including the shot noise - propagated into the final correlation. This (and the related values for - 1,2,3) does not include sample variance, so it is always an underestimate - of the actual variance. - vargam1: The variance of :math:`\Gamma_1`. - vargam2: The variance of :math:`\Gamma_2`. - vargam3: The variance of :math:`\Gamma_3`. + vargam0: The variance estimate of :math:`\Gamma_0`, computed according to ``var_method`` + (default: ``'shot'``). + vargam1: The variance estimate of :math:`\Gamma_1`. + vargam2: The variance estimate of :math:`\Gamma_2`. + vargam3: The variance estimate of :math:`\Gamma_3`. The typical usage pattern is as follows:: >>> ggg = treecorr.GGGCorrelation(config) - >>> ggg.process(cat) # Compute auto-correlation. - >>> ggg.process(cat1, cat2, cat3) # Compute cross-correlation. - >>> ggg.write(file_name) # Write out to a file. - >>> gam0 = ggg.gam0, etc. # Access gamma values directly. - >>> gam0r = ggg.gam0r # Or access real and imag parts separately. + >>> ggg.process(cat) # Compute the auto-correlation. + >>> # ggg.process(cat1, cat2, cat3) # ... or the cross-correlation. + >>> ggg.write(file_name) # Write out to a file. + >>> gam0 = ggg.gam0, etc. # Access gamma values directly. + >>> gam0r = ggg.gam0r # Or access real and imaginary parts separately. >>> gam0i = ggg.gam0i + See also: `KGGCorrelation`, `NGGCorrelation`, `GGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -228,13 +229,13 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" gam0r The real part of the estimator of :math:`\Gamma_0` - gam0i The imag part of the estimator of :math:`\Gamma_0` + gam0i The imaginary part of the estimator of :math:`\Gamma_0` gam1r The real part of the estimator of :math:`\Gamma_1` - gam1i The imag part of the estimator of :math:`\Gamma_1` + gam1i The imaginary part of the estimator of :math:`\Gamma_1` gam2r The real part of the estimator of :math:`\Gamma_2` - gam2i The imag part of the estimator of :math:`\Gamma_2` + gam2i The imaginary part of the estimator of :math:`\Gamma_2` gam3r The real part of the estimator of :math:`\Gamma_3` - gam3i The imag part of the estimator of :math:`\Gamma_3` + gam3i The imaginary part of the estimator of :math:`\Gamma_3` sigma_gam0 The sqrt of the variance estimate of :math:`\Gamma_0` sigma_gam1 The sqrt of the variance estimate of :math:`\Gamma_1` sigma_gam2 The sqrt of the variance estimate of :math:`\Gamma_2` @@ -413,8 +414,9 @@ def calculateMap3(self, *, R=None, k2=1, k3=1): ``m2_uform`` = 'Crittenden'. Parameters: - R (array): The R values at which to calculate the aperture mass statistics. - (default: None, which means use self.rnom1d) + R (:class:`numpy.ndarray`): + The R values at which to calculate the aperture mass statistics. + (default: None, which means to use self.rnom1d) k2 (float): If given, the ratio R2/R1 in the SKL formulae. (default: 1) k3 (float): If given, the ratio R3/R1 in the SKL formulae. (default: 1) @@ -650,11 +652,12 @@ def writeMap3(self, file_name, *, R=None, file_type=None, precision=None): Parameters: file_name (str): The name of the file to write to. - R (array): The R values at which to calculate the statistics. - (default: None, which means use self.rnom) + R (:class:`numpy.ndarray`): + The R values at which to calculate the statistics. + (default: None, which means to use self.rnom) file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) """ self.logger.info('Writing Map^3 from GGG correlations to %s',file_name) diff --git a/treecorr/kgcorrelation.py b/treecorr/kgcorrelation.py index 58d17f1b..8fb58fc5 100644 --- a/treecorr/kgcorrelation.py +++ b/treecorr/kgcorrelation.py @@ -33,10 +33,10 @@ class KGCorrelation(BaseKZCorrelation): correlations of some survey property, such as seeing, with shear, where "kappa" would really be the measured property, e.g. the observed sizes of the stars. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. - In addition to the attributes common to all `Corr3` subclasses, objects of this class + In addition to the attributes common to all `Corr2` subclasses, objects of this class hold the following attributes: Attributes: @@ -56,15 +56,18 @@ class KGCorrelation(BaseKZCorrelation): The typical usage pattern is as follows: >>> kg = treecorr.KGCorrelation(config) - >>> kg.process(cat1,cat2) # Calculate the cross-correlation - >>> kg.write(file_name) # Write out to a file. - >>> xi = kg.xi # Or access the correlation function directly. + >>> kg.process(cat1, cat2) # Compute the cross-correlation. + >>> kg.write(file_name) # Write out to a file. + >>> xi, xi_im = kg.xi, kg.xi_im # Or access the correlation function directly. + + See also: `KKCorrelation`, `NGCorrelation`, `KGGCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -124,7 +127,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result file_name (str): The name of the file to write to. file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) diff --git a/treecorr/kggcorrelation.py b/treecorr/kggcorrelation.py index 71e1994f..edb56d5b 100644 --- a/treecorr/kggcorrelation.py +++ b/treecorr/kggcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: nnncorrelation +.. module:: kggcorrelation """ import numpy as np @@ -47,7 +47,7 @@ class KGGCorrelation(Corr3): names you find most convenient: ``gam0``, ``gam1``, ``gam2`` and ``gam3`` are all valid attributes, which return the corresponding value. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -58,26 +58,34 @@ class KGGCorrelation(Corr3): gam1: The 1st "natural" correlation function, :math:`\Gamma_1`. gam2: The 2nd "natural" correlation function, :math:`\Gamma_2`. gam3: The 3rd "natural" correlation function, :math:`\Gamma_3`. - vargam0: The variance estimate of :math:`\Gamma_0`, only including the shot noise. - vargam1: The variance estimate of :math:`\Gamma_1`, only including the shot noise. - vargam2: The variance estimate of :math:`\Gamma_2`, only including the shot noise. - vargam3: The variance estimate of :math:`\Gamma_3`, only including the shot noise. + vargam0: The variance estimate of :math:`\Gamma_0`, computed according to ``var_method`` + (default: ``'shot'``). + vargam1: The variance estimate of :math:`\Gamma_1`, computed according to ``var_method`` + (default: ``'shot'``). + vargam2: The variance estimate of :math:`\Gamma_2`, computed according to ``var_method`` + (default: ``'shot'``). + vargam3: The variance estimate of :math:`\Gamma_3`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows:: >>> kgg = treecorr.KGGCorrelation(config) - >>> kgg.process(cat1, cat2) # Compute cross-correlation of two fields. - >>> kgg.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> kgg.write(file_name) # Write out to a file. - >>> gam0 = kgg.gam0, etc. # Access gamma values directly. - >>> gam0r = kgg.gam0r # Or access real and imag parts separately. + >>> kgg.process(cat1, cat2) # Compute the cross-correlation of two fields. + >>> # kgg.process(cat1, cat2, cat3) # ... or of three fields. + >>> kgg.write(file_name) # Write out to a file. + >>> gam0 = kgg.gam0, etc. # Access gamma values directly. + >>> gam0r = kgg.gam0r # Or access real and imaginary parts separately. >>> gam0i = kgg.gam0i + See also: `GKGCorrelation`, `GGKCorrelation`, `KKGCorrelation`, `GGGCorrelation`, + `KGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -209,9 +217,9 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" gam0r The real part of the estimator of :math:`\Gamma_0` - gam0i The imag part of the estimator of :math:`\Gamma_0` + gam0i The imaginary part of the estimator of :math:`\Gamma_0` gam2r The real part of the estimator of :math:`\Gamma_2` - gam2i The imag part of the estimator of :math:`\Gamma_2` + gam2i The imaginary part of the estimator of :math:`\Gamma_2` sigma_gam0 The sqrt of the variance estimate of :math:`\Gamma_0` sigma_gam2 The sqrt of the variance estimate of :math:`\Gamma_2` """) @@ -261,7 +269,7 @@ class GKGCorrelation(Corr3): names you find most convenient: ``gam0``, ``gam1``, ``gam2`` and ``gam3`` are all valid attributes, which return the corresponding value. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -272,26 +280,34 @@ class GKGCorrelation(Corr3): gam1: The 1st "natural" correlation function, :math:`\Gamma_1`. gam2: The 2nd "natural" correlation function, :math:`\Gamma_2`. gam3: The 3rd "natural" correlation function, :math:`\Gamma_3`. - vargam0: The variance estimate of :math:`\Gamma_0`, only including the shot noise. - vargam1: The variance estimate of :math:`\Gamma_1`, only including the shot noise. - vargam2: The variance estimate of :math:`\Gamma_2`, only including the shot noise. - vargam3: The variance estimate of :math:`\Gamma_3`, only including the shot noise. + vargam0: The variance estimate of :math:`\Gamma_0`, computed according to ``var_method`` + (default: ``'shot'``). + vargam1: The variance estimate of :math:`\Gamma_1`, computed according to ``var_method`` + (default: ``'shot'``). + vargam2: The variance estimate of :math:`\Gamma_2`, computed according to ``var_method`` + (default: ``'shot'``). + vargam3: The variance estimate of :math:`\Gamma_3`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows:: >>> gkg = treecorr.GKGCorrelation(config) - >>> gkg.process(cat1, cat2, cat1) # Compute cross-correlation of two fields. - >>> gkg.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> gkg.write(file_name) # Write out to a file. - >>> gam0 = gkg.gam0, etc. # Access gamma values directly. - >>> gam0r = gkg.gam0r # Or access real and imag parts separately. + >>> gkg.process(cat1, cat2, cat1) # Compute the cross-correlation of two fields. + >>> # gkg.process(cat1, cat2, cat3) # ... or of three fields. + >>> gkg.write(file_name) # Write out to a file. + >>> gam0 = gkg.gam0, etc. # Access gamma values directly. + >>> gam0r = gkg.gam0r # Or access real and imaginary parts separately. >>> gam0i = gkg.gam0i + See also: `KGGCorrelation`, `GGKCorrelation`, `KKGCorrelation`, `GGGCorrelation`, + `KGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -421,9 +437,9 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" gam0r The real part of the estimator of :math:`\Gamma_0` - gam0i The imag part of the estimator of :math:`\Gamma_0` + gam0i The imaginary part of the estimator of :math:`\Gamma_0` gam1r The real part of the estimator of :math:`\Gamma_1` - gam1i The imag part of the estimator of :math:`\Gamma_1` + gam1i The imaginary part of the estimator of :math:`\Gamma_1` sigma_gam0 The sqrt of the variance estimate of :math:`\Gamma_0` sigma_gam1 The sqrt of the variance estimate of :math:`\Gamma_1` """) @@ -452,8 +468,8 @@ class GGKCorrelation(Corr3): r"""This class handles the calculation and storage of a 3-point shear-shear-scalar correlation function. - With this class, point 1 of the triangle (i.e. the vertex opposite d1) is the one with the - shear value. Use `GKGCorrelation` and `KGGCorrelation` for classes with the shear in the + With this class, point 3 of the triangle (i.e. the vertex opposite d3) is the one with the + scalar value. Use `GKGCorrelation` and `KGGCorrelation` for classes with the scalar in the other two positions. For the shear projection, we follow the lead of the 3-point shear-shear-shear correlation @@ -473,7 +489,7 @@ class GGKCorrelation(Corr3): names you find most convenient: ``gam0``, ``gam1``, ``gam2`` and ``gam3`` are all valid attributes, which return the corresponding value. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -484,26 +500,34 @@ class GGKCorrelation(Corr3): gam1: The 1st "natural" correlation function, :math:`\Gamma_1`. gam2: The 2nd "natural" correlation function, :math:`\Gamma_2`. gam3: The 3rd "natural" correlation function, :math:`\Gamma_3`. - vargam0: The variance estimate of :math:`\Gamma_0`, only including the shot noise. - vargam1: The variance estimate of :math:`\Gamma_1`, only including the shot noise. - vargam2: The variance estimate of :math:`\Gamma_2`, only including the shot noise. - vargam3: The variance estimate of :math:`\Gamma_3`, only including the shot noise. + vargam0: The variance estimate of :math:`\Gamma_0`, computed according to ``var_method`` + (default: ``'shot'``). + vargam1: The variance estimate of :math:`\Gamma_1`, computed according to ``var_method`` + (default: ``'shot'``). + vargam2: The variance estimate of :math:`\Gamma_2`, computed according to ``var_method`` + (default: ``'shot'``). + vargam3: The variance estimate of :math:`\Gamma_3`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows:: >>> ggk = treecorr.GGKCorrelation(config) - >>> ggk.process(cat1, cat2) # Compute cross-correlation of two fields. - >>> ggk.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> ggk.write(file_name) # Write out to a file. - >>> gam0 = ggk.gam0, etc. # Access gamma values directly. - >>> gam0r = ggk.gam0r # Or access real and imag parts separately. + >>> ggk.process(cat1, cat2) # Compute the cross-correlation of two fields. + >>> # ggk.process(cat1, cat2, cat3) # ... or of three fields. + >>> ggk.write(file_name) # Write out to a file. + >>> gam0 = ggk.gam0, etc. # Access gamma values directly. + >>> gam0r = ggk.gam0r # Or access real and imaginary parts separately. >>> gam0i = ggk.gam0i + See also: `KGGCorrelation`, `GKGCorrelation`, `KKGCorrelation`, `GGGCorrelation`, + `KGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -633,9 +657,9 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" gam0r The real part of the estimator of :math:`\Gamma_0` - gam0i The imag part of the estimator of :math:`\Gamma_0` + gam0i The imaginary part of the estimator of :math:`\Gamma_0` gam1r The real part of the estimator of :math:`\Gamma_1` - gam1i The imag part of the estimator of :math:`\Gamma_1` + gam1i The imaginary part of the estimator of :math:`\Gamma_1` sigma_gam0 The sqrt of the variance estimate of :math:`\Gamma_0` sigma_gam1 The sqrt of the variance estimate of :math:`\Gamma_1` """) diff --git a/treecorr/kkcorrelation.py b/treecorr/kkcorrelation.py index 005ec52f..280ab2d2 100644 --- a/treecorr/kkcorrelation.py +++ b/treecorr/kkcorrelation.py @@ -34,10 +34,10 @@ class KKCorrelation(Corr2): correlations of the CMB temperature fluctuations, where "kappa" would really be :math:`\Delta T`. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. - In addition to the attributes common to all `Corr3` subclasses, objects of this class + In addition to the attributes common to all `Corr2` subclasses, objects of this class hold the following attributes: Attributes: @@ -56,16 +56,19 @@ class KKCorrelation(Corr2): The typical usage pattern is as follows: >>> kk = treecorr.KKCorrelation(config) - >>> kk.process(cat) # For auto-correlation. - >>> kk.process(cat1,cat2) # For cross-correlation. - >>> kk.write(file_name) # Write out to a file. - >>> xi = kk.xi # Or access the correlation function directly. + >>> kk.process(cat) # Compute the auto-correlation. + >>> # kk.process(cat1, cat2) # ... or the cross-correlation. + >>> kk.write(file_name) # Write out to a file. + >>> xi = kk.xi # Or access the correlation function directly. + + See also: `NKCorrelation`, `KGCorrelation`, `KKKCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -142,7 +145,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result file_name (str): The name of the file to write to. file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) diff --git a/treecorr/kkgcorrelation.py b/treecorr/kkgcorrelation.py index 9712564c..61a45e4f 100644 --- a/treecorr/kkgcorrelation.py +++ b/treecorr/kkgcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: nnncorrelation +.. module:: kkgcorrelation """ import numpy as np @@ -26,14 +26,14 @@ class KKGCorrelation(Corr3): r"""This class handles the calculation and storage of a 3-point scalar-scalar-shear correlation function. - See the doc string of `Corr3` for a description of how the triangles + See the docstring of `Corr3` for a description of how the triangles are binned. With this class, point 3 of the triangle (i.e. the vertex opposite d3) is the one with the shear value. Use `KGKCorrelation` and `GKKCorrelation` for classes with the shear in the other two positions. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -41,24 +41,28 @@ class KKGCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance estimate, only including the shot noise propagated into the - final correlation. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows:: >>> kkg = treecorr.KKGCorrelation(config) - >>> kkg.process(cat1, cat2) # Compute cross-correlation of two fields. - >>> kkg.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> kkg.write(file_name) # Write out to a file. - >>> zeta = kkg.zeta # Access correlation function. - >>> zetar = kkg.zetar # Or access real and imag parts separately. + >>> kkg.process(cat1, cat2) # Compute the cross-correlation of two fields. + >>> # kkg.process(cat1, cat2, cat3) # ... or of three fields. + >>> kkg.write(file_name) # Write out to a file. + >>> zeta = kkg.zeta # Access the correlation function. + >>> zetar = kkg.zetar # Or access real and imaginary parts separately. >>> zetai = kkg.zetai + See also: `KGKCorrelation`, `GKKCorrelation`, `KGGCorrelation`, `KKKCorrelation`, + `KGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -124,7 +128,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" zetar The real part of the estimator of :math:`\zeta` - zetai The imag part of the estimator of :math:`\zeta` + zetai The imaginary part of the estimator of :math:`\zeta` sigma_zeta The sqrt of the variance estimate of :math:`\zeta` """) @@ -147,14 +151,14 @@ class KGKCorrelation(Corr3): r"""This class handles the calculation and storage of a 3-point scalar-shear-scalar correlation function. - See the doc string of `Corr3` for a description of how the triangles + See the docstring of `Corr3` for a description of how the triangles are binned. With this class, point 2 of the triangle (i.e. the vertex opposite d2) is the one with the shear value. Use `KKGCorrelation` and `GKKCorrelation` for classes with the shear in the other two positions. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -162,24 +166,28 @@ class KGKCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance estimate, only including the shot noise propagated into the - final correlation. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows:: >>> kgk = treecorr.KGKCorrelation(config) - >>> kgk.process(cat1, cat2, cat1) # Compute cross-correlation of two fields. - >>> kgk.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> kgk.write(file_name) # Write out to a file. - >>> zeta = kgk.zeta # Access correlation function. - >>> zetar = kgk.zetar # Or access real and imag parts separately. + >>> kgk.process(cat1, cat2, cat1) # Compute the cross-correlation of two fields. + >>> # kgk.process(cat1, cat2, cat3) # ... or of three fields. + >>> kgk.write(file_name) # Write out to a file. + >>> zeta = kgk.zeta # Access the correlation function. + >>> zetar = kgk.zetar # Or access real and imaginary parts separately. >>> zetai = kgk.zetai + See also: `KKGCorrelation`, `GKKCorrelation`, `KGGCorrelation`, `KKKCorrelation`, + `KGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -245,7 +253,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" zetar The real part of the estimator of :math:`\zeta` - zetai The imag part of the estimator of :math:`\zeta` + zetai The imaginary part of the estimator of :math:`\zeta` sigma_zeta The sqrt of the variance estimate of :math:`\zeta` """) @@ -268,14 +276,14 @@ class GKKCorrelation(Corr3): r"""This class handles the calculation and storage of a 3-point shear-scalar-scalar correlation function. - See the doc string of `Corr3` for a description of how the triangles + See the docstring of `Corr3` for a description of how the triangles are binned. With this class, point 1 of the triangle (i.e. the vertex opposite d1) is the one with the shear value. Use `KGKCorrelation` and `KKGCorrelation` for classes with the shear in the other two positions. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -283,24 +291,28 @@ class GKKCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance estimate, only including the shot noise propagated into the - final correlation. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows:: >>> gkk = treecorr.GKKCorrelation(config) - >>> gkk.process(cat1, cat2) # Compute cross-correlation of two fields. - >>> gkk.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> gkk.write(file_name) # Write out to a file. - >>> zeta = gkk.zeta # Access correlation function. - >>> zetar = gkk.zetar # Or access real and imag parts separately. + >>> gkk.process(cat1, cat2) # Compute the cross-correlation of two fields. + >>> # gkk.process(cat1, cat2, cat3) # ... or of three fields. + >>> gkk.write(file_name) # Write out to a file. + >>> zeta = gkk.zeta # Access the correlation function. + >>> zetar = gkk.zetar # Or access real and imaginary parts separately. >>> zetai = gkk.zetai + See also: `KKGCorrelation`, `KGKCorrelation`, `KGGCorrelation`, `KKKCorrelation`, + `KGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -366,7 +378,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" zetar The real part of the estimator of :math:`\zeta` - zetai The imag part of the estimator of :math:`\zeta` + zetai The imaginary part of the estimator of :math:`\zeta` sigma_zeta The sqrt of the variance estimate of :math:`\zeta` """) diff --git a/treecorr/kkkcorrelation.py b/treecorr/kkkcorrelation.py index 735e4a4e..99d506fd 100644 --- a/treecorr/kkkcorrelation.py +++ b/treecorr/kkkcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: nnncorrelation +.. module:: kkkcorrelation """ import numpy as np @@ -33,7 +33,7 @@ class KKKCorrelation(Corr3): correlations of the CMB temperature fluctuations, where "kappa" would really be :math:`\Delta T`. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -41,23 +41,25 @@ class KKKCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance of :math:`\zeta`, only including the shot noise propagated into - the final correlation. This does not include sample variance, so it is always - an underestimate of the actual variance. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows: >>> kkk = treecorr.KKKCorrelation(config) - >>> kkk.process(cat) # Compute auto-correlation. - >>> kkk.process(cat1, cat2, cat3) # Compute cross-correlation. - >>> kkk.write(file_name) # Write out to a file. - >>> zeta = kkk.zeta # Access zeta directly. + >>> kkk.process(cat) # Compute the auto-correlation. + >>> # kkk.process(cat1, cat2, cat3) # ... or the cross-correlation. + >>> kkk.write(file_name) # Write out to a file. + >>> zeta = kkk.zeta # Access zeta directly. + + See also: `KKGCorrelation`, `NKKCorrelation`, `KKCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: diff --git a/treecorr/kqcorrelation.py b/treecorr/kqcorrelation.py index f339b910..adf8d1f6 100644 --- a/treecorr/kqcorrelation.py +++ b/treecorr/kqcorrelation.py @@ -26,10 +26,10 @@ class KQCorrelation(BaseKZCorrelation): r"""This class handles the calculation and storage of a 2-point scalar-quatrefoil correlation function, where a quatrefoil is any field with spin-4 rotational properties. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. - In addition to the attributes common to all `Corr3` subclasses, objects of this class + In addition to the attributes common to all `Corr2` subclasses, objects of this class hold the following attributes: Attributes: @@ -49,15 +49,18 @@ class KQCorrelation(BaseKZCorrelation): The typical usage pattern is as follows: >>> kq = treecorr.KQCorrelation(config) - >>> kq.process(cat1,cat2) # Calculate the cross-correlation - >>> kq.write(file_name) # Write out to a file. - >>> xi = kq.xi # Or access the correlation function directly. + >>> kq.process(cat1, cat2) # Compute the cross-correlation. + >>> kq.write(file_name) # Write out to a file. + >>> xi, xi_im = kq.xi, kq.xi_im # Or access the correlation function directly. + + See also: `NQCorrelation`, `QQCorrelation`, `KZCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -117,7 +120,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result file_name (str): The name of the file to write to. file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) diff --git a/treecorr/ktcorrelation.py b/treecorr/ktcorrelation.py index 42bc9ba3..a9d17040 100644 --- a/treecorr/ktcorrelation.py +++ b/treecorr/ktcorrelation.py @@ -26,10 +26,10 @@ class KTCorrelation(BaseKZCorrelation): r"""This class handles the calculation and storage of a 2-point scalar-trefoil correlation function, where a trefoil is any field with spin-3 rotational properties. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. - In addition to the attributes common to all `Corr3` subclasses, objects of this class + In addition to the attributes common to all `Corr2` subclasses, objects of this class hold the following attributes: Attributes: @@ -49,15 +49,18 @@ class KTCorrelation(BaseKZCorrelation): The typical usage pattern is as follows: >>> kt = treecorr.KTCorrelation(config) - >>> kt.process(cat1,cat2) # Calculate the cross-correlation - >>> kt.write(file_name) # Write out to a file. - >>> xi = kt.xi # Or access the correlation function directly. + >>> kt.process(cat1, cat2) # Compute the cross-correlation. + >>> kt.write(file_name) # Write out to a file. + >>> xi, xi_im = kt.xi, kt.xi_im # Or access the correlation function directly. + + See also: `NTCorrelation`, `TTCorrelation`, `KZCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -101,9 +104,9 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result fell into each bin meanlogr The mean value :math:`\langle \log(r)\rangle` of pairs that fell into each bin - xi The real part of correlation function, + xi The real part of the correlation function, :math:`\xi(r) = \langle \kappa\, t_R\rangle` - xi_im The imaginary part of correlation function. + xi_im The imaginary part of the correlation function. sigma The sqrt of the variance estimate of both of these weight The total weight contributing to each bin npairs The total number of pairs in each bin @@ -117,7 +120,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result file_name (str): The name of the file to write to. file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) diff --git a/treecorr/kvcorrelation.py b/treecorr/kvcorrelation.py index 2097454a..f5af7325 100644 --- a/treecorr/kvcorrelation.py +++ b/treecorr/kvcorrelation.py @@ -26,10 +26,10 @@ class KVCorrelation(BaseKZCorrelation): r"""This class handles the calculation and storage of a 2-point scalar-vector correlation function. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. - In addition to the attributes common to all `Corr3` subclasses, objects of this class + In addition to the attributes common to all `Corr2` subclasses, objects of this class hold the following attributes: Attributes: @@ -50,15 +50,18 @@ class KVCorrelation(BaseKZCorrelation): The typical usage pattern is as follows: >>> kv = treecorr.KVCorrelation(config) - >>> kv.process(cat1,cat2) # Calculate the cross-correlation - >>> kv.write(file_name) # Write out to a file. - >>> xi = kv.xi # Or access the correlation function directly. + >>> kv.process(cat1, cat2) # Compute the cross-correlation. + >>> kv.write(file_name) # Write out to a file. + >>> xi, xi_im = kv.xi, kv.xi_im # Or access the correlation function directly. + + See also: `NVCorrelation`, `VVCorrelation`, `KZCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -118,7 +121,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result file_name (str): The name of the file to write to. file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) diff --git a/treecorr/kzcorrelation.py b/treecorr/kzcorrelation.py index 9eb720a3..45d48e49 100644 --- a/treecorr/kzcorrelation.py +++ b/treecorr/kzcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: kvcorrelation +.. module:: kzcorrelation """ import numpy as np @@ -89,13 +89,13 @@ class KZCorrelation(BaseKZCorrelation): r"""This class handles the calculation and storage of a 2-point scalar-complex correlation function, where the complex field is taken to have spin-0 rotational properties. If the spin-0 field is real, you should instead use `KKCorrelation` as it will be faster. - This class is intended for correlations of a scalar field with a complex values that + This class is intended for correlations of a scalar field with complex values that don't change with orientation. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. - In addition to the attributes common to all `Corr3` subclasses, objects of this class + In addition to the attributes common to all `Corr2` subclasses, objects of this class hold the following attributes: Attributes: @@ -114,16 +114,19 @@ class KZCorrelation(BaseKZCorrelation): The typical usage pattern is as follows: - >>> kv = treecorr.KZCorrelation(config) - >>> kv.process(cat1,cat2) # Calculate the cross-correlation - >>> kv.write(file_name) # Write out to a file. - >>> xi = kv.xi # Or access the correlation function directly. + >>> kz = treecorr.KZCorrelation(config) + >>> kz.process(cat1, cat2) # Compute the cross-correlation. + >>> kz.write(file_name) # Write out to a file. + >>> xi, xi_im = kz.xi, kz.xi_im # Or access the correlation function directly. + + See also: `NZCorrelation`, `ZZCorrelation`, `KVCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -167,9 +170,9 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result fell into each bin meanlogr The mean value :math:`\langle \log(r)\rangle` of pairs that fell into each bin - xi The real part of correlation function, + xi The real part of the correlation function, :math:`xi(r) = \langle \kappa\, z\rangle` - xi_im The imaginary part of correlation function. + xi_im The imaginary part of the correlation function. sigma The sqrt of the variance estimate of both of these weight The total weight contributing to each bin npairs The total number of pairs in each bin @@ -183,7 +186,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result file_name (str): The name of the file to write to. file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) diff --git a/treecorr/ngcorrelation.py b/treecorr/ngcorrelation.py index df4166fb..f9584581 100644 --- a/treecorr/ngcorrelation.py +++ b/treecorr/ngcorrelation.py @@ -28,10 +28,10 @@ class NGCorrelation(BaseNZCorrelation): function. This is the tangential shear profile around lenses, commonly referred to as galaxy-galaxy lensing. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. - In addition to the attributes common to all `Corr3` subclasses, objects of this class + In addition to the attributes common to all `Corr2` subclasses, objects of this class hold the following attributes: Attributes: @@ -45,7 +45,7 @@ class NGCorrelation(BaseNZCorrelation): .. note:: - The default method for estimating the variance and covariance attributes (``varxi``, + The default method for estimating the variance and covariance attributes (``varxi`` and ``cov``) is 'shot', which only includes the shape noise propagated into the final correlation. This does not include sample variance, so it is always an underestimate of the actual variance. To get better estimates, you need to set @@ -55,15 +55,18 @@ class NGCorrelation(BaseNZCorrelation): The typical usage pattern is as follows: >>> ng = treecorr.NGCorrelation(config) - >>> ng.process(cat1,cat2) # Compute the cross-correlation. - >>> ng.write(file_name) # Write out to a file. - >>> xi = ng.xi # Or access the correlation function directly. + >>> ng.process(cat1, cat2) # Compute the cross-correlation. + >>> ng.write(file_name) # Write out to a file. + >>> xi, xi_im = ng.xi, ng.xi_im # Or access the correlation function directly. + + See also: `NKCorrelation`, `GGCorrelation`, `NGGCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -104,7 +107,14 @@ def calculateXi(self, *, rg=None): After calling this function, the attributes ``xi``, ``xi_im``, ``varxi``, and ``cov`` will correspond to the compensated values (if rg is provided). The raw, uncompensated values - are available as ``rawxi``, ``raw_xi_im``, and ``raw_varxi``. + are available as ``raw_xi``, ``raw_xi_im``, and ``raw_varxi``. + + .. note:: + + The returned variance estimate (``varxi``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr2.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: rg (NGCorrelation): The cross-correlation using random locations as the lenses @@ -140,9 +150,9 @@ def write(self, file_name, *, rg=None, file_type=None, precision=None, fell into each bin gamT The real part of the mean tangential shear, :math:`\langle \gamma_T \rangle(r)` - gamX The imag part of the mean tangential shear, + gamX The imaginary part of the mean tangential shear, :math:`\langle \gamma_\times \rangle(r)` - sigma The sqrt of the variance estimate of either of these + sigma The sqrt of the variance estimate of each of these weight The total weight contributing to each bin npairs The total number of pairs in each bin ========== ============================================================= @@ -157,7 +167,7 @@ def write(self, file_name, *, rg=None, file_type=None, precision=None, (RG), if desired. (default: None) file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) @@ -201,8 +211,9 @@ def calculateNMap(self, *, R=None, rg=None, m2_uform=None): to the derivations of :math:`T_+` and :math:`T_-` in Schneider et al. (2002). Parameters: - R (array): The R values at which to calculate the aperture mass statistics. - (default: None, which means use self.rnom) + R (:class:`numpy.ndarray`): + The R values at which to calculate the aperture mass statistics. + (default: None, which means to use self.rnom) rg (NGCorrelation): The cross-correlation using random locations as the lenses (RG), if desired. (default: None) m2_uform (str): Which form to use for the aperture mass, as described above. @@ -255,7 +266,7 @@ def calculateNMap(self, *, R=None, rg=None, m2_uform=None): def writeNMap(self, file_name, *, R=None, rg=None, m2_uform=None, file_type=None, precision=None): - r"""Write the cross correlation of the foreground galaxy counts with the aperture mass + r"""Write the cross-correlation of the foreground galaxy counts with the aperture mass based on the correlation function to the file, file_name. If rg is provided, the compensated calculation will be used for :math:`\xi`. @@ -270,21 +281,22 @@ def writeNMap(self, file_name, *, R=None, rg=None, m2_uform=None, file_type=None R The radius of the aperture. NMap An estimate of :math:`\langle N_{ap} M_{ap} \rangle(R)` NMx An estimate of :math:`\langle N_{ap} M_\times \rangle(R)` - sig_nmap The sqrt of the variance estimate of either of these + sig_nmap The sqrt of the variance estimate of each of these ========== ========================================================= Parameters: file_name (str): The name of the file to write to. - R (array): The R values at which to calculate the aperture mass statistics. - (default: None, which means use self.rnom) + R (:class:`numpy.ndarray`): + The R values at which to calculate the aperture mass statistics. + (default: None, which means to use self.rnom) rg (NGCorrelation): The cross-correlation using random locations as the lenses (RG), if desired. (default: None) m2_uform (str): Which form to use for the aperture mass. (default: 'Crittenden'; this value can also be given in the constructor in the config dict.) file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) """ self.logger.info('Writing NMap from NG correlations to %s',file_name) @@ -312,9 +324,9 @@ def writeNorm(self, file_name, *, gg, dd, rr, R=None, dr=None, rg=None, This function computes these combinations and outputs them to a file. - - if rg is provided, the compensated calculation will be used for + - If rg is provided, the compensated calculation will be used for :math:`\langle N_{ap} M_{ap} \rangle`. - - if dr is provided, the compensated calculation will be used for + - If dr is provided, the compensated calculation will be used for :math:`\langle N_{ap}^2 \rangle`. See `calculateNMap` for an explanation of the ``m2_uform`` parameter. @@ -327,7 +339,7 @@ def writeNorm(self, file_name, *, gg, dd, rr, R=None, dr=None, rg=None, R The radius of the aperture NMap An estimate of :math:`\langle N_{ap} M_{ap} \rangle(R)` NMx An estimate of :math:`\langle N_{ap} M_\times \rangle(R)` - sig_nmap The sqrt of the variance estimate of either of these + sig_nmap The sqrt of the variance estimate of each of these Napsq An estimate of :math:`\langle N_{ap}^2 \rangle(R)` sig_napsq The sqrt of the variance estimate of :math:`\langle N_{ap}^2 \rangle` Mapsq An estimate of :math:`\langle M_{ap}^2 \rangle(R)` @@ -344,20 +356,21 @@ def writeNorm(self, file_name, *, gg, dd, rr, R=None, dr=None, rg=None, gg (GGCorrelation): The auto-correlation of the shear field dd (NNCorrelation): The auto-correlation of the lens counts (DD) rr (NNCorrelation): The auto-correlation of the random field (RR) - R (array): The R values at which to calculate the aperture mass statistics. - (default: None, which means use self.rnom) + R (:class:`numpy.ndarray`): + The R values at which to calculate the aperture mass statistics. + (default: None, which means to use self.rnom) dr (NNCorrelation): The cross-correlation of the data with randoms (DR), if desired, in which case the Landy-Szalay estimator will be calculated. (default: None) rd (NNCorrelation): The cross-correlation of the randoms with data (RD), if - desired. (default: None, which means use rd=dr) + desired. (default: None, which means to use rd=dr) rg (NGCorrelation): The cross-correlation using random locations as the lenses (RG), if desired. (default: None) m2_uform (str): Which form to use for the aperture mass. (default: 'Crittenden'; this value can also be given in the constructor in the config dict.) file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) """ self.logger.info('Writing Norm from NG correlations to %s',file_name) diff --git a/treecorr/nggcorrelation.py b/treecorr/nggcorrelation.py index d2d462b6..6c7bd4b5 100644 --- a/treecorr/nggcorrelation.py +++ b/treecorr/nggcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: nnncorrelation +.. module:: nggcorrelation """ import numpy as np @@ -26,7 +26,7 @@ class NGGCorrelation(Corr3): r"""This class handles the calculation and storage of a 3-point count-shear-shear correlation function. - With this class, points 2 and 3 of the triangle (i.e. the vertces opposite d2,d3) are the ones + With this class, points 2 and 3 of the triangle (i.e. the vertices opposite d2, d3) are the ones with the shear values. Use `GNGCorrelation` and `GGNCorrelation` for classes with the shears in the other positions. @@ -47,7 +47,7 @@ class NGGCorrelation(Corr3): names you find most convenient: ``gam0``, ``gam1``, ``gam2`` and ``gam3`` are all valid attributes, which return the corresponding value. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -58,28 +58,36 @@ class NGGCorrelation(Corr3): gam1: The 1st "natural" correlation function, :math:`\Gamma_1`. gam2: The 2nd "natural" correlation function, :math:`\Gamma_2`. gam3: The 3rd "natural" correlation function, :math:`\Gamma_3`. - vargam0: The variance estimate of :math:`\Gamma_0`, only including the shot noise. - vargam1: The variance estimate of :math:`\Gamma_1`, only including the shot noise. - vargam2: The variance estimate of :math:`\Gamma_2`, only including the shot noise. - vargam3: The variance estimate of :math:`\Gamma_3`, only including the shot noise. + vargam0: The variance estimate of :math:`\Gamma_0`, computed according to ``var_method`` + (default: ``'shot'``). + vargam1: The variance estimate of :math:`\Gamma_1`, computed according to ``var_method`` + (default: ``'shot'``). + vargam2: The variance estimate of :math:`\Gamma_2`, computed according to ``var_method`` + (default: ``'shot'``). + vargam3: The variance estimate of :math:`\Gamma_3`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows:: >>> ngg = treecorr.NGGCorrelation(config) - >>> ngg.process(cat1, cat2) # Compute cross-correlation of two fields. - >>> ngg.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> ngg.write(file_name) # Write out to a file. - >>> rgg.process(rand, cat2) # Compute cross-correlation with randoms. - >>> ngg.calculateZeta(rgg=rgg) # Calculate zeta using randoms - >>> gam0 = ngg.gam0, etc. # Access gamma values directly. - >>> gam0r = ngg.gam0r # Or access real and imag parts separately. + >>> ngg.process(cat1, cat2) # Compute the cross-correlation of two fields. + >>> # ngg.process(cat1, cat2, cat3) # ... or of three fields. + >>> ngg.write(file_name) # Write out to a file. + >>> rgg.process(rand, cat2) # Compute the random cross-correlation. + >>> ngg.calculateZeta(rgg=rgg) # Calculate zeta using randoms. + >>> gam0 = ngg.gam0, etc. # Access gamma values directly. + >>> gam0r = ngg.gam0r # Or access real and imaginary parts separately. >>> gam0i = ngg.gam0i + See also: `GNGCorrelation`, `GGNCorrelation`, `NNGCorrelation`, `GGGCorrelation`, + `NGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -273,9 +281,16 @@ def calculateGam(self, *, rgg=None): After calling this function, the attributes ``gam0``, ``gam2``, ``vargam0``, ``vargam2``, and ``cov`` will correspond to the compensated values (if rgg is provided). - The raw, uncompensated values are available as ``raw_gam0`` , ``raw_gam2``, + The raw, uncompensated values are available as ``raw_gam0``, ``raw_gam2``, ``raw_vargam0``, and ``raw_vargam2``. + .. note:: + + The returned variance estimates (``vargam0`` and ``vargam2``) are computed according + to this object's ``var_method`` setting, specified when constructing the object + (default: ``'shot'``). Internally, this method calls `Corr3.estimate_cov`; see that + method for details about available variance and covariance estimation schemes. + Parameters: rgg (NGGCorrelation): The cross-correlation using random locations as the lenses (RGG), if desired. (default: None) @@ -350,9 +365,9 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" gam0r The real part of the estimator of :math:`\Gamma_0` - gam0i The imag part of the estimator of :math:`\Gamma_0` + gam0i The imaginary part of the estimator of :math:`\Gamma_0` gam2r The real part of the estimator of :math:`\Gamma_2` - gam2i The imag part of the estimator of :math:`\Gamma_2` + gam2i The imaginary part of the estimator of :math:`\Gamma_2` sigma_gam0 The sqrt of the variance estimate of :math:`\Gamma_0` sigma_gam2 The sqrt of the variance estimate of :math:`\Gamma_2` """) @@ -381,7 +396,7 @@ class GNGCorrelation(Corr3): r"""This class handles the calculation and storage of a 3-point shear-count-shear correlation function. - With this class, points 1 and 3 of the triangle (i.e. the vertces opposite d2,d3) are the ones + With this class, points 1 and 3 of the triangle (i.e. the vertices opposite d1, d3) are the ones with the shear values. Use `NGGCorrelation` and `GGNCorrelation` for classes with the shears in the other positions. @@ -402,7 +417,7 @@ class GNGCorrelation(Corr3): names you find most convenient: ``gam0``, ``gam1``, ``gam2`` and ``gam3`` are all valid attributes, which return the corresponding value. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -413,28 +428,36 @@ class GNGCorrelation(Corr3): gam1: The 1st "natural" correlation function, :math:`\Gamma_1`. gam2: The 2nd "natural" correlation function, :math:`\Gamma_2`. gam3: The 3rd "natural" correlation function, :math:`\Gamma_3`. - vargam0: The variance estimate of :math:`\Gamma_0`, only including the shot noise. - vargam1: The variance estimate of :math:`\Gamma_1`, only including the shot noise. - vargam2: The variance estimate of :math:`\Gamma_2`, only including the shot noise. - vargam3: The variance estimate of :math:`\Gamma_3`, only including the shot noise. + vargam0: The variance estimate of :math:`\Gamma_0`, computed according to ``var_method`` + (default: ``'shot'``). + vargam1: The variance estimate of :math:`\Gamma_1`, computed according to ``var_method`` + (default: ``'shot'``). + vargam2: The variance estimate of :math:`\Gamma_2`, computed according to ``var_method`` + (default: ``'shot'``). + vargam3: The variance estimate of :math:`\Gamma_3`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows:: >>> gng = treecorr.GNGCorrelation(config) - >>> gng.process(cat1, cat2, cat1) # Compute cross-correlation of two fields. - >>> gng.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> gng.write(file_name) # Write out to a file. - >>> grg.process(cat1, rand, cat1) # Compute cross-correlation with randoms. - >>> gng.calculateZeta(grg=grg) # Calculate zeta using randoms - >>> gam0 = gng.gam0, etc. # Access gamma values directly. - >>> gam0r = gng.gam0r # Or access real and imag parts separately. + >>> gng.process(cat1, cat2, cat1) # Compute the cross-correlation of two fields. + >>> # gng.process(cat1, cat2, cat3) # ... or of three fields. + >>> gng.write(file_name) # Write out to a file. + >>> grg.process(cat1, rand, cat1) # Compute the random cross-correlation. + >>> gng.calculateZeta(grg=grg) # Calculate zeta using randoms. + >>> gam0 = gng.gam0, etc. # Access gamma values directly. + >>> gam0r = gng.gam0r # Or access real and imaginary parts separately. >>> gam0i = gng.gam0i + See also: `NGGCorrelation`, `GGNCorrelation`, `NNGCorrelation`, `GGGCorrelation`, + `NGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -626,9 +649,16 @@ def calculateGam(self, *, grg=None): After calling this function, the attributes ``gam0``, ``gam1``, ``vargam0``, ``vargam1``, and ``cov`` will correspond to the compensated values (if grg is provided). - The raw, uncompensated values are available as ``raw_gam0`` , ``raw_gam1``, + The raw, uncompensated values are available as ``raw_gam0``, ``raw_gam1``, ``raw_vargam0``, and ``raw_vargam1``. + .. note:: + + The returned variance estimates (``vargam0`` and ``vargam1``) are computed according + to this object's ``var_method`` setting, specified when constructing the object + (default: ``'shot'``). Internally, this method calls `Corr3.estimate_cov`; see that + method for details about available variance and covariance estimation schemes. + Parameters: grg (GNGCorrelation): The cross-correlation using random locations as the lenses (GRG), if desired. (default: None) @@ -703,9 +733,9 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" gam0r The real part of the estimator of :math:`\Gamma_0` - gam0i The imag part of the estimator of :math:`\Gamma_0` + gam0i The imaginary part of the estimator of :math:`\Gamma_0` gam1r The real part of the estimator of :math:`\Gamma_1` - gam1i The imag part of the estimator of :math:`\Gamma_1` + gam1i The imaginary part of the estimator of :math:`\Gamma_1` sigma_gam0 The sqrt of the variance estimate of :math:`\Gamma_0` sigma_gam1 The sqrt of the variance estimate of :math:`\Gamma_1` """) @@ -734,7 +764,7 @@ class GGNCorrelation(Corr3): r"""This class handles the calculation and storage of a 3-point shear-shear-count correlation function. - With this class, points 1 and 2 of the triangle (i.e. the vertces opposite d2,d3) are the ones + With this class, points 1 and 2 of the triangle (i.e. the vertices opposite d1, d2) are the ones with the shear values. Use `NGGCorrelation` and `GNGCorrelation` for classes with the shears in the other positions. @@ -755,7 +785,7 @@ class GGNCorrelation(Corr3): names you find most convenient: ``gam0``, ``gam1``, ``gam2`` and ``gam3`` are all valid attributes, which return the corresponding value. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -766,28 +796,36 @@ class GGNCorrelation(Corr3): gam1: The 1st "natural" correlation function, :math:`\Gamma_1`. gam2: The 2nd "natural" correlation function, :math:`\Gamma_2`. gam3: The 3rd "natural" correlation function, :math:`\Gamma_3`. - vargam0: The variance estimate of :math:`\Gamma_0`, only including the shot noise. - vargam1: The variance estimate of :math:`\Gamma_1`, only including the shot noise. - vargam2: The variance estimate of :math:`\Gamma_2`, only including the shot noise. - vargam3: The variance estimate of :math:`\Gamma_3`, only including the shot noise. + vargam0: The variance estimate of :math:`\Gamma_0`, computed according to ``var_method`` + (default: ``'shot'``). + vargam1: The variance estimate of :math:`\Gamma_1`, computed according to ``var_method`` + (default: ``'shot'``). + vargam2: The variance estimate of :math:`\Gamma_2`, computed according to ``var_method`` + (default: ``'shot'``). + vargam3: The variance estimate of :math:`\Gamma_3`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows:: >>> ggn = treecorr.GGNCorrelation(config) - >>> ggn.process(cat1, cat2) # Compute cross-correlation of two fields. - >>> ggn.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> ggn.write(file_name) # Write out to a file. - >>> ggr.process(cat1, rand) # Compute cross-correlation with randoms. - >>> ggn.calculateZeta(ggr=ggr) # Calculate zeta using randoms - >>> gam0 = ggn.gam0, etc. # Access gamma values directly. - >>> gam0r = ggn.gam0r # Or access real and imag parts separately. + >>> ggn.process(cat1, cat2) # Compute the cross-correlation of two fields. + >>> # ggn.process(cat1, cat2, cat3) # ... or of three fields. + >>> ggn.write(file_name) # Write out to a file. + >>> ggr.process(cat1, rand) # Compute the random cross-correlation. + >>> ggn.calculateZeta(ggr=ggr) # Calculate zeta using randoms. + >>> gam0 = ggn.gam0, etc. # Access gamma values directly. + >>> gam0r = ggn.gam0r # Or access real and imaginary parts separately. >>> gam0i = ggn.gam0i + See also: `NGGCorrelation`, `GNGCorrelation`, `NNGCorrelation`, `GGGCorrelation`, + `NGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -979,9 +1017,16 @@ def calculateGam(self, *, ggr=None): After calling this function, the attributes ``gam0``, ``gam1``, ``vargam0``, ``vargam1``, and ``cov`` will correspond to the compensated values (if ggr is provided). - The raw, uncompensated values are available as ``raw_gam0`` , ``raw_gam1``, + The raw, uncompensated values are available as ``raw_gam0``, ``raw_gam1``, ``raw_vargam0``, and ``raw_vargam1``. + .. note:: + + The returned variance estimates (``vargam0`` and ``vargam1``) are computed according + to this object's ``var_method`` setting, specified when constructing the object + (default: ``'shot'``). Internally, this method calls `Corr3.estimate_cov`; see that + method for details about available variance and covariance estimation schemes. + Parameters: ggr (GGNCorrelation): The cross-correlation using random locations as the lenses (GGR), if desired. (default: None) @@ -1003,7 +1048,7 @@ def calculateGam(self, *, ggr=None): if (ggr.npatch3 not in (1,self.npatch3) or ggr.npatch1 != self.npatch1 or ggr.npatch2 != self.npatch2): - raise RuntimeError("GRG must be run with the same patches as GDG") + raise RuntimeError("GGR must be run with the same patches as GGD") if len(self.results) > 0: # If there are any ggr patch pairs that aren't in results (e.g. due to different @@ -1056,9 +1101,9 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" gam0r The real part of the estimator of :math:`\Gamma_0` - gam0i The imag part of the estimator of :math:`\Gamma_0` + gam0i The imaginary part of the estimator of :math:`\Gamma_0` gam1r The real part of the estimator of :math:`\Gamma_1` - gam1i The imag part of the estimator of :math:`\Gamma_1` + gam1i The imaginary part of the estimator of :math:`\Gamma_1` sigma_gam0 The sqrt of the variance estimate of :math:`\Gamma_0` sigma_gam1 The sqrt of the variance estimate of :math:`\Gamma_1` """) diff --git a/treecorr/nkcorrelation.py b/treecorr/nkcorrelation.py index 6315a4d3..25db2b37 100644 --- a/treecorr/nkcorrelation.py +++ b/treecorr/nkcorrelation.py @@ -34,10 +34,10 @@ class NKCorrelation(Corr2): correlations of non-shear quantities, e.g. the sizes or concentrations of galaxies, around a set of lenses, where "kappa" would be the measurements of these quantities. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. - In addition to the attributes common to all `Corr3` subclasses, objects of this class + In addition to the attributes common to all `Corr2` subclasses, objects of this class hold the following attributes: Attributes: @@ -59,15 +59,18 @@ class NKCorrelation(Corr2): The typical usage pattern is as follows: >>> nk = treecorr.NKCorrelation(config) - >>> nk.process(cat1,cat2) # Compute the cross-correlation function. - >>> nk.write(file_name) # Write out to a file. - >>> xi = nk.xi # Or access the correlation function directly. + >>> nk.process(cat1, cat2) # Compute the cross-correlation. + >>> nk.write(file_name) # Write out to a file. + >>> xi = nk.xi # Or access the correlation function directly. + + See also: `NGCorrelation`, `KKCorrelation`, `NKKCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -158,7 +161,14 @@ def calculateXi(self, *, rk=None): After calling this function, the attributes ``xi``, ``varxi`` and ``cov`` will correspond to the compensated values (if rk is provided). The raw, uncompensated values are - available as ``rawxi`` and ``raw_varxi``. + available as ``raw_xi`` and ``raw_varxi``. + + .. note:: + + The returned variance estimate (``varxi``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr2.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: rk (NKCorrelation): The cross-correlation using random locations as the lenses (RK), @@ -234,7 +244,7 @@ def write(self, file_name, * ,rk=None, file_type=None, precision=None, that fell into each bin kappa The mean value :math:`\langle \kappa\rangle(r)` sigma The sqrt of the variance estimate of - :math:`\langle \kappa\rangle` + :math:`\langle \kappa\rangle(r)` weight The total weight contributing to each bin npairs The total number of pairs in each bin ========== ========================================================= @@ -249,7 +259,7 @@ def write(self, file_name, * ,rk=None, file_type=None, precision=None, if desired. (default: None) file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) diff --git a/treecorr/nkkcorrelation.py b/treecorr/nkkcorrelation.py index 55159f07..02fb39ac 100644 --- a/treecorr/nkkcorrelation.py +++ b/treecorr/nkkcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: nnkcorrelation +.. module:: nkkcorrelation """ import numpy as np @@ -30,7 +30,7 @@ class NKKCorrelation(Corr3): scalar value. Use `KNKCorrelation` and `KKNCorrelation` for classes with the scalar in the other two positions. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -38,26 +38,30 @@ class NKKCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance estimate, only including the shot noise propagated into the - final correlation. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows: >>> nkk = treecorr.NKKCorrelation(config) - >>> nkk.process(cat1, cat2) # Compute cross-correlation of two fields. - >>> nkk.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> nkk.write(file_name) # Write out to a file. - >>> rkk.process(rand, cat2) # Compute cross-correlation with randoms. - >>> nkk.calculateZeta(rkk=rkk) # Calculate zeta using randoms - >>> zeta = nkk.zeta # Access correlation function - >>> zetar = nkk.zetar # Or access real and imaginary parts separately + >>> nkk.process(cat1, cat2) # Compute the cross-correlation of two fields. + >>> # nkk.process(cat1, cat2, cat3) # ... or of three fields. + >>> nkk.write(file_name) # Write out to a file. + >>> rkk.process(rand, cat2) # Compute the random cross-correlation. + >>> nkk.calculateZeta(rkk=rkk) # Calculate zeta using randoms. + >>> zeta = nkk.zeta # Access the correlation function. + >>> zetar = nkk.zetar # Or access real and imaginary parts separately. >>> zetai = nkk.zetai + See also: `KNKCorrelation`, `KKNCorrelation`, `NNKCorrelation`, `KKKCorrelation`, + `NKCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -153,7 +157,14 @@ def calculateZeta(self, *, rkk=None): After calling this function, the attributes ``zeta``, ``varzeta`` and ``cov`` will correspond to the compensated values (if rkk is provided). The raw, uncompensated values - are available as ``rawxi`` and ``raw_varxi``. + are available as ``raw_zeta`` and ``raw_varzeta``. + + .. note:: + + The returned variance estimate (``varzeta``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr3.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: rkk (NKKCorrelation): The cross-correlation using random locations as the lenses (RKK), @@ -255,7 +266,7 @@ class KNKCorrelation(Corr3): scalar value. Use `NKKCorrelation` and `KKNCorrelation` for classes with the scalar in the other two positions. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -263,26 +274,30 @@ class KNKCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance estimate, only including the shot noise propagated into the - final correlation. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows: - >>> knk = treecorr.NKKCorrelation(config) - >>> knk.process(cat1, cat2, cat1) # Compute cross-correlation of two fields. - >>> knk.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> knk.write(file_name) # Write out to a file. - >>> krk.process(cat1, rand, cat1) # Compute cross-correlation with randoms. - >>> knk.calculateZeta(krk=krk) # Calculate zeta using randoms - >>> zeta = knk.zeta # Access correlation function - >>> zetar = knk.zetar # Or access real and imaginary parts separately + >>> knk = treecorr.KNKCorrelation(config) + >>> knk.process(cat1, cat2, cat1) # Compute the cross-correlation of two fields. + >>> # knk.process(cat1, cat2, cat3) # ... or of three fields. + >>> knk.write(file_name) # Write out to a file. + >>> krk.process(cat1, rand, cat1) # Compute the random cross-correlation. + >>> knk.calculateZeta(krk=krk) # Calculate zeta using randoms. + >>> zeta = knk.zeta # Access the correlation function. + >>> zetar = knk.zetar # Or access real and imaginary parts separately. >>> zetai = knk.zetai + See also: `NKKCorrelation`, `KKNCorrelation`, `NNKCorrelation`, `KKKCorrelation`, + `NKCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -378,10 +393,17 @@ def calculateZeta(self, *, krk=None): After calling this function, the attributes ``zeta``, ``varzeta`` and ``cov`` will correspond to the compensated values (if krk is provided). The raw, uncompensated values - are available as ``rawxi`` and ``raw_varxi``. + are available as ``raw_zeta`` and ``raw_varzeta``. + + .. note:: + + The returned variance estimate (``varzeta``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr3.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: - krk (KNKCorrelation): The cross-correlation using random locations as the lenses (RKK), + krk (KNKCorrelation): The cross-correlation using random locations as the lenses (KRK), if desired. (default: None) Returns: @@ -480,7 +502,7 @@ class KKNCorrelation(Corr3): scalar value. Use `NKKCorrelation` and `KNKCorrelation` for classes with the scalar in the other two positions. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -488,26 +510,30 @@ class KKNCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance estimate, only including the shot noise propagated into the - final correlation. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows: >>> kkn = treecorr.KKNCorrelation(config) - >>> kkn.process(cat1, cat2) # Compute cross-correlation of two fields. - >>> kkn.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> kkn.write(file_name) # Write out to a file. - >>> kkr.process(cat1, rand) # Compute cross-correlation with randoms. - >>> kkn.calculateZeta(kkr=kkr) # Calculate zeta using randoms - >>> zeta = kkn.zeta # Access correlation function - >>> zetar = kkn.zetar # Or access real and imaginary parts separately + >>> kkn.process(cat1, cat2) # Compute the cross-correlation of two fields. + >>> # kkn.process(cat1, cat2, cat3) # ... or of three fields. + >>> kkn.write(file_name) # Write out to a file. + >>> kkr.process(cat1, rand) # Compute the random cross-correlation. + >>> kkn.calculateZeta(kkr=kkr) # Calculate zeta using randoms. + >>> zeta = kkn.zeta # Access the correlation function. + >>> zetar = kkn.zetar # Or access real and imaginary parts separately. >>> zetai = kkn.zetai + See also: `NKKCorrelation`, `KNKCorrelation`, `NNKCorrelation`, `KKKCorrelation`, + `NKCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -598,15 +624,22 @@ def calculateZeta(self, *, kkr=None): - If kkr is None, the simple correlation function (self.zeta) is returned. - If kkr is not None, then a compensated calculation is done: - :math:`\zeta = (DKK - RKK)`, where DKK represents the correlation of the kappa - field with the data points and RKK represents the correlation with random points. + :math:`\zeta = (KKD - KKR)`, where KKD represents the correlation of the kappa + field with the data points and KKR represents the correlation with random points. After calling this function, the attributes ``zeta``, ``varzeta`` and ``cov`` will correspond to the compensated values (if kkr is provided). The raw, uncompensated values - are available as ``rawxi`` and ``raw_varxi``. + are available as ``raw_zeta`` and ``raw_varzeta``. + + .. note:: + + The returned variance estimate (``varzeta``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr3.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: - kkr (KKNCorrelation): The cross-correlation using random locations as the lenses (RKK), + kkr (KKNCorrelation): The cross-correlation using random locations as the lenses (KKR), if desired. (default: None) Returns: diff --git a/treecorr/nncorrelation.py b/treecorr/nncorrelation.py index 1c5c3ed5..537177f5 100644 --- a/treecorr/nncorrelation.py +++ b/treecorr/nncorrelation.py @@ -25,9 +25,9 @@ class NNCorrelation(Corr2): r"""This class handles the calculation and storage of a 2-point count-count correlation - function. i.e. the regular density correlation function. + function, i.e. the regular density correlation function. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr2` subclasses, objects of this class @@ -47,19 +47,22 @@ class NNCorrelation(Corr2): The typical usage pattern is as follows: >>> nn = treecorr.NNCorrelation(config) - >>> nn.process(cat) # For auto-correlation. - >>> nn.process(cat1,cat2) # For cross-correlation. - >>> rr.process... # Likewise for random-random correlations - >>> dr.process... # If desired, also do data-random correlations - >>> rd.process... # For cross-correlations, also do the reverse. - >>> nn.write(file_name,rr=rr,dr=dr,rd=rd) # Write out to a file. - >>> xi,varxi = nn.calculateXi(rr=rr,dr=dr,rd=rd) # Or get correlation function directly. + >>> nn.process(cat) # Compute the auto-correlation. + >>> # nn.process(cat1, cat2) # ... or the cross-correlation. + >>> rr.process(...) # Likewise for random-random correlations. + >>> dr.process(...) # If desired, also do data-random correlations. + >>> rd.process(...) # For cross-correlations, also do the reverse. + >>> xi, varxi = nn.calculateXi(rr=rr, dr=dr, rd=rd) # Calculate the correlation function. + >>> nn.write(file_name, rr=rr, dr=dr, rd=rd) # Write out to a file. + + See also: `NGCorrelation`, `NKCorrelation`, `NNNCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -145,7 +148,7 @@ def finalize(self): The `Corr2.process_auto` and `Corr2.process_cross` commands accumulate values in each bin, so they can be called multiple times if appropriate. Afterwards, this command - finishes the calculation of meanr, meanlogr by dividing by the total weight. + finishes the calculation of ``meanr`` and ``meanlogr`` by dividing by the total weight. """ self._finalize() @@ -207,7 +210,7 @@ def getWeight(self): """The weight array for the current correlation object as a 1-d array. This is the weight array corresponding to `getStat`. In this case, it is the denominator - RR from the calculation done by calculateXi(). + RR from the calculation done by `calculateXi`. """ if self._rr_weight is not None: return self._rr_weight.ravel() @@ -216,20 +219,20 @@ def getWeight(self): def calculateXi(self, *, rr, dr=None, rd=None): r"""Calculate the correlation function given another correlation function of random - points using the same mask, and possibly cross correlations of the data and random. + points using the same mask, and possibly cross-correlations of the data and random. The rr value is the `NNCorrelation` function for random points. - For a signal that involves a cross correlations, there should be two random + For a signal that involves a cross-correlation, there should be two random cross-correlations: data-random and random-data, given as dr and rd. - If dr is None, the simple correlation function :math:`\xi = (DD/RR - 1)` is used. - - if dr is given and rd is None, then the Landy-Szalay estimate + - If dr is given and rd is None, then the Landy-Szalay estimate :math:`\xi = (DD - 2DR + RR)/RR` is used. - If dr and rd are both given, then :math:`\xi = (DD - DR - RD + RR)/RR` is used. - This is the correlate of the Landy-Szalay statistic for cross-correlations, where + This is the analog of the Landy-Szalay statistic for cross-correlations, where the first and second catalogs do not represent the same underlying field. - where DD is the data NN correlation function, which is the current object. + Here DD is the data NN correlation function, which is the current object. .. note:: @@ -239,9 +242,16 @@ def calculateXi(self, *, rr, dr=None, rd=None): estimates, you need to set ``var_method`` to something else and use patches in the input catalog(s). cf. `Covariance Estimates`. - After calling this method, you can use the `Corr2.estimate_cov` method or use this - correlation object in the `estimate_multi_cov` function. Also, the calculated xi and - varxi returned from this function will be available as attributes. + After calling this method, you can use this correlation object in the + `estimate_multi_cov` function. Also, the calculated xi and varxi returned from this + function will be available as attributes. + + .. note:: + + The returned variance estimate (``varxi``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr2.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: rr (NNCorrelation): The auto-correlation of the random field (RR) @@ -249,7 +259,7 @@ def calculateXi(self, *, rr, dr=None, rd=None): desired, in which case the Landy-Szalay estimator will be calculated. (default: None) rd (NNCorrelation): The cross-correlation of the randoms with data (RD), if - desired. (default: None, which means use rd=dr) + desired. (default: None, which means to use rd=dr) Returns: Tuple containing: @@ -476,11 +486,11 @@ def write(self, file_name, *, rr=None, dr=None, rd=None, file_type=None, precisi dr (NNCorrelation): The cross-correlation of the data with randoms (DR), if desired. (default: None) rd (NNCorrelation): The cross-correlation of the randoms with data (RD), if - desired. (default: None, which means use rd=dr) + desired. (default: None, which means to use rd=dr) file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. @@ -578,9 +588,11 @@ def from_file(cls, file_name, *, file_type=None, logger=None, rng=None): file_name (str): The name of the file to read in. file_type (str): The type of file ('ASCII', 'FITS', or 'HDF'). (default: determine the type automatically from the extension of file_name.) - logger (Logger): If desired, a logger object to use for logging. (default: None) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for bootstrap - random number generation. (default: None) + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object to use for logging. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + bootstrap random number generation. (default: None) Returns: An NNCorrelation object, constructed from the information in the file. @@ -688,12 +700,13 @@ def calculateNapSq(self, *, rr, R=None, dr=None, rd=None, m2_uform=None): Parameters: rr (NNCorrelation): The auto-correlation of the random field (RR) - R (array): The R values at which to calculate the aperture mass statistics. - (default: None, which means use self.rnom) + R (:class:`numpy.ndarray`): + The R values at which to calculate the aperture mass statistics. + (default: None, which means to use self.rnom) dr (NNCorrelation): The cross-correlation of the data with randoms (DR), if desired. (default: None) rd (NNCorrelation): The cross-correlation of the randoms with data (RD), if - desired. (default: None, which means use rd=dr) + desired. (default: None, which means to use rd=dr) m2_uform (str): Which form to use for the aperture mass. (default: 'Crittenden'; this value can also be given in the constructor in the config dict.) diff --git a/treecorr/nngcorrelation.py b/treecorr/nngcorrelation.py index 92f35125..559787cc 100644 --- a/treecorr/nngcorrelation.py +++ b/treecorr/nngcorrelation.py @@ -30,7 +30,7 @@ class GNNCorrelation(Corr3): shear value. Use `NGNCorrelation` and `NNGCorrelation` for classes with the shear in the other two positions. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -38,27 +38,31 @@ class GNNCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance estimate, only including the shot noise propagated into the - final correlation. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows: >>> gnn = treecorr.GNNCorrelation(config) - >>> gnn.process(cat1, cat2) # Compute cross-correlation of two fields. - >>> gnn.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> grr.process(cat1, rand) # Compute cross-correlation with randoms. - >>> gdr.process(cat1, cat2, rand) # Compute cross-correlation with randoms and data - >>> gnn.write(file_name) # Write out to a file. - >>> gnn.calculateZeta(grr=grr, gdr=gdr) # Calculate zeta using randoms - >>> zeta = gnn.zeta # Access correlation function - >>> zetar = gnn.zetar # Or access real and imaginary parts separately + >>> gnn.process(cat1, cat2) # Compute the cross-correlation of two fields. + >>> # gnn.process(cat1, cat2, cat3) # ... or of three fields. + >>> grr.process(cat1, rand) # Compute the random cross-correlation. + >>> gdr.process(cat1, cat2, rand) # Optionally compute data-random cross-correlation. + >>> gnn.write(file_name) # Write out to a file. + >>> gnn.calculateZeta(grr=grr, gdr=gdr) # Calculate zeta using randoms. + >>> zeta = gnn.zeta # Access the correlation function. + >>> zetar = gnn.zetar # Or access real and imaginary parts separately. >>> zetai = gnn.zetai + See also: `NGNCorrelation`, `NNGCorrelation`, `NGGCorrelation`, `NNNCorrelation`, + `NGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -160,22 +164,29 @@ def _clear(self): def calculateZeta(self, *, grr=None, gdr=None, grd=None): r"""Calculate the correlation function given another correlation function of random - points using the same mask, and possibly cross correlations of the data and random. + points using the same mask, and possibly cross-correlations of the data and random. The grr value is the `GNNCorrelation` function for random points with the shear field. - One can also provide a cross correlation of the count data with randoms and the shear. + One can also provide a cross-correlation of the count data with randoms and the shear. - If grr is None, the simple correlation function (self.zeta) is returned. - If only grr is given the compensated value :math:`\zeta = GDD - GRR` is returned. - - if gdr is given and grd is None (or vice versa), then :math:`\zeta = GDD - 2GDR + GRR` + - If gdr is given and grd is None (or vice versa), then :math:`\zeta = GDD - 2GDR + GRR` is returned. - If gdr and grd are both given, then :math:`\zeta = GDD - GDR - GRD + GRR` is returned. - where GDD is the data GNN correlation function, which is the current object. + Here GDD is the data GNN correlation function, which is the current object. + + After calling this method, you can use this correlation object in the + `estimate_multi_cov` function. Also, the calculated zeta and varzeta returned from this + function will be available as attributes. + + .. note:: - After calling this method, you can use the `Corr2.estimate_cov` method or use this - correlation object in the `estimate_multi_cov` function. Also, the calculated zeta and - varzeta returned from this function will be available as attributes. + The returned variance estimate (``varzeta``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr3.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: grr (GNNCorrelation): The correlation of the random points with the shear field @@ -299,7 +310,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" zetar The real part of the estimator of :math:`\zeta` - zetai The imag part of the estimator of :math:`\zeta` + zetai The imaginary part of the estimator of :math:`\zeta` sigma_zeta The sqrt of the variance estimate of :math:`\zeta`. """) @@ -327,7 +338,7 @@ class NGNCorrelation(Corr3): shear value. Use `GNNCorrelation` and `NNGCorrelation` for classes with the shear in the other two positions. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -335,27 +346,31 @@ class NGNCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance estimate, only including the shot noise propagated into the - final correlation. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows: - >>> ngn = treecorr.GNNCorrelation(config) - >>> ngn.process(cat1, cat2, cat1) # Compute cross-correlation of two fields. - >>> ngn.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> rgr.process(rand, cat2, rand) # Compute cross-correlation with randoms. - >>> dgr.process(cat1, cat2, rand) # Compute cross-correlation with randoms and data - >>> ngn.write(file_name) # Write out to a file. - >>> ngn.calculateZeta(rgr=rgr, dgr=dgr) # Calculate zeta using randoms - >>> zeta = ngn.zeta # Access correlation function - >>> zetar = ngn.zetar # Or access real and imaginary parts separately + >>> ngn = treecorr.NGNCorrelation(config) + >>> ngn.process(cat1, cat2, cat1) # Compute the cross-correlation of two fields. + >>> # ngn.process(cat1, cat2, cat3) # ... or of three fields. + >>> rgr.process(rand, cat2, rand) # Compute the random cross-correlation. + >>> dgr.process(cat1, cat2, rand) # Optionally compute data-random cross-correlation. + >>> ngn.write(file_name) # Write out to a file. + >>> ngn.calculateZeta(rgr=rgr, dgr=dgr) # Calculate zeta using randoms. + >>> zeta = ngn.zeta # Access the correlation function. + >>> zetar = ngn.zetar # Or access real and imaginary parts separately. >>> zetai = ngn.zetai + See also: `GNNCorrelation`, `NNGCorrelation`, `NGGCorrelation`, `NNNCorrelation`, + `NGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -458,22 +473,29 @@ def _clear(self): def calculateZeta(self, *, rgr=None, dgr=None, rgd=None): r"""Calculate the correlation function given another correlation function of random - points using the same mask, and possibly cross correlations of the data and random. + points using the same mask, and possibly cross-correlations of the data and random. The rgr value is the `NGNCorrelation` function for random points with the shear field. - One can also provide a cross correlation of the count data with randoms and the shear. + One can also provide a cross-correlation of the count data with randoms and the shear. - If rgr is None, the simple correlation function (self.zeta) is returned. - If only rgr is given the compensated value :math:`\zeta = DGD - RGR` is returned. - - if dgr is given and rgd is None (or vice versa), then :math:`\zeta = DGD - 2DGR + RGR` + - If dgr is given and rgd is None (or vice versa), then :math:`\zeta = DGD - 2DGR + RGR` is returned. - If dgr and rgd are both given, then :math:`\zeta = DGD - DGR - RGD + RGR` is returned. - where DGD is the data NGN correlation function, which is the current object. + Here DGD is the data NGN correlation function, which is the current object. + + After calling this method, you can use this correlation object in the + `estimate_multi_cov` function. Also, the calculated zeta and varzeta returned from this + function will be available as attributes. - After calling this method, you can use the `Corr2.estimate_cov` method or use this - correlation object in the `estimate_multi_cov` function. Also, the calculated zeta and - varzeta returned from this function will be available as attributes. + .. note:: + + The returned variance estimate (``varzeta``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr3.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: rgr (NGNCorrelation): The correlation of the random points with the shear field @@ -597,7 +619,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" zetar The real part of the estimator of :math:`\zeta` - zetai The imag part of the estimator of :math:`\zeta` + zetai The imaginary part of the estimator of :math:`\zeta` sigma_zeta The sqrt of the variance estimate of :math:`\zeta`. """) @@ -625,7 +647,7 @@ class NNGCorrelation(Corr3): shear value. Use `GNNCorrelation` and `NGNCorrelation` for classes with the shear in the other two positions. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -633,27 +655,31 @@ class NNGCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance estimate, only including the shot noise propagated into the - final correlation. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows: >>> nng = treecorr.NNGCorrelation(config) - >>> nng.process(cat1, cat2) # Compute cross-correlation of two fields. - >>> nng.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> rrg.process(rand, cat2) # Compute cross-correlation with randoms. - >>> drg.process(cat1, rand, cat2) # Compute cross-correlation with randoms and data - >>> nng.write(file_name) # Write out to a file. - >>> nng.calculateZeta(rrg=rrg, drg=drg) # Calculate zeta using randoms - >>> zeta = nng.zeta # Access correlation function - >>> zetar = nng.zetar # Or access real and imaginary parts separately + >>> nng.process(cat1, cat2) # Compute the cross-correlation of two fields. + >>> # nng.process(cat1, cat2, cat3) # ... or of three fields. + >>> rrg.process(rand, cat2) # Compute the random cross-correlation. + >>> drg.process(cat1, rand, cat2) # Optionally compute data-random cross-correlation. + >>> nng.write(file_name) # Write out to a file. + >>> nng.calculateZeta(rrg=rrg, drg=drg) # Calculate zeta using randoms. + >>> zeta = nng.zeta # Access the correlation function. + >>> zetar = nng.zetar # Or access real and imaginary parts separately. >>> zetai = nng.zetai + See also: `GNNCorrelation`, `NGNCorrelation`, `NGGCorrelation`, `NNNCorrelation`, + `NGCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -756,22 +782,29 @@ def _clear(self): def calculateZeta(self, *, rrg=None, drg=None, rdg=None): r"""Calculate the correlation function given another correlation function of random - points using the same mask, and possibly cross correlations of the data and random. + points using the same mask, and possibly cross-correlations of the data and random. The rrg value is the `NNGCorrelation` function for random points with the shear field. - One can also provide a cross correlation of the count data with randoms and the shear. + One can also provide a cross-correlation of the count data with randoms and the shear. - If rrg is None, the simple correlation function (self.zeta) is returned. - If only rrg is given the compensated value :math:`\zeta = DDG - RRG` is returned. - - if drg is given and rdg is None (or vice versa), then :math:`\zeta = DDG - 2DRG + RRG` + - If drg is given and rdg is None (or vice versa), then :math:`\zeta = DDG - 2DRG + RRG` is returned. - If drg and rdg are both given, then :math:`\zeta = DDG - DRG - RDG + RRG` is returned. - where DDG is the data NNG correlation function, which is the current object. + Here DDG is the data NNG correlation function, which is the current object. + + After calling this method, you can use this correlation object in the + `estimate_multi_cov` function. Also, the calculated zeta and varzeta returned from this + function will be available as attributes. + + .. note:: - After calling this method, you can use the `Corr2.estimate_cov` method or use this - correlation object in the `estimate_multi_cov` function. Also, the calculated zeta and - varzeta returned from this function will be available as attributes. + The returned variance estimate (``varzeta``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr3.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: rrg (NNGCorrelation): The correlation of the random points with the shear field @@ -895,7 +928,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result write.__doc__ = Corr3.write.__doc__.format( r""" zetar The real part of the estimator of :math:`\zeta` - zetai The imag part of the estimator of :math:`\zeta` + zetai The imaginary part of the estimator of :math:`\zeta` sigma_zeta The sqrt of the variance estimate of :math:`\zeta`. """) diff --git a/treecorr/nnkcorrelation.py b/treecorr/nnkcorrelation.py index 0107853e..6dfdfbfb 100644 --- a/treecorr/nnkcorrelation.py +++ b/treecorr/nnkcorrelation.py @@ -30,7 +30,7 @@ class KNNCorrelation(Corr3): scalar value. Use `NKNCorrelation` and `NNKCorrelation` for classes with the scalar in the other two positions. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -38,27 +38,31 @@ class KNNCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance estimate, only including the shot noise propagated into the - final correlation. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows: >>> knn = treecorr.KNNCorrelation(config) - >>> knn.process(cat1, cat2) # Compute cross-correlation of two fields. - >>> knn.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> krr.process(cat1, rand) # Compute cross-correlation with randoms. - >>> kdr.process(cat1, cat2, rand) # Compute cross-correlation with randoms and data - >>> knn.write(file_name) # Write out to a file. - >>> knn.calculateZeta(krr=krr, kdr=kdr) # Calculate zeta using randoms - >>> zeta = knn.zeta # Access correlation function - >>> zetar = knn.zetar # Or access real and imaginary parts separately + >>> knn.process(cat1, cat2) # Compute the cross-correlation of two fields. + >>> # knn.process(cat1, cat2, cat3) # ... or of three fields. + >>> krr.process(cat1, rand) # Compute the random cross-correlation. + >>> kdr.process(cat1, cat2, rand) # Optionally compute data-random cross-correlation. + >>> knn.write(file_name) # Write out to a file. + >>> knn.calculateZeta(krr=krr, kdr=kdr) # Calculate zeta using randoms. + >>> zeta = knn.zeta # Access the correlation function. + >>> zetar = knn.zetar # Or access real and imaginary parts separately. >>> zetai = knn.zetai + See also: `NKNCorrelation`, `NNKCorrelation`, `NKKCorrelation`, `NNNCorrelation`, + `NKCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -152,22 +156,29 @@ def _clear(self): def calculateZeta(self, *, krr=None, kdr=None, krd=None): r"""Calculate the correlation function given another correlation function of random - points using the same mask, and possibly cross correlations of the data and random. + points using the same mask, and possibly cross-correlations of the data and random. The krr value is the `KNNCorrelation` function for random points with the scalar field. - One can also provide a cross correlation of the count data with randoms and the scalar. + One can also provide a cross-correlation of the count data with randoms and the scalar. - If krr is None, the simple correlation function (self.zeta) is returned. - If only krr is given the compensated value :math:`\zeta = KDD - KRR` is returned. - - if kdr is given and krd is None (or vice versa), then :math:`\zeta = KDD - 2KDR + KRR` + - If kdr is given and krd is None (or vice versa), then :math:`\zeta = KDD - 2KDR + KRR` is returned. - If kdr and krd are both given, then :math:`\zeta = KDD - KDR - KRD + KRR` is returned. - where KDD is the data KNN correlation function, which is the current object. + Here KDD is the data KNN correlation function, which is the current object. + + After calling this method, you can use this correlation object in the + `estimate_multi_cov` function. Also, the calculated zeta and varzeta returned from this + function will be available as attributes. + + .. note:: - After calling this method, you can use the `Corr2.estimate_cov` method or use this - correlation object in the `estimate_multi_cov` function. Also, the calculated zeta and - varzeta returned from this function will be available as attributes. + The returned variance estimate (``varzeta``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr3.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: krr (KNNCorrelation): The correlation of the random points with the scalar field @@ -328,7 +339,7 @@ class NKNCorrelation(Corr3): scalar value. Use `KNNCorrelation` and `NNKCorrelation` for classes with the scalar in the other two positions. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -336,27 +347,31 @@ class NKNCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance estimate, only including the shot noise propagated into the - final correlation. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows: >>> nkn = treecorr.NKNCorrelation(config) - >>> nkn.process(cat1, cat2, cat1) # Compute cross-correlation of two fields. - >>> nkn.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> rkr.process(rand, cat2, rand) # Compute cross-correlation with randoms. - >>> dkr.process(cat1, cat2, rand) # Compute cross-correlation with randoms and data - >>> nkn.write(file_name) # Write out to a file. - >>> nkn.calculateZeta(rkr=rkr, dkr=dkr) # Calculate zeta using randoms - >>> zeta = nkn.zeta # Access correlation function - >>> zetar = nkn.zetar # Or access real and imaginary parts separately + >>> nkn.process(cat1, cat2, cat1) # Compute the cross-correlation of two fields. + >>> # nkn.process(cat1, cat2, cat3) # ... or of three fields. + >>> rkr.process(rand, cat2, rand) # Compute the random cross-correlation. + >>> dkr.process(cat1, cat2, rand) # Optionally compute data-random cross-correlation. + >>> nkn.write(file_name) # Write out to a file. + >>> nkn.calculateZeta(rkr=rkr, dkr=dkr) # Calculate zeta using randoms. + >>> zeta = nkn.zeta # Access the correlation function. + >>> zetar = nkn.zetar # Or access real and imaginary parts separately. >>> zetai = nkn.zetai + See also: `KNNCorrelation`, `NNKCorrelation`, `NKKCorrelation`, `NNNCorrelation`, + `NKCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -450,22 +465,29 @@ def _clear(self): def calculateZeta(self, *, rkr=None, dkr=None, rkd=None): r"""Calculate the correlation function given another correlation function of random - points using the same mask, and possibly cross correlations of the data and random. + points using the same mask, and possibly cross-correlations of the data and random. The rkr value is the `NKNCorrelation` function for random points with the scalar field. - One can also provide a cross correlation of the count data with randoms and the scalar. + One can also provide a cross-correlation of the count data with randoms and the scalar. - If rkr is None, the simple correlation function (self.zeta) is returned. - If only rkr is given the compensated value :math:`\zeta = DKD - RKR` is returned. - - if dkr is given and rkd is None (or vice versa), then :math:`\zeta = DKD - 2DKR + RKR` + - If dkr is given and rkd is None (or vice versa), then :math:`\zeta = DKD - 2DKR + RKR` is returned. - If dkr and rkd are both given, then :math:`\zeta = DKD - DKR - RKD + RKR` is returned. - where DKD is the data NKN correlation function, which is the current object. + Here DKD is the data NKN correlation function, which is the current object. + + After calling this method, you can use this correlation object in the + `estimate_multi_cov` function. Also, the calculated zeta and varzeta returned from this + function will be available as attributes. - After calling this method, you can use the `Corr2.estimate_cov` method or use this - correlation object in the `estimate_multi_cov` function. Also, the calculated zeta and - varzeta returned from this function will be available as attributes. + .. note:: + + The returned variance estimate (``varzeta``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr3.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: rkr (NKNCorrelation): The correlation of the random points with the scalar field @@ -626,7 +648,7 @@ class NNKCorrelation(Corr3): scalar value. Use `KNNCorrelation` and `NKNCorrelation` for classes with the scalar in the other two positions. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr3` subclasses, objects of this class @@ -634,27 +656,31 @@ class NNKCorrelation(Corr3): Attributes: zeta: The correlation function, :math:`\zeta`. - varzeta: The variance estimate, only including the shot noise propagated into the - final correlation. + varzeta: The variance estimate of :math:`\zeta`, computed according to ``var_method`` + (default: ``'shot'``). The typical usage pattern is as follows: >>> nnk = treecorr.NNKCorrelation(config) - >>> nnk.process(cat1, cat2) # Compute cross-correlation of two fields. - >>> nnk.process(cat1, cat2, cat3) # Compute cross-correlation of three fields. - >>> rrk.process(rand, cat2) # Compute cross-correlation with randoms. - >>> drk.process(cat1, rand, cat2) # Compute cross-correlation with randoms and data - >>> nnk.write(file_name) # Write out to a file. - >>> nnk.calculateZeta(rrk=rrk, drk=drk) # Calculate zeta using randoms - >>> zeta = nnk.zeta # Access correlation function - >>> zetar = nnk.zetar # Or access real and imaginary parts separately + >>> nnk.process(cat1, cat2) # Compute the cross-correlation of two fields. + >>> # nnk.process(cat1, cat2, cat3) # ... or of three fields. + >>> rrk.process(rand, cat2) # Compute the random cross-correlation. + >>> drk.process(cat1, rand, cat2) # Optionally compute data-random cross-correlation. + >>> nnk.write(file_name) # Write out to a file. + >>> nnk.calculateZeta(rrk=rrk, drk=drk) # Calculate zeta using randoms. + >>> zeta = nnk.zeta # Access the correlation function. + >>> zetar = nnk.zetar # Or access real and imaginary parts separately. >>> zetai = nnk.zetai + See also: `KNNCorrelation`, `NKNCorrelation`, `NKKCorrelation`, `NNNCorrelation`, + `NKCorrelation`. + Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -748,22 +774,29 @@ def _clear(self): def calculateZeta(self, *, rrk=None, drk=None, rdk=None): r"""Calculate the correlation function given another correlation function of random - points using the same mask, and possibly cross correlations of the data and random. + points using the same mask, and possibly cross-correlations of the data and random. The rrk value is the `NNKCorrelation` function for random points with the scalar field. - One can also provide a cross correlation of the count data with randoms and the scalar. + One can also provide a cross-correlation of the count data with randoms and the scalar. - If rrk is None, the simple correlation function (self.zeta) is returned. - If only rrk is given the compensated value :math:`\zeta = DDK - RRK` is returned. - - if drk is given and rdk is None (or vice versa), then :math:`\zeta = DDK - 2DRK + RRK` + - If drk is given and rdk is None (or vice versa), then :math:`\zeta = DDK - 2DRK + RRK` is returned. - If drk and rdk are both given, then :math:`\zeta = DDK - DRK - RDK + RRK` is returned. - where DDK is the data NNK correlation function, which is the current object. + Here DDK is the data NNK correlation function, which is the current object. + + After calling this method, you can use this correlation object in the + `estimate_multi_cov` function. Also, the calculated zeta and varzeta returned from this + function will be available as attributes. + + .. note:: - After calling this method, you can use the `Corr2.estimate_cov` method or use this - correlation object in the `estimate_multi_cov` function. Also, the calculated zeta and - varzeta returned from this function will be available as attributes. + The returned variance estimate (``varzeta``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr3.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: rrk (NNKCorrelation): The correlation of the random points with the scalar field diff --git a/treecorr/nnncorrelation.py b/treecorr/nnncorrelation.py index 2eddcbe4..6da5327f 100644 --- a/treecorr/nnncorrelation.py +++ b/treecorr/nnncorrelation.py @@ -27,24 +27,28 @@ class NNNCorrelation(Corr3): """This class handles the calculation and storage of a 3-point count-count-count correlation function. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr3` for a description of how the triangles are binned along with the attributes related to the different binning options. The typical usage pattern is as follows: >>> nnn = treecorr.NNNCorrelation(config) - >>> nnn.process(cat) # Compute auto-correlation. - >>> rrr.process(rand) # Compute auto-correlation of random field. - >>> drr.process(cat,rand) # If desired, compute data-random cross-correlations - >>> rdd.process(rand,cat) # Also compute cross-correlation with two data and one random - >>> nnn.write(file_name, rrr=rrr, drr=drr, ...) # Write out to a file. - >>> zeta,varzeta = nnn.calculateZeta(rrr=rrr, drr=drr, rdd=rdd) # Calculate zeta + >>> nnn.process(cat) # Compute the auto-correlation. + >>> # nnn.process(cat1, cat2, cat3) # ... or the cross-correlation. + >>> rrr.process(rand) # Compute the random auto-correlation. + >>> drr.process(cat, rand) # Optionally compute data-random cross-correlation. + >>> rdd.process(rand, cat) # Also compute random-data cross-correlation. + >>> zeta, varzeta = nnn.calculateZeta(rrr=rrr, drr=drr, rdd=rdd) # Calculate zeta. + >>> nnn.write(file_name, rrr=rrr, drr=drr, ...) # Write out to a file. + + See also: `NNCorrelation`, `NNGCorrelation`, `NNKCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr3`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -224,7 +228,7 @@ def toSAS(self, *, target=None, **kwargs): def calculateZeta(self, *, rrr, drr=None, rdd=None): r"""Calculate the 3pt function given another 3pt function of random - points using the same mask, and possibly cross correlations of the data and random. + points using the same mask, and possibly cross-correlations of the data and random. There are two possible formulae that are currently supported. @@ -251,7 +255,7 @@ def calculateZeta(self, *, rrr, drr=None, rdd=None): .. note:: - One might thing the formula should be :math:`\zeta = (DDD-3RDD+3DRR-RRR)/RRR` + One might think the formula should be :math:`\zeta = (DDD-3RDD+3DRR-RRR)/RRR` by analogy with the 2pt Landy-Szalay formula. However, the way these are calculated, the object we are calling RDD already includes triangles where R is in each of the 3 locations. So it is really more like RDD + DRD + DDR. @@ -266,10 +270,17 @@ def calculateZeta(self, *, rrr, drr=None, rdd=None): .. note:: This method is not valid for bin_type='LogMultipole'. I don't think there is - a straightforward way to go directly from the multipole expoansion of DDD and + a straightforward way to go directly from the multipole expansion of DDD and RRR to Zeta. Normally one would instead convert both to LogSAS binning (cf. `toSAS`) and then call `calculateZeta` with those. + .. note:: + + The returned variance estimate (``varzeta``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr3.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. + Parameters: rrr (NNNCorrelation): The auto-correlation of the random field (RRR) drr (NNNCorrelation): DRR if desired. (default: None) @@ -311,7 +322,7 @@ def calculateZeta(self, *, rrr, drr=None, rdd=None): else: self.zeta = self.weight - rdd.weight * rddf + drr.weight * drrf - denom - # Divide by DRR in all cases. + # Divide by RRR in all cases. if np.any(rrr.weight == 0): self.logger.warning("Warning: Some bins for the randoms had no triangles.") denom[rrr.weight==0] = 1. # guard against division by 0. @@ -553,7 +564,7 @@ def write(self, file_name, *, rrr=None, drr=None, rdd=None, file_type=None, prec file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. @@ -678,9 +689,11 @@ def from_file(cls, file_name, *, file_type=None, logger=None, rng=None): file_name (str): The name of the file to read in. file_type (str): The type of file ('ASCII', 'FITS', or 'HDF'). (default: determine the type automatically from the extension of file_name.) - logger (Logger): If desired, a logger object to use for logging. (default: None) - rng (RandomState): If desired, a numpy.random.RandomState instance to use for bootstrap - random number generation. (default: None) + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object to use for logging. (default: None) + rng (:class:`numpy.random.Generator`): + If desired, a ``Generator`` instance to use for + bootstrap random number generation. (default: None) Returns: An NNNCorrelation object, constructed from the information in the file. diff --git a/treecorr/nqcorrelation.py b/treecorr/nqcorrelation.py index 4c4efac8..a61572ed 100644 --- a/treecorr/nqcorrelation.py +++ b/treecorr/nqcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: ngcorrelation +.. module:: nqcorrelation """ import numpy as np @@ -26,7 +26,7 @@ class NQCorrelation(BaseNZCorrelation): r"""This class handles the calculation and storage of a 2-point count-quatrefoil correlation function, where a quatrefoil is any field with spin-4 rotational properties. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr2` subclasses, objects of this class @@ -53,15 +53,18 @@ class NQCorrelation(BaseNZCorrelation): The typical usage pattern is as follows: >>> nq = treecorr.NQCorrelation(config) - >>> nq.process(cat1,cat2) # Compute the cross-correlation. - >>> nq.write(file_name) # Write out to a file. - >>> xi = nq.xi # Or access the correlation function directly. + >>> nq.process(cat1, cat2) # Compute the cross-correlation. + >>> nq.write(file_name) # Write out to a file. + >>> xi, xi_im = nq.xi, nq.xi_im # Or access the correlation function directly. + + See also: `KQCorrelation`, `QQCorrelation`, `NZCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -103,7 +106,7 @@ def calculateXi(self, *, rq=None): After calling this function, the attributes ``xi``, ``xi_im``, ``varxi``, and ``cov`` will correspond to the compensated values (if rq is provided). The raw, uncompensated values - are available as ``rawxi``, ``raw_xi_im``, and ``raw_varxi``. + are available as ``raw_xi``, ``raw_xi_im``, and ``raw_varxi``. Parameters: rq (NQCorrelation): The cross-correlation using random locations as the lenses @@ -141,7 +144,7 @@ def write(self, file_name, *, rq=None, file_type=None, precision=None, center points qR_im The mean imaginary part of the quatrefoil field relative to the center points - sigma The sqrt of the variance estimate of either of these + sigma The sqrt of the variance estimate of each of these weight The total weight contributing to each bin npairs The total number of pairs in each bin ========== ============================================================= @@ -156,7 +159,7 @@ def write(self, file_name, *, rq=None, file_type=None, precision=None, (RQ), if desired. (default: None) file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) diff --git a/treecorr/ntcorrelation.py b/treecorr/ntcorrelation.py index b3441f5d..89ff6709 100644 --- a/treecorr/ntcorrelation.py +++ b/treecorr/ntcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: ngcorrelation +.. module:: ntcorrelation """ import numpy as np @@ -26,7 +26,7 @@ class NTCorrelation(BaseNZCorrelation): r"""This class handles the calculation and storage of a 2-point count-trefoil correlation function, where a trefoil is any field with spin-3 rotational properties. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr2` subclasses, objects of this class @@ -53,15 +53,18 @@ class NTCorrelation(BaseNZCorrelation): The typical usage pattern is as follows: >>> nt = treecorr.NTCorrelation(config) - >>> nt.process(cat1,cat2) # Compute the cross-correlation. - >>> nt.write(file_name) # Write out to a file. - >>> xi = nt.xi # Or access the correlation function directly. + >>> nt.process(cat1, cat2) # Compute the cross-correlation. + >>> nt.write(file_name) # Write out to a file. + >>> xi, xi_im = nt.xi, nt.xi_im # Or access the correlation function directly. + + See also: `KTCorrelation`, `TTCorrelation`, `NZCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -103,7 +106,7 @@ def calculateXi(self, *, rt=None): After calling this function, the attributes ``xi``, ``xi_im``, ``varxi``, and ``cov`` will correspond to the compensated values (if rt is provided). The raw, uncompensated values - are available as ``rawxi``, ``raw_xi_im``, and ``raw_varxi``. + are available as ``raw_xi``, ``raw_xi_im``, and ``raw_varxi``. Parameters: rt (NTCorrelation): The cross-correlation using random locations as the lenses @@ -141,7 +144,7 @@ def write(self, file_name, *, rt=None, file_type=None, precision=None, center points. tR_im The mean imaginary part of the trefoil field relative to the center points. - sigma The sqrt of the variance estimate of either of these + sigma The sqrt of the variance estimate of each of these weight The total weight contributing to each bin npairs The total number of pairs in each bin ========== ============================================================= @@ -156,7 +159,7 @@ def write(self, file_name, *, rt=None, file_type=None, precision=None, (RT), if desired. (default: None) file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) diff --git a/treecorr/nvcorrelation.py b/treecorr/nvcorrelation.py index 849c1c56..fbd52193 100644 --- a/treecorr/nvcorrelation.py +++ b/treecorr/nvcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: ngcorrelation +.. module:: nvcorrelation """ import numpy as np @@ -26,7 +26,7 @@ class NVCorrelation(BaseNZCorrelation): r"""This class handles the calculation and storage of a 2-point count-vector correlation function. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr2` subclasses, objects of this class @@ -53,15 +53,18 @@ class NVCorrelation(BaseNZCorrelation): The typical usage pattern is as follows: >>> nv = treecorr.NVCorrelation(config) - >>> nv.process(cat1,cat2) # Compute the cross-correlation. - >>> nv.write(file_name) # Write out to a file. - >>> xi = nv.xi # Or access the correlation function directly. + >>> nv.process(cat1, cat2) # Compute the cross-correlation. + >>> nv.write(file_name) # Write out to a file. + >>> xi, xi_im = nv.xi, nv.xi_im # Or access the correlation function directly. + + See also: `KVCorrelation`, `VVCorrelation`, `NZCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -102,7 +105,7 @@ def calculateXi(self, *, rv=None): After calling this function, the attributes ``xi``, ``xi_im``, ``varxi``, and ``cov`` will correspond to the compensated values (if rv is provided). The raw, uncompensated values - are available as ``rawxi``, ``raw_xi_im``, and ``raw_varxi``. + are available as ``raw_xi``, ``raw_xi_im``, and ``raw_varxi``. Parameters: rv (NVCorrelation): The cross-correlation using random locations as the lenses @@ -139,7 +142,7 @@ def write(self, file_name, *, rv=None, file_type=None, precision=None, vR The mean radial vector, :math:`\langle v_R \rangle(r)` vT The mean counter-clockwise tangential vector, :math:`\langle v_T \rangle(r)`. - sigma The sqrt of the variance estimate of either of these + sigma The sqrt of the variance estimate of each of these weight The total weight contributing to each bin npairs The total number of pairs in each bin ========== ============================================================= @@ -154,7 +157,7 @@ def write(self, file_name, *, rv=None, file_type=None, precision=None, (RV), if desired. (default: None) file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) diff --git a/treecorr/nzcorrelation.py b/treecorr/nzcorrelation.py index df0ff245..32acc470 100644 --- a/treecorr/nzcorrelation.py +++ b/treecorr/nzcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: ngcorrelation +.. module:: nzcorrelation """ import numpy as np @@ -101,7 +101,9 @@ def calculateXi(self, rz=None): self._rz = rz if rz.npatch1 not in (1,self.npatch1) or rz.npatch2 != self.npatch2: - raise RuntimeError(f"R{self._letter2} must be run with the same patches as D{self._letter2}") + raise RuntimeError( + f"R{self._letter2} must be run with the same patches as D{self._letter2}" + ) if len(self.results) > 0: # If there are any rz patch pairs that aren't in results (e.g. due to different @@ -172,10 +174,10 @@ class NZCorrelation(BaseNZCorrelation): r"""This class handles the calculation and storage of a 2-point count-complex correlation function, where the complex field is taken to have spin-0 rotational properties. If the spin-0 field is real, you should instead use `NKCorrelation` as it will be faster. - This class is intended for correlations of a scalar field with a complex values that + This class is intended for correlations of a scalar field with complex values that don't change with orientation. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr2` subclasses, objects of this class @@ -202,15 +204,18 @@ class NZCorrelation(BaseNZCorrelation): The typical usage pattern is as follows: >>> nz = treecorr.NZCorrelation(config) - >>> nz.process(cat1,cat2) # Compute the cross-correlation. - >>> nz.write(file_name) # Write out to a file. - >>> xi = nz.xi # Or access the correlation function directly. + >>> nz.process(cat1, cat2) # Compute the cross-correlation. + >>> nz.write(file_name) # Write out to a file. + >>> xi, xi_im = nz.xi, nz.xi_im # Or access the correlation function directly. + + See also: `NKCorrelation`, `NGCorrelation`, `KZCorrelation`, `ZZCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: @@ -251,7 +256,14 @@ def calculateXi(self, *, rz=None): After calling this function, the attributes ``xi``, ``xi_im``, ``varxi``, and ``cov`` will correspond to the compensated values (if rz is provided). The raw, uncompensated values - are available as ``rawxi``, ``raw_xi_im``, and ``raw_varxi``. + are available as ``raw_xi``, ``raw_xi_im``, and ``raw_varxi``. + + .. note:: + + The returned variance estimate (``varxi``) is computed according to this object's + ``var_method`` setting, specified when constructing the object (default: ``'shot'``). + Internally, this method calls `Corr2.estimate_cov`; see that method for details + about available variance and covariance estimation schemes. Parameters: rz (NZCorrelation): The cross-correlation using random locations as the lenses @@ -285,9 +297,10 @@ def write(self, file_name, *, rz=None, file_type=None, precision=None, into each bin meanlogr The mean value :math:`\langle \log(r) \rangle` of pairs that fell into each bin - z_real The mean real component, :math:`\langle real(z) \rangle(r)` - z_imag The mean imaginary component, :math:`\langle imag(z) \rangle(r)`. - sigma The sqrt of the variance estimate of either of these + z_real The mean real component, :math:`\langle \operatorname{Re}(z) \rangle(r)` + z_imag The mean imaginary component, + :math:`\langle \operatorname{Im}(z) \rangle(r)`. + sigma The sqrt of the variance estimate of each of these weight The total weight contributing to each bin npairs The total number of pairs in each bin ========== ============================================================= @@ -302,7 +315,7 @@ def write(self, file_name, *, rz=None, file_type=None, precision=None, (RZ), if desired. (default: None) file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) diff --git a/treecorr/qqcorrelation.py b/treecorr/qqcorrelation.py index 2b14e1c5..4f768c74 100644 --- a/treecorr/qqcorrelation.py +++ b/treecorr/qqcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: ggcorrelation +.. module:: qqcorrelation """ import numpy as np @@ -26,7 +26,7 @@ class QQCorrelation(BaseZZCorrelation): r"""This class handles the calculation and storage of a 2-point quatrefoil-quatrefoil correlation function, where a quatrefoil is any field with spin-4 rotational properties. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr2` subclasses, objects of this class @@ -54,16 +54,19 @@ class QQCorrelation(BaseZZCorrelation): The typical usage pattern is as follows: >>> qq = treecorr.QQCorrelation(config) - >>> qq.process(cat) # For auto-correlation. - >>> qq.process(cat1,cat2) # For cross-correlation. - >>> qq.write(file_name) # Write out to a file. - >>> xip = qq.xip # Or access the correlation function directly. + >>> qq.process(cat) # Compute the auto-correlation. + >>> # qq.process(cat1, cat2) # ... or the cross-correlation. + >>> qq.write(file_name) # Write out to a file. + >>> xip, xim = qq.xip, qq.xim # Or access the correlation functions directly. + + See also: `NQCorrelation`, `KQCorrelation`, `GGCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: diff --git a/treecorr/reader.py b/treecorr/reader.py index bd7b57fd..2eafa073 100644 --- a/treecorr/reader.py +++ b/treecorr/reader.py @@ -12,19 +12,16 @@ # and/or other materials provided with the distribution. """ -This is a set of helper classes that read data for treecorr.Catalog objects. -They have a bit of a mixed bag of functions, not intended to be complete, -just to cover the needs of that class. - -HDF and FITS files differ in that the former can have different length columns -in the same data group, unlike fits HDUs, which have columns of all the same length. -Where possible we check for that here, and a user would have to be fairly determined to -trigger this. - -The other difference is that fitsio reads a structured array, whereas h5py will -read a dictionary of arrays. This does not cause problems here, since both are -indexed by string, but may prevent usage elsewhere. If so we could convert them to -both provide dicts. +This module provides helper classes that read data for `treecorr.Catalog` objects. +The interfaces are intentionally minimal and cover only what Catalog needs. + +HDF and FITS files differ in that HDF groups can have columns with different lengths, +unlike FITS table HDUs, where all columns have the same length. +Where possible, we check for this. + +Another difference is that fitsio reads a structured array, whereas h5py reads +a dictionary of arrays. This does not cause problems here, since both are +indexed by string, but it may matter elsewhere. """ import numpy as np @@ -365,9 +362,8 @@ def __init__(self, file_name, *, delimiter=None, comment_marker='#', logger=None """ Parameters: file_name (str): The file name. - delimiter (str): What delimiter to use between values. (default: None, - which means any whitespace) - comment_marker (str): What token indicates a comment line. (default: '#') + delimiter (str): Ignored for Parquet input. + comment_marker (str): Ignored for Parquet input. """ try: import pandas @@ -402,7 +398,7 @@ def __contains__(self, ext): def check_valid_ext(self, ext): """Check if an extension is valid for reading, and raise ValueError if not. - None is the only valid extension for ASCII files. + None is the only valid extension for Parquet files. Parameters: ext (str): The extension to check. @@ -429,7 +425,7 @@ def read(self, cols, s=slice(None), *, ext=None): def row_count(self, col=None, *, ext=None): """Count the number of rows in the named extension and column - Unlike in FitsReader, col is required. + For Parquet input, this is just the number of rows in the DataFrame. Parameters: col (str): The column to use. (ignored) @@ -482,7 +478,7 @@ def __init__(self, file_name, *, logger=None): else: # There is a bug in earlier fitsio versions that prevents slicing. - # It you hit this assert, upgrade your fitsio version. + # If you hit this assert, upgrade your fitsio version. assert fitsio.__version__ > '1.0.6' self._file = None # Only works inside a with block. @@ -548,7 +544,7 @@ def read(self, cols, s=slice(None), *, ext=None): Parameters: cols (str/list): The name(s) of column(s) to read. s (slice/array): A slice object or selection of integers to read. (default: all) - ext (str/int)): The FITS extension to use. (default: 1) + ext (str/int): The FITS extension to use. (default: 1) Returns: The data as a recarray or simple numpy array as appropriate diff --git a/treecorr/ttcorrelation.py b/treecorr/ttcorrelation.py index f2d80836..8e7b2776 100644 --- a/treecorr/ttcorrelation.py +++ b/treecorr/ttcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: ggcorrelation +.. module:: ttcorrelation """ import numpy as np @@ -26,7 +26,7 @@ class TTCorrelation(BaseZZCorrelation): r"""This class handles the calculation and storage of a 2-point trefoil-trefoil correlation function, where a trefoil is any field with spin-3 rotational properties. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr2` subclasses, objects of this class @@ -54,16 +54,19 @@ class TTCorrelation(BaseZZCorrelation): The typical usage pattern is as follows: >>> tt = treecorr.TTCorrelation(config) - >>> tt.process(cat) # For auto-correlation. - >>> tt.process(cat1,cat2) # For cross-correlation. - >>> tt.write(file_name) # Write out to a file. - >>> xip = tt.xip # Or access the correlation function directly. + >>> tt.process(cat) # Compute the auto-correlation. + >>> # tt.process(cat1, cat2) # ... or the cross-correlation. + >>> tt.write(file_name) # Write out to a file. + >>> xip, xim = tt.xip, tt.xim # Or access the correlation functions directly. + + See also: `NTCorrelation`, `KTCorrelation`, `GGCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: diff --git a/treecorr/util.py b/treecorr/util.py index c97d0654..1ab364dc 100644 --- a/treecorr/util.py +++ b/treecorr/util.py @@ -34,7 +34,8 @@ def set_max_omp_threads(num_threads, logger=None): further TreeCorr functions :param num_threads: The target maximum number of threads to allow. None means no limit. - :param logger: If desired, a logger object for logging any warnings here. (default: None) + :param logger: If desired, a ``Logger`` object for logging any warnings here. + (default: None) """ global max_omp_threads max_omp_threads=num_threads @@ -43,7 +44,8 @@ def set_omp_threads(num_threads, logger=None): """Set the number of OpenMP threads to use in the C++ layer. :param num_threads: The target number of threads to use - :param logger: If desired, a logger object for logging any warnings here. (default: None) + :param logger: If desired, a ``Logger`` object for logging any warnings here. + (default: None) :returns: The number of threads OpenMP reports that it will use. Typically this matches the input, but OpenMP reserves the right not to comply with @@ -90,7 +92,7 @@ def get_omp_threads(): :returns: The number of threads OpenMP reports that it will use. """ - # Some OMP implemenations have a bug where if omp_get_max_threads() is called + # Some OMP implementations have a bug where if omp_get_max_threads() is called # (which is what this function does), it sets something called thread affinity. # The upshot of that is that multiprocessing (i.e. not even just omp threading) is # confined to a single hardware thread. Yeah, it's idiotic, but that seems to be @@ -146,7 +148,7 @@ def make_writer(file_name, precision=4, file_type=None, logger=None): return writer def make_reader(file_name, file_type=None, logger=None): - """Factory function to make a writer instance of the correct type. + """Factory function to make a reader instance of the correct type. """ # Figure out which file type to use. file_type = parse_file_type(file_type, file_name, output=False, logger=logger) @@ -162,9 +164,9 @@ def make_reader(file_name, file_type=None, logger=None): return reader class LRU_Cache(object): - """ Simplified Least Recently Used Cache. - Mostly stolen from http://code.activestate.com/recipes/577970-simplified-lru-cache/, - but added a method for dynamic resizing. The least recently used cached item is + """Simplified Least Recently Used Cache. + Based on http://code.activestate.com/recipes/577970-simplified-lru-cache/, + with an added method for dynamic resizing. The least recently used cached item is overwritten on a cache miss. Note: This has additional functionality beyond what functools.lru_cache provides. @@ -172,7 +174,7 @@ class LRU_Cache(object): 2. The key is only on the args, not kwargs, so a logger can be provided as a kwarg without triggering a cache miss. - :param user_function: A python function to cache. + :param user_function: A Python function to cache. :param maxsize: Maximum number of inputs to cache. [Default: 1024] Usage @@ -182,7 +184,7 @@ class LRU_Cache(object): >>> >>> v1 = slow_function(*k1) # Calling function is slow >>> v1 = slow_function(*k1) # Calling again with same args is still slow - >>> cache = galsim.utilities.LRU_Cache(slow_function) + >>> cache = treecorr.util.LRU_Cache(slow_function) >>> v1 = cache(*k1) # Returns slow_function(*k1), slowly the first time >>> v1 = cache(*k1) # Returns slow_function(*k1) again, but fast this time. @@ -242,7 +244,7 @@ def last_value(self): return self.root[0][3] def resize(self, maxsize): - """ Resize the cache. Increasing the size of the cache is non-destructive, i.e., + """ Resize the cache. Increasing the size of the cache is non-destructive, i.e. previously cached inputs remain in the cache. Decreasing the size of the cache will necessarily remove items from the cache if the cache is already filled. Items are removed in least recently used order. @@ -329,7 +331,7 @@ def parse_metric(metric, coords, coords2=None, coords3=None): if metric in ['Rperp', 'OldRperp', 'FisherRperp'] and coords != '3d': raise ValueError("%s metric is only valid for catalogs with 3d positions."%metric) if metric == 'Rlens' and auto: - raise ValueError("Rlens metric is only valid for cross correlations.") + raise ValueError("Rlens metric is only valid for cross-correlations.") if metric == 'Rlens' and coords != '3d': raise ValueError("Rlens metric is only valid for catalogs with 3d positions.") if metric == 'Arc' and coords not in ['spherical', '3d']: @@ -400,11 +402,12 @@ def parse_xyzsep(args, kwargs, _coords): :param dec: The declination of the location for which to count nearby points. :param sep: The separation distance as an angle - For all angle parameters (ra, dec, sep), this quantity may be a coord.Angle instance, or + For all angle parameters (ra, dec, sep), this quantity may be a + :class:`coord.Angle` instance, or units maybe be provided as ra_units, dec_units or sep_units respectively. - Finally, in cases where ra, dec are allowed, a coord.CelestialCoord instance may be - provided as the first argument. + Finally, in cases where ra, dec are allowed, a :class:`coord.CelestialCoord` instance + may be provided as the first argument. :returns: The effective (x, y, z, sep) as a tuple. """ @@ -599,8 +602,8 @@ def slow_function_to_be_used_as_a_property(self): x = ... # Some slow calculation. return x - Base on an answer from http://stackoverflow.com/a/6849299 - This implementation taken from GalSim utilities.py + Based on an answer from http://stackoverflow.com/a/6849299 + This implementation was taken from GalSim utilities.py """ def __init__(self, fget): self.fget = fget diff --git a/treecorr/vvcorrelation.py b/treecorr/vvcorrelation.py index 21cec804..3678f114 100644 --- a/treecorr/vvcorrelation.py +++ b/treecorr/vvcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: ggcorrelation +.. module:: vvcorrelation """ import numpy as np @@ -26,7 +26,7 @@ class VVCorrelation(BaseZZCorrelation): r"""This class handles the calculation and storage of a 2-point vector-vector correlation function. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr2` subclasses, objects of this class @@ -54,16 +54,19 @@ class VVCorrelation(BaseZZCorrelation): The typical usage pattern is as follows: >>> vv = treecorr.VVCorrelation(config) - >>> vv.process(cat) # For auto-correlation. - >>> vv.process(cat1,cat2) # For cross-correlation. - >>> vv.write(file_name) # Write out to a file. - >>> xip = vv.xip # Or access the correlation function directly. + >>> vv.process(cat) # Compute the auto-correlation. + >>> # vv.process(cat1, cat2) # ... or the cross-correlation. + >>> vv.write(file_name) # Write out to a file. + >>> xip, xim = vv.xip, vv.xim # Or access the correlation functions directly. + + See also: `NVCorrelation`, `KVCorrelation`, `GGCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: diff --git a/treecorr/writer.py b/treecorr/writer.py index 7855a3cc..d3340d7c 100644 --- a/treecorr/writer.py +++ b/treecorr/writer.py @@ -28,7 +28,8 @@ def __init__(self, file_name, *, precision=4, logger=None): Parameters: file_name: The file name. precision: The number of digits of precision to output. - logger: If desired, a logger object for logging. (default: None) + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None) """ self.file_name = file_name self.logger = logger @@ -51,11 +52,11 @@ def write(self, col_names, columns, *, params=None, ext=None): """Write some columns to an output ASCII file with the given column names. Parameters: - col_names: A list of columns names for the given columns. These will be written + col_names: A list of column names for the given columns. These will be written in a header comment line at the top of the output file. columns: A list of numpy arrays with the data to write. params: A dict of extra parameters to write at the top of the output file. - ext: Optional ext name for these data. (default: None) + ext: Optional extension name for these data. (default: None) """ ncol = len(col_names) data = np.empty( (len(columns[0]), ncol) ) @@ -80,7 +81,7 @@ def write_array(self, data, *, ext=None): Parameters: data: The array to write. - ext: Optional ext name for these data. (default: None) + ext: Optional extension name for these data. (default: None) """ if ext is not None: s = '## %s\n'%ext @@ -103,7 +104,8 @@ def __init__(self, file_name, *, logger=None): """ Parameters: file_name: The file name. - logger: If desired, a logger object for logging. (default: None) + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None) """ try: import fitsio # noqa: F401 @@ -126,15 +128,15 @@ def file(self): return self._file def write(self, col_names, columns, *, params=None, ext=None): - """Write some columns to an output ASCII file with the given column names. + """Write some columns to an output FITS table with the given column names. - If name is not None, then it is used as the name of the extension for these data. + If ``ext`` is not None, then it is used as the name of the extension for these data. Parameters: - col_names: A list of columns names for the given columns. + col_names: A list of column names for the given columns. columns: A list of numpy arrays with the data to write. - params: A dict of extra parameters to write at the top of the output file. - ext: Optional ext name for these data. (default: None) + params: A dict of extra parameters to write in the extension header. + ext: Optional extension name for these data. (default: None) """ data = np.empty(len(columns[0]), dtype=[ (c,'f8') for c in col_names ]) for (c, col) in zip(col_names, columns): @@ -146,7 +148,7 @@ def write_array(self, data, *, ext=None): Parameters: data: The array to write. - ext: Optional ext name for these data. (default: None) + ext: Optional extension name for these data. (default: None) """ self.file.write(data, extname=ext) @@ -162,13 +164,14 @@ def __exit__(self, exc_type, exc_value, exc_traceback): class HdfWriter(object): """Writer interface for HDF5 files. - Uses h5py to read columns, etc. + Uses h5py to write columns, etc. """ def __init__(self, file_name, *, logger=None): """ Parameters: file_name: The file name. - logger: If desired, a logger object for logging. (default: None) + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None) """ try: import h5py # noqa: F401 @@ -191,14 +194,14 @@ def file(self): return self._file def write(self, col_names, columns, *, params=None, ext=None): - """Write some columns to an output ASCII file with the given column names. + """Write some columns to an output HDF5 group with the given column names. - If name is not None, then it is used as the name of the extension for these data. + If ``ext`` is not None, then it is used as the name of the group for these data. Parameters: - col_names: A list of columns names for the given columns. + col_names: A list of column names for the given columns. columns: A list of numpy arrays with the data to write. - params: A dict of extra parameters to write at the top of the output file. + params: A dict of extra parameters to write in the group attributes. ext: Optional group name for these data. (default: None) """ if ext is not None: @@ -215,7 +218,7 @@ def write_array(self, data, *, ext=None): Parameters: data: The array to write. - ext: Optional ext name for these data. (default: None) + ext: Optional group name for these data. (default: None) """ if ext is not None: hdf = self.file.create_group(ext) diff --git a/treecorr/zzcorrelation.py b/treecorr/zzcorrelation.py index 33b60ccc..0ea5c5ce 100644 --- a/treecorr/zzcorrelation.py +++ b/treecorr/zzcorrelation.py @@ -12,7 +12,7 @@ # and/or other materials provided with the distribution. """ -.. module:: ggcorrelation +.. module:: zzcorrelation """ import numpy as np @@ -115,8 +115,8 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result that fell into each bin xip The real part of the :math:`\xi_+` correlation function xim The real part of the :math:`\xi_-` correlation function - xip_im The imag part of the :math:`\xi_+` correlation function - xim_im The imag part of the :math:`\xi_-` correlation function + xip_im The imaginary part of the :math:`\xi_+` correlation function + xim_im The imaginary part of the :math:`\xi_-` correlation function sigma_xip The sqrt of the variance estimate of :math:`\xi_+` sigma_xim The sqrt of the variance estimate of :math:`\xi_-` weight The total weight contributing to each bin @@ -131,7 +131,7 @@ def write(self, file_name, *, file_type=None, precision=None, write_patch_result file_name (str): The name of the file to write to. file_type (str): The type of file to write ('ASCII' or 'FITS'). (default: determine the type automatically from the extension of file_name.) - precision (int): For ASCII output catalogs, the desired precision. (default: 4; + precision (int): For ASCII output files, the desired precision. (default: 4; this value can also be given in the constructor in the config dict.) write_patch_results (bool): Whether to write the patch-based results as well. (default: False) @@ -172,7 +172,7 @@ class ZZCorrelation(BaseZZCorrelation): r"""This class handles the calculation and storage of a 2-point correlation function of two complex spin-0 fields. If either spin-0 field is real, you should instead use `KZCorrelation` as it will be faster, and if both are real, you should use `KKCorrelation`. - This class is intended for correlations of scalar fields with a complex values that + This class is intended for correlations of scalar fields with complex values that don't change with orientation. To be consistent with the other spin correlation functions, we compute two quantities: @@ -185,7 +185,7 @@ class ZZCorrelation(BaseZZCorrelation): There is no projection along the line connecting the two points as there is for the other complex fields, since the field values don't change with orientation. - See the doc string of `Corr3` for a description of how the triangles are binned along + See the docstring of `Corr2` for a description of how the pairs are binned along with the attributes related to the different binning options. In addition to the attributes common to all `Corr2` subclasses, objects of this class @@ -213,16 +213,19 @@ class ZZCorrelation(BaseZZCorrelation): The typical usage pattern is as follows: >>> zz = treecorr.ZZCorrelation(config) - >>> zz.process(cat) # For auto-correlation. - >>> zz.process(cat1,cat2) # For cross-correlation. - >>> zz.write(file_name) # Write out to a file. - >>> xip = zz.xip # Or access the correlation function directly. + >>> zz.process(cat) # Compute the auto-correlation. + >>> # zz.process(cat1, cat2) # ... or the cross-correlation. + >>> zz.write(file_name) # Write out to a file. + >>> xip, xim = zz.xip, zz.xim # Or access the correlation functions directly. + + See also: `NZCorrelation`, `KZCorrelation`, `VVCorrelation`. Parameters: config (dict): A configuration dict that can be used to pass in kwargs if desired. - This dict is allowed to have addition entries besides those listed + This dict is allowed to have additional entries besides those listed in `Corr2`, which are ignored here. (default: None) - logger: If desired, a logger object for logging. (default: None, in which case + logger (:class:`logging.Logger`): + If desired, a ``Logger`` object for logging. (default: None, in which case one will be built according to the config dict's verbose level.) Keyword Arguments: