threeML.utils.statistics package

Submodules

threeML.utils.statistics.likelihood_functions module

threeML.utils.statistics.likelihood_functions.half_chi2(y, yerr, expectation)

threeML.utils.statistics.likelihood_functions.poisson_log_likelihood_ideal_bkg(observed_counts, expected_bkg_counts, expected_model_counts)

Poisson log-likelihood for the case where the background has no uncertainties:

L = sum_{i=0}^{N}~o_i~log{(m_i + b_i)} - (m_i + b_i) - log{o_i!}

Parameters

• observed_counts –

• expected_bkg_counts –

• expected_model_counts –

Returns

(log_like vector, background vector)

threeML.utils.statistics.likelihood_functions.poisson_observed_gaussian_background(observed_counts, background_counts, background_error, expected_model_counts)

threeML.utils.statistics.likelihood_functions.poisson_observed_poisson_background(observed_counts, background_counts, exposure_ratio, expected_model_counts)

threeML.utils.statistics.likelihood_functions.poisson_observed_poisson_background_xs(observed_counts, background_counts, exposure_ratio, expected_model_counts)

Profile log-likelihood for the case when the observed counts are Poisson distributed, and the background counts are Poisson distributed as well (typical for X-ray analysis with aperture photometry). This has been derived by Keith Arnaud (see the Xspec manual, Wstat statistic)

threeML.utils.statistics.likelihood_functions.regularized_log(vector)

A function which is log(vector) where vector > 0, and zero otherwise.

Parameters

vector –

Returns

threeML.utils.statistics.likelihood_functions.xlogy(x, y)

A function which is 0 if x is 0, and x * log(y) otherwise. This is to fix the fact that for a machine 0 * log(inf) is nan, instead of 0.

Parameters

• x –

• y –

Returns

threeML.utils.statistics.likelihood_functions.xlogy_one(x, y)

A function which is 0 if x is 0, and x * log(y) otherwise. This is to fix the fact that for a machine 0 * log(inf) is nan, instead of 0.

Parameters

• x –

• y –

Returns

threeML.utils.statistics.stats_tools module

class threeML.utils.statistics.stats_tools.PoissonResiduals(Non, Noff, alpha=1.0)

Bases: object

This class implements a way to compute residuals for a Poisson distribution mapping them to residuals of a standard normal distribution. The probability of obtaining the observed counts given the expected one is computed, and then transformed “in unit of sigma”, i.e., the sigma value corresponding to that probability is computed.

The algorithm implemented here uses different branches so that it is fairly accurate between -36 and +36 sigma.

NOTE: if the expected number of counts is not very high, then the Poisson distribution is skewed and so the probability of obtaining a downward fluctuation at a given sigma level is not the same as obtaining the same fluctuation in the upward direction. Therefore, the distribution of residuals is not expected to be symmetric in that case. The sigma level at which this effect is visible depends strongly on the expected number of counts. Under normal circumstances residuals are expected to be a few sigma at most, in which case the effect becomes important for expected number of counts <~ 15-20.

significance_one_side()

class threeML.utils.statistics.stats_tools.Significance(Non, Noff, alpha=1)

Bases: object

Implements equations in Li&Ma 1983

known_background()

Compute the significance under the hypothesis that there is no uncertainty in the background. In other words, compute the probability of obtaining the observed counts given the expected counts from the background, then transform it in sigma.

NOTE: this is reliable for expected counts >~10-15 if the significance is not very high. The higher the expected counts, the more reliable the significance estimation. As rule of thumb, you need at least 25 counts to have reliable estimates up to 5 sigma.

NOTE 2: if you use to compute residuals in units of sigma, you should not expected them to be symmetrically distributed around 0 unless the expected number of counts is high enough for all bins (>~15). This is due to the fact that the Poisson distribution is very skewed at low counts.

Returns

significance vector

li_and_ma(assign_sign=True)

Compute the significance using the formula from Li & Ma 1983, which is appropriate when both background and observed signal are counts coming from a Poisson distribution.

Parameters

assign_sign – whether to assign a sign to the significance, according to the sign of the net counts

Non - alpha * Noff, so that excesses will have positive significances and defects negative significances :return:

li_and_ma_equivalent_for_gaussian_background(sigma_b)

Compute the significance using the formula from Vianello 2018 (https://iopscience.iop.org/article/10.3847/1538-4365/aab780/meta), which is appropriate when the observation is Poisson distributed but the background has been modeled and thus has Gaussian distributed errors.

Parameters

sigma_b – The gaussian 1 sigma errors on the background

Returns

threeML.utils.statistics.stats_tools.aic(log_like, n_parameters, n_data_points)

The Aikake information criterion. A model comparison tool based of infomormation theory. It assumes that N is large i.e., that the model is approaching the CLT.

threeML.utils.statistics.stats_tools.bic(log_like, n_parameters, n_data_points)

The Bayesian information criterion.

threeML.utils.statistics.stats_tools.dic(bayes_analysis)

elpd_DIC = log p(y|mean(parameters)) - p_DIC

the first term is the deviance at the mean of the posterior and p_DIC is the effective number of free parameters:

p_DIC = 2(log p(y|mean(parameters)) - 1/N sum(log p(y|parameters_s), 1,N) )

DIC = -2*elpd_DIC

the effective number of free parameters can be negative if the mean is the mean is far from the mode

Parameters

bayes_analysis – a bayesian analysis object

Return dic, effective number of free parameters

threeML.utils.statistics.stats_tools.sqrt_sum_of_squares(arg)

Parameters

arg – and array of number to be squared and summed

Returns

the sqrt of the sum of the squares

threeML.utils.statistics.stats_tools.waic(bayesian_trace)

Module contents