cardinal_pythonlib.probability

Original code copyright (C) 2009-2022 Rudolf Cardinal (rudolf@pobox.com).

This file is part of cardinal_pythonlib.

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Miscellaneous probability functions.

cardinal_pythonlib.probability.bayes_posterior(prior: float, likelihood: float, marginal_likelihood: float) → float[source]

Returns P(H | D), the posterior probability of hypothesis H, given data D.

Parameters:

prior¶ – P(H), the prior belief in H
likelihood¶ – P(D | H), the probability of observing D given H
marginal_likelihood¶ – P(D), the probability of observing D; also called evidence or model evidence.

Returns:

P(H | D), the probability of H given D.

Return type:

float

Bayes’ rule:

$P(A | B) P(B) = P(B | A) P(A) = P(A \land B) P(H | D) = \frac{ P(D | H) \cdot P(H) }{ P(D) } \text{posterior} = \frac { \text{likelihood} \cdot \text{prior} } { \text{marginal likelihood} }$

cardinal_pythonlib.probability.ln(x: float) → float[source]

Version of math.log() that treats log(0) as -inf, rather than crashing with ValueError: math domain error.

Parameters:: x¶ – parameter
Returns:: ln(x), the natural logarithm of x
Return type:: float

See https://stackoverflow.com/questions/42980201/logarithm-of-zero-in-python.

For speed, use from math import log as math_log, etc.:

Note also that although floats can’t be “too close to zero” to cause an error, other things (like Decimal objects) can. See https://stackoverflow.com/questions/19095774/python-math-domain-errors-in-math-log-function

Exception catching is likely faster:

cardinal_pythonlib.probability.log10(x: float) → float[source]

Version of math.log10() that treats log(0) as -inf, rather than crashing with ValueError: math domain error.

Parameters:: x¶ – parameter
Returns:: log10(x), the logarithm to base 10 of x
Return type:: float

See https://stackoverflow.com/questions/42980201/logarithm-of-zero-in-python.

cardinal_pythonlib.probability.log_bayes_posterior(log_prior: float, log_likelihood: float, log_marginal_likelihood: float) → float[source]

Exactly equivalent to bayes_posterior(), but using log probability.

Parameters:

log_prior¶ – $\log P(H)$
log_likelihood¶ – $\log P(D | H)$
log_marginal_likelihood¶ – $\log P(D)$

Returns:

$\log P(H | D)$

Return type:

float

cardinal_pythonlib.probability.log_likelihood_ratio_from_p(p_d_given_h: float, p_d_given_not_h: float) → float[source]

Returns

$\ln \frac{ P(D | H) }{ P(D | \neg H) } = \ln P(D | H) - \ln P(D | \neg H)$

Parameters:

p_d_given_h¶ – $P(D | H)$
p_d_given_not_h¶ – $P(D | \neg H)$

Returns:

log likelihood ratio, as above

Return type:

float

cardinal_pythonlib.probability.log_odds_from_1_in_n(n: float) → float[source]

If the chance of something occurring are 1 in n, then its probability is $1/n$ , and its odds are $\frac{1}{n - 1}$ . This function returns the log of those odds.

Parameters:: n¶ – n, as above
Returns:: $\ln \frac{1}{n - 1} = \ln 1 - \ln (n - 1) = -\ln(n - 1)$
Return type:: float

cardinal_pythonlib.probability.log_odds_from_probability(p: float) → float[source]

Returns log odds from a probability.

Parameters:: p¶ – probability
Returns:: ln(odds)
Return type:: float

See

cardinal_pythonlib.probability.log_posterior_odds(log_prior_odds: float, log_likelihood_ratio: float) → float[source]: Exactly as for posterior_odds(), but with log odds.

cardinal_pythonlib.probability.log_posterior_odds_from_bool_d_pdh_pdnh(log_prior_odds: float, d: bool, p_d_given_h: float, p_d_given_not_h: float) → float[source]

Calculates posterior odds.

Parameters:

log_prior_odds¶ – log prior odds of H, $ln(\frac{ P(H) }{ P(\neg H) })$
d¶ – whether D is true or not
p_d_given_h¶ – $P(D | H)$
p_d_given_not_h¶ – $P(D | \neg H)$

Returns:

log posterior odds of H, $ln(\frac{ P(H | D) }{ P(\neg H | D) })$

Return type:

float

cardinal_pythonlib.probability.log_posterior_odds_from_pdh_pdnh(log_prior_odds: float, p_d_given_h: float, p_d_given_not_h: float) → float[source]

Calculates posterior odds.

Parameters:

log_prior_odds¶ – log prior odds of H, $ln(\frac{ P(H) }{ P(\neg H) })$
p_d_given_h¶ – $P(D | H)$
p_d_given_not_h¶ – $P(D | \neg H)$

Returns:

log posterior odds of H, $ln(\frac{ P(H | D) }{ P(\neg H | D) })$

Return type:

float

cardinal_pythonlib.probability.log_probability_from_log_odds(log_odds: float) → float[source]

No obvious quick form for this:

$o &= \frac{p}{1 - p} \\ \log o &= \log p - \log (1 - p) \\$

Parameters:: log_odds¶ – ln(o); natural log of o
Returns:: ln(p)
Return type:: float

cardinal_pythonlib.probability.odds_from_probability(p: float) → float[source]

Returns odds, given a probability.

$o = \frac{p}{1 - p}$

Parameters:: p¶ – probability
Returns:: odds
Return type:: float

cardinal_pythonlib.probability.posterior_odds(prior_odds: float, likelihood_ratio: float) → float[source]

Returns the posterior odds for a hypothesis, given the prior odds and the likelihood ratio.

$P(A | B) &= P(B | A) \frac{ P(A) }{ P(B) } \\ P(\neg A | B) &= P(B | \neg A) \frac{ P(\neg A) }{ P(B) } \\ \frac{ P(A | B) }{ P(\neg A | B) } &= \frac{ P(B | A) }{ P(B | \neg A) } \frac{ P(A) }{ P(\neg A) } \\ \text{posterior odds} &= \text{likelihood ratio} \cdot \text{prior odds}$