Search results
Results From The WOW.Com Content Network
In probability theory and statistics, a categorical distribution (also called a generalized Bernoulli distribution, multinoulli distribution [1]) is a discrete probability distribution that describes the possible results of a random variable that can take on one of K possible categories, with the probability of each category separately specified.
In probability theory, statistics, and machine learning, the continuous Bernoulli distribution [1] [2] [3] is a family of continuous probability distributions parameterized by a single shape parameter (,), defined on the unit interval [,], by:
One can compute this directly, without using a probability distribution (distribution-free classifier); one can estimate the probability of a label given an observation, (| =) (discriminative model), and base classification on that; or one can estimate the joint distribution (,) (generative model), from that compute the conditional probability ...
It represents a discrete probability distribution concentrated at 0 — a degenerate distribution — it is a Distribution (mathematics) in the generalized function sense; but the notation treats it as if it were a continuous distribution. The Kent distribution on the two-dimensional sphere.
In machine learning, the observations are often known as instances, the explanatory variables are termed features (grouped into a feature vector), and the possible categories to be predicted are classes. Other fields may use different terminology: e.g. in community ecology, the term "classification" normally refers to cluster analysis.
For example, a sequence of Bernoulli trials is interpreted as the Bernoulli process. One may generalize this to include continuous time Lévy processes, and many Lévy processes can be seen as limits of i.i.d. variables—for instance, the Wiener process is the limit of the Bernoulli process.
Mechanisms for discretizing continuous data include Fayyad & Irani's MDL method, [2] which uses mutual information to recursively define the best bins, CAIM, CACC, Ameva, and many others [3] Many machine learning algorithms are known to produce better models by discretizing continuous attributes.
In machine learning, the Gumbel distribution is sometimes employed to generate samples from the categorical distribution. This technique is called "Gumbel-max trick" and is a special example of "reparameterization tricks". [15]