Demystifying the difference between generative and discriminative machine learning models

Machine learning is one of the exciting fields of study that gives machines the ability to learn and become more accurate at predicting outcomes. It overlaps with and receives ideas from artificial intelligence and many other related technologies. Today, machine learning is evolved from pattern recognition and the concept that computers can learn without being programmed to performing specific tasks. ML algorithms are able to recognize spoken words, mine data and build applications that learn from data, and improve their accuracy over time.

In this article, we will explore two machine learning models, generative and discriminating, their importance, and differences.

Generative Machine Learning Model

The generative model is considered as a class of statistical models that can generate new data instances. This model is typically used to estimate probabilities, modeling data points and distinguishing between classes based on these probabilities. As they often rely on Bayes theorem, generative models can tackle a more complex task than analogous discriminative models. Generative modeling is used in unsupervised machine learning as a means to describe phenomena in data, enabling computers to understand the real world.

Examples of generative models are:

• Naive Bayes or Bayesian networks
• Gaussian Mixture Model (GMM)
• Hidden Markov model
• Linear Discriminant Analysis (LDA)

Discriminative Machine Learning Model

Discriminative model refers to a class of models used in statistical classification, especially in supervised machine learning. Also known as conditional models, generative modeling learns the boundary between classes or labels in a dataset. It tends to model the joint probability of data points and can create new instances using probability estimates and maximum likelihood. Unlike the generative models, discriminative models have the advantage of being more robust to outliers.

Discriminative models in machine learning are:

• Logistic regression
• Support vector machine
• Decision tree
• Random forests

Generative vs. Discriminative Machine Learning Model

Generative models try to model how data is placed throughout the space, while discriminative models attempt to draw boundaries in the data space. Generative modeling contrasts with discriminative modeling, which recognizes existing data and can be used to classify data. Generative modeling produces something and discriminative modeling identifies tags and sorts data.

Generative models are useful for unsupervised machine learning tasks and are impacted by the presence of outliers over discriminative models. Conversely, discriminative models are useful for supervised machine learning tasks and are more robust to outliers. Discriminative models are computationally cheap compared to generative models. In maths, discriminative machine learning trains a model which is done by learning model parameters that maximize the conditional probability P(Y|X). On the other side, generative machine learning trains a model to learn parameters, maximizing the joint probability of P(X, Y).

Furthermore, humans can adopt these two different approaches to machine learning models while learning an artificial language. These models have not previously been explored in human learning. However, it is related to known effects of causal direction, classification vs. inference learning, and observational vs. feedback learning. In a nutshell, generative and discriminative models leverage diverse approaches to solve object categorization problems