Why GFlowNets are Useful? Talk About How They are Trained

In this article, we provide a guide to building your first GFlowNets in TensorFlow 2.0 implementation

GFlowNets is an ML technique for generating compositional objects at a frequency proportional to their associated reward. It was introduced at NeurIPS by Emmanuel Bengio. They live somewhere at the intersection of reinforcement learning, deep generative models, and energy-based probabilistic modeling. GFlowNet is very useful in a combinatorial domain, drug molecule synthesis.

Generative Flow Networks are a DL technique for building objects at a frequency proportional to the expected reward of those objects in an environment.  They allow neural nets to model distributions over data structures like graphs, the sample from them as well as estimate all kinds of probabilistic quantities which otherwise look intractable. In this article, we are a guide to building your first Generative Flow Networks in TensorFlow 2.0 implementation.

Generative Flow Networks:

Generative Flow Networks have been introduced as a method to sample a diverse set of candidates in an active learning context, with a training objective that makes them approximately sample in proportion to a given reward function. It aimed at bridging the gap between SOTA AI and human intelligence by introducing system 2 inductive biases in neural nets. They amortize in a single but trained generative pass, the work is typically done by computationally expensive MCMC methods.

GFlowNets was to sample the distribution of trajectories whose probability is proportional to a positive return. It can be used to estimate joint probability distributions and corresponding marginal distributions and are particularly interesting to represent distributions over composite objects like sets and graphs. It is a probabilistic model that builds objects. And it selects one building block at random according to the predicted distribution. GFlowNet can turn a given energy function into samples but it does it in an amortized way, converting the cost of a lot of very expensive MCMC trajectories into the cost training of a generative model.

Flow Networks:

A flow network is a directed graph with sources and sinks and edges carrying some amount of flow between them through intermediate nodes of pipes of water. The motivation of such a flow network is iterative black-box optimization, where the agent has to calculate the reward for a large batch of candidates for each round. Some real-life problems like the flow of liquids through pipes, the current through wires, and the delivery of goods can be modeled using flow networks.

How GFlowNets are trained:

The Flow part of GFlowNets is water flows from the origin, through all the actions, and flows out at terminal states where rewards are known. This goal is to distribute water through the pipes such that it flows out in volumes proportional to the rewards.

Generative Flow Networks will stick with Trajectory Balance. It loss function is one way to achieve that goal. For each trajectory, the forward flow of probabilities. A key point is that the reward and terminal state are fixed values in your training data, but the trajectories are learned.

Build a GFlowNet in TensorFlow 2.0:

TensorFlow 2.0 is a library that provides a comprehensive ecosystem of tools for developers, researchers, and organizations who want to build scalable Machine Learning and Deep Learning applications. While the Bengio lab implemented their GFlowNets using PyTorch, and TensorFlow 2.0. A properly trained GFlowNet generates objects with probability proportional to the reward.