Skip to main content

Tensorflow Probability

Motivation​

model = tf.keras.Sequential([...])

model.compile(optimizer='adam',
              loss=lambda y, dist: -dist.log_prob(y)')

rv_y_given_x = model(x)
rv_y_given_x.prob(y)
rv_y_given_x.mean()
rv_y_given_x.variance()

Installation​

pip install tensorflow-probability

or

pip install tfp-nightly

Example: Regression​

Learn known unknowns​

model = tf.keras.Sequential([
    tf.keras.layers.Dense(hidden_units, ...),
    tf.keras.layers.Dense(1+1), # mean and variance
    tfp.layers.DistributionLambda(lambda t:
        tfd.Normal(loca=t[..., 0], scale=tf.softplus(t[..., 1])))
})

Learn unknown unknowns​

model = tf.keras.Sequential([
    tf.keras.layers.DenseVariational(hidden_units, ...),
    tf.keras.layers.DenseVariational(1),
    tfp.layers.DistributionLambda(lambda t:
        tfd.Normal(loca=t[..., 0], scale=1))
})

Learn known and unknown unknowns​

model = tf.keras.Sequential([
    tf.keras.layers.DenseVariational(hidden_units, ...),
    tf.keras.layers.DenseVariational(1+1),
    tfp.layers.DistributionLambda(lambda t:
        tfd.Normal(loca=t[..., 0], scale=tf.softplus(t[..., 1])))
})

Uncertainty in the loss function​

model = tf.keras.Sequential([
    tf.keras.layers.Dense(hidden_units, ...),
    tf.keras.layers.Dense(1), # mean and variance
    tfp.layers.VariationalGaussianProcess(
        num_inducing_points, kernel_provider)
})