mlpack::nca::SoftmaxErrorFunction< MetricType > Class Template Reference

The "softmax" stochastic neighbor assignment probability function. More...

List of all members.

Public Member Functions

 SoftmaxErrorFunction (const arma::mat &dataset, const arma::Col< size_t > &labels, MetricType metric=MetricType())
 Initialize with the given kernel; useful when the kernel has some state to store, which is set elsewhere.
double Evaluate (const arma::mat &covariance, const size_t i)
 Evaluate the softmax objective function for the given covariance matrix on only one point of the dataset.
double Evaluate (const arma::mat &covariance)
 Evaluate the softmax function for the given covariance matrix.
const arma::mat GetInitialPoint () const
 Get the initial point.
void Gradient (const arma::mat &covariance, const size_t i, arma::mat &gradient)
 Evaluate the gradient of the softmax function for the given covariance matrix on only one point of the dataset.
void Gradient (const arma::mat &covariance, arma::mat &gradient)
 Evaluate the gradient of the softmax function for the given covariance matrix.
size_t NumFunctions () const
 Get the number of functions the objective function can be decomposed into.

Private Member Functions

void Precalculate (const arma::mat &coordinates)
 Precalculate the denominators and numerators that will make up the p_ij, but only if the coordinates matrix is different than the last coordinates the Precalculate() method was run with.

Private Attributes

const arma::mat & dataset
 The dataset.
arma::vec denominators
 Holds denominators for calculation of p_ij, for the non-separable Evaluate() and Gradient().
const arma::Col< size_t > & labels
 Labels for each point in the dataset.
arma::mat lastCoordinates
 Last coordinates. Used for the non-separable Evaluate() and Gradient().
MetricType metric
 The instantiated metric.
arma::vec p
 Holds calculated p_i, for the non-separable Evaluate() and Gradient().
bool precalculated
 False if nothing has ever been precalculated (only at construction time).
arma::mat stretchedDataset
 Stretched dataset. Kept internal to avoid memory reallocations.

Detailed Description

template<typename MetricType = metric::SquaredEuclideanDistance>
class mlpack::nca::SoftmaxErrorFunction< MetricType >

The "softmax" stochastic neighbor assignment probability function.

The actual function is

p_ij = (exp(-|| A x_i - A x_j || ^ 2)) / (sum_{k != i} (exp(-|| A x_i - A x_k || ^ 2)))

where x_n represents a point and A is the current scaling matrix.

This class is more flexible than the original paper, allowing an arbitrary metric function to be used in place of || A x_i - A x_j ||^2, meaning that the squared Euclidean distance is not the only allowed metric for NCA. However, that is probably the best way to use this class.

In addition to the standard Evaluate() and Gradient() functions which MLPACK optimizers use, overloads of Evaluate() and Gradient() are given which only operate on one point in the dataset. This is useful for optimizers like stochastic gradient descent (see mlpack::optimization::SGD).

Definition at line 52 of file nca_softmax_error_function.hpp.


Constructor & Destructor Documentation

template<typename MetricType = metric::SquaredEuclideanDistance>
mlpack::nca::SoftmaxErrorFunction< MetricType >::SoftmaxErrorFunction ( const arma::mat &  dataset,
const arma::Col< size_t > &  labels,
MetricType  metric = MetricType() 
)

Initialize with the given kernel; useful when the kernel has some state to store, which is set elsewhere.

If no kernel is given, an empty kernel is used; this way, you can call the constructor with no arguments. A reference to the dataset we will be optimizing over is also required.

Parameters:
dataset Matrix containing the dataset.
labels Vector of class labels for each point in the dataset.
kernel Instantiated kernel (optional).

Member Function Documentation

template<typename MetricType = metric::SquaredEuclideanDistance>
double mlpack::nca::SoftmaxErrorFunction< MetricType >::Evaluate ( const arma::mat &  covariance,
const size_t  i 
)

Evaluate the softmax objective function for the given covariance matrix on only one point of the dataset.

This is the separable implementation, where the objective function is decomposed into the sum of many objective functions, and here, only one of those constituent objective functions is returned.

Parameters:
covariance Covariance matrix of Mahalanobis distance.
i Index of point to use for objective function.
template<typename MetricType = metric::SquaredEuclideanDistance>
double mlpack::nca::SoftmaxErrorFunction< MetricType >::Evaluate ( const arma::mat &  covariance  ) 

Evaluate the softmax function for the given covariance matrix.

This is the non-separable implementation, where the objective function is not decomposed into the sum of several objective functions.

Parameters:
covariance Covariance matrix of Mahalanobis distance.
template<typename MetricType = metric::SquaredEuclideanDistance>
const arma::mat mlpack::nca::SoftmaxErrorFunction< MetricType >::GetInitialPoint (  )  const

Get the initial point.

template<typename MetricType = metric::SquaredEuclideanDistance>
void mlpack::nca::SoftmaxErrorFunction< MetricType >::Gradient ( const arma::mat &  covariance,
const size_t  i,
arma::mat &  gradient 
)

Evaluate the gradient of the softmax function for the given covariance matrix on only one point of the dataset.

This is the separable implementation, where the objective function is decomposed into the sum of many objective functions, and here, only one of those constituent objective functions is returned.

Parameters:
covariance Covariance matrix of Mahalanobis distance.
i Index of point to use for objective function.
gradient Matrix to store the calculated gradient in.
template<typename MetricType = metric::SquaredEuclideanDistance>
void mlpack::nca::SoftmaxErrorFunction< MetricType >::Gradient ( const arma::mat &  covariance,
arma::mat &  gradient 
)

Evaluate the gradient of the softmax function for the given covariance matrix.

This is the non-separable implementation, where the objective function is not decomposed into the sum of several objective functions.

Parameters:
covariance Covariance matrix of Mahalanobis distance.
gradient Matrix to store the calculated gradient in.
template<typename MetricType = metric::SquaredEuclideanDistance>
size_t mlpack::nca::SoftmaxErrorFunction< MetricType >::NumFunctions (  )  const [inline]

Get the number of functions the objective function can be decomposed into.

This is just the number of points in the dataset.

Definition at line 124 of file nca_softmax_error_function.hpp.

template<typename MetricType = metric::SquaredEuclideanDistance>
void mlpack::nca::SoftmaxErrorFunction< MetricType >::Precalculate ( const arma::mat &  coordinates  )  [private]

Precalculate the denominators and numerators that will make up the p_ij, but only if the coordinates matrix is different than the last coordinates the Precalculate() method was run with.

This method is only called by the non-separable Evaluate() and Gradient().

This will update last_coordinates_ and stretched_dataset_, and also calculate the p_i and denominators_ which are used in the calculation of p_i or p_ij. The calculation will be O((n * (n + 1)) / 2), which is not great.

Parameters:
coordinates Coordinates matrix to use for precalculation.

Member Data Documentation

template<typename MetricType = metric::SquaredEuclideanDistance>
const arma::mat& mlpack::nca::SoftmaxErrorFunction< MetricType >::dataset [private]

The dataset.

Definition at line 128 of file nca_softmax_error_function.hpp.

template<typename MetricType = metric::SquaredEuclideanDistance>
arma::vec mlpack::nca::SoftmaxErrorFunction< MetricType >::denominators [private]

Holds denominators for calculation of p_ij, for the non-separable Evaluate() and Gradient().

Definition at line 143 of file nca_softmax_error_function.hpp.

template<typename MetricType = metric::SquaredEuclideanDistance>
const arma::Col<size_t>& mlpack::nca::SoftmaxErrorFunction< MetricType >::labels [private]

Labels for each point in the dataset.

Definition at line 130 of file nca_softmax_error_function.hpp.

template<typename MetricType = metric::SquaredEuclideanDistance>
arma::mat mlpack::nca::SoftmaxErrorFunction< MetricType >::lastCoordinates [private]

Last coordinates. Used for the non-separable Evaluate() and Gradient().

Definition at line 136 of file nca_softmax_error_function.hpp.

template<typename MetricType = metric::SquaredEuclideanDistance>
MetricType mlpack::nca::SoftmaxErrorFunction< MetricType >::metric [private]

The instantiated metric.

Definition at line 133 of file nca_softmax_error_function.hpp.

template<typename MetricType = metric::SquaredEuclideanDistance>
arma::vec mlpack::nca::SoftmaxErrorFunction< MetricType >::p [private]

Holds calculated p_i, for the non-separable Evaluate() and Gradient().

Definition at line 140 of file nca_softmax_error_function.hpp.

template<typename MetricType = metric::SquaredEuclideanDistance>
bool mlpack::nca::SoftmaxErrorFunction< MetricType >::precalculated [private]

False if nothing has ever been precalculated (only at construction time).

Definition at line 146 of file nca_softmax_error_function.hpp.

template<typename MetricType = metric::SquaredEuclideanDistance>
arma::mat mlpack::nca::SoftmaxErrorFunction< MetricType >::stretchedDataset [private]

Stretched dataset. Kept internal to avoid memory reallocations.

Definition at line 138 of file nca_softmax_error_function.hpp.


The documentation for this class was generated from the following file:

Generated on 13 Aug 2014 for MLPACK by  doxygen 1.6.1