sqfa.model

Class implementing the Supervised Quadratic Feature Analysis (SQFA) model.

Classes

SQFA(n_dim[, feature_noise, n_filters, ...])

Supervised Quadratic Feature Analysis (SQFA) model.

SecondMomentsSQFA(n_dim[, feature_noise, ...])

Second-moments Supervised Quadratic Feature Analysis (SQFA) model.
class sqfa.model.SQFA(n_dim, feature_noise=0, n_filters=2, filters=None, distance_fun=None, constraint='sphere')

Bases: SecondMomentsSQFA

Supervised Quadratic Feature Analysis (SQFA) model. This version of the model uses both the means and the covariances of the data, and uses distances (or approximations) in the manifold of normal distributions.

Methods

add_module(name, module)

Add a child module to the current module.

apply(fn)

Apply fn recursively to every submodule (as returned by .children()) as well as self.

bfloat16()

Casts all floating point parameters and buffers to bfloat16 datatype.

buffers([recurse])

Return an iterator over module buffers.

children()

Return an iterator over immediate children modules.

compile(*args, **kwargs)

Compile this Module's forward using torch.compile().

cpu()

Move all model parameters and buffers to the CPU.

cuda([device])

Move all model parameters and buffers to the GPU.

double()

Casts all floating point parameters and buffers to double datatype.

eval()

Set the module in evaluation mode.

extra_repr()

Return the extra representation of the module.

fit([X, y, data_statistics, max_epochs, lr, ...])

Fit the SQFA model to data using the LBFGS optimizer.

fit_pca([X, data_statistics])

Fit the SQFA filters to the data using PCA.

float()

Casts all floating point parameters and buffers to float datatype.

forward(*input)

Define the computation performed at every call.

get_buffer(target)

Return the buffer given by target if it exists, otherwise throw an error.

get_class_distances(data_statistics[, ...])

Compute the pairwise lower bounds to the Fisher-Rao distances.

get_extra_state()

Return any extra state to include in the module's state_dict.

get_parameter(target)

Return the parameter given by target if it exists, otherwise throw an error.

get_submodule(target)

Return the submodule given by target if it exists, otherwise throw an error.

half()

Casts all floating point parameters and buffers to half datatype.

ipu([device])

Move all model parameters and buffers to the IPU.

load_state_dict(state_dict[, strict, assign])

Copy parameters and buffers from state_dict into this module and its descendants.

modules()

Return an iterator over all modules in the network.

mtia([device])

Move all model parameters and buffers to the MTIA.

named_buffers([prefix, recurse, ...])

Return an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself.

named_children()

Return an iterator over immediate children modules, yielding both the name of the module as well as the module itself.

named_modules([memo, prefix, remove_duplicate])

Return an iterator over all modules in the network, yielding both the name of the module as well as the module itself.

named_parameters([prefix, recurse, ...])

Return an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself.

parameters([recurse])

Return an iterator over module parameters.

register_backward_hook(hook)

Register a backward hook on the module.

register_buffer(name, tensor[, persistent])

Add a buffer to the module.

register_forward_hook(hook, *[, prepend, ...])

Register a forward hook on the module.

register_forward_pre_hook(hook, *[, ...])

Register a forward pre-hook on the module.

register_full_backward_hook(hook[, prepend])

Register a backward hook on the module.

register_full_backward_pre_hook(hook[, prepend])

Register a backward pre-hook on the module.

register_load_state_dict_post_hook(hook)

Register a post-hook to be run after module's load_state_dict() is called.

register_load_state_dict_pre_hook(hook)

Register a pre-hook to be run before module's load_state_dict() is called.

register_module(name, module)

Alias for add_module().

register_parameter(name, param)

Add a parameter to the module.

register_state_dict_post_hook(hook)

Register a post-hook for the state_dict() method.

register_state_dict_pre_hook(hook)

Register a pre-hook for the state_dict() method.

requires_grad_([requires_grad])

Change if autograd should record operations on parameters in this module.

set_extra_state(state)

Set extra state contained in the loaded state_dict.

set_submodule(target, module)

Set the submodule given by target if it exists, otherwise throw an error.

share_memory()

See torch.Tensor.share_memory_().

state_dict(*args[, destination, prefix, ...])

Return a dictionary containing references to the whole state of the module.

to(*args, **kwargs)

Move and/or cast the parameters and buffers.

to_empty(*, device[, recurse])

Move the parameters and buffers to the specified device without copying storage.

train([mode])

Set the module in training mode.

transform(data_points)

Transform data to feature space.

transform_scatters(data_scatters)

Transform data scatter matrices to feature space scatter matrices.

type(dst_type)

Casts all parameters and buffers to dst_type.

xpu([device])

Move all model parameters and buffers to the XPU.

zero_grad([set_to_none])

Reset gradients of all model parameters.

__call__
fit(X=None, y=None, data_statistics=None, max_epochs=300, lr=0.1, estimator='empirical', pairwise=False, show_progress=True, return_loss=False, **kwargs)

Fit the SQFA model to data using the LBFGS optimizer.

Parameters:

  • X (torch.Tensor) – Input data of shape (n_samples, n_dim). If data_statistics is None, then X and y must be provided.

  • y (torch.Tensor) – Labels of shape (n_samples,). If data_statistics is None, then X and y must be provided. Labels must be integers starting from 0.

  • data_statistics (dict) – Dictionary containing the fields ‘means’ and ‘covariances’

  • max_epochs (int, optional) – Number of max training epochs. By default 50.

  • lr (float) – Learning rate for the optimizer. Default is 0.1.

  • estimator – Covariance estimator to use. Options are “empirical”, and “oas”. Default is “empirical”.

  • pairwise (bool) – If True, then filters are optimized pairwise (the first 2 filters are optimized together, then held fixed and the next 2 filters are optimized together, etc.). If False, all filters are optimized together. Default is False.

  • show_progress (bool) – If True, show a progress bar during training. Default is True.

  • return_loss (bool) – If True, return the loss after training. Default is False.

  • **kwargs – Additional keyword arguments passed to the NAdam optimizer.

get_class_distances(data_statistics, regularized=False)

Compute the pairwise lower bounds to the Fisher-Rao distances.

Parameters:

  • data_statistics (torch.Tensor or dict) –

    • If a torch.Tensor, should have shape (n_classes, n_dim, n_dim) and contain the scatter matrices (second moments) of the data for each class.

    • If a dict, it should contain fields ‘means’ and ‘covariances’.

  • regularized (bool) – If True, regularize the distances by adding a small value to the diagonal of the transformed scatter matrices. Default is False.

Returns:

Pairwise distances between the transformed feature scatter matrices.

Return type:

torch.Tensor shape (n_classes, n_classes)

class sqfa.model.SecondMomentsSQFA(n_dim, feature_noise=0, n_filters=2, filters=None, distance_fun=None, constraint='sphere')

Bases: Module

Second-moments Supervised Quadratic Feature Analysis (SQFA) model. This version of the model uses only the second moment matrices of the data, and distances in the SPD manifold.

Methods

add_module(name, module)

Add a child module to the current module.

apply(fn)

Apply fn recursively to every submodule (as returned by .children()) as well as self.

bfloat16()

Casts all floating point parameters and buffers to bfloat16 datatype.

buffers([recurse])

Return an iterator over module buffers.

children()

Return an iterator over immediate children modules.

compile(*args, **kwargs)

Compile this Module's forward using torch.compile().

cpu()

Move all model parameters and buffers to the CPU.

cuda([device])

Move all model parameters and buffers to the GPU.

double()

Casts all floating point parameters and buffers to double datatype.

eval()

Set the module in evaluation mode.

extra_repr()

Return the extra representation of the module.

fit([X, y, data_statistics, max_epochs, lr, ...])

Fit the second-moments SQFA model to data using the LBFGS optimizer.

fit_pca([X, data_statistics])

Fit the SQFA filters to the data using PCA.

float()

Casts all floating point parameters and buffers to float datatype.

forward(*input)

Define the computation performed at every call.

get_buffer(target)

Return the buffer given by target if it exists, otherwise throw an error.

get_class_distances(data_statistics[, ...])

Compute the pairwise distances between the feature scatter matrices of the different classes.

get_extra_state()

Return any extra state to include in the module's state_dict.

get_parameter(target)

Return the parameter given by target if it exists, otherwise throw an error.

get_submodule(target)

Return the submodule given by target if it exists, otherwise throw an error.

half()

Casts all floating point parameters and buffers to half datatype.

ipu([device])

Move all model parameters and buffers to the IPU.

load_state_dict(state_dict[, strict, assign])

Copy parameters and buffers from state_dict into this module and its descendants.

modules()

Return an iterator over all modules in the network.

mtia([device])

Move all model parameters and buffers to the MTIA.

named_buffers([prefix, recurse, ...])

Return an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself.

named_children()

Return an iterator over immediate children modules, yielding both the name of the module as well as the module itself.

named_modules([memo, prefix, remove_duplicate])

Return an iterator over all modules in the network, yielding both the name of the module as well as the module itself.

named_parameters([prefix, recurse, ...])

Return an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself.

parameters([recurse])

Return an iterator over module parameters.

register_backward_hook(hook)

Register a backward hook on the module.

register_buffer(name, tensor[, persistent])

Add a buffer to the module.

register_forward_hook(hook, *[, prepend, ...])

Register a forward hook on the module.

register_forward_pre_hook(hook, *[, ...])

Register a forward pre-hook on the module.

register_full_backward_hook(hook[, prepend])

Register a backward hook on the module.

register_full_backward_pre_hook(hook[, prepend])

Register a backward pre-hook on the module.

register_load_state_dict_post_hook(hook)

Register a post-hook to be run after module's load_state_dict() is called.

register_load_state_dict_pre_hook(hook)

Register a pre-hook to be run before module's load_state_dict() is called.

register_module(name, module)

Alias for add_module().

register_parameter(name, param)

Add a parameter to the module.

register_state_dict_post_hook(hook)

Register a post-hook for the state_dict() method.

register_state_dict_pre_hook(hook)

Register a pre-hook for the state_dict() method.

requires_grad_([requires_grad])

Change if autograd should record operations on parameters in this module.

set_extra_state(state)

Set extra state contained in the loaded state_dict.

set_submodule(target, module)

Set the submodule given by target if it exists, otherwise throw an error.

share_memory()

See torch.Tensor.share_memory_().

state_dict(*args[, destination, prefix, ...])

Return a dictionary containing references to the whole state of the module.

to(*args, **kwargs)

Move and/or cast the parameters and buffers.

to_empty(*, device[, recurse])

Move the parameters and buffers to the specified device without copying storage.

train([mode])

Set the module in training mode.

transform(data_points)

Transform data to feature space.

transform_scatters(data_scatters)

Transform data scatter matrices to feature space scatter matrices.

type(dst_type)

Casts all parameters and buffers to dst_type.

xpu([device])

Move all model parameters and buffers to the XPU.

zero_grad([set_to_none])

Reset gradients of all model parameters.

__call__
fit(X=None, y=None, data_statistics=None, max_epochs=300, lr=0.1, estimator='empirical', pairwise=False, show_progress=True, return_loss=False, **kwargs)

Fit the second-moments SQFA model to data using the LBFGS optimizer.

Parameters:

  • X (torch.Tensor) – Input data of shape (n_samples, n_dim). If data_statistics is None, then X and y must be provided.

  • y (torch.Tensor) – Labels of shape (n_samples,). If data_statistics is None, then X and y must be provided. Labels must be integers starting from 0.

  • data_statistics (torch.Tensor or dict) –

    • If a torch.Tensor, should have shape (n_classes, n_dim, n_dim) and contain the scatter matrices (second moments) of the data for each class.

    • If a dict, it should contain fields ‘means’ and ‘covariances’

  • max_epochs (int, optional) – Number of max training epochs. By default 50.

  • lr (float) – Learning rate for the optimizer. Default is 0.1.

  • estimator – Covariance estimator to use. Options are “empirical”, and “oas”. Default is “empirical”.

  • pairwise (bool) – If True, then filters are optimized pairwise (the first 2 filters are optimized together, then held fixed and the next 2 filters are optimized together, etc.). If False, all filters are optimized together. Default is False.

  • show_progress (bool) – If True, show a progress bar during training. Default is True.

  • return_loss (bool) – If True, return the loss after training. Default is False.

  • **kwargs – Additional keyword arguments passed to the NAdam optimizer.

fit_pca(X=None, data_statistics=None)

Fit the SQFA filters to the data using PCA. This can be used to initialize the filters before training.

Parameters:

  • X (torch.Tensor) – Input data of shape (n_samples, n_dim).

  • data_statistics (torch.Tensor) – Tensor of shape (n_classes, n_dim, n_dim) with the second moments of the data for each class. If None, then X and y must be provided. Default is None.

get_class_distances(data_statistics, regularized=False)

Compute the pairwise distances between the feature scatter matrices of the different classes.

Parameters:

  • data_statistics (torch.Tensor or dict) –

    • If a torch.Tensor, should have shape (n_classes, n_dim, n_dim) and contain the scatter matrices (second moments) of the data for each class.

    • If a dict, it should contain fields ‘means’ and ‘covariances’.

  • regularized (bool) – If True, regularize the distances by adding a small value to the diagonal of the transformed scatter matrices. Default is False.

Returns:

Pairwise distances between the transformed feature scatter matrices.

Return type:

torch.Tensor shape (n_classes, n_classes)

transform(data_points)

Transform data to feature space.

Parameters:

data_points (torch.Tensor) – Input data of shape (n_samples, n_dim).

Returns:

Data transformed to feature space.

Return type:

torch.Tensor shape (n_samples, n_filters)

transform_scatters(data_scatters)

Transform data scatter matrices to feature space scatter matrices.

Parameters:

data_scatters (torch.Tensor) – Tensor of shape (n_classes, n_dim, n_dim), with second moment or covariance matrices.

Returns:

Covariances of the transformed features.

Return type:

torch.Tensor shape (n_classes, n_filters, n_filters)