Expectation Maximization infrastructure¶
The machine learning algorithms distributed with ProSper are based on latent variable probabilistic data models and are trained with variational Expectation Maximization (EM) learning algorithm. The source code is therefore organized under an EM module.
Expectation Maximization Algorithm¶
The Expectation Maximization (EM) algorithm is used to optimize probabilistic models with latent random variables. It is an iterative algorithm that optimizes with respect to the posterior distribution in the E-step and and proceeds with an optimization of the model parameters in the M step. A simple implementation is given in the EM class. The more technical components however are contained in the Model classes.
-
class
prosper.em.EM(model=None, anneal=None, data=None, lparams=None, mpi_comm=None)¶ This class drives the EM algorithm.
-
run(verbose=False)¶ Run a complete cooling-cycle
When verbose is True a progress message is printed for every step via
dlog.progress(...)()
-
step()¶ Execute a single EM-Step
-
-
class
prosper.em.Model(comm=<Mock id='140062113491152'>)¶ Model Base Class.
Includes knowledge about parameters, data generation, model specific functions, E and M step.
Specific models will be subclasses of this abstract base class.
-
generate_data(model_params, N)¶ Generate datapoints according to the model.
Given the model parameters model_params return a dataset of N datapoints.
-
noisify_params(model_params, anneal)¶ Noisify model params.
Noisify the given model parameters according to self.noise_policy and the annealing object provided. The noise_policy of some model parameter PARAM will only be applied if the annealing object provides a noise strength via PARAM_noise.
-
standard_init(data)¶ Initialize a set of model parameters in some sane way.
Return value is model_parameter dictionary
-
step(anneal, model_params, my_data)¶
-
Component Analysis Models¶
Component Analysis Models refers to models with multiple latent variables may contribute to the same datapoints to
-
class
prosper.em.camodels.CAModel(D, H, Hprime, gamma, to_learn=['W', 'pi', 'sigma'], comm=<Mock id='140062088152512'>)¶ Abstract base class for Sparse Coding models with binary latent variables and expectation tuncation (ET) based training scheme.
This
-
check_params(model_params)¶ Perform a sanity check on the model parameters. Throw an exception if there are major violations; correct the parameter in case of minor violations
-
compute_lpj(anneal, model_params, my_data)¶ Determine candidates and compute log-pseudo-joint.
-
generate_data(model_params, my_N)¶ Generate data according to the model. Internally uses generate_data_from_hidden.
- Parameters
This method does _not_ obey gamma: The generated data may have more than gamma active causes for a given datapoint.
-
inference(anneal, model_params, test_data, topK=10, logprob=False, adaptive=True, Hprime_max=None, gamma_max=None)¶ Perform inference with the learned model on test data and return the top K configurations with their posterior probabilities. :param anneal: Annealing schedule, e.g., em.anneal :type anneal: prosper.em.annealling.Annealing :param model_params: Learned model parameters, e.g., em.lparams :type model_params: dict :param test_data: The test data stored in field ‘y’. Candidates stored in ‘candidates’ (optional). :type test_data: dict :param topK: The number of returned configurations :type topK: int :param logprob: Return probability or log probability :type logprob: boolean :param adaptive: Adjust Hprime, gamma to be greater than the number of active units in the MAP state :type adaptive: boolean :param Hprime_max: Upper limit for Hprime adjustment :type Hprime_max: int :param gamma_max: Upper limit for gamma adjustment :type gamma_max: int
-
select_partial_data(anneal, my_data)¶ Select a partial data-set from my_data and return it.
The fraction of datapoints selected is determined by anneal[‘partial’]. If anneal[‘partial’] is equal to either 1 or 0 the whole dataset will be returned.
-
standard_init(data)¶ Standard onitial estimation for model parameters.
This implementation
W and sigma.
each W raw is set to the average over the data plus WGN of mean zero and var sigma/4. sigma is set to the variance of the data around the computed mean. pi is set to 1./H . Returns a dict with the estimated parameter set with entries “W”, “pi” and “sigma”.
-
step(anneal, model_params, my_data)¶ Perform an EM-step
-
-
prosper.em.camodels.generate_state_matrix(Hprime, gamma)¶ Full combinatorics of Hprime-dim binary vectors with at most gamma ones.
- Parameters
Hprime (int) – Vector length
gamma – Maximum number of ones
gamma – int
Binary Sparse Coding¶
-
class
prosper.em.camodels.bsc_et.BSC_ET(D, H, Hprime, gamma, to_learn=['W', 'pi', 'sigma'], comm=<Mock id='140062026281256'>)¶ Binary Sparse Coding
Implements learning and inference of a Binary Sparse coding model under a variational approximation
-
comm¶ - Type
MPI communicator
-
no_states¶ number of different states of latent variables except singleton states and zero state
- Type
(.., Hprime) ndarray
-
single_state_matrix¶ matrix that holds all possible singleton states
- Type
((K-1)*H, H) ndarray
-
state_abs¶ number of non-zero elements in the rows of the state_matrix
- Type
(no_states, ) ndarray
-
state_matrix¶ latent variable states taken into account during the em algorithm
- Type
(no_states, Hprime) ndarray
-
states¶ the differnt values that a latent variable can take must include 0 and one more integer
- Type
(K,) ndarray
-
to_learn¶ list of strings included in model_params.keys() that specify which parameters are going to be optimized
- Type
References
[1] M. Henniges, G. Puertas, J. Bornschein, J. Eggert, and J. Lücke (2010). Binary Sparse Coding. Proc. LVA/ICA 2010, LNCS 6365, 450-457.
[2] J. Lücke and J. Eggert (2010). Expectation Truncation and the Benefits of Preselection in Training Generative Models. Journal of Machine Learning Research 11:2855-2900.
-
E_step(anneal, model_params, my_data)¶ BSC E_step
my_data variables used:
my_data[‘y’] Datapoints my_data[‘can’] Candidate H’s according to selection func.
Annealing variables used:
anneal[‘T’] Temperature for det. annealing anneal[‘N_cut_factor’] 0.: no truncation; 1. trunc. according to model
-
M_step(anneal, model_params, my_suff_stat, my_data)¶ BSC M_step
my_data variables used:
my_data[‘y’] Datapoints my_data[‘candidates’] Candidate H’s according to selection func.
Annealing variables used:
anneal[‘T’] Temperature for det. annealing anneal[‘N_cut_factor’] 0.: no truncation; 1. trunc. according to model
Generate data according to the MCA model while the latents are given in my_hdata[‘s’].
This method does _not_ obey gamma: The generated data may have more than gamma active causes for a given datapoint.
-
select_Hprimes(model_params, data)¶ Return a new data-dictionary which has been annotated with a data[‘candidates’] dataset. A set of self.Hprime candidates will be selected.
-
Ternary Sparse Coding¶
-
class
prosper.em.camodels.tsc_et.TSC_ET(D, H, Hprime, gamma, to_learn=['W', 'pi', 'sigma'], comm=<Mock id='140062026563424'>)¶ Ternary Sparse Coding
Implements learning and inference of a Ternary Sparse coding model under a variational approximation
-
comm¶ - Type
MPI communicator
-
no_states¶ number of different states of latent variables except singleton states and zero state
- Type
(.., Hprime) ndarray
-
single_state_matrix¶ matrix that holds all possible singleton states
- Type
((K-1)*H, H) ndarray
-
state_abs¶ number of non-zero elements in the rows of the state_matrix
- Type
(no_states, ) ndarray
-
state_matrix¶ latent variable states taken into account during the em algorithm
- Type
(no_states, Hprime) ndarray
-
states¶ the differnt values that a latent variable can take must include 0 and one more integer
- Type
(K,) ndarray
-
to_learn¶ list of strings included in model_params.keys() that specify which parameters are going to be optimized
- Type
References
[1] G. Exarchakis, M. Henniges, J. Eggert, and J. Lücke (2012). Ternary Sparse Coding. International Conference on Latent Variable Analysis and Signal Separation (LVA/ICA), 204-212.
[2] J. Lücke and J. Eggert (2010). Expectation Truncation and the Benefits of Preselection in Training Generative Models. Journal of Machine Learning Research 11:2855-2900.
-
E_step(anneal, model_params, my_data)¶ E step for Teranary Sparse Coding Identifies approximate posterior information for Ternary Sparse Coding
- Parameters
anneal (Annealing object) –
- contains information related to annealing
- anneal[‘T’]: scalar
Temperature for det. annealing
- anneal[‘N_cut_factor’]: scalar
0.: no truncation; 1. trunc. according to model
model_params (dict) –
- dictionary of parameters
- model_params[‘W’]: ndarray
dictionary
- model_params[‘sigma’]: float
standard deviation of gaussian noise
- model_params[‘pi’]: float
prior parameter
my_data (dict) –
- datapoints dictionary
- my_data[‘y’]: ndarray
Datapoints
- my_data[‘can’]: ndarray
Candidate H’s according to selection func.
- Returns
- dict[‘logpj’]
Approximate joint of datapoints and latent variable states
- Return type
-
M_step(anneal, model_params, my_suff_stat, my_data)¶ Ternary Sparse Coding M-Step
This function is responsible for finding the optimal model parameters given an approximation of the posterior distribution.
- Parameters
anneal (Annealing object) –
- Annealing type obje ct containing training schedule information
anneal[‘T’] : Temperature for det. annealing anneal[‘N_cut_factor’]: 0. no truncation; 1. trunc. according to model
model_params (dict) –
- dictionary containing model parameters
- model_params[‘W’]: (H,D) ndarray
linear dictionary
- model_params[‘pi’]: (K,) ndarray
prior parameters
- model_params[‘sigma’]: float
standard deviation of noise model
my_suff_stat (dict) –
- dictionary containing inforamtion about the joint distribution
- my_suff_stat[‘logpj’]: (my_N,no_states) ndarray
logarithm of joint of data and latent variable states
my_data (dict) –
- data dictionary
- my_data[‘y’]: (my_N,D) ndarray
datapoints
- my_data[‘candidates’]: (my_n,Hprime)
Candidate H’s according to selection func.
- Returns
- dictionary containing updated model parameters
- dict[‘W’]: (H,D) ndarray
linear dictionary
- dict[‘pi’]: (K,) ndarray
prior parameters
- dict[‘sigma’]: float
standard deviation of noise model
- Return type
-
generate_data(model_params, my_N)¶ - Parameters
- Returns
dict –
- returns generated data
- dict[‘y’]: (my_N, D) ndarray
generated data
- dict[‘s’]: (my_N, H) ndarray
latent variable states that generated the data
Deleted Parameters
——————
noise_on (bool, optional) – flag to control deterministic/stochastic generation. If True gaussian noise with standard deviation model_params[‘sigma’] is added to the data
gs ((my_N, H), optional) – ground truth latent variables. This option is used for generating artificial data with particular latent variables. Defaults to randomly sampled latent variables from the prior
gp ((my_N, H), optional) – ground truth posterior. This option is used for generating data that have a particular true posterior distribution. Defaults to randomly sampled latent variables from the prior
-
inference(anneal, model_params, test_data, topK=10, logprob=False, abs_marginal=True, adaptive=True, Hprime_max=None, gamma_max=None)¶ Perform inference with the learned model on test data and return the top K configurations with their posterior probabilities.
- Parameters
anneal (Annealing object) – annealing information
model_params (dict) – dictionary with model parameters
test_data (dict) – data dictionary. The data in this case are ndarray under the key ‘y’.
topK (int, optional) – the number of most probable latent variable states to be returned
logprob (bool, optional) – the probabilities of the most probable latent variable states
abs_marginal (bool, optional) – Description
adaptive (bool, optional) – if set to True it will run inference again for datapoints with gamma active latent variables in the top state using setting gamma=gamma+1 and Hprime=Hprime+1
Hprime_max (None, optional) – if adaptive is True it will stop Hprime from increasing above this integer. None defaults to H.
gamma_max (None, optional) – if adaptive is True it will stop gamma from increasing above this integer. None defaults to H.
- Returns
- a dictionary with posterior information
- dict[‘s’]: (batchsize, topK, H) ndarray
the topK most probable vectors
- dict[‘m’]: (batchsize, H) ndarray
latent variable marginal distribution
- dict[‘am’]: (batchsize, H) ndarray
absolote latent variable marginal distribution
- dict[‘p’]: (batchsize, topK) ndarray
probabilities of topK latent variable states
- dict[‘gamma’]: int
sparseness approximation parameter
- dict[‘Hprime’]: int
truncation approximation parameter
- Return type
-
select_Hprimes(model_params, data)¶ Return a new data-dictionary which has been annotated with a data[‘candidates’] dataset. A set of self.Hprime candidates will be selected.
- Parameters
- Returns
- dataset dictionary
- data[‘y’]: (my_n,D) ndarray
datapoints
- data[‘candidates’]: (my_n,) ndarray
indices of the best explained datapoints
- Return type
-
Discrete Sparse Coding¶
-
class
prosper.em.camodels.dsc_et.DSC_ET(D, H, Hprime, gamma, states=array([-1., 0., 1.]), to_learn=['W', 'pi', 'sigma'], comm=<Mock id='140062024662376'>)¶ Discrete Sparse Coding
Implements learning and inference of a Discrete Sparse coding model under a variational approximation
-
comm¶ - Type
MPI communicator
-
no_states¶ number of different states of latent variables except singleton states and zero state
- Type
(.., Hprime) ndarray
-
single_state_matrix¶ matrix that holds all possible singleton states
- Type
((K-1)*H, H) ndarray
-
state_abs¶ number of non-zero elements in the rows of the state_matrix
- Type
(no_states, ) ndarray
-
state_matrix¶ latent variable states taken into account during the em algorithm
- Type
(no_states, Hprime) ndarray
-
states¶ the differnt values that a latent variable can take must include 0 and one more integer
- Type
(K,) ndarray
-
to_learn¶ list of strings included in model_params.keys() that specify which parameters are going to be optimized
- Type
References
[1] G. Exarchakis, and J. Lücke (2017). Discrete Sparse Coding. Neural Computation, 29(11), 2979-3013.
[2] J. Lücke and J. Eggert (2010). Expectation Truncation and the Benefits of Preselection in Training Generative Models. Journal of Machine Learning Research 11:2855-2900.
-
E_step(anneal, model_params, my_data)¶ Discrete Sparse Coding E-step
This function is responsible for finding an approximation of the posterior distribution given the model parameters.
- Parameters
anneal (Annealing object) –
- Annealing type obje ct containing training schedule information
anneal[‘T’] : Temperature for det. annealing anneal[‘N_cut_factor’]: 0. no truncation; 1. trunc. according to model
model_params (dict) –
- dictionary containing model parameters
- model_params[‘W’]: (H,D) ndarray
linear dictionary
- model_params[‘pi’]: (K,) ndarray
prior parameters
- model_params[‘sigma’]: float
standard deviation of noise model
my_data (dict) –
- data dictionary
- my_data[‘y’]: (my_N,D) ndarray
datapoints
- my_data[‘candidates’]: (my_n,Hprime)
Candidate H’s according to selection func.
- Returns
- returns information about the approximation posterior
- dict[‘logpj’]: (my_n,no_states) ndarray
an approximation of the logarithm of the joint distribution
- Return type
-
M_step(anneal, model_params, my_suff_stat, my_data)¶ Discrete Sparse Coding M-Step
This function is responsible for finding the optimal model parameters given an approximation of the posterior distribution.
- Parameters
anneal (Annealing object) –
- Annealing type obje ct containing training schedule information
anneal[‘T’] : Temperature for det. annealing anneal[‘N_cut_factor’]: 0. no truncation; 1. trunc. according to model
model_params (dict) –
- dictionary containing model parameters
- model_params[‘W’]: (H,D) ndarray
linear dictionary
- model_params[‘pi’]: (K,) ndarray
prior parameters
- model_params[‘sigma’]: float
standard deviation of noise model
my_suff_stat (dict) –
- dictionary containing inforamtion about the joint distribution
- my_suff_stat[‘logpj’]: (my_N,no_states) ndarray
logarithm of joint of data and latent variable states
my_data (dict) –
- data dictionary
- my_data[‘y’]: (my_N,D) ndarray
datapoints
- my_data[‘candidates’]: (my_n,Hprime)
Candidate H’s according to selection func.
- Returns
- dictionary containing updated model parameters
- dict[‘W’]: (H,D) ndarray
linear dictionary
- dict[‘pi’]: (K,) ndarray
prior parameters
- dict[‘sigma’]: float
standard deviation of noise model
- Return type
-
check_params(model_params)¶ Sanity check.
Sanity-check the given model parameters. Raises an exception if something is severely wrong.
- Parameters
model_params (dict) –
- dictionary of model parameters
- model_params[‘W’]: (H,D) ndarray
linear dictionary
- model_params[‘pi’]: (K,) ndarray
prior parameters
- model_params[‘sigma’]: float
standard deviation of noise model
- Returns
- model parameters
- model_params[‘W’]: (H,D) ndarray
linear dictionary
- model_params[‘pi’]: (K,) ndarray
prior parameters
- model_params[‘sigma’]: float
standard deviation of noise model
- Return type
-
free_energy(model_params, my_data)¶ Deprecated
-
gain(old_parameters, new_parameters)¶ Deprecated
-
generate_data(model_params, my_N, noise_on=True, gs=None, gp=None)¶ - Parameters
model_params (dict) –
- model parameters
- model_params[‘W’]: (H,D) ndarray
linear dictionary
- model_params[‘pi’]: (K,) ndarray
prior parameters
- model_params[‘sigma’]: float
standard deviation of noise model
my_N (int) – number of datapoints for this process
noise_on (bool, optional) – flag to control deterministic/stochastic generation. If True gaussian noise with standard deviation model_params[‘sigma’] is added to the data
gs ((my_N, H), optional) – ground truth latent variables. This option is used for generating artificial data with particular latent variables. Defaults to randomly sampled latent variables from the prior
gp ((my_N, H), optional) – ground truth posterior. This option is used for generating data that have a particular true posterior distribution. Defaults to randomly sampled latent variables from the prior
- Returns
- returns generated data
- dict[‘y’]: (my_N, D) ndarray
generated data
- dict[‘s’]: (my_N, H) ndarray
latent variable states that generated the data
- Return type
-
get_likelihood(D, sigma, logpj_all, N)¶ Data likelihood This functions computes the approximate likelihood of the data from the approximation of the joint
- Parameters
- Returns
the approximate likelihood value
- Return type
-
inference(anneal, model_params, test_data, topK=10, logprob=False, adaptive=True, Hprime_max=None, gamma_max=None)¶ Perform inference with the learned model on test data and return the top K configurations with their posterior probabilities.
- Parameters
anneal (Annealing object) – annealing information
model_params (dict) – dictionary with model parameters
test_data (dict) – data dictionary. The data in this case are ndarray under the key ‘y’.
topK (int, optional) – the number of most probable latent variable states to be returned
logprob (bool, optional) – the probabilities of the most probable latent variable states
adaptive (bool, optional) – if set to True it will run inference again for datapoints with gamma active latent variables in the top state using setting gamma=gamma+1 and Hprime=Hprime+1
Hprime_max (None, optional) – if adaptive is True it will stop Hprime from increasing above this integer. None defaults to H.
gamma_max (None, optional) – if adaptive is True it will stop gamma from increasing above this integer. None defaults to H.
- Returns
a dictionary with posterior information
- Return type
-
noisify_params(model_params, anneal)¶ Noisify model params.
Noisify the given model parameters according to self._noise_policy and the annealing object provided. The noise_policy of some model parameter PARAM will only be applied if the annealing object provides a noise strength via PARAM_noise.
- Parameters
model_params (dict) –
- dictionary containing model parameters
- model_params[‘W’]: (H,D) ndarray
linear dictionary
- model_params[‘pi’]: (K,) ndarray
prior parameters
- model_params[‘sigma’]: float
standard deviation of noise model
anneal (Annealing object) –
- Annealing type object containing training schedule information
anneal[‘W_noise’] : standard deviation of noise to be added to the dictionary anneal[‘pi_noise’]: standard deviation of noise to be added to the prior anneal[‘sigma_noise’]: standard deviation of noise to be added to the standard deviation of the noise model
- Returns
- model_paramsdict
- dictionary containing model parameters
- model_params[‘W’]: (H,D) ndarray
linear dictionary
- model_params[‘pi’]: (K,) ndarray
prior parameters
- model_params[‘sigma’]: float
standard deviation of noise model
- Return type
-
select_Hprimes(model_params, data)¶ Return a new data-dictionary which has been annotated with a data[‘candidates’] dataset. A set of self.Hprime candidates will be selected.
- Parameters
- Returns
- dataset dictionary
- data[‘y’]: (my_n,D) ndarray
datapoints
- data[‘candidates’]: (my_n,) ndarray
indices of the best explained datapoints
- Return type
-
select_partial_data(anneal, my_data)¶ Select a partial data-set from my_data and return it.
The fraction of datapoints selected is determined by anneal[‘partial’]. If anneal[‘partial’] is equal to either 1 or 0 the whole dataset will be returned.
- Parameters
anneal (Annealing object) –
- Annealing type object containing training schedule information
anneal[‘partial’] : fraction of the data to return
my_data (dict) –
- dictionary of the dataset
- my_data[‘y’]: (my_N, D) ndarray
the datapoints
- Returns
- dictionary of the dataset
- my_data[‘y’]: (my_N*anneal[‘partial’], D) ndarray
The updated datapoints
- Return type
-
standard_init(data)¶ Standard onitial estimation for model parameters.
This implementation each “W” raw is set to the average over the data plus white Gaussian noise of mean zero and standard deviation “sigma”/4. sigma is set to the variance of the data around the computed mean. “pi” is set to 1./H . Returns a dict with the estimated parameter set with entries “W”, “pi” and “sigma”.
-
Spike-and-Slab Sparse Coding¶
-
class
prosper.em.camodels.gsc_et.GSC(D, H, Hprime=0, gamma=0, sigma_sq_type='scalar', to_learn=['W', 'pi', 'mu', 'sigma_sq', 'psi_sq'], comm=<Mock id='140062023302784'>)¶ -
check_params(model_params)¶ Sanity check.
Sanity-check the given model parameters. Raises an exception if something is severely wrong.
-
compute_lpj(anneal, model_params, my_data)¶ Determine candidates and compute log-pseudo-joint.
-
generate_data(model_params, my_N)¶ given ground truth model parameters, generate data of size my_N
Generate data according to the MCA model while the latents are given in my_hdata[‘s’].
This method does _not_ obey gamma: The generated data may have more than gamma active causes for a given datapoint.
-
resume_init(h5_result_file)¶ Initialize model parameters to previously inferred values.
-
standard_init(my_data)¶ Standard Initial of the model parameters.
-
Maximum Component Analysis¶
-
class
prosper.em.camodels.mca_et.MCA_ET(D, H, Hprime, gamma, to_learn=['W', 'pi', 'sigma'], comm=<Mock id='140061936755600'>)¶ -
E_step(anneal, model_params, my_data)¶ MCA E_step
my_data variables used:
my_data[‘y’] Datapoints my_data[‘can’] Candidate H’s according to selection func.
Annealing variables used:
anneal[‘T’] Temperature for det. annealing AND softmax anneal[‘N_cut_factor’] 0.: no truncation; 1. trunc. according to model
-
M_step(anneal, model_params, my_suff_stat, my_data)¶ MCA M_step
my_data variables used:
my_data[‘y’] Datapoints my_data[‘candidates’] Candidate H’s according to selection func.
Annealing variables used:
anneal[‘T’] Temperature for det. annealing AND softmax anneal[‘N_cut_factor’] 0.: no truncation; 1. trunc. according to model
-
check_params(model_params)¶ Sanity-check the given model parameters. Raises an exception if something is severely wrong.
-
generate_data(model_params, my_N)¶ Generate data according to the MCA model.
This method does _not_ obey gamma: The generated data may have more than gamma active causes for a given datapoint.
-
select_Hprimes(model_params, data)¶ Return a new data-dictionary which has been annotated with a data[‘candidates’] dataset. A set of self.Hprime candidates will be selected.
-
Maximum Magnitude Component Analysis¶
-
class
prosper.em.camodels.mmca_et.MMCA_ET(D, H, Hprime, gamma, to_learn=['W', 'pi', 'sigma'], comm=<Mock id='140061936736688'>)¶ -
E_step(anneal, model_params, my_data)¶ MCA E_step
my_data variables used:
my_data[‘y’] Datapoints my_data[‘can’] Candidate H’s according to selection func.
Annealing variables used:
anneal[‘T’] Temperature for det. annealing AND softmax anneal[‘N_cut_factor’] 0.: no truncation; 1. trunc. according to model
-
M_step(anneal, model_params, my_suff_stat, my_data)¶ MCA M_step
my_data variables used:
my_data[‘y’] Datapoints my_data[‘candidates’] Candidate H’s according to selection func.
Annealing variables used:
anneal[‘T’] Temperature for det. annealing AND softmax anneal[‘N_cut_factor’] 0.: no truncation; 1. trunc. according to model
-
check_params(model_params)¶ Sanity-check the given model parameters. Raises an exception if something is severely wrong.
Generate data according to the MCA model while the latents are given in my_hdata[‘s’].
-
select_Hprimes(model_params, data)¶ Return a new data-dictionary which has been annotated with a data[‘candidates’] dataset. A set of self.Hprime candidates will be selected.
-
Mixture Models¶
Mixture Models refers to models where a single latent variable is responsible for the generation of a datapoint
-
class
prosper.em.mixturemodels.MixtureModel(D, H, to_learn=['W', 'pies'], comm=<Mock id='140061935930896'>)¶ -
check_params(model_params)¶ Sanity check.
Sanity-check the given model parameters. Raises an exception if something is severely wrong.
-
generate_data(model_params, my_N)¶ Generate data according to the model. Internally uses generate_data_from_hidden.
This method does _not_ obey gamma: The generated data may have more than gamma active causes for a given datapoint.
-
inference(anneal, model_params, my_data, no_maps=10)¶ To be implemented
-
select_partial_data(anneal, data)¶ Select a partial data-set from data and return it.
The fraction of datapoints selected is determined by anneal[‘partial’]. If anneal[‘partial’] is equal to either 1 or 0 the whole dataset will be returned.
-
standard_init(data)¶ Standard Initial Estimation for W and sigma.
each W raw is set to the average over the data plus WGN of mean zero and var sigma/4. sigma is set to the variance of the data around the computed mean. pi is set to 1./H . Returns a dict with the estimated parameter set with entries “W”, “pi” and “sigma”.
-
step(anneal, model_params, data)¶ Perform an EM-step
-
Mixture of Gaussians¶
Standard mixture of Gaussians model:
-
class
prosper.em.mixturemodels.MoG.MoG(D, H, to_learn=['pies', 'W', 'sigmas_sq'], sigmas_sq_type='full', comm=<Mock id='140061936307112'>)¶ -
check_params(model_params)¶ Sanity check.
Sanity-check the given model parameters. Raises an exception if something is severely wrong.
Generate datapoints according to the model.
Given the model parameters model_params return a dataset of N datapoints.
-
resume_init(h5_output)¶ Standard Initial Estimation for W, sigma and mu.
-
standard_init(my_data)¶ Standard Initial Estimation for W, sigmas and pies.
-
Mixture of Gaussians¶
Standard mixture model with a Poisson observation noise model:
-
class
prosper.em.mixturemodels.MoP.MoP(D, H, to_learn=['pies', 'W'], A=nan, comm=<Mock id='140061936156064'>)¶ -
check_params(model_params)¶ Sanity check.
Sanity-check the given model parameters. Raises an exception if something is severely wrong.
Generate datapoints according to the model.
Given the model parameters model_params return a dataset of N datapoints.
-
resume_init(h5_output)¶ Standard Initial Estimation for W, sigma and mu.
-
standard_init(my_data)¶ Standard Initial Estimation for W, sigma and mu.
-
Annealing¶
The annealing module holds utilities relevant to making minor modifications to the training process.
Annealing Class¶
This is a generic class inheritted by all annealing objects:
-
class
prosper.em.annealing.Annealing¶ Base class for implementations of annealing schemes.
Implementations deriving from this class control the cooling schedule and provide some additional control functions used in the EM algorithm.
-
next(gain)¶ Returns a (accept, T, finished)-tuple.
- accept is a boolean and indicates if the parameters changed by gain last iteration,
EM should accept the new parameters or if it should bae the next iteration on the old ones.
- finished is also a boolean and indicate whether the cooling has finished and EM should
drop out of the loop.
T is the temperature EM should use in the next iteration
-
reset()¶ Reset the cooling-cycle. This call returs the initial cooling temperature that will be used for the first step.
-
Linear Annealing¶
The linear annealing class is an example annealing class that makes changes at hyperparameters changing with linear rate over EM iterations.
-
class
prosper.em.annealing.LinearAnnealing(steps=80)¶ -
as_dict()¶ Return all annealing parameters with their current value as dict.
-
next(gain=0.0)¶ Step forward by one step.
After calling this method, this annealing object will potentially return different values for all its values.
-
reset()¶ Reset the cooling-cycle. This call returs the initial cooling temperature that will be used for the first step.
-