amber.architect
Contents
Rationale
This module amber.architect
can be devided into four major components. It’s easy to imagine
a model space to sample models from, and a training environment as a placeholder for
different components to interact. Currently the model space is pre-defined, but we can make it
dynamic and evolving in the future.
The remaining two components do the most of the heavy-lifting. Search algorithms, such as a controller
recurrent neural network vs a genetic algorithm, are separated by sub-packages (a folder with __init__.py
)
and each variant of a search algorithm is a separate file. This makes it easy to re-use code within
a search algorithm, relatively independent across different searchers, and let them share the same
configurations of model space and training environments (although it’s possible only certain combinations
of searcher - model space - train env are viable).
The manager is perhaps the most tedious. The general workflow is:
takes a model architecture
arch
as input,passes
arch
to the modelermodel=amber.modeler.model_fn(arch)
,trains the model
model.fit(train_data)
,evaluates the model’s reward
reward=reward_fn(model)
,stores the model in a buffer
buffer_fn.store(model, reward)
,returns the reward signal to the search algorithm.
Each of the steps has variantions, but the overall layout should almost always stay as described above.
Model Space
Model space to perform architecture search
- class BranchedModelSpace(subspaces, concat_op='concatenate', **kwargs)[source]
Bases:
amber.architect.modelSpace.ModelSpace
- Parameters
subspaces (list) – A list of ModelSpace. First element is a list of input branches. Second element is a stem model space
concat_op (str) – string identifier for how to concatenate different input branches
- property branch_to_layer
- property layer_to_branch
- class ModelSpace(**kwargs)[source]
Bases:
object
Model Space constructor
Provides utility functions for holding “states” / “operations” that the controller must use to train and predict. Also provides a more convenient way to define the model search space
There are several ways to construct a model space. For example, one way is to initialize an empty
ModelSpace
then iteratively add layers to it, where each layer has a number of candidate operations:>>> def get_model_space(out_filters=64, num_layers=9): >>> model_space = ModelSpace() >>> num_pool = 4 >>> expand_layers = [num_layers//num_pool*i-1 for i in range(1, num_pool)] >>> for i in range(num_layers): >>> model_space.add_layer(i, [ >>> Operation('conv1d', filters=out_filters, kernel_size=8, activation='relu'), >>> Operation('conv1d', filters=out_filters, kernel_size=4, activation='relu'), >>> Operation('maxpool1d', filters=out_filters, pool_size=4, strides=1), >>> Operation('avgpool1d', filters=out_filters, pool_size=4, strides=1), >>> Operation('identity', filters=out_filters), >>> ]) >>> if i in expand_layers: >>> out_filters *= 2 >>> return model_space
Alternatively, ModelSpace can also be constructed from a dictionary.
- add_layer(layer_id, layer_states=None)[source]
Add a new layer to model space
- Parameters
layer_id (int) – The layer id of which layer to be added. Can be incontinuous to previous layers.
layer_states (list of amber.architect.Operation) – A list of
Operation
object to be added.
- Returns
Boolean value of Whether the model space is valid after inserting this layer
- Return type
- add_state(layer_id, state)[source]
Append a new state/operation to a layer
- Parameters
layer_id (int) – Which layer to append a new operation.
state (amber.architect.State) – The new operation object to be appended.
- delete_layer(layer_id)[source]
Delete an entire layer and its associated values
- Parameters
layer_id (int) – which layer index to be deleted
- Returns
Boolean value of Whether the model space is valid after inserting this layer
- Return type
- delete_state(layer_id, state_id)[source]
Delete an operation from layer
- Parameters
layer_id (int) – Which layer to delete an operation
state_id (int) – Which operation index to be deleted
- static from_dict(d)[source]
Static method for creating a ModelSpace from a Dictionary or List
- Parameters
d (dict or list) – A dictionary or list specifying candidate operations for each layer
- Returns
The constructed model space from the given dict/list
- Return type
amber.architect.ModelSpace
- get_random_model_states()[source]
Get a random combination of model operations throughout each layer
- Returns
model_states – A list of randomly sampled model operations
- Return type
- Operation
alias of
amber.architect.modelSpace.State
- class State(Layer_type, **kwargs)[source]
Bases:
object
The Amber internal holder for a computational operation at any layer
- Parameters
Layer_type (str) – The string for the operation type; supports most commonly used
tf.keras.layers
typeskwargs – Operation/layer specifications are parsed through keyword arguments
- Variables
Notes
Any attributes that are not specified in
Layer_attributes
will use the default value as defined intf.keras.layers
. For example, if you do not specifyactivation
inLayer_attributes
, it will uselinear
.Examples
For example, to create a 1D-convolutional operation with ReLU activation, kernel size=8, number of kernels=32:
>>> from amber.architect import State >>> op = State("conv1d", filters=32, kernel_size=8, activation='relu')
- get_layer_shortname(layer)[source]
Get the short name for a computational operation of a layer, useful in converting a Layer object to a string as ID or when plotting
- Parameters
layer (amber.architect.Operation) – The
Operation
object for any layer.- Returns
sn – The unique short name for this operation
- Return type
Todo
Consider refactoring
layer
tooperation
Train Environment
Training environment provides interactions between several components within architect and outside.
- class ControllerTrainEnvironment(controller, manager, max_episode=100, max_step_per_ep=2, logger=None, resume_prev_run=False, should_plot=True, initial_buffering_queue=15, working_dir='.', entropy_converge_epsilon=0.01, squeezed_action=True, with_input_blocks=False, with_skip_connection=True, save_controller=True, continuous_run=False, verbose=0, **kwargs)[source]
Bases:
object
The training evnrionment employs
controller
model andmanager
to mange data and reward, creates a reinforcement learning environment- Parameters
controller (amber.architect.BaseController) – The controller to search architectures in this environment.
manager (amber.architect.BaseManager) – The manager to interact with modelers in this environment.
max_episode (int) – Maximum number of controller steps. Each controller step will sample
max_step_per_ep
child model architectures.max_step_per_ep (int) – The number of child model architectures to sample in each controller step (aka
max_episode
).logger (logging.Logger or None) – The logger to use in environment
resume_prev_run (bool) – If true, try to reload existing controller and history in the given working directory. Default is False.
should_plot (bool) – If false, turn off the plots at the end of training. Default is False.
initial_buffering_queue (int)
- working_dirstr
File path to working directory
- squeezed_actionbool
If false, will expect each entry in architecture sequence to be strictly one-hot encoded; if true, some entries will be categorically encoded. Categorical encoding is consistent with current settings in
amber.architect.GeneralController
, thus is recommendedsqueezed_action=True
. Default is True.- with_input_blocksbool
Whether the controller will search for input blocks. Default is False.
- with_skip_connectionbool
Whether the controller will search for residual/skip connections. Default is True.
- save_controllerbool
Whether to save the final controller parameters in working directory after training (i.e. reaching the max controller steps). Default is True.
- verbosebool or int
Verbosity level
Notes
- Note if either with_input_blocks or with_skip_connection, a list of integers will be used to represent the
sequential model architecture and wiring, instead of a list of amber.architect.Operation
Todo
Refactor the rest of attributes to be private.
- class EnasTrainEnv(*args, **kwargs)[source]
Bases:
amber.architect.trainEnv.ControllerTrainEnvironment
- Params:
time_budget: defaults to 72 hours
- class MultiManagerEnvironment(data_descriptive_features, is_enas='auto', *args, **kwargs)[source]
Bases:
amber.architect.trainEnv.EnasTrainEnv
MultiManagerEnvironment is an environment that allows one controller to interact with multiple EnasManagers
Controller(s)
Implementations of NAS controller for searching architectures
Changelog
Aug. 7, 2018: initial
Feb. 6. 2019: finished initial OperationController
Jun. 17. 2019: separated to OperationController and GeneralController
Aug. 15, 2020: updated documentations
- class GeneralController(model_space, buffer_type='ordinal', with_skip_connection=True, share_embedding=None, use_ppo_loss=False, kl_threshold=0.05, skip_connection_unique_connection=False, buffer_size=15, batch_size=5, session=None, train_pi_iter=20, lstm_size=32, lstm_num_layers=2, lstm_keep_prob=1.0, tanh_constant=None, temperature=None, optim_algo='adam', skip_target=0.8, skip_weight=None, rescale_advantage_by_reward=False, name='controller', verbose=0, **kwargs)[source]
Bases:
amber.architect.controller.generalController.BaseController
GeneralController for neural architecture search
- This class searches for two steps:
computational operations for each layer
skip connections for each layer from all previous layers [optional]
It is a modified version of enas: https://github.com/melodyguan/enas . Notable modifications include: dissection of sampling and training processes to enable better understanding of controller behaviors, buffering and logging; loss function can be optimized by either REINFORCE or PPO.
Todo
Refactor the rest of the attributes to private.
- Parameters
model_space (amber.architect.ModelSpace) – A ModelSpace object constructed to perform architecture search for.
with_skip_connection (bool) – If false, will not search residual connections and only search for computation operations per layer. Default is True.
share_embedding (dict) – a Dictionary defining which child-net layers will share the softmax and embedding weights during Controller training and sampling. For example,
{1:0, 2:0}
means layer 1 and 2 will share the embedding with layer 0.use_ppo_loss (bool) – If true, use PPO loss for optimization instead of REINFORCE. Default is False.
kl_threshold (float) – If KL-divergence between the sampling probabilities of updated controller parameters and that of original parameters exceeds kl_threshold within a single controller training step, triggers early-stopping to halt the controller training. Default is 0.05.
buffer_size (int) – amber.architect.Buffer stores only the sampled architectures from the last
buffer_size
number of from previous controller steps, where each step has a number of sampled architectures as specified inamber.architect.ControllerTrainEnv
.batch_size (int) – How many architectures in a batch to train the controller
session (tf.Session) – The session where the controller tensors is placed
train_pi_iter (int) – The number of epochs/iterations to train controller policy in one controller step.
lstm_size (int) – The size of hidden units for stacked LSTM, i.e. controller RNN.
lstm_num_layers (int) – The number of stacked layers for stacked LSTM, i.e. controller RNN.
lstm_keep_prob (float) – keep_prob = 1 - dropout probability for stacked LSTM.
tanh_constant (float) – If not None, the logits for each multivariate classification will be transformed by
tf.tanh
then multiplied by tanh_constant. This can avoid over-confident controllers asserting probability=1 or 0 caused by logit going to +/- inf. Default is None.temperature (float) – The temperature is a scale factor to logits. Higher temperature will flatten the probabilities among different classes, while lower temperature will freeze them. Default is None, i.e. 1.
optim_algo (str) – Optimizer for controller RNN. Can choose from [“adam”, “sgd”, “rmsprop”]. Default is “adam”.
skip_target (float) – The expected proportion of skip connections, i.e. the proportion of 1’s in the skip/extra connections in the output arc_seq
skip_weight (float) – The weight for skip connection kl-divergence from the expected skip_target
name (str) – The name for this Controller instance; all
tf.Tensors
will be placed under this VariableScope. This name determines which tensors will be initialized when a new Controller instance is created.
- Variables
weights (list of tf.Variable) – The list of all trainable
tf.Variable
in this controllermodel_space (amber.architect.ModelSpace) – The model space which the controller will be searching from.
buffer (amber.architect.Buffer) – The Buffer object stores the history architectures, computes the rewards, and gets feed dict for training.
session (tf.Session) – The reference to the session that hosts this controller instance.
- static convert_arc_to_onehot(controller)[source]
Convert a categorical architecture sequence to a one-hot encoded architecture sequence
- Parameters
controller (amber.architect.controller) – An instance of controller
- Returns
onehot_list – a one-hot encoded architecture sequence
- Return type
- get_action(*args, **kwargs)[source]
Get a sampled architecture/action and its corresponding probabilities give current controller policy parameters.
The generated architecture is the out-going information from controller to manager. which in turn will feedback the reward signal for storage and training by the controller.
- Parameters
None
- Returns
onehots –
The sampled architecture sequence. In particular, the architecture sequence is ordered as:
[categorical_operation_0, categorical_operation_1, binary_skip_0, categorical_operation_2, binary_skip_0, binary_skip_1, ...]
- Return type
- probslist of ndarray
The probabilities associated with each sampled operation and residual connection. Shapes will vary depending on each layer’s specification in ModelSpace for operation, and the layer number for residual connections.
- get_weights(**kwargs)[source]
Get the current controller weights in a numpy array
- Parameters
None
- Returns
weights – A list of numpy array for each weights in controller
- Return type
- load_weights(filepath, **kwargs)[source]
Load the controller weights from a hdf5 file
- Parameters
filepath (str) – file path to saved weights
- Return type
None
- static remove_files(files, working_dir='.')[source]
Static method for removing files
- Parameters
files (list of str) – files to be removed
working_dir (str) – filepath to working directory
- Return type
None
- save_weights(filepath, **kwargs)[source]
Save current controller weights to a hdf5 file
- Parameters
filepath (str) – file path to save the weights
- Return type
None
- set_weights(weights, **kwargs)[source]
Set the current controller weights
- Parameters
weights (list of numpy.ndarray) – A list of numpy array for each weights in controller
- Return type
None
- store(state, prob, action, reward, *args, **kwargs)[source]
Store all necessary information and rewards for a given architecture
This is the receiving method for controller to interact with manager by storing the rewards for a given architecture. The architecture and its probabilities can be generated by
get_action()
method.- Parameters
state (list) – The state for which the action and probabilities are drawn.
prob (list of ndarray) – A list of probabilities for each operation and skip connections.
action (list) –
A list of architecture tokens ordered as:
[categorical_operation_0, categorical_operation_1, binary_skip_0, categorical_operation_2, binary_skip_0, binary_skip_1, ...]
reward (float) – Reward for this architecture, as evaluated by
amber.architect.manager
- Return type
None
- train(episode, working_dir)[source]
Train the controller policy parameters for one step.
- Parameters
episode (int) – Total number of epochs to train the controller. Each epoch will iterate over all architectures stored in buffer.
working_dir (str) – Filepath to working directory to store (possible) intermediate results
- Returns
aloss – Average controller loss for this train step
- Return type
Notes
Consider renaming this method to
train_step()
to better reflect its function, and avoid confusion with the training function in environmentControllerTrainEnv.train()
- class MultiIOController(num_output_blocks=2, with_output_blocks=True, output_block_unique_connection=True, output_block_diversity_weight=None, **kwargs)[source]
Bases:
amber.architect.controller.multiioController.MultiInputController
Example
from amber.bootstrap.dense_skipcon_space import get_model_space from amber.architect.controller.multiioController import MultiIOController import numpy as np import tensorflow as tf s = tf.Session() model_space = get_model_space(5) controller = MultiIOController(model_space=model_space, output_block_unique_connection=True, session=s) a1, p1 = controller.get_action() a2, p2 = controller.get_action() a_batch = np.array([a1,a2]) p_batch = [np.concatenate(x) for x in zip(*[p1,p2])] feed_dict = {controller.input_arc[i]: a_batch[:, [i]] for i in range(a_batch.shape[1])} feed_dict.update({controller.advantage: np.array([1., -1.]).reshape((2,1))}) feed_dict.update({controller.old_probs[i]: p_batch[i] for i in range(len(controller.old_probs))}) feed_dict.update({controller.reward: np.array([1., 1.]).reshape((2,1))}) print(s.run(controller.onehot_log_prob, feed_dict)) for _ in range(100): s.run(controller.train_op, feed_dict=feed_dict) if _%20==0: print(s.run(controller.loss, feed_dict)) print(s.run(controller.onehot_log_prob, feed_dict))
Notes
Placeholder for now.
- class MultiInputController(model_space, buffer_type='ordinal', with_skip_connection=True, with_input_blocks=True, share_embedding=None, use_ppo_loss=False, kl_threshold=0.05, num_input_blocks=2, input_block_unique_connection=True, skip_connection_unique_connection=False, buffer_size=15, batch_size=5, session=None, train_pi_iter=20, lstm_size=32, lstm_num_layers=2, lstm_keep_prob=1.0, tanh_constant=None, temperature=None, skip_target=0.8, skip_weight=0.5, optim_algo='adam', name='controller', *args, **kwargs)[source]
Bases:
amber.architect.controller.generalController.GeneralController
DOCSTRING
- Parameters
model_space
with_skip_connection
with_input_blocks
share_embedding (dict) – a Dictionary defining which child-net layers will share the softmax and embedding weights during Controller training and sampling
use_ppo_loss
kl_threshold
num_input_blocks
input_block_unique_connection
buffer_size
batch_size
session
train_pi_iter
lstm_size
lstm_num_layers
lstm_keep_prob
tanh_constant
temperature
lr_init
lr_dec_start
lr_dec_every
lr_dec_rate
l2_reg
clip_mode
grad_bound
use_critic
bl_dec
optim_algo
sync_replicas
num_aggregate
num_replicas
skip_target (float) – the expected proportion of skip connections, i.e. the proportion of 1’s in the skip/extra connections in the output arc_seq
skip_weight (float) – the weight for skip connection kl-divergence from the expected skip_target
name (str) – name for the instance; also used for all tensor variable scopes
- Variables
g_emb (tf.Tensor) – initial controller hidden state tensor; to be learned
Placeholder –
Note
This class derived from GeneralController adds the input feature block selection upon the Base class. Since the selection is inherently embedded in the NAS cell rolling-out, the sampler and trainer methods are overwritten.
- TODO:
needs to evaluate how different ways of connecting inputs will affect search performance; e.g. connect input before operation or after?
- class OperationController(state_space, controller_units, embedding_dim=5, optimizer='rmsprop', discount_factor=0.0, clip_val=0.2, beta=0.001, kl_threshold=0.05, train_pi_iter=100, lr_pi=0.005, buffer_size=50, batch_size=5, verbose=0)[source]
Bases:
amber.architect.controller.generalController.BaseController
- example state_space for a 2-layer conv-net:
- state_space = [[‘conv3’, ‘conv5’, ‘conv7’], [‘maxp2’, ‘avgp2’],
[‘conv3’, ‘conv5’, ‘conv7’], [‘maxp2’, ‘avgp2’]]
- class ZeroShotController(data_description_config, *args, **kwargs)[source]
Bases:
amber.architect.controller.generalController.GeneralController
General Controller
General controller for searching computational operation per layer, and residual connection
- class GeneralController(model_space, buffer_type='ordinal', with_skip_connection=True, share_embedding=None, use_ppo_loss=False, kl_threshold=0.05, skip_connection_unique_connection=False, buffer_size=15, batch_size=5, session=None, train_pi_iter=20, lstm_size=32, lstm_num_layers=2, lstm_keep_prob=1.0, tanh_constant=None, temperature=None, optim_algo='adam', skip_target=0.8, skip_weight=None, rescale_advantage_by_reward=False, name='controller', verbose=0, **kwargs)[source]
Bases:
amber.architect.controller.generalController.BaseController
GeneralController for neural architecture search
- This class searches for two steps:
computational operations for each layer
skip connections for each layer from all previous layers [optional]
It is a modified version of enas: https://github.com/melodyguan/enas . Notable modifications include: dissection of sampling and training processes to enable better understanding of controller behaviors, buffering and logging; loss function can be optimized by either REINFORCE or PPO.
Todo
Refactor the rest of the attributes to private.
- Parameters
model_space (amber.architect.ModelSpace) – A ModelSpace object constructed to perform architecture search for.
with_skip_connection (bool) – If false, will not search residual connections and only search for computation operations per layer. Default is True.
share_embedding (dict) – a Dictionary defining which child-net layers will share the softmax and embedding weights during Controller training and sampling. For example,
{1:0, 2:0}
means layer 1 and 2 will share the embedding with layer 0.use_ppo_loss (bool) – If true, use PPO loss for optimization instead of REINFORCE. Default is False.
kl_threshold (float) – If KL-divergence between the sampling probabilities of updated controller parameters and that of original parameters exceeds kl_threshold within a single controller training step, triggers early-stopping to halt the controller training. Default is 0.05.
buffer_size (int) – amber.architect.Buffer stores only the sampled architectures from the last
buffer_size
number of from previous controller steps, where each step has a number of sampled architectures as specified inamber.architect.ControllerTrainEnv
.batch_size (int) – How many architectures in a batch to train the controller
session (tf.Session) – The session where the controller tensors is placed
train_pi_iter (int) – The number of epochs/iterations to train controller policy in one controller step.
lstm_size (int) – The size of hidden units for stacked LSTM, i.e. controller RNN.
lstm_num_layers (int) – The number of stacked layers for stacked LSTM, i.e. controller RNN.
lstm_keep_prob (float) – keep_prob = 1 - dropout probability for stacked LSTM.
tanh_constant (float) – If not None, the logits for each multivariate classification will be transformed by
tf.tanh
then multiplied by tanh_constant. This can avoid over-confident controllers asserting probability=1 or 0 caused by logit going to +/- inf. Default is None.temperature (float) – The temperature is a scale factor to logits. Higher temperature will flatten the probabilities among different classes, while lower temperature will freeze them. Default is None, i.e. 1.
optim_algo (str) – Optimizer for controller RNN. Can choose from [“adam”, “sgd”, “rmsprop”]. Default is “adam”.
skip_target (float) – The expected proportion of skip connections, i.e. the proportion of 1’s in the skip/extra connections in the output arc_seq
skip_weight (float) – The weight for skip connection kl-divergence from the expected skip_target
name (str) – The name for this Controller instance; all
tf.Tensors
will be placed under this VariableScope. This name determines which tensors will be initialized when a new Controller instance is created.
- Variables
weights (list of tf.Variable) – The list of all trainable
tf.Variable
in this controllermodel_space (amber.architect.ModelSpace) – The model space which the controller will be searching from.
buffer (amber.architect.Buffer) – The Buffer object stores the history architectures, computes the rewards, and gets feed dict for training.
session (tf.Session) – The reference to the session that hosts this controller instance.
- static convert_arc_to_onehot(controller)[source]
Convert a categorical architecture sequence to a one-hot encoded architecture sequence
- Parameters
controller (amber.architect.controller) – An instance of controller
- Returns
onehot_list – a one-hot encoded architecture sequence
- Return type
- get_action(*args, **kwargs)[source]
Get a sampled architecture/action and its corresponding probabilities give current controller policy parameters.
The generated architecture is the out-going information from controller to manager. which in turn will feedback the reward signal for storage and training by the controller.
- Parameters
None
- Returns
onehots –
The sampled architecture sequence. In particular, the architecture sequence is ordered as:
[categorical_operation_0, categorical_operation_1, binary_skip_0, categorical_operation_2, binary_skip_0, binary_skip_1, ...]
- Return type
- probslist of ndarray
The probabilities associated with each sampled operation and residual connection. Shapes will vary depending on each layer’s specification in ModelSpace for operation, and the layer number for residual connections.
- get_weights(**kwargs)[source]
Get the current controller weights in a numpy array
- Parameters
None
- Returns
weights – A list of numpy array for each weights in controller
- Return type
- load_weights(filepath, **kwargs)[source]
Load the controller weights from a hdf5 file
- Parameters
filepath (str) – file path to saved weights
- Return type
None
- static remove_files(files, working_dir='.')[source]
Static method for removing files
- Parameters
files (list of str) – files to be removed
working_dir (str) – filepath to working directory
- Return type
None
- save_weights(filepath, **kwargs)[source]
Save current controller weights to a hdf5 file
- Parameters
filepath (str) – file path to save the weights
- Return type
None
- set_weights(weights, **kwargs)[source]
Set the current controller weights
- Parameters
weights (list of numpy.ndarray) – A list of numpy array for each weights in controller
- Return type
None
- store(state, prob, action, reward, *args, **kwargs)[source]
Store all necessary information and rewards for a given architecture
This is the receiving method for controller to interact with manager by storing the rewards for a given architecture. The architecture and its probabilities can be generated by
get_action()
method.- Parameters
state (list) – The state for which the action and probabilities are drawn.
prob (list of ndarray) – A list of probabilities for each operation and skip connections.
action (list) –
A list of architecture tokens ordered as:
[categorical_operation_0, categorical_operation_1, binary_skip_0, categorical_operation_2, binary_skip_0, binary_skip_1, ...]
reward (float) – Reward for this architecture, as evaluated by
amber.architect.manager
- Return type
None
- train(episode, working_dir)[source]
Train the controller policy parameters for one step.
- Parameters
episode (int) – Total number of epochs to train the controller. Each epoch will iterate over all architectures stored in buffer.
working_dir (str) – Filepath to working directory to store (possible) intermediate results
- Returns
aloss – Average controller loss for this train step
- Return type
Notes
Consider renaming this method to
train_step()
to better reflect its function, and avoid confusion with the training function in environmentControllerTrainEnv.train()
MultiIO Controller
- class MultiIOController(num_output_blocks=2, with_output_blocks=True, output_block_unique_connection=True, output_block_diversity_weight=None, **kwargs)[source]
Bases:
amber.architect.controller.multiioController.MultiInputController
Example
from amber.bootstrap.dense_skipcon_space import get_model_space from amber.architect.controller.multiioController import MultiIOController import numpy as np import tensorflow as tf s = tf.Session() model_space = get_model_space(5) controller = MultiIOController(model_space=model_space, output_block_unique_connection=True, session=s) a1, p1 = controller.get_action() a2, p2 = controller.get_action() a_batch = np.array([a1,a2]) p_batch = [np.concatenate(x) for x in zip(*[p1,p2])] feed_dict = {controller.input_arc[i]: a_batch[:, [i]] for i in range(a_batch.shape[1])} feed_dict.update({controller.advantage: np.array([1., -1.]).reshape((2,1))}) feed_dict.update({controller.old_probs[i]: p_batch[i] for i in range(len(controller.old_probs))}) feed_dict.update({controller.reward: np.array([1., 1.]).reshape((2,1))}) print(s.run(controller.onehot_log_prob, feed_dict)) for _ in range(100): s.run(controller.train_op, feed_dict=feed_dict) if _%20==0: print(s.run(controller.loss, feed_dict)) print(s.run(controller.onehot_log_prob, feed_dict))
Notes
Placeholder for now.
- class MultiInputController(model_space, buffer_type='ordinal', with_skip_connection=True, with_input_blocks=True, share_embedding=None, use_ppo_loss=False, kl_threshold=0.05, num_input_blocks=2, input_block_unique_connection=True, skip_connection_unique_connection=False, buffer_size=15, batch_size=5, session=None, train_pi_iter=20, lstm_size=32, lstm_num_layers=2, lstm_keep_prob=1.0, tanh_constant=None, temperature=None, skip_target=0.8, skip_weight=0.5, optim_algo='adam', name='controller', *args, **kwargs)[source]
Bases:
amber.architect.controller.generalController.GeneralController
DOCSTRING
- Parameters
model_space
with_skip_connection
with_input_blocks
share_embedding (dict) – a Dictionary defining which child-net layers will share the softmax and embedding weights during Controller training and sampling
use_ppo_loss
kl_threshold
num_input_blocks
input_block_unique_connection
buffer_size
batch_size
session
train_pi_iter
lstm_size
lstm_num_layers
lstm_keep_prob
tanh_constant
temperature
lr_init
lr_dec_start
lr_dec_every
lr_dec_rate
l2_reg
clip_mode
grad_bound
use_critic
bl_dec
optim_algo
sync_replicas
num_aggregate
num_replicas
skip_target (float) – the expected proportion of skip connections, i.e. the proportion of 1’s in the skip/extra connections in the output arc_seq
skip_weight (float) – the weight for skip connection kl-divergence from the expected skip_target
name (str) – name for the instance; also used for all tensor variable scopes
- Variables
g_emb (tf.Tensor) – initial controller hidden state tensor; to be learned
Placeholder –
Note
This class derived from GeneralController adds the input feature block selection upon the Base class. Since the selection is inherently embedded in the NAS cell rolling-out, the sampler and trainer methods are overwritten.
- TODO:
needs to evaluate how different ways of connecting inputs will affect search performance; e.g. connect input before operation or after?
Operation Controller
A pure Keras-implementation of NAS ZZJ Aug. 7, 2018
- class OperationController(state_space, controller_units, embedding_dim=5, optimizer='rmsprop', discount_factor=0.0, clip_val=0.2, beta=0.001, kl_threshold=0.05, train_pi_iter=100, lr_pi=0.005, buffer_size=50, batch_size=5, verbose=0)[source]
Bases:
amber.architect.controller.generalController.BaseController
- example state_space for a 2-layer conv-net:
- state_space = [[‘conv3’, ‘conv5’, ‘conv7’], [‘maxp2’, ‘avgp2’],
[‘conv3’, ‘conv5’, ‘conv7’], [‘maxp2’, ‘avgp2’]]
Zero-Shot Controller (AMBIENT)
Zero-shot controller takes in descriptive features from dataset when calling .get_action() method, then generate new architectures for new tasks using a trained controller based on the data-descriptive features for the new task.
ZZ, May 13, 2020
- class ZeroShotController(data_description_config, *args, **kwargs)[source]
Bases:
amber.architect.controller.generalController.GeneralController
Manager
Manager class for streamlining downstream build and evaluation given an architecture.
Manager is the class that takes in architecture designs from an architecture search/optimization algorithm, then
interacts with amber.modeler
to build and train the model according to architecture, and finally calls
amber.architect.rewards
to evaluate the trained model rewards to feedback the architecture designer.
- class DistributedGeneralManager(devices, train_data_kwargs, validate_data_kwargs, do_resample=False, *args, **kwargs)[source]
Bases:
amber.architect.manager.GeneralManager
Distributed manager will place all tensors of any child models to a pre-assigned GPU device
- class GeneralManager(train_data, validation_data, model_fn, reward_fn, store_fn, working_dir='.', save_full_model=False, epochs=5, child_batchsize=128, verbose=0, fit_kwargs=None, predict_kwargs=None, evaluate_kwargs=None, **kwargs)[source]
Bases:
amber.architect.manager.BaseNetworkManager
Manager creates child networks, train them on a dataset, and retrieve rewards.
- Parameters
train_data (tuple, string or generator) – Training data to be fed to
keras.models.Model.fit
.validation_data (tuple, string, or generator) – Validation data. The data format is understood similarly to train_data.
model_fn (amber.modeler) – A callable function to build and implement child models given an architecture sequence.
reward_fn (amber.architect.rewards) – A callable function to evaluate the rewards on a trained model and the validation dataset.
store_fn (amber.architect.store) – A callable function to store necessary information (such as predictions, model architectures, and a variety of plots etc.) for the given child model.
working_dir (str) – File path for working directory.
save_full_model (bool) – If true, save the full model beside the model weights. Default is False.
epochs (int) – The total number of epochs to train the child model.
child_batchsize (int) – The batch size for training the child model.
fit_kwargs (dict or None) – Keyword arguments for model.fit
predict_kwargs (dict or None) – Keyword arguments for model.predict
evaluate_kwargs (dict or None) – Keyword arguments for model.evaluate
verbose (bool or int) – Verbose level. 0=non-verbose, 1=verbose, 2=less verbose.
kwargs (dict) – Other keyword arguments parsed.
- Variables
train_data (tuple or generator) – The unpacked training data
validation_data (tuple or generator) – The unpacked validation data
model_fn (amber.modeler) – Reference to the callable function to build and implement child models given an architecture sequence.
reward_fn (amber.architect.rewards) – Reference to the callable function to evaluate the rewards on a trained model and the validation dataset.
store_fn (amber.architect.store) – Reference to the callable function to store necessary information (such as predictions, model architectures, and a variety of plots etc.) for the given child model.
working_dir (str) – File path to working directory
Todo
Refactor the rest of attributes as private.
Update the description of
train_data
andvalidation_data
to more flexible unpacking, once it’s added:If it's tuple, expects it to be a tuple of numpy.array of (x,y); if it's string, expects it to be the file path to a compiled training data; if it's a generator, expects it yield a batch of training features and samples.
- get_rewards(trial, model_arc, **kwargs)[source]
The reward getter for a given model architecture
- Parameters
trial (int) – An integer number indicating the trial for this architecture
model_arc (list) – The list of architecture sequence
- Returns
this_reward (float) – The reward signal as determined by
reward_fn(model, val_data)
loss_and_metrics (dict) – A dictionary of auxillary information for this model, such as loss, and other metrics (as in
tf.keras.metrics
)
Buffer
Buffer class for holding reinforcement-learning histories
Buffer must have the following functionality:
store a new incoming architecture and its associated reward and probability
get data from the stored buffers
- class Buffer(max_size, discount_factor=0.0, ewa_beta=None, is_squeeze_dim=False, rescale_advantage_by_reward=False, clip_advantage=10.0)[source]
Bases:
object
The ordinal buffer class
Buffer stores the sampled architectures, computed bias-adjusted rewards/advantages, and make feed-dict related through get_data method.
The main attributes of
Buffer
can be divided into short-term and long-term: short-term buffers are directly appending new entry to the lists; while long-term buffers will concatenate short-term buffers, perform temporal discounts and/or adjust for baseline rewards.An entire short-term buffer will correspond to one entry in long-term buffer.
Short-term buffers will be emptied when converted to long-term, and long-term buffers will be kept at a maximum length specified by
max_size
argument.- Parameters
max_size (int) – The maximum number of controller steps of sampled architectures to store. Stored data beyond the last
max_size
will be dumped to disk.discount_factor (float) – Temporal discount factor. Not used for now.
ewa_beta (float) – The average is approx. over the past 1/(1-ewa_beta)
is_squeeze_dim (bool) – If True, controller samples a sequence of tokens, instead of one-hot arrays. Default is False.
rescale_advantage_by_reward (bool) – If True, the advantage will be divided by reward to rescale it as a proportion of increase/decrease of reward
clip_advantage (float) – Clip the advantage to the value of
clip_advantage
if the advantage exceeds it. Default is 10.0.
- Variables
r_bias (float) – The moving baseline for reward; used to compute advantage = this_reward - r_bias.
state_buffer (list) – The list of short-term states; each entry corresponds to one architecture generated by
amber.architect.BaseController.get_action()
.action_buffer (list) – The list of short-term categorical-encoded architectures; each entry is a list, corresponding to one architecture generated by
amber.architect.BaseController.get_action()
prob_buffer (list) – The list of short-term probabilities; each entry is a list of numpy.ndarray, where each numpy.ndarray is probability.
reward_buffer (list) – The list of short-term rewards.
lt_sbuffer (list of numpy.array) – The list of long-term states; each entry is a numpy.array concatenated from several short-term buffer to facilitate batching in
get_data()
.lt_abuffer (list of numpy.array) – The list of long-term architectures; each entry is a numpy.array concatenated from several short-term buffer architectures.
lt_pbuffer (list of list of numpy.array) – The list of long-term probabilities; each entry is a list of numpy.array corresponding to several short-term buffer probabilities. Each numpy.array is concatenated probabilities of the same layer and type from several short-term entries.
lt_adbuffer (list of list of floats) – The list of long-term advantages; each entry is floats of rewards corresponding to one short-term buffer.
lt_rmbuffer (list of floats) – The list of reward mean for each short-term buffer.
lt_nrbuffer (list of numpy.array) – The list of original rewards for each short-term buffer.
Todo
Refactor the rest of attributes to be private.
- discount_rewards()[source]
Discount rewards by temporal discount factor. This is not currently used in architecture searching.
Example
behavior of discounted reward, given
reward_buffer=[1,2,3]
:if buffer.discount_factor=0.1: ([array([[1.23]]), array([[2.3]]), array([[3.]])], [1, 2, 3]) if buffer.discount_factor=0.9: ([array([[5.23]]), array([[4.7]]), array([[3.]])], [1, 2, 3])
- finish_path(state_space, global_ep, working_dir, *args, **kwargs)[source]
Finish path of short-term buffer and convert to long-term buffer
An entire short-term buffer will correspond to one entry in long-term buffer. After conversion, this will also dump buffers to file and empty the short-term buffer.
- Parameters
state_space (amber.architect.ModelSpace) – The model space to perform search on.
global_ep (int) – The global number of controller steps
working_dir (str) – File path to working directory.
- get_data(bs, shuffle=True)[source]
Get a batched data
size of buffer: (traj, traj_len, data_shape)
- Parameters
bs (int) – batch size
shuffle (bool) – Randomly shuffle the index before yielding data. Default is True.
- Yields
list – A list of batched data; entries are ordered as:
[state, prob, arc_seq, advantages, rewards]
. Each entry is a batch of data points as specified bybs
.
- class MultiManagerBuffer(max_size, ewa_beta=None, is_squeeze_dim=False, rescale_advantage_by_reward=False, clip_advantage=3.0, **kwargs)[source]
Bases:
object
- property lt_abuffer
For legacy use
- property lt_adbuffer
For legacy use
- property lt_pbuffer
For legacy use
- store(prob, action, reward, description, manager_index)[source]
- Parameters
prob (list) – A list of np.array of different sizes corresponding to each layer
action (list) – A list of np.array of one-hot actions
reward (float) – Float number of reward to be maximized
description (1D-array like) – A vector of data descriptor
manager_index (int) – Integer number showing the index for which the reward is coming from
- class ReplayBuffer(*args, **kwargs)[source]
Bases:
amber.architect.buffer.Buffer
DOCSTRING : TODO
- get_buffer(arg)[source]
Getter method for getting a buffer class from a string
- Parameters
arg (str or callable) – return the buffer constructor corresponding to that identifier; if is callable, assume it’s a buffer constructor already, do nothing and return it
- Returns
A callable buffer constructor
- Return type
amber.architect.buffer
- Raises
ValueError – If the given
arg
is not string nor callable, orarg
is string but is not understood/implemented
Reward
Reward function for processing a train child model
Reward function takes in trained model and validation data as input, returns reward
- class KnowledgeReward(knowledge_function, Lambda, loss_c=None, knowledge_c=None)[source]
Bases:
amber.architect.reward.Reward
DOCSTRING: To be added after it’s thoroughly tested
- Parameters
knowledge_function
Lambda
loss_c
knowledge_c
- class LossAucReward(method='auc', knowledge_function=None, Lambda=1, loss_c=None, knowledge_c=None, *args, **kwargs)[source]
Bases:
amber.architect.reward.Reward
Reward function for evaluating AUC as reward
This reward function employs a scorer to evaluate a trained model’s prediction. Scorers can be parsed as strings for AUROC and AUPR; or a callable function that takes
y_true
andy_score
as input arguments.In the case of multi-tasking, the average scorer outputs across all tasks will be returned as the reward.
- Parameters
method (str or callable) – The method to use to implement scorer. If is string, expects either “auc”/”auroc” or “aupr”/”auprc”. Otherwise must be callable (see details in Example). Default is “auc”.
knowledge_function (amber.objective) – The augmented rewards for considering additional biological knowledge and how consistent the trained model is. Default is None.
Lambda (float) – The weight of augmented knowledge reward, compared to regular validation data-based loss as 1.
loss_c (float or None) – If is None, return the negative loss as is. A constant to scale loss reward, if specified. Default is None.
knowledge_c (float or None) – If is None, return the knowledge reward as is. A constant to scale knowledge reward, if specified. Default is None.
Examples
Constructing the reward function for AUPR on the basis of validation data:
>>> from amber.architect.reward import LossAucReard >>> reward_fn = LossAucReward(method="aupr") >>> reward_fn(model, val_data)
Alternatively, use the spearman correlation instead of AUPR for regression tasks:
>>> from amber.architect.reward import LossAucReard >>> import scipy.stats as ss >>> reward_fn = LossAucReward(method=lambda y_true, y_score: ss.spearmanr(y_true, y_score).correlation) >>> reward_fn(model, val_data)
- class LossReward(validation_steps=None, *args, **kwargs)[source]
Bases:
amber.architect.reward.Reward
The most basic reward function; returns negative loss as rewards
- MockReward(train_history_list, metric, stringify_states, metric_name_dict, Lambda=1.0)[source]
MockReward is a resampler from a train history
MockReward returns a callable reward function that works similarly to a real one : takes in an architecture sequence, and return its reward signals. However, the rewards are all pre-computed in the train history files instead of training from scratch, which is useful for fast benchmarking of architecture searching algorithms
- Parameters
train_history_list (list of strings) – A list of file paths to train history files
metric (list or set) – A specified subset of metrices to return
stringify_states (bool) – If true, will expect the architecture sequence to be categorically encoded and use it to stringify according to the given model space.
metric_name_dict (dict) – A dictionary mapping metric name (keys) to their indices (values)
Lambda (float) – The weight for augmented reward, compared to the loss-derived reward on the basis of validation data has a weight of 1.
- Returns
reward_fn – A callable reward function
- Return type
callable
See also
amber.architect.reward.LossReward
A regular loss-based reward function
Store
Store functions will take care of storage and post-processing related matters after a child model is trained.
- get_store_fn(arg)[source]
The getter function that returns a callable store function from a string
- Parameters
arg (str) – The string identifier for a particular store function. Current choices are: - general - model_plot - minimal
- Returns
A callable store function
- Return type
callable
- store_general(trial, model, hist, data, pred, loss_and_metrics, working_dir='.', save_full_model=False, *args, **kwargs)[source]
- store_with_hessian(trial, model, hist, data, pred, loss_and_metrics, working_dir='.', save_full_model=False, knowledge_func=None)[source]
Common Operations
- get_kl_divergence_n_entropy(curr_prediction, curr_onehot, old_prediction, old_onehotpred)[source]
compute approx return kl, ent