amber.architect

Rationale

This module amber.architect can be devided into four major components. It’s easy to imagine a model space to sample models from, and a training environment as a placeholder for different components to interact. Currently the model space is pre-defined, but we can make it dynamic and evolving in the future.

The remaining two components do the most of the heavy-lifting. Search algorithms, such as a controller recurrent neural network vs a genetic algorithm, are separated by sub-packages (a folder with __init__.py) and each variant of a search algorithm is a separate file. This makes it easy to re-use code within a search algorithm, relatively independent across different searchers, and let them share the same configurations of model space and training environments (although it’s possible only certain combinations of searcher - model space - train env are viable).

The manager is perhaps the most tedious. The general workflow is:

1. takes a model architecture arch as input,

2. passes arch to the modeler model=amber.modeler.model_fn(arch),

3. trains the model model.fit(train_data),

4. evaluates the model’s reward reward=reward_fn(model),

5. stores the model in a buffer buffer_fn.store(model, reward),

6. returns the reward signal to the search algorithm.

Each of the steps has variantions, but the overall layout should almost always stay as described above.

Model Space

Model space to perform architecture search

class BranchedModelSpace(subspaces, concat_op='concatenate', **kwargs)

Bases: amber.architect.modelSpace.ModelSpace

Parameters

• subspaces (list) – A list of ModelSpace. First element is a list of input branches. Second element is a stem model space

• concat_op (str) – string identifier for how to concatenate different input branches

property branch_to_layer

property layer_to_branch

class ModelSpace(**kwargs)

Bases: object

Model Space constructor

Provides utility functions for holding “states” / “operations” that the controller must use to train and predict. Also provides a more convenient way to define the model search space

There are several ways to construct a model space. For example, one way is to initialize an empty ModelSpace then iteratively add layers to it, where each layer has a number of candidate operations:

>>> def get_model_space(out_filters=64, num_layers=9):
>>>    model_space = ModelSpace()
>>>    num_pool = 4
>>>    expand_layers = [num_layers//num_pool*i-1 for i in range(1, num_pool)]
>>>    for i in range(num_layers):
>>>        model_space.add_layer(i, [
>>>            Operation('conv1d', filters=out_filters, kernel_size=8, activation='relu'),
>>>            Operation('conv1d', filters=out_filters, kernel_size=4, activation='relu'),
>>>            Operation('maxpool1d', filters=out_filters, pool_size=4, strides=1),
>>>            Operation('avgpool1d', filters=out_filters, pool_size=4, strides=1),
>>>            Operation('identity', filters=out_filters),
>>>      ])
>>>        if i in expand_layers:
>>>            out_filters *= 2
>>>    return model_space

Alternatively, ModelSpace can also be constructed from a dictionary.

add_layer(layer_id, layer_states=None)

Add a new layer to model space

Parameters

• layer_id (int) – The layer id of which layer to be added. Can be incontinuous to previous layers.

• layer_states (list of amber.architect.Operation) – A list of Operation object to be added.

Returns

Boolean value of Whether the model space is valid after inserting this layer

Return type

bool

add_state(layer_id, state)

Append a new state/operation to a layer

Parameters

• layer_id (int) – Which layer to append a new operation.

• state (amber.architect.State) – The new operation object to be appended.

delete_layer(layer_id)

Delete an entire layer and its associated values

Parameters

layer_id (int) – which layer index to be deleted

Returns

Boolean value of Whether the model space is valid after inserting this layer

Return type

bool

delete_state(layer_id, state_id)

Delete an operation from layer

Parameters

• layer_id (int) – Which layer to delete an operation

• state_id (int) – Which operation index to be deleted

static from_dict(d)

Static method for creating a ModelSpace from a Dictionary or List

Parameters

d (dict or list) – A dictionary or list specifying candidate operations for each layer

Returns

The constructed model space from the given dict/list

Return type

amber.architect.ModelSpace

get_random_model_states()

Get a random combination of model operations throughout each layer

Returns

model_states – A list of randomly sampled model operations

Return type

list

get_space_size()

Get the total model space size by the product of all candidate operations across all layers. No residual connections are considered.

Returns

size – The total number of possible combinations of operations.

Return type

int

print_state_space()

print out the model space in a nice layout (not so nice yet)

Operation

alias of amber.architect.modelSpace.State

class State(Layer_type, **kwargs)

Bases: object

The Amber internal holder for a computational operation at any layer

Parameters

• Layer_type (str) – The string for the operation type; supports most commonly used tf.keras.layers types

• kwargs – Operation/layer specifications are parsed through keyword arguments

Variables

• Layer_type (str) – The string for the operation type.

• Layer_attributes (dict) – The dictionary that holds key-value pairs for all specification for this layer.

Notes

Any attributes that are not specified in Layer_attributes will use the default value as defined in tf.keras.layers. For example, if you do not specify activation in Layer_attributes, it will use linear.

Examples

For example, to create a 1D-convolutional operation with ReLU activation, kernel size=8, number of kernels=32:

>>> from amber.architect import State
>>> op = State("conv1d", filters=32, kernel_size=8, activation='relu')

get_layer_shortname(layer)

Get the short name for a computational operation of a layer, useful in converting a Layer object to a string as ID or when plotting

Parameters

layer (amber.architect.Operation) – The Operation object for any layer.

Returns

sn – The unique short name for this operation

Return type

str

Todo

Consider refactoring layer to operation

Train Environment

Training environment provides interactions between several components within architect and outside.

class ControllerTrainEnvironment(controller, manager, max_episode=100, max_step_per_ep=2, logger=None, resume_prev_run=False, should_plot=True, initial_buffering_queue=15, working_dir='.', entropy_converge_epsilon=0.01, squeezed_action=True, with_input_blocks=False, with_skip_connection=True, save_controller=True, continuous_run=False, verbose=0, **kwargs)

Bases: object

The training evnrionment employs controller model and manager to mange data and reward, creates a reinforcement learning environment

Parameters

• controller (amber.architect.BaseController) – The controller to search architectures in this environment.

• manager (amber.architect.BaseManager) – The manager to interact with modelers in this environment.

• max_episode (int) – Maximum number of controller steps. Each controller step will sample max_step_per_ep child model architectures.

• max_step_per_ep (int) – The number of child model architectures to sample in each controller step (aka max_episode).

• logger (logging.Logger or None) – The logger to use in environment

• resume_prev_run (bool) – If true, try to reload existing controller and history in the given working directory. Default is False.

• should_plot (bool) – If false, turn off the plots at the end of training. Default is False.

• initial_buffering_queue (int)

working_dirstr

File path to working directory

squeezed_actionbool

If false, will expect each entry in architecture sequence to be strictly one-hot encoded; if true, some entries will be categorically encoded. Categorical encoding is consistent with current settings in amber.architect.GeneralController, thus is recommended squeezed_action=True. Default is True.

with_input_blocksbool

Whether the controller will search for input blocks. Default is False.

with_skip_connectionbool

Whether the controller will search for residual/skip connections. Default is True.

save_controllerbool

Whether to save the final controller parameters in working directory after training (i.e. reaching the max controller steps). Default is True.

verbosebool or int

Verbosity level

Notes

Note if either with_input_blocks or with_skip_connection, a list of integers will be used to represent the

sequential model architecture and wiring, instead of a list of amber.architect.Operation

Todo

Refactor the rest of attributes to be private.

clean()

reset()

restore()

step(action_prob)

train()

Performs training for controller

class EnasTrainEnv(*args, **kwargs)

Bases: amber.architect.trainEnv.ControllerTrainEnvironment

Params:

time_budget: defaults to 72 hours

train()

class MultiManagerEnvironment(data_descriptive_features, is_enas='auto', *args, **kwargs)

Bases: amber.architect.trainEnv.EnasTrainEnv

MultiManagerEnvironment is an environment that allows one controller to interact with multiple EnasManagers

train()

class ParallelMultiManagerEnvironment(processes=2, enable_manager_sampling=False, *args, **kwargs)

Bases: amber.architect.trainEnv.MultiManagerEnvironment

restore()

compute_entropy(prob_states)

get_controller_history(fn='train_history.csv')

get_controller_states(model)

set_controller_states(model, states)

Controller(s)

Implementations of NAS controller for searching architectures

Changelog

• Aug. 7, 2018: initial

• Feb. 6. 2019: finished initial OperationController

• Jun. 17. 2019: separated to OperationController and GeneralController

• Aug. 15, 2020: updated documentations

class BaseController(*args, **kwargs)

Bases: object

abstract class for controllers

remove_files(*args, **kwargs)

store(*args, **kwargs)

train(*args, **kwargs)

class GeneralController(model_space, buffer_type='ordinal', with_skip_connection=True, share_embedding=None, use_ppo_loss=False, kl_threshold=0.05, skip_connection_unique_connection=False, buffer_size=15, batch_size=5, session=None, train_pi_iter=20, lstm_size=32, lstm_num_layers=2, lstm_keep_prob=1.0, tanh_constant=None, temperature=None, optim_algo='adam', skip_target=0.8, skip_weight=None, rescale_advantage_by_reward=False, name='controller', verbose=0, **kwargs)

Bases: amber.architect.controller.generalController.BaseController

GeneralController for neural architecture search

This class searches for two steps:

• computational operations for each layer

• skip connections for each layer from all previous layers [optional]

It is a modified version of enas: https://github.com/melodyguan/enas . Notable modifications include: dissection of sampling and training processes to enable better understanding of controller behaviors, buffering and logging; loss function can be optimized by either REINFORCE or PPO.

Todo

Refactor the rest of the attributes to private.

Parameters

• model_space (amber.architect.ModelSpace) – A ModelSpace object constructed to perform architecture search for.

• with_skip_connection (bool) – If false, will not search residual connections and only search for computation operations per layer. Default is True.

• share_embedding (dict) – a Dictionary defining which child-net layers will share the softmax and embedding weights during Controller training and sampling. For example, {1:0, 2:0} means layer 1 and 2 will share the embedding with layer 0.

• use_ppo_loss (bool) – If true, use PPO loss for optimization instead of REINFORCE. Default is False.

• kl_threshold (float) – If KL-divergence between the sampling probabilities of updated controller parameters and that of original parameters exceeds kl_threshold within a single controller training step, triggers early-stopping to halt the controller training. Default is 0.05.

• buffer_size (int) – amber.architect.Buffer stores only the sampled architectures from the last buffer_size number of from previous controller steps, where each step has a number of sampled architectures as specified in amber.architect.ControllerTrainEnv.

• batch_size (int) – How many architectures in a batch to train the controller

• session (tf.Session) – The session where the controller tensors is placed

• train_pi_iter (int) – The number of epochs/iterations to train controller policy in one controller step.

• lstm_size (int) – The size of hidden units for stacked LSTM, i.e. controller RNN.

• lstm_num_layers (int) – The number of stacked layers for stacked LSTM, i.e. controller RNN.

• lstm_keep_prob (float) – keep_prob = 1 - dropout probability for stacked LSTM.

• tanh_constant (float) – If not None, the logits for each multivariate classification will be transformed by tf.tanh then multiplied by tanh_constant. This can avoid over-confident controllers asserting probability=1 or 0 caused by logit going to +/- inf. Default is None.

• temperature (float) – The temperature is a scale factor to logits. Higher temperature will flatten the probabilities among different classes, while lower temperature will freeze them. Default is None, i.e. 1.

• optim_algo (str) – Optimizer for controller RNN. Can choose from [“adam”, “sgd”, “rmsprop”]. Default is “adam”.

• skip_target (float) – The expected proportion of skip connections, i.e. the proportion of 1’s in the skip/extra connections in the output arc_seq

• skip_weight (float) – The weight for skip connection kl-divergence from the expected skip_target

• name (str) – The name for this Controller instance; all tf.Tensors will be placed under this VariableScope. This name determines which tensors will be initialized when a new Controller instance is created.

Variables

• weights (list of tf.Variable) – The list of all trainable tf.Variable in this controller

• model_space (amber.architect.ModelSpace) – The model space which the controller will be searching from.

• buffer (amber.architect.Buffer) – The Buffer object stores the history architectures, computes the rewards, and gets feed dict for training.

• session (tf.Session) – The reference to the session that hosts this controller instance.

static convert_arc_to_onehot(controller)

Convert a categorical architecture sequence to a one-hot encoded architecture sequence

Parameters

controller (amber.architect.controller) – An instance of controller

Returns

onehot_list – a one-hot encoded architecture sequence

Return type

list

get_action(*args, **kwargs)

Get a sampled architecture/action and its corresponding probabilities give current controller policy parameters.

The generated architecture is the out-going information from controller to manager. which in turn will feedback the reward signal for storage and training by the controller.

Parameters

None

Returns

onehots –

The sampled architecture sequence. In particular, the architecture sequence is ordered as:

[categorical_operation_0,
categorical_operation_1, binary_skip_0,
categorical_operation_2, binary_skip_0, binary_skip_1,
...]

Return type

list

probslist of ndarray

The probabilities associated with each sampled operation and residual connection. Shapes will vary depending on each layer’s specification in ModelSpace for operation, and the layer number for residual connections.

get_weights(**kwargs)

Get the current controller weights in a numpy array

Parameters

None

Returns

weights – A list of numpy array for each weights in controller

Return type

list

load_weights(filepath, **kwargs)

Load the controller weights from a hdf5 file

Parameters

filepath (str) – file path to saved weights

Return type

None

static remove_files(files, working_dir='.')

Static method for removing files

Parameters

• files (list of str) – files to be removed

• working_dir (str) – filepath to working directory

Return type

None

save_weights(filepath, **kwargs)

Save current controller weights to a hdf5 file

Parameters

filepath (str) – file path to save the weights

Return type

None

set_weights(weights, **kwargs)

Set the current controller weights

Parameters

weights (list of numpy.ndarray) – A list of numpy array for each weights in controller

Return type

None

store(state, prob, action, reward, *args, **kwargs)

Store all necessary information and rewards for a given architecture

This is the receiving method for controller to interact with manager by storing the rewards for a given architecture. The architecture and its probabilities can be generated by get_action() method.

Parameters

• state (list) – The state for which the action and probabilities are drawn.

• prob (list of ndarray) – A list of probabilities for each operation and skip connections.

• action (list) –

  A list of architecture tokens ordered as:

  [categorical_operation_0,
  categorical_operation_1, binary_skip_0,
  categorical_operation_2, binary_skip_0, binary_skip_1,
  ...]
  

• reward (float) – Reward for this architecture, as evaluated by amber.architect.manager

Return type

None

train(episode, working_dir)

Train the controller policy parameters for one step.

Parameters

• episode (int) – Total number of epochs to train the controller. Each epoch will iterate over all architectures stored in buffer.

• working_dir (str) – Filepath to working directory to store (possible) intermediate results

Returns

aloss – Average controller loss for this train step

Return type

float

Notes

Consider renaming this method to train_step() to better reflect its function, and avoid confusion with the training function in environment ControllerTrainEnv.train()

class MultiIOController(num_output_blocks=2, with_output_blocks=True, output_block_unique_connection=True, output_block_diversity_weight=None, **kwargs)

Bases: amber.architect.controller.multiioController.MultiInputController

Example

from amber.bootstrap.dense_skipcon_space import get_model_space
from amber.architect.controller.multiioController import MultiIOController
import numpy as np
import tensorflow as tf
s = tf.Session()
model_space = get_model_space(5)
controller = MultiIOController(model_space=model_space, output_block_unique_connection=True, session=s)
a1, p1 = controller.get_action()
a2, p2 = controller.get_action()
a_batch = np.array([a1,a2])
p_batch = [np.concatenate(x) for x in zip(*[p1,p2])]
feed_dict = {controller.input_arc[i]: a_batch[:, [i]]
             for i in range(a_batch.shape[1])}
feed_dict.update({controller.advantage: np.array([1., -1.]).reshape((2,1))})
feed_dict.update({controller.old_probs[i]: p_batch[i]
                  for i in range(len(controller.old_probs))})
feed_dict.update({controller.reward: np.array([1., 1.]).reshape((2,1))})
print(s.run(controller.onehot_log_prob, feed_dict))
for _ in range(100):
    s.run(controller.train_op, feed_dict=feed_dict)
    if _%20==0: print(s.run(controller.loss, feed_dict))
print(s.run(controller.onehot_log_prob, feed_dict))

Notes

Placeholder for now.

class MultiInputController(model_space, buffer_type='ordinal', with_skip_connection=True, with_input_blocks=True, share_embedding=None, use_ppo_loss=False, kl_threshold=0.05, num_input_blocks=2, input_block_unique_connection=True, skip_connection_unique_connection=False, buffer_size=15, batch_size=5, session=None, train_pi_iter=20, lstm_size=32, lstm_num_layers=2, lstm_keep_prob=1.0, tanh_constant=None, temperature=None, skip_target=0.8, skip_weight=0.5, optim_algo='adam', name='controller', *args, **kwargs)

Bases: amber.architect.controller.generalController.GeneralController

DOCSTRING

Parameters

• model_space

• with_skip_connection

• with_input_blocks

• share_embedding (dict) – a Dictionary defining which child-net layers will share the softmax and embedding weights during Controller training and sampling

• use_ppo_loss

• kl_threshold

• num_input_blocks

• input_block_unique_connection

• buffer_size

• batch_size

• session

• train_pi_iter

• lstm_size

• lstm_num_layers

• lstm_keep_prob

• tanh_constant

• temperature

• lr_init

• lr_dec_start

• lr_dec_every

• lr_dec_rate

• l2_reg

• clip_mode

• grad_bound

• use_critic

• bl_dec

• optim_algo

• sync_replicas

• num_aggregate

• num_replicas

• skip_target (float) – the expected proportion of skip connections, i.e. the proportion of 1’s in the skip/extra connections in the output arc_seq

• skip_weight (float) – the weight for skip connection kl-divergence from the expected skip_target

• name (str) – name for the instance; also used for all tensor variable scopes

Variables

• g_emb (tf.Tensor) – initial controller hidden state tensor; to be learned

• Placeholder –

Note

This class derived from GeneralController adds the input feature block selection upon the Base class. Since the selection is inherently embedded in the NAS cell rolling-out, the sampler and trainer methods are overwritten.

TODO:

needs to evaluate how different ways of connecting inputs will affect search performance; e.g. connect input before operation or after?

class OperationController(state_space, controller_units, embedding_dim=5, optimizer='rmsprop', discount_factor=0.0, clip_val=0.2, beta=0.001, kl_threshold=0.05, train_pi_iter=100, lr_pi=0.005, buffer_size=50, batch_size=5, verbose=0)

Bases: amber.architect.controller.generalController.BaseController

example state_space for a 2-layer conv-net:

state_space = [[‘conv3’, ‘conv5’, ‘conv7’], [‘maxp2’, ‘avgp2’],

[‘conv3’, ‘conv5’, ‘conv7’], [‘maxp2’, ‘avgp2’]]

get_action(seed)

remove_files(files, working_dir='.')

store(state, prob, action, reward)

train(episode, working_dir)

called only when path finishes

class ZeroShotController(data_description_config, *args, **kwargs)

Bases: amber.architect.controller.generalController.GeneralController

get_action(description_feature, *args, **kwargs)

store(prob, action, reward, description, manager_index, *args, **kwargs)

train(episode, working_dir)

General Controller

General controller for searching computational operation per layer, and residual connection

class BaseController(*args, **kwargs)

Bases: object

abstract class for controllers

remove_files(*args, **kwargs)

store(*args, **kwargs)

train(*args, **kwargs)

class GeneralController(model_space, buffer_type='ordinal', with_skip_connection=True, share_embedding=None, use_ppo_loss=False, kl_threshold=0.05, skip_connection_unique_connection=False, buffer_size=15, batch_size=5, session=None, train_pi_iter=20, lstm_size=32, lstm_num_layers=2, lstm_keep_prob=1.0, tanh_constant=None, temperature=None, optim_algo='adam', skip_target=0.8, skip_weight=None, rescale_advantage_by_reward=False, name='controller', verbose=0, **kwargs)

Bases: amber.architect.controller.generalController.BaseController

GeneralController for neural architecture search

This class searches for two steps:

• computational operations for each layer

• skip connections for each layer from all previous layers [optional]

It is a modified version of enas: https://github.com/melodyguan/enas . Notable modifications include: dissection of sampling and training processes to enable better understanding of controller behaviors, buffering and logging; loss function can be optimized by either REINFORCE or PPO.

Todo

Refactor the rest of the attributes to private.

Parameters

• model_space (amber.architect.ModelSpace) – A ModelSpace object constructed to perform architecture search for.

• with_skip_connection (bool) – If false, will not search residual connections and only search for computation operations per layer. Default is True.

• share_embedding (dict) – a Dictionary defining which child-net layers will share the softmax and embedding weights during Controller training and sampling. For example, {1:0, 2:0} means layer 1 and 2 will share the embedding with layer 0.

• use_ppo_loss (bool) – If true, use PPO loss for optimization instead of REINFORCE. Default is False.

• kl_threshold (float) – If KL-divergence between the sampling probabilities of updated controller parameters and that of original parameters exceeds kl_threshold within a single controller training step, triggers early-stopping to halt the controller training. Default is 0.05.

• buffer_size (int) – amber.architect.Buffer stores only the sampled architectures from the last buffer_size number of from previous controller steps, where each step has a number of sampled architectures as specified in amber.architect.ControllerTrainEnv.

• batch_size (int) – How many architectures in a batch to train the controller

• session (tf.Session) – The session where the controller tensors is placed

• train_pi_iter (int) – The number of epochs/iterations to train controller policy in one controller step.

• lstm_size (int) – The size of hidden units for stacked LSTM, i.e. controller RNN.

• lstm_num_layers (int) – The number of stacked layers for stacked LSTM, i.e. controller RNN.

• lstm_keep_prob (float) – keep_prob = 1 - dropout probability for stacked LSTM.

• tanh_constant (float) – If not None, the logits for each multivariate classification will be transformed by tf.tanh then multiplied by tanh_constant. This can avoid over-confident controllers asserting probability=1 or 0 caused by logit going to +/- inf. Default is None.

• temperature (float) – The temperature is a scale factor to logits. Higher temperature will flatten the probabilities among different classes, while lower temperature will freeze them. Default is None, i.e. 1.

• optim_algo (str) – Optimizer for controller RNN. Can choose from [“adam”, “sgd”, “rmsprop”]. Default is “adam”.

• skip_target (float) – The expected proportion of skip connections, i.e. the proportion of 1’s in the skip/extra connections in the output arc_seq

• skip_weight (float) – The weight for skip connection kl-divergence from the expected skip_target

• name (str) – The name for this Controller instance; all tf.Tensors will be placed under this VariableScope. This name determines which tensors will be initialized when a new Controller instance is created.

Variables

• weights (list of tf.Variable) – The list of all trainable tf.Variable in this controller

• model_space (amber.architect.ModelSpace) – The model space which the controller will be searching from.

• buffer (amber.architect.Buffer) – The Buffer object stores the history architectures, computes the rewards, and gets feed dict for training.

• session (tf.Session) – The reference to the session that hosts this controller instance.

static convert_arc_to_onehot(controller)

Convert a categorical architecture sequence to a one-hot encoded architecture sequence

Parameters

controller (amber.architect.controller) – An instance of controller

Returns

onehot_list – a one-hot encoded architecture sequence

Return type

list

get_action(*args, **kwargs)

Get a sampled architecture/action and its corresponding probabilities give current controller policy parameters.

The generated architecture is the out-going information from controller to manager. which in turn will feedback the reward signal for storage and training by the controller.

Parameters

None

Returns

onehots –

The sampled architecture sequence. In particular, the architecture sequence is ordered as:

[categorical_operation_0,
categorical_operation_1, binary_skip_0,
categorical_operation_2, binary_skip_0, binary_skip_1,
...]

Return type

list

probslist of ndarray

The probabilities associated with each sampled operation and residual connection. Shapes will vary depending on each layer’s specification in ModelSpace for operation, and the layer number for residual connections.

get_weights(**kwargs)

Get the current controller weights in a numpy array

Parameters

None

Returns

weights – A list of numpy array for each weights in controller

Return type

list

load_weights(filepath, **kwargs)

Load the controller weights from a hdf5 file

Parameters

filepath (str) – file path to saved weights

Return type

None

static remove_files(files, working_dir='.')

Static method for removing files

Parameters

• files (list of str) – files to be removed

• working_dir (str) – filepath to working directory

Return type

None

save_weights(filepath, **kwargs)

Save current controller weights to a hdf5 file

Parameters

filepath (str) – file path to save the weights

Return type

None

set_weights(weights, **kwargs)

Set the current controller weights

Parameters

weights (list of numpy.ndarray) – A list of numpy array for each weights in controller

Return type

None

store(state, prob, action, reward, *args, **kwargs)

Store all necessary information and rewards for a given architecture

This is the receiving method for controller to interact with manager by storing the rewards for a given architecture. The architecture and its probabilities can be generated by get_action() method.

Parameters

• state (list) – The state for which the action and probabilities are drawn.

• prob (list of ndarray) – A list of probabilities for each operation and skip connections.

• action (list) –

  A list of architecture tokens ordered as:

  [categorical_operation_0,
  categorical_operation_1, binary_skip_0,
  categorical_operation_2, binary_skip_0, binary_skip_1,
  ...]
  

• reward (float) – Reward for this architecture, as evaluated by amber.architect.manager

Return type

None

train(episode, working_dir)

Train the controller policy parameters for one step.

Parameters

• episode (int) – Total number of epochs to train the controller. Each epoch will iterate over all architectures stored in buffer.

• working_dir (str) – Filepath to working directory to store (possible) intermediate results

Returns

aloss – Average controller loss for this train step

Return type

float

Notes

Consider renaming this method to train_step() to better reflect its function, and avoid confusion with the training function in environment ControllerTrainEnv.train()

MultiIO Controller

class MultiIOController(num_output_blocks=2, with_output_blocks=True, output_block_unique_connection=True, output_block_diversity_weight=None, **kwargs)

Bases: amber.architect.controller.multiioController.MultiInputController

Example

from amber.bootstrap.dense_skipcon_space import get_model_space
from amber.architect.controller.multiioController import MultiIOController
import numpy as np
import tensorflow as tf
s = tf.Session()
model_space = get_model_space(5)
controller = MultiIOController(model_space=model_space, output_block_unique_connection=True, session=s)
a1, p1 = controller.get_action()
a2, p2 = controller.get_action()
a_batch = np.array([a1,a2])
p_batch = [np.concatenate(x) for x in zip(*[p1,p2])]
feed_dict = {controller.input_arc[i]: a_batch[:, [i]]
             for i in range(a_batch.shape[1])}
feed_dict.update({controller.advantage: np.array([1., -1.]).reshape((2,1))})
feed_dict.update({controller.old_probs[i]: p_batch[i]
                  for i in range(len(controller.old_probs))})
feed_dict.update({controller.reward: np.array([1., 1.]).reshape((2,1))})
print(s.run(controller.onehot_log_prob, feed_dict))
for _ in range(100):
    s.run(controller.train_op, feed_dict=feed_dict)
    if _%20==0: print(s.run(controller.loss, feed_dict))
print(s.run(controller.onehot_log_prob, feed_dict))

Notes

Placeholder for now.

class MultiInputController(model_space, buffer_type='ordinal', with_skip_connection=True, with_input_blocks=True, share_embedding=None, use_ppo_loss=False, kl_threshold=0.05, num_input_blocks=2, input_block_unique_connection=True, skip_connection_unique_connection=False, buffer_size=15, batch_size=5, session=None, train_pi_iter=20, lstm_size=32, lstm_num_layers=2, lstm_keep_prob=1.0, tanh_constant=None, temperature=None, skip_target=0.8, skip_weight=0.5, optim_algo='adam', name='controller', *args, **kwargs)

Bases: amber.architect.controller.generalController.GeneralController

DOCSTRING

Parameters

• model_space

• with_skip_connection

• with_input_blocks

• share_embedding (dict) – a Dictionary defining which child-net layers will share the softmax and embedding weights during Controller training and sampling

• use_ppo_loss

• kl_threshold

• num_input_blocks

• input_block_unique_connection

• buffer_size

• batch_size

• session

• train_pi_iter

• lstm_size

• lstm_num_layers

• lstm_keep_prob

• tanh_constant

• temperature

• lr_init

• lr_dec_start

• lr_dec_every

• lr_dec_rate

• l2_reg

• clip_mode

• grad_bound

• use_critic

• bl_dec

• optim_algo

• sync_replicas

• num_aggregate

• num_replicas

• skip_target (float) – the expected proportion of skip connections, i.e. the proportion of 1’s in the skip/extra connections in the output arc_seq

• skip_weight (float) – the weight for skip connection kl-divergence from the expected skip_target

• name (str) – name for the instance; also used for all tensor variable scopes

Variables

• g_emb (tf.Tensor) – initial controller hidden state tensor; to be learned

• Placeholder –

Note

This class derived from GeneralController adds the input feature block selection upon the Base class. Since the selection is inherently embedded in the NAS cell rolling-out, the sampler and trainer methods are overwritten.

TODO:

needs to evaluate how different ways of connecting inputs will affect search performance; e.g. connect input before operation or after?

Operation Controller

A pure Keras-implementation of NAS ZZJ Aug. 7, 2018

class OperationController(state_space, controller_units, embedding_dim=5, optimizer='rmsprop', discount_factor=0.0, clip_val=0.2, beta=0.001, kl_threshold=0.05, train_pi_iter=100, lr_pi=0.005, buffer_size=50, batch_size=5, verbose=0)

Bases: amber.architect.controller.generalController.BaseController

example state_space for a 2-layer conv-net:

state_space = [[‘conv3’, ‘conv5’, ‘conv7’], [‘maxp2’, ‘avgp2’],

[‘conv3’, ‘conv5’, ‘conv7’], [‘maxp2’, ‘avgp2’]]

get_action(seed)

remove_files(files, working_dir='.')

store(state, prob, action, reward)

train(episode, working_dir)

called only when path finishes

build_actor(inputs, rnn_units, input_dim, maxlen, state_space, scope, trainable=True)

get_indexer(t)

get_optimizer(name: str, lr: float, decay: float = 0.995)

one_hot_encoder(val, num_classes, offset=0)

parse_action_str(action_onehot, state_space)

Zero-Shot Controller (AMBIENT)

Zero-shot controller takes in descriptive features from dataset when calling .get_action() method, then generate new architectures for new tasks using a trained controller based on the data-descriptive features for the new task.

ZZ, May 13, 2020

class ZeroShotController(data_description_config, *args, **kwargs)

Bases: amber.architect.controller.generalController.GeneralController

get_action(description_feature, *args, **kwargs)

store(prob, action, reward, description, manager_index, *args, **kwargs)

train(episode, working_dir)

Manager

Manager class for streamlining downstream build and evaluation given an architecture.

Manager is the class that takes in architecture designs from an architecture search/optimization algorithm, then interacts with amber.modeler to build and train the model according to architecture, and finally calls amber.architect.rewards to evaluate the trained model rewards to feedback the architecture designer.

class BaseNetworkManager(*args, **kwargs)

Bases: object

get_rewards(trial, model_arc)

class DistributedGeneralManager(devices, train_data_kwargs, validate_data_kwargs, do_resample=False, *args, **kwargs)

Bases: amber.architect.manager.GeneralManager

Distributed manager will place all tensors of any child models to a pre-assigned GPU device

close_handler()

get_rewards(trial, model_arc, remap_device=None, **kwargs)

class GeneralManager(train_data, validation_data, model_fn, reward_fn, store_fn, working_dir='.', save_full_model=False, epochs=5, child_batchsize=128, verbose=0, fit_kwargs=None, predict_kwargs=None, evaluate_kwargs=None, **kwargs)

Bases: amber.architect.manager.BaseNetworkManager

Manager creates child networks, train them on a dataset, and retrieve rewards.

Parameters

• train_data (tuple, string or generator) – Training data to be fed to keras.models.Model.fit.

• validation_data (tuple, string, or generator) – Validation data. The data format is understood similarly to train_data.

• model_fn (amber.modeler) – A callable function to build and implement child models given an architecture sequence.

• reward_fn (amber.architect.rewards) – A callable function to evaluate the rewards on a trained model and the validation dataset.

• store_fn (amber.architect.store) – A callable function to store necessary information (such as predictions, model architectures, and a variety of plots etc.) for the given child model.

• working_dir (str) – File path for working directory.

• save_full_model (bool) – If true, save the full model beside the model weights. Default is False.

• epochs (int) – The total number of epochs to train the child model.

• child_batchsize (int) – The batch size for training the child model.

• fit_kwargs (dict or None) – Keyword arguments for model.fit

• predict_kwargs (dict or None) – Keyword arguments for model.predict

• evaluate_kwargs (dict or None) – Keyword arguments for model.evaluate

• verbose (bool or int) – Verbose level. 0=non-verbose, 1=verbose, 2=less verbose.

• kwargs (dict) – Other keyword arguments parsed.

Variables

• train_data (tuple or generator) – The unpacked training data

• validation_data (tuple or generator) – The unpacked validation data

• model_fn (amber.modeler) – Reference to the callable function to build and implement child models given an architecture sequence.

• reward_fn (amber.architect.rewards) – Reference to the callable function to evaluate the rewards on a trained model and the validation dataset.

• store_fn (amber.architect.store) – Reference to the callable function to store necessary information (such as predictions, model architectures, and a variety of plots etc.) for the given child model.

• working_dir (str) – File path to working directory

• verbose (bool or int) – Verbose level

Todo

• Refactor the rest of attributes as private.

• Update the description of train_data and validation_data to more flexible unpacking, once it’s added:

  If it's tuple, expects it to be a tuple of numpy.array of
  (x,y); if it's string, expects it to be the file path to a compiled training data; if it's a generator, expects
  it yield a batch of training features and samples.
  

get_rewards(trial, model_arc, **kwargs)

The reward getter for a given model architecture

Parameters

• trial (int) – An integer number indicating the trial for this architecture

• model_arc (list) – The list of architecture sequence

Returns

• this_reward (float) – The reward signal as determined by reward_fn(model, val_data)

• loss_and_metrics (dict) – A dictionary of auxillary information for this model, such as loss, and other metrics (as in tf.keras.metrics)

Buffer

Buffer class for holding reinforcement-learning histories

Buffer must have the following functionality:

• store a new incoming architecture and its associated reward and probability

• get data from the stored buffers

class Buffer(max_size, discount_factor=0.0, ewa_beta=None, is_squeeze_dim=False, rescale_advantage_by_reward=False, clip_advantage=10.0)

Bases: object

The ordinal buffer class

Buffer stores the sampled architectures, computed bias-adjusted rewards/advantages, and make feed-dict related through get_data method.

The main attributes of Buffer can be divided into short-term and long-term: short-term buffers are directly appending new entry to the lists; while long-term buffers will concatenate short-term buffers, perform temporal discounts and/or adjust for baseline rewards.

An entire short-term buffer will correspond to one entry in long-term buffer.

Short-term buffers will be emptied when converted to long-term, and long-term buffers will be kept at a maximum length specified by max_size argument.

Parameters

• max_size (int) – The maximum number of controller steps of sampled architectures to store. Stored data beyond the last max_size will be dumped to disk.

• discount_factor (float) – Temporal discount factor. Not used for now.

• ewa_beta (float) – The average is approx. over the past 1/(1-ewa_beta)

• is_squeeze_dim (bool) – If True, controller samples a sequence of tokens, instead of one-hot arrays. Default is False.

• rescale_advantage_by_reward (bool) – If True, the advantage will be divided by reward to rescale it as a proportion of increase/decrease of reward

• clip_advantage (float) – Clip the advantage to the value of clip_advantage if the advantage exceeds it. Default is 10.0.

Variables

• r_bias (float) – The moving baseline for reward; used to compute advantage = this_reward - r_bias.

• state_buffer (list) – The list of short-term states; each entry corresponds to one architecture generated by amber.architect.BaseController.get_action().

• action_buffer (list) – The list of short-term categorical-encoded architectures; each entry is a list, corresponding to one architecture generated by amber.architect.BaseController.get_action()

• prob_buffer (list) – The list of short-term probabilities; each entry is a list of numpy.ndarray, where each numpy.ndarray is probability.

• reward_buffer (list) – The list of short-term rewards.

• lt_sbuffer (list of numpy.array) – The list of long-term states; each entry is a numpy.array concatenated from several short-term buffer to facilitate batching in get_data().

• lt_abuffer (list of numpy.array) – The list of long-term architectures; each entry is a numpy.array concatenated from several short-term buffer architectures.

• lt_pbuffer (list of list of numpy.array) – The list of long-term probabilities; each entry is a list of numpy.array corresponding to several short-term buffer probabilities. Each numpy.array is concatenated probabilities of the same layer and type from several short-term entries.

• lt_adbuffer (list of list of floats) – The list of long-term advantages; each entry is floats of rewards corresponding to one short-term buffer.

• lt_rmbuffer (list of floats) – The list of reward mean for each short-term buffer.

• lt_nrbuffer (list of numpy.array) – The list of original rewards for each short-term buffer.

Todo

Refactor the rest of attributes to be private.

discount_rewards()

Discount rewards by temporal discount factor. This is not currently used in architecture searching.

Example

behavior of discounted reward, given reward_buffer=[1,2,3]:

if buffer.discount_factor=0.1:
    ([array([[1.23]]), array([[2.3]]), array([[3.]])], [1, 2, 3])
if buffer.discount_factor=0.9:
    ([array([[5.23]]), array([[4.7]]), array([[3.]])], [1, 2, 3])

finish_path(state_space, global_ep, working_dir, *args, **kwargs)

Finish path of short-term buffer and convert to long-term buffer

An entire short-term buffer will correspond to one entry in long-term buffer. After conversion, this will also dump buffers to file and empty the short-term buffer.

Parameters

• state_space (amber.architect.ModelSpace) – The model space to perform search on.

• global_ep (int) – The global number of controller steps

• working_dir (str) – File path to working directory.

get_data(bs, shuffle=True)

Get a batched data

size of buffer: (traj, traj_len, data_shape)

Parameters

• bs (int) – batch size

• shuffle (bool) – Randomly shuffle the index before yielding data. Default is True.

Yields

list – A list of batched data; entries are ordered as: [state, prob, arc_seq, advantages, rewards]. Each entry is a batch of data points as specified by bs.

store(state=None, prob=None, action=None, reward=None, *args, **kwargs)

class MultiManagerBuffer(max_size, ewa_beta=None, is_squeeze_dim=False, rescale_advantage_by_reward=False, clip_advantage=3.0, **kwargs)

Bases: object

dump_buffer(model_space, global_ep, working_dir)

finish_path(state_space, global_ep, working_dir, *args, **kwargs)

get_data(bs, shuffle=True)

property lt_abuffer

For legacy use

property lt_adbuffer

For legacy use

property lt_pbuffer

For legacy use

reset_short_term()

store(prob, action, reward, description, manager_index)

Parameters

• prob (list) – A list of np.array of different sizes corresponding to each layer

• action (list) – A list of np.array of one-hot actions

• reward (float) – Float number of reward to be maximized

• description (1D-array like) – A vector of data descriptor

• manager_index (int) – Integer number showing the index for which the reward is coming from

stratified_reward_mean()

Returns

for each reward (key), the average reward (value) computed in current short-term buffer

Return type

dict

class ReplayBuffer(*args, **kwargs)

Bases: amber.architect.buffer.Buffer

DOCSTRING : TODO

get_advantage()

get_data(bs, shuffle=True)

get_buffer(arg)

Getter method for getting a buffer class from a string

Parameters

arg (str or callable) – return the buffer constructor corresponding to that identifier; if is callable, assume it’s a buffer constructor already, do nothing and return it

Returns

A callable buffer constructor

Return type

amber.architect.buffer

Raises

ValueError – If the given arg is not string nor callable, or arg is string but is not understood/implemented

parse_action_str(action_onehot, state_space)

parse_action_str_squeezed(action_onehot, state_space)

Reward

Reward function for processing a train child model

Reward function takes in trained model and validation data as input, returns reward

class KnowledgeReward(knowledge_function, Lambda, loss_c=None, knowledge_c=None)

Bases: amber.architect.reward.Reward

DOCSTRING: To be added after it’s thoroughly tested

Parameters

• knowledge_function

• Lambda

• loss_c

• knowledge_c

class LossAucReward(method='auc', knowledge_function=None, Lambda=1, loss_c=None, knowledge_c=None, *args, **kwargs)

Bases: amber.architect.reward.Reward

Reward function for evaluating AUC as reward

This reward function employs a scorer to evaluate a trained model’s prediction. Scorers can be parsed as strings for AUROC and AUPR; or a callable function that takes y_true and y_score as input arguments.

In the case of multi-tasking, the average scorer outputs across all tasks will be returned as the reward.

Parameters

• method (str or callable) – The method to use to implement scorer. If is string, expects either “auc”/”auroc” or “aupr”/”auprc”. Otherwise must be callable (see details in Example). Default is “auc”.

• knowledge_function (amber.objective) – The augmented rewards for considering additional biological knowledge and how consistent the trained model is. Default is None.

• Lambda (float) – The weight of augmented knowledge reward, compared to regular validation data-based loss as 1.

• loss_c (float or None) – If is None, return the negative loss as is. A constant to scale loss reward, if specified. Default is None.

• knowledge_c (float or None) – If is None, return the knowledge reward as is. A constant to scale knowledge reward, if specified. Default is None.

Examples

Constructing the reward function for AUPR on the basis of validation data:

>>> from amber.architect.reward import LossAucReard
>>> reward_fn = LossAucReward(method="aupr")
>>> reward_fn(model, val_data)

Alternatively, use the spearman correlation instead of AUPR for regression tasks:

>>> from amber.architect.reward import LossAucReard
>>> import scipy.stats as ss
>>> reward_fn = LossAucReward(method=lambda y_true, y_score: ss.spearmanr(y_true, y_score).correlation)
>>> reward_fn(model, val_data)

min(data)

For dealing with non-valid model

class LossReward(validation_steps=None, *args, **kwargs)

Bases: amber.architect.reward.Reward

The most basic reward function; returns negative loss as rewards

MockReward(train_history_list, metric, stringify_states, metric_name_dict, Lambda=1.0)

MockReward is a resampler from a train history

MockReward returns a callable reward function that works similarly to a real one : takes in an architecture sequence, and return its reward signals. However, the rewards are all pre-computed in the train history files instead of training from scratch, which is useful for fast benchmarking of architecture searching algorithms

Parameters

• train_history_list (list of strings) – A list of file paths to train history files

• metric (list or set) – A specified subset of metrices to return

• stringify_states (bool) – If true, will expect the architecture sequence to be categorically encoded and use it to stringify according to the given model space.

• metric_name_dict (dict) – A dictionary mapping metric name (keys) to their indices (values)

• Lambda (float) – The weight for augmented reward, compared to the loss-derived reward on the basis of validation data has a weight of 1.

Returns

reward_fn – A callable reward function

Return type

callable

See also

amber.architect.reward.LossReward

A regular loss-based reward function

class Reward(*args, **kwargs)

Bases: object

class SparseCategoricalReward(method='auc', knowledge_function=None, Lambda=1, loss_c=None, knowledge_c=None, *args, **kwargs)

Bases: amber.architect.reward.LossAucReward

Store

Store functions will take care of storage and post-processing related matters after a child model is trained.

get_store_fn(arg)

The getter function that returns a callable store function from a string

Parameters

arg (str) – The string identifier for a particular store function. Current choices are: - general - model_plot - minimal

Returns

A callable store function

Return type

callable

store_general(trial, model, hist, data, pred, loss_and_metrics, working_dir='.', save_full_model=False, *args, **kwargs)

store_minimal(trial, model, working_dir='.', save_full_model=False, **kwargs)

store_with_hessian(trial, model, hist, data, pred, loss_and_metrics, working_dir='.', save_full_model=False, knowledge_func=None)

store_with_model_plot(trial, model, hist, data, pred, loss_and_metrics, working_dir='.', save_full_model=False, *args, **kwargs)

write_pred_to_disk(fn, y_pred, y_obs, metadata=None, metrics=None)

Common Operations

batch_norm1d(x, is_training, name='bn', decay=0.9, epsilon=1e-05, data_format='NWC')

batchify(x, y=None, batch_size=None, shuffle=True, drop_remainder=True)

count_model_params(tf_variables)

create_bias(name, shape, initializer=None)

create_weight(name, shape, initializer=None, trainable=True, seed=None)

get_keras_train_ops(loss, tf_variables, optim_algo, **kwargs)

get_kl_divergence_n_entropy(curr_prediction, curr_onehot, old_prediction, old_onehotpred)

compute approx return kl, ent

get_tf_layer(fn_str)

get_tf_loss(loss, y_true, y_pred)

get_tf_metrics(m)

lstm(x, prev_c, prev_h, w)

numpy_shuffle_in_unison(List)

proximal_policy_optimization_loss(curr_prediction, curr_onehot, old_prediction, old_onehotpred, rewards, advantage, clip_val, beta=None)

stack_lstm(x, prev_c, prev_h, w)

unpack_data(data, unroll_generator_x=False, unroll_generator_y=False, callable_kwargs=None)