CommandLine Interface (CLI)


Overall wrapper class for building an AMBER App.

class Amber(types, specs=None)[source]

Bases: object

The main wrapper class for AMBER

For most direct users, this is the main entry for running an AMBER model search. The purpose of this class can be summarized as follows:

  • provides a unified set of class methods for managing AMBER initialization, run, and storage

  • provides class attributes that reference to underlying AMBER components, for fine-tuning and debugging

  • provides the base class for future developments

The class can be initialized by passing two dictionaries together, types and specs, then an Amber instance will be constructed at initialization. Alternatively, if one just passed types, then only place holders will be built, and tensors can be constructed later using from_dict method.

This class facilitates the CLI and GUI caller, and should always be maintained.

  • types (dict) – The types dictionary specifies the type of controller, manager to be constructed. It should have the following keys: [‘controller_type’, ‘modeler_type’, ‘knowledge_fn_type’, ‘reward_fn_type’, ‘manager_type’, ‘env_type’].

  • specs (dict, or None) – Parameter specifications for each type. It can be different depending on different types of AMBER components being constructed. Most of the specification dictionary works by just copy-pasting from an example script, while it can also get very long and detailed for customized runs.

  • type_dict (dict) –

  • is_built (bool) –

  • session (tf.Session, or None) –

  • model_space (amber.architect.ModelSpace) –

  • controller (amber.architect.BaseController) –

  • manager (amber.architect.GeneralManager) –

  • env (amber.architect.ControllerTrainEnvironment) –



class AmberSpecifications(name, types=None)[source]

Bases: object

Creates a set of parameter specification for given AMBER component types

KEY_ATTR_MAP = {'controller': '_controller', 'knowledge_fn': '_knowledge_fn', 'manager': '_manager', 'model_builder': '_model_builder', 'model_space': '_model_space', 'reward_fn': '_reward_fn', 'train_env': '_train_env'}
property controller
property manager
property types


This script wraps around the AMBER package by serializing a string to an AMBER object. Useful in order to get a working TrainEnv from a configuration text file.

class DataToParse(obj, method=None, kws=None)[source]

Bases: object

A unified wrapper for reading and caching data files. Given a input file path, determines the file types by the file extensions, and uses corresponding data reader to unpack the data object, and return the connected file handler.

We need this because the configuration files only inputs a file path as string, and we need proper ways to handle the data files in different types.

Currently supports data in pickle, h5py, and numpy. File types are automatically determined in _extention(), but can also be overwrite by specifying method=STRING.

  • obj (str, or Callable) – a data object of flexible type. If str, expect it to be a filepath. If Callable, expect it to return a dataset when called with kws as keyword arguments. Other types are under development as needed.

  • method (str, or None) – method to unpack the data. If None, will try to automatically determine by _extension method.

  • kws (dict, or None) – keyword arguments to pass to a callable object


The class is often useful in constructing an Amber object, when passing irregular data files to a Manager:

from amber.getter import DataToParse
data = DataToParse("./data/example.npy")
arr = data.unpack()
# the same syntax also works if the file is h5py
data = DataToParse("./data/example.h5")
with data.unpack() as store:
    arr = store['X'][()]

It is also used internally in serializing an Amber object, e.g.

knowledge_data = DataToParse(var_dict['knowledge_data']).unpack()

returns a file handler connected to the data file

get_controller(controller_type, model_space, session, **kwargs)[source]

Serialize a controller of given type, using the specified model space, and a tf session if needed (can be None otherwise)

This step constructs the neural network architecture search algorithm, one of the two components in the final training environment.

The search algorithm takes two neccessary arguments as inputs: a string to specify what search algorithm to use, and a model space to sample model architectures from.

A tensorflow session may be passed to run training and inference, if the controller is implemented in Tensorflow.

  • controller_type (str) – string for the controller type

  • model_space (amber.architect.ModelSpace) – a model space instance for the search algorithms

  • session (tf.Session, or None) – a tensorflow session

  • kwargs (dict) – keyword arguments for controller, such as buffer type

get_knowledge_fn(knowledge_fn_type, knowledge_data_dict, *args, **kwargs)[source]

Serialize a knowledge function of given type, for a specified interpretation-based data in knowledge data dictionary

See the documentation for reward_fn for how the knowledge function is combined into consideration in the model’s reward.

get_manager(manager_type, model_fn, reward_fn, data_dict, session, *args, **kwargs)[source]

Serialize a manager of given type.

This step constructs the child neural network manager, one of the two components in the final training environment.

The manager has many dependent workers to construct, train, evaluate, and save the child models.

  1. The most crucial workers are:

    • model_fn : a builder worker that converts architecture to a trainable model

    • reward_fn: an evaluation worker for scoring the trained model; the evaluation can be composite and have multiple components, such as validation loss + interpretation consistency.

  2. The other workers can be passed through keyword arguments, such as:

    • store_fn : a function that specifies what to do for a trained model, e.g. save the weights, plot the model, or perform other relevant post-processing tasks

  • manager_type (str) – string for the manager type

  • model_fn (amber.modeler.BaseModeler) – an instance of amber model builer

  • reward_fn (amber.architect.reward) – an instance of amber reward

  • data_dict (dictionary) – a dictionary for data filepaths, must have keys “train_data” and “validation_data”


work-in-progress : maps GUI inputs to construct an AMBER object, for input and output nodes, and model space


Serialize a model space, most often from a python dictionary, or a pre-defined amber.architect.ModelSpace


arg (dict, or amber.architect.ModelSpace)


Suppose we want to construct a very simple two-hidden-layer model space for building convolutional neural nets.

We need to first specify the input and output node shapes, which are fixed regardless of the network architecture, then the remaining two layers will be implemented as a ModelSpace like so:

from amber.getter import get_model_space
ms = get_model_space([
        # first layer convolution
        [{'Layer_type': 'conv1d', 'kernel_size':4}, {'Layer_type': 'conv1d', 'kernel_size':8} ],
        # second layer pooling
        [{'Layer_type': 'globalmaxpool1d'}, {'Layer_type': 'globalavgpool1d'} ]
# output: StateSpace with 2 layers and 4 total combinations
get_modeler(model_fn_type, model_space, session, *args, **kwargs)[source]

Serialize a model builder of the given type.

Getting a modeler is one of the two essential steps in constructing a manager.

The model builder / modeler will often depend on the model space where the search algorithm (i.e. controller) is sampling architectures from. This is particularly true for efficient NAS (enas)-based modelers, because a super-graph or super-net is built first, then sub-graphs are trained and sampled from the super-graph. See the ENAS paper by Pham et al. 2018 for more details.

Notably, there are also exceptions where a modeler does not depend on a model space, e.g. if the search algorithm returns a list of amber.architect.Operation instead of integers representing tokens, then passing a model space is redundent.

  • model_fn_type (str) – string for the model builder type

  • model_space (amber.architect.ModelSpace) – model space where the search algorithms sample models

  • session (tf.Session) – tensorflow session, can be None if not applicable for certain model builders

get_reward_fn(reward_fn_type, knowledge_fn, *args, **kwargs)[source]

Serialize a reward function for model evaluations.

Getting a reward function is one of the two essential steps in constructing a manager.

A reward function is usually the primary objective we are trying to optimize, such as the accuracy for classification, or the Pearson correlation/variance explained (R^2) for regression. It can be as simple as the negative of the loss function; since it’s a reward, a larger value is what the search algorithm is optimizing.

Of course it is also possible to use a composite of several metrics, such as

\[Reward(model) = Accuracy + \lambda_1 AUROC + \lambda_2 AUPR\]

This can be easily done by passing a custom method argument to amber.architect.reward.LossAucReward.

See the documentation for LossAucReward.

Please note, that the reward function is always applied to evaluate the trained model’s performance on the validation data. The validation data is specified in the manager’s data_dict.

In a special case, we can also augment the validation data-based reward functions with additional interpretation-based functions, called knowledge functions in AMBER.

What’s the difference of the knowledege function versus the validation data-based metrics? Apparently the interpretation-based knowledge data do not fall within the prediction task that the model is built for, otherwise this would have been appended to the validation data.

Think of the knowledge function as a general similar measure, but not doing a forward pass of the neural nework, but by other transformations such as saliency (taking derivative w.r.t. to data), network pruning, or knowledge distillation. Such transformation’s output will be compared to some interpretation-based data; because in certain domains such as biology, we have learned so much about how a particular system works, and having a way to encode these pre-existing knowledege into a set of interpretation-based data, and measure if the neural network makes predictions in consistency with our knowledge, is extremely useful.

Then the optimization of neural network architecture becomes a typical multi-objective optimization problem.

See the preprint manuscript here for joint optimization of prediction accuracy and motif extraction from convolution neural networks arXiv.

get_train_env(env_type, controller, manager, *args, **kwargs)[source]

Serialize a controller training environment, given controller and manager objects

This is the last step of getting a working Amber run - build an environment for controller and manager to interact.

The environment depends on

  • a search algorithm for optimizing neural network architectures, and

  • a manager for coordinating several types of works, including modelers, objective evaluations, buffer of history, and post-train processing.

  • env_type (str) – string for the environment type

  • controller (amber.architect.GeneralController) – an instance inherited from amber controller, to be included as the agent in the training environment

  • manager (amber.architect.GeneralManager) – an instance inherited from amber manager, to be included in the training environment

  • TypeError: – if env_type is not a string

  • ValueError: – if env_type string is not found in current implementations


work-in-progress : maps GUI inputs to construct an AMBER object


helper function that converts a dictionary’s values to data objects by creating DataToParse instances