ncalab

Submodules

Attributes

`__CURRENT_PATH`	Project root directory.
`ROOT_PATH`	Directory in which training weights are stored.
`WEIGHTS_PATH`	Example task directory.
`TASK_PATH`
`DEFAULT_RANDOM_SEED`
`animator_style_dark`
`animator_styles`

Classes

`AutoStepper`	Helps selecting number of timesteps based on NCA activity.
`GrowingNCADataset`	An abstract class representing a `Dataset`.
`DiceScore`	Pytorch Module that computes the Dice overlap score between two images.
`DiceLoss`	Base class for all neural network modules.
`DiceBCELoss`	Combination of Dice and BCE Loss between two images.
`FocalLoss`	Base class for all neural network modules.
`AbstractNCAModel`	Abstract base class for NCA models.
`AbstractNCAHead`	Base class for all neural network modules.
`BasicNCAPerception`
`AbstractNCARule`	Base class for all neural network modules.
`MLPNCARule`	NCA rule module based on a two-layer Multi-Layer-Perceptron (MLP).
`CascadeNCA`	Chain multiple instances of the same NCA model, operating at different
`ClassificationNCAModel`	Abstract base class for NCA models.
`ClassificationNCAHead`	Base class for all neural network modules.
`DepthNCAModel`	NCA model for monocular depth estimation.
`GrowingNCAModel`	NCA Model class for "growing" tasks, in which a structure is grown from a single seed pixel.
`SegmentationNCAModel`	Model used for image segmentation.
`Prediction`	Stores the result of an NCA prediction, including the number of steps it took.
`BasicNCATrainer`	Trainer class for any model subclassing BasicNCA.
`ParameterSet`
`ParameterSearch`
`Pool`	Sample pool that retains previous predictions. Also applies damaging patterns to
`BasicNCATrainer`	Trainer class for any model subclassing BasicNCA.
`TrainingHistory`	Stores data about the training progress. Populated during training
`EarlyStopping`	Early stopping helper class.
`TrainValRecord`	Helper class, storing a training / validation data split to generate
`SplitDefinition`	Stores a k-fold cross-validation split.
`KFoldCrossValidationTrainer`
`AbstractNCAModel`	Abstract base class for NCA models.
`Prediction`	Stores the result of an NCA prediction, including the number of steps it took.
`UncertaintyEstimator`
`NQM`	Variance over multiple predictions.
`MCMC`	Markov-Chain Monte Carlo
`AnimatorStyle`
`Animator`	Responsible for rendering NCA predictions as GIFs.
`Prediction`	Stores the result of an NCA prediction, including the number of steps it took.
`Visual`	Base class for tensorboard visuals.
`VisualBinaryImageClassification`	Base class for tensorboard visuals.
`VisualRGBImageClassification`	Base class for tensorboard visuals.
`VisualMultiImageClassification`	Base class for tensorboard visuals.
`VisualBinaryImageSegmentation`	Base class for tensorboard visuals.
`VisualDepthEstimation`	Base class for tensorboard visuals.
`VisualGrowing`	Base class for tensorboard visuals.

Functions

`get_compute_device`(→ torch.device)	Obtain a pytorch compute device handle based on input string.
`pad_input`(→ torch.Tensor)	Pads the BCWH input tensor along its channel dimension to match the expected number of
`print_NCALab_banner`()	Show NCALab banner on terminal.
`print_mascot`(message)	Show help text in a speech bubble.
`fix_random_seed`([seed])	Fixes the random seed for all pseudo-random number generators,
`unwrap`(x)	Panics if x is None, otherwise returns x.
`intepret_range_parameter`(→ int)	Interpret a range parameter that is passed for NCA timesteps.
`draw_segmentation_overlay`(image, mask, style)
`unwrap`(x)	Panics if x is None, otherwise returns x.
`abbreviate_label`(L[, max_len])
`show_image_row`(ax, images[, vmin, vmax, cmap, ...])	Shows a row of images next to each other.

Package Contents

class ncalab.AutoStepper(min_steps: int = 10, max_steps: int = 100, plateau: int = 5, verbose: bool = False, threshold: float = 0.01)

Helps selecting number of timesteps based on NCA activity.

Parameters:

min_steps (int, optional) – Minimum number of timesteps to always execute, defaults to 10.
max_steps (int, optional) – Terminate after maximum number of steps, defaults to 100.
plateau (int) – Number of steps that is considered a plateau, defaults to 5.
verbose (bool) – Whether to log interruption to stdout, defaults to False.
threshold (float) – Score threshold, defaults to 1e-2.

min_steps = 10

max_steps = 100

plateau = 5

verbose = False

threshold = 0.01

cooldown = 0

hidden_i: torch.Tensor | None = None

hidden_i_1: torch.Tensor | None = None

_score() → torch.Tensor

Calculates activity score.

Method check() uses this score to determine if the NCA is inactive.

Returns:: Activity score estimate.
Return type:: torch.Tensor

_check(step: int) → bool

Checks whether to interrupt inference after the current step.

Parameters:: step (int) – Current NCA inference step.
Returns:: Whether to interrupt inference after the current step.
Return type:: bool

run(nca: ncalab.models.AbstractNCAModel, x)

__call__(*args: Any, **kwargs: Any) → Any

class ncalab.GrowingNCADataset(image: numpy.ndarray, num_channels: int, batch_size: int = 8)

Bases: torch.utils.data.Dataset

An abstract class representing a Dataset.

All datasets that represent a map from keys to data samples should subclass it. All subclasses should overwrite __getitem__(), supporting fetching a data sample for a given key. Subclasses could also optionally overwrite __len__(), which is expected to return the size of the dataset by many Sampler implementations and the default options of DataLoader. Subclasses could also optionally implement __getitems__(), for speedup batched samples loading. This method accepts list of indices of samples of batch and returns list of samples.

Note

DataLoader by default constructs an index sampler that yields integral indices. To make it work with a map-style dataset with non-integral indices/keys, a custom sampler must be provided.

Dedicated dataset for “growing” tasks, like growing emoji.

The idea is to train a model solely for the purpose to generate (“grow”) a fixed image. Hence, this Dataset class only stores multiple copies of the same image.

Parameters:

[np.ndarray] (image) – Input image.
[int] (batch_size) – Total number of image channels (including hidden)
[int] – Output batch size. Defaults to 8.

batch_size = 8

image

seed

__len__()

__getitem__(idx)

class ncalab.DiceScore

Bases: torch.nn.Module

Pytorch Module that computes the Dice overlap score between two images.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x: torch.Tensor, y: torch.Tensor, smooth: float = 1.0) → torch.Tensor

Parameters:

x (torch.Tensor) – Reference Input
y (torch.Tensor) – Other Input
smooth (float) – Smooting factor, defaults to 1.0

Returns:

Dice score

Return type:

torch.Tensor

class ncalab.DiceLoss

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F


class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

dicescore

forward(x: torch.Tensor, y: torch.Tensor, smooth: float = 1.0) → torch.Tensor

class ncalab.DiceBCELoss

Bases: torch.nn.Module

Combination of Dice and BCE Loss between two images.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

dicescore

forward(x: torch.Tensor, y: torch.Tensor, smooth: float = 1.0) → torch.Tensor

Parameters:

x (torch.Tensor) – Reference Input
y (torch.Tensor) – Other Input
smooth (float) – Smooting factor, defaults to 1.0

Returns:

Dice score

Return type:

torch.Tensor

class ncalab.FocalLoss(weight=None, gamma=2, device='cpu')

Bases: torch.nn.modules.loss._WeightedLoss

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F


class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

gamma = 2

weight = None

device = 'cpu'

ce_loss

forward(_input, _target)

class ncalab.AbstractNCAModel(device: torch.device, num_image_channels: int, num_hidden_channels: int, num_output_channels: int, plot_function: ncalab.visualization.Visual | None = None, validation_metric: str | None = None, fire_rate: float = 0.5, hidden_size: int = 128, use_alive_mask: bool = False, immutable_image_channels: bool = True, num_learned_filters: int = 0, filter_padding: Literal['zero', 'reflect', 'replicate', 'circular'] = 'reflect', use_laplace: bool = False, kernel_size: int = 3, pad_noise: bool = False, use_temporal_encoding: bool = False, rule_type: type[ncalab.models.basicNCA.abstractNCArule.AbstractNCARule] = MLPNCARule, rule_args=None, training_timesteps: int | Tuple[int, int] = 100, inference_timesteps: int | Tuple[int, int] = 100)

Bases: torch.nn.Module, abc.ABC

Abstract base class for NCA models.

BasicNCAModel is a composition of an NCA backbone model (called “rule”), and an (optional) head module for downstream tasks.

Parameters:

device – Pytorch device descriptor.
num_image_channels – Number of channels reserved for input image.
num_hidden_channels – Number of hidden channels (communication channels).
num_output_channels – Number of output channels.
validation_metric
fire_rate – Fire rate for stochastic weight update. Defaults to 0.5.
hidden_size – Number of neurons in hidden layer. Defaults to 128.
use_alive_mask – Whether to use alive masking (channel 3) during training. Defaults to False.
immutable_image_channels – If image channels should be fixed during inference, which is the case for most segmentation or classification problems. Defaults to True.
num_learned_filters – Number of learned filters. If zero, use two sobel filters instead. Defaults to 2.
filter_padding – Padding type to use. Might affect reliance on spatial cues. Defaults to “circular”.
use_laplace – Whether to use Laplace filter (only if num_learned_filters == 0)
kernel_size – Filter kernel size (only for learned filters)
pad_noise – Whether to pad input image tensor with noise in hidden / output channels
use_temporal_encoding
rule_type
rule_args
training_timesteps
inference_timesteps

device

num_image_channels

num_hidden_channels

num_output_channels

num_channels

fire_rate = 0.5

hidden_size = 128

use_alive_mask = False

immutable_image_channels = True

num_learned_filters = 0

use_laplace = False

kernel_size = 3

filter_padding = 'reflect'

pad_noise = False

use_temporal_encoding = False

plot_function = None

validation_metric = None

training_timesteps = 100

inference_timesteps = 100

perception

input_vector_size

rule_type

rule_args = None

rule

head: ncalab.models.basicNCA.abstractNCAhead.AbstractNCAHead | None = None

metrics: Dict[str, torchmetrics.Metric]

_define_rule() → ncalab.models.basicNCA.abstractNCArule.AbstractNCARule

prepare_input(x: torch.Tensor) → torch.Tensor

Preprocess input. Intended to be overwritten by subclass, if preprocessing is necessary.

Parameters:: [torch.Tensor] (x) – Input tensor to preprocess.
Returns:: Processed tensor.

_alive(x)

_update(x: torch.Tensor, step: int) → torch.Tensor

Compute residual cell update.

Parameters:

[torch.Tensor] (x) – Input tensor, BCWH
[int] (step) – Current timestep, required for computing temporal encoding.

Returns:

Residual cell update, BCWH.

_forward_step(x: torch.Tensor, step: int)

forward(x: torch.Tensor, steps: int = 1) → ncalab.prediction.Prediction

Parameters:

[torch.Tensor] (x) – Input image, padded along the channel dimension, BCWH.
[int] (steps) – Time steps in forward pass.

Returns [Prediction]:

Prediction object.

_post_forward_step(x: torch.Tensor) → torch.Tensor

loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) → Dict[str, torch.Tensor]

Compute loss. Needs to be overloaded by any subclass. Please note that the returned dict needs to hold “total” key in which the total loss is stored, which is typically a weighted sum of other losses. The total loss is backpropagated, whereas the other losses are sent to tensorboard.

Parameters:

[torch.Tensor] (label) – Input image, BCWH.
[torch.Tensor] – Ground truth, BCWH.

Returns:

Dictionary of identifiers mapped to computed losses.

finetune(freeze_head: bool = False)

Prepare model for fine tuning by freezing everything except the final layer, and setting to “train” mode.

Param:: freeze_head

predict(image: torch.Tensor, steps: int | Tuple[int, int] | None = None) → ncalab.prediction.Prediction

Make an NCA prediction, performing multiple forward passes to yield a final result.

Parameters:

image (torch.Tensor) – Input image, BCWH.
steps (Optional[int]) – Time steps

Returns:

Prediction object.

Return type:

Prediction

record(image: torch.Tensor, steps: int | Tuple[int, int] | None = None) → List[ncalab.prediction.Prediction]

Record predictions for all time steps and return the resulting sequence of predictions.

Parameters:: image (torch.Tensor) – Input image, BCWH.
Returns:: List of Prediction objects.
Return type:: List[Prediction]

validate(dataloader: torch.utils.data.DataLoader, steps: int | None = None) → Tuple[Dict[str, float], List[ncalab.prediction.Prediction]]

Make a prediction on an image of the validation set and return metrics computed with respect to a labelled validation image.

Parameters:

[torch.utils.data.DataLoader] (dataloader) – Dataloader for validation images
[int] (steps) – Inference steps

Returns [Tuple[float, List[Prediction]]]:

Validation metric, predicted image BCWH

_to_dict() → Dict[str, Any]

to_dict() → Dict[str, Any]

classmethod from_dict(d: Dict[str, Any])

num_trainable_parameters() → int

Returns the number of trainable model parameters.

Returns:: Number of trainable parameters.
Return type:: int

save(path: str | os.PathLike)

static load(model: AbstractNCAModel, path: str | os.PathLike) → AbstractNCAModel

post_prediction(prediction: ncalab.prediction.Prediction) → ncalab.prediction.Prediction

class ncalab.AbstractNCAHead

Bases: torch.nn.Module, abc.ABC

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F


class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

optimizer = None

forward(x: torch.Tensor) → torch.Tensor

Parameters:: x (torch.Tensor) – Input tensor
Returns:: NotImplemented, subclasses are required to implement this method.

abstractmethod freeze(freeze_last: bool = True) → None

Freeze head weights.

Parameters:: freeze_last (bool, optional) – Whether to freeze the last layer (if applicable), defaults to True
Returns:: NotImplemented, subclasses are required to implement this method.

class ncalab.BasicNCAPerception(nca: ncalab.models.basicNCA.AbstractNCAModel)

nca

_define_filters()

Define list of perception filters, based on parameters passed in constructor.

Parameters:: [int] (num_learned_filters) – Number of learned filters in perception filter bank.

perceive(x: torch.Tensor, step: int) → torch.Tensor

freeze()

class ncalab.AbstractNCARule(device: torch.device, input_size: int, hidden_size: int, output_size: int)

Bases: torch.nn.Module, abc.ABC

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F


class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

device

input_size

hidden_size

output_size

abstractmethod freeze(freeze_last: bool = False) → None

class ncalab.MLPNCARule(device: torch.device, input_size: int, hidden_size: int, output_size: int, nonlinearity: type[torch.nn.Module] = nn.ReLU)

Bases: ncalab.models.basicNCA.abstractNCArule.AbstractNCARule

NCA rule module based on a two-layer Multi-Layer-Perceptron (MLP).

Parameters:

nn (_type_) – _description_
device (torch.device) – Compute device
input_size (int) – Input neurons
hidden_size (int) – Hidden neurons
output_size (int) – Output neurons
nonlinearity (type[nn.Module], optional) – Activation function, defaults to nn.ReLU

nonlinearity

_build_network()

_initialize_network()

Initialize network weights of the MLP.

We assume that the default initialization of the first layer is good enough. Since the final layer is purely linear and unbiased, we initalize with 0.

forward(x: torch.Tensor) → torch.Tensor

Parameters:: x (torch.Tensor) – BCWH perception vector
Returns:: BCWH residual update
Return type:: torch.Tensor

freeze(freeze_last: bool = False)

Freeze the first layer of the NCA rule network and, optionally, the final layer.

Parameters:: freeze_last (bool, optional) – _description_, defaults to False

class ncalab.CascadeNCA(wrapped: ncalab.models.basicNCA.AbstractNCAModel, scales: List[int], steps: List[int], single_model: bool = True)

Bases: ncalab.models.basicNCA.AbstractNCAModel

Chain multiple instances of the same NCA model, operating at different image scales.

The idea is to use this model as a wrapper and drop-in replacement for an existing model. For instance, if we created a model nca = SegmentationNCA(…) and all code to interface with it, we could instead write cascade = CascadeNCA(SegmentationNCA(…), scales, steps) without the need for adjusting any of the interfacing code.

This is still highly experimental. In the future, we’ll work on a cleaner interface for this.

Parameters:

wrapped (ncalab.AbstractNCAModel) – Backbone model based on AbstractNCAModel.
scales (List[int]) – List of scales to operate at, e.g. [4, 2, 1].
steps (List[int]) – List of number of NCA inference time steps.
single_model (bool) – Only train a single instance of the NCA model

loss

Compute loss. Needs to be overloaded by any subclass. Please note that the returned dict needs to hold “total” key in which the total loss is stored, which is typically a weighted sum of other losses. The total loss is backpropagated, whereas the other losses are sent to tensorboard.

Parameters:

[torch.Tensor] (label) – Input image, BCWH.
[torch.Tensor] – Ground truth, BCWH.

Returns:

Dictionary of identifiers mapped to computed losses.

finetune

Prepare model for fine tuning by freezing everything except the final layer, and setting to “train” mode.

Param:: freeze_head

prepare_input

Preprocess input. Intended to be overwritten by subclass, if preprocessing is necessary.

Parameters:: [torch.Tensor] (x) – Input tensor to preprocess.
Returns:: Processed tensor.

head

metrics

wrapped

scales

steps

single_model = True

models: List[ncalab.models.basicNCA.AbstractNCAModel]

forward(x: torch.Tensor, *args, **kwargs) → ncalab.prediction.Prediction

Parameters:

x (torch.Tensor) – Input image tensor, BCWH.
steps (torch.Tensor) – Unused, as steps are defined in constructor.

Returns:

Prediction object

Return type:

Prediction

record(image: torch.Tensor, steps: int | Tuple[int, int] | None = None) → List[ncalab.prediction.Prediction]

Records predictions for all time steps and returns the resulting sequence of predictions.

Takes care of scaling the image in between steps.

Parameters:: image (torch.Tensor) – Input image, BCWH.
Returns:: List of Prediction objects.
Return type:: List[Prediction]

class ncalab.ClassificationNCAModel(device: torch.device, num_image_channels: int, num_hidden_channels: int, num_classes: int, fire_rate: float = 0.8, hidden_size: int = 128, use_alive_mask: bool = False, pixel_wise_loss: bool = False, num_learned_filters: int = 2, filter_padding: Literal['zero', 'reflect', 'replicate', 'circular'] = 'reflect', use_laplace: bool = False, kernel_size: int = 3, pad_noise: bool = False, use_temporal_encoding: bool = False, use_classifier: bool = True, class_names: List[str] | None = None, avg_pool_size: int = 8, lambda_hidden: float = 0, **kwargs)

Bases: ncalab.models.basicNCA.AbstractNCAModel

Abstract base class for NCA models.

BasicNCAModel is a composition of an NCA backbone model (called “rule”), and an (optional) head module for downstream tasks.

Parameters:

device – Pytorch device descriptor.
num_image_channels – _description_
num_hidden_channels – _description_
num_classes – _description_
fire_rate – Fire rate for stochastic weight update. Defaults to 0.8.
hidden_size – Number of neurons in hidden layer. Defaults to 128.
use_alive_mask – Whether to use alive masking (channel 3) during training. Defaults to False.
pixel_wise_loss – Whether a prediction per pixel is desired, like in self-classifying MNIST. Defaults to False.
num_learned_filters – Number of learned filters. If zero, use two sobel filters instead. Defaults to 2.
filter_padding – Padding type to use. Might affect reliance on spatial cues. Defaults to “circular”.
pad_noise – Whether to pad input image tensor with noise in hidden / output channels

_num_classes

pixel_wise_loss = False

use_classifier = True

avg_pool_size = 8

lambda_hidden = 0

metrics

focal_loss

property num_classes: int

classify(image: torch.Tensor, steps: int = 100, reduce: bool = False) → torch.Tensor

Predict classification for an input image.

Parameters:

image – Input image.
steps – Inference steps. Defaults to 100.
reduce – Return a single softmax probability. Defaults to False.

Returns:

Single class index or vector of logits.

loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) → Dict[str, torch.Tensor]

Return the classification loss.

Parameters:

pred – Prediction.
label – Ground truth.

Returns:

Dictionary of identifiers mapped to computed losses.

post_prediction(prediction: ncalab.prediction.Prediction) → ncalab.prediction.Prediction

class ncalab.ClassificationNCAHead(num_hidden_channels: int, num_classes: int, device: torch.device, avg_pool_size: int, hidden_size: int = 32)

Bases: ncalab.models.basicNCA.abstractNCAhead.AbstractNCAHead

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F


class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:: training (bool) – Boolean represents whether this module is in training or evaluation mode.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

num_hidden_channels

num_classes

device

avg_pool_size

classifier

forward(x)

Parameters:: x (torch.Tensor) – Input tensor
Returns:: NotImplemented, subclasses are required to implement this method.

freeze(freeze_last: bool = False)

Freeze head weights.

Parameters:: freeze_last (bool, optional) – Whether to freeze the last layer (if applicable), defaults to True
Returns:: NotImplemented, subclasses are required to implement this method.

class ncalab.DepthNCAModel(device: torch.device, num_image_channels: int = 3, num_hidden_channels: int = 18, fire_rate: float = 0.8, hidden_size: int = 128, num_learned_filters: int = 2, pad_noise: bool = False, **kwargs)

Bases: ncalab.models.basicNCA.AbstractNCAModel

NCA model for monocular depth estimation.

Parameters:

device – Pytorch device descriptor.
num_image_channels – Number of channels reserved for input image.
num_hidden_channels – Number of hidden channels (communication channels).
num_output_channels – Number of output channels.
validation_metric
fire_rate – Fire rate for stochastic weight update. Defaults to 0.5.
hidden_size – Number of neurons in hidden layer. Defaults to 128.
use_alive_mask – Whether to use alive masking (channel 3) during training. Defaults to False.
immutable_image_channels – If image channels should be fixed during inference, which is the case for most segmentation or classification problems. Defaults to True.
num_learned_filters – Number of learned filters. If zero, use two sobel filters instead. Defaults to 2.
filter_padding – Padding type to use. Might affect reliance on spatial cues. Defaults to “circular”.
use_laplace – Whether to use Laplace filter (only if num_learned_filters == 0)
kernel_size – Filter kernel size (only for learned filters)
pad_noise – Whether to pad input image tensor with noise in hidden / output channels
use_temporal_encoding
rule_type
rule_args
training_timesteps
inference_timesteps

vignette = None

loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) → Dict[str, torch.Tensor]

Parameters:

image – Input image, BCWH.
label – Ground truth.

Returns:

Dictionary of identifiers mapped to computed losses.

class ncalab.GrowingNCAModel(device: torch.device, num_image_channels: int = 4, num_hidden_channels: int = 16, fire_rate: float = 0.5, hidden_size: int = 128, use_alive_mask: bool = False, lambda_hidden: float = 0.0, **kwargs)

Bases: ncalab.models.basicNCA.AbstractNCAModel

NCA Model class for “growing” tasks, in which a structure is grown from a single seed pixel.

This specialization of the BasicNCAModel has some interesting properties. For instance, it has no output channels, as the growing task directly manipulates the input image channels.

Parameters:

[torch.device] (device) – Pytorch device descriptor.
[int] (hidden_size) – Number of channels reserved for input image. Defaults to 4.
[int] – Number of hidden channels (communication channels). Defaults to 16.
[float] (fire_rate) – Stochastic weight update. Defaults to 0.5.
[int] – Default number of nodes in hidden layer. Defaults to 128.
[bool] (use_alive_mask) – Whether to use alive masking. Defaults to False.

lambda_hidden = 0.0

loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) → Dict[str, torch.Tensor]

Implements a simple MSE loss between target and prediction.

Parameters:

pred – Prediction
label – Target

Returns [Tensor]:

MSE Loss

make_seed(width: int, height: int) → torch.Tensor

grow(seed: torch.Tensor, steps: int = 100) → List[numpy.ndarray]

Run the growth process and return the resulting output sequence.

Parameters:

[torch.Tensor] (seed) – Seed image, can be generated through make_seed.
[int] (steps) – Number of inference steps. Defaults to 100.

Returns [List[np.ndarray]]:

Sequence of output images.

class ncalab.SegmentationNCAModel(device: torch.device, num_image_channels: int = 3, num_hidden_channels: int = 16, num_classes: int = 1, fire_rate: float = 0.8, hidden_size: int = 128, num_learned_filters: int = 2, pad_noise: bool = False, filter_padding: Literal['zero', 'reflect', 'replicate', 'circular'] = 'circular', lambda_hidden: float = 0.001, **kwargs)

Bases: ncalab.models.basicNCA.AbstractNCAModel

Model used for image segmentation.

Uses Dice score as the default validation metric. Currently, only binary segmentation masks are supported.

Parameters:

[torch.device] (device) – Compute device.
[int] (learned_filters) – Number of image channels. Defaults to 3.
[int] – Number of hidden channels. Defaults to 16.
[int] – Number of classes. Defaults to 1.
[float] (fire_rate) – NCA fire rate. Defaults to 0.8.
[int] – Number of neurons in hidden layer. Defaults to 128.
[int] – Number of learned filters. If 0, use sobel. Defaults to 2.
[bool] (pad_noise) – Whether to pad input images with noise. Defaults to True.
[str] (filter_padding) – Padding type to use. Might affect reliance on spatial cues. Defaults to “circular”.

num_classes = 1

metrics

lambda_hidden = 0.001

bce_loss

dice_loss

loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) → Dict[str, torch.Tensor]

Compute Dice loss.

Parameters:

pred – Prediction.
label – Ground truth.

Returns:

Dictionary of identifiers mapped to computed losses.

_post_forward_step(x: torch.Tensor) → torch.Tensor

post_prediction(prediction: ncalab.prediction.Prediction) → ncalab.prediction.Prediction

ncalab.__CURRENT_PATH: Project root directory.

ncalab.ROOT_PATH: Directory in which training weights are stored.

ncalab.WEIGHTS_PATH: Example task directory.

ncalab.TASK_PATH

class ncalab.Prediction(model, steps: int, output_image: torch.Tensor, logits: torch.Tensor, head_prediction: torch.Tensor | None = None)

Stores the result of an NCA prediction, including the number of steps it took.

Sequences are typically stored by BasicNCAModel’s “record” function, and are returned as a list of Prediction objects.

Constructor is typically not called explicitly. Rather, the forward pass of BasicNCAModel (and its subclasses) is responsible for filling its attributes.

Parameters:

model (ncalab.BasicNCAModel) – Reference to model used for prediction.
steps (int) – Number of steps taken for the prediction.
output_image (torch.Tensor) – Output image tensor.

model

steps

output_image

_output_array: numpy.ndarray | None = None

logits

_logits_array: numpy.ndarray | None = None

head_prediction = None

_head_prediction_array: numpy.ndarray | None = None

property image_channels: torch.Tensor

Convenience property to access the image channels as a Tensor.

Returns:: BCWH Tensor
Return type:: torch.Tensor

property hidden_channels: torch.Tensor

Convenience property to access the hidden channels as a Tensor.

Returns:: BCWH Tensor
Return type:: torch.Tensor

property output_channels: torch.Tensor

Convenience property to access the output channels as a Tensor.

Returns:: BCWH Tensor
Return type:: torch.Tensor

property output_array: numpy.ndarray

Convenience property to access the whole output image in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call.

Returns:: Numpy array in BCWH format
Return type:: np.ndarray

property image_channels_np: numpy.ndarray

Convenience property to access the output image channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call.

Returns:: Numpy array in BCWH format
Return type:: np.ndarray

property hidden_channels_np: numpy.ndarray

Convenience property to access the hidden image channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call.

Returns:: Numpy array in BCWH format
Return type:: np.ndarray

property output_channels_np: numpy.ndarray

Convenience property to access the image’s output channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call.

Returns:: Numpy array in BCWH format
Return type:: np.ndarray

property head_prediction_array: numpy.ndarray | None

property logits_array: numpy.ndarray

class ncalab.BasicNCATrainer(nca: ncalab.models.basicNCA.AbstractNCAModel, model_path: pathlib.Path | pathlib.PosixPath | None, gradient_clipping: bool = False, lr: float | None = None, lr_gamma: float = 0.99, adam_betas=(0.9, 0.95), batch_repeat: int = 2, max_epochs: int = 200, optimizer_method: str = 'adam', pool: ncalab.training.pool.Pool | None = None, lr_scheduler: torch.optim.lr_scheduler.LRScheduler | None = None)

Trainer class for any model subclassing BasicNCA.

Parameters:

nca (ncalab.AbstractNCAModel) – NCA model instance to train.
model_path (Path | PosixPath, optional) – Path to saved models. If None, models are not saved, defaults to None.
gradient_clipping (bool, optional) – Whether to clip gradients, defaults to False.
lr (float, optional) – Initial learning rate, defaults to 16e-4.
lr_gamma (float, optional) – Exponential learning rate decay, defaults to 0.9999.
adam_betas (tuple, optional) – Beta values for Adam optimizer, defaults to (0.9, 0.95).
batch_repeat – How often each batch will be duplicated, dfaults to 2.
max_epochs – Maximum number of epochs in training, defaults to 200.
optimizer_method (str, optional) – Optimization method, defaults to ‘adam’.
pool (ncalab.Pool) – Sample pool object.

nca

model_path

gradient_clipping = False

lr_gamma = 0.99

adam_betas = (0.9, 0.95)

batch_repeat = 2

max_epochs = 200

optimizer_method = 'adam'

pool = None

lr_scheduler = None

info() → str

Shows a markdown-formatted info string with training parameters. Useful for showing info on tensorboard to keep track of parameter changes.

Returns [str]:: Markdown-formatted info string.

_train_iteration(x: torch.Tensor, y: torch.Tensor, optimizer: torch.optim.Optimizer, head_optimizer: torch.optim.Optimizer | None, total_batch_iterations: int, summary_writer: torch.utils.tensorboard.SummaryWriter | None = None) → Tuple[ncalab.prediction.Prediction, Dict[str, torch.Tensor]]

Run a single training iteration.

Parameters:

x – Input training images.
y – Input training labels.
steps – Number of NCA inference time steps.
optimizer – Optimizer.
total_batch_iterations (int) – Total training batch iterations
summary_writer (SummaryWriter, optional) – Tensorboard SummaryWriter

Returns:

Predicted image.

Return type:

Tuple[Prediction, Dict[str, torch.Tensor]]

train(dataloader_train: torch.utils.data.DataLoader, dataloader_val: torch.utils.data.DataLoader | None = None, dataloader_test: torch.utils.data.DataLoader | None = None, save_every: int | None = None, summary_writer: torch.utils.tensorboard.SummaryWriter | None = None, plot_function: ncalab.visualization.Visual | None = None, earlystopping: ncalab.training.earlystopping.EarlyStopping | None = None) → ncalab.training.traininghistory.TrainingHistory

Execute basic NCA training loop with a single function call.

Parameters:

[DataLoader] (dataloader_val) – Training DataLoader
[DataLoader] – Validation DataLoader
[int] (save_every) – How often to save model state (in epochs). Useful for very small datasets, like growing lizard.

:param summary_writer [SummaryWriter] Tensorboard SummaryWriter. Defaults to None. :param plot_function: Plot function override. If None, use model’s default. Defaults to None. :param earlystopping (EarlyStopping, optional): EarlyStopping object. Defaults to None.

Returns [TrainingHistory]:: TrainingHistory object.

class ncalab.ParameterSet(**kwargs)

params

mutable

combinations = []

index = 0

is_mutable(key)

info()

next() → Dict[str, Any]

num_combinations()

__len__()

__next__()

__iter__()

class ncalab.ParameterSearch(device, model_class, model_params: ParameterSet, trainer_params: ParameterSet)

device

model_class

model_params

trainer_params

info() → str: Generate information string with a summary of the search to run.

search(dataloader_train: torch.utils.data.DataLoader, dataloader_val: torch.utils.data.DataLoader | None = None)

Run search.

Parameters:

[DataLoader] (dataloader_val) – Training DataLoader.
[DataLoader] – Validation DataLoader. Defaults to None.

__call__(*args, **kwargs): Shorthand for running the search.

class ncalab.Pool(n_seed: int = 1, damage: bool = False, p_damage: float = 0.2)

Sample pool that retains previous predictions. Also applies damaging patterns to images to increase the robustness of the trained NCA.

Parameters:

n_seed (int, optional) – How many seed images to retain, defaults to 1
damage (bool, optional) – Whether to apply damaging patterns, defaults to False
p_damage (float, optional) – Probability at which a damaging pattern is applied, defaults to 0.2

n_seed = 1

damage = False

batch: torch.Tensor | None = None

p_damage = 0.2

update(batch: torch.Tensor)

Parameters:: batch – BCWH

sample(seed: torch.Tensor) → torch.Tensor

Parameters:: seed – BCWH
Returns:: BCWH

class ncalab.BasicNCATrainer(nca: ncalab.models.basicNCA.AbstractNCAModel, model_path: pathlib.Path | pathlib.PosixPath | None, gradient_clipping: bool = False, lr: float | None = None, lr_gamma: float = 0.99, adam_betas=(0.9, 0.95), batch_repeat: int = 2, max_epochs: int = 200, optimizer_method: str = 'adam', pool: ncalab.training.pool.Pool | None = None, lr_scheduler: torch.optim.lr_scheduler.LRScheduler | None = None)

Trainer class for any model subclassing BasicNCA.

Parameters:

nca (ncalab.AbstractNCAModel) – NCA model instance to train.
model_path (Path | PosixPath, optional) – Path to saved models. If None, models are not saved, defaults to None.
gradient_clipping (bool, optional) – Whether to clip gradients, defaults to False.
lr (float, optional) – Initial learning rate, defaults to 16e-4.
lr_gamma (float, optional) – Exponential learning rate decay, defaults to 0.9999.
adam_betas (tuple, optional) – Beta values for Adam optimizer, defaults to (0.9, 0.95).
batch_repeat – How often each batch will be duplicated, dfaults to 2.
max_epochs – Maximum number of epochs in training, defaults to 200.
optimizer_method (str, optional) – Optimization method, defaults to ‘adam’.
pool (ncalab.Pool) – Sample pool object.

nca

model_path

gradient_clipping = False

lr_gamma = 0.99

adam_betas = (0.9, 0.95)

batch_repeat = 2

max_epochs = 200

optimizer_method = 'adam'

pool = None

lr_scheduler = None

info() → str

Shows a markdown-formatted info string with training parameters. Useful for showing info on tensorboard to keep track of parameter changes.

Returns [str]:: Markdown-formatted info string.

_train_iteration(x: torch.Tensor, y: torch.Tensor, optimizer: torch.optim.Optimizer, head_optimizer: torch.optim.Optimizer | None, total_batch_iterations: int, summary_writer: torch.utils.tensorboard.SummaryWriter | None = None) → Tuple[ncalab.prediction.Prediction, Dict[str, torch.Tensor]]

Run a single training iteration.

Parameters:

x – Input training images.
y – Input training labels.
steps – Number of NCA inference time steps.
optimizer – Optimizer.
total_batch_iterations (int) – Total training batch iterations
summary_writer (SummaryWriter, optional) – Tensorboard SummaryWriter

Returns:

Predicted image.

Return type:

Tuple[Prediction, Dict[str, torch.Tensor]]

train(dataloader_train: torch.utils.data.DataLoader, dataloader_val: torch.utils.data.DataLoader | None = None, dataloader_test: torch.utils.data.DataLoader | None = None, save_every: int | None = None, summary_writer: torch.utils.tensorboard.SummaryWriter | None = None, plot_function: ncalab.visualization.Visual | None = None, earlystopping: ncalab.training.earlystopping.EarlyStopping | None = None) → ncalab.training.traininghistory.TrainingHistory

Execute basic NCA training loop with a single function call.

Parameters:

[DataLoader] (dataloader_val) – Training DataLoader
[DataLoader] – Validation DataLoader
[int] (save_every) – How often to save model state (in epochs). Useful for very small datasets, like growing lizard.

:param summary_writer [SummaryWriter] Tensorboard SummaryWriter. Defaults to None. :param plot_function: Plot function override. If None, use model’s default. Defaults to None. :param earlystopping (EarlyStopping, optional): EarlyStopping object. Defaults to None.

Returns [TrainingHistory]:: TrainingHistory object.

class ncalab.TrainingHistory(path: pathlib.Path | pathlib.PosixPath | None, metrics: Dict[str, float], current_epoch: int, current_model: ncalab.models.AbstractNCAModel, best_accuracy: float = 0, best_epoch: int = 0, best_model: ncalab.models.AbstractNCAModel | None = None, verbose: bool = True)

Stores data about the training progress. Populated during training with ncalab.training.BasicNCATrainer.

Parameters:

path (Optional[Path | PosixPath]) – Save and load path.
metrics (Dict[str, float]) – Dict of validation metrics
current_epoch (int) – Current training epoch.
current_model (AbstractNCAModel) – Currently trained model.
best_accuracy (float, optional) – Best validation accuracy, defaults to 0
best_epoch (int, optional) – Epoch of best validation accuracy, defaults to 0
best_model (Optional[AbstractNCAModel], optional) – Model with best validation accuracy, defaults to None
verbose (bool, optional) – Whether to print updates of validation accuracy, defaults to True

path

metrics

current_epoch

current_model

best_accuracy = 0

best_epoch = 0

best_model = None

verbose = True

created_timestamp

modified_timestamp

loss: List[float] = []

update(epoch: int, model: ncalab.models.AbstractNCAModel, accuracy: float, overwrite: bool = False)

Populates history with current iteration’s values.

Automatically recognizes changes in accuracy.

Parameters:

epoch (int) – Current epoch
model (AbstractNCAModel) – Current model
accuracy (float) – Current accuracy, based on model’s validation metric
overwrite (bool, optional) – Whether to overwrite best accuracy even with no improvement, defaults to False

save(): Saves history and model checkpoint.

to_dict() → Dict

Return dict of recorded values

Returns:: Dict of recorded values
Return type:: Dict

class ncalab.EarlyStopping(patience: int, min_delta: float = 1e-06)

Early stopping helper class. Helps to stop the training if no change in validation metrics is observed.

Parameters:

patience (int) – Steps to wait until stopping the training.
min_delta (float) – Minimum deviation until counter is reset, defaults to 1e-6.

patience

min_delta = 1e-06

best_accuracy = 0.0

counter = 0

done() → bool

Checks whether the training can be stopped.

Needs to be queried in training loop, once per epoch.

Returns:: Whether to stop the training or not.
Return type:: bool

step(accuracy: float)

Increases internal counter if accuracy doesn’t improve, otherwise resets the counter.

Needs to be called in training loop, once per epoch.

Parameters:: accuracy (float) – Validation accuracy.

class ncalab.TrainValRecord(train: List[str], val: List[str])

Helper class, storing a training / validation data split to generate respective DataLoader objects.

Parameters:

train (List[str]) – List of training image file paths
val (List[str]) – List of validation image file paths

train

val

dataloaders(DatasetType: Type, path: pathlib.Path | pathlib.PosixPath, transform=None, batch_sizes=None): Generate a pair of training and validation DataLoader objects, based on a given DataSet subtype.

class ncalab.SplitDefinition

Stores a k-fold cross-validation split.

folds = []

dataloader_test = None

static read(path: pathlib.PosixPath) → SplitDefinition

Reads json files with split definitions, similar to those created by nnUNet.

Format is like

[
    {
        "train": [ "filename0", "filename1",... ]
        "val": [ "filename2", "filename3",... ]
    },
    {
        ...
    }
]

Parameters:: path – Path to JSON file containing split definition.
Returns:: SplitDefinition object
Return type:: SplitDefinition

__len__() → int

__getitem__(idx) → TrainValRecord

class ncalab.KFoldCrossValidationTrainer(trainer: ncalab.training.trainer.BasicNCATrainer, split: SplitDefinition)

Parameters:

[BasicNCATrainer] (trainer) – BasicNCATrainer, to train each individual fold.
[SplitDefinition] (split) – Definition of the split used for k-fold cross-training.

trainer

model_prototype

model_name

split

train(DatasetType: Type, datapath: pathlib.Path | pathlib.PosixPath, transform, batch_sizes: None | Dict = None, save_every: int | None = None) → List[ncalab.training.traininghistory.TrainingHistory]

Run training loop with a single function call.

Parameters:

[Type] (DatasetType) – Type of dataset class to use.
[Path] (datapath) – _description_
transform – Data transform, e.g. initialized via Albumentations.
batch_sizes – Dict of batch sizes per set, e.g. {“train”: 8, “val”: 16}. Defaults to None.
[int] (save_every) – _description_. Defaults to None.
plot_function – Plot function override. If None, use model’s default. Defaults to None.

Returns [List[TrainingHistory]]:

List of TrainingHistory objects, one per fold.

class ncalab.AbstractNCAModel(device: torch.device, num_image_channels: int, num_hidden_channels: int, num_output_channels: int, plot_function: ncalab.visualization.Visual | None = None, validation_metric: str | None = None, fire_rate: float = 0.5, hidden_size: int = 128, use_alive_mask: bool = False, immutable_image_channels: bool = True, num_learned_filters: int = 0, filter_padding: Literal['zero', 'reflect', 'replicate', 'circular'] = 'reflect', use_laplace: bool = False, kernel_size: int = 3, pad_noise: bool = False, use_temporal_encoding: bool = False, rule_type: type[ncalab.models.basicNCA.abstractNCArule.AbstractNCARule] = MLPNCARule, rule_args=None, training_timesteps: int | Tuple[int, int] = 100, inference_timesteps: int | Tuple[int, int] = 100)

Bases: torch.nn.Module, abc.ABC

Abstract base class for NCA models.

BasicNCAModel is a composition of an NCA backbone model (called “rule”), and an (optional) head module for downstream tasks.

Parameters:

device – Pytorch device descriptor.
num_image_channels – Number of channels reserved for input image.
num_hidden_channels – Number of hidden channels (communication channels).
num_output_channels – Number of output channels.
validation_metric
fire_rate – Fire rate for stochastic weight update. Defaults to 0.5.
hidden_size – Number of neurons in hidden layer. Defaults to 128.
use_alive_mask – Whether to use alive masking (channel 3) during training. Defaults to False.
immutable_image_channels – If image channels should be fixed during inference, which is the case for most segmentation or classification problems. Defaults to True.
num_learned_filters – Number of learned filters. If zero, use two sobel filters instead. Defaults to 2.
filter_padding – Padding type to use. Might affect reliance on spatial cues. Defaults to “circular”.
use_laplace – Whether to use Laplace filter (only if num_learned_filters == 0)
kernel_size – Filter kernel size (only for learned filters)
pad_noise – Whether to pad input image tensor with noise in hidden / output channels
use_temporal_encoding
rule_type
rule_args
training_timesteps
inference_timesteps

device

num_image_channels

num_hidden_channels

num_output_channels

num_channels

fire_rate = 0.5

hidden_size = 128

use_alive_mask = False

immutable_image_channels = True

num_learned_filters = 0

use_laplace = False

kernel_size = 3

filter_padding = 'reflect'

pad_noise = False

use_temporal_encoding = False

plot_function = None

validation_metric = None

training_timesteps = 100

inference_timesteps = 100

perception

input_vector_size

rule_type

rule_args = None

rule

head: ncalab.models.basicNCA.abstractNCAhead.AbstractNCAHead | None = None

metrics: Dict[str, torchmetrics.Metric]

_define_rule() → ncalab.models.basicNCA.abstractNCArule.AbstractNCARule

prepare_input(x: torch.Tensor) → torch.Tensor

Preprocess input. Intended to be overwritten by subclass, if preprocessing is necessary.

Parameters:: [torch.Tensor] (x) – Input tensor to preprocess.
Returns:: Processed tensor.

_alive(x)

_update(x: torch.Tensor, step: int) → torch.Tensor

Compute residual cell update.

Parameters:

[torch.Tensor] (x) – Input tensor, BCWH
[int] (step) – Current timestep, required for computing temporal encoding.

Returns:

Residual cell update, BCWH.

_forward_step(x: torch.Tensor, step: int)

forward(x: torch.Tensor, steps: int = 1) → ncalab.prediction.Prediction

Parameters:

[torch.Tensor] (x) – Input image, padded along the channel dimension, BCWH.
[int] (steps) – Time steps in forward pass.

Returns [Prediction]:

Prediction object.

_post_forward_step(x: torch.Tensor) → torch.Tensor

loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) → Dict[str, torch.Tensor]

Compute loss. Needs to be overloaded by any subclass. Please note that the returned dict needs to hold “total” key in which the total loss is stored, which is typically a weighted sum of other losses. The total loss is backpropagated, whereas the other losses are sent to tensorboard.

Parameters:

[torch.Tensor] (label) – Input image, BCWH.
[torch.Tensor] – Ground truth, BCWH.

Returns:

Dictionary of identifiers mapped to computed losses.

finetune(freeze_head: bool = False)

Prepare model for fine tuning by freezing everything except the final layer, and setting to “train” mode.

Param:: freeze_head

predict(image: torch.Tensor, steps: int | Tuple[int, int] | None = None) → ncalab.prediction.Prediction

Make an NCA prediction, performing multiple forward passes to yield a final result.

Parameters:

image (torch.Tensor) – Input image, BCWH.
steps (Optional[int]) – Time steps

Returns:

Prediction object.

Return type:

Prediction

record(image: torch.Tensor, steps: int | Tuple[int, int] | None = None) → List[ncalab.prediction.Prediction]

Record predictions for all time steps and return the resulting sequence of predictions.

Parameters:: image (torch.Tensor) – Input image, BCWH.
Returns:: List of Prediction objects.
Return type:: List[Prediction]

validate(dataloader: torch.utils.data.DataLoader, steps: int | None = None) → Tuple[Dict[str, float], List[ncalab.prediction.Prediction]]

Make a prediction on an image of the validation set and return metrics computed with respect to a labelled validation image.

Parameters:

[torch.utils.data.DataLoader] (dataloader) – Dataloader for validation images
[int] (steps) – Inference steps

Returns [Tuple[float, List[Prediction]]]:

Validation metric, predicted image BCWH

_to_dict() → Dict[str, Any]

to_dict() → Dict[str, Any]

classmethod from_dict(d: Dict[str, Any])

num_trainable_parameters() → int

Returns the number of trainable model parameters.

Returns:: Number of trainable parameters.
Return type:: int

save(path: str | os.PathLike)

static load(model: AbstractNCAModel, path: str | os.PathLike) → AbstractNCAModel

post_prediction(prediction: ncalab.prediction.Prediction) → ncalab.prediction.Prediction

class ncalab.Prediction(model, steps: int, output_image: torch.Tensor, logits: torch.Tensor, head_prediction: torch.Tensor | None = None)

Stores the result of an NCA prediction, including the number of steps it took.

Sequences are typically stored by BasicNCAModel’s “record” function, and are returned as a list of Prediction objects.

Constructor is typically not called explicitly. Rather, the forward pass of BasicNCAModel (and its subclasses) is responsible for filling its attributes.

Parameters:

model (ncalab.BasicNCAModel) – Reference to model used for prediction.
steps (int) – Number of steps taken for the prediction.
output_image (torch.Tensor) – Output image tensor.

model

steps

output_image

_output_array: numpy.ndarray | None = None

logits

_logits_array: numpy.ndarray | None = None

head_prediction = None

_head_prediction_array: numpy.ndarray | None = None

property image_channels: torch.Tensor

Convenience property to access the image channels as a Tensor.

Returns:: BCWH Tensor
Return type:: torch.Tensor

property hidden_channels: torch.Tensor

Convenience property to access the hidden channels as a Tensor.

Returns:: BCWH Tensor
Return type:: torch.Tensor

property output_channels: torch.Tensor

Convenience property to access the output channels as a Tensor.

Returns:: BCWH Tensor
Return type:: torch.Tensor

property output_array: numpy.ndarray

Convenience property to access the whole output image in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call.

Returns:: Numpy array in BCWH format
Return type:: np.ndarray

property image_channels_np: numpy.ndarray

Convenience property to access the output image channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call.

Returns:: Numpy array in BCWH format
Return type:: np.ndarray

property hidden_channels_np: numpy.ndarray

Convenience property to access the hidden image channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call.

Returns:: Numpy array in BCWH format
Return type:: np.ndarray

property output_channels_np: numpy.ndarray

Convenience property to access the image’s output channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call.

Returns:: Numpy array in BCWH format
Return type:: np.ndarray

property head_prediction_array: numpy.ndarray | None

property logits_array: numpy.ndarray

class ncalab.UncertaintyEstimator(nca: ncalab.models.AbstractNCAModel)

Parameters:: nca (AbstractNCAModel) – Trained NCA model

nca

_estimate(image: torch.Tensor) → Tuple[torch.Tensor, List[ncalab.prediction.Prediction]]

Internal uncertainty estimation method.

Parameters:: image (torch.Tensor) – Input image sample
Returns:: Float tensor of uncertainty heatmap (BCWH), predictions, reduced uncertainty score
Return type:: Tuple[torch.Tensor, List[Prediction], float]

estimate(image: torch.Tensor, reduce: str = 'mean') → Tuple[torch.Tensor, List[ncalab.prediction.Prediction], torch.Tensor]

Estimate predictive uncertainty.

Parameters:

image (torch.Tensor) – Input image sample
reduce (str) – Reduction strategy, defaults to “mean”.

Returns:

Float tensor of uncertainty heatmap (BCWH), final prediction, reduced uncertainty score for batch (BC)

Return type:

Tuple[torch.Tensor, List[Prediction], torch.Tensor]

__call__(*args, **kwargs)

class ncalab.NQM(nca: ncalab.models.AbstractNCAModel, N: int = 10, normalize=False)

Bases: UncertaintyEstimator

Variance over multiple predictions.

Parameters:: nca (AbstractNCAModel) – Trained NCA model

N = 10

normalize = False

_estimate(image: torch.Tensor) → Tuple[torch.Tensor, List[ncalab.prediction.Prediction]]

Internal uncertainty estimation method.

Parameters:: image (torch.Tensor) – Input image sample
Returns:: Float tensor of uncertainty heatmap (BCWH), predictions, reduced uncertainty score
Return type:: Tuple[torch.Tensor, List[Prediction], float]

class ncalab.MCMC(nca: ncalab.models.AbstractNCAModel, N_last: int = 10, normalize=False)

Bases: UncertaintyEstimator

Markov-Chain Monte Carlo

Parameters:: nca (AbstractNCAModel) – Trained NCA model

N_last = 10

normalize = False

_estimate(image: torch.Tensor) → Tuple[torch.Tensor, List[ncalab.prediction.Prediction]]

Internal uncertainty estimation method.

Parameters:: image (torch.Tensor) – Input image sample
Returns:: Float tensor of uncertainty heatmap (BCWH), predictions, reduced uncertainty score
Return type:: Tuple[torch.Tensor, List[Prediction], float]

ncalab.get_compute_device(device: str = 'cuda:0') → torch.device

Obtain a pytorch compute device handle based on input string. If user tries to get a CUDA device, but none is available, defaults to CPU.

Parameters:: device (str) – Device string, defaults to “cuda:0”.
Returns:: Pytorch compute device.
Return type:: torch.device

ncalab.pad_input(x: torch.Tensor, nca: ncalab.models.AbstractNCAModel, noise: bool = True, mean: float = 0.5, std: float = 0.225) → torch.Tensor

Pads the BCWH input tensor along its channel dimension to match the expected number of channels required by the NCA model. Pads with either Gaussian noise (parameterized by mean and std) or zeros, depending on the “noise” parameter.

Parameters:

x (torch.Tensor) – Input image tensor, BCWH.
nca (ncalab.BasicNCAModel) – NCA model definition.
noise (bool, optional) – Whether to pad with noise. Otherwise zeros, defaults to True.
mean (float, optional) – Mean (mu) of Gaussian noise distribution, defaults to 0.5.
std (float, optional) – Standard deviation (sigma) of Gaussian noise distribution, defaults to 0.225.

Returns:

Input tensor, BCWH, padded along the channel dimension.

Return type:

torch.Tensor

ncalab.print_NCALab_banner(): Show NCALab banner on terminal.

ncalab.print_mascot(message: str)

Show help text in a speech bubble.

Parameters:: message (str) – Message to display.

ncalab.DEFAULT_RANDOM_SEED = 1337

ncalab.fix_random_seed(seed: int = DEFAULT_RANDOM_SEED)

Fixes the random seed for all pseudo-random number generators, including Python-native, Numpy and Pytorch.

Parameters:: seed (int, optional) – Random seed, defaults to DEFAULT_RANDOM_SEED.

ncalab.unwrap(x: Any)

Panics if x is None, otherwise returns x.

This is a useful shorthand for cases such as x = unwrap(some_object).do_something() in which we are 99% certain that some_object is not None and want to avoid a mypy complaint.

Parameters:: x (Any) – Any kind of object.
Raises:: RuntimeError – If x is None.
Returns:: Just passes through the input x if it is not None.

ncalab.intepret_range_parameter(x: int | Tuple[int, int]) → int

Interpret a range parameter that is passed for NCA timesteps.

If the parameter is a single int, just return it as is. If the parameter is a two-valued tuple, interpret it as a [min,max) and randomly sample from that range.

Parameters:: x (int | Tuple[int, int]) – _description_
Raises:: TypeError – If something else than an int or a tuple was passed.
Returns:: _description_
Return type:: int

class ncalab.AnimatorStyle(color_background, color_overlay, color_title, color_progress, underline: bool = True, progress_h: int = 3)

color_background

color_overlay

color_title

color_progress

underline = True

progress_h = 3

apply(fig, ax)

ncalab.animator_style_dark

ncalab.animator_styles

ncalab.draw_segmentation_overlay(image, mask, style)

class ncalab.Animator(nca: ncalab.models.AbstractNCAModel, seed: torch.Tensor, steps: int | None = None, interval: int = 100, repeat: bool = True, repeat_delay: int = 10000, overlay: bool = False, show_timestep: bool = True, hidden: bool = False, show_input: bool = False, style: str | AnimatorStyle = 'dark')

Responsible for rendering NCA predictions as GIFs.

Parameters:

nca (ncalab.AbstractNCAModel) – NCA model instance
seed (torch.Tensor) – Input image for the NCA model
steps (int, optional) – Number of NCA prediction steps per sample, defaults to 100
interval (int, optional) – Time of each frame (milliseconds), defaults to 100
repeat (bool, optional) – Whether to loop the animation, defaults to True
repeat_delay (int, optional) – Time after which the animation is repeated (milliseconds), defaults to 10000
overlay (bool, optional) – Whether to overlay output channel (segmentation mask), defaults to False
show_timestep (bool, optional) – Whether to display timestep in caption, defaults to True

animation_fig

save(path: str | pathlib.Path)

Save generated figure as GIF

Parameters:: path (str | Path) – Output path

class ncalab.Prediction(model, steps: int, output_image: torch.Tensor, logits: torch.Tensor, head_prediction: torch.Tensor | None = None)

Stores the result of an NCA prediction, including the number of steps it took.

Sequences are typically stored by BasicNCAModel’s “record” function, and are returned as a list of Prediction objects.

Constructor is typically not called explicitly. Rather, the forward pass of BasicNCAModel (and its subclasses) is responsible for filling its attributes.

Parameters:

model (ncalab.BasicNCAModel) – Reference to model used for prediction.
steps (int) – Number of steps taken for the prediction.
output_image (torch.Tensor) – Output image tensor.

model

steps

output_image

_output_array: numpy.ndarray | None = None

logits

_logits_array: numpy.ndarray | None = None

head_prediction = None

_head_prediction_array: numpy.ndarray | None = None

property image_channels: torch.Tensor

Convenience property to access the image channels as a Tensor.

Returns:: BCWH Tensor
Return type:: torch.Tensor

property hidden_channels: torch.Tensor

Convenience property to access the hidden channels as a Tensor.

Returns:: BCWH Tensor
Return type:: torch.Tensor

property output_channels: torch.Tensor

Convenience property to access the output channels as a Tensor.

Returns:: BCWH Tensor
Return type:: torch.Tensor

property output_array: numpy.ndarray

Convenience property to access the whole output image in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call.

Returns:: Numpy array in BCWH format
Return type:: np.ndarray

property image_channels_np: numpy.ndarray

Convenience property to access the output image channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call.

Returns:: Numpy array in BCWH format
Return type:: np.ndarray

property hidden_channels_np: numpy.ndarray

Convenience property to access the hidden image channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call.

Returns:: Numpy array in BCWH format
Return type:: np.ndarray

property output_channels_np: numpy.ndarray

Convenience property to access the image’s output channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call.

Returns:: Numpy array in BCWH format
Return type:: np.ndarray

property head_prediction_array: numpy.ndarray | None

property logits_array: numpy.ndarray

ncalab.unwrap(x: Any)

Panics if x is None, otherwise returns x.

This is a useful shorthand for cases such as x = unwrap(some_object).do_something() in which we are 99% certain that some_object is not None and want to avoid a mypy complaint.

Parameters:: x (Any) – Any kind of object.
Raises:: RuntimeError – If x is None.
Returns:: Just passes through the input x if it is not None.

ncalab.abbreviate_label(L, max_len=8)

ncalab.show_image_row(ax, images, vmin=None, vmax=None, cmap=None, overlays=None, overlay_vmin=None, overlay_vmax=None, overlay_cmap=None, label: str = '', colorbar: bool = False, x_index: bool = False, normalize: bool = False)

Shows a row of images next to each other.

Parameters:

ax – Axis object.
images – List of grayscale, RGB or RGBA images, can be CWH or WHC.
vmin – Minimum value to clip channel values, defaults to None
vmax – Maximum value to clip channel values, defaults to None
cmap – matplotlib colormap to apply, defaults to None
overlays – _description_, defaults to None
overlay_vmin – _description_, defaults to None
overlay_vmax – _description_, defaults to None
overlay_cmap – _description_, defaults to None
label – y-axis label next to first image, defaults to “”
colorbar – Whether to display a colorbar next to the last image, defaults to False
x_index – Whether to show the batch index below each image, defaults to False
normalize – Whether to normalize images across batch

class ncalab.Visual

Base class for tensorboard visuals.

show(model, image: numpy.ndarray, prediction: ncalab.prediction.Prediction, label: numpy.ndarray) → matplotlib.figure.Figure

class ncalab.VisualBinaryImageClassification

Bases: Visual

Base class for tensorboard visuals.

show(model, image: numpy.ndarray, prediction: ncalab.prediction.Prediction, label: numpy.ndarray) → matplotlib.figure.Figure

class ncalab.VisualRGBImageClassification

Bases: Visual

Base class for tensorboard visuals.

show(model, image: numpy.ndarray, prediction: ncalab.prediction.Prediction, label: numpy.ndarray) → matplotlib.figure.Figure

class ncalab.VisualMultiImageClassification

Bases: Visual

Base class for tensorboard visuals.

new_instance

class ncalab.VisualBinaryImageSegmentation

Bases: Visual

Base class for tensorboard visuals.

show(model, image: numpy.ndarray, prediction: ncalab.prediction.Prediction, label: numpy.ndarray) → matplotlib.figure.Figure

class ncalab.VisualDepthEstimation

Bases: Visual

Base class for tensorboard visuals.

show(model, image: numpy.ndarray, prediction: ncalab.prediction.Prediction, label: numpy.ndarray) → matplotlib.figure.Figure

class ncalab.VisualGrowing

Bases: Visual

Base class for tensorboard visuals.

show(model, image: numpy.ndarray, prediction: ncalab.prediction.Prediction, label: numpy.ndarray) → matplotlib.figure.Figure