ncalab.models.applications ========================== .. py:module:: ncalab.models.applications Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/ncalab/models/applications/classification/index /autoapi/ncalab/models/applications/depth/index /autoapi/ncalab/models/applications/growing/index /autoapi/ncalab/models/applications/segmentation/index Classes ------- .. autoapisummary:: ncalab.models.applications.ClassificationNCAModel ncalab.models.applications.ClassificationNCAHead ncalab.models.applications.DepthNCAModel ncalab.models.applications.GrowingNCAModel ncalab.models.applications.SegmentationNCAModel Package Contents ---------------- .. py:class:: ClassificationNCAModel(device: torch.device, num_image_channels: int, num_hidden_channels: int, num_classes: int, fire_rate: float = 0.8, hidden_size: int = 128, use_alive_mask: bool = False, pixel_wise_loss: bool = False, num_learned_filters: int = 2, filter_padding: Literal['zero', 'reflect', 'replicate', 'circular'] = 'reflect', use_laplace: bool = False, kernel_size: int = 3, pad_noise: bool = False, use_temporal_encoding: bool = False, use_classifier: bool = True, class_names: Optional[List[str]] = None, avg_pool_size: int = 8, lambda_hidden: float = 0, **kwargs) Bases: :py:obj:`ncalab.models.basicNCA.AbstractNCAModel` Abstract base class for NCA models. BasicNCAModel is a composition of an NCA backbone model (called "rule"), and an (optional) head module for downstream tasks. :param device: Pytorch device descriptor. :param num_image_channels: _description_ :param num_hidden_channels: _description_ :param num_classes: _description_ :param fire_rate: Fire rate for stochastic weight update. Defaults to 0.8. :param hidden_size: Number of neurons in hidden layer. Defaults to 128. :param use_alive_mask: Whether to use alive masking (channel 3) during training. Defaults to False. :param pixel_wise_loss: Whether a prediction per pixel is desired, like in self-classifying MNIST. Defaults to False. :param num_learned_filters: Number of learned filters. If zero, use two sobel filters instead. Defaults to 2. :param filter_padding: Padding type to use. Might affect reliance on spatial cues. Defaults to "circular". :param pad_noise: Whether to pad input image tensor with noise in hidden / output channels .. py:attribute:: _num_classes .. py:attribute:: pixel_wise_loss :value: False .. py:attribute:: use_classifier :value: True .. py:attribute:: avg_pool_size :value: 8 .. py:attribute:: lambda_hidden :value: 0 .. py:attribute:: metrics .. py:attribute:: focal_loss .. py:property:: num_classes :type: int .. py:method:: classify(image: torch.Tensor, steps: int = 100, reduce: bool = False) -> torch.Tensor Predict classification for an input image. :param image: Input image. :param steps: Inference steps. Defaults to 100. :param reduce: Return a single softmax probability. Defaults to False. :returns: Single class index or vector of logits. .. py:method:: loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) -> Dict[str, torch.Tensor] Return the classification loss. :param pred: Prediction. :param label: Ground truth. :returns: Dictionary of identifiers mapped to computed losses. .. py:method:: post_prediction(prediction: ncalab.prediction.Prediction) -> ncalab.prediction.Prediction .. py:class:: ClassificationNCAHead(num_hidden_channels: int, num_classes: int, device: torch.device, avg_pool_size: int, hidden_size: int = 32) Bases: :py:obj:`ncalab.models.basicNCA.abstractNCAhead.AbstractNCAHead` Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:: import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) Submodules assigned in this way will be registered, and will also have their parameters converted when you call :meth:`to`, etc. .. note:: As per the example above, an ``__init__()`` call to the parent class must be made before assignment on the child. :ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool Initialize internal Module state, shared by both nn.Module and ScriptModule. .. py:attribute:: num_hidden_channels .. py:attribute:: num_classes .. py:attribute:: device .. py:attribute:: avg_pool_size .. py:attribute:: classifier .. py:method:: forward(x) :param x: Input tensor :type x: torch.Tensor :returns: NotImplemented, subclasses are required to implement this method. .. py:method:: freeze(freeze_last: bool = False) Freeze head weights. :param freeze_last: Whether to freeze the last layer (if applicable), defaults to True :type freeze_last: bool, optional :returns: NotImplemented, subclasses are required to implement this method. .. py:class:: DepthNCAModel(device: torch.device, num_image_channels: int = 3, num_hidden_channels: int = 18, fire_rate: float = 0.8, hidden_size: int = 128, num_learned_filters: int = 2, pad_noise: bool = False, **kwargs) Bases: :py:obj:`ncalab.models.basicNCA.AbstractNCAModel` NCA model for monocular depth estimation. :param device: Pytorch device descriptor. :param num_image_channels: Number of channels reserved for input image. :param num_hidden_channels: Number of hidden channels (communication channels). :param num_output_channels: Number of output channels. :param validation_metric: :param fire_rate: Fire rate for stochastic weight update. Defaults to 0.5. :param hidden_size: Number of neurons in hidden layer. Defaults to 128. :param use_alive_mask: Whether to use alive masking (channel 3) during training. Defaults to False. :param immutable_image_channels: If image channels should be fixed during inference, which is the case for most segmentation or classification problems. Defaults to True. :param num_learned_filters: Number of learned filters. If zero, use two sobel filters instead. Defaults to 2. :param filter_padding: Padding type to use. Might affect reliance on spatial cues. Defaults to "circular". :param use_laplace: Whether to use Laplace filter (only if num_learned_filters == 0) :param kernel_size: Filter kernel size (only for learned filters) :param pad_noise: Whether to pad input image tensor with noise in hidden / output channels :param use_temporal_encoding: :param rule_type: :param rule_args: :param training_timesteps: :param inference_timesteps: .. py:attribute:: vignette :value: None .. py:method:: loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) -> Dict[str, torch.Tensor] :param image: Input image, BCWH. :param label: Ground truth. :returns: Dictionary of identifiers mapped to computed losses. .. py:class:: GrowingNCAModel(device: torch.device, num_image_channels: int = 4, num_hidden_channels: int = 16, fire_rate: float = 0.5, hidden_size: int = 128, use_alive_mask: bool = False, lambda_hidden: float = 0.0, **kwargs) Bases: :py:obj:`ncalab.models.basicNCA.AbstractNCAModel` NCA Model class for "growing" tasks, in which a structure is grown from a single seed pixel. This specialization of the BasicNCAModel has some interesting properties. For instance, it has no output channels, as the growing task directly manipulates the input image channels. :param device [torch.device]: Pytorch device descriptor. :param num_image_channels [int]: Number of channels reserved for input image. Defaults to 4. :param num_hidden_channels [int]: Number of hidden channels (communication channels). Defaults to 16. :param fire_rate [float]: Stochastic weight update. Defaults to 0.5. :param hidden_size [int]: Default number of nodes in hidden layer. Defaults to 128. :param use_alive_mask [bool]: Whether to use alive masking. Defaults to False. .. py:attribute:: lambda_hidden :value: 0.0 .. py:method:: loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) -> Dict[str, torch.Tensor] Implements a simple MSE loss between target and prediction. :param pred: Prediction :param label: Target :returns [Tensor]: MSE Loss .. py:method:: make_seed(width: int, height: int) -> torch.Tensor .. py:method:: grow(seed: torch.Tensor, steps: int = 100) -> List[numpy.ndarray] Run the growth process and return the resulting output sequence. :param seed [torch.Tensor]: Seed image, can be generated through make_seed. :param steps [int]: Number of inference steps. Defaults to 100. :returns [List[np.ndarray]]: Sequence of output images. .. py:class:: SegmentationNCAModel(device: torch.device, num_image_channels: int = 3, num_hidden_channels: int = 16, num_classes: int = 1, fire_rate: float = 0.8, hidden_size: int = 128, num_learned_filters: int = 2, pad_noise: bool = False, filter_padding: Literal['zero', 'reflect', 'replicate', 'circular'] = 'circular', lambda_hidden: float = 0.001, **kwargs) Bases: :py:obj:`ncalab.models.basicNCA.AbstractNCAModel` Model used for image segmentation. Uses Dice score as the default validation metric. Currently, only binary segmentation masks are supported. :param device [torch.device]: Compute device. :param num_image_channels [int]: Number of image channels. Defaults to 3. :param num_hidden_channels [int]: Number of hidden channels. Defaults to 16. :param num_classes [int]: Number of classes. Defaults to 1. :param fire_rate [float]: NCA fire rate. Defaults to 0.8. :param hidden_size [int]: Number of neurons in hidden layer. Defaults to 128. :param learned_filters [int]: Number of learned filters. If 0, use sobel. Defaults to 2. :param pad_noise [bool]: Whether to pad input images with noise. Defaults to True. :param filter_padding [str]: Padding type to use. Might affect reliance on spatial cues. Defaults to "circular". .. py:attribute:: num_classes :value: 1 .. py:attribute:: metrics .. py:attribute:: lambda_hidden :value: 0.001 .. py:attribute:: bce_loss .. py:attribute:: dice_loss .. py:method:: loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) -> Dict[str, torch.Tensor] Compute Dice loss. :param pred: Prediction. :param label: Ground truth. :returns: Dictionary of identifiers mapped to computed losses. .. py:method:: _post_forward_step(x: torch.Tensor) -> torch.Tensor .. py:method:: post_prediction(prediction: ncalab.prediction.Prediction) -> ncalab.prediction.Prediction