ncalab.uncertainty ================== .. py:module:: ncalab.uncertainty Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/ncalab/uncertainty/uncertainty_estimator/index Classes ------- .. autoapisummary:: ncalab.uncertainty.BasicNCAModel ncalab.uncertainty.Prediction ncalab.uncertainty.UncertaintyEstimator ncalab.uncertainty.NQM ncalab.uncertainty.MCMC Package Contents ---------------- .. py:class:: BasicNCAModel(device: torch.device, num_image_channels: int, num_hidden_channels: int, num_output_channels: int, plot_function: Optional[ncalab.visualization.Visual] = None, validation_metric: Optional[str] = None, fire_rate: float = 0.5, hidden_size: int = 128, use_alive_mask: bool = False, immutable_image_channels: bool = True, num_learned_filters: int = 2, filter_padding: str = 'reflect', use_laplace: bool = False, kernel_size: int = 3, pad_noise: bool = False, use_temporal_encoding: bool = False, rule_type: type[ncalab.models.basicNCA.basicNCArule.BasicNCARule] = BasicNCARule, training_timesteps: int | Tuple[int, int] = 100, inference_timesteps: int | Tuple[int, int] = 100) Bases: :py:obj:`torch.nn.Module` Abstract base class for NCA models. BasicNCAModel is a composition of an NCA backbone model (called "rule"), and an (optional) head module for downstream tasks. :param device: Pytorch device descriptor. :param num_image_channels: Number of channels reserved for input image. :param num_hidden_channels: Number of hidden channels (communication channels). :param num_output_channels: Number of output channels. :param fire_rate: Fire rate for stochastic weight update. Defaults to 0.5. :param hidden_size: Number of neurons in hidden layer. Defaults to 128. :param use_alive_mask: Whether to use alive masking (channel 3) during training. Defaults to False. :param immutable_image_channels: If image channels should be fixed during inference, which is the case for most segmentation or classification problems. Defaults to True. :param num_learned_filters: Number of learned filters. If zero, use two sobel filters instead. Defaults to 2. :param filter_padding: Padding type to use. Might affect reliance on spatial cues. Defaults to "circular". :param use_laplace: Whether to use Laplace filter (only if num_learned_filters == 0) :param kernel_size: Filter kernel size (only for learned filters) :param pad_noise: Whether to pad input image tensor with noise in hidden / output channels .. py:attribute:: device .. py:attribute:: num_image_channels .. py:attribute:: num_hidden_channels .. py:attribute:: num_output_channels .. py:attribute:: num_channels .. py:attribute:: fire_rate :value: 0.5 .. py:attribute:: hidden_size :value: 128 .. py:attribute:: use_alive_mask :value: False .. py:attribute:: immutable_image_channels :value: True .. py:attribute:: num_learned_filters :value: 2 .. py:attribute:: use_laplace :value: False .. py:attribute:: kernel_size :value: 3 .. py:attribute:: filter_padding :value: 'reflect' .. py:attribute:: pad_noise :value: False .. py:attribute:: use_temporal_encoding :value: False .. py:attribute:: plot_function :value: None .. py:attribute:: validation_metric :value: None .. py:attribute:: training_timesteps :value: 100 .. py:attribute:: inference_timesteps :value: 100 .. py:attribute:: perception .. py:attribute:: input_vector_size .. py:attribute:: rule_type .. py:attribute:: rule .. py:attribute:: head :type: ncalab.models.basicNCA.basicNCAhead.BasicNCAHead | None :value: None .. py:method:: _define_rule() -> ncalab.models.basicNCA.basicNCArule.BasicNCARule .. py:method:: prepare_input(x: torch.Tensor) -> torch.Tensor Preprocess input. Intended to be overwritten by subclass, if preprocessing is necessary. :param x [torch.Tensor]: Input tensor to preprocess. :returns: Processed tensor. .. py:method:: _alive(x) .. py:method:: _update(x: torch.Tensor, step: int) -> torch.Tensor Compute residual cell update. :param x [torch.Tensor]: Input tensor, BCWH :param step [int]: Current timestep, required for computing temporal encoding. :returns: Residual cell update, BCWH. .. py:method:: _forward_step(x: torch.Tensor, step: int) .. py:method:: forward(x: torch.Tensor, steps: int = 1) -> ncalab.prediction.Prediction :param x [torch.Tensor]: Input image, padded along the channel dimension, BCWH. :param steps [int]: Time steps in forward pass. :returns [Prediction]: Prediction object. .. py:method:: loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) -> Dict[str, torch.Tensor] Compute loss. Needs to be overloaded by any subclass. Please note that the returned dict needs to hold "total" key in which the total loss is stored, which is typically a weighted sum of other losses. The total loss is backpropagated, whereas the other losses are sent to tensorboard. :param image [torch.Tensor]: Input image, BCWH. :param label [torch.Tensor]: Ground truth, BCWH. :returns: Dictionary of identifiers mapped to computed losses. .. py:method:: finetune(freeze_head: bool = False) Prepare model for fine tuning by freezing everything except the final layer, and setting to "train" mode. :param: freeze_head .. py:method:: metrics(pred: ncalab.prediction.Prediction, label: torch.Tensor) -> Dict[str, float] Return dict of standard evaluation metrics, given a prediction and corresponding ground truth label. :param pred: Prediction. :type pred: Prediction :param label: Ground truth label. :type label: torch.Tensor :returns: Dict of metrics, mapped by their names. :rtype: Dict[str, float] .. py:method:: predict(image: torch.Tensor, steps: int = 100) -> ncalab.prediction.Prediction Make an NCA prediction, performing multiple forward passes to yield a final result. :param image: Input image, BCWH. :type image: torch.Tensor :param steps: Time steps :type steps: int :returns: Prediction object. :rtype: Prediction .. py:method:: record(image: torch.Tensor, steps: Optional[int] = None) -> List[ncalab.prediction.Prediction] Record predictions for all time steps and return the resulting sequence of predictions. :param image: Input image, BCWH. :type image: torch.Tensor :returns: List of Prediction objects. :rtype: List[Prediction] .. py:method:: validate(image: torch.Tensor, label: torch.Tensor, steps: Optional[int] = None) -> Optional[Tuple[Dict[str, float], ncalab.prediction.Prediction]] Make a prediction on an image of the validation set and return metrics computed with respect to a labelled validation image. :param image [torch.Tensor]: Input image, BCWH :param label [torch.Tensor]: Ground truth label :param steps [int]: Inference steps :returns [Tuple[float, Prediction]]: Validation metric, predicted image BCWH .. py:method:: _to_dict() -> Dict[str, Any] .. py:method:: to_dict() -> Dict[str, Any] .. py:method:: num_trainable_parameters() -> int Returns the number of trainable model parameters. :return: Number of trainable parameters. :rtype: int .. py:class:: Prediction(model, steps: int, output_image: torch.Tensor, head_prediction: Optional[torch.Tensor] = None) Stores the result of an NCA prediction, including the number of steps it took. Sequences are typically stored by BasicNCAModel's "record" function, and are returned as a list of Prediction objects. Constructor is typically not called explicitly. Rather, the forward pass of BasicNCAModel (and its subclasses) is responsible for filling its attributes. :param model: Reference to model used for prediction. :type model: ncalab.BasicNCAModel :param steps: Number of steps taken for the prediction. :type steps: int :param output_image: Output image tensor. :type output_image: torch.Tensor .. py:attribute:: model .. py:attribute:: steps .. py:attribute:: output_image .. py:attribute:: _output_array :type: Optional[numpy.ndarray] :value: None .. py:attribute:: head_prediction :value: None .. py:attribute:: _head_prediction_array :type: Optional[numpy.ndarray] :value: None .. py:property:: image_channels :type: torch.Tensor Convenience property to access the image channels as a Tensor. :returns: BCWH Tensor :rtype: torch.Tensor .. py:property:: hidden_channels :type: torch.Tensor Convenience property to access the hidden channels as a Tensor. :returns: BCWH Tensor :rtype: torch.Tensor .. py:property:: output_channels :type: torch.Tensor Convenience property to access the output channels as a Tensor. :returns: BCWH Tensor :rtype: torch.Tensor .. py:property:: output_array :type: numpy.ndarray Convenience property to access the whole output image in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call. :returns: Numpy array in BCWH format :rtype: np.ndarray .. py:property:: image_channels_np :type: numpy.ndarray Convenience property to access the output image channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call. :returns: Numpy array in BCWH format :rtype: np.ndarray .. py:property:: hidden_channels_np :type: numpy.ndarray Convenience property to access the hidden image channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call. :returns: Numpy array in BCWH format :rtype: np.ndarray .. py:property:: output_channels_np :type: numpy.ndarray Convenience property to access the image's output channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call. :returns: Numpy array in BCWH format :rtype: np.ndarray .. py:property:: head_prediction_array :type: numpy.ndarray | None .. py:class:: UncertaintyEstimator(nca: ncalab.models.BasicNCAModel) :param nca: Trained NCA model :type nca: BasicNCAModel .. py:attribute:: nca .. py:method:: _estimate(image: torch.Tensor) -> Tuple[torch.Tensor, List[ncalab.prediction.Prediction]] Internal uncertainty estimation method. :param image: Input image sample :type image: torch.Tensor :return: Float tensor of uncertainty heatmap (BCWH), predictions, reduced uncertainty score :rtype: Tuple[torch.Tensor, List[Prediction], float] .. py:method:: estimate(image: torch.Tensor, reduce: str = 'mean') -> Tuple[torch.Tensor, List[ncalab.prediction.Prediction], torch.Tensor] Estimate predictive uncertainty. :param image: Input image sample :type image: torch.Tensor :param reduce: Reduction strategy, defaults to "mean". :type reduce: str :return: Float tensor of uncertainty heatmap (BCWH), final prediction, reduced uncertainty score for batch (BC) :rtype: Tuple[torch.Tensor, List[Prediction], torch.Tensor] .. py:method:: __call__(*args, **kwargs) .. py:class:: NQM(nca: ncalab.models.BasicNCAModel, N: int = 10, normalize=False) Bases: :py:obj:`UncertaintyEstimator` Variance over multiple predictions. :param nca: Trained NCA model :type nca: BasicNCAModel .. py:attribute:: N :value: 10 .. py:attribute:: normalize :value: False .. py:method:: _estimate(image: torch.Tensor) -> Tuple[torch.Tensor, List[ncalab.prediction.Prediction]] Internal uncertainty estimation method. :param image: Input image sample :type image: torch.Tensor :return: Float tensor of uncertainty heatmap (BCWH), predictions, reduced uncertainty score :rtype: Tuple[torch.Tensor, List[Prediction], float] .. py:class:: MCMC(nca: ncalab.models.BasicNCAModel, N_last: int = 10, normalize=False) Bases: :py:obj:`UncertaintyEstimator` Markov-Chain Monte Carlo :param nca: Trained NCA model :type nca: BasicNCAModel .. py:attribute:: N_last :value: 10 .. py:attribute:: normalize :value: False .. py:method:: _estimate(image: torch.Tensor) -> Tuple[torch.Tensor, List[ncalab.prediction.Prediction]] Internal uncertainty estimation method. :param image: Input image sample :type image: torch.Tensor :return: Float tensor of uncertainty heatmap (BCWH), predictions, reduced uncertainty score :rtype: Tuple[torch.Tensor, List[Prediction], float]