ncalab.training =============== .. py:module:: ncalab.training Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/ncalab/training/distillation_trainer/index /autoapi/ncalab/training/earlystopping/index /autoapi/ncalab/training/kfold/index /autoapi/ncalab/training/pool/index /autoapi/ncalab/training/trainer/index /autoapi/ncalab/training/traininghistory/index /autoapi/ncalab/training/trainingparameters/index Classes ------- .. autoapisummary:: ncalab.training.EarlyStopping ncalab.training.BasicNCATrainer ncalab.training.TrainingHistory ncalab.training.TrainValRecord ncalab.training.SplitDefinition ncalab.training.KFoldCrossValidationTrainer ncalab.training.BasicNCAModel ncalab.training.Prediction ncalab.training.Visual ncalab.training.EarlyStopping ncalab.training.Pool ncalab.training.TrainingHistory ncalab.training.BasicNCATrainer ncalab.training.BasicNCAModel ncalab.training.TrainingStatus ncalab.training.TrainingHistory Functions --------- .. autoapisummary:: ncalab.training.intepret_range_parameter ncalab.training.pad_input ncalab.training.unwrap Package Contents ---------------- .. py:class:: EarlyStopping(patience: int, min_delta: float = 1e-06) Early stopping helper class. Helps to stop the training if no change in validation metrics is observed. :param patience: Steps to wait until stopping the training. :type patience: int :param min_delta: Minimum deviation until counter is reset, defaults to 1e-6. :type min_delta: float .. py:attribute:: patience .. py:attribute:: min_delta :value: 1e-06 .. py:attribute:: best_accuracy :value: 0.0 .. py:attribute:: counter :value: 0 .. py:method:: done() -> bool Checks whether the training can be stopped. Needs to be queried in training loop, once per epoch. :returns: Whether to stop the training or not. :rtype: bool .. py:method:: step(accuracy: float) Increases internal counter if accuracy doesn't improve, otherwise resets the counter. Needs to be called in training loop, once per epoch. :param accuracy: Validation accuracy. :type accuracy: float .. py:class:: BasicNCATrainer(nca: ncalab.models.basicNCA.BasicNCAModel, model_path: Optional[pathlib.Path | pathlib.PosixPath], gradient_clipping: bool = False, lr: Optional[float] = None, lr_gamma: float = 0.99, adam_betas=(0.9, 0.95), batch_repeat: int = 2, max_epochs: int = 200, optimizer_method: str = 'adam', pool: Optional[ncalab.training.pool.Pool] = None) Trainer class for any model subclassing BasicNCA. :param nca: NCA model instance to train. :type nca: ncalab.BasicNCAModel :param model_path: Path to saved models. If None, models are not saved, defaults to None. :type model_path: Path | PosixPath, optional :param gradient_clipping: Whether to clip gradients, defaults to False. :type gradient_clipping: bool, optional :param lr: Initial learning rate, defaults to 16e-4. :type lr: float, optional :param lr_gamma: Exponential learning rate decay, defaults to 0.9999. :type lr_gamma: float, optional :param adam_betas: Beta values for Adam optimizer, defaults to (0.9, 0.95). :type adam_betas: tuple, optional :param batch_repeat: How often each batch will be duplicated, dfaults to 2. :param max_epochs: Maximum number of epochs in training, defaults to 200. :param optimizer_method: Optimization method, defaults to 'adam'. :type optimizer_method: str, optional :param pool: Sample pool object. :type pool: ncalab.Pool .. py:attribute:: nca .. py:attribute:: model_path .. py:attribute:: gradient_clipping :value: False .. py:attribute:: lr_gamma :value: 0.99 .. py:attribute:: adam_betas :value: (0.9, 0.95) .. py:attribute:: batch_repeat :value: 2 .. py:attribute:: max_epochs :value: 200 .. py:attribute:: optimizer_method :value: 'adam' .. py:attribute:: pool :value: None .. py:method:: info() -> str Shows a markdown-formatted info string with training parameters. Useful for showing info on tensorboard to keep track of parameter changes. :returns [str]: Markdown-formatted info string. .. py:method:: _train_iteration(x: torch.Tensor, y: torch.Tensor, optimizer: torch.optim.Optimizer, total_batch_iterations: int, summary_writer: Optional[torch.utils.tensorboard.SummaryWriter] = None) -> Tuple[ncalab.prediction.Prediction, Dict[str, torch.Tensor]] Run a single training iteration. :param x: Input training images. :param y: Input training labels. :param steps: Number of NCA inference time steps. :param optimizer: Optimizer. :param total_batch_iterations: Total training batch iterations :type total_batch_iterations: int :param summary_writer: Tensorboard SummaryWriter :type summary_writer: SummaryWriter, optional :returns: Predicted image. :rtype: Tuple[Prediction, Dict[str, torch.Tensor]] .. py:method:: train(dataloader_train: torch.utils.data.DataLoader, dataloader_val: Optional[torch.utils.data.DataLoader] = None, dataloader_test: Optional[torch.utils.data.DataLoader] = None, save_every: Optional[int] = None, summary_writer: Optional[torch.utils.tensorboard.SummaryWriter] = None, plot_function: Optional[ncalab.visualization.Visual] = None, earlystopping: Optional[ncalab.training.earlystopping.EarlyStopping] = None) -> ncalab.training.traininghistory.TrainingHistory Execute basic NCA training loop with a single function call. :param dataloader_train [DataLoader]: Training DataLoader :param dataloader_val [DataLoader]: Validation DataLoader :param save_every [int]: How often to save model state (in epochs). Useful for very small datasets, like growing lizard. :param summary_writer [SummaryWriter] Tensorboard SummaryWriter. Defaults to None. :param plot_function: Plot function override. If None, use model's default. Defaults to None. :param earlystopping (EarlyStopping, optional): EarlyStopping object. Defaults to None. :returns [TrainingHistory]: TrainingHistory object. .. py:class:: TrainingHistory(path: Optional[pathlib.Path | pathlib.PosixPath], metrics: Dict[str, float], current_epoch: int, current_model: ncalab.models.BasicNCAModel, best_accuracy: float = 0, best_epoch: int = 0, best_model: Optional[ncalab.models.BasicNCAModel] = None, verbose: bool = True) Stores data about the training progress. Populated during training with ncalab.training.BasicNCATrainer. :param path: Save and load path. :type path: Optional[Path | PosixPath] :param metrics: Dict of validation metrics :type metrics: Dict[str, float] :param current_epoch: Current training epoch. :type current_epoch: int :param current_model: Currently trained model. :type current_model: BasicNCAModel :param best_accuracy: Best validation accuracy, defaults to 0 :type best_accuracy: float, optional :param best_epoch: Epoch of best validation accuracy, defaults to 0 :type best_epoch: int, optional :param best_model: Model with best validation accuracy, defaults to None :type best_model: Optional[BasicNCAModel], optional :param verbose: Whether to print updates of validation accuracy, defaults to True :type verbose: bool, optional .. py:attribute:: path .. py:attribute:: metrics .. py:attribute:: current_epoch .. py:attribute:: current_model .. py:attribute:: best_accuracy :value: 0 .. py:attribute:: best_epoch :value: 0 .. py:attribute:: best_model :value: None .. py:attribute:: verbose :value: True .. py:attribute:: created_timestamp .. py:attribute:: modified_timestamp .. py:method:: update(epoch: int, model: ncalab.models.BasicNCAModel, accuracy: float, overwrite: bool = False) Populates history with current iteration's values. Automatically recognizes changes in accuracy. :param epoch: Current epoch :type epoch: int :param model: Current model :type model: BasicNCAModel :param accuracy: Current accuracy, based on model's validation metric :type accuracy: float :param overwrite: Whether to overwrite best accuracy even with no improvement, defaults to False :type overwrite: bool, optional .. py:method:: save() Saves history and model checkpoint. .. py:method:: to_dict() -> Dict Return dict of recorded values :return: Dict of recorded values :rtype: Dict .. py:class:: TrainValRecord(train: List[str], val: List[str]) Helper class, storing a training / validation data split to generate respective DataLoader objects. :param train: List of training image file paths :type train: List[str] :param val: List of validation image file paths :type val: List[str] .. py:attribute:: train .. py:attribute:: val .. py:method:: dataloaders(DatasetType: Type, path: pathlib.Path | pathlib.PosixPath, transform=None, batch_sizes=None) Generate a pair of training and validation DataLoader objects, based on a given DataSet subtype. .. py:class:: SplitDefinition Stores a k-fold cross-validation split. .. py:attribute:: folds :value: [] .. py:attribute:: dataloader_test :value: None .. py:method:: read(path: pathlib.PosixPath) -> SplitDefinition :staticmethod: Reads json files with split definitions, similar to those created by nnUNet. Format is like .. highlight:: python .. code-block:: python [ { "train": [ "filename0", "filename1",... ] "val": [ "filename2", "filename3",... ] }, { ... } ] :param path: Path to JSON file containing split definition. :returns: SplitDefinition object :rtype: SplitDefinition .. py:method:: __len__() -> int .. py:method:: __getitem__(idx) -> TrainValRecord .. py:class:: KFoldCrossValidationTrainer(trainer: ncalab.training.trainer.BasicNCATrainer, split: SplitDefinition) :param trainer [BasicNCATrainer]: BasicNCATrainer, to train each individual fold. :param split [SplitDefinition]: Definition of the split used for k-fold cross-training. .. py:attribute:: trainer .. py:attribute:: model_prototype .. py:attribute:: model_name .. py:attribute:: split .. py:method:: train(DatasetType: Type, datapath: pathlib.Path | pathlib.PosixPath, transform, batch_sizes: None | Dict = None, save_every: int | None = None) -> List[ncalab.training.traininghistory.TrainingHistory] Run training loop with a single function call. :param DatasetType [Type]: Type of dataset class to use. :param datapath [Path]: _description_ :param transform: Data transform, e.g. initialized via Albumentations. :param batch_sizes: Dict of batch sizes per set, e.g. {"train": 8, "val": 16}. Defaults to None. :param save_every [int]: _description_. Defaults to None. :param plot_function: Plot function override. If None, use model's default. Defaults to None. :returns [List[TrainingHistory]]: List of TrainingHistory objects, one per fold. .. py:class:: BasicNCAModel(device: torch.device, num_image_channels: int, num_hidden_channels: int, num_output_channels: int, plot_function: Optional[ncalab.visualization.Visual] = None, validation_metric: Optional[str] = None, fire_rate: float = 0.5, hidden_size: int = 128, use_alive_mask: bool = False, immutable_image_channels: bool = True, num_learned_filters: int = 2, filter_padding: str = 'reflect', use_laplace: bool = False, kernel_size: int = 3, pad_noise: bool = False, use_temporal_encoding: bool = False, rule_type: type[ncalab.models.basicNCA.basicNCArule.BasicNCARule] = BasicNCARule, training_timesteps: int | Tuple[int, int] = 100, inference_timesteps: int | Tuple[int, int] = 100) Bases: :py:obj:`torch.nn.Module` Abstract base class for NCA models. BasicNCAModel is a composition of an NCA backbone model (called "rule"), and an (optional) head module for downstream tasks. :param device: Pytorch device descriptor. :param num_image_channels: Number of channels reserved for input image. :param num_hidden_channels: Number of hidden channels (communication channels). :param num_output_channels: Number of output channels. :param fire_rate: Fire rate for stochastic weight update. Defaults to 0.5. :param hidden_size: Number of neurons in hidden layer. Defaults to 128. :param use_alive_mask: Whether to use alive masking (channel 3) during training. Defaults to False. :param immutable_image_channels: If image channels should be fixed during inference, which is the case for most segmentation or classification problems. Defaults to True. :param num_learned_filters: Number of learned filters. If zero, use two sobel filters instead. Defaults to 2. :param filter_padding: Padding type to use. Might affect reliance on spatial cues. Defaults to "circular". :param use_laplace: Whether to use Laplace filter (only if num_learned_filters == 0) :param kernel_size: Filter kernel size (only for learned filters) :param pad_noise: Whether to pad input image tensor with noise in hidden / output channels .. py:attribute:: device .. py:attribute:: num_image_channels .. py:attribute:: num_hidden_channels .. py:attribute:: num_output_channels .. py:attribute:: num_channels .. py:attribute:: fire_rate :value: 0.5 .. py:attribute:: hidden_size :value: 128 .. py:attribute:: use_alive_mask :value: False .. py:attribute:: immutable_image_channels :value: True .. py:attribute:: num_learned_filters :value: 2 .. py:attribute:: use_laplace :value: False .. py:attribute:: kernel_size :value: 3 .. py:attribute:: filter_padding :value: 'reflect' .. py:attribute:: pad_noise :value: False .. py:attribute:: use_temporal_encoding :value: False .. py:attribute:: plot_function :value: None .. py:attribute:: validation_metric :value: None .. py:attribute:: training_timesteps :value: 100 .. py:attribute:: inference_timesteps :value: 100 .. py:attribute:: perception .. py:attribute:: input_vector_size .. py:attribute:: rule_type .. py:attribute:: rule .. py:attribute:: head :type: ncalab.models.basicNCA.basicNCAhead.BasicNCAHead | None :value: None .. py:method:: _define_rule() -> ncalab.models.basicNCA.basicNCArule.BasicNCARule .. py:method:: prepare_input(x: torch.Tensor) -> torch.Tensor Preprocess input. Intended to be overwritten by subclass, if preprocessing is necessary. :param x [torch.Tensor]: Input tensor to preprocess. :returns: Processed tensor. .. py:method:: _alive(x) .. py:method:: _update(x: torch.Tensor, step: int) -> torch.Tensor Compute residual cell update. :param x [torch.Tensor]: Input tensor, BCWH :param step [int]: Current timestep, required for computing temporal encoding. :returns: Residual cell update, BCWH. .. py:method:: _forward_step(x: torch.Tensor, step: int) .. py:method:: forward(x: torch.Tensor, steps: int = 1) -> ncalab.prediction.Prediction :param x [torch.Tensor]: Input image, padded along the channel dimension, BCWH. :param steps [int]: Time steps in forward pass. :returns [Prediction]: Prediction object. .. py:method:: loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) -> Dict[str, torch.Tensor] Compute loss. Needs to be overloaded by any subclass. Please note that the returned dict needs to hold "total" key in which the total loss is stored, which is typically a weighted sum of other losses. The total loss is backpropagated, whereas the other losses are sent to tensorboard. :param image [torch.Tensor]: Input image, BCWH. :param label [torch.Tensor]: Ground truth, BCWH. :returns: Dictionary of identifiers mapped to computed losses. .. py:method:: finetune(freeze_head: bool = False) Prepare model for fine tuning by freezing everything except the final layer, and setting to "train" mode. :param: freeze_head .. py:method:: metrics(pred: ncalab.prediction.Prediction, label: torch.Tensor) -> Dict[str, float] Return dict of standard evaluation metrics, given a prediction and corresponding ground truth label. :param pred: Prediction. :type pred: Prediction :param label: Ground truth label. :type label: torch.Tensor :returns: Dict of metrics, mapped by their names. :rtype: Dict[str, float] .. py:method:: predict(image: torch.Tensor, steps: int = 100) -> ncalab.prediction.Prediction Make an NCA prediction, performing multiple forward passes to yield a final result. :param image: Input image, BCWH. :type image: torch.Tensor :param steps: Time steps :type steps: int :returns: Prediction object. :rtype: Prediction .. py:method:: record(image: torch.Tensor, steps: Optional[int] = None) -> List[ncalab.prediction.Prediction] Record predictions for all time steps and return the resulting sequence of predictions. :param image: Input image, BCWH. :type image: torch.Tensor :returns: List of Prediction objects. :rtype: List[Prediction] .. py:method:: validate(image: torch.Tensor, label: torch.Tensor, steps: Optional[int] = None) -> Optional[Tuple[Dict[str, float], ncalab.prediction.Prediction]] Make a prediction on an image of the validation set and return metrics computed with respect to a labelled validation image. :param image [torch.Tensor]: Input image, BCWH :param label [torch.Tensor]: Ground truth label :param steps [int]: Inference steps :returns [Tuple[float, Prediction]]: Validation metric, predicted image BCWH .. py:method:: _to_dict() -> Dict[str, Any] .. py:method:: to_dict() -> Dict[str, Any] .. py:method:: num_trainable_parameters() -> int Returns the number of trainable model parameters. :return: Number of trainable parameters. :rtype: int .. py:class:: Prediction(model, steps: int, output_image: torch.Tensor, head_prediction: Optional[torch.Tensor] = None) Stores the result of an NCA prediction, including the number of steps it took. Sequences are typically stored by BasicNCAModel's "record" function, and are returned as a list of Prediction objects. Constructor is typically not called explicitly. Rather, the forward pass of BasicNCAModel (and its subclasses) is responsible for filling its attributes. :param model: Reference to model used for prediction. :type model: ncalab.BasicNCAModel :param steps: Number of steps taken for the prediction. :type steps: int :param output_image: Output image tensor. :type output_image: torch.Tensor .. py:attribute:: model .. py:attribute:: steps .. py:attribute:: output_image .. py:attribute:: _output_array :type: Optional[numpy.ndarray] :value: None .. py:attribute:: head_prediction :value: None .. py:attribute:: _head_prediction_array :type: Optional[numpy.ndarray] :value: None .. py:property:: image_channels :type: torch.Tensor Convenience property to access the image channels as a Tensor. :returns: BCWH Tensor :rtype: torch.Tensor .. py:property:: hidden_channels :type: torch.Tensor Convenience property to access the hidden channels as a Tensor. :returns: BCWH Tensor :rtype: torch.Tensor .. py:property:: output_channels :type: torch.Tensor Convenience property to access the output channels as a Tensor. :returns: BCWH Tensor :rtype: torch.Tensor .. py:property:: output_array :type: numpy.ndarray Convenience property to access the whole output image in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call. :returns: Numpy array in BCWH format :rtype: np.ndarray .. py:property:: image_channels_np :type: numpy.ndarray Convenience property to access the output image channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call. :returns: Numpy array in BCWH format :rtype: np.ndarray .. py:property:: hidden_channels_np :type: numpy.ndarray Convenience property to access the hidden image channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call. :returns: Numpy array in BCWH format :rtype: np.ndarray .. py:property:: output_channels_np :type: numpy.ndarray Convenience property to access the image's output channels in the format of a numpy array. Brings the entire tensor to CPU on demand, and only at the first call. :returns: Numpy array in BCWH format :rtype: np.ndarray .. py:property:: head_prediction_array :type: numpy.ndarray | None .. py:function:: intepret_range_parameter(x: int | Tuple[int, int]) -> int Interpret a range parameter that is passed for NCA timesteps. If the parameter is a single int, just return it as is. If the parameter is a two-valued tuple, interpret it as a [min,max) and randomly sample from that range. :param x: _description_ :type x: int | Tuple[int, int] :raises TypeError: If something else than an int or a tuple was passed. :return: _description_ :rtype: int .. py:function:: pad_input(x: torch.Tensor, nca: ncalab.models.BasicNCAModel, noise: bool = True, mean: float = 0.5, std: float = 0.225) -> torch.Tensor Pads the BCWH input tensor along its channel dimension to match the expected number of channels required by the NCA model. Pads with either Gaussian noise (parameterized by mean and std) or zeros, depending on the "noise" parameter. :param x: Input image tensor, BCWH. :type x: torch.Tensor :param nca: NCA model definition. :type nca: ncalab.BasicNCAModel :param noise: Whether to pad with noise. Otherwise zeros, defaults to True. :type noise: bool, optional :param mean: Mean (mu) of Gaussian noise distribution, defaults to 0.5. :type mean: float, optional :param std: Standard deviation (sigma) of Gaussian noise distribution, defaults to 0.225. :type std: float, optional :returns: Input tensor, BCWH, padded along the channel dimension. :rtype: torch.Tensor .. py:function:: unwrap(x: Any) Panics if x is None, otherwise returns x. This is a useful shorthand for cases such as ``x = unwrap(some_object).do_something()`` in which we are 99% certain that some_object is not None and want to avoid a mypy complaint. :param x: Any kind of object. :type x: Any :raises RuntimeError: If x is None. :return: Just passes through the input x if it is not None. .. py:class:: Visual Base class for tensorboard visuals. .. py:method:: show(model, image: numpy.ndarray, prediction: ncalab.prediction.Prediction, label: numpy.ndarray) -> matplotlib.figure.Figure .. py:class:: EarlyStopping(patience: int, min_delta: float = 1e-06) Early stopping helper class. Helps to stop the training if no change in validation metrics is observed. :param patience: Steps to wait until stopping the training. :type patience: int :param min_delta: Minimum deviation until counter is reset, defaults to 1e-6. :type min_delta: float .. py:attribute:: patience .. py:attribute:: min_delta :value: 1e-06 .. py:attribute:: best_accuracy :value: 0.0 .. py:attribute:: counter :value: 0 .. py:method:: done() -> bool Checks whether the training can be stopped. Needs to be queried in training loop, once per epoch. :returns: Whether to stop the training or not. :rtype: bool .. py:method:: step(accuracy: float) Increases internal counter if accuracy doesn't improve, otherwise resets the counter. Needs to be called in training loop, once per epoch. :param accuracy: Validation accuracy. :type accuracy: float .. py:class:: Pool(n_seed: int = 1, damage: bool = False, p_damage: float = 0.2) Sample pool that retains previous predictions. Also applies damaging patterns to images to increase the robustness of the trained NCA. :param n_seed: How many seed images to retain, defaults to 1 :type n_seed: int, optional :param damage: Whether to apply damaging patterns, defaults to False :type damage: bool, optional :param p_damage: Probability at which a damaging pattern is applied, defaults to 0.2 :type p_damage: float, optional .. py:attribute:: n_seed :value: 1 .. py:attribute:: damage :value: False .. py:attribute:: batch :type: torch.Tensor | None :value: None .. py:attribute:: p_damage :value: 0.2 .. py:method:: update(batch: torch.Tensor) :param batch: BCWH .. py:method:: sample(seed: torch.Tensor) -> torch.Tensor :param seed: BCWH :return: BCWH .. py:class:: TrainingHistory(path: Optional[pathlib.Path | pathlib.PosixPath], metrics: Dict[str, float], current_epoch: int, current_model: ncalab.models.BasicNCAModel, best_accuracy: float = 0, best_epoch: int = 0, best_model: Optional[ncalab.models.BasicNCAModel] = None, verbose: bool = True) Stores data about the training progress. Populated during training with ncalab.training.BasicNCATrainer. :param path: Save and load path. :type path: Optional[Path | PosixPath] :param metrics: Dict of validation metrics :type metrics: Dict[str, float] :param current_epoch: Current training epoch. :type current_epoch: int :param current_model: Currently trained model. :type current_model: BasicNCAModel :param best_accuracy: Best validation accuracy, defaults to 0 :type best_accuracy: float, optional :param best_epoch: Epoch of best validation accuracy, defaults to 0 :type best_epoch: int, optional :param best_model: Model with best validation accuracy, defaults to None :type best_model: Optional[BasicNCAModel], optional :param verbose: Whether to print updates of validation accuracy, defaults to True :type verbose: bool, optional .. py:attribute:: path .. py:attribute:: metrics .. py:attribute:: current_epoch .. py:attribute:: current_model .. py:attribute:: best_accuracy :value: 0 .. py:attribute:: best_epoch :value: 0 .. py:attribute:: best_model :value: None .. py:attribute:: verbose :value: True .. py:attribute:: created_timestamp .. py:attribute:: modified_timestamp .. py:method:: update(epoch: int, model: ncalab.models.BasicNCAModel, accuracy: float, overwrite: bool = False) Populates history with current iteration's values. Automatically recognizes changes in accuracy. :param epoch: Current epoch :type epoch: int :param model: Current model :type model: BasicNCAModel :param accuracy: Current accuracy, based on model's validation metric :type accuracy: float :param overwrite: Whether to overwrite best accuracy even with no improvement, defaults to False :type overwrite: bool, optional .. py:method:: save() Saves history and model checkpoint. .. py:method:: to_dict() -> Dict Return dict of recorded values :return: Dict of recorded values :rtype: Dict .. py:class:: BasicNCATrainer(nca: ncalab.models.basicNCA.BasicNCAModel, model_path: Optional[pathlib.Path | pathlib.PosixPath], gradient_clipping: bool = False, lr: Optional[float] = None, lr_gamma: float = 0.99, adam_betas=(0.9, 0.95), batch_repeat: int = 2, max_epochs: int = 200, optimizer_method: str = 'adam', pool: Optional[ncalab.training.pool.Pool] = None) Trainer class for any model subclassing BasicNCA. :param nca: NCA model instance to train. :type nca: ncalab.BasicNCAModel :param model_path: Path to saved models. If None, models are not saved, defaults to None. :type model_path: Path | PosixPath, optional :param gradient_clipping: Whether to clip gradients, defaults to False. :type gradient_clipping: bool, optional :param lr: Initial learning rate, defaults to 16e-4. :type lr: float, optional :param lr_gamma: Exponential learning rate decay, defaults to 0.9999. :type lr_gamma: float, optional :param adam_betas: Beta values for Adam optimizer, defaults to (0.9, 0.95). :type adam_betas: tuple, optional :param batch_repeat: How often each batch will be duplicated, dfaults to 2. :param max_epochs: Maximum number of epochs in training, defaults to 200. :param optimizer_method: Optimization method, defaults to 'adam'. :type optimizer_method: str, optional :param pool: Sample pool object. :type pool: ncalab.Pool .. py:attribute:: nca .. py:attribute:: model_path .. py:attribute:: gradient_clipping :value: False .. py:attribute:: lr_gamma :value: 0.99 .. py:attribute:: adam_betas :value: (0.9, 0.95) .. py:attribute:: batch_repeat :value: 2 .. py:attribute:: max_epochs :value: 200 .. py:attribute:: optimizer_method :value: 'adam' .. py:attribute:: pool :value: None .. py:method:: info() -> str Shows a markdown-formatted info string with training parameters. Useful for showing info on tensorboard to keep track of parameter changes. :returns [str]: Markdown-formatted info string. .. py:method:: _train_iteration(x: torch.Tensor, y: torch.Tensor, optimizer: torch.optim.Optimizer, total_batch_iterations: int, summary_writer: Optional[torch.utils.tensorboard.SummaryWriter] = None) -> Tuple[ncalab.prediction.Prediction, Dict[str, torch.Tensor]] Run a single training iteration. :param x: Input training images. :param y: Input training labels. :param steps: Number of NCA inference time steps. :param optimizer: Optimizer. :param total_batch_iterations: Total training batch iterations :type total_batch_iterations: int :param summary_writer: Tensorboard SummaryWriter :type summary_writer: SummaryWriter, optional :returns: Predicted image. :rtype: Tuple[Prediction, Dict[str, torch.Tensor]] .. py:method:: train(dataloader_train: torch.utils.data.DataLoader, dataloader_val: Optional[torch.utils.data.DataLoader] = None, dataloader_test: Optional[torch.utils.data.DataLoader] = None, save_every: Optional[int] = None, summary_writer: Optional[torch.utils.tensorboard.SummaryWriter] = None, plot_function: Optional[ncalab.visualization.Visual] = None, earlystopping: Optional[ncalab.training.earlystopping.EarlyStopping] = None) -> ncalab.training.traininghistory.TrainingHistory Execute basic NCA training loop with a single function call. :param dataloader_train [DataLoader]: Training DataLoader :param dataloader_val [DataLoader]: Validation DataLoader :param save_every [int]: How often to save model state (in epochs). Useful for very small datasets, like growing lizard. :param summary_writer [SummaryWriter] Tensorboard SummaryWriter. Defaults to None. :param plot_function: Plot function override. If None, use model's default. Defaults to None. :param earlystopping (EarlyStopping, optional): EarlyStopping object. Defaults to None. :returns [TrainingHistory]: TrainingHistory object. .. py:class:: BasicNCAModel(device: torch.device, num_image_channels: int, num_hidden_channels: int, num_output_channels: int, plot_function: Optional[ncalab.visualization.Visual] = None, validation_metric: Optional[str] = None, fire_rate: float = 0.5, hidden_size: int = 128, use_alive_mask: bool = False, immutable_image_channels: bool = True, num_learned_filters: int = 2, filter_padding: str = 'reflect', use_laplace: bool = False, kernel_size: int = 3, pad_noise: bool = False, use_temporal_encoding: bool = False, rule_type: type[ncalab.models.basicNCA.basicNCArule.BasicNCARule] = BasicNCARule, training_timesteps: int | Tuple[int, int] = 100, inference_timesteps: int | Tuple[int, int] = 100) Bases: :py:obj:`torch.nn.Module` Abstract base class for NCA models. BasicNCAModel is a composition of an NCA backbone model (called "rule"), and an (optional) head module for downstream tasks. :param device: Pytorch device descriptor. :param num_image_channels: Number of channels reserved for input image. :param num_hidden_channels: Number of hidden channels (communication channels). :param num_output_channels: Number of output channels. :param fire_rate: Fire rate for stochastic weight update. Defaults to 0.5. :param hidden_size: Number of neurons in hidden layer. Defaults to 128. :param use_alive_mask: Whether to use alive masking (channel 3) during training. Defaults to False. :param immutable_image_channels: If image channels should be fixed during inference, which is the case for most segmentation or classification problems. Defaults to True. :param num_learned_filters: Number of learned filters. If zero, use two sobel filters instead. Defaults to 2. :param filter_padding: Padding type to use. Might affect reliance on spatial cues. Defaults to "circular". :param use_laplace: Whether to use Laplace filter (only if num_learned_filters == 0) :param kernel_size: Filter kernel size (only for learned filters) :param pad_noise: Whether to pad input image tensor with noise in hidden / output channels .. py:attribute:: device .. py:attribute:: num_image_channels .. py:attribute:: num_hidden_channels .. py:attribute:: num_output_channels .. py:attribute:: num_channels .. py:attribute:: fire_rate :value: 0.5 .. py:attribute:: hidden_size :value: 128 .. py:attribute:: use_alive_mask :value: False .. py:attribute:: immutable_image_channels :value: True .. py:attribute:: num_learned_filters :value: 2 .. py:attribute:: use_laplace :value: False .. py:attribute:: kernel_size :value: 3 .. py:attribute:: filter_padding :value: 'reflect' .. py:attribute:: pad_noise :value: False .. py:attribute:: use_temporal_encoding :value: False .. py:attribute:: plot_function :value: None .. py:attribute:: validation_metric :value: None .. py:attribute:: training_timesteps :value: 100 .. py:attribute:: inference_timesteps :value: 100 .. py:attribute:: perception .. py:attribute:: input_vector_size .. py:attribute:: rule_type .. py:attribute:: rule .. py:attribute:: head :type: ncalab.models.basicNCA.basicNCAhead.BasicNCAHead | None :value: None .. py:method:: _define_rule() -> ncalab.models.basicNCA.basicNCArule.BasicNCARule .. py:method:: prepare_input(x: torch.Tensor) -> torch.Tensor Preprocess input. Intended to be overwritten by subclass, if preprocessing is necessary. :param x [torch.Tensor]: Input tensor to preprocess. :returns: Processed tensor. .. py:method:: _alive(x) .. py:method:: _update(x: torch.Tensor, step: int) -> torch.Tensor Compute residual cell update. :param x [torch.Tensor]: Input tensor, BCWH :param step [int]: Current timestep, required for computing temporal encoding. :returns: Residual cell update, BCWH. .. py:method:: _forward_step(x: torch.Tensor, step: int) .. py:method:: forward(x: torch.Tensor, steps: int = 1) -> ncalab.prediction.Prediction :param x [torch.Tensor]: Input image, padded along the channel dimension, BCWH. :param steps [int]: Time steps in forward pass. :returns [Prediction]: Prediction object. .. py:method:: loss(pred: ncalab.prediction.Prediction, label: torch.Tensor) -> Dict[str, torch.Tensor] Compute loss. Needs to be overloaded by any subclass. Please note that the returned dict needs to hold "total" key in which the total loss is stored, which is typically a weighted sum of other losses. The total loss is backpropagated, whereas the other losses are sent to tensorboard. :param image [torch.Tensor]: Input image, BCWH. :param label [torch.Tensor]: Ground truth, BCWH. :returns: Dictionary of identifiers mapped to computed losses. .. py:method:: finetune(freeze_head: bool = False) Prepare model for fine tuning by freezing everything except the final layer, and setting to "train" mode. :param: freeze_head .. py:method:: metrics(pred: ncalab.prediction.Prediction, label: torch.Tensor) -> Dict[str, float] Return dict of standard evaluation metrics, given a prediction and corresponding ground truth label. :param pred: Prediction. :type pred: Prediction :param label: Ground truth label. :type label: torch.Tensor :returns: Dict of metrics, mapped by their names. :rtype: Dict[str, float] .. py:method:: predict(image: torch.Tensor, steps: int = 100) -> ncalab.prediction.Prediction Make an NCA prediction, performing multiple forward passes to yield a final result. :param image: Input image, BCWH. :type image: torch.Tensor :param steps: Time steps :type steps: int :returns: Prediction object. :rtype: Prediction .. py:method:: record(image: torch.Tensor, steps: Optional[int] = None) -> List[ncalab.prediction.Prediction] Record predictions for all time steps and return the resulting sequence of predictions. :param image: Input image, BCWH. :type image: torch.Tensor :returns: List of Prediction objects. :rtype: List[Prediction] .. py:method:: validate(image: torch.Tensor, label: torch.Tensor, steps: Optional[int] = None) -> Optional[Tuple[Dict[str, float], ncalab.prediction.Prediction]] Make a prediction on an image of the validation set and return metrics computed with respect to a labelled validation image. :param image [torch.Tensor]: Input image, BCWH :param label [torch.Tensor]: Ground truth label :param steps [int]: Inference steps :returns [Tuple[float, Prediction]]: Validation metric, predicted image BCWH .. py:method:: _to_dict() -> Dict[str, Any] .. py:method:: to_dict() -> Dict[str, Any] .. py:method:: num_trainable_parameters() -> int Returns the number of trainable model parameters. :return: Number of trainable parameters. :rtype: int .. py:class:: TrainingStatus(*args, **kwds) Bases: :py:obj:`enum.Enum` Encodes last status of the training. .. py:attribute:: STATUS_NONE :value: 0 .. py:attribute:: STATUS_RUNNING :value: 1 .. py:attribute:: STATUS_DONE :value: 2 .. py:class:: TrainingHistory(path: Optional[pathlib.Path | pathlib.PosixPath], metrics: Dict[str, float], current_epoch: int, current_model: ncalab.models.BasicNCAModel, best_accuracy: float = 0, best_epoch: int = 0, best_model: Optional[ncalab.models.BasicNCAModel] = None, verbose: bool = True) Stores data about the training progress. Populated during training with ncalab.training.BasicNCATrainer. :param path: Save and load path. :type path: Optional[Path | PosixPath] :param metrics: Dict of validation metrics :type metrics: Dict[str, float] :param current_epoch: Current training epoch. :type current_epoch: int :param current_model: Currently trained model. :type current_model: BasicNCAModel :param best_accuracy: Best validation accuracy, defaults to 0 :type best_accuracy: float, optional :param best_epoch: Epoch of best validation accuracy, defaults to 0 :type best_epoch: int, optional :param best_model: Model with best validation accuracy, defaults to None :type best_model: Optional[BasicNCAModel], optional :param verbose: Whether to print updates of validation accuracy, defaults to True :type verbose: bool, optional .. py:attribute:: path .. py:attribute:: metrics .. py:attribute:: current_epoch .. py:attribute:: current_model .. py:attribute:: best_accuracy :value: 0 .. py:attribute:: best_epoch :value: 0 .. py:attribute:: best_model :value: None .. py:attribute:: verbose :value: True .. py:attribute:: created_timestamp .. py:attribute:: modified_timestamp .. py:method:: update(epoch: int, model: ncalab.models.BasicNCAModel, accuracy: float, overwrite: bool = False) Populates history with current iteration's values. Automatically recognizes changes in accuracy. :param epoch: Current epoch :type epoch: int :param model: Current model :type model: BasicNCAModel :param accuracy: Current accuracy, based on model's validation metric :type accuracy: float :param overwrite: Whether to overwrite best accuracy even with no improvement, defaults to False :type overwrite: bool, optional .. py:method:: save() Saves history and model checkpoint. .. py:method:: to_dict() -> Dict Return dict of recorded values :return: Dict of recorded values :rtype: Dict