ncalab.training =============== .. py:module:: ncalab.training Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/ncalab/training/earlystopping/index /autoapi/ncalab/training/kfold/index /autoapi/ncalab/training/pool/index /autoapi/ncalab/training/trainer/index /autoapi/ncalab/training/traininghistory/index /autoapi/ncalab/training/trainingparameters/index Classes ------- .. autoapisummary:: ncalab.training.EarlyStopping ncalab.training.BasicNCATrainer ncalab.training.TrainingHistory ncalab.training.TrainValRecord ncalab.training.SplitDefinition ncalab.training.KFoldCrossValidationTrainer ncalab.training.Pool ncalab.training.BasicNCATrainer ncalab.training.TrainingHistory Package Contents ---------------- .. py:class:: EarlyStopping(patience: int, min_delta: float = 1e-06) Early stopping helper class. Helps to stop the training if no change in validation metrics is observed. :param patience: Steps to wait until stopping the training. :type patience: int :param min_delta: Minimum deviation until counter is reset, defaults to 1e-6. :type min_delta: float .. py:attribute:: patience .. py:attribute:: min_delta :value: 1e-06 .. py:attribute:: best_accuracy :value: 0.0 .. py:attribute:: counter :value: 0 .. py:method:: done() -> bool Checks whether the training can be stopped. Needs to be queried in training loop, once per epoch. :returns: Whether to stop the training or not. :rtype: bool .. py:method:: step(accuracy: float) Increases internal counter if accuracy doesn't improve, otherwise resets the counter. Needs to be called in training loop, once per epoch. :param accuracy: Validation accuracy. :type accuracy: float .. py:class:: BasicNCATrainer(nca: ncalab.models.basicNCA.AbstractNCAModel, model_path: Optional[pathlib.Path | pathlib.PosixPath], gradient_clipping: bool = False, lr: Optional[float] = None, lr_gamma: float = 0.99, adam_betas=(0.9, 0.95), batch_repeat: int = 2, max_epochs: int = 200, optimizer_method: str = 'adam', pool: Optional[ncalab.training.pool.Pool] = None, lr_scheduler: Optional[torch.optim.lr_scheduler.LRScheduler] = None) Trainer class for any model subclassing BasicNCA. :param nca: NCA model instance to train. :type nca: ncalab.AbstractNCAModel :param model_path: Path to saved models. If None, models are not saved, defaults to None. :type model_path: Path | PosixPath, optional :param gradient_clipping: Whether to clip gradients, defaults to False. :type gradient_clipping: bool, optional :param lr: Initial learning rate, defaults to 16e-4. :type lr: float, optional :param lr_gamma: Exponential learning rate decay, defaults to 0.9999. :type lr_gamma: float, optional :param adam_betas: Beta values for Adam optimizer, defaults to (0.9, 0.95). :type adam_betas: tuple, optional :param batch_repeat: How often each batch will be duplicated, dfaults to 2. :param max_epochs: Maximum number of epochs in training, defaults to 200. :param optimizer_method: Optimization method, defaults to 'adam'. :type optimizer_method: str, optional :param pool: Sample pool object. :type pool: ncalab.Pool .. py:attribute:: nca .. py:attribute:: model_path .. py:attribute:: gradient_clipping :value: False .. py:attribute:: lr_gamma :value: 0.99 .. py:attribute:: adam_betas :value: (0.9, 0.95) .. py:attribute:: batch_repeat :value: 2 .. py:attribute:: max_epochs :value: 200 .. py:attribute:: optimizer_method :value: 'adam' .. py:attribute:: pool :value: None .. py:attribute:: lr_scheduler :value: None .. py:method:: info() -> str Shows a markdown-formatted info string with training parameters. Useful for showing info on tensorboard to keep track of parameter changes. :returns [str]: Markdown-formatted info string. .. py:method:: _train_iteration(x: torch.Tensor, y: torch.Tensor, optimizer: torch.optim.Optimizer, head_optimizer: torch.optim.Optimizer | None, total_batch_iterations: int, summary_writer: Optional[torch.utils.tensorboard.SummaryWriter] = None) -> Tuple[ncalab.prediction.Prediction, Dict[str, torch.Tensor]] Run a single training iteration. :param x: Input training images. :param y: Input training labels. :param steps: Number of NCA inference time steps. :param optimizer: Optimizer. :param total_batch_iterations: Total training batch iterations :type total_batch_iterations: int :param summary_writer: Tensorboard SummaryWriter :type summary_writer: SummaryWriter, optional :returns: Predicted image. :rtype: Tuple[Prediction, Dict[str, torch.Tensor]] .. py:method:: train(dataloader_train: torch.utils.data.DataLoader, dataloader_val: Optional[torch.utils.data.DataLoader] = None, dataloader_test: Optional[torch.utils.data.DataLoader] = None, save_every: Optional[int] = None, summary_writer: Optional[torch.utils.tensorboard.SummaryWriter] = None, plot_function: Optional[ncalab.visualization.Visual] = None, earlystopping: Optional[ncalab.training.earlystopping.EarlyStopping] = None) -> ncalab.training.traininghistory.TrainingHistory Execute basic NCA training loop with a single function call. :param dataloader_train [DataLoader]: Training DataLoader :param dataloader_val [DataLoader]: Validation DataLoader :param save_every [int]: How often to save model state (in epochs). Useful for very small datasets, like growing lizard. :param summary_writer [SummaryWriter] Tensorboard SummaryWriter. Defaults to None. :param plot_function: Plot function override. If None, use model's default. Defaults to None. :param earlystopping (EarlyStopping, optional): EarlyStopping object. Defaults to None. :returns [TrainingHistory]: TrainingHistory object. .. py:class:: TrainingHistory(path: Optional[pathlib.Path | pathlib.PosixPath], metrics: Dict[str, float], current_epoch: int, current_model: ncalab.models.AbstractNCAModel, best_accuracy: float = 0, best_epoch: int = 0, best_model: Optional[ncalab.models.AbstractNCAModel] = None, verbose: bool = True) Stores data about the training progress. Populated during training with ncalab.training.BasicNCATrainer. :param path: Save and load path. :type path: Optional[Path | PosixPath] :param metrics: Dict of validation metrics :type metrics: Dict[str, float] :param current_epoch: Current training epoch. :type current_epoch: int :param current_model: Currently trained model. :type current_model: AbstractNCAModel :param best_accuracy: Best validation accuracy, defaults to 0 :type best_accuracy: float, optional :param best_epoch: Epoch of best validation accuracy, defaults to 0 :type best_epoch: int, optional :param best_model: Model with best validation accuracy, defaults to None :type best_model: Optional[AbstractNCAModel], optional :param verbose: Whether to print updates of validation accuracy, defaults to True :type verbose: bool, optional .. py:attribute:: path .. py:attribute:: metrics .. py:attribute:: current_epoch .. py:attribute:: current_model .. py:attribute:: best_accuracy :value: 0 .. py:attribute:: best_epoch :value: 0 .. py:attribute:: best_model :value: None .. py:attribute:: verbose :value: True .. py:attribute:: created_timestamp .. py:attribute:: modified_timestamp .. py:attribute:: loss :type: List[float] :value: [] .. py:method:: update(epoch: int, model: ncalab.models.AbstractNCAModel, accuracy: float, overwrite: bool = False) Populates history with current iteration's values. Automatically recognizes changes in accuracy. :param epoch: Current epoch :type epoch: int :param model: Current model :type model: AbstractNCAModel :param accuracy: Current accuracy, based on model's validation metric :type accuracy: float :param overwrite: Whether to overwrite best accuracy even with no improvement, defaults to False :type overwrite: bool, optional .. py:method:: save() Saves history and model checkpoint. .. py:method:: to_dict() -> Dict Return dict of recorded values :return: Dict of recorded values :rtype: Dict .. py:class:: TrainValRecord(train: List[str], val: List[str]) Helper class, storing a training / validation data split to generate respective DataLoader objects. :param train: List of training image file paths :type train: List[str] :param val: List of validation image file paths :type val: List[str] .. py:attribute:: train .. py:attribute:: val .. py:method:: dataloaders(DatasetType: Type, path: pathlib.Path | pathlib.PosixPath, transform=None, batch_sizes=None) Generate a pair of training and validation DataLoader objects, based on a given DataSet subtype. .. py:class:: SplitDefinition Stores a k-fold cross-validation split. .. py:attribute:: folds :value: [] .. py:attribute:: dataloader_test :value: None .. py:method:: read(path: pathlib.PosixPath) -> SplitDefinition :staticmethod: Reads json files with split definitions, similar to those created by nnUNet. Format is like .. highlight:: python .. code-block:: python [ { "train": [ "filename0", "filename1",... ] "val": [ "filename2", "filename3",... ] }, { ... } ] :param path: Path to JSON file containing split definition. :returns: SplitDefinition object :rtype: SplitDefinition .. py:method:: __len__() -> int .. py:method:: __getitem__(idx) -> TrainValRecord .. py:class:: KFoldCrossValidationTrainer(trainer: ncalab.training.trainer.BasicNCATrainer, split: SplitDefinition) :param trainer [BasicNCATrainer]: BasicNCATrainer, to train each individual fold. :param split [SplitDefinition]: Definition of the split used for k-fold cross-training. .. py:attribute:: trainer .. py:attribute:: model_prototype .. py:attribute:: model_name .. py:attribute:: split .. py:method:: train(DatasetType: Type, datapath: pathlib.Path | pathlib.PosixPath, transform, batch_sizes: None | Dict = None, save_every: int | None = None) -> List[ncalab.training.traininghistory.TrainingHistory] Run training loop with a single function call. :param DatasetType [Type]: Type of dataset class to use. :param datapath [Path]: _description_ :param transform: Data transform, e.g. initialized via Albumentations. :param batch_sizes: Dict of batch sizes per set, e.g. {"train": 8, "val": 16}. Defaults to None. :param save_every [int]: _description_. Defaults to None. :param plot_function: Plot function override. If None, use model's default. Defaults to None. :returns [List[TrainingHistory]]: List of TrainingHistory objects, one per fold. .. py:class:: Pool(n_seed: int = 1, damage: bool = False, p_damage: float = 0.2) Sample pool that retains previous predictions. Also applies damaging patterns to images to increase the robustness of the trained NCA. :param n_seed: How many seed images to retain, defaults to 1 :type n_seed: int, optional :param damage: Whether to apply damaging patterns, defaults to False :type damage: bool, optional :param p_damage: Probability at which a damaging pattern is applied, defaults to 0.2 :type p_damage: float, optional .. py:attribute:: n_seed :value: 1 .. py:attribute:: damage :value: False .. py:attribute:: batch :type: torch.Tensor | None :value: None .. py:attribute:: p_damage :value: 0.2 .. py:method:: update(batch: torch.Tensor) :param batch: BCWH .. py:method:: sample(seed: torch.Tensor) -> torch.Tensor :param seed: BCWH :return: BCWH .. py:class:: BasicNCATrainer(nca: ncalab.models.basicNCA.AbstractNCAModel, model_path: Optional[pathlib.Path | pathlib.PosixPath], gradient_clipping: bool = False, lr: Optional[float] = None, lr_gamma: float = 0.99, adam_betas=(0.9, 0.95), batch_repeat: int = 2, max_epochs: int = 200, optimizer_method: str = 'adam', pool: Optional[ncalab.training.pool.Pool] = None, lr_scheduler: Optional[torch.optim.lr_scheduler.LRScheduler] = None) Trainer class for any model subclassing BasicNCA. :param nca: NCA model instance to train. :type nca: ncalab.AbstractNCAModel :param model_path: Path to saved models. If None, models are not saved, defaults to None. :type model_path: Path | PosixPath, optional :param gradient_clipping: Whether to clip gradients, defaults to False. :type gradient_clipping: bool, optional :param lr: Initial learning rate, defaults to 16e-4. :type lr: float, optional :param lr_gamma: Exponential learning rate decay, defaults to 0.9999. :type lr_gamma: float, optional :param adam_betas: Beta values for Adam optimizer, defaults to (0.9, 0.95). :type adam_betas: tuple, optional :param batch_repeat: How often each batch will be duplicated, dfaults to 2. :param max_epochs: Maximum number of epochs in training, defaults to 200. :param optimizer_method: Optimization method, defaults to 'adam'. :type optimizer_method: str, optional :param pool: Sample pool object. :type pool: ncalab.Pool .. py:attribute:: nca .. py:attribute:: model_path .. py:attribute:: gradient_clipping :value: False .. py:attribute:: lr_gamma :value: 0.99 .. py:attribute:: adam_betas :value: (0.9, 0.95) .. py:attribute:: batch_repeat :value: 2 .. py:attribute:: max_epochs :value: 200 .. py:attribute:: optimizer_method :value: 'adam' .. py:attribute:: pool :value: None .. py:attribute:: lr_scheduler :value: None .. py:method:: info() -> str Shows a markdown-formatted info string with training parameters. Useful for showing info on tensorboard to keep track of parameter changes. :returns [str]: Markdown-formatted info string. .. py:method:: _train_iteration(x: torch.Tensor, y: torch.Tensor, optimizer: torch.optim.Optimizer, head_optimizer: torch.optim.Optimizer | None, total_batch_iterations: int, summary_writer: Optional[torch.utils.tensorboard.SummaryWriter] = None) -> Tuple[ncalab.prediction.Prediction, Dict[str, torch.Tensor]] Run a single training iteration. :param x: Input training images. :param y: Input training labels. :param steps: Number of NCA inference time steps. :param optimizer: Optimizer. :param total_batch_iterations: Total training batch iterations :type total_batch_iterations: int :param summary_writer: Tensorboard SummaryWriter :type summary_writer: SummaryWriter, optional :returns: Predicted image. :rtype: Tuple[Prediction, Dict[str, torch.Tensor]] .. py:method:: train(dataloader_train: torch.utils.data.DataLoader, dataloader_val: Optional[torch.utils.data.DataLoader] = None, dataloader_test: Optional[torch.utils.data.DataLoader] = None, save_every: Optional[int] = None, summary_writer: Optional[torch.utils.tensorboard.SummaryWriter] = None, plot_function: Optional[ncalab.visualization.Visual] = None, earlystopping: Optional[ncalab.training.earlystopping.EarlyStopping] = None) -> ncalab.training.traininghistory.TrainingHistory Execute basic NCA training loop with a single function call. :param dataloader_train [DataLoader]: Training DataLoader :param dataloader_val [DataLoader]: Validation DataLoader :param save_every [int]: How often to save model state (in epochs). Useful for very small datasets, like growing lizard. :param summary_writer [SummaryWriter] Tensorboard SummaryWriter. Defaults to None. :param plot_function: Plot function override. If None, use model's default. Defaults to None. :param earlystopping (EarlyStopping, optional): EarlyStopping object. Defaults to None. :returns [TrainingHistory]: TrainingHistory object. .. py:class:: TrainingHistory(path: Optional[pathlib.Path | pathlib.PosixPath], metrics: Dict[str, float], current_epoch: int, current_model: ncalab.models.AbstractNCAModel, best_accuracy: float = 0, best_epoch: int = 0, best_model: Optional[ncalab.models.AbstractNCAModel] = None, verbose: bool = True) Stores data about the training progress. Populated during training with ncalab.training.BasicNCATrainer. :param path: Save and load path. :type path: Optional[Path | PosixPath] :param metrics: Dict of validation metrics :type metrics: Dict[str, float] :param current_epoch: Current training epoch. :type current_epoch: int :param current_model: Currently trained model. :type current_model: AbstractNCAModel :param best_accuracy: Best validation accuracy, defaults to 0 :type best_accuracy: float, optional :param best_epoch: Epoch of best validation accuracy, defaults to 0 :type best_epoch: int, optional :param best_model: Model with best validation accuracy, defaults to None :type best_model: Optional[AbstractNCAModel], optional :param verbose: Whether to print updates of validation accuracy, defaults to True :type verbose: bool, optional .. py:attribute:: path .. py:attribute:: metrics .. py:attribute:: current_epoch .. py:attribute:: current_model .. py:attribute:: best_accuracy :value: 0 .. py:attribute:: best_epoch :value: 0 .. py:attribute:: best_model :value: None .. py:attribute:: verbose :value: True .. py:attribute:: created_timestamp .. py:attribute:: modified_timestamp .. py:attribute:: loss :type: List[float] :value: [] .. py:method:: update(epoch: int, model: ncalab.models.AbstractNCAModel, accuracy: float, overwrite: bool = False) Populates history with current iteration's values. Automatically recognizes changes in accuracy. :param epoch: Current epoch :type epoch: int :param model: Current model :type model: AbstractNCAModel :param accuracy: Current accuracy, based on model's validation metric :type accuracy: float :param overwrite: Whether to overwrite best accuracy even with no improvement, defaults to False :type overwrite: bool, optional .. py:method:: save() Saves history and model checkpoint. .. py:method:: to_dict() -> Dict Return dict of recorded values :return: Dict of recorded values :rtype: Dict