Package rawDeepLearningClassifier

Class DLControl

java.lang.Object
PamController.PamControlledUnit
rawDeepLearningClassifier.DLControl
All Implemented Interfaces:
ClipDisplayParent, PamSettings, SettingsNameProvider
public class DLControl extends PamControlledUnit implements PamSettings, ClipDisplayParent
Module which uses an external deep learning classifier to identify any data unit containing raw data.

For example, the module can be used to apply a deep learning model to analyse raw sound data, clips or click detections.

The module is essentially a PAMGuard wrapper for the jdl4pam library which does most of the working in loading models and setting up spectrogram transforms so models receive the correct input data. The majority of the code for the module involves interfacing with PAMGuard's data management system (e.g. binary files), visualisation tools such as spectrogram graphics and the TDDisplayFX and creating a settings GUI so users can load models easily and visualise how they work.

Currently the jdl4pam library supports three types of deep learning model, Generic, AnimalSpot and Ketos.

Generic
Generic models allows users to load almost any type of model and manually assign the types of data transform and input shape. This means that the user has to get the settings exactly right or the model will not work. It is the most complex way to run a deep learning model and generally not recommenced. However, users can export a settings file which makes it easier to set up for another user.

AnimalSpot
AnimalSpot is a framework for training acoustic deep learning models using Pytorch. Users can load a .py model which contains embedded metadata so that PMAGuard knows the exact transforms required for the model input. This makes deploying models in PAMGuard very easy - users require little or no experience to get this working.

Ketos
Ketos is a framework for training acoustic deep learning models using TensorFlow. Users can load a .ktpb model which contains embedded metadata so that PMAGuard knows the exact transforms required for the model input. Like AnimalSpot, this makes deploying Ketos models in PAMGuard very straightforward.

Author:
Jamie Macaulay

  • Field Details

    • PLUGIN_BUILD

      public static final boolean PLUGIN_BUILD
      PLUGIN_BUILD boolean is set to true so that the class loader isn't changed. When ready to compile into a fatjar set to true. When using eclipse set to false.
      See Also:

    • PROCESSING_START

      public static final int PROCESSING_START
      Flag for processing start
      See Also:

    • NEW_PARAMS

      public static final int NEW_PARAMS
      Flag to indicate a setup is required
      See Also:

    • PROCESSING_END

      public static final int PROCESSING_END
      See Also:

  • Constructor Details

    • DLControl

      public DLControl(String unitName)
      Constructor for the DL Control.
      Parameters:
      unitName - - the unit name.

  • Method Details

    • getDLModels

      public ArrayList<DLClassiferModel> getDLModels()
      Get the available deep learning models
      Returns:
      the available deep learning models.

    • getDLModel

      public DLClassiferModel getDLModel(String string)
      Get a model by it's name.
      Parameters:
      the - name the model.
      Returns:
      the corresponding model object or null if no model with the name exists.

    • getDLModel

      public DLClassiferModel getDLModel()
      Get the current deep learning model.
      Returns:
      the current deep learning model.

    • getDLParams

      public RawDLParams getDLParams()
      Get basic parameters.
      Returns:
      parameters class.

    • getSettingsReference

      public Serializable getSettingsReference()
      Specified by:
      getSettingsReference in interface PamSettings
      Returns:
      The serialisable object that will be stored

    • getSettingsVersion

      public long getSettingsVersion()
      Specified by:
      getSettingsVersion in interface PamSettings
      Returns:
      An integer version number for the settings

    • restoreSettings

      public boolean restoreSettings(PamControlledUnitSettings pamControlledUnitSettings)
      Specified by:
      restoreSettings in interface PamSettings
      Parameters:
      pamControlledUnitSettings -
      Returns:
      true if successful The object performs final checks (if needed) and then casts the settings data pamcontrolledunitSettings.settings into the correct type and uses as required

    • getSegmenter

      public SegmenterProcess getSegmenter()
      Get the segmenter process. This breaks raw data into chunks, combines into groups and sends to DL classifiers.
      Returns:
      the segmenter process.

    • getSettingsPane

      public DLSettingsPane getSettingsPane()
      Get the settings pane.
      Returns:
      the settings pane.

    • showSettingsDialog

      public void showSettingsDialog(Frame parentFrame)
      Show settings dialog.
      Parameters:
      parentFrame - - the frame.

    • getSidePanel

      public PamSidePanel getSidePanel()
      Description copied from class: PamControlledUnit
      Gets a reference to a small panel to be displayed along the left hand edge of the main tab panel. Side panels should be small since they are always visible and any space they take will be taken from the main tab panel.

      It is possible for a PamControlled unit to have a side panel without having a pamTabPanel.

      Overrides:
      getSidePanel in class PamControlledUnit
      Returns:
      a pamSidePanel object.
      See Also:

    • createDetectionMenu

      public JMenuItem createDetectionMenu(Frame parentFrame)
      Description copied from class: PamControlledUnit
      Create a JMenu object containing MenuItems associated with PamProcesses
      Overrides:
      createDetectionMenu in class PamControlledUnit
      Parameters:
      parentFrame - The owner frame of the menu
      Returns:
      reference to a JMenu which can be added to an existing menu or menu bar

      Note that if multiple views are to use the same menu, then they should each create a new menu (by setting Create to true) the first time they call this method.

    • getDLClassifyProcess

      public DLClassifyProcess getDLClassifyProcess()
      Get the deep learning classification process. This handles running the current deep learning model.
      Returns:
      the deep learning classification process.

    • getGUI

      public PamControlledUnitGUI getGUI(int flag)
      Get the GUI for the PAMControlled unit. This has multiple GUI options which are instantiated depending on the view type.
      Overrides:
      getGUI in class PamControlledUnit
      Parameters:
      flag - . The GUI type flag defined in PAMGuiManager.
      Returns:
      the GUI for the PamControlledUnit unit.

    • setParams

      public void setParams(RawDLParams newParams)

    • checkModelParams

      public void checkModelParams()
      Called when setParams is called, which should have new model params after the dialog was closed. Puts these into dlParams so they get serialised with rest of XML.

    • getParentDataBlock

      public PamDataBlock getParentDataBlock()
      Get the parent data block.
      Returns:
      the parent data block.

    • getNumClasses

      public int getNumClasses()
      Get the number of classes for the current classifier.
      Returns:
      the number of classes.

    • update

      public void update(int processingFlag)
      Called whenever offline processing is occurring
      Parameters:
      processingFlag -

    • getClassNameManager

      public DLClassNameManager getClassNameManager()

    • getAnnotationType

      public DLAnnotationType getAnnotationType()
      Convenience function to get the DLAnnotationType from the DLClassification process.
      Returns:
      the DLAnnotationType

    • getDataSelector

      public DataSelector getDataSelector()
      The current data selector.
      Returns:
      the current data selector.

    • createDataSelector

      public void createDataSelector(PamDataBlock<?> source)
      Get the data selector.
      Parameters:
      source - - the source data block

    • getDlClassifierChooser

      public DLClassifierChooser getDlClassifierChooser()
      Get the classifier chooser. The classifier chooser chooses which classifier use based on a selected file or URL.
      Returns:
      the classifier chooser.

    • getDownloadManager

      public DLDownloadManager getDownloadManager()
      Get the download manager for downloading models offline.
      Returns:
      the download manager.

    • getDefaultModelManager

      public DLDefaultModelManager getDefaultModelManager()
      Get the default model manager. This handles the default models that can be downloaded.
      Returns:
      the default model manager.

    • isGroupDetections

      public boolean isGroupDetections()
      Check whether group detections are being used. If true then detections are saved to a group and all detections within a segment are then passed to the deep learning classifier.

    • setGroupDetections

      public void setGroupDetections(boolean groupDetections)
      Set whether to use group detections. If true then detections are saved to a group and all detections within a segment are then passed to the deep learning classifier.
      Parameters:
      groupDetections - - true to use group detections.

    • getClipDataBlock

      public ClipDisplayDataBlock<ClipDataUnit> getClipDataBlock()
      Specified by:
      getClipDataBlock in interface ClipDisplayParent
      Returns:
      the data block who's clips are to be displayed

    • getDisplayName

      public String getDisplayName()
      Description copied from interface: ClipDisplayParent
      A name (though I don't think this ever gets used by current displays)
      Specified by:
      getDisplayName in interface ClipDisplayParent
      Returns:
      display name

    • getClipDecorations

      public ClipDisplayDecorations getClipDecorations(ClipDisplayUnit clipDisplayUnit)
      Description copied from interface: ClipDisplayParent
      Add additional functionality and controls to a clip display unit
      Specified by:
      getClipDecorations in interface ClipDisplayParent
      Parameters:
      clipDisplayUnit - display unit to decorate.

    • displaySettingChange

      public void displaySettingChange()
      Specified by:
      displaySettingChange in interface ClipDisplayParent