Package rawDeepLearningClassifier

Class DLControl

java.lang.Object

PamController.PamControlledUnit

rawDeepLearningClassifier.DLControl

All Implemented Interfaces:

ClipDisplayParent, PamSettings, SettingsNameProvider

public class DLControl
extends PamControlledUnit
implements PamSettings, ClipDisplayParent

Module which uses an external deep learning classifier to identify any data unit containing raw data.

For example, the module can be used to apply a deep learning model to analyse raw sound data, clips or click detections.

The module is essentially a PAMGuard wrapper for the jdl4pam library which does most of the working in loading models and setting up spectrogram transforms so models receive the correct input data. The majority of the code for the module involves interfacing with PAMGuard's data management system (e.g. binary files), visualisation tools such as spectrogram graphics and the TDDisplayFX and creating a settings GUI so users can load models easily and visualise how they work.

Currently the jdl4pam library supports three types of deep learning model, Generic, AnimalSpot and Ketos.

Generic
Generic models allows users to load almost any type of model and manually assign the types of data transform and input shape. This means that the user has to get the settings exactly right or the model will not work. It is the most complex way to run a deep learning model and generally not recommenced. However, users can export a settings file which makes it easier to set up for another user.

AnimalSpot
AnimalSpot is a framework for training acoustic deep learning models using Pytorch. Users can load a .py model which contains embedded metadata so that PMAGuard knows the exact transforms required for the model input. This makes deploying models in PAMGuard very easy - users require little or no experience to get this working.

Ketos
Ketos is a framework for training acoustic deep learning models using TensorFlow. Users can load a .ktpb model which contains embedded metadata so that PMAGuard knows the exact transforms required for the model input. Like AnimalSpot, this makes deploying Ketos models in PAMGuard very straightforward.

Author:

Jamie Macaulay

Field Summary

Fields

Modifier and Type	Field	Description
static final int	NEW_PARAMS	Flag to indicate a setup is required
static final boolean	PLUGIN_BUILD	PLUGIN_BUILD boolean is set to true so that the class loader isn't changed.
static final int	PROCESSING_END
static final int	PROCESSING_START	Flag for processing start

Fields inherited from class PamController.PamControlledUnit

isMixed, isViewer

Constructor Summary

Constructors

Constructor	Description
DLControl(String unitName)	Constructor for the DL Control.

Method Summary

Modifier and Type	Method	Description
void	checkModelParams()	Called when setParams is called, which should have new model params after the dialog was closed.
void	createDataSelector(PamDataBlock<?> source)	Get the data selector.
JMenuItem	createDetectionMenu(Frame parentFrame)	Create a JMenu object containing MenuItems associated with PamProcesses
void	displaySettingChange()
DLAnnotationType	getAnnotationType()	Convenience function to get the DLAnnotationType from the DLClassification process.
DLClassNameManager	getClassNameManager()
ClipDisplayDataBlock<ClipDataUnit>	getClipDataBlock()
ClipDisplayDecorations	getClipDecorations(ClipDisplayUnit clipDisplayUnit)	Add additional functionality and controls to a clip display unit
DataSelector	getDataSelector()	The current data selector.
DLDefaultModelManager	getDefaultModelManager()	Get the default model manager.
String	getDisplayName()	A name (though I don't think this ever gets used by current displays)
DLClassifierChooser	getDlClassifierChooser()	Get the classifier chooser.
DLClassifyProcess	getDLClassifyProcess()	Get the deep learning classification process.
DLClassiferModel	getDLModel()	Get the current deep learning model.
DLClassiferModel	getDLModel(String string)	Get a model by it's name.
ArrayList<DLClassiferModel>	getDLModels()	Get the available deep learning models
RawDLParams	getDLParams()	Get basic parameters.
DLDownloadManager	getDownloadManager()	Get the download manager for downloading models offline.
PamControlledUnitGUI	getGUI(int flag)	Get the GUI for the PAMControlled unit.
int	getNumClasses()	Get the number of classes for the current classifier.
PamDataBlock	getParentDataBlock()	Get the parent data block.
SegmenterProcess	getSegmenter()	Get the segmenter process.
DLSettingsPane	getSettingsPane()	Get the settings pane.
Serializable	getSettingsReference()
long	getSettingsVersion()
PamSidePanel	getSidePanel()	Gets a reference to a small panel to be displayed along the left hand edge of the main tab panel.
boolean	isGroupDetections()	Check whether group detections are being used.
boolean	restoreSettings(PamControlledUnitSettings pamControlledUnitSettings)
void	setGroupDetections(boolean groupDetections)	Set whether to use group detections.
void	setParams(RawDLParams newParams)
void	showSettingsDialog(Frame parentFrame)	Show settings dialog.
void	update(int processingFlag)	Called whenever offline processing is occurring

Methods inherited from class PamController.PamControlledUnit

addOfflineTaskGroup, addOtherRelatedMenuItems, addPamProcess, addPamProcess, addRelatedMenuItems, canClose, canPlayViewerSound, createDisplayMenu, createFileMenu, createHelpMenu, flushDataBlockBuffers, getBackupInformation, getFrameNumber, getGuiFrame, getInstanceIndex, getModuleStatus, getModuleStatusManager, getModuleSummary, getModuleSummary, getNumOfflineTaskGroups, getNumPamProcesses, getOfflineState, getOfflineTaskGroup, getPamConfiguration, getPamController, getPamModuleInfo, getPamProcess, getPamView, getPlugin, getShortUnitType, getTabClipCopier, getTabPanel, getTabSpecificMenuBar, getToolbarComponent, getUnitName, getUnitTaskManager, getUnitTaskManager, getUnitType, getVerboseLevel, gotoTab, isInMainConfiguration, isNetRx, isViewer, notifyModelChanged, pamClose, pamHasStopped, pamToStart, playViewerSound, removePamProcess, removeUnit, rename, saveViewerData, setFrameNumber, setModuleStatusManager, setPamConfiguration, setPamController, setPamModuleInfo, setPamView, setSidePanel, setTabPanel, setToolbarComponent, setUnitTaskManager, setupControlledUnit, stopViewerSound, tellModule, terminalPrint, terminalPrintln, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface PamController.PamSettings

getUnitType

Methods inherited from interface PamController.SettingsNameProvider

getUnitName

Field Details

PLUGIN_BUILD

public static final boolean PLUGIN_BUILD

PLUGIN_BUILD boolean is set to true so that the class loader isn't changed. When ready to compile into a fatjar set to true. When using eclipse set to false.

See Also:

• Constant Field Values

PROCESSING_START

public static final int PROCESSING_START

Flag for processing start

See Also:

• Constant Field Values

NEW_PARAMS

public static final int NEW_PARAMS

Flag to indicate a setup is required

See Also:

• Constant Field Values

PROCESSING_END

public static final int PROCESSING_END

See Also:

• Constant Field Values

Constructor Details

DLControl

public DLControl(String unitName)

Constructor for the DL Control.

Parameters:

unitName - - the unit name.

Method Details

getDLModels

public ArrayList<DLClassiferModel> getDLModels()

Get the available deep learning models

Returns:

the available deep learning models.

getDLModel

public DLClassiferModel getDLModel(String string)

Get a model by it's name.

Parameters:

the - name the model.

Returns:

the corresponding model object or null if no model with the name exists.

getDLModel

public DLClassiferModel getDLModel()

Get the current deep learning model.

Returns:

the current deep learning model.

getDLParams

public RawDLParams getDLParams()

Get basic parameters.

Returns:

parameters class.

getSettingsReference

public Serializable getSettingsReference()

Specified by:

getSettingsReference in interface PamSettings

Returns:

The serialisable object that will be stored

getSettingsVersion

public long getSettingsVersion()

Specified by:

getSettingsVersion in interface PamSettings

Returns:

An integer version number for the settings

restoreSettings

public boolean restoreSettings(PamControlledUnitSettings pamControlledUnitSettings)

Specified by:

restoreSettings in interface PamSettings

Parameters:

pamControlledUnitSettings -

Returns:

true if successful The object performs final checks (if needed) and then casts the settings data pamcontrolledunitSettings.settings into the correct type and uses as required

getSegmenter

public SegmenterProcess getSegmenter()

Get the segmenter process. This breaks raw data into chunks, combines into groups and sends to DL classifiers.

Returns:

the segmenter process.

getSettingsPane

public DLSettingsPane getSettingsPane()

Get the settings pane.

Returns:

the settings pane.

showSettingsDialog

public void showSettingsDialog(Frame parentFrame)

Show settings dialog.

Parameters:

parentFrame - - the frame.

getSidePanel

public PamSidePanel getSidePanel()

Description copied from class: PamControlledUnit

Gets a reference to a small panel to be displayed along the left hand edge of the main tab panel. Side panels should be small since they are always visible and any space they take will be taken from the main tab panel.

It is possible for a PamControlled unit to have a side panel without having a pamTabPanel.

Overrides:

getSidePanel in class PamControlledUnit

Returns:

a pamSidePanel object.

See Also:

• PamSidePanel
• PamTabPanel

createDetectionMenu

public JMenuItem createDetectionMenu(Frame parentFrame)

Description copied from class: PamControlledUnit

Create a JMenu object containing MenuItems associated with PamProcesses

Overrides:

createDetectionMenu in class PamControlledUnit

Parameters:

parentFrame - The owner frame of the menu

Returns:

reference to a JMenu which can be added to an existing menu or menu bar

Note that if multiple views are to use the same menu, then they should each create a new menu (by setting Create to true) the first time they call this method.

getDLClassifyProcess

public DLClassifyProcess getDLClassifyProcess()

Get the deep learning classification process. This handles running the current deep learning model.

Returns:

the deep learning classification process.

getGUI

public PamControlledUnitGUI getGUI(int flag)

Get the GUI for the PAMControlled unit. This has multiple GUI options which are instantiated depending on the view type.

Overrides:

getGUI in class PamControlledUnit

Parameters:

flag - . The GUI type flag defined in PAMGuiManager.

Returns:

the GUI for the PamControlledUnit unit.

setParams

public void setParams(RawDLParams newParams)

checkModelParams

public void checkModelParams()

Called when setParams is called, which should have new model params after the dialog was closed. Puts these into dlParams so they get serialised with rest of XML.

getParentDataBlock

public PamDataBlock getParentDataBlock()

Get the parent data block.

Returns:

the parent data block.

getNumClasses

public int getNumClasses()

Get the number of classes for the current classifier.

Returns:

the number of classes.

update

public void update(int processingFlag)

Called whenever offline processing is occurring

Parameters:

processingFlag -

getClassNameManager

public DLClassNameManager getClassNameManager()

getAnnotationType

public DLAnnotationType getAnnotationType()

Convenience function to get the DLAnnotationType from the DLClassification process.

Returns:

the DLAnnotationType

getDataSelector

public DataSelector getDataSelector()

The current data selector.

Returns:

the current data selector.

createDataSelector

public void createDataSelector(PamDataBlock<?> source)

Get the data selector.

Parameters:

source - - the source data block

getDlClassifierChooser

public DLClassifierChooser getDlClassifierChooser()

Get the classifier chooser. The classifier chooser chooses which classifier use based on a selected file or URL.

Returns:

the classifier chooser.

getDownloadManager

public DLDownloadManager getDownloadManager()

Get the download manager for downloading models offline.

Returns:

the download manager.

getDefaultModelManager

public DLDefaultModelManager getDefaultModelManager()

Get the default model manager. This handles the default models that can be downloaded.

Returns:

the default model manager.

isGroupDetections

public boolean isGroupDetections()

Check whether group detections are being used. If true then detections are saved to a group and all detections within a segment are then passed to the deep learning classifier.

setGroupDetections

public void setGroupDetections(boolean groupDetections)

Set whether to use group detections. If true then detections are saved to a group and all detections within a segment are then passed to the deep learning classifier.

Parameters:

groupDetections - - true to use group detections.

getClipDataBlock

public ClipDisplayDataBlock<ClipDataUnit> getClipDataBlock()

Specified by:

getClipDataBlock in interface ClipDisplayParent

Returns:

the data block who's clips are to be displayed

getDisplayName

public String getDisplayName()

Description copied from interface: ClipDisplayParent

A name (though I don't think this ever gets used by current displays)

Specified by:

getDisplayName in interface ClipDisplayParent

Returns:

display name

getClipDecorations

public ClipDisplayDecorations getClipDecorations(ClipDisplayUnit clipDisplayUnit)

Description copied from interface: ClipDisplayParent

Add additional functionality and controls to a clip display unit

Specified by:

getClipDecorations in interface ClipDisplayParent

Parameters:

clipDisplayUnit - display unit to decorate.

displaySettingChange

public void displaySettingChange()

Specified by:

displaySettingChange in interface ClipDisplayParent