Binary Storage
Binary Storage
Overview
The database is not well suited for all types of data, particularly those such as clicks or whistle contours which have a varying length. The binary storage system stores PAMGuard output in a proprietary data format which is simple and efficient and can handle any type of data.
Data stored using the binary store can be reviewed with the PAMGuard viewer. It is also possible to open the binary files using your own software written in other programming languages such as Matlab or R.
Adding a Binary Storage module
From the File>Add Modules>Utilities menu select ‘Binary Storage’ to add binary storage to your configuration.
Configuring the Binary Store
To configure the binary store, go to the File>Binary Storage… menu to open the Binary Storage dialog.
Select the folder where you want to store the data and optionally set the other three options.
Storing data in folders arranged by date can help with data backup and archiving.
Some PAMGuard detectors make additional measures of background noise. These can be stored either with the binary data in the pgdf files or in separate pgnf files (see below). The former has the advantage of creating fewer files, the latter may make it easier to load and process noise data (e.g. to assess detector performance) in R or Matlab.
Automatically starting new files is important in order that individual files do not get excessively large.
The binary store can start new files at fixed time intervals, when files reach a certain size or both (i.e. which ever condition occurs first).
Output Files
Output files are identified by a long name which contains the module type, the module name, the datablock name, the date and the time. e.g. Clicks from a Click Detector called “Porpoise Detector” will have a name like “Click_Detector_Porpoise_Detector_Clicks_20100331_142400.pgdf”
pgdf Files
pgdf files are the main files which contain the binary data. All pgdf files have a common header and footer structure (i.e. the data at the start and at the end of each file). Individual units of data are however different for each module.
pgdf files can contain more than one type of data. In particular, some detectors also include regular measurements of background noise in their binary output.
pgnf Files
Some detectors (currently the Click detector the Whistle and Moan detector and the GPL detector) make and store measures of background noise. These can be important in assessing detector performance. The background data can either be stored with the detection data in the pgdf files or written to separate pgnf files. In either case, the stored data are the same, it’s a choice of fewer files or whether you want to be able to access noise data relatively quickly without having to load all binary data as well.
pgdx Files
pgdx files are indexing files. They contain headers and footers which are identical to those in the pgdf files, but no data. pgdx files speed up indexing processes at the start of a PAMGuard viewer session.
psfx Files
psfx files contain PAMGuard configuration data. These are basically the same data are as stored in the standard PAMGuard configuration files, but wrapped with a date and other information.
One of these is written each time PAMGuard is started, providing a permanent record of PAMGuard settings. psfx files can be loaded into the PAMGuard Viewer from the viewer data map or relaoded as with any other configuration file.