Uinique Data Identifiers
Unique Data Identifiers (UID’s)
Overview
A New system of uniquely identifying individual PAMGuard data units was introduced in Version 1.16.00.
The system of Unique Identifiers (UID’s) makes it easier to identify individual data units on the different PAMGuard displays and to link to data when manipulating it with other software such as Matlab.
Introducing the UID system involved significant changes to both the the PAMGuard binary file system and to the PAMGuard database. Older datasets will be automatically updated to the new format which is NOT BACKWARDS COMPATIBLE.
You are therefore strongly advised to back up databases and binary files before attempting to use the new version.
Stored Data
In the PAMGuard database, all entries will have an additional UID column.
In the binary store, all data units will contain an additional UID field in general header data for each data unit.
In the binary store the UID’s are stored as a 64 bit integer. In the database the UId’s will be stored as a 64 bit integer if the underlying database supports 64 bit integers, otherwise they will be stored as a 32 bit integer. 32 bit integers allow for up to 2 billion individual UID’s which will be more than enough for any reasonable amount of data. There are a small number of very special circumstances where we think that 64 bit storage will be necessary, for instance the monitoring of fixed hydrophone installations over periods of many years.
New data
UID’s will automatically be added to any new data collected with PAMGuard version 2.00.00 or later.
Converting old data
If older data, collected with versions prior to 1.16.00 are opened with a version supporting UID’s, then a dialog will be shown to inform you that you need to make the data conversion. If you don’t want to convert, then press ‘Cancel’ and exit PAMGuard immediately.
If you do decide to proceed with the conversion, the the UID Repair tool Dialog will be shown. For safety, binary data will be copied to a new folder as the conversion takes place. By default, this will be the old folder name with “_withUID” appended to the folder name. You can change the new binary storage folder if you wish to at this point. If you still want to proceed, press OK and wait.
Conversion of a large dataset can take several hours. The time taken will depend on how much data there is, but also how many click events have been marked in the click detector. These pose a particular problem for conversion since the same UID has to be written to both the binary files and the OfflineClicks database table.
When conversion is complete, it’s generally a good idea to restart PAMGuard to ensure that all datamaps are correct