Etere HSM archives files into LTO tapes using the Etere eXchange Format (EXF), a format able to store multiple objects (essence, metadata, etc) in one logical container including directory structure and permissions.
Currently, the structure used by Etere for archiving data on LTO tapes is defined as shown in the following model:
File mark
Header
Asset Form XML
File mark
File data
File mark
where:
▪File mark: An marker placed at the beginning, middle (between the xml and the file) and end of the file to separate the segments that compose it.
▪Header: A group of data (which size is 556 bytes in total) that details key information on the archived file:
Data Value :
Data name - StreamID
Data type : DWORD
Data size : 4 bytes
Data Value : Fixed to 1234567890
Data name - FileAttributeData
Data type : WIN32_FILE_ATTRIBUTE_DATA
Data size : 36 bytes
Data Value : Attribute information for a file or directory including "FileAttributes", "CreationTime", "LastAccessTime", "LastWriteTime", "FileSizeHigh" and "FileSizeLow".
NB: For detailed information refer to the link: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365739(v=vs.85).aspx
Data name - IdArchive
Data type : DWORD
Data size : 4 bytes
Data Value : Internal reference
Data name - OriginalPath
Data type : String [255]
Data size : 256 bytes
Data Value : Single-byte characters.
NB: The first byte stores the length of the string.
Data name - FileName
Data type : String [255]
Data size : 256 bytes
Data Value : Single-byte characters.
NB: The first byte stores the length of the string.
▪Asset Form XML: A XML file containing all data managed for the asset (e.g. ID, code, description, type, duration, timecodes, aspect, quality, audio, validity, parental guidance, related-season, artists, roles, genres, media, EDL, SMPTE and several more).
It's worth mentioning that this XML file is created through the Etere's XML Asset Full export function.
▪File Data: The essence of the asset content.
Consistency Check
When processing EXF file format, HSM Data Pump would perform a consistency check on data format before writing to database. The consistency check results would be written into log file for assessment. The consistency check includes:
•Start and End data block
•Data position on tape
•The congruent in file size
Some possible error messages resulted from consistency check are listed below:
! Consistency check! Archive was not performed: checked iniblock=0, endblock=0, remaining=0
! Consistency check! Archive was not performed: checked iniblock before=66079, after=66079
! Consistency check! Not enough written blocks: expected=6707, written=0
NB: Please note that in order to be considered as a valid structure, the Header.StreamID value must be 1234567890 and a File mark must be placed at the end (after the File data) of the file.
Note
The EXF has been discontinued from Etere version 25.2 onwards. Since then, tapes were written in Archive Exchange Format (AXF).