Storage Stream Library
Storage Stream Library is a component for use in Win32 and Win64 (Windows XP/Vista/7/8/10), OSX, iOS, Android and Linux software.
Reads and writes a versatile meta-data/binary stream format, for storing settings, presets or any kind of textual and/or binary data.
- Supports text and binary frames (with 11 built-in frame formats)
- Frames can be structured inside frames (multi-level hierarchy)
- Access directly all frame datas as a TMemoryStream (completely customized frame contents)
- Frames can be compressed and can be grouped individualy too
- Load the stream or dynamicaly access the stream frames on-demand from source medium
- Save the stream to a standard TStream, send the stream over network, store it in a database blob field, etc.
- 64 bit data sizes (supports files and content >4GB)
- CRC32 integrity checking on stream and frame level
- Fully unicode implementation
- Pure Delphi code, no external dependencies
- Delphi XE2 64bit and OSX, Delphi XE5 iOS and Android, Lazarus/Free pascal compatible
Storage Stream Library in shareware and commercial software?
The component is free for use in free software. If you like it and use it in a shareware or commercial (or any other money making - advertising, in app. selling, etc.) product you need one of the licenses.
Delphi 2009 and above.
To use the library with Lazarus/Free Pascal 'generics.collections.pas' is required (included in the 'Lazarus-FPC' folder, or download the latest version from: https://github.com/maciej-izak/generics.collections).
Loading and opening
The Storage Stream can be accessed in 2 modes: either loading the whole stream into memory (LoadFromFile(), LoadFromStream()) or just scan the stream and parse the frame structure (OpenFile()/OpenStream()) and access the frame streams directly from the source medium (file/memory).
The latter can be used for "unlimited" size files as the frame stream data is not loaded into memory at all and only access a frame content on-demand.
If you opened the file stream in read and write mode it's not possible to modify the frame's content size but it's possible to overwrite it.
It's not possible to use the setter functions on a stream that was opened (not loaded), but you can call the frame's Deflash() method to load the frame content into memory and change it after if needed - the setter functions will work then.
Note: When opening a stream with OpenStream() the TStream must be alive until calling Close() or freeing the TStorageStream class.
Frame name encoding
By default frame names are written as unicode strings (UTF-16). Set 'FrameNameEncoding := ssfneUTF8;' to write UTF-8 frame IDs. UTF-8 format results smaller file sizes if no unicode characters are used for frame names, but requires a bit more processing so is a bit slower.
The library automatically reads the frame names according to the encoding specified in the stream.
There is a global 'StorageStreamGlobalDefaultFrameNameEncoding' variable that all newly created TSorageStream classes use automatically. 'TSorageStream.Clear' re-sets the encoding to the global default value.
The root setter functions create new frame if one does not exists, or change the frame content if a frame specified by FrameID exists. Both functions work on frames belonging to the currently set 'GroupingContext' (see below 'Grouping').
To explicilty add a new text frame belonging to group 3 for example:
StorageStream.AddFrame('My new frame').SetText('Some text').GroupIdentifier := 3;
When using the SetStream() setter functions the 'DataStream' TStream object must be alive until calling 'TSorageStream.Clear' or freeing the 'TSorageStream' object.
Use the provided AddFrame(), AddFrameFromFile(), AddFrameVirtual(), DeleteFrame(), DeleteAllFrames() and Clear() functions for manipulating the frames list. If you manipulate the Frames TList manually (eg. exchange items) always call ReIndexFrameList() afterwards so that to have the frames' Indexes valid.
Every frame has a 'GroupIdentifier' which is 0 by default. If 'GroupingIdentity' is True the 'GroupIdentifier' is stored when saving the stream.
TStorageStream has a 'GroupingContext' value that can be used to automaticaly work with the specified frames that belong to this group. Set 'GroupingContext' to a non-zero value and all newly added frames will have this 'GroupIdentifier'. All frame ID based look-up (getter and setter) functions will only return frames that belong to this group. On the contrary 'Index' based getter and setter function always work directly on frames specified by 'Index'.
'GroupingContext' is used in frame's context, to work with sub-frames, 'GroupingContext' needs to be set for the parent frame that has sub-frames that you wish to work with.
Major version number is used internaly by the library, use the 'MinorVersion' to specify version of your stream before saving it.
When a new instance is created the 'MajorVersion' is automaticaly set to the latest implementation version. Saving the stream creates a stream with the latest version format. To save the stream with a legacy format set 'MajorVersion' to the desired legacy version number before saving the stream.
When loading an existing stream 'MajorVersion' is set to the loaded stream format. To re-save the stream in the latest version format set 'MajorVersion := STORAGE_STREAM_FORMAT_LATEST_VERSION;' before re-saving the stream.
CRC32 error checking can be used on stream level and on frame level, both at the same time if needed.
Stream level: To calculate and write CRC32 for the stream set 'StorageStream.CRC.WriteCRC := True;'. When loading/opening a stream that has CRC32 specified in the stream, enable validation with 'StorageStream.CRC.CRCStatusScan := sscrcssValidate;' before loading or opening a stream. If this variable is set to 'sscrcssOmit' (default), then the CRC check will not be performed, the CRC32 value will be just parsed and not validated. Validating the CRC requires more processing so it's slower.
Frame level: To write CRC32 values for individual frames, enable the frame's 'CRCWrite := True'. Do this for all particular frames. To set this for all frames (and sub-frames) use SetWriteCRC32ForFrames(True), or to disable CRC32 writing 'SetWriteCRC32ForFrames(False)'. To check if a frame's CRC is valid use 'TStorageStreamFrame.CRC32Valid'. There is also a callback function, 'OnFrameCRC32Error' event for the base class, if that's assigned it will be called with the invalid CRC32 frame as the parameter on load/open if there is a CRC mismatch.
Turning on and using CRC functions (or assigning the 'OnFrameCRC32Error' event) slows down loading and saving functionality and is off by default.