emCompress-ToGo User Guide & Reference Manual
On-target compression system.
Introducing emCompress-ToGo
This section presents an overview of emCompress-ToGo, its structure,
and its capabilities.
What is emCompress-ToGo?
emCompress-ToGo is a compression system that is able to reduce
the size of data that must be compressed or decompressed by a small
target microcontroller. Typical uses of emCompress-ToGo are:
- Compress and decompress communication data over a limited-bandwidth link.
- Decompress firmware images that must be dynamically expanded on device reprogramming.
- Decompress configuration bitstreams to program FPGA and CPLD devices.
- Permanent files for embedded web server static content.
Of course, emCompress-ToGo is not limited to these applications, it
can be used whenever it’s beneficial to reduce the size of dynamic data.
Features
emCompress-ToGo is written in standard ANSI C and can run on virtually
any CPU. Here’s a list summarizing the main features of emCompress-ToGo:
- Clean ISO/ANSI C source code.
- Small decompressor ROM footprint.
- Essentially zero-RAM compression and decompression.
- Easy-to-understand and simple-to-use API.
- Simple configuration.
Package content
emCompress-ToGo is provided in source code and contains everything required.
The following table shows the content of the emCompress-ToGo Package:
| Files | Description |
|---|
| Config | Configuration header files. |
| Doc | emCompress-ToGo documentation. |
| COMPRESS | emCompress-ToGo source code. |
| SEGGER | SEGGER software component source code used in emCompress-ToGo. |
| Windows | Supporting applications in binary form. |
Sample applications
emCompress-ToGo ships with a number of sample applications that show how
to integrate shell capability into your application. Each sample application
adds an additional capability and is a small step from the previous sample.
The sample applications are:
| Application | Description |
|---|
| CTG_M2M_Encode.c | Memory to memory compression. |
| CTG_M2M_Decode.c | Memory to memory decompression. |
| CTG_M2F_Encode.c | Memory to function compression. |
| CTG_F2M_Decode.c | Function to memory decompression. |
| CTG_F2M_Encode.c | Function to memory compression. |
| CTG_M2F_Decode.c | Memory to function decompression. |
| CTG_BenchEncode.c | Benchmark for all compression modes. |
| CTG_BenchDecode.c | Benchmark for all decompression modes. |
| CTG_Util.c | Compress and decompress files. |
| CTG_Test.c | Compressor and decompressor test application. |
| CTG_Optimize.c | Search for optimal compressor parameters. |
A note on the samples
Each sample that we present in this section is written in a style
that makes it easy to describe and that fits comfortably within the
margins of printed paper. Therefore, it may well be that you would
rewrite the sample to have a slightly different structure that fits
better, but please keep in mind that these examples are written with
clarity as the prime objective, and to that end we sacrifice some
brevity and efficiency.
Where to find the sample code
Target-independent code
All target-independent samples are included in the Application
directory of the emCompress-ToGo distribution.
The entry point for all target-dependent code is the function MainTask(),
which is usually run by the host operating system in a task context. To
run this code on a PC or workstation, simply call MainTask() from main():
void MainTask(void);
int main(int argc, char **argv) {
MainTask();
return 0;
}
Host-only code
All applications that require a file system and are intended to run
on a PC or workstation are included in the Windows/COMPRESS
directory of the emCompress-ToGo distribution.
Recommended project structure
We recommend keeping emCompress-ToGo separate from your application files.
It is good practice to keep all the program files (including the
header files) together in the COMPRESS subdirectory of your project’s
root directory. This practice has the advantage of being very easy to
update to newer versions of emCompress-ToGo by simply replacing the COMPRESS
and SEGGER directories. Your application files can be stored
anywhere.
Warning
When updating to a newer emCompress-ToGo version: as files may have
been added, moved or deleted, the project directories may need to be
updated accordingly.
Include directories
You should make sure that the include path contains the following directories
(the order of inclusion is of no importance):
Warning
Always make sure that you have only one version of each file!
It is frequently a major problem when updating to a new version of emCompress-ToGo
if you have old files included and therefore mix different versions. If you
keep emCompress-ToGo in the directories as suggested (and only in these), this
type of problem cannot occur. When updating to a newer version, you should
be able to keep your configuration files and leave them unchanged. For
safety reasons, we recommend backing up (or at least renaming) the
COMPRESS directories before updating.
Using emCompress-ToGo
emCompress-ToGo divides into two parts:
- A compressor that is responsible for compressing data using
SEGGER’s SMASH-2 algorithm, and
- A decompressor that is responsible for decompression data
compressed with the SMASH-2 algorithm.
Memory-to-memory compression
The most straightforward way of compressing data is to use a simple
memory-to-memory compression function. For this purpose, emCompress-ToGo
offers CTG_CompressM2M() where the “M2M” suffix indicates that the
source is memory and the destination is memory.
This example compresses some text and displays the compressed size:
/*********************************************************************
* (c) SEGGER Microcontroller GmbH *
* The Embedded Experts *
* www.segger.com *
**********************************************************************
-------------------------- END-OF-HEADER -----------------------------
File : CTG_M2M_Encode.c
Purpose : Compress memory to memory
*/
#include "CTG.h"
#include <stdio.h>
static const U8 _aInput[] = {
"Jabberwocky\n BY LEWIS CARROLL\n\n\n"
"'Twas brillig, and the slithy toves\n Did gyre and gimble in the wabe:\n"
"All mimsy were the borogoves,\n And the mome raths outgrabe.\n\n"
"\"Beware the Jabberwock, my son!\n The jaws that bite, the claws that catch!\n"
"Beware the Jubjub bird, and shun\n The frumious Bandersnatch!\"\n\n"
"He took his vorpal sword in hand;\n Long time the manxome foe he sought-\n"
"So rested he by the Tumtum tree\n And stood awhile in thought.\n\n"
"And, as in uffish thought he stood,\n The Jabberwock, with eyes of flame,\n"
"Came whiffling through the tulgey wood,\n And burbled as it came!\n\n"
"One, two! One, two! And through and through\n The vorpal blade went snicker-snack!\n\n"
"He left it dead, and with its head\n He went galumphing back.\n\n"
"\"And hast thou slain the Jabberwock?\n Come to my arms, my beamish boy!\n"
"O frabjous day! Callooh! Callay!\"\n\n He chortled in his joy.\n"
"'Twas brillig, and the slithy toves\n Did gyre and gimble in the wabe:\n"
"All mimsy were the borogoves,\n And the mome raths outgrabe.\n"
};
static U8 _aOutput[sizeof(_aInput)*2];
void MainTask(void);
void MainTask(void) {
int Status;
//
Status = CTG_CompressM2M(&_aInput[0], sizeof(_aInput),
&_aOutput[0], sizeof(_aOutput),
0);
if (Status >= 0) {
printf("Compressed %u to %d bytes.\n", sizeof(_aInput), Status);
} else {
printf("Compression error.\n");
}
}
Include emCompress-ToGo header
All of emCompress-ToGo’s API is made available by including
the single header file CTG.h.
Uncompressed input data
The array _aInput contains some text which is the
data to be compressed.
Compressed output
The array _aOutput will contain the compressed bitstream.
It is possible for some uncompressible data to expand, so the
output array is made double size in case this happens. (This
happens to be far larger than required: the maximum size that
uncompressible data expands to is documented in SMASH-2 format and bitstream expansion.)
Run compressor to compress data
The function CTG_CompressM2M() compresses the input, writes
the compressed bitstream, and delivers a status to the application
which indicates whether the compression succeeded or not.
The compression function is provided the input and output arrays,
their sizes, and an additional “flags” parameter that customizes
the compression algorithm: the default of zero is used here, and
this parameter is described in Adjusting compression parameters.
Interpret the result
Any negative status code is an error, indicating that compression
failed. The only failure that can happen, in this case, is that
the output array is not big enough to contain the compressed data.
We don’t expect compression to ever fail, as the output array is
of sufficient capacity here, but the status is decoded anyway.
Compile and test
To build the program, compile CTG_M2M_Encode.c with the emCompress-ToGo
source code in COMPRESS and SEGGER software components in
SEGGER. After it’s compiled and linked without errors, running
the example prints the uncompressed and compressed sizes:
C:> CTG_M2M_Encode.exe
Compressed 1011 to 700 bytes.
C:> _
Memory-to-memory decompression
Now that we have some compressed data, it’s time to decompress it.
The function CTG_DecompressM2M() uses the same memory-to-memory pattern.
This example compresses some text, decompresses it, and checks that
the decompressed image is identical to the image given to the compressor:
/*********************************************************************
* (c) SEGGER Microcontroller GmbH *
* The Embedded Experts *
* www.segger.com *
**********************************************************************
-------------------------- END-OF-HEADER -----------------------------
File : CTG_M2M_Decode.c
Purpose : Compress and decompress memory to memory
*/
#include "CTG.h"
#include <stdio.h>
static const U8 _aInput[] = {
"Jabberwocky\n BY LEWIS CARROLL\n\n\n"
"'Twas brillig, and the slithy toves\n Did gyre and gimble in the wabe:\n"
"All mimsy were the borogoves,\n And the mome raths outgrabe.\n\n"
"\"Beware the Jabberwock, my son!\n The jaws that bite, the claws that catch!\n"
"Beware the Jubjub bird, and shun\n The frumious Bandersnatch!\"\n\n"
"He took his vorpal sword in hand;\n Long time the manxome foe he sought-\n"
"So rested he by the Tumtum tree\n And stood awhile in thought.\n\n"
"And, as in uffish thought he stood,\n The Jabberwock, with eyes of flame,\n"
"Came whiffling through the tulgey wood,\n And burbled as it came!\n\n"
"One, two! One, two! And through and through\n The vorpal blade went snicker-snack!\n\n"
"He left it dead, and with its head\n He went galumphing back.\n\n"
"\"And hast thou slain the Jabberwock?\n Come to my arms, my beamish boy!\n"
"O frabjous day! Callooh! Callay!\"\n\n He chortled in his joy.\n"
"'Twas brillig, and the slithy toves\n Did gyre and gimble in the wabe:\n"
"All mimsy were the borogoves,\n And the mome raths outgrabe.\n"
};
static U8 _aOutput[sizeof(_aInput)*2];
static U8 _aMirror[sizeof(_aInput)];
void MainTask(void);
void MainTask(void) {
int EncodeLen;
int DecodeLen;
//
EncodeLen = CTG_CompressM2M(&_aInput[0], sizeof(_aInput),
&_aOutput[0], sizeof(_aOutput),
0);
if (EncodeLen < 0) {
printf("Compression error.\n");
} else {
printf("Compressed %u to %d bytes.\n", sizeof(_aInput), EncodeLen);
DecodeLen = CTG_DecompressM2M(&_aOutput[0], EncodeLen,
&_aMirror[0], sizeof(_aMirror));
if (DecodeLen < 0) {
printf("Decompression error.\n");
} else if (DecodeLen != sizeof(_aInput)) {
printf("Decompression expected-length error.\n");
} else {
printf("Decompressed %d to %d bytes.\n", EncodeLen, DecodeLen);
if (memcmp(_aInput, _aMirror, DecodeLen) == 0) {
printf("Images match!\n");
} else {
printf("Images do not match!\n");
}
}
}
}
Introduce a mirror
The array _aMirror is introduced to be a mirror-image of the input
data after the compression-decompression process. If the input and
mirrored input are not identical, there is an error in the
compression-decompression process.
Compress the input
This proceeds as in the previous example, and the length of the encoded
data is stored because we need it later.
Decompress the compressed data
Calling CTG_DecompressM2M() decompresses the compressed data. Notice
that we do not need to pass an additional “flags” parameter when
decompressing as the compression flags are stored, along with the
compressed image, in the bitstream to be recoverd by the decompressor.
Interpret the result
CTG_DecompressM2M() returns a status code the same as CTG_CompressM2M()
does. However, it is possible for the compressed bitstream to be
corrupt, so additional errors can be reported by the decompressor.
Check lengths
If the length of the decompressed data does not match the length of
the original data, that’s a problem. We don’t expect this to be the
case.
Report lengths
As a check, the decompression phase statistics are shown.
Check images mirror each other
Once the compressed bitstream has decompressed correctly, an additional
check ensures that the decompressed data is identical to the original
data. If this isn’t the case, there is an error in the compression-decompression
process.
Compile and run
To build the program, compile CTG_M2M_Decode.c with the emCompress-ToGo
source code in COMPRESS and SEGGER software components in
SEGGER. After it’s compiled and linked without errors, running
the example prints the uncompressed and compressed sizes:
C:> CTG_M2M_Decode.exe
Compressed 1011 to 700 bytes.
Decompressed 700 to 1011 bytes.
Images match!
C:> _
Memory-to-function compression
Rather than the all-at-once compression offered by CTG_CompressM2M(),
emCompress-ToGo can compress the input and write the output incrementally
through a user-provided function. In this way it’s possible for the
client, for instance, to write the output to a file or to memory, or
even to a network connection. A further example is compressing an FPGA
configuration bitstream where the uncompressed FPGA bitstream is too
large to fit into memory—in this case, the compressed bitstream is
decompressed and written to the FPGA by the user-provided function as
it is decompressed.
This example compresses some text and sends the compressed output
through a function which stores it into memory.
/*********************************************************************
* (c) SEGGER Microcontroller GmbH *
* The Embedded Experts *
* www.segger.com *
**********************************************************************
-------------------------- END-OF-HEADER -----------------------------
File : CTG_M2F_Encode.c
Purpose : Compress memory to function.
*/
#include "CTG.h"
#include <stdio.h>
static const U8 _aInput[] = {
"Jabberwocky\n BY LEWIS CARROLL\n\n\n"
"'Twas brillig, and the slithy toves\n Did gyre and gimble in the wabe:\n"
"All mimsy were the borogoves,\n And the mome raths outgrabe.\n\n"
"\"Beware the Jabberwock, my son!\n The jaws that bite, the claws that catch!\n"
"Beware the Jubjub bird, and shun\n The frumious Bandersnatch!\"\n\n"
"He took his vorpal sword in hand;\n Long time the manxome foe he sought-\n"
"So rested he by the Tumtum tree\n And stood awhile in thought.\n\n"
"And, as in uffish thought he stood,\n The Jabberwock, with eyes of flame,\n"
"Came whiffling through the tulgey wood,\n And burbled as it came!\n\n"
"One, two! One, two! And through and through\n The vorpal blade went snicker-snack!\n\n"
"He left it dead, and with its head\n He went galumphing back.\n\n"
"\"And hast thou slain the Jabberwock?\n Come to my arms, my beamish boy!\n"
"O frabjous day! Callooh! Callay!\"\n\n He chortled in his joy.\n"
"'Twas brillig, and the slithy toves\n Did gyre and gimble in the wabe:\n"
"All mimsy were the borogoves,\n And the mome raths outgrabe.\n"
};
static U8 _aOutput[sizeof(_aInput)*2];
typedef struct {
U8 * pOutput;
unsigned OutputLen;
unsigned Cursor;
} CTG_WR_CONTEXT;
static int _Wr(const U8 *pData, unsigned DataLen, void *pWrCtx) {
CTG_WR_CONTEXT * pCtx;
unsigned N;
//
pCtx = (CTG_WR_CONTEXT *)pWrCtx;
for (N = 0; N < DataLen; ++N) {
if (pCtx->Cursor >= pCtx->OutputLen) {
return CTG_STATUS_OUTPUT_OVERFLOW;
}
pCtx->pOutput[pCtx->Cursor++] = *pData++;
}
return N;
}
void MainTask(void);
void MainTask(void) {
CTG_WR_CONTEXT WrCtx;
int EncodeLen;
//
WrCtx.pOutput = &_aOutput[0];
WrCtx.OutputLen = sizeof(_aOutput);
WrCtx.Cursor = 0;
EncodeLen = CTG_CompressM2F(&_aInput[0], sizeof(_aInput),
_Wr, &WrCtx,
0);
if (EncodeLen < 0) {
printf("Compression error.\n");
} else {
printf("Compressed %u to %d bytes.\n", sizeof(_aInput), EncodeLen);
}
}
The “write context”
The user-defined type CTG_WR_CONTEXT contains state information
that is relevant by the output function. In this case there is
a pointer to the start of the output buffer, pOutput, the size
of the output buffer, OutputLen, and a pointer to the next
output position to be written, Cursor.
Accept write parameters
The function _Wr is responsible for taking the data given to it
and somehow acting upon it. In this case we store the data to the
output buffer, but other implementations could write the output to
a file or to a network connection.
Recover context
The incoming parameter pWrCtx is a blind pointer, but we know
that a pointer to the client’s “write context” is passed in, so the
pointer is unblinded by a cast.
Write data
A for loop iterates over the incoming data bytes and transfers
them to the output buffer. If the output buffer becomes full, an
error indication is returned which immediately terminates compression.
Set up write context
The write context is set up to deliver the compressed data into the
array _aOutput.
Compress the data
The call to CTG_CompressM2F() passes in the array to compress,
the function to call when compressed data are ready, and a pointer
to the user’s write context. On return, any error returned by the
write function is propagated to the client.
Compile and run
To build the program, compile CTG_M2F_Encode.c with the emCompress-ToGo
source code in COMPRESS and SEGGER software components in
SEGGER. After it’s compiled and linked without errors, running
the example prints the uncompressed and compressed sizes:
C:> CTG_M2F_Encode.exe
Compressed 1011 to 700 bytes.
C:> _
Function-to-memory decompression
Just as CTG_CompressM2F() can deliver its compressed data using a
function, decompression can receive its compressed data using a function.
In this way it’s possible for the client, for instance, to read the
input from a file or network connection and write the output to memory.
This example decompresses a valid bitstream delivered through repeated
function calls and stores it into memory.
/*********************************************************************
* (c) SEGGER Microcontroller GmbH *
* The Embedded Experts *
* www.segger.com *
**********************************************************************
-------------------------- END-OF-HEADER -----------------------------
File : CTG_F2M_Decode.c
Purpose : Decompress function to memory.
*/
#include "CTG.h"
#include <stdio.h>
static const U8 _aInput[] = {
0xA2, 0x24, 0x8C, 0x18, 0x31, 0x65, 0xE4, 0xDC, 0x79, 0x33, 0x66, 0x4D,
0x9E, 0x06, 0x0A, 0x40, 0x80, 0x10, 0x92, 0x05, 0x04, 0x93, 0x22, 0x57,
0x92, 0x4C, 0x01, 0x31, 0x24, 0x88, 0x14, 0x29, 0x4F, 0x98, 0x30, 0x45,
0x73, 0x09, 0x27, 0xA8, 0xDC, 0x09, 0x33, 0x07, 0x84, 0x18, 0x39, 0x69,
0xD8, 0xB0, 0x49, 0x73, 0x86, 0x05, 0x88, 0x30, 0x6E, 0xC8, 0x80, 0xA0,
0x83, 0xA6, 0x0C, 0x88, 0xB9, 0xA4, 0x13, 0x93, 0x97, 0xA8, 0x37, 0x76,
0xCA, 0xCC, 0xDD, 0xF8, 0x05, 0x10, 0x49, 0x8B, 0xD6, 0x99, 0x3C, 0x72,
0x11, 0xDF, 0x88, 0x33, 0x69, 0xDA, 0x88, 0x61, 0x4B, 0x54, 0x1A, 0xB7,
0x9F, 0x0B, 0x87, 0x30, 0xA5, 0x93, 0x1D, 0x41, 0x21, 0x01, 0x44, 0x5B,
0x84, 0xE6, 0x32, 0xEC, 0x2E, 0xC4, 0xA3, 0x3A, 0x25, 0x31, 0x6F, 0xE4,
0xBC, 0x39, 0x3B, 0x16, 0x58, 0x86, 0x34, 0x04, 0x6D, 0x8A, 0xDA, 0xBC,
0x69, 0x4B, 0x88, 0x9C, 0xB0, 0x44, 0x43, 0xDF, 0x9B, 0x3A, 0x74, 0xCE,
0x92, 0x0C, 0x05, 0x2E, 0xD7, 0x08, 0x11, 0x42, 0xCA, 0x42, 0x67, 0xAF,
0x8F, 0x2B, 0x4C, 0x35, 0xDA, 0xC2, 0xCA, 0x9C, 0x37, 0x6E, 0x42, 0x86,
0x34, 0x50, 0x55, 0x4E, 0x4D, 0x98, 0xBB, 0x8C, 0x67, 0x27, 0x1C, 0x42,
0x33, 0x75, 0x29, 0xCB, 0xC4, 0x4D, 0x1A, 0xC3, 0xB6, 0x14, 0x19, 0x4B,
0xC6, 0x18, 0xB4, 0x0E, 0x0E, 0x24, 0xA4, 0x8E, 0x18, 0xB5, 0x44, 0x3A,
0x47, 0x0E, 0xD9, 0x2C, 0xDB, 0x1C, 0x34, 0x75, 0xDC, 0x06, 0x52, 0x62,
0x46, 0x4E, 0x5D, 0x1A, 0x4C, 0x1F, 0x61, 0x40, 0xA8, 0xEA, 0x50, 0x37,
0xC7, 0xED, 0x39, 0x10, 0xB9, 0x62, 0x20, 0x95, 0x82, 0x37, 0x6F, 0xD6,
0x80, 0x40, 0x93, 0x16, 0x35, 0xBB, 0xD4, 0x0A, 0x4E, 0x58, 0xCA, 0x36,
0x97, 0x6A, 0x42, 0xE4, 0x4C, 0x10, 0xBA, 0x59, 0xB3, 0x33, 0xE4, 0x81,
0x29, 0x35, 0x39, 0xCB, 0x54, 0x7C, 0x30, 0x29, 0x6B, 0x8B, 0x14, 0xDE,
0x55, 0xCD, 0xCC, 0x5B, 0x32, 0xE7, 0xAC, 0x90, 0x73, 0x06, 0x0D, 0x9D,
0xCB, 0x6C, 0x99, 0xF2, 0x96, 0xF6, 0x63, 0x8B, 0xD4, 0x96, 0xFD, 0xB3,
0x49, 0xBD, 0xD6, 0x18, 0xAA, 0x90, 0x47, 0x97, 0x48, 0xA2, 0x11, 0x2A,
0x1B, 0x68, 0x97, 0xAC, 0x84, 0x77, 0x52, 0xE9, 0x35, 0xCC, 0x87, 0x96,
0x7B, 0x12, 0x6F, 0x97, 0xEA, 0xDB, 0x64, 0x1A, 0xA9, 0x4C, 0x1D, 0x33,
0x66, 0xA9, 0x06, 0xDA, 0x50, 0xFD, 0x64, 0x6F, 0x78, 0xE9, 0xBC, 0xEB,
0x77, 0xA0, 0xA3, 0xDC, 0xD5, 0xE3, 0x40, 0x94, 0xC9, 0x4B, 0x09, 0x31,
0x20, 0xB3, 0xD4, 0x13, 0xDF, 0xA4, 0xA4, 0x4E, 0x19, 0x6A, 0x2C, 0xCA,
0x30, 0x0D, 0x17, 0x6C, 0xA1, 0xD4, 0x1A, 0xA1, 0xC5, 0xBF, 0x65, 0xF3,
0xED, 0xA4, 0x1A, 0x6C, 0xCE, 0x32, 0x17, 0xF1, 0x40, 0x4E, 0x57, 0x0F,
0x31, 0x75, 0xE4, 0xFA, 0x05, 0x2D, 0xA8, 0x2D, 0x9C, 0xEB, 0x42, 0xA0,
0xA6, 0x4B, 0xC2, 0x13, 0xB7, 0xB3, 0x64, 0xD6, 0x84, 0x00, 0x19, 0x27,
0xA7, 0xF9, 0x27, 0xA5, 0xF9, 0x35, 0x9F, 0x1D, 0xD8, 0xD0, 0xB6, 0x56,
0x18, 0x09, 0x8B, 0xFF, 0xFA, 0x23, 0xDC, 0xC2, 0x28, 0x26, 0x94, 0x96,
0xBA, 0x52, 0x9A, 0x96, 0x9A, 0x31, 0xB9, 0x52, 0xAB, 0x11, 0x43, 0x97,
0x59, 0x74, 0x25, 0x9B, 0x89, 0xAC, 0x4C, 0x23, 0x7F, 0x99, 0xC9, 0xA4,
0x82, 0x50, 0x88, 0x64, 0xC8, 0xA7, 0x4B, 0xCF, 0xC0, 0x59, 0x08, 0xC5,
0x56, 0xE0, 0x52, 0x6A, 0x0B, 0x25, 0x56, 0x0E, 0xF3, 0x67, 0xAE, 0x8B,
0xF8, 0x36, 0x16, 0x79, 0xC3, 0xF4, 0xF1, 0x84, 0xED, 0x33, 0x1B, 0x5B,
0xC4, 0xCF, 0x90, 0x87, 0xA1, 0x33, 0x70, 0xBC, 0x74, 0x1F, 0x24, 0x8C,
0x9C, 0x36, 0x73, 0x7F, 0x46, 0xD9, 0x48, 0xD1, 0x67, 0x37, 0x62, 0xDE,
0xE4, 0xB5, 0x13, 0x4F, 0x7D, 0x8E, 0xEC, 0x52, 0xBB, 0xE7, 0x40, 0x26,
0x2C, 0x82, 0x2C, 0x86, 0x31, 0xB6, 0xF8, 0x04, 0xDA, 0x0C, 0x56, 0xD0,
0x07, 0x59, 0x56, 0x62, 0x2C, 0x7C, 0xC9, 0xA1, 0x3B, 0xC3, 0xDC, 0x71,
0xF4, 0x4C, 0xB2, 0x2A, 0xEF, 0x1B, 0x19, 0x14, 0xF8, 0xE2, 0x55, 0x55
};
static U8 _aOutput[sizeof(_aInput)*2];
typedef struct {
const U8 * pInput;
unsigned InputLen;
unsigned Cursor;
} CTG_RD_CONTEXT;
static int _Rd(U8 *pData, unsigned DataLen, void *pRdCtx) {
CTG_RD_CONTEXT * pCtx;
unsigned N;
//
pCtx = (CTG_RD_CONTEXT *)pRdCtx;
for (N = 0; N < DataLen; ++N) {
if (pCtx->Cursor >= pCtx->InputLen) {
break;
}
*pData++ = pCtx->pInput[pCtx->Cursor++];
}
return N;
}
void MainTask(void);
void MainTask(void) {
CTG_RD_CONTEXT RdCtx;
int DecodeLen;
//
RdCtx.pInput = &_aInput[0];
RdCtx.InputLen = sizeof(_aInput);
RdCtx.Cursor = 0;
DecodeLen = CTG_DecompressF2M(_Rd, &RdCtx,
&_aOutput[0], sizeof(_aOutput));
if (DecodeLen < 0) {
printf("Decompression error.\n");
} else {
printf("Decompressed %u to %d bytes.\n", sizeof(_aInput), DecodeLen);
}
}
The “read context”
The user-defined type CTG_RD_CONTEXT contains state information
that is relevant by the input function. In this case there is
a pointer to the start of the input buffer, pInput, the size
of the input buffer, InputLen, and a pointer to the next
input position to be read, Cursor.
Accept read parameters
The function _Rd is responsible for reading source data and
returning the number of bytes read to the caller. In this case
we read the data from the input buffer, but other implementations
could read from a file or from a network connection.
Recover context
The incoming parameter pRdCtx is a blind pointer, but we know
that a pointer to the client’s “read context” is passed in, so the
pointer is unblinded by a cast.
Read data
Return as much data as possible up to ’DataLen’.
The function returns less bytes as ’DataLen’ or even 0 at the
end of the input data.
Set up read context
The read context is set up to read the compressed data from the
array _aInput.
Decompress the data
The call to CTG_DecompressF2M() passes in the function to call
when more data are required, a pointer to the user’s read context,
and the array that receives the decompressed data. On return,
any error returned by the read function is propagated to the client.
Compile and run
To build the program, compile CTG_F2M_Decode.c with the emCompress-ToGo
source code in COMPRESS and SEGGER software components in
SEGGER. After it’s compiled and linked without errors, running
the example prints the uncompressed and compressed sizes:
C:> CTG_F2M_Decode.exe
Decompressed 576 to 1089 bytes.
C:> _
Adjusting compression parameters
In all examples the Flags parameter has been set to zero. This
parameter adjusts how the compressor works and affects its runtime performance,
i.e. how long it takes to find matches, and the compression ratio.
As you might imagine, better compression
takes longer, and in a real-time system you may wish to favor compression
speed to reduce latency on a communications channel or you may wish to
favor size when writing sensor data or a log file to a file system.
Currently the flags passed into the compressor control the window size.
Window size
The window size affects how much history is examined to find and compress
a “string match”: when a substring is received that matches part of the
history, a reference to the match position is encoded in place of the
matched string. Clearly, having more history provides more opportunities
for a match but also reduces the runtime performance of the compressor
because more history needs to be searched. There is also another
consequence of larger windows: with larger windows, you have longer
distances which, when encoded, require more bits in the output bitstream.
emCompress-ToGo has the capability of selecting history sizes from 256 bytes to
16 KB in powers of two. For low-latency real time systems, a small history
of 256 bytes to 1 KB should suffice. For good compression, a medium history
of 2KB or 4 KB is an excellent choice.
If this parameter is not specified, it defaults to a match distance of
256 bytes.
See Compressor encoding flags for the compression flags you can select.
A small example
The effect of the window size upon compression for a sample input file (HTML
code, size 324751 bytes) is shown in the following table.
| Window size | Compressed file size | |
|---|
| 256 bytes | 121586 bytes | 37.4 % |
| 512 bytes | 103809 bytes | 31.9 % |
| 1k | 91455 bytes | 28.1 % |
| 2k | 67199 bytes | 20.6 % |
| 4k | 51234 bytes | 15.7 % |
| 8k | 48802 bytes | 15.0 % |
| 16k | 47588 bytes | 14.6 % |
Function-to-memory compression
The compression modes function-to-memory and function-to-function
require an additional parameter of a “work buffer” to maintain history
information to make forward match decisions. These modes are useful when
the uncompressed data cannot fit into the target’s memory or the source
data is stored externally, such as in a file or in a serial flash, and
can only be read incrementally.
The size of this work buffer can be determined using the macro
CTG_COMPRESS_WS_SIZE. The compression functions
ensure that the work buffer is large enough and is consistent with the selected
compression flags: if there is a problem, the compression function returns
an error.
The functions CTG_CompressF2M() and CTG_CompressF2F() use the same
functional parameters seen in function-to-memory decompression, and as
such this mechanism is not described further. The example CTG_F2M_Encode.c
demonstrates the concepts that have been used before and shows how the
buffer is provided:
/*********************************************************************
* (c) SEGGER Microcontroller GmbH *
* The Embedded Experts *
* www.segger.com *
**********************************************************************
-------------------------- END-OF-HEADER -----------------------------
File : CTG_F2M_Encode.c
Purpose : Compress function to memory
*/
#include "CTG.h"
#include <stdio.h>
static const U8 _aInput[] = {
"Jabberwocky\n BY LEWIS CARROLL\n\n\n"
"'Twas brillig, and the slithy toves\n Did gyre and gimble in the wabe:\n"
"All mimsy were the borogoves,\n And the mome raths outgrabe.\n\n"
"\"Beware the Jabberwock, my son!\n The jaws that bite, the claws that catch!\n"
"Beware the Jubjub bird, and shun\n The frumious Bandersnatch!\"\n\n"
"He took his vorpal sword in hand;\n Long time the manxome foe he sought-\n"
"So rested he by the Tumtum tree\n And stood awhile in thought.\n\n"
"And, as in uffish thought he stood,\n The Jabberwock, with eyes of flame,\n"
"Came whiffling through the tulgey wood,\n And burbled as it came!\n\n"
"One, two! One, two! And through and through\n The vorpal blade went snicker-snack!\n\n"
"He left it dead, and with its head\n He went galumphing back.\n\n"
"\"And hast thou slain the Jabberwock?\n Come to my arms, my beamish boy!\n"
"O frabjous day! Callooh! Callay!\"\n\n He chortled in his joy.\n"
"'Twas brillig, and the slithy toves\n Did gyre and gimble in the wabe:\n"
"All mimsy were the borogoves,\n And the mome raths outgrabe.\n"
};
static U8 _aOutput[sizeof(_aInput)*2];
typedef struct {
const U8 * pInput;
unsigned InputLen;
unsigned Cursor;
} CTG_RD_CONTEXT;
static int _Rd(U8 *pData, unsigned DataLen, void *pRdCtx) {
CTG_RD_CONTEXT * pCtx;
unsigned N;
//
pCtx = (CTG_RD_CONTEXT *)pRdCtx;
for (N = 0; N < DataLen; ++N) {
if (pCtx->Cursor >= pCtx->InputLen) {
break;
}
*pData++ = pCtx->pInput[pCtx->Cursor++];
}
return N;
}
void MainTask(void);
void MainTask(void) {
CTG_RD_CONTEXT RdCtx;
int Status;
U8 aWork[CTG_COMPRESS_WS_SIZE(CTG_FLAG_WINDOW_SIZE_1K)];
//
RdCtx.pInput = &_aInput[0];
RdCtx.InputLen = sizeof(_aInput);
RdCtx.Cursor = 0;
//
Status = CTG_CompressF2M(_Rd, &RdCtx,
&_aOutput[0], sizeof(_aOutput),
&aWork[0], sizeof(aWork),
CTG_FLAG_WINDOW_SIZE_1K);
if (Status >= 0) {
printf("Compressed %u to %d bytes.\n", sizeof(_aInput), Status);
} else {
printf("Compression error.\n");
}
}
Declare work buffer
The work buffer is sized to the compression parameters we will select.
Provide read context
The “read function and context” mechanism is identical to the
decompress function-to-memory example.
Provide work buffer
The work buffer and its size are provided to the compression function.
Compress with nondefault flags
The selected flag of a 1 KB maximum match distance is
consistent with the work buffer provided to the function.
Memory-to-function decompression
The decompression modes memory-to-function and function-to-function
require an additional parameter of a “work buffer” to maintain history
information to copy backward references.
The size of this work buffer can be determined using the macro
CTG_DECOMPRESS_WS_SIZE.
The decompression functions ensure that the work buffer is large enough
and is consistent with the selected compression flags: if there is a problem,
the decompression function returns an error.
The functions CTG_DecompressM2F() and CTG_DecompressF2F() use the same
functional parameters seen before and this mechanism is not described further.
The example CTG_M2F_Decode.c demonstrates the concepts that have been used
before and shows how the buffer is provided:
/*********************************************************************
* (c) SEGGER Microcontroller GmbH *
* The Embedded Experts *
* www.segger.com *
**********************************************************************
-------------------------- END-OF-HEADER -----------------------------
File : CTG_M2F_Decode.c
Purpose : Decompress memory to function.
*/
#include "CTG.h"
#include <stdio.h>
static const U8 _aInput[] = {
0xA2, 0x24, 0x8C, 0x18, 0x31, 0x65, 0xE4, 0xDC, 0x79, 0x33, 0x66, 0x4D,
0x9E, 0x06, 0x0A, 0x40, 0x80, 0x10, 0x92, 0x05, 0x04, 0x93, 0x22, 0x57,
0x92, 0x4C, 0x01, 0x31, 0x24, 0x88, 0x14, 0x29, 0x4F, 0x98, 0x30, 0x45,
0x73, 0x09, 0x27, 0xA8, 0xDC, 0x09, 0x33, 0x07, 0x84, 0x18, 0x39, 0x69,
0xD8, 0xB0, 0x49, 0x73, 0x86, 0x05, 0x88, 0x30, 0x6E, 0xC8, 0x80, 0xA0,
0x83, 0xA6, 0x0C, 0x88, 0xB9, 0xA4, 0x13, 0x93, 0x97, 0xA8, 0x37, 0x76,
0xCA, 0xCC, 0xDD, 0xF8, 0x05, 0x10, 0x49, 0x8B, 0xD6, 0x99, 0x3C, 0x72,
0x11, 0xDF, 0x88, 0x33, 0x69, 0xDA, 0x88, 0x61, 0x4B, 0x54, 0x1A, 0xB7,
0x9F, 0x0B, 0x87, 0x30, 0xA5, 0x93, 0x1D, 0x41, 0x21, 0x01, 0x44, 0x5B,
0x84, 0xE6, 0x32, 0xEC, 0x2E, 0xC4, 0xA3, 0x3A, 0x25, 0x31, 0x6F, 0xE4,
0xBC, 0x39, 0x3B, 0x16, 0x58, 0x86, 0x34, 0x04, 0x6D, 0x8A, 0xDA, 0xBC,
0x69, 0x4B, 0x88, 0x9C, 0xB0, 0x44, 0x43, 0xDF, 0x9B, 0x3A, 0x74, 0xCE,
0x92, 0x0C, 0x05, 0x2E, 0xD7, 0x08, 0x11, 0x42, 0xCA, 0x42, 0x67, 0xAF,
0x8F, 0x2B, 0x4C, 0x35, 0xDA, 0xC2, 0xCA, 0x9C, 0x37, 0x6E, 0x42, 0x86,
0x34, 0x50, 0x55, 0x4E, 0x4D, 0x98, 0xBB, 0x8C, 0x67, 0x27, 0x1C, 0x42,
0x33, 0x75, 0x29, 0xCB, 0xC4, 0x4D, 0x1A, 0xC3, 0xB6, 0x14, 0x19, 0x4B,
0xC6, 0x18, 0xB4, 0x0E, 0x0E, 0x24, 0xA4, 0x8E, 0x18, 0xB5, 0x44, 0x3A,
0x47, 0x0E, 0xD9, 0x2C, 0xDB, 0x1C, 0x34, 0x75, 0xDC, 0x06, 0x52, 0x62,
0x46, 0x4E, 0x5D, 0x1A, 0x4C, 0x1F, 0x61, 0x40, 0xA8, 0xEA, 0x50, 0x37,
0xC7, 0xED, 0x39, 0x10, 0xB9, 0x62, 0x20, 0x95, 0x82, 0x37, 0x6F, 0xD6,
0x80, 0x40, 0x93, 0x16, 0x35, 0xBB, 0xD4, 0x0A, 0x4E, 0x58, 0xCA, 0x36,
0x97, 0x6A, 0x42, 0xE4, 0x4C, 0x10, 0xBA, 0x59, 0xB3, 0x33, 0xE4, 0x81,
0x29, 0x35, 0x39, 0xCB, 0x54, 0x7C, 0x30, 0x29, 0x6B, 0x8B, 0x14, 0xDE,
0x55, 0xCD, 0xCC, 0x5B, 0x32, 0xE7, 0xAC, 0x90, 0x73, 0x06, 0x0D, 0x9D,
0xCB, 0x6C, 0x99, 0xF2, 0x96, 0xF6, 0x63, 0x8B, 0xD4, 0x96, 0xFD, 0xB3,
0x49, 0xBD, 0xD6, 0x18, 0xAA, 0x90, 0x47, 0x97, 0x48, 0xA2, 0x11, 0x2A,
0x1B, 0x68, 0x97, 0xAC, 0x84, 0x77, 0x52, 0xE9, 0x35, 0xCC, 0x87, 0x96,
0x7B, 0x12, 0x6F, 0x97, 0xEA, 0xDB, 0x64, 0x1A, 0xA9, 0x4C, 0x1D, 0x33,
0x66, 0xA9, 0x06, 0xDA, 0x50, 0xFD, 0x64, 0x6F, 0x78, 0xE9, 0xBC, 0xEB,
0x77, 0xA0, 0xA3, 0xDC, 0xD5, 0xE3, 0x40, 0x94, 0xC9, 0x4B, 0x09, 0x31,
0x20, 0xB3, 0xD4, 0x13, 0xDF, 0xA4, 0xA4, 0x4E, 0x19, 0x6A, 0x2C, 0xCA,
0x30, 0x0D, 0x17, 0x6C, 0xA1, 0xD4, 0x1A, 0xA1, 0xC5, 0xBF, 0x65, 0xF3,
0xED, 0xA4, 0x1A, 0x6C, 0xCE, 0x32, 0x17, 0xF1, 0x40, 0x4E, 0x57, 0x0F,
0x31, 0x75, 0xE4, 0xFA, 0x05, 0x2D, 0xA8, 0x2D, 0x9C, 0xEB, 0x42, 0xA0,
0xA6, 0x4B, 0xC2, 0x13, 0xB7, 0xB3, 0x64, 0xD6, 0x84, 0x00, 0x19, 0x27,
0xA7, 0xF9, 0x27, 0xA5, 0xF9, 0x35, 0x9F, 0x1D, 0xD8, 0xD0, 0xB6, 0x56,
0x18, 0x09, 0x8B, 0xFF, 0xFA, 0x23, 0xDC, 0xC2, 0x28, 0x26, 0x94, 0x96,
0xBA, 0x52, 0x9A, 0x96, 0x9A, 0x31, 0xB9, 0x52, 0xAB, 0x11, 0x43, 0x97,
0x59, 0x74, 0x25, 0x9B, 0x89, 0xAC, 0x4C, 0x23, 0x7F, 0x99, 0xC9, 0xA4,
0x82, 0x50, 0x88, 0x64, 0xC8, 0xA7, 0x4B, 0xCF, 0xC0, 0x59, 0x08, 0xC5,
0x56, 0xE0, 0x52, 0x6A, 0x0B, 0x25, 0x56, 0x0E, 0xF3, 0x67, 0xAE, 0x8B,
0xF8, 0x36, 0x16, 0x79, 0xC3, 0xF4, 0xF1, 0x84, 0xED, 0x33, 0x1B, 0x5B,
0xC4, 0xCF, 0x90, 0x87, 0xA1, 0x33, 0x70, 0xBC, 0x74, 0x1F, 0x24, 0x8C,
0x9C, 0x36, 0x73, 0x7F, 0x46, 0xD9, 0x48, 0xD1, 0x67, 0x37, 0x62, 0xDE,
0xE4, 0xB5, 0x13, 0x4F, 0x7D, 0x8E, 0xEC, 0x52, 0xBB, 0xE7, 0x40, 0x26,
0x2C, 0x82, 0x2C, 0x86, 0x31, 0xB6, 0xF8, 0x04, 0xDA, 0x0C, 0x56, 0xD0,
0x07, 0x59, 0x56, 0x62, 0x2C, 0x7C, 0xC9, 0xA1, 0x3B, 0xC3, 0xDC, 0x71,
0xF4, 0x4C, 0xB2, 0x2A, 0xEF, 0x1B, 0x19, 0x14, 0xF8, 0xE2, 0x55, 0x55
};
static U8 _aOutput[sizeof(_aInput)*2];
typedef struct {
U8 * pOutput;
unsigned OutputLen;
unsigned Cursor;
} CTG_WR_CONTEXT;
static int _Wr(const U8 *pData, unsigned DataLen, void *pWrCtx) {
CTG_WR_CONTEXT * pCtx;
unsigned N;
//
pCtx = (CTG_WR_CONTEXT *)pWrCtx;
for (N = 0; N < DataLen; ++N) {
if (pCtx->Cursor >= pCtx->OutputLen) {
return CTG_STATUS_OUTPUT_OVERFLOW;
}
pCtx->pOutput[pCtx->Cursor++] = *pData++;
}
return N;
}
void MainTask(void);
void MainTask(void) {
CTG_WR_CONTEXT WrCtx;
U8 aWork[CTG_DECOMPRESS_WS_SIZE(CTG_FLAG_WINDOW_SIZE_1K)];
int DecodeLen;
//
WrCtx.pOutput = &_aOutput[0];
WrCtx.OutputLen = sizeof(_aOutput);
WrCtx.Cursor = 0;
DecodeLen = CTG_DecompressM2F(&_aInput[0], sizeof(_aInput),
_Wr, &WrCtx,
&aWork[0], sizeof(aWork));
if (DecodeLen < 0) {
printf("Decompression error.\n");
} else {
printf("Decompressed %u to %d bytes.\n", sizeof(_aInput), DecodeLen);
}
}
Declare work buffer
The work buffer is sized to the compression parameters used to
compress the input stream. In this case the window size was
selected to be 1 KB and therefore the buffer is sized appropriately.
Provide work buffer
The work buffer and its size are provided to the decompression
function to use and check correctness.
Fast compression
emCompress-ToGo also provides compression functions that are much faster than
the ordinary ones, but require an additional or bigger “work buffer” in RAM
to speed up the operation.
The size of this work buffer can be determined using the macros
CTG_FCOMPRESS_WS_SIZE() and CTG_FCOMPRESS_M_WS_SIZE().
See CTG_FastCompressM2M().
For details on the performance see Runtime performance.
Compression of ARM thumb-2 program code
Executable firmware of ARM thumb-2 instructions is usually hard to compress.
emCompress-ToGo provides compression functions that achieve slightly better compression rates
when operating on thumb-2 code. See CTG_CompressT2() and CTG_DecompressT2().
emCompress-ToGo API
This section describes the public API for emCompress-ToGo. Any functions
or data structures that are not described here but are exposed through
inclusion of the CTG.h header file must be considered
private and subject to change.
Preprocessor definitions
Version number
Description
Symbol expands to a number that identifies the specific emCompress-ToGo release.
Definition
#define CTG_VERSION 34000
Symbols
| Definition | Description |
|---|
| CTG_VERSION | Format is “Mmmrr” so, for example, 31000 corresponds to version 3.10. |
Function status values
Description
Compression and decompression errors.
Definition
#define CTG_STATUS_OUTPUT_OVERFLOW (-100)
#define CTG_STATUS_INPUT_UNDERFLOW (-101)
#define CTG_STATUS_BUFFER_TOO_SMALL (-102)
#define CTG_STATUS_BAD_PARAMETER (-103)
#define CTG_STATUS_DECODING_ERROR (-104)
Symbols
| Definition | Description |
|---|
| CTG_STATUS_OUTPUT_OVERFLOW | The output buffer is not big enough to contain the compressed or decompressed output. |
| CTG_STATUS_INPUT_UNDERFLOW | The input buffer contains a bitstream that is truncated. |
| CTG_STATUS_BUFFER_TOO_SMALL | The work buffer provided to the compressor or decompressor is too small. |
| CTG_STATUS_BAD_PARAMETER | A parameter given to the function is invalid. |
| CTG_STATUS_DECODING_ERROR | Error while decoding, bad encoded data stream. |
Additional information
Values returned as errors by the compression and decompression
functions. All errors are negative.
Workspace buffer sizes
Description
Gives the size of the workspace required for compression and
decompression depending on the ’Flags’ parameter (CTG_FLAG_WINDOW_SIZE_…)
used during compression.
Definition
#define CTG_COMPRESS_WS_SIZE(Flags) ((0x100uL << (Flags)) + CTG_MAX_REFERENCE_LEN)
#define CTG_DECOMPRESS_WS_SIZE(Flags) (0x100uL << (Flags))
#define CTG_FCOMPRESS_M_WS_SIZE(Flags) ((0x200uL << (Flags)) + 512u)
#define CTG_FCOMPRESS_WS_SIZE(Flags) ((0x300uL << (Flags)) + CTG_MAX_REFERENCE_LEN + 512u)
#define CTG_COMPRESS_T2_WS_SIZE(Flags) (CTG_COMPRESS_WS_SIZE(Flags) + CTG_T2_PRECONDITION_BUFF_SIZE)
#define CTG_DECOMPRESS_T2_WS_SIZE(Flags) (CTG_DECOMPRESS_WS_SIZE(Flags) + CTG_T2_PRECONDITION_BUFF_SIZE)
Symbols
| Definition | Description |
|---|
| CTG_COMPRESS_WS_SIZE(Flags) | Gives the workspace size required for compression functions depending on the window size used. |
| CTG_DECOMPRESS_WS_SIZE(Flags) | Gives the workspace size required for decompression functions depending on the window size used for compression. |
| CTG_FCOMPRESS_M_WS_SIZE(Flags) | Gives the workspace size required for the fast compression functions from memory depending on the window size used. |
| CTG_FCOMPRESS_WS_SIZE(Flags) | Gives the workspace size required for the fast compression functions depending on the window size used. |
| CTG_COMPRESS_T2_WS_SIZE(Flags) | Gives the workspace size required for compression functions depending on the window size used. |
| CTG_DECOMPRESS_T2_WS_SIZE(Flags) | Gives the workspace size required for decompression functions depending on the window size used for compression. |
Compressor encoding flags
Window size parameter
Description
Sets the window size to use when compressing.
Definition
#define CTG_FLAG_WINDOW_SIZE_256 0u
#define CTG_FLAG_WINDOW_SIZE_512 1u
#define CTG_FLAG_WINDOW_SIZE_1K 2u
#define CTG_FLAG_WINDOW_SIZE_2K 3u
#define CTG_FLAG_WINDOW_SIZE_4K 4u
#define CTG_FLAG_WINDOW_SIZE_8K 5u
#define CTG_FLAG_WINDOW_SIZE_16K 6u
Symbols
| Definition | Description |
|---|
| CTG_FLAG_WINDOW_SIZE_256 | Window set to 256 bytes |
| CTG_FLAG_WINDOW_SIZE_512 | Window set to 512 bytes |
| CTG_FLAG_WINDOW_SIZE_1K | Window set to 1K bytes |
| CTG_FLAG_WINDOW_SIZE_2K | Window set to 2K bytes |
| CTG_FLAG_WINDOW_SIZE_4K | Window set to 4K bytes |
| CTG_FLAG_WINDOW_SIZE_8K | Window set to 8K bytes |
| CTG_FLAG_WINDOW_SIZE_16K | Window set to 16K bytes |
Additional information
Smaller windows will make compression run faster, larger
windows tend to make the compressed output smaller.
Data types
CTG_STREAM
Description
Streaming interface for compression / decompression.
Type definition
typedef struct {
unsigned AvailIn;
const U8 * pIn;
int Flush;
unsigned AvailOut;
U8 * pOut;
} CTG_STREAM;
Structure members
| Member | Description |
|---|
| AvailIn | Number of elements available as input octets. |
| pIn | Pointer to available input octets. |
| Flush | Set to 1, if no more input data are available. |
| AvailOut | Size of buffer for output data. |
| pOut | Pointer to the buffer for output data. |
CTG_RD_FUNC
Description
Read data form source. The function must provide at least one byte of
data to the caller, except if the end of the input data is reached.
If the function returns a negative value, the compression/decompression
operation will be aborted and the value if forward to
the caller of the compression/decompression function.
Type definition
typedef int (CTG_RD_FUNC)(U8 * pData,
unsigned DataLen,
void * pRdCtx);
Parameters
| Parameter | Description |
|---|
| pData | Pointer to object that receives the data. |
| DataLen | Maximum number of bytes to read into object, will be nonzero. |
| pRdCtx | Client-supplied read context. |
Return value
| < 0 | Error reading data. |
| = 0 | Success, end of stream, no further data. |
| > 0 | Success, number of bytes placed into object, more data to read. |
CTG_WR_FUNC
Description
Write data to sink. The function must consume all ’DataLen’ bytes of data.
If the function returns a negative value, the compression/decompression
operation will be aborted and the value if forward to
the caller of the compression/decompression function.
Type definition
typedef int (CTG_WR_FUNC)(const U8 * pData,
unsigned DataLen,
void * pWrCtx);
Parameters
| Parameter | Description |
|---|
| pData | Pointer to object to write. |
| DataLen | Number of bytes to write. |
| pWrCtx | Client-supplied read context. |
Return value
| < 0 | Error writing data. |
| = 0 | Success. |
Compression functions
emCompress-ToGo defines the following compression functions:
CTG_CompressM2M()
Description
Compress, memory to memory.
Prototype
int CTG_CompressM2M(const U8 * pInput,
unsigned InputLen,
U8 * pOutput,
unsigned OutputSize,
unsigned Flags);
Parameters
| Parameter | Description |
|---|
| pInput | Pointer to octet string to encode. |
| InputLen | Byte length of the input octet string. |
| pOutput | Pointer to the buffer that receives the compressed output. |
| OutputSize | Size of the output buffer in bytes. |
| Flags | Window size to be used for compression, see CTG_FLAG_WINDOW_SIZE_… macros. |
Return value
| ≥ 0 | Success, compressed output length. |
| < 0 | Error indication. |
Example
See Memory-to-memory compression.
See also
Function status values and Compressor encoding flags.
CTG_CompressM2F()
Description
Compress, memory to function.
Prototype
int CTG_CompressM2F(const U8 * pInput,
unsigned InputLen,
CTG_WR_FUNC * pfWr,
void * pWrCtx,
unsigned Flags);
Parameters
| Parameter | Description |
|---|
| pInput | Pointer to octet string to encode. |
| InputLen | Octet length of the input octet string. |
| pfWr | Pointer to function that writes output. |
| pWrCtx | Pointer to user-supplied write context. |
| Flags | Encoding flags. |
Return value
| ≥ 0 | Success, compressed output length. |
| < 0 | Error indication. |
Example
See Memory-to-function compression.
See also
Function status values and Compressor encoding flags.
CTG_CompressF2M()
Description
Compress, function to memory.
Prototype
int CTG_CompressF2M(CTG_RD_FUNC * pfRd,
void * pRdCtx,
U8 * pOutput,
unsigned OutputSize,
U8 * pWork,
unsigned WorkLen,
unsigned Flags);
Parameters
| Parameter | Description |
|---|
| pfRd | Pointer to function that reads input. |
| pRdCtx | Pointer to user-supplied context for read function. |
| pOutput | Pointer to buffer that receives the compressed output. |
| OutputSize | Size of the output buffer in bytes. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
| Flags | Encoding flags. |
Return value
| ≥ 0 | Success, compressed output length. |
| < 0 | Error indication. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_COMPRESS_WS_SIZE(Flags).
This requirement is enforced by the compressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned
by the compress functions.
Example
See Function-to-memory compression.
See also
Function status values and Compressor encoding flags.
CTG_CompressF2F()
Description
Compress, function to function.
Prototype
int CTG_CompressF2F(CTG_RD_FUNC * pfRd,
void * pRdCtx,
CTG_WR_FUNC * pfWr,
void * pWrCtx,
U8 * pWork,
unsigned WorkLen,
unsigned Flags);
Parameters
| Parameter | Description |
|---|
| pfRd | Pointer to function that reads input. |
| pRdCtx | Pointer to user-supplied context for read function. |
| pfWr | Pointer to function that writes output. |
| pWrCtx | Pointer to user-supplied write context. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
| Flags | Encoding flags. |
Return value
| ≥ 0 | Success, compressed output length. |
| < 0 | Error indication. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_COMPRESS_WS_SIZE(Flags).
This requirement is enforced by the compressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned
by the compress functions.
Example
See Function-to-memory compression.
See also
Function status values and Compressor encoding flags.
CTG_FastCompressM2M()
Description
Compress, memory to memory. Much faster than CTG_CompressM2M().
Prototype
int CTG_FastCompressM2M(const U8 * pInput,
unsigned InputLen,
U8 * pOutput,
unsigned OutputSize,
U16 * pWork,
unsigned WorkLen,
unsigned Flags);
Parameters
| Parameter | Description |
|---|
| pInput | Pointer to octet string to encode. |
| InputLen | Byte length of the input octet string. |
| pOutput | Pointer to the buffer that receives the compressed output. |
| OutputSize | Size of the output buffer in bytes. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer in bytes. |
| Flags | Window size to be used for compression, see CTG_FLAG_WINDOW_SIZE_… macros. |
Return value
| ≥ 0 | Success, compressed output length. |
| < 0 | Error indication. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_FCOMPRESS_M_WS_SIZE(Flags).
This requirement is enforced by the compressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned
by the compress function.
See also
Function status values and Compressor encoding flags.
CTG_CompressInit()
Description
Initialize for compression with CTG_Compress().
Prototype
void CTG_CompressInit(CTG_COMPRESS_CTX * pCtx,
U8 * pWork,
unsigned WorkLen,
unsigned Flags);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to compress context to be initialized. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
| Flags | Window size to be used for compression, see CTG_FLAG_WINDOW_SIZE_… macros. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_COMPRESS_WS_SIZE(Flags).
This requirement is enforced by the compressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned
by the compress function.
Example
See CTG_Util.c sample application.
See also
Compressor encoding flags.
CTG_Compress()
Description
Run compress on (part of) data.
The function will process as many data from the input stream as possible
and copy the compressed data to the output stream.
While processing input data, pStream->pIn is increased and pStream->AvailIn is reduced.
While generating output data, pStream->pOut is increased and pStream->AvailOut is reduced.
The function stops if either pStream->AvailIn or pStream->AvailOut reaches 0.
On calling this function, the data stream must meet the following conditions:
pStream->AvailOut > 0 && (pStream->AvailIn > 0 || pStream->Flush > 0).
Prototype
int CTG_Compress(CTG_COMPRESS_CTX * pCtx,
CTG_STREAM * pStream);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to compress context, created by CTG_CompressInit(). |
| pStream | Pointer to data stream (input and output). |
Return value
| < 0 | Error indication. |
| = 0 | Processing suspended, needs more input data or more output buffer. Then the function must be called again to resume processing. |
| > 0 | Compression finished successfully. |
Example
See CTG_Util.c sample application.
See also
Function status values
CTG_FastCompressInit()
Description
Initialize for compression with CTG_FastCompress().
Prototype
void CTG_FastCompressInit(CTG_FCOMPRESS_CTX * pCtx,
U16 * pWork,
unsigned WorkLen,
unsigned Flags);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to compress context to be initialized. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
| Flags | Window size to be used for compression, see CTG_FLAG_WINDOW_SIZE_… macros. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_FCOMPRESS_WS_SIZE(Flags).
This requirement is enforced by the compressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned
by the compress function.
See also
Compressor encoding flags.
CTG_FastCompress()
Description
Run compress on (part of) data. Much faster than CTG_Compress().
The function will process as many data from the input stream as possible
and copy the compressed data to the output stream.
While processing input data, pStream->pIn is increased and pStream->AvailIn is reduced.
While generating output data, pStream->pOut is increased and pStream->AvailOut is reduced.
The function stops if either pStream->AvailIn or pStream->AvailOut reaches 0.
On calling this function, the data stream must meet the following conditions:
pStream->AvailOut > 0 && (pStream->AvailIn > 0 || pStream->Flush > 0).
Prototype
int CTG_FastCompress(CTG_FCOMPRESS_CTX * pCtx,
CTG_STREAM * pStream);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to compress context, created by CTG_FastCompressInit(). |
| pStream | Pointer to data stream (input and output). |
Return value
| < 0 | Error indication. |
| = 0 | Processing suspended, needs more input data or more output buffer. Then the function must be called again to resume processing. |
| > 0 | Compression finished successfully. |
See also
Function status values
CTG_CompressReadInit()
Description
Initialize for compression with CTG_CompressRead().
Prototype
void CTG_CompressReadInit(CTG_COMPRESS_RD_CTX * pCtx,
U8 * pWork,
unsigned WorkLen,
U8 * pBuff,
unsigned BuffSize,
CTG_RD_FUNC * pfRd,
void * pRdCtx,
unsigned Flags);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to compress context to be initialized. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
| pBuff | Pointer to a buffer used for input data. This buffer is used when calling the read function: pfRd(pBuff, BuffSize, pRdCtx). |
| BuffSize | Size of the buffer in bytes. Every buffer size ≥ 1 will work. However, small sizes will decrease performance. |
| pfRd | Pointer to function that reads input (uncompressed data). |
| pRdCtx | Pointer to user-supplied context for read function. |
| Flags | Window size to be used for compression, see CTG_FLAG_WINDOW_SIZE_… macros. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_COMPRESS_WS_SIZE(Flags).
This requirement is enforced by the compressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned
by the compress function.
See also
Compressor encoding flags.
CTG_CompressRead()
Description
Read compressed data from a compression-stream. Input data to be compressed
is fetched using the callback read function given via CTG_CompressReadInit().
To read the whole compressed data
this function must be called repeatedly until it returns a value < NumBytes.
Prototype
int CTG_CompressRead(CTG_COMPRESS_RD_CTX * pCtx,
U8 * pBuff,
unsigned NumBytes);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to compress context, created by CTG_CompressReadInit(). |
| pBuff | Pointer to the buffer to store the compressed data. |
| NumBytes | Number of bytes to be read. |
Return value
| ≥ 0 | Success, number of bytes read from stream. A value < NumBytes indicates the end of the compressed data. |
| < 0 | Error indication. |
See also
Function status values
CTG_CompressWriteInit()
Description
Initialize for compression with CTG_CompressWrite().
Prototype
void CTG_CompressWriteInit(CTG_COMPRESS_WR_CTX * pCtx,
U8 * pWork,
unsigned WorkLen,
U8 * pBuff,
unsigned BuffSize,
CTG_WR_FUNC * pfWr,
void * pWrCtx,
unsigned Flags);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to compress context to be initialized. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
| pBuff | Pointer to a buffer used to collect output data. This buffer is used when calling the write function: pfWr(pBuff, Len, pWrCtx). |
| BuffSize | Size of the buffer in bytes. Every buffer size ≥ 1 will work. However, small sizes will decrease performance. |
| pfWr | Pointer to function that writes output (compressed data). |
| pWrCtx | Pointer to user-supplied write context. |
| Flags | Window size to be used for compression, see CTG_FLAG_WINDOW_SIZE_… macros. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_COMPRESS_WS_SIZE(Flags).
This requirement is enforced by the compressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned
by the compress function.
See also
Compressor encoding flags.
CTG_CompressWrite()
Description
Write data to be compressed to a compression-stream. Compressed data is
output using the callback write function given via CTG_CompressWriteInit().
This function must be called repeatedly until all data to be compressed are
written. To indicate the end of the data to be compressed, the function must
be called with ’NumBytes’ = 0.
Prototype
int CTG_CompressWrite( CTG_COMPRESS_WR_CTX * pCtx,
const U8 * pData,
unsigned NumBytes);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to compress context, created by CTG_CompressWriteInit(). |
| pData | Pointer to the uncompressed data to be compressed. |
| NumBytes | Number of bytes to write. |
Return value
| = 0 | Success. |
| < 0 | Error indication. |
See also
Function status values.
Example
static int _Fd;
//
// Write callback function that stores data into a file.
// The function is called for every 512 bytes of compressed data.
//
static int _WriteCb(const U8 *pData, unsigned DataLen, void *pWrCtx) {
FileWrite(_Fd, pData, DataLen);
return 0;
}
//
// Main function: Collect sensor data and write them compressed to a file.
//
void main() {
U8 Len;
U8 Buff[100];
U8 FileBuff[512];
U8 WorkBuff[CTG_COMPRESS_WS_SIZE(CTG_FLAG_WINDOW_SIZE_1K)];
CTG_COMPRESS_WR_CTX Ctx;
_Fd = FileOpen("SensorData.bin", "w");
CTG_CompressWriteInit(&Ctx, WorkBuff, sizeof(WorkBuff),
FileBuff, sizeof(FileBuff),
_WriteCb, NULL, CTG_FLAG_WINDOW_SIZE_1K);
//
// Super loop
//
while (...) {
if (..more sensor data available..) {
//
// Get sonsor data (variable length)
//
Len = GenSensorData(Buff);
//
// Store length in front of data
//
CTG_CompressWrite(&Ctx, &Len, sizeof(Len));
CTG_CompressWrite(&Ctx, Buff, Len);
}
//
// Handle other tasks
//
....
}
//
// Finish compression
//
CTG_CompressWrite(&Ctx, NULL, 0);
FileClose(_Fd);
}
CTG_CompressT2Init()
Description
Initialize for compression with CTG_CompressT2().
Prototype
void CTG_CompressT2Init(CTG_COMPRESS_T2_CTX * pCtx,
U8 * pWork,
unsigned WorkLen,
unsigned Flags);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to compress context to be initialized. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
| Flags | Window size to be used for compression, see CTG_FLAG_WINDOW_SIZE_… macros. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_COMPRESS_T2_WS_SIZE(Flags).
This requirement is enforced by the compressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned
by the compress function.
CTG_CompressT2()
Description
Run compress for thumb-2 code on (part of) data.
The function will process as many data from the input stream as possible
and copy the compressed data to the output stream.
While processing input data, pStream->pIn is increased and pStream->AvailIn is reduced.
While generating output data, pStream->pOut is increased and pStream->AvailOut is reduced.
The function stops if either pStream->AvailIn or pStream->AvailOut reaches 0.
On calling this function, the data stream must meet the following conditions:
pStream->AvailOut > 0 && (pStream->AvailIn > 0 || pStream->Flush > 0).
Prototype
int CTG_CompressT2(CTG_COMPRESS_T2_CTX * pCtx,
CTG_STREAM * pStream);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to compress context, created by CTG_CompressT2Init(). |
| pStream | Pointer to data stream (input and output). |
Return value
| < 0 | Error indication. |
| = 0 | Processing suspended, needs more input data or more output buffer. Then the function must be called again to resume processing. |
| > 0 | Compression finished successfully. |
Decompression functions
emCompress-ToGo defines the following decompression functions:
CTG_DecompressM2M()
Description
Decompress, memory to memory.
Prototype
int CTG_DecompressM2M(const U8 * pInput,
unsigned InputLen,
U8 * pOutput,
unsigned OutputSize);
Parameters
| Parameter | Description |
|---|
| pInput | Pointer to object that contains the compressed input. |
| InputLen | Octet length of the input object. |
| pOutput | Pointer to buffer that receives the decompressed output. |
| OutputSize | Size of the output buffer in bytes. |
Return value
| < 0 | Error decoding bitstream. |
| ≥ 0 | Number of bytes output. |
Example
See Memory-to-memory decompression.
See also
Function status values
CTG_DecompressM2F()
Description
Decompress, memory to function.
Prototype
int CTG_DecompressM2F(const U8 * pInput,
unsigned InputLen,
CTG_WR_FUNC * pfWr,
void * pWrCtx,
U8 * pWork,
unsigned WorkLen);
Parameters
| Parameter | Description |
|---|
| pInput | Pointer to object that contains the compressed input. |
| InputLen | Octet length of the input object. |
| pfWr | Pointer to function that writes output. |
| pWrCtx | Pointer to user-supplied write context. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
Return value
| < 0 | Error decoding bitstream. |
| ≥ 0 | Number of bytes output. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_DECOMPRESS_WS_SIZE(Flag),
where ’Flag’ is the parameter that was used during compression of the data
(Flag = CTG_FLAG_WINDOW_SIZE_…).
This requirement is enforced by the decompressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned.
Example
See Memory-to-function decompression.
See also
Function status values
CTG_DecompressF2M()
Description
Decompress, function to memory.
Prototype
int CTG_DecompressF2M(CTG_RD_FUNC pfRd,
void * pRdCtx,
U8 * pOutput,
unsigned OutputSize);
Parameters
| Parameter | Description |
|---|
| pfRd | Pointer to function that reads input. |
| pRdCtx | Pointer to user-supplied context for read function. |
| pOutput | Pointer to buffer that receives the decompressed output. |
| OutputSize | Size of the output buffer in bytes. |
Return value
| < 0 | Error decoding bitstream. |
| ≥ 0 | Number of bytes output. |
Example
See Function-to-memory decompression.
See also
Function status values
CTG_DecompressF2F()
Description
Decompress, function to function.
Prototype
int CTG_DecompressF2F(CTG_RD_FUNC pfRd,
void * pRdCtx,
CTG_WR_FUNC * pfWr,
void * pWrCtx,
U8 * pWork,
unsigned WorkLen);
Parameters
| Parameter | Description |
|---|
| pfRd | Pointer to function that reads input. |
| pRdCtx | Pointer to user-supplied read context. |
| pfWr | Pointer to function that writes output. |
| pWrCtx | Pointer to user-supplied write context. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
Return value
| < 0 | Error decoding bitstream. |
| ≥ 0 | Number of bytes output. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_DECOMPRESS_WS_SIZE(Flag),
where ’Flag’ is the parameter that was used during compression of the data
(Flag = CTG_FLAG_WINDOW_SIZE_…).
This requirement is enforced by the decompressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned.
Example
See Memory-to-function decompression.
See also
Function status values
CTG_DecompressInit()
Description
Initialize for decompression with CTG_Decompress().
Prototype
void CTG_DecompressInit(CTG_DECOMPRESS_CTX * pCtx,
U8 * pWork,
unsigned WorkLen);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to decompress context to be initialized. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_DECOMPRESS_WS_SIZE(Flags),
where ’Flags’ is the parameter that was used during compression of the data
(Flag = CTG_FLAG_WINDOW_SIZE_…).
This requirement is enforced by the decompressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned
by the decompress function.
Example
See CTG_Util.c sample application.
See also
Compressor encoding flags.
CTG_Decompress()
Description
Run decompress on (part of) data.
The function will process as many data from the input stream as possible
and copy the decompressed data to the output stream.
While processing input data, pStream->pIn is increased and pStream->AvailIn is reduced.
While generating output data, pStream->pOut is increased and pStream->AvailOut is reduced.
The function stops if either pStream->AvailIn or pStream->AvailOut reaches 0.
On calling this function, the data stream must meet the following conditions:
pStream->AvailOut > 0 && (pStream->AvailIn > 0 || pStream->Flush > 0).
Prototype
int CTG_Decompress(CTG_DECOMPRESS_CTX * pCtx,
CTG_STREAM * pStream);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to decompress context, created by CTG_DecompressInit(). |
| pStream | Pointer to data stream (input and output). |
Return value
| < 0 | Error decoding bitstream. |
| = 0 | Processing suspended, needs more input data or more output buffer. Then the function must be called again to resume processing. |
| > 0 | Decompression finished successfully. |
Example
See CTG_Util.c sample application.
See also
Function status values
CTG_DecompressReadInit()
Description
Initialize for decompression with CTG_DecompressRead().
Prototype
void CTG_DecompressReadInit(CTG_DECOMPRESS_RD_CTX * pCtx,
U8 * pWork,
unsigned WorkLen,
U8 * pBuff,
unsigned BuffSize,
CTG_RD_FUNC * pfRd,
void * pRdCtx);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to decompress context to be initialized. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
| pBuff | Pointer to a buffer used for input data. This buffer is used when calling the read function: pfRd(pBuff, BuffSize, pRdCtx). |
| BuffSize | Size of the buffer in bytes. Every buffer size ≥ 1 will work. However, small sizes will decrease performance. |
| pfRd | Pointer to function that reads input (compressed data). |
| pRdCtx | Pointer to user-supplied context for read function. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_DECOMPRESS_WS_SIZE(Flag),
where ’Flag’ is the parameter that was used during compression of the data
(Flag = CTG_FLAG_WINDOW_SIZE_…).
This requirement is enforced by the decompressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned
by the decompress function.
See also
Compressor encoding flags.
CTG_DecompressRead()
Description
Read decompressed data from a decompression-stream. Compressed input data to be
decompressed is fetched using the callback read function given via CTG_DecompressReadInit().
To read the whole decompressed data
this function must be called repeatedly until it returns a value < NumBytes.
Prototype
int CTG_DecompressRead(CTG_DECOMPRESS_RD_CTX * pCtx,
U8 * pBuff,
unsigned NumBytes);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to decompress context, created by CTG_DecompressReadInit(). |
| pBuff | Pointer to the buffer to store the decompressed data. |
| NumBytes | Number of bytes to be read. |
Return value
| ≥ 0 | Success, number of bytes read from stream. A value < NumBytes indicates the end of the decompressed data. |
| < 0 | Error indication. |
See also
Function status values
Example
static int _Fd;
//
// Read callback function that reads data from a file.
// On EOF the function returns 0.
//
static int _ReadCb(U8 *pData, unsigned DataLen, void *pWrCtx) {
return FileRead(_Fd, pData, DataLen);
}
//
// Main function: Read sensor data from a compressed file and process them.
//
void main() {
U8 Len;
U8 Buff[100];
U8 FileBuff[512];
U8 WorkBuff[CTG_COMPRESS_WS_SIZE(CTG_FLAG_WINDOW_SIZE_1K)];
CTG_DECOMPRESS_RD_CTX Ctx;
_Fd = FileOpen("SensorData.bin", "r");
CTG_DecompressReadInit(&Ctx, WorkBuff, sizeof(WorkBuff),
FileBuff, sizeof(FileBuff),
_ReadCb, NULL);
//
// Each sensor data record is preceded by its length.
// Read all records until end of the file.
//
while (CTG_DecompressRead(&Ctx, &Len, sizeof(Len)) == sizeof(Len)) {
CTG_DecompressRead(&Ctx, Len, Buff);
//
// Process data in Buff.
//
....
}
FileClose(_Fd);
}
CTG_DecompressWriteInit()
Description
Initialize for decompression with CTG_DecompressWrite().
Prototype
void CTG_DecompressWriteInit(CTG_DECOMPRESS_WR_CTX * pCtx,
U8 * pWork,
unsigned WorkLen,
U8 * pBuff,
unsigned BuffSize,
CTG_WR_FUNC * pfWr,
void * pWrCtx);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to decompress context to be initialized. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
| pBuff | Pointer to a buffer used to collect output data. This buffer is used when calling the write function: pfWr(pBuff, Len, pWrCtx). |
| BuffSize | Size of the buffer in bytes. Every buffer size ≥ 1 will work. However, small sizes will decrease performance. |
| pfWr | Pointer to function that writes output (decompressed data). |
| pWrCtx | Pointer to user-supplied write context. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_DECOMPRESS_WS_SIZE(Flag),
where ’Flag’ is the parameter that was used during compression of the data
(Flag = CTG_FLAG_WINDOW_SIZE_…).
This requirement is enforced by the decompressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned
by the decompress function.
See also
Compressor encoding flags.
CTG_DecompressWrite()
Description
Write compressed data to be decompressed to a stream. Decompressed data is
output using the callback write function given via CTG_DecompressWriteInit().
This function must be called repeatedly until all compressed data are
written. To indicate the end of the compressed data, the function must
be called with ’NumBytes’ = 0.
Prototype
int CTG_DecompressWrite( CTG_DECOMPRESS_WR_CTX * pCtx,
const U8 * pData,
unsigned NumBytes);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to compress context, created by CTG_CompressReadInit(). |
| pData | Pointer to the compressed data to be decompressed. |
| NumBytes | Number of bytes to write. |
Return value
| = 0 | Success. |
| < 0 | Error indication. |
See also
Function status values
CTG_DecompressT2Init()
Description
Initialize for decompression with CTG_DecompressT2().
Prototype
void CTG_DecompressT2Init(CTG_DECOMPRESS_T2_CTX * pCtx,
U8 * pWork,
unsigned WorkLen);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to decompress context to be initialized. |
| pWork | Pointer to work buffer. |
| WorkLen | Length of work buffer. |
Additional information
The work buffer size (value of WorkLen) must be at least CTG_DECOMPRESS_T2_WS_SIZE(Flags),
where ’Flags’ is the parameter that was used during compression of the data
(Flag = CTG_FLAG_WINDOW_SIZE_…).
This requirement is enforced by the decompressor and, should
the buffer be too small, CTG_STATUS_BUFFER_TOO_SMALL is returned
by the decompress function.
CTG_DecompressT2()
Description
Run decompress for thumb-2 code on (part of) data.
The function will process as many data from the input stream as possible
and copy the decompressed data to the output stream.
While processing input data, pStream->pIn is increased and pStream->AvailIn is reduced.
While generating output data, pStream->pOut is increased and pStream->AvailOut is reduced.
The function stops if either pStream->AvailIn or pStream->AvailOut reaches 0.
On calling this function, the data stream must meet the following conditions:
pStream->AvailOut > 0 && (pStream->AvailIn > 0 || pStream->Flush > 0).
Prototype
int CTG_DecompressT2(CTG_DECOMPRESS_T2_CTX * pCtx,
CTG_STREAM * pStream);
Parameters
| Parameter | Description |
|---|
| pCtx | Pointer to decompress context, created by CTG_DecompressT2Init(). |
| pStream | Pointer to data stream (input and output). |
Return value
| < 0 | Error decoding bitstream. |
| = 0 | Processing suspended, needs more input data or more output buffer. Then the function must be called again to resume processing. |
| > 0 | Decompression finished successfully. |
Utility functions
emCompress-ToGo defines the following utility functions:
CTG_GetStatusText()
Description
Decode emCompress-ToGo status code.
Prototype
char *CTG_GetStatusText(int Status);
Parameters
| Parameter | Description |
|---|
| Status | Code to decode. |
Return value
Non-zero pointer to status description.
CTG_GetCopyrightText()
Description
Get emCompress-ToGo copyright as printable string.
Prototype
char *CTG_GetCopyrightText(void);
Return value
Zero-terminated copyright string.
CTG_GetVersionText()
Description
Get emCompress-ToGo version as printable string.
Prototype
char *CTG_GetVersionText(void);
Return value
Zero-terminated version string.
Complete applications
This section presents two full applications developed with emCompress-ToGo:
- A compression-decompression utility to compress and decompress files.
- A self-test utility.
Compressor-decompressor utility
emCompress-ToGo ships with an application, in source form, which compresses
files into SMASH-2 format. These files can be fed into the emCompress-ToGo
decompressor, byte for byte, and be decompressed on target hardware.
For details, see CTG_Util.c sample application..
Command line and options
The emCompress-ToGo example compression application accepts the command
line options described in the following sections.
The command line syntax is:
CTG_Util [options] inputfile [outputfile]
The output file is optional: if given, the compressed bitstream is
written to it.
The example application displays some information relating to the compression
process and selected parameters.
Example
C:> CTG_Util.exe -wb=14 FPGA.input
emCompress-ToGo Compression Utility V3.20 compiled Mar 11 2019 10:59:40
Copyright (c) 2015-2019 SEGGER Microcontroller GmbH www.segger.com
Statistics:
Original: 114618 bytes
Compressed: 25402 bytes, or 22.16% of original
Saving: 89216 bytes, or 77.84% of original
Standardized: 1.77 bits/byte
C:> _
Set window bits (-wb)
Syntax
-wb=number
Description
Set the number of window bits, and therefore the window size, to
use when compressing. Acceptable values are 8 through 14 inclusive
with the default being 11.
This parameter sets the maximum match distance and, therefore,
the maximum number of octets that must be stored to satisfy
references made by the compressor (when not compressing from
memory) and the decompressor (when not decompressing to memory).
Increasing the window size will usually increase compression
ratios and reduce the size of the compressed bitstream.
Compress (-c)
Syntax
-c
Description
Compress input file to output file. This is the default action.
Decompress (-d)
Syntax
-d
Description
Decompress input file to output file.
Run silently (-q)
Syntax
-q
Description
Instructs the utility to run quietly and not display compression
and decompression statistics.
Verbose (-v)
Syntax
-v
Description
Instructs the utility to display compression and decompression statistics.
ARM thumb-2 instruction compression mode (-t2)
Syntax
-t2
Description
Uses preconditioning of data for better compression of ARM thumb-2 program code.
Compressor and decompressor self-test
This utility CTG_Test.c takes as input a single file and runs all compressors
and decompressors over the file to ensure that a compression-decompression
cycle results in the original content. This provides an assurance
of compressor and decompressor correctness after compilation and linking.
This section describes the memory requirement in terms of RAM and ROM
that emCompress-ToGo requires for decompression which can be used to obtain
sufficient estimates for most target systems.
Reentrancy
emCompress-ToGo is fully reentrant when compressing and decompressing a
bitstream: there is no requirement to lock any shared data nor is
there any static data requirement associated with its use.
Configuration
There is no configuration required in order to use emCompress-ToGo in your
target system beyond setting compiler options for code generation
strategy and setting up paths to include files.
emCompress-ToGo uses SEGGER’s Small Microcontroller Advanced
Super-High (SMASH-2) format to compress data. The proprietary
SMASH-2 format is an excellent all-round, tunable format.
When used in emCompress-ToGo, the maximum expansion of a bitstream is one bit
per byte plus 20 bits of header information and end-of-stream marker.
Hence, the number of bits to encode an
8n-bit bitstream is 9n+20; round this up to the next multiple
of 8 bits to determine the maximum buffer size required in bytes.
Target system configuration
The following table shows the hardware and the toolchain details of
the real-world target system used for benchmarking:
| Detail | Description |
|---|
| CPU | STM32F7, Cortex-M7, 200 MHz, run from flash |
| Toolchain | Embedded Studio for ARM Release 4.12 (gcc compiler) |
| Model | Thumb-2 instructions |
| Compiler options | Highest size optimization |
ROM and RAM use
The amount of ROM and RAM that emCompress-ToGo uses for compression and
decompression varies depending upon the function used.
The following table measures the total ROM required for a single
function. The compressor and decompressor do not require any static
data themselves, any working data is transitory and provided by the
client to the compressor and decompressor (if needed) and is shown
in the “RAM (client)” column. The size of the RAM required depends
on the window size selected upon compression.
Compression
Decompression
Target system configuration
The following table shows the hardware and the toolchain details of
the real-world target system used for benchmarking:
| Detail | Description |
|---|
| CPU | STM32F7, Cortex-M7, 200 MHz, run from flash |
| Toolchain | Embedded Studio for ARM Release 4.12 (gcc compiler) |
| Model | Thumb-2 instructions |
| Compiler options | -O2 optimization |
The following tables show the average throughput of the compression and decompression
functions calculated over a set of different input files. The actual performance
depends on the data to be compressed / decompressed.
Decompression
Compression, memory to memory, CTG_CompressM2M().
| Window size | CTG_CompressM2M() | CTG_FastCompressM2M() |
|---|
| 256 bytes | 410 kB/sec | 1510 kB/sec |
| 512 bytes | 263 kB/sec | 1130 kB/sec |
| 1k | 171 kB/sec | 850 kB/sec |
| 2k | 107 kB/sec | 580 kB/sec |
| 4k | 67 kB/sec | 390 kB/sec |
| 8k | 44 kB/sec | 250 kB/sec |
| 16k | 31 kB/sec | 180 kB/sec |
Compression, streaming, CTG_Compress().
| Window size | CTG_Compress() | CTG_FastCompress() |
|---|
| 256 bytes | 220 kB/sec | 1010 kB/sec |
| 512 bytes | 133 kB/sec | 750 kB/sec |
| 1k | 79 kB/sec | 540 kB/sec |
| 2k | 46 kB/sec | 350 kB/sec |
| 4k | 26 kB/sec | 240 kB/sec |
| 8k | 15 kB/sec | 165 kB/sec |
| 16k | 9 kB/sec | 120 kB/sec |
Glossary
Bitstream
A sequence of bits read on bit-by-bit basis.
Codec
Coder-decoder. A device or algorithm capable of coding or decoding a digital data stream. A lossless compressor and decompressor combination constitutes a codec.
Compressor
An algorithm that attempts to find redundancy in data and remove that redundancy thereby compressing the data to use fewer bits.
Decompressor
An algorithm that reverses the effect of compression and recovers the original data from the encoded bitstream.
KB
Kilobyte. Defined as either 1,024 or 1,000 bytes by context. In the microcontroller world and this manual it is understood to be 1,024 bytes and is routinely shortened further to “K” when describing microcontroller RAM or flash sizes.
LZSS
Lempel--Ziv--Storer--Szymanski. A compression scheme that is based on LZ77.
SMASH
Small Microcontroller Advanced Super-High format. SEGGER’s proprietary format for compressing data.
Indexes
Index of functions
Index of types