Compressing const wchar_t strings in program memory

Question

I am building an avr program with many wchar_t strings. At the moment the way I store them in to the chip is this:

const _flash wchar_t* const  Greek_text[] =
{
    [ACCELERATION_TEXT] = (const _flash wchar_t[]) {L"Επιταγχυνση"},
    [ACCELERATION_SHORT_TEXT] = (const _flash wchar_t[]) {L"Επιταγχ."},
    [REDUCED_TEXT] = (const _flash wchar_t[]) {L"Μειωμενη"},
    [FULL_TEXT] = (const _flash wchar_t[]) {L"Μεγιστη"}
}

I am looking for a way to store them while being compressed. One way that I can think of, is removing the Unicode prefix for the specific language, but the only way of doing this to my knowledge, is storing the lower half of each word manually in unsigned chars. Is there a more practical way of doing something like this?

David Grayson · Answer

It looks like all your strings are Greek. You could assign each letter or character that appears in your strings to a number between 1 and 255 and then just have char strings where each char contains one of those encoded numbers. This would halve the space of each individual string, but you might need a lookup table in your program to convert those codes back to unicode, and that table could occupy up to 512 bytes in your program memory depending on how many different characters you use.

People have already developed encodings like this before, so maybe you could just use a standard one, like Windows-1253.

Figuring out a good system to author these encoded strings and insert them into your program is another challenge. I'd suggest writing a simple Ruby program like the one below. (You'd save the program itself as a UTF-8 text file on your computer and edit it using a standard text editor.)

string_table = {
  ACCELERATION_TEXT: 'Επιταγχυνση',
  ACCELERATION_SHORT_TEXT: 'Επιταγχ.',
  REDUCED_TEXT: 'Μειωμενη',
  FULL_TEXT: 'Μεγιστη',
}

string_table.each do |key, value|
  hex = value.encode('Windows-1253').each_byte.map { |b| '\x%02x' % b }.join
  puts "[#{key}] = \"#{hex}\","
end

The program above outputs this:

[ACCELERATION_TEXT] = "\xc5\xf0\xe9\xf4\xe1\xe3\xf7\xf5\xed\xf3\xe7",
[ACCELERATION_SHORT_TEXT] = "\xc5\xf0\xe9\xf4\xe1\xe3\xf7\x2e",
[REDUCED_TEXT] = "\xcc\xe5\xe9\xf9\xec\xe5\xed\xe7",
[FULL_TEXT] = "\xcc\xe5\xe3\xe9\xf3\xf4\xe7",

Also, another way to maybe save space (while increasing CPU time used) would be to get rid of the array of pointers you are creating. This would save two bytes per string. Just store all the strings next to each other in memory. The strings would be separated by null characters and you can optionally have a double null character marking the end of the table. To look up a string in the table, you start reading bytes from the beginning of the table one at a time and count how many null characters you have seen.

const _flash char greek_text[] =
   "\xc5\xf0\xe9\xf4\xe1\xe3\xf7\xf5\xed\xf3\xe7\0"
   "\xc5\xf0\xe9\xf4\xe1\xe3\xf7\x2e\0"
   "\xcc\xe5\xe9\xf9\xec\xe5\xed\xe7\0"
   "\xcc\xe5\xe3\xe9\xf3\xf4\xe7\0";

Compressing const wchar_t strings in program memory

1 Answers