Change UTF-8 coding of file into ISO 8859-15 in tcl

Theoretical Physics 0
Viewed 35

I have written a code in Tcl which starts by getting the file My_Text_File.txt into it:

set myfile [open My_Text_File.txt]
set file_data [read $myfile]

The file My_Text_File.txt is encoded in UTF-8. But this file must be encoded in ISO 8859-15 (also referred to as Latin-9). Is there a way to extend a Tcl code in a way that it changes a UTF-8 encoded text file to an ISO 8859-15 encoded one?

I would like to emphasize that the change from UTF-8 to ISO 8859-15 must be done inside the Tcl code.

Thanks in advance!

tcl
2 Answers

You have to read your original file, converting from UTF-8 to tcl's native Unicode encoding, and then write the contents to a temporary file using ISO-8859-15 encoding, and finally replace the original with the temporary. tcl has a few commands to make it easy:

#!/usr/bin/env tclsh

# See `encoding names` for the list of character encodings supported
# by your version of tcl

proc convert_file {file to_encoding {from_encoding}} {
    set infile [open $file]
    # Assume original file is in the default system encoding if no
    # explicit from encoding is given.
    if {$from_encoding ne ""} {
        chan configure $infile -encoding $from_encoding
    }

    # Create a temporary file to write the re-encoded text to
    set outfile [file tempfile temp_name]
    chan configure $outfile -encoding $to_encoding

    # Efficiently read everything from one channel and write to another.
    chan copy $infile $outfile
    chan close $infile
    chan close $outfile

    # Rename the temporary file to the original
    file copy -force $temp_name $file
    file delete -force $temp_name
}


convert_file My_Text_File.txt iso8859-15 utf-8
userU2LF0

If the file isn't too big, it's easy to just read everything into memory and write it out again with the new encoding.

# Very simple conversion utility
proc convertFile {filename fromEncoding toEncoding} {
    # Read a file in a given encoding
    set f [open $filename]
    chan configure $f -encoding $fromEncoding
    set contents [chan read $f]
    chan close $f

    # Write a file in a given encoding
    set f [open $filename w]
    chan configure $f -encoding $toEncoding
    chan puts -nonewline $f $contents
    chan close $f
}

# Apply to the particular case we care about
convertFile My_Text_File.txt utf-8 iso8859-15

For a large file, you need to stream the data from one file to another (and then you can rename the target file afterwards).

Beware! Converting UTF-8 to ISO 8859-15 can lose information if there are characters in the source text that are not present in the target encoding.

Donal Fellows0
Related