Decompressing .Z file stored in Azure ADLS Gen2

user_9395516 0
Viewed 151

I have a .Z file stored in Azure ADLS Gen2. I want to decompress the file in the ADLS, I tried decompressing using ADF and C# but found that .Z is not supported. Also I tried using Apache Common Compress Lib for decompression, but unable to read the file in InputStream.

Can anyone have any idea, how we can decompress the file using Apache lib in Scala.

azure compression scala
1 Answers

.Z files are .gzip files so you could try this approach

import java.io.{BufferedReader, File, FileInputStream, InputStreamReader}
import java.util.zip.GZIPInputStream
object UnzipFiles {

  def decompressGzipOrZFiles(file: File, encode: String): BufferedReader = {
    val fis = new FileInputStream(file)
    val gzis = new GZIPInputStream(fis)
    val isr = new InputStreamReader(gzis, encode)
    new BufferedReader(isr)
  }

  def main(args: Array[String]): Unit = {
    val path = new File("/home/cloudera/files/my_file.Z")
    // print to the console
    decompressGzipOrZFiles(path,"UTF-8").lines().toArray.foreach(println)
  }
}

or you could follow this too

    def uncompressGzip(myFileDotZorGzip: String): Unit = {
      import java.io.FileInputStream
      import java.util.zip.GZIPInputStream
      try {
        val gzipInputStream = new GZIPInputStream(new FileInputStream(myFileDotZorGzip))
        try {
          val tam = 128
          val buffer = new Array[Byte](tam)
          do {
            gzipInputStream.read(buffer)
            gzipInputStream.skip(tam)
            //do something with data
            print(buffer.foreach(b => print(b.toChar)))
          } while(gzipInputStream.read() != -1)
        } finally {
          if (gzipInputStream != null) gzipInputStream.close()
        }
      }
    }

I hope this helps.

user_85714980
Related