Scala - try-catch inside for loop with yield

user_4819127 0
Viewed 1878

I'm writing a Scala app using some 3rd party library. When iterating over a collection from that library an exception occurs, which I want to ignore, and go on with the iteration. The whole thing is inside a for loop with yield.

val myFuntionalSequence = for {
  mailing <- mailingCollection
} yield (mailing.getName, mailing.getSubject)

As said, the error occurs inside the iteration, so this line:

mailing <- mailingCollection

If I would put a try catch around the whole loop, then I cannot continue with the iteration. I have a non-functional solution to have the same output as above, but I want to keep the whole app in a functional style. This is what I came up within a non-functional way:

case class MyElement(name: String, subject: String)

...

var myNonFunctionalList = scala.collection.mutable.ListBuffer[MyElement]()

while(mailingIterator.hasNext) {
  try {
    val mailing = mailingIterator.next()
    myNonFunctionalList += MyElement(mailing.getName, mailing.getSubject)
  } catch {
    case e: Exception => println("Error")
  }
}

My question is, do you know a functional way of trying to iterate through a for loop and on error skipping that element and only returning the elements where the iteration worked?

Thanks, Felix

yield for-loop try-catch scala
2 Answers

UPDATE: I updated the answer to correct my misunderstanding of the question, as Tim pointed out in his comment.

As mentioned in my comment, I will assume that by "continue with the iteration" you mean just not to throw the exception and still return the results up until that point. I will also assume that you want to maintain the laziness of the Iterator.

If so, I would effectively decorate the java.util.Iterator to protect against their bug, but returning a Scala Iterator so that you still have the laziness that the original library should you need it. Something (non-prod-ready) on the line of:

def scalaIterator[A](it: java.util.Iterator[A]): Iterator[Option[A]] = new Iterator[Option[A]](){
  private var hasBoomed: Boolean = false
  override def hasNext = !hasBoomed && it.hasNext

  override def next() = {
    Try(it.next()) match {
      case Failure(_) =>
        hasBoomed = true
        None
      case Success(value) => Some(value)
    }
  }
}

Now, assuming you library call return Mailing instances, you should wrap it in Option:

val myFuntionalSequence: Iterator[Option[(String, String)]] = for{
  mailingOpt <- scalaIterator(thirdPartyIterator)
} yield mailingOpt.map(mailing => (mailing.name, mailing.subject))

That will return a Iterator[Option[(String, Int)]].

Example results could be (when materialized): 

Some(mailing1Pair),Some(mailing2Pair),None // in case of an exception
Some(mailing1Pair),Some(mailing2Pair),Some(mailing3Pair) // no exception

Now, what if you actually don't care about that final None? And what if all you actually want is a List[(String, Int)] that contains all the successfully returned Mailings? In that case, you would do: 

val flattened: Iterator[(String, String)] = myFuntionalSequence.flatten

Now comes the really functional part ;)

But what if you actually want to know whether there was an exception? That is, you want to return None if there was at least one exception, or a Some(List(someMailing1Pair, etc....)) otherwise. Basically, what you want is a way to transform List[Option[(String, String)]] into an Option[List[(String, String)]].

Enter Traverse, from the Cats library: 

implicit cats.implicits._
val myFuntionalSeuence: Iterator[Option[(String, String)]] = ...
val flipped: Option[List[(String, String)]] = myFuntionalSeuence.toList.sequence //notice the .toList here: we are materializing a lazy collection here

...at which point you treat it as any other Option. This approach is generic, it works with anything that implements Traverse and I recommend that you would read the docs I linked: you will find plenty of goodies! :)

user_9471140

If you want to remain functional then you need a recursive function to unpick the unreliable iterator.

This is untested code, but it might look like this:

def safeIterate[T](i: Iterator[T]): List[Try[T]] = {
  @annotation.tailrec
  def loop(res: List[Try[T]]): List[Try[T]] =
    if (i.hasNext) {
      loop(Try(i.next) +: res)
    } else {
      res.reverse
    }

  loop(Nil)
}

You can check each Try value to see which iterations succeeded or failed. It you just want the success values then you can call .flatMap(_.toOption) on the List. Or use this version of safeIterate:

def safeIterate[T](i: Iterator[T]): List[T] = {
  @annotation.tailrec
  def loop(res: List[T]): List[T] =
    if (i.hasNext) {
      Try(i.next) match {
        case Success(t) => loop(t +: res)
        case _ => loop(res)
      }
    } else {
      res.reverse
    }

  loop(Nil)
}

Someone smarter than me can probably make this return another Iterator rather than a List.

user_76626700
Related