Scala better approach or solution - get average diff between list of records

user_16298345 0
Viewed 48

Below are series of record logs:

1,1000,open

2,1000,open

1,1002,close

2,1002,close

The first field is user id, second field time epoch, and third field status of the session. The session time for the user is the time difference between adjacent open close.For example in the above case.

for user 1 it is 1002-1000 =2

The goal is to find the average session time per user for the record logs

since we are not given exact details of the dataframe of the input we are free to make assumptions for the input datatype

This is my below appraoch: Code:

  object Test extends App {

  case class UserLog(id: Int, epoch: Long, state: String)

  case class StateDifference(currentState: String, currentDiffence: Long)

  val ll = List(UserLog(1, 1000, "open"), UserLog(2, 1000, "open"), UserLog(1, 1002, "close"), UserLog(2, 1002, "close"))

  val result = ll.foldLeft(Map[Int, StateDifference]()) { (userLogMap, cur) =>
    userLogMap.get(cur.id) match {
      case Some(userLog) if userLog.currentState == "open" && cur.state == "close" => userLogMap + (cur.id -> StateDifference(cur.state, (cur.epoch - userLog.currentDiffence)))
      case _ => userLogMap + (cur.id -> StateDifference(cur.state, cur.epoch))
    }
  }
  for ((k, v) <- result) {
    println(s"$k -> ${v.currentDiffence}")
  }

}

O/P: 1 -> 2 2 -> 2

Instead of case classes we can also take the input in List of tuples, then maybe use sliding func, groupBy to ..etc

Shootout amzing scala ppl with differnt solutions, better approach are really highly appretaited!!

immutability data-structures scala
1 Answers

You can use groupMap to get all records of the same user.
Then, we can pair each record with the next one, filter all the pairs that are Open -> Close and compute the duration of those sessions and finally calculate the average of those durations:

def avergeSessionTimePerUser(records: List[UserLog]): Map[Int, Long] =
  records
    .groupMap(record => record.id)(record => (record.state -> record.epoch))
    .view
    .mapValues { logs =>
      val (sum, count) =
        (logs lazyZip logs.tail).collect {
          case ((State.Open, openTime), (State.Close, closeTime)) =>
            closeTime - openTime
        }.foldLeft((0L -> 0L)) {
          case ((accSum, accCount), sessionDuration) =>
            (accSum + sessionDuration) -> (accCount + 1)
        }
      
      sum / count
    }.toMap

You can see the code running here.

user_41114040
Related