How to parse a JSON object (which is a list) into simple Scala class objects in scala?

Question

I've spent too much time trying to make this work, I'm new to Scala.

Basically I make a request to an API and get the following response:

[
  {
    "id": "bde585ea-43ad-4e62-9f20-ea721193e0a5",
    "clientId": "account",
    "realm":"test-realm-uqrw"
    "name": "${client_account}",
    "rootUrl": "${authBaseUrl}",
    "baseUrl": "/realms/test-realm-uqrw/account/",
    "surrogateAuthRequired": false,
    "enabled": true,
    "alwaysDisplayInConsole": false,
    "clientAuthenticatorType": "client-secret",
    "defaultRoles": [
      "manage-account",
      "view-profile"
    ],
    "redirectUris": [
      "/realms/test-realm-uqrw/account/*"
    ],
    "webOrigins": [],
    "protocol": "openid-connect",
    "attributes": {},
    "authenticationFlowBindingOverrides": {},
    "fullScopeAllowed": false,
    "nodeReRegistrationTimeout": 0,
    "defaultClientScopes": [
      "web-origins",
      "role_list",

    ],

    "access": {
      "view": true,
      "configure": true,
      "manage": true
    }
  },
  {..another object of the same type, different values },
  {..another object of the same type, different values }
]

I just need to extract the "id" field from any of those objects(I match by the realm attribute later on). Is there a simple way to convert that json list into a List[] of Map[String, Any]? I say Any because the type of values are so varied - booleans, strings, maps, lists.

I've tried a couple of methods (internal tools) and Jackson(error: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of scala.collection.immutable.List(no Creators, like default construct, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information) and the closest I got was to a weird list of Tuples giving me the incorrect result (because of incorrect processing).

What is a simple way to do this? Or am I destined to create a custom class for this API response? Or can I just walk down this JSON document (I just want one value from one of the objects in that array) and extract the value?

user_3298917 · Answer

One of the native and modern is Circe, in your case solution might look something like:

import io.circe._, io.circe.parser._, io.circe.generic.auto._, io.circe.syntax._

case class Response(
    id: String,
    clientId: String,
    realm: String,
    name: String,
    rootUrl: String,
    baseUrl: String,
    surrogateAuthRequired: Boolean,
    enabled: Boolean,
    alwaysDisplayInConsole: Boolean,
    clientAuthenticatorType: String,
    defaultRoles: List[String],
    redirectUris: List[String],
    webOrigins: List[String],
    protocol: String,
    fullScopeAllowed: Boolean,
    nodeReRegistrationTimeout: Int,
    defaultClientScopes: List[String],
    access: Access
)

case class Access(view: Boolean, configure: Boolean, manage: Boolean)

val json =
  s"""
       |[
       |  {
       |    "id": "bde585ea-43ad-4e62-9f20-ea721193e0a5",
       |    "clientId": "account",
       |    "realm":"test-realm-uqrw",
       |    "name": "client_account",
       |    "rootUrl": "authBaseUrl",
       |    "baseUrl": "/realms/test-realm-uqrw/account/",
       |    "surrogateAuthRequired": false,
       |    "enabled": true,
       |    "alwaysDisplayInConsole": false,
       |    "clientAuthenticatorType": "client-secret",
       |    "defaultRoles": [
       |      "manage-account",
       |      "view-profile"
       |    ],
       |    "redirectUris": [
       |      "/realms/test-realm-uqrw/account/*"
       |    ],
       |    "webOrigins": [],
       |    "protocol": "openid-connect",
       |    "fullScopeAllowed": false,
       |    "nodeReRegistrationTimeout": 0,
       |    "defaultClientScopes": [
       |      "web-origins",
       |      "role_list"
       |    ],
       |
       |    "access": {
       |      "view": true,
       |      "configure": true,
       |      "manage": true
       |    }
       |  }
       |]
       |""".stripMargin

println(parse(json).flatMap(_.as[List[Response]]))

Which will printout:

Right(List(Response(bde585ea-43ad-4e62-9f20-ea721193e0a5,account,test-realm-uqrw,client_account,authBaseUrl,/realms/test-realm-uqrw/account/,false,true,false,client-secret,List(manage-account, view-profile),List(/realms/test-realm-uqrw/account/*),List(),openid-connect,false,0,List(web-origins, role_list),Access(true,true,true))))

Scatie: https://scastie.scala-lang.org/5OpAUTjSTEWWTrH4X24vAg

The biggest advantage - unlike Jackson it's not based on runtime reflection, and instead, compile-time derivations.

UPDATE

As @LuisMiguelMejíaSuárez correctly suggested in comments section, if you would like to fetch only id field you can do it without full model parsing, like:

import io.circe._, io.circe.parser._

val json =
  s"""
       |[
       |  {
       |    "id": "bde585ea-43ad-4e62-9f20-ea721193e0a5"
       |  },
       |  {
       |    "id": "bde585ea-43ad-4e62-9f20-ea721193e0a6"
       |  }
       |]
       |""".stripMargin

println(parse(json).map(_.hcursor.values.map(_.map(_.hcursor.downField("id").as[String]))))

Print out:

Right(Some(Vector(Right(bde585ea-43ad-4e62-9f20-ea721193e0a5), Right(bde585ea-43ad-4e62-9f20-ea721193e0a6))))

Scatie: https://scastie.scala-lang.org/bSSZdLPyTJWcup2KIb4zAw

But be careful - manual JSON manipulations, something usually used in edge cases. I'd suggest to go with model derivation even for simple cases.

user_228843 · Answer

Use jsoniter-scala FTW!

It is handy in derivation and the most efficient in runtime. Extraction of JSON values is where it shines at most.

Please add the following dependencies:

libraryDependencies ++= Seq(
  // Use the %%% operator instead of %% for Scala.js  
  "com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-core"   % "2.6.4",
  // Use the "provided" scope instead when the "compile-internal" scope is not supported  
  "com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-macros" % "2.6.4" % "compile-internal"
)

Then for just id values no need to define fields or data structures for other values.

Just define a simplest data structure and parse immediately to it:

import com.github.plokhotnyuk.jsoniter_scala.macros._
import com.github.plokhotnyuk.jsoniter_scala.core._
import java.util.UUID

val json = """[
             |  {
             |    "id": "bde585ea-43ad-4e62-9f20-ea721193e0a5",
             |    "clientId": "account",
             |    "realm":"test-realm-uqrw",
             |    "name": "${client_account}",
             |    "rootUrl": "${authBaseUrl}",
             |    "baseUrl": "/realms/test-realm-uqrw/account/",
             |    "surrogateAuthRequired": false,
             |    "enabled": true,
             |    "alwaysDisplayInConsole": false,
             |    "clientAuthenticatorType": "client-secret",
             |    "defaultRoles": [
             |      "manage-account",
             |      "view-profile"
             |    ],
             |    "redirectUris": [
             |      "/realms/test-realm-uqrw/account/*"
             |    ],
             |    "webOrigins": [],
             |    "protocol": "openid-connect",
             |    "attributes": {},
             |    "authenticationFlowBindingOverrides": {},
             |    "fullScopeAllowed": false,
             |    "nodeReRegistrationTimeout": 0,
             |    "defaultClientScopes": [
             |      "web-origins",
             |      "role_list",
             |
             |    ],
             |
             |    "access": {
             |      "view": true,
             |      "configure": true,
             |      "manage": true
             |    }
             |  }
             |]""".stripMargin.getBytes("UTF-8")

case class Response(id: UUID)

implicit val codec: JsonValueCodec[List[Response]] = JsonCodecMaker.make

val responses = readFromArray(json)

println(responses)
println(responses.map(_.id))

Expected output:

List(Response(bde585ea-43ad-4e62-9f20-ea721193e0a5))
List(bde585ea-43ad-4e62-9f20-ea721193e0a5)

Feel free to ask for help here or in the gitter chat if it is need to handle your data differently or yet more efficiently.

user_7585327 · Answer

You can use play json, it is very simple.

import play.api.libs.json._

case class Access(view: Boolean, configure: Boolean, manage: Boolean)
case class Response(
    id: String,
    clientId: String,
    realm: String,
    name: String,
    rootUrl: String,
    baseUrl: String,
    surrogateAuthRequired: Boolean,
    enabled: Boolean,
    alwaysDisplayInConsole: Boolean,
    clientAuthenticatorType: String,
    defaultRoles: List[String],
    redirectUris: List[String],
    webOrigins: List[String],
    protocol: String,
    fullScopeAllowed: Boolean,
    nodeReRegistrationTimeout: Int,
    defaultClientScopes: List[String],
    access: Access
)



val string =
  s"""
       |[
       |  {
       |    "id": "bde585ea-43ad-4e62-9f20-ea721193e0a5",
       |    "clientId": "account",
       |    "realm":"test-realm-uqrw",
       |    "name": "client_account",
       |    "rootUrl": "authBaseUrl",
       |    "baseUrl": "/realms/test-realm-uqrw/account/",
       |    "surrogateAuthRequired": false,
       |    "enabled": true,
       |    "alwaysDisplayInConsole": false,
       |    "clientAuthenticatorType": "client-secret",
       |    "defaultRoles": [
       |      "manage-account",
       |      "view-profile"
       |    ],
       |    "redirectUris": [
       |      "/realms/test-realm-uqrw/account/*"
       |    ],
       |    "webOrigins": [],
       |    "protocol": "openid-connect",
       |    "fullScopeAllowed": false,
       |    "nodeReRegistrationTimeout": 0,
       |    "defaultClientScopes": [
       |      "web-origins",
       |      "role_list"
       |    ],
       |
       |    "access": {
       |      "view": true,
       |      "configure": true,
       |      "manage": true
       |    }
       |  }
       |]
       |""".stripMargin

implicit val ac = Json.format[Access]
implicit val res = Json.format[Response]

println(Json.parse(string).asInstanceOf[JsArray].value.map(_.as[Response]))

to avoid exception-

val responseOpt = Json.parse(string) match {
        case JsArray(value: collection.IndexedSeq[JsValue]) => value.map(_.asOpt[Response])
        case _ => Seq.empty
      }

see : https://scastie.scala-lang.org/RBUHhxxIQAGcKgk9a9iwIA

Here is the doc : https://www.playframework.com/documentation/2.8.x/ScalaJson

user_2359227 · Answer

Another option using play-json, is to define a path:

val jsPath = JsPath \ "id"

Then to apply it:

jsPath(Json.parse(jsonString))

Code run at Scastie.

How to parse a JSON object (which is a list) into simple Scala class objects in scala?

4 Answers