Jython - Scala array/list not JSON serializable within python script

Viewed 138

I have a Scala class that wraps an avro record with getters and setters. Using Jython to allow users to write python scripts to process the Avro record and ultimately do a json.dumps on the new processed record.

The issue is, if the user wants to grab a value that is an Array from the record, the interpreter complains that the object is not JSON serializable.

import json
json.dumps(<AClass>.getArray('myArray'))

The AClass is made available to any given python script at run time. Scala AClass:

class AClass {
  def getArray(fieldName: String): Array[Integer] = { 
    val value: GenericData.Array[T] = [....]
        value
      .asInstanceOf[GenericData.Array[T]]
      .asScala
      .toArray[T]
  }
}

I've tried a few other return types, 1) List[Integer], 2) mutable.Buffer[Integer], just the plain Avro generic Array 3) GenericData.Array[T]. All give the same serialization error with the slightly varying objects:

  1. Runtime exception occurred during Python processing. TypeError: List(1, 2, 3) is not JSON serializable.
  2. ... Buffer(1, 2, 3) is not JSON serializable
  3. ... [1, 2, 3] is not JSON serializable.

Now it seems that if we were to convert it to a list() from within the python script, it works fine. This gave some leads but need it to happen at the Scala level.

import json
json.dumps(list(<AClass>.getArray('myArray')))

Is there any way to achieve this? What Scala / Java list type would translate directly into the python list type and/or be JSON serializable within the Jython py interpreter?

1 Answers
Related