First of all, the return type of the second sum should be (Int => Int) => Int, because it's supposed to return a function that accepts an Int => Int function, not an Int number.
Unfortunately you can't iterate over x using recursion of ret_fun, because x is not its argument. However you could use an indirect recursion of sum
def sum(x: Int): (Int => Int) => Int = {
def ret_fun(f: Int => Int): Int =
if (x == 1) f(1)
else f(x) + sum(x - 1)(f)
ret_fun
}
I imagine it may defeat the purpose of your exercise, because it's essentially the same as saying:
def sum(x: Int)(f: Int => Int): Int =
if (x == 1) f(1)
else f(x) + sum(x - 1)(f)
If your idea was to implement ret_fun using direct recursion and achieve the full analogy with the first example, let me comment on why it's problematic.
The reason is that the stop condition x == 1 is defined in terms of x and not in terms of f(x), so it's hard to determine which invocation should recurse and which shouldn't without changing the logic of the algorithm
def sum(x: Int): (Int => Int) => Int = {
def ret_fun(f: Int => Int): Int =
if (???) f(1)
else f(x) + ret_fun(x => f(x - 1))
ret_fun
}
We can, for instance, try if (f(x) == 0) 0, but as I said it will pretty much break the algorithm and will calculate f(x) + f(x - 1) + ... + 0 instead of f(x) + f(x - 1) + ... + f(1)