Implementing a gradient descent from a single point in Numpy?

user_11028689 0
Viewed 1164

I have encountered this question in an online test. I am looking for advice on what approach to use, rather than a full solution.

You are walking on a mountain. You want to descend to the lowest point on the mountain and choose to apply a gradient descent to plan your route. the height at any location x,y is described by the function h:

h(x,y) = x*y**2*(sin(Pi*x) + cos (2Pi*y))

image

Implement a function that takes as input the floats x and y, which represent your position on the mountain. This returns a list of floats (dx,dy) which represents the gradient at this position. The function accepts float x, float y as parameters and is expected to return a float array. Feel free to import numpy.

def gradient(x,y):
   # your code

I have looked up the np.gradient() function which could give me 'a single array corresponding to the derivatives'. However, this takes an array as input yet I have a single point as an input?

So how would you convert this into an array? or, maybe I am expected to write a gradient descent algorithm from scratch using np.zero() arrays instead of np.gradient()? I would be grateful for any analogies, links and snippets of code with a similar approach, yet I'd really like to have a go at tackling this problem myself first.

gradient-descent numpy-ndarray python-3.x numpy
2 Answers

wasnt able to figure out an array solution yet,

reverting to SymPy library I got this code to retrieve a list of

dx, dy at point x,y given your function:

import sympy as sym
from sympy.parsing.sympy_parser import parse_expr

class FunZ:
    
    def __init__(self, f):  
  
        self.x, self.y  = sym.symbols('x y')
        
        self.f = parse_expr(f)
        
        # print('f : ', self.f)
        
    def evalu(self, xx, yy):
    
        return float(self.f.subs({self.x: xx, self.y: yy}).evalf())
    

    def derX(self, xx, yy):        
        
        self.dx = sym.diff(self.f, self.x)
        
        # print('dx : ', self.dx)
        
        return float(self.dx.subs({self.x: xx, self.y: yy}).evalf())
    
    def derY(self, xx, yy):
        
        self.dy = sym.diff(self.f, self.y)
        
        # print('dy :', self.dy)
        
        return float(self.dy.subs({self.x: xx, self.y: yy}).evalf())
    
    def derXY(self, xx, yy):
        
        return [float(self.derX(xx, yy)), float(self.derY(xx, yy))]
    


xx = -3
yy = -2.54

funz = FunZ('x * y ** 2 * (sin(pi * x) + cos(2 * pi * y))')

# print('evalu : ',funz.evalu(xx,yy))

print(
      funz.evalu(xx,yy), type(funz.evalu(xx,yy)),'\n',
      funz.derX(xx, yy), type(funz.derX(xx, yy)),'\n',
      funz.derY(xx, yy), type(funz.derY(xx, yy)),'\n'
      )

print('_________________')

print(funz.derXY(xx,yy), type(funz.derXY(xx, yy)))

output:

18.74673336701243 <class 'float'> 
 54.55598636936226 <class 'float'> 
 15.481918816962425 <class 'float'> 

_________________
[54.55598636936226, 15.481918816962425] <class 'list'>

not sure code is right, got it visualized throught:

import numpy as np

import sympy as sym
from sympy.parsing.sympy_parser import parse_expr

import matplotlib.pyplot as plt


class FunZ:
    
    def __init__(self, f):  
  
        self.x, self.y  = sym.symbols('x y')
        
        self.f = parse_expr(f)
        
        # print('f : ', self.f)
        
    def evalu(self, xx, yy):
    
        return float(self.f.subs({self.x: xx, self.y: yy}).evalf())
    

    def derX(self, xx, yy):        
        
        self.dx = sym.diff(self.f, self.x)
        
        # print('dx : ', self.dx)
        
        return float(self.dx.subs({self.x: xx, self.y: yy}).evalf())
    
    def derY(self, xx, yy):
        
        self.dy = sym.diff(self.f, self.y)
        
        # print('dy :', self.dy)
        
        return float(self.dy.subs({self.x: xx, self.y: yy}).evalf())
    
    def derXY(self, xx, yy):
        
        return [float(self.derX(xx, yy)), float(self.derY(xx, yy))]

XX = np.linspace(-3, 3, 100)

YY = np.linspace(-3, 3, 100)

funz = FunZ('x * y ** 2 * (sin(pi * x) + cos(2 * pi * y))')

ij = [(x, y, funz.evalu(x, y)) for x in XX for y in YY]


arr = np.array(ij, dtype=float)

# print(arr, arr.size, arr.shape, arr.dtype)


der_x = [(a, b, funz.derX(a, b)) for a in XX for b in YY] 

derX = np.array(der_x)

# print(derX, derX.size, derX.shape, derX.dtype)


der_y = [(a, b, funz.derY(a, b)) for a in XX for b in YY] 

derY = np.array(der_y)

# print(derY, derY.size, derY.shape, derY.dtype)


x = arr[:, 0]
y = arr[:, 1]

data = arr[:, 2]

fig = plt.figure()
ax = fig.add_subplot(221, projection="3d")
ax.plot_trisurf(x, y, data, color="red", alpha=0.5)

ax2 = fig.add_subplot(223, projection="3d")
ax2.plot_trisurf(
    x, y, derX[:, 2], color="blue", alpha=0.2) 

ax3 = fig.add_subplot(224, projection="3d")
ax3.plot_trisurf(
    x, y, derY[:, 2], color="green", alpha=0.2)  
plt.show()

output is:

output

the code, especially the latter is very very slow, so I would be interested in a faster solution too, but given my poor knowlegde of python I can only wait for suggestions

user_98770650

On second thought, I believe the solution here it simpler than it seems.. it is possible to take derivatives of h(x,y) mathematically, to get something like dh/dx = y^2sin (Pix) + Piy^2cos(Pix) + y^2cos(2Piy)..and dh/dy=.. and so on. Then we can do a float of two values like this, mylist = [dx, dy] and return an array of floats:

np.array(mylist, dtype = float).

or, using numpy.gradient():

def gradient(x,y):
    import numpy as np
    f = np.array([x,y], dtype = float)
    result = np.gradient(f)
return result

pity I could not test it. please vote if that solves it.

user_110286890
Related