parse nested function to extract each inner function in python

user_14998952 0
Viewed 216

I have a nested expression as below

 expression = 'position(\'a\' IN Concat("function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ,  "function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ))'

I want the output as by retreiving nested function first and then outer functions

['Concat("function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ,  "function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" )','position(\'a\' IN Concat("function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ,  "function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ))']

Below is the code I have tried

 result = []
 for i in range(len(expression)):
     if expression[i]=="(":
        a.append(i)
     elif expression[i]==")":
          fromIdx=a.pop()
          fromIdx2=max(a[-1],expression.rfind(",", 0, fromIdx))
          flag=False
          for (fromIndex, toIndex) in first_Index:
             if fromIdx2 + 1 >= fromIndex and i <= toIndex:
                 flag=True
                 break
          if flag==False:
             result.append(expression[fromIdx2+1:i+1])

But this works only if expression is separated by ',' for ex:

expression = 'position(\'a\' , Concat("function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ,  "function_test"."PRODUCT_CATEGORIES"."CATEGORY_NAME" ))'

and result for this expression from my code will be correct as exprected. In first expression ,I mentioned ,there is IN operator instead of ',' hence my code doesnt work.

Please help

python
1 Answers

If you want it to be reliable, you need a full-fledged SQL parser. Fortunately, there is an out-of-box solution for that: https://pypi.org/project/sqlparse/. As soon as you have a parsed token tree, you can walk through it and do what you need:

import sqlparse

def extract_functions(tree):
    res = []
    def visit(token):
        if token.is_group:
            for child in token.tokens:
                visit(child)
        if isinstance(token, sqlparse.sql.Function):
            res.append(token.value)
    visit(tree)
    return res

extract_functions(sqlparse.parse(expression)[0])

Explanation. sqlparse.parse(expression) parses the string and returns a tuple of statements. As there is only one statement in the example, we can just take the first element. If there are many statements, you should rather iterate over all tuple elements.

extract_functions recursively walks over a parsed token tree depth first (since you want inner calls appear before outer ones) using token.is_group to determine if the current token is a leaf, tests if the current token is a function, and if it is, appends its string representation (token.value) to the result list.

user_112554470
Related