I'm attempting to create a parser to translate a "custom" file into JSON so I can more easily manipulate its contents (For argument's sake, call the "custom" formate a .qwerty).
I've already created a Lexer which breaks down the file into individual lexemes (tokens) which structure is [token_type, token_value]. Now I am struggling to parse the lexemes into their correct dictionaries, as it is difficult to insert data into a sub-sub-dictionary since Keys aren't constant. As well as insert data into arrays stored in dictionaries.
It should be noted I am attempting to sequentially parse tokens into an actual python json object then dump the json object.
An example of the file can be seen below, along with what the end result is meant to resemble.
FILE: ABC.querty
Dict_abc_1{
Dict_abc_2{
HeaderGUID="";
Version_TPI="999";
EncryptionType="0";
}
Dict_abc_3{
FamilyName="John Doe";
}
Dict_abc_4{
Array_abc{
{TimeStamp="2018-11-07 01:00:00"; otherinfo="";}
{TimeStamp="2018-11-07 01:00:00"; otherinfo="";}
{TimeStamp="2018-11-07 01:00:00"; otherinfo="";}
{TimeStamp="2018-11-07 02:53:57"; otherinfo="";}
{TimeStamp="2018-11-07 02:53:57"; otherinfo="";}
}
Dict_abc_5{
LastContact="2018-11-08 01:00:00";
BatteryStatus=99;
BUStatus=PowerOn;
LastCallTime="2018-11-08 01:12:46";
LastSuccessPoll="2018-11-08 01:12:46";
CallResult=Successful;
}
}
}
Code=999999;
FILE: ABC.json
{
"Dict_abc_1":{
"Dict_abc_2":{
"HeaderGUID":"",
"Version_TPI":"999",
"EncryptionType":"0"
},
"Dict_abc_3":{
"FamilyName":"John Doe"
},
"Dict_abc_4":{
"Array_abc":[
{"TimeStamp":"2018-11-07 01:00:00", "otherinfo":""},
{"TimeStamp":"2018-11-07 01:00:00", "otherinfo":""},
{"TimeStamp":"2018-11-07 01:00:00", "otherinfo":""},
{"TimeStamp":"2018-11-07 02:53:57", "otherinfo":""},
{"TimeStamp":"2018-11-07 02:53:57", "otherinfo":""}
],
"Dict_abc_5":{
"LastContact":"2018-11-08 01:00:00",
"BatteryStatus":99,
"BUStatus":"PowerOn",
"LastCallTime":"2018-11-08 01:12:46",
"LastSuccessPoll":"2018-11-08 01:12:46",
"CallResult":"Successful"
}
}
},
"Code":999999
}
Additional token information, Token types can either be (with possible values)
- IDENTIFIER contain the name of the variable identifier
- VARIABLE containing actual data belonging to the parent IDENTIFIER
- OPERATOR equal "="
- OPEN_BRACKET equal "{"
- CLOSE_BRACKET equal "}"
An example of ABC.querty's lexemes can be seen HERE
fundamental logical extract of main.py
def main():
content = open_file(file_name) ## read file
lexer = Lexer(content) ## create lexer class
tokens = lexer.tokenize() ## create lexems as seen in pastebin
parser = Parser(tokens).parse() ## create parser class given tokens
print(json.dumps(parser, sort_keys=True,indent=4, separators=(',', ': ')))
parser.py
import re
class Parser(object):
def __init__(self, tokens):
self.tokens = tokens
self.token_index = 0
self.json_object = {}
self.current_object = {}
self.path = [self.json_object]
def parse(self):
while self.token_index < len(self.tokens):
token = self.getToken()
token_type = token[0]
token_value = token[1]
print("%s \t %s" % (token_type, token_value))
if token_type in "IDENTIFIER":
self.increment()
identifier_type = self.getToken()
if identifier_type[0] in "OPEN_BRACKET":
identifier_two_type = self.getToken(1)
if identifier_two_type[0] in ["OPERATOR","IDENTIFIER"]:
## make dict in current dict
pass
elif identifier_two_type[0] in "OPEN_BRACKET":
## make array in current dict
pass
elif identifier_type[0] in "OPERATOR":
## insert data into current dict
pass
if token_type in "CLOSE_BRACKET":
identifier_type = self.getToken()
if "OPEN_BRACKET" in identifier_type[0]:
#still in array of current dict
pass
elif "IDENTIFIER" in identifier_type[0]:
self.changeDirectory()
else:
#end script
pass
self.increment()
print(self.path)
return self.json_object
def changeDirectory(self):
if len(self.path) > 0:
self.path = self.path.pop()
self.current_object = -1
def increment(self):
if self.token_index < len(self.tokens):
self.token_index+=1
def getToken(self, x=0):
return self.tokens[self.token_index+x]
Additional parse information, Currently, I was trying to store the current dictionary in a path array to allow me to insert into dictionaries and arrays within dictionaries.
Any suggestions or solutions are very much appreciated,
Thanks.