Append text to a group in a html document based on condition

user_13377970 0
Viewed 178

I am parsing a large html doc. I have used groups to "group" the text and separate using \n\n. The entire text is within the <font> </font> tags of the doc.

Each group has 5 fields, Serial#.........,Cust#...........,Customer Name...,BILL TO NO NAME.,DATE......

I need to use the Cust#........... from each "group" and compare it against every other group in the list to look for a duplicate Cust#............

If a duplicate is found then I need to append BILL TO NO NAME. into each group with the duplicate Cust#...........

Sample html:

Serial#......... 12345678974566321\nCust#........... 123456\nCustomer Name... Humpfrey Bear\nBILL TO NO NAME. Bill To: 001166 - Some Company\nDATE...... 01/01/00\n\n'Serial#......... sgfdsfd546545645\nCust#........... 123456\nCustomer Name... Humpfrey Bear\nBILL TO NO NAME. Bill To: 0165487 - Some Other Company\nDATE...... 01/01/00\n\n'Serial#......... Jgfdhdgfhgfdh4545\nCust#........... 88483\nCustomer Name... John Smith\nBILL TO NO NAME. Bill To: 0146897 - Some Company\nDATE...... 01/01/00\n\n'Serial#......... JF2SJads5dsafdsaf\nCust#........... 015648\nCustomer Name... Eric Cantona\nBILL TO NO NAME. Bill To: 8888154 - Man Utd\nDATE...... 01/01/00\n\n'Serial#......... JdsfrfdsgHG091797\nCust#........... 015648\nCustomer Name... Eric Cantona\nBILL TO NO NAME. Bill To: 9876524 - Big Big Company\nDATE...... 01/01/00\n\n'

The ouput I need is: Serial#......... 12345678974566321\nCust#........... 123456\nCustomer Name... Humpfrey Bear\nBILL TO NO NAME. Bill To: 001166 - Some Company Bill To: 0165487 - Some Other Company\nDATE...... 01/01/00\n\n',Serial#......... JF2SJads5dsafdsaf\nCust#........... 015648\nCustomer Name... Eric Cantona\nBILL TO NO NAME. Bill To: 8888154 - Man Utd Bill To: 9876524 - Big Big Company\nDATE...... 01/01/00\n\n'

My output has omitted the duplciates based on Cust#......xxxxx, but I am just trying to show my expected result a little clearer. I can sort the duplicates out later.

A shortened version of what I have thus far the rest is insignificant.

from bs4 import BeautifulSoup
import re
import urllib
import os



with open(r'.html', 'r') as f:
   html_string = f.read()

soup = BeautifulSoup(html_string, 'html.parser')

groups = soup.font.text.replace('Serial#', 'xxxSerial#').split('xxx')
beautifulsoup python
2 Answers

It's not exactly like the expected output in your question, but it may get you close enough:

bills = """[your html above]"""
groups = bills.replace('Serial#','xxxSerial#').split('xxx')
cust_nums = []
ng= []
for group in groups[1:]:
    items = group.split('\n')
    cn = items[1].split('. ')[1]
    bill = items[3].split('. ')[1]
    if cn not in cust_nums:
        cust_nums.append(items[1].split('. ')[1])            
    else:
        items[3]+=' '+bill
        ng.append(items[:-2])
ng

Output:

[['Serial#......... sgfdsfd546545645',
  'Cust#........... 123456',
  'Customer Name... Humpfrey Bear',
  'BILL TO NO NAME. Bill To: 0165487 - Some Other Company Bill To: 0165487 - Some Other Company',
  'DATE...... 01/01/00'],
 ['Serial#......... JdsfrfdsgHG091797',
  'Cust#........... 015648',
  'Customer Name... Eric Cantona',
  'BILL TO NO NAME. Bill To: 9876524 - Big Big Company Bill To: 9876524 - Big Big Company',
  'DATE...... 01/01/00']]
user_94480900

If your goal is to identify duplicate Cust# values, here's one approach:

import re

sep = "Serial#"

def get_cust_number(entry):
    """ Isolate the customer ID number. """
    pattern = re.compile("Cust#[^0-9]+(\d+)")
    return pattern.findall(entry)[0]

# return a list of dicts, each element has full entry and customer id
parsed = [{"full_entry": sep + x, "cust_id": get_cust_number(x)} 
          for x in data.split(sep) if x]

# example parsed element
{'full_entry': "Serial#......... 12345678974566321\nCust#........... 123456\nCustomer Name... Humpfrey Bear\nBILL TO NO NAME. Bill To: 001166 - Some Company\nDATE...... 01/01/00\n\n'",
  'cust_id': '123456'}

Now use a set to look for duplicate Cust# and store the duplicate entries in dupes:

seen = set()
dupes = list()

# iterate over each entry, store :cust_id: values in :seen: set.
# if :cust_id: is already in :seen:, it is a dupe.  store it in :dupes:.
for entry in parsed:
    if entry['cust_id'] in seen:
        dupes.append(entry['full_entry'])
    else:
        seen.add(entry['cust_id'])
        
dupes
["Serial#......... sgfdsfd546545645\nCust#........... 123456\nCustomer Name... Humpfrey Bear\nBILL TO NO NAME. Bill To: 0165487 - Some Other Company\nDATE...... 01/01/00\n\n'",
 "Serial#......... JdsfrfdsgHG091797\nCust#........... 015648\nCustomer Name... Eric Cantona\nBILL TO NO NAME. Bill To: 9876524 - Big Big Company\nDATE...... 01/01/00\n\n'"]

I don't understand why your output has the second "Humpfrey Bear" entry but the first "Eric Cantona" entry. This answer generates a list of only duplicate outputs (meaning the first entry isn't there).

user_27999410
Related