Placing word documents into dataframe

Question

I have a couple word documents and i want to collate their contents into a data frame before exporting into an excel. So far i have this code:

import docx2txt

my_word_files = glob.glob(r"C:\Users\.......\*.docx")

for file in my_word_files:
    word = docx2txt.process(file)

This turns the word documents into strings and the contents look like the below:

Questions

: What is your full name?

John Smith

etc....

Each question starts with a ':' and each answer is wrapped inbetween two . What i want to do is turn it onto a dataframe which would look like the below:

What is your full name?   Question2    Question3   etc...
John Smith                Answer2      Answer3

With each row being the answers from each word file so that everything is collated nicely.

user_9375102 · Answer

Assuming you are starting with a pandas dataframe, we can apply some logical operations before pivoting to your desired output.

from io import StringIO
import pandas as pd
import numpy as np

d = """: What is your full name?



John Smith


: What is your Age

25


: Where are you located

London, fam.


Engurrland


"""

df = pd.read_csv(StringIO(d),sep='	',header=None)

First, lets build a sequence so we can have a logical order of question + answers and remove any values with <> in the body.

df = df[~df[0].str.contains('<|>')]

df['sequence'] = df.groupby(df[0].str.contains('^:').cumsum()).cumcount()

df['questionSequence'] = np.where(
    df[0].str.contains("^:"), df.groupby(df[0].str.contains("^:")).cumcount(), np.nan
)

df['questionSequence'] = df['questionSequence'].ffill()

print(df)

                          0     sequence  questionSequence
0   : What is your full name?         0               0.0
2                  John Smith         1               0.0
4          : What is your Age         0               1.0
6                          25         1               1.0
9     : Where are you located         0               2.0
11               London, fam.         1               2.0
14                 Engurrland         2               2.0

Next, we want to split out the Question column to create a separate question & answer column and remove the pesky colon while we are at it.

df['question'] = np.where(df['sequence'].eq(0),df[0],np.nan)
df['question'] = df['question'].ffill().str.strip(': ')

This is shaping up quite nicely, now lets filter out the questions from column 0 by filtering on the sequence column.

df1 = df1 = df[(df['sequence'].ne(0))].copy()

print(df1)

              0    sequence  questionSequence                 question
2     John Smith         1               0.0  What is your full name?
6             25         1               1.0         What is your Age
11  London, fam.         1               2.0    Where are you located
14    Engurrland         2               2.0    Where are you located

finally, we will use a multi index to create your pivot in the same order of the questions.

final = pd.crosstab(
    df1["sequence"], [df1["questionSequence"], df1["question"]], df1[0], aggfunc="first"
)

Placing word documents into dataframe

1 Answers