Parse file string into arraylist of arraylist

user_8838089 0
Viewed 1488

I'm trying to write the following method- to summarize it's supposed to take a file and parse individual words and punctuation (not apostrophe's)from the string into arraylist of arraylist. I've been struggling with this for hours and can't find anything that would help me understand where I'm going wrong - any help would be appreciated:

/**
 * Opens and reads the contents of the input file specified in fileName. The input file is read
 * line by line. Each line is split into words and punction (excluding the apostrophe) and
 * stored in an ArrayList of Strings. These ArrayLists representing the line are stored in an
 * ArrayList of ArrayLists of Strings. Specifically, they are put in the ArrayList fileByLine
 * that is passed in as a parameter.
 *
 * For example, a file containing the following: 
 * Lorem ipsum dolor sit amet, consectetur adipiscing elit. Don'ec elementum tortor in mauris 
 * consequat vulputate.
 *
 * Would produce an ArrayList of ArrayLists containing 2 ArrayLists of Strings.
 * The first ArrayList would contain:
 *   "Lorem", "ipsum", "dolor", "sit", "amet", ",", "consectetur", "adipiscing", "elit", ".", 
 *   "Don'ec", "elementum", "tortor", "in", "mauris"
 * The second Arraylist would contain:
 *   "consequat", "vulputate", "."
 *
 * Note 1: The text file is assumed to be UTF-8.
 * Note 2: There are no assumption about the length of the file or the length of the lines.
 * Note 3: All single quotes (') are assumed to be apostrophes.
 *
 * When opening the file, any FileNotFoundException is caught and the error message 
 * "Exception: File 'fileName' not found." followed by a new line is output,
 * where fileName is the name of the file that the method attempted to open.
 *
 * @param fileName The name of the input text file to parse.
 * @param fileByLine Reference to ArrayList to contain the contents of the file line by line, 
 *                   where each line is an ArrayList of Strings.
 * @throws IOException if an I/O error occurs when closing the file. FileNotFoundException is
 *                     caught when opening the file.
 */
public static void readInputFile(String fileName,
  ArrayList < ArrayList < String > > fileByLine) throws IOException {
  FileInputStream fileByteStream = null;
  Scanner inFS = null;
  ArrayList < String > splitByWord = new ArrayList < String > ();
  String line = "";
  String word = "";



  //trying to open the file
  try {
    fileByteStream = new FileInputStream(fileName); //May throw file not found exception
    inFS = new Scanner(fileByteStream);

    //Loop that cycles through the file
    while (inFS.hasNextLine()) {
      line = inFS.nextLine();
      for (String wordPunctuation: line.split("//s")) { //splits each line by whitespace
        word = "";
        for (int i = 0; i < wordPunctuation.length(); i++) { //splits punctuation from each word
          Character letter = wordPunctuation.charAt(i);
          if (Character.isLetter(letter) || letter == '\'') {
            word = word + letter;
          }
          //else {
          //word = letter.toString();
          //}
        }
        splitByWord.add(word); //adds each word to an ArrayList
      }
      fileByLine.add(splitByWord); //adds the above created ArrayList to an ArrayList
    }
  } catch (FileNotFoundException excpt) {
    System.out.print("Exception: File 'fileName' not found.\n" + fileName);
  } finally {
    if (fileName != null) {
      inFS.close();
    }
  }

}
punctuation arraylist java string
3 Answers

i don't really understand what you are really looking for but i wrote the following code which reads a file and returns all words that are not punctuation ... hope it helps !

public ArrayList<String> readFiles(File file) throws FileNotFoundException{
    ArrayList<String> liste=new ArrayList<String>();
    Scanner scanner = new Scanner(file).useDelimiter("[.,;\n\t:\\s(){}\"/?!«»]+");
     while(scanner.hasNext()) {
         liste.add(scanner.next());             
     }
     scanner.close();
     return liste;
}
user_72645440

Oki now i might be understanding it so try this following code.

public ArrayList<ArrayList<String>> readFiles(File file) throws FileNotFoundException{
     ArrayList<ArrayList<String>> liste=new ArrayList<ArrayList<String>>();
     Scanner scanner = new Scanner(file).useDelimiter("\n");

     while(scanner.hasNext()) {
         ArrayList<String> l=new ArrayList<String>();
         String[] s=scanner.next().split("[.,;\n\t:\\s(){}\"/?!«»]+");
         for (int i = 0; i < s.length; i++) {
            l.add(s[i]);
         }
         liste.add(l);           
     }
     scanner.close();
     return liste;
}
user_72645440

I am glad your code comments give an example that shows what you are trying to do. If you can use java 8 then this will work for you.

Here is what my output looks like:

java-8 list list

I did not use any special text formatting, just the default toString from the ArrayList (which uses comma as separators between elements)

The main method:

public static void main(String[] args) throws IOException {
   String filePath = "the_path_to_your_data_file";
   extractWordsAndPunctuation(filePath)
   .stream()
   .forEach(System.out::println);
}

The java 8 streaming functionalities are fit for such a problem.

Here is the extractWordsAndPunctuation method:

public static List<List<String>> extractWordsAndPunctuation(String filePath) throws IOException {
   return Files.lines(Paths.get(filePath)).map(line -> line.split("//s"))
   .map(lineArray -> 
   Arrays.asList(lineArray)).map(PonctuatedWords::extractPunctuation)
                .collect(Collectors.toList());
}

The Files class is from java nio and since java 8, it has a lines method.

lines(Path path) Read all lines from a file as a Stream.

As you can see the logic is very close to your own. The first map method splits the line and maps it to a string array. The second map method turns it into a List<String> and the third map includes punctuations also into the list. Finally .collect(Collectors.toList()) collects each List<String> into a global List<List<String>> and we get the desired result.

The extractPunctuation method:

public static List<String> extractPunctuation(List<String> source) {
   return reduce(source.stream().map(subwordsExtractor).collect(Collectors.toList()));
}

The source parameter here is the words from a line before extracting the punctuations as words of their own. The subwordsExtractor method takes care of that.

public static Function<String, List<String>> subwordsExtractor = PonctuatedWords::extractSubWords;

again, the extractSubWords method logic is based on yours:

public static List<String> extractSubWords(String word) {
   List<String> result = new ArrayList<>();
   String subWord = "";
   for (int i = 0; i < word.length(); i++) {
      if (Character.isLetter(word.charAt(i)) || word.charAt(i) == '\'') {
         subWord += word.charAt(i);
      } else {
         if (!("".equals(subWord))) {
            result.add(subWord);
            subWord = "" + word.charAt(i);
      }
    }
   } 
   if (!"".equals(subWord)) {
      result.add(subWord);
   }
   return result;
}

The complete example:

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.stream.Collectors;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.function.Function;

public class PonctuatedWords {

    public static List<String> extractSubWords(String word) {
        List<String> result = new ArrayList<>();
        String subWord = "";
        for (int i = 0; i < word.length(); i++) {
            if (Character.isLetter(word.charAt(i)) || word.charAt(i) == '\'') {
                subWord += word.charAt(i);
            } else {
                if (!("".equals(subWord))) {
                    result.add(subWord);
                    subWord = "" + word.charAt(i);
                }
            }
        }

        if (!"".equals(subWord)) {
            result.add(subWord);
        }

        return result;
    }

    public static Function<String, List<String>> subwordsExtractor = PonctuatedWords::extractSubWords;

    public static List<String> reduce(List<List<String>> source) {
        List<String> result = new ArrayList<>();
        for (List<String> strings : source) {
            result.addAll(strings);
        }
        return result;
    }

    public static List<String> extractPunctuation(List<String> source) {
        return reduce(source.stream().map(subwordsExtractor).collect(Collectors.toList()));
    }

    public static List<List<String>> extractWordsAndPunctuation(String filePath) throws IOException {
        return Files.lines(Paths.get(filePath)).map(line -> line.split("//s"))
                .map(lineArray -> Arrays.asList(lineArray)).map(PonctuatedWords::extractPunctuation)
                .collect(Collectors.toList());
    }

    public static void main(String[] args) throws IOException {
        String filePath = "the_path_to_your_data_file";
        extractWordsAndPunctuation(filePath)
        .stream()
        .forEach(System.out::println);
    }

}
user_44377800
Related