Reading from ifstream won't read whitespace

user_298661 0
Viewed 60174

I'm implementing a custom lexer in C++ and when attempting to read in whitespace, the ifstream won't read it out. I'm reading character by character using >>, and all the whitespace is gone. Is there any way to make the ifstream keep all the whitespace and read it out to me? I know that when reading whole strings, the read will stop at whitespace, but I was hoping that by reading character by character, I would avoid this behaviour.

Attempted: .get(), recommended by many answers, but it has the same effect as std::noskipws, that is, I get all the spaces now, but not the new-line character that I need to lex some constructs.

Here's the offending code (extended comments truncated)

while(input >> current) {
    always_next_struct val = always_next_struct(next);
    if (current == L' ' || current == L'\n' || current == L'\t' || current == L'\r') {
        continue;
    }
    if (current == L'/') {
        input >> current;
        if (current == L'/') {
            // explicitly empty while loop
            while(input.get(current) && current != L'\n');
            continue;
        }

I'm breaking on the while line and looking at every value of current as it comes in, and \r or \n are definitely not among them- the input just skips to the next line in the input file.

c++11 c++
10 Answers

You could just Wrap the stream in a std::streambuf_iterator to get data with all whitespaces and newlines like this .

           /*Open the stream in default mode.*/
            std::ifstream myfile("myfile.txt");

            if(myfile.good()) {
                /*Read data using streambuffer iterators.*/
    vector<char> buf((std::istreambuf_iterator<char>(myfile)), (std::istreambuf_iterator<char>()));

                /*str_buf holds all the data including whitespaces and newline .*/
                string str_buf(buf.begin(),buf.end());

                myfile.close();
            }
user_62196260

By default, this skipws flag is already set on the ifstream object, so we must disable it. The ifstream object has these default flags because of std::basic_ios::init, called on every new ios_base object (more details). Any of the following would work:

in_stream.unsetf(std::ios_base::skipws);
in_stream >> std::noskipws; // Using the extraction operator, same as below
std::noskipws(in_stream); // Explicitly calling noskipws instead of using operator>>

Other flags are listed on cpp reference.

user_73658660

Just Use getline.

while (getline(input,current))
{
      cout<<current<<"\n";

}
user_38592200
Related