How to read and print contents of text file line by line?

So, I have no clue on how to have CMD echo lines from a *.txt text file one at a time with a tiny delay to make it seem like it’s processing.

Is this even possible with a batch alone?

I’ve tried doing research, but I can’t find sufficient text manipulation to be able to do this, but I do know how to make a pause between each command and how to do loops.

3 Answers 3

Let us assume the text file TestFile.txt should be output line by line which is an ANSI encoded text file with just ASCII characters containing this text:

The batch file below outputs this text file line by line with one second delay between each line with the exception of second line which is completely empty.

The strange looking character ¿ after eol= is an inverted question mark with hexadecimal Unicode value 00BF used to output third line correct. A line with an inverted question mark at beginning would not be output because of this redefinition of end of line character.

This batch file code is not designed to output any type of text file with any type of character encoding independent on which characters contains the text file. The Windows command line environment is not designed for output of any text file.

It is also possible to use a different, unquoted syntax to specify the FOR options delims , eol and usebackq to define an empty list of delimiters and no end of line character:

Thanks goes to aschipfl for this alternate syntax of the three FOR options with using escape character ^ to escape the equal signs and spaces in not double quoted options string to get interpreted by cmd.exe the string usebackq delims= eol= as one argument string for for /F .

There is ( instead of a space as usually used to output also correct line 7 with just a tab and some normal spaces. See also DosTips forum topic ECHO. FAILS to give text or blank line — Instead use ECHO/. echo/%%I does not correct output line 9 starting with a question mark.

It is not possible to define with an option that FOR does not ignore empty lines. But it is possible with FIND or FINDSTR to output a text file with all lines with a line number at beginning and so having no empty line anymore. The line number is enclosed in square brackets (FIND) or separated with a colon (FINDSTR) from rest of the line. It would be possible to assign to loop variable only the string after first sequence of ] or : after line number which in most cases means the entire line as in text file. But if a line in text file starts by chance with ] or : , FOR would remove this delimiter character too. The solution is this code:

The entire line with line number and colon output by FINDSTR executed in a separate command processes started by FOR with %ComSpec% /c and the command line within ‘ as additional arguments is assigned to loop variable I which is assigned next to environment variable Line .

Then delayed expansion is enabled as needed for next line which results in pushing address of current environment variables list on stack as well as current directory path, state of command extensions and state of delayed expansion before creating a copy of the current environment variables list.

Next the value of environment variable Line is output, but with substituting everything up to first colon by nothing which results in the output of the real line as stored in text file without the line number and the colon inserted at beginning by FINDSTR.

Finally the created copy of environment variables list is deleted from memory, and previous states of delayed expansion and command extension are popped from stack and set as well as the current directory path is set again as current directory and previous address of environment variables list is restored to restore the list of environment variables.

It is of course not very efficient to run for each line in text file the commands setlocal EnableDelayedExpansion and endlocal doing much more than just enabling/disabling delayed expansion, but this is necessary here to get lines with an exclamation mark correct assigned to environment variable Line and process next correct the value of Line . The efficiency loss is not really problematic here because of the delay of one second between output of each line.

For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.

Solarian Programmer

My programming ramblings

C Programming — read a file line by line with fgets and getline, implement a portable getline version

Posted on April 3, 2019 by Paul

In this article, I will show you how to read a text file line by line in C using the standard C function fgets and the POSIX getline function. At the end of the article, I will write a portable implementation of the getline function that can be used with any standard C compiler.

Reading a file line by line is a trivial problem in many programming languages, but not in C. The standard way of reading a line of text in C is to use the fgets function, which is fine if you know in advance how long a line of text could be.

You can find all the code examples and the input file at the GitHub repo for this article.

Let’s start with a simple example of using fgets to read chunks from a text file. :

For testing the code I’ve used a simple dummy file, lorem.txt. This is a piece from the output of the above program on my machine:

The code prints the content of the chunk array, as filled after every call to fgets, and a marker string.

If you watch carefully, by scrolling the above text snippet to the right, you can see that the output was truncated to 127 characters per line of text. This was expected because our code can store an entire line from the original text file only if the line can fit inside our chunk array.

What if you need to have the entire line of text available for further processing and not a piece of line ? A possible solution is to copy or concatenate chunks of text in a separate line buffer until we find the end of line character.

Let’s start by creating a line buffer that will store the chunks of text, initially this will have the same length as the chunk array:

Next, we are going to append the content of the chunk array to the end of the line string, until we find the end of line character. If necessary, we’ll resize the line buffer:

Please note, that in the above code, every time the line buffer needs to be resized its capacity is doubled.

This is the result of running the above code on my machine. For brevity, I kept only the first lines of output:

You can see that, this time, we can print full lines of text and not fixed length chunks like in the initial approach.

Let’s modify the above code in order to print the line length instead of the actual text:

This is the result of running the modified code on my machine:

In the next example, I will show you how to use the getline function available on POSIX systems like Linux, Unix and macOS. Microsoft Visual Studio doesn’t have an equivalent function, so you won’t be able to easily test this example on a Windows system. However, you should be able to test it if you are using Cygwin or Windows Subsystem for Linux.

Please note, how simple is to use POSIX’s getline versus manually buffering chunks of line like in my previous example. It is unfortunate that the standard C library doesn’t include an equivalent function.

When you use getline, don’t forget to free the line buffer when you don’t need it anymore. Also, calling getline more than once will overwrite the line buffer, make a copy of the line content if you need to keep it for further processing.

This is the result of running the above getline example on a Linux machine:

It is interesting to note, that for this particular case the getline function on Linux resizes the line buffer to a max of 960 bytes. If you run the same code on macOS the line buffer is resized to 1024 bytes. This is due to the different ways in which getline is implemented on different Unix like systems.

As mentioned before, getline is not present in the C standard library. It could be an interesting exercise to implement a portable version of this function. The idea here is not to implement the most performant version of getline, but rather to implement a simple replacement for non POSIX systems.

We are going to take the above example and replace the POSIX’s getline version with our own implementation, say my_getline. Obviously, if you are on a POSIX system, you should use the version provided by the operating system, which was tested by countless users and tuned for optimal performance.

The POSIX getline function has this signature:

Since ssize_t is also a POSIX defined type, usually a 64 bits signed integer, this is how we are going to declare our version:

In principle we are going to implement the function using the same approach as in one of the above examples, where I’ve defined a line buffer and kept copying chunks of text in the buffer until we found the end of line character:

This is how we can use the above function, for simplicity I kept the code and the function definition in the same file:

The above code gives the same results as the code that uses the POSIX’s getline function:

I’ve also tested the code with the Microsoft C compiler and gives identical results for the same input file.

If you want to learn more about C99/C11 I would recommend reading 21st Century C: C Tips from the New School by Ben Klemens:

Read file line by line using ifstream in C++

The contents of file.txt are:

Where 5 3 is a coordinate pair. How do I process this data line by line in C++?

I am able to get the first line, but how do I get the next line of the file?

8 Answers 8

First, make an ifstream :

The two standard methods are:

Assume that every line consists of two numbers and read token by token:

Line-based parsing, using string streams:

You shouldn’t mix (1) and (2), since the token-based parsing doesn’t gobble up newlines, so you may end up with spurious empty lines if you use getline() after token-based extraction got you to the end of a line already.

Use ifstream to read data from a file:

If you really need to read line by line, then do this:

But you probably just need to extract coordinate pairs:

Update:

In your code you use ofstream myfile; , however the o in ofstream stands for output . If you want to read from the file (input) use ifstream . If you want to both read and write use fstream .

Reading a file line by line in C++ can be done in some different ways.

[Fast] Loop with std::getline()

The simplest approach is to open an std::ifstream and loop using std::getline() calls. The code is clean and easy to understand.

[Fast] Use Boost’s file_description_source

Another possibility is to use the Boost library, but the code gets a bit more verbose. The performance is quite similar to the code above (Loop with std::getline()).

[Fastest] Use C code

If performance is critical for your software, you may consider using the C language. This code can be 4-5 times faster than the C++ versions above, see benchmark below

Benchmark — Which one is faster?

I have done some performance benchmarks with the code above and the results are interesting. I have tested the code with ASCII files that contain 100,000 lines, 1,000,000 lines and 10,000,000 lines of text. Each line of text contains 10 words in average. The program is compiled with -O3 optimization and its output is forwarded to /dev/null in order to remove the logging time variable from the measurement. Last, but not least, each piece of code logs each line with the printf() function for consistency.

The results show the time (in ms) that each piece of code took to read the files.

The performance difference between the two C++ approaches is minimal and shouldn’t make any difference in practice. The performance of the C code is what makes the benchmark impressive and can be a game changer in terms of speed.

What’s the fastest way to read a text file line-by-line?

I want to read a text file line by line. I wanted to know if I’m doing it as efficiently as possible within the .NET C# scope of things.

This is what I’m trying so far:

8 Answers 8

To find the fastest way to read a file line by line you will have to do some benchmarking. I have done some small tests on my computer but you cannot expect that my results apply to your environment.

Using StreamReader.ReadLine

This is basically your method. For some reason you set the buffer size to the smallest possible value (128). Increasing this will in general increase performance. The default size is 1,024 and other good choices are 512 (the sector size in Windows) or 4,096 (the cluster size in NTFS). You will have to run a benchmark to determine an optimal buffer size. A bigger buffer is — if not faster — at least not slower than a smaller buffer.

The FileStream constructor allows you to specify FileOptions. For example, if you are reading a large file sequentially from beginning to end, you may benefit from FileOptions.SequentialScan . Again, benchmarking is the best thing you can do.

Using File.ReadLines

This is very much like your own solution except that it is implemented using a StreamReader with a fixed buffer size of 1,024. On my computer this results in slightly better performance compared to your code with the buffer size of 128. However, you can get the same performance increase by using a larger buffer size. This method is implemented using an iterator block and does not consume memory for all lines.

Using File.ReadAllLines

This is very much like the previous method except that this method grows a list of strings used to create the returned array of lines so the memory requirements are higher. However, it returns String[] and not an IEnumerable allowing you to randomly access the lines.