How to read text file line by line which is excessing characters in batch file? Limiting the line length is allowed.(Windows, batch script)

user_8360655 0
Viewed 224
##test.txt##
First line = 1;*|:12345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345
Second line = 5;*|:3215432;*|:21543215432154321543215432154321543215432154321543215432154321543215;*|:543215;*|:5432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321543215432154321


SetLocal EnableDelayedexpansion
for /F "tokens=* delims=" %%a in ('type "test.txt"') do (
            set "Line=%%a"
            echo Line: "!Line!"
)

Obviously, the above code cannot read 2 lines in test.txt, since each line is over the limit(8191 characters).

By typing

type "test.txt"

It still resulted in the 2 lines as in the test.txt file. However, as soon as 2 lines get in the for loop, the lines cannot be read.

I do not need the whole length of line, so I tried to preprocess the text file, by deleting excessing part of the string. For example, limit the line length as 8100 in the above example.

1.How do I edit the above text file without using notepad(any GUI editors) in windows environment?

2.If I do not need to preprocess the file, what is the trick to avoid the above issue?

  • Even I cannot put carrot(^), which extends the line of command. What I want is it is fine to split the string and take first parts, but I do not want to do it manually via notepad or other GUI file editors.
cmd batch-file windows string
2 Answers

You can use a chunk reader, it splits each line into chunks of a maximum size of 1023 characters, each.

@echo off
setlocal EnableDelayedExpansion

set pos=0
set line=
<long_text.txt (
    FOR /F "tokens=1 delims=:" %%1 in ('findstr /o "^" long_text.txt') DO (
        set new_pos=%%1
        set /a size=new_pos-pos
        set /a "chunks=(size-1+1022) / 1023"
        if defined line echo Line: !line! chunks=!chunks!

        set /a pos=new_pos
        for /L %%# in (1 1 !chunks!) do (
            set "partial="
            set /p partial=     
            if defined partial (
                echo   #!line! chunk %%# -- !partial:~0,10! ... !partial:~-10! 
            )
        )
        set /a line+=1
    )
)

How it works

The outer FOR /F .. findstr /O loop is used to determine the length of each line, by calculating the positional difference between two lines.
The line length is used to calculate how many chunks has to be read to fetch the whole line.

A line itself is read by set /p (it reads from the redirection of <long_text.txt). set /p has a build in limit of 1023 characters.

Therefore set /p is used chunk times.

To calculate how many chunks are used for a single line, the line length has to be divided by 1023, but 1 character has to be subtracted, because the LF doesn't count (but the CR). The +1022 is a result of the fact, that the remaining characters after the last complete chunk has to be read, too.
The last chunk can be empty, if the line is a multiple of 1023 (also an empty line).

The only remaining point is the last line.
The last line will not be read by this technique, but it's easy enough to append one empty line before.

user_4631150

If you are on a supported Windows system, PowerShell will be available. Change the 20 to the maximum number of characters you need from each line.

powershell -NoLogo -NoProfile -Command ^
    "Get-Content -Path '.\longlines.txt' |" ^
        "ForEach-Object { $_.substring(0,[Math]::min($_.Length, 20)) }"
user_4479010
Related