Linux binary file standard input matches

Today grep made a log file and reported «Binary file (standard input) matches»

Opened the file with vi and looked at it, there are non-character garbled

This problem can be solved by adding —text or -a in grep

Intelligent Recommendation

When grep shows «matching to binary»

the reason grepIf you hit\000 NULCharacter, it will assume the file is a binary file, and grep match the default ignore binary data. So to usegrep -aProperties: binary data is not ignored. grepof-aor-.

Input and output (binary file, text file, text format option)

Input and output 1 Numpy binary file 1.1 numpy.save() 1.2 nuumpy.load() 1.3 numpy.savez() 2 Text file 2.1 numpy.savetxt() 2.2 &nb.

4.3 with grep search for text in the file

Command: grep Uses: text search Usage: grep match_pattern filename Supporting multiple formats match -E: support for regular expressions, egrep equivalent -i: Ignore capitalization styles -e: Matching.

How to solve the problem when python is packaged into exe file and the window flashes after clicking? An input technique

1. The first situation, Our packaged exe file, after normal operation, the window is closed, it is too late to see the operation log, how to solve it? The answer is to add an input prompt sentence at .

Solve text truncation problem when NSTextContainer pagination

Solve text truncation problem when NSTextContainer pagination NSTextContainer can be used with NSLayoutManager to page large text files. However, there will be problems during paging, and the display .

More Recommendation

Solve the problem of garbled characters when opening a txt text file in Sublime text3

Compilation is not easy, if reprinted, please state the source: Menghuihekou:http://blog.csdn.net/zxc514257857/article/details/79505982 Install Package Control Select View—>Show C.

Solve the problem of date display 0000-00-00 when importing mysql database, excel or text file import

I didn’t want to write about the resolution of bugs, but I made some mistakes in several posts on the Internet, please correct me, hahaha Importing MySQL If you use a text file or excel file to .

Solve the problem that the corresponding Design is not updated when the Text code is modified in the activity_main.xml file in Android studio

Solve the problem that the corresponding Design is not updated when the Text code is modified in the activity_main.xml file in Android studio problem Solution result problem When I used Android studio.

[Unity] Solve the problem that the text entered in the Android text input box is white

The project encountered white text in the input box on a colleague’s Samsung mobile phone. The problem was that the back panel of his input box was also white, making the text invisible. solutio.

One way to solve bert long text matches

introduction bert1After the proposed transfer learning to open the door, first by training corpus unsupervised common language model, and then fine-tuning based on their corpus (finetune) model to ach.

Источник

Marko Sutic’s Database Blog

Saturday, July 12, 2008

Binary file (standard input) matches

One of the most important obligations for DBA’s is regularly checking alert log file.

To help myself I’ve written shell script which connects to several unix machines and examines alert log files looking for ORA- errors or ‘Checkpoint not complete’ lines.

Script was working fine for several months but yesterday it reported ‘Binary file (standard input) matches’.

I’ve searched for that error and found solution:

Why does grep report «Binary file matches»?

If grep listed all matching «lines» from a binary file, it would probably generate output that is not useful, and it might even muck up your display. So GNU grep suppresses output from files that appear to be binary files. To force GNU grep to output lines even from files that appear to be binary, use the `-a’ or `—binary-files=text’ option. To eliminate the «Binary file matches» messages, use the `-I’ or `—binary-files=without-match’ option.

In my script I’m using egrep and grep powerful unix tools to search for specified pattern, so my problem was there. I don’t know what happened with that alert log file but grep supressed output like it was binary file.

To fix that problem I’ve added -a along the grep lines and everything worked fine.

Update:
I’ve found what was causing this error and fixed little bug in my script. On specified database server logrotate was started with compression option. I’m analyzing whole log files (rotated with logrotate also), so in this case I’ve tried to read compressed logfile with cat tool. As I was reading binary file with cat output was little fuzzy and unreadable. To fix this I’ve used less tool which shows compressed log file in decompressed version as a text. Now script works like a clock.

Источник

Linux egrep command

On Unix-like operating systems, the egrep command searches for a text pattern, using extended regular expressions to perform the match. Running egrep is equivalent to running grep with the -E option.

This page covers the GNU/Linux version of egrep.

Syntax

Options

-A NUM, —after-context=NUM	Print NUM lines of trailing context after matching lines. Places a line containing — between contiguous groups of matches.
-a, —text	Process a binary file as if it were text; this is equivalent to the —binary-files=text option.
-B NUM, —before-context=NUM	Print NUM lines of leading context before matching lines. Places a line containing — between contiguous groups of matches.
-C NUM, —context=NUM	Print NUM lines of output context. Places a line containing — between contiguous groups of matches.
-b, —byte-offset	Print the byte offset in the input file before each line of output.
—binary-files=TYPE	If the first few bytes of a file indicate that the file contains binary data, assume that the file is of type TYPE. By default, TYPE is binary, and grep normally outputs either a one-line message saying that a binary file matches, or no message if there is no match. If TYPE is without-match, grep assumes that a binary file does not match; this is equivalent to the -I option. If TYPE is text, grep processes a binary file as if it were text; this is equivalent to the -a option. Warning: grep —binary-files=text might output binary garbage, which can have nasty side effects if the output is a terminal and if the terminal driver interprets some of it as commands.
—colour[=WHEN], —color[=WHEN]	Surround the matching string with the marker find in GREP_COLOR environment variable. WHEN may be ‘never‘, ‘always‘, or ‘auto‘
-c, —count	Suppress normal output; instead print a count of matching lines for each input file. With the -v, —invert-match option (see below), count non-matching lines.
-D ACTION, —devices=ACTION	If an input file is a device, FIFO or socket, use ACTION to process it. By default, ACTION is read, which means that devices are read as if they were ordinary files. If ACTION is skip, devices are silently skipped.
-d ACTION, —directories=ACTION	If an input file is a directory, use ACTION to process it. By default, ACTION is read, which means that directories are read as if they were ordinary files. If ACTION is skip, directories are silently skipped. If ACTION is recurse, grep reads all files under each directory, recursively; this is equivalent to the -r option.
-e PATTERN, —regexp=PATTERN	Use PATTERN as the pattern; useful to protect patterns beginning with «—«.
-F, —fixed-strings	Interpret PATTERN as a list of fixed strings, separated by newlines, that may be matched.
-P, —perl-regexp	Interpret PATTERN as a Perl regular expression.
-f FILE, —file=FILE	Obtain patterns from FILE, one per line. The empty file contains zero patterns, and therefore matches nothing.
-G, —basic-regexp	Interpret PATTERN as a basic regular expression (see below). This is the default.
-H, —with-filename	Print the file name for each match.
-h, —no-filename	Suppress the prefixing of file names on output when multiple files are searched.
—help	Output a brief help message.
-I	Process a binary file as if it did not contain matching data; this is equivalent to the —binary-files=without-match option.
-i, —ignore-case	Ignore case distinctions in both the PATTERN and the input files.
-L, —files-without-match	Suppress normal output; instead print the name of each input file from which no output would normally print. The scanning stops on the first match.
-l, —files-with-matches	Suppress normal output; instead print the name of each input file from which output would normally print. The scanning stops on the first match.
-m NUM, —max-count=NUM	Stop reading a file after NUM matching lines. If the input is standard input from a regular file, and NUM matching lines are output, grep ensures that the standard input is positioned to after the last matching line before exiting, regardless of the presence of trailing context lines. This enables a calling process to resume a search. When grep stops after NUM matching lines, it outputs any trailing context lines. When the -c or —count option is also used, grep does not output a count greater than NUM. When the -v or —invert-match option is also used, grep stops after outputting NUM non-matching lines.
—mmap	If possible, use the mmap system call to read input, instead of the default read system call. In some situations, —mmap yields better performance. However, —mmap can cause undefined behavior (including core dumps) if an input file shrinks while grep is operating, or if an I/O error occurs.
-n, —line-number	Prefix each line of output with the line number within its input file.
-o, —only-matching	Show only the part of a matching line that matches PATTERN.
—label=LABEL	Displays input actually coming from standard input as input coming from file LABEL. This is especially useful for tools like zgrep, e.g., gzip -cd foo.gz \|grep -H —label=foo something
—line-buffered	Use line buffering. This can incur a performance penalty.
-q, —quiet, —silent	Be quiet; do not write anything to standard output. Exit immediately with zero status if any match is found, even if an error was detected. Also see the -s or —no-messages option.
-R, -r, —recursive	Read all files under each directory, recursively; this is equivalent to the -d recurse option. Modified by:

Recurse in directories only searching file matching PATTERN.

Recurse in directories skip file matching PATTERN. -s, —no-messages Suppress error messages about nonexistent or unreadable files. Portability note: unlike GNU grep, traditional grep did not conform to POSIX.2, because traditional grep lacked a -q option and its -s option behaved like GNU grep’s -q option. Shell scripts intended to be portable to traditional grep should avoid both -q and -s and should redirect output to /dev/null instead. -U, —binary Treat the file(s) as binary. By default, under MS-DOS and Microsoft Windows, grep guesses the file type by looking at the contents of the first 32 KB read from the file. If grep decides the file is a text file, it strips the CR characters from the original file contents (to make regular expressions with ^ and $ work correctly). Specifying -U overrules this guesswork, causing all files to be read and passed to the matching mechanism verbatim; if the file is a text file with CR/LF pairs at the end of each line, this causes some regular expressions to fail. This option has no effect on platforms other than MS-DOS and MS-Windows. -u, —unix-byte-offsets Report Unix-style byte offsets. This switch causes grep to report byte offsets as if the file were Unix-style text file, i.e., with CR characters stripped off. This produces results identical to running grep on a Unix machine. This option has no effect unless -b option is also used; it has no effect on platforms other than MS-DOS and MS-Windows. -V, —version Print the version number of grep to standard error. This version number should be included in all bug reports (see below). -v, —invert-match Invert the sense of matching, to select non-matching lines. -w, —word-regexp Select only those lines containing matches that form whole words. The test is that the matching substring must either be at the beginning of the line, or preceded by a non-word constituent character. Similarly, it must be either at the end of the line or followed by a non-word constituent character. Word constituent characters are letters, digits, and the underscore. -x, —line-regexp Select only those matches that exactly match the whole line. -y Obsolete synonym for -i. -Z, —null Output a zero byte (the ASCII NULL character) instead of the character that normally follows a file name. For example, grep -lZ outputs a zero byte after each file name instead of the usual newline. This option makes the output unambiguous, even in the presence of file names containing unusual characters like newlines. This option can be used with commands like find -print0, perl -0, sort -z, and xargs -0 to process arbitrary file names, even those that contain newline characters.

Regular expressions

A regular expression is a pattern that describes a set of strings. Regular expressions are constructed analogously to arithmetic expressions, using various operators to combine smaller expressions.

Grep understands two different versions of regular expression syntax: «basic» and «extended.» In GNU grep, there is no difference in available functionality using either syntax. In other implementations, basic regular expressions are less powerful. The following description applies to extended regular expressions, which are used in egrep; differences for basic regular expressions are summarized afterwards.

The fundamental building blocks are the regular expressions that match a single character. Most characters, including all letters and digits, are regular expressions that match themselves. Any metacharacter with special meaning may be quoted by preceding it with a backslash.

A bracket expression is a list of characters enclosed by [ and ]. It matches any single character in that list; if the first character of the list is the caret ^ then it matches any character not in the list. For example, the regular expression [0123456789] matches any single digit.

Within a bracket expression, a range expression consists of two characters separated by a hyphen («-«). It matches any single character that sorts between the two characters, inclusive, using the locale’s collating sequence and character set. For example, in the default C locale, [a-d] is equivalent to [abcd]. Many locales sort characters in dictionary order, and in these locales [a-d] is often not equivalent to [abcd]; it might be equivalent to [aBbCcDd], for example. To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the LC_ALL environment variable to the value C.

Finally, certain named classes of characters are predefined within bracket expressions, as follows. Their names are self explanatory, and they are [:alnum:], [:alpha:], [:cntrl:], [:digit:], [:graph:], [:lower:], [:print:], [:punct:], [:space:], [:upper:], and [:xdigit:]. For example, [[:alnum:]] means [0-9A-Za-z], except the latter form depends upon the C locale and the ASCII character encoding, whereas the former is independent of locale and character set. (Note that the brackets in these class names are part of the symbolic names, and must be included in addition to the brackets delimiting the bracket list.) Most metacharacters lose their special meaning inside lists. To include a literal ] place it first in the list. Similarly, to include a literal ^ place it anywhere but first. Finally, to include a literal — place it last.

The period . matches any single character. The symbol \w is a synonym for [[:alnum:]] and \W is a synonym for [^[:alnum]].

The caret ^ and the dollar sign («$«) are metacharacters that respectively match the empty string at the beginning and end of a line. The symbols \ respectively match the empty string at the beginning and end of a word. The symbol \b matches the empty string at the edge of a word, and \B matches the empty string provided it’s not at the edge of a word.

A regular expression may be followed by one of several repetition operators:

?	The preceding item is optional and matched at most once.
*	The preceding item will be matched zero or more times.
+	The preceding item will be matched one or more times.
<n>	The preceding item is matched exactly n times.
<n,>	The preceding item is matched n or more times.
<n,m>	The preceding item is matched at least n times, but not more than m times.

Two regular expressions may be concatenated; the resulting regular expression matches any string formed by concatenating two substrings that respectively match the concatenated subexpressions.

Two regular expressions may be joined by the infix operator |; the resulting regular expression matches any string matching either subexpression.

Repetition takes precedence over concatenation, which in turn takes precedence over alternation. A whole subexpression may be enclosed in parentheses to override these precedence rules.

The backreference \n, where n is a single digit, matches the substring previously matched by the nth parenthesized subexpression of the regular expression.

In basic regular expressions the metacharacters ?, +, <, |, (, and ) lose their special meaning; instead use the backslashed versions \?, \+, \<, \|, $, and $.

Traditional egrep did not support the <metacharacter, and some egrep implementations support \ <instead, so portable scripts should avoid <in egrep patterns and should use [ <]to match a literal <.

GNU egrep attempts to support traditional usage by assuming that <is not special if it would be the start of an invalid interval specification. For example, the shell command egrep ‘ <1'searches for the two-character string <1instead of reporting a syntax error in the regular expression. POSIX.2 allows this behavior as an extension, but portable scripts should avoid it.

Environment variables

Grep‘s behavior is affected by the following environment variables:

A locale LC_foo is specified by examining the three environment variables LC_ALL, LC_foo, LANG, in that order. The first of these variables that is set specifies the locale. For example, if LC_ALL is not set, but LC_MESSAGES is set to pt_BR, then Brazilian Portuguese is used for the LC_MESSAGES locale. The C locale is used if none of these environment variables are set, or if the locale catalog is not installed, or if grep was not compiled with national language support (NLS).

GREP_OPTIONS

This variable specifies default options to be placed in front of any explicit options. For example, if GREP_OPTIONS is ‘—binary-files=without-match —directories=skip‘, grep behaves as if the two options —binary-files=without-match and —directories=skip had been specified before any explicit options. Option specifications are separated by whitespace. A backslash escapes the next character, so it can be used to specify an option containing whitespace or a backslash.

GREP_COLOR

Specifies the marker for highlighting.

LC_ALL, LC_COLLATE, LANG

These variables specify the LC_COLLATE locale, which determines the collating sequence used to interpret range expressions like [a-z].

LC_ALL, LC_CTYPE, LANG

These variables specify the LC_CTYPE locale, which determines the type of characters, e.g., which characters are whitespace.

LC_ALL, LC_MESSAGES, LANG

These variables specify the LC_MESSAGES locale, which determines the language that grep uses for messages. The default C locale uses American English messages.

POSIXLY_CORRECT

If set, grep behaves as POSIX.2 requires; otherwise, grep behaves more like other GNU programs. POSIX.2 requires that options that follow file names must be treated as file names; by default, such options are permuted to the front of the operand list and are treated as options. Also, POSIX.2 requires that unrecognized options be diagnosed as «illegal», but since they are not really against the law the default is to diagnose them as «invalid». POSIXLY_CORRECT also disables _N_GNU_nonoption_argv_flags_, described below.

_N_GNU_nonoption_argv_flags_

(Here N is grep’s numeric process ID.) If the ith character of this environment variable’s value is 1, do not consider the ith operand of grep to be an option, even if it appears to be one. A shell can put this variable in the environment for each command it runs, specifying which operands are the results of file name wildcard expansion and therefore should not be treated as options. This behavior is available only with the GNU C library, and only when POSIXLY_CORRECT is not set.

Examples

Search for patterns of support help and windows in the file myfile.txt.

Match any lines in myfile.txt which begin a line with an alphabetic word that also ends the line.

Count the number of lines in myfile.txt which begin with the word ‘begin‘ or end with the word ‘end‘.

fgrep — Filter text which matches a fixed-character string.
grep — Filter text which matches a regular expression.
sed — A utility for filtering and transforming text.
sh — The Bourne shell command interpreter.

Источник

Linux binary file standard input matches

grep возвращает «Бинарный файл (стандартный ввод) соответствует« при попытке найти строковый шаблон в файле

3 ответа