Linux unzip from pipe

How to redirect output of wget as input to unzip?

I have to download a file from this link. The file download is a zip file which I will have to unzip in the current folder.

Normally, I would download it first, then run the unzip command.

But in this way, I need to execute two commands, wait for the completion of first one to execute the next one, also, I must know the name of the file temp.zip to give it to unzip .

Is it possible to redirect output of wget to unzip ? Something like

But it didn’t work.

bash: wget http://www.vim.org/scripts/download_script.php?src_id=11834 -O temp.zip : ambiguous redirect

Also, wget got executed twice, and downloaded the file twice.

9 Answers 9

You have to download your files to a temp file, because (quoting the unzip man page):

Archives read from standard input are not yet supported, except with funzip (and then only the first member of the archive can be extracted).

Just bring the commands together:

But in order to make it more flexible you should probably put it into a script so you save some typing and in order to make sure you don’t accidentally overwrite something you could use the mktemp command to create a safe filename for your temp file:

This is a repost of my answer to a similar question:

The ZIP file format includes a directory (index) at the end of the archive. This directory says where, within the archive each file is located and thus allows for quick, random access, without reading the entire archive.

This would appear to pose a problem when attempting to read a ZIP archive through a pipe, in that the index is not accessed until the very end and so individual members cannot be correctly extracted until after the file has been entirely read and is no longer available. As such it appears unsurprising that most ZIP decompressors simply fail when the archive is supplied through a pipe.

The directory at the end of the archive is not the only location where file meta information is stored in the archive. In addition, individual entries also include this information in a local file header, for redundancy purposes.

Источник

Unzipping files that are flying in through a pipe

Can I make unzip or any similar programs work on the standard output? The situation is I’m downloading a zip file, which is supposed to be unzipped on the fly.

12 Answers 12

While a zip file is in fact a container format, there’s no reason why it can’t be read from a pipe (stdin) if the file can fit into memory easily enough. Here’s a Python script that takes a zip file as standard input and extracts the contents to the current directory or to a specified directory if specified.

This script can be minified to one line and created as an alias.

Now unzip the output of wget easily.

This is unlikely to work how you expect. Zip is not just a compression format, but also a container format. It rolls up the jobs of both tar and gzip.bzip2 into one. Having said that, if your zip has a single file, you can use unzip -p to extract the files to stdout. If you have more than one file, there’s no way for you to tell where they start and stop.

As for reading from stdin, the unzip man page has this sentence:

Archives read from standard input are not yet supported, except with funzip (and then only the first member of the archive can be extracted).

You might have some luck with funzip.

I like to use curl because it is installed by default (the -L is needed for redirects which often occur):

However, bsdtar is not installed by default, and I could not get funzip to work.

This is a repost of my answer to a similar question:

The ZIP file format includes a directory (index) at the end of the archive. This directory says where, within the archive each file is located and thus allows for quick, random access, without reading the entire archive.

This would appear to pose a problem when attempting to read a ZIP archive through a pipe, in that the index is not accessed until the very end and so individual members cannot be correctly extracted until after the file has been entirely read and is no longer available. As such it appears unsurprising that most ZIP decompressors simply fail when the archive is supplied through a pipe.

The directory at the end of the archive is not the only location where file meta information is stored in the archive. In addition, individual entries also include this information in a local file header, for redundancy purposes.

Although not every ZIP decompressor will use local file headers when the index is unavailable, the tar and cpio front ends to libarchive (a.k.a. bsdtar and bsdcpio) can and will do so when reading through a pipe, meaning that the following is possible:

What you want to do is, make unzip take a ZIPped file on its standard input rather than as an argument. This is usually easily supported by gzip and tar kind of tools with a — argument. But the standard unzip does not do that (though, it does support extraction to a pipe). However, all is not lost.

Look at funzip manual page.

funzip without a file argument acts as a filter; that is, it assumes that a ZIP archive (or a gzip’d file) is being piped into standard input, and it extracts the first member from the archive to stdout. When stdin comes from a tty device, funzip assumes that this cannot be a stream of (binary) compressed data and shows a short help text, instead. If there is a file argument, then input is read from the specified file instead of from stdin.

Given the limitation on single-member extraction, funzip is most useful in conjunction with a secondary archiver program such as tar(1). The following section includes an example illustrating this usage in the case of disk backups to tape.

This goes well with the idea that most linux archives are usually TAR’ed and then ZIPped in some way (gzip, bzip, et al). This will work for you if you have a tar.ZIP .

It is worth noting that funzip is written by Info-ZIP original author Mark Adler. He writes in the funzip man page,

however, no such update is seen around. I suspect that Mark found it unnecessary since other archiving methods worked easily with TAR.

Источник

How do you specify filenames within a zip when creating it on the command line from a pipe?

I’m trying to create a zip file from file contents which are being piped in, e.g.

This writes the zip correctly, but when you open the zip, the file within it is called «-«. Is there any way of specifying what the filename of the piped in data should be within the zip?

6 Answers 6

You can do this.

this is done from the notion that i have 3 files in the dir.

and the file itself, the result is then

From the Linux Zip Man page

If the file list is specified as -@, zip takes the list of input files from standard input.

You can use a named pipe, and send the request output to it, while zipping from it.

I couldn’t manage with the PHP answer (out of memory on bigger mysql dumps), and the FIFO was not working as I wanted, so my solution is to rename the file inside the ZIP archive after running the dump, using zipnote (which is included with the zip package on Debian).

From what i can gather you cannot do both with the zip command, i mean you cannot both specify the filenames and pipe the content. You can either pipe the contents and the resulting file is — or you can pipe the filenames with -@ .

That does not mean that doing so is impossible using other techniques. I outline one of those below. It does mean that you have to have PHP installed and the zip extension loaded.

There could be a whole bunch of other ways to do it. But this is the easiest that I know of. Oh and it’s a one-liner too.

This is a working example using PHP

To run on windows just swap single and double quotes. Or just place the script in a file.

Источник

How to unzip a zip file from the Terminal?

Just downloaded a .zip file from the internet. I want to use the terminal to unzip the file. What is the correct way to do this?

9 Answers 9

If the unzip command isn’t already installed on your system, then run:

After installing the unzip utility, if you want to extract to a particular destination folder, you can use:

If the source and destination directories are the same, you can simply do:

You can simply use unzip .

A more useful tool is 7z , which zips and unzips a range of compression formats, notably lzma , usually the protocol offering the highest compression rates.

This command installs 7z :

This command lists the contents of the zip:

This command extracts the contents of the zip:

to extract to yourpath/somedir

If you want to extract to an absolute path, use

Using scripting tools: Perl and Python

Many answers here mention tools that require installation, but nobody has mentioned that two of Ubuntu’s scripting languages, Perl and Python, already come with all the necessary modules that allow you to unzip a zip archive, which means you don’t need to install anything else. Just use either of the two scripts presented below to do the job. They’re fairly short and can even be condensed to a one-liner command if we wanted to.

Python

If the source and destination directories are the same, you can simply do:

I prefer bsdtar to unzip / zip . For extracting, they are pretty similar:

However for zipping, bsdtar wins. Say you have this input:

and want this in the zip file:

This is easy with bsdtar :

zip does not have the -d option like unzip, so you have no way to achieve the above unless you cd first.

Here is the detailed description of options that I find useful:

Install unzip

So First of all we need to install unzip on our system if it’s not installed. unzip command is used to extract files from a ZIP archive.

Run the following command to install unzip

Now Follow the steps below:

UnZip File

OPTION 1 – If the Zip File is in the same directory/folder in which your terminal is and we want to extract it in the present working directory.

Use the following command to achieve the above described scenario

if the zip file is protected with some password, then use the following command :

Please make sure you use -P (capital P) not -p because the are different options.

OPTION 2 – If the zip file is not present in the same directory and we want to extract/unzip the file in different directory.

Use the following command to achieve the above described scenario

if we does not use option -d the file will be extracted to present working directory.

And if the zip file is password protected we can also use -P .

use tar Command in Linux / Unix

tar is an acronym for Tape Archive. tar command is used to Manipulates archives in Linux/Unix. System administrators uses tar command frequently to rip a bunch of files or directories into highly compressed archive which are called tarball or tar , bzip and gzip in Linux/Unix system.

tar Syntax

tar required Flags

tar optional Flags

Examples

Create tar Archive File by Compressing an Directory or a Single File

The terminal command below will create a .tar file called sample_dir.tar with a directory /home/codebind/sample_dir or sample_dir in present working directory.

Here’s what those flags (-cvf) actually mean

-c, —create – create a new archive

-x, —extract, —get – extract files from an archive

-f, —file ARCHIVE – use archive file or device ARCHIVE

Create tar.gz or tgz Archive File by Compressing an Directory or a Single File

The terminal command below will create a .tar.gz file called sample_dir.tar.gz with a directory /home/codebind/sample_dir or sample_dir in present working directory.

Notice that we have added extra flag -z to the command.Here’s what the flag -z actually mean

-z, —gzip, —gunzip —ungzip – Compress the archive with gzip

The command bellow will create a .tgz file. One this to notice is tar.gz and tgz both are similar.

Compressing Multiple Directories or Files at Once

Let’s say, For example we want to compress the sample_dir directory, the java_test directory, and the abc.py file to a tar file called sample_dir.tar.gz .

Run the following command to achieve the goal above.

Create .bzip2 Archive File by Compressing an Directory or a Single File

Notice that we have added extra flag -f to the command.Here’s what the flag -f actually mean

-f, —file ARCHIVE – use archive file or device ARCHIVE

Extract .tar Archive File

We can extract or untar the compressed file using the tar command. The command below will extract the contents of sample_dir.tar to the present directory.

The following command will extract or untar files in specified Directory i.e. /home/codebind/dir_name in this case.

we have added extra flag -C to the command.Here’s what the flag -C actually mean

-C, —directory DIR – change to directory DIR

Источник

Примеры использования zip и unzip в linux

Сегодня я хотел сделать пост памятку по работе в Unix подобных системах с утилитами zip и unzip. Также рассмотреть несколько интересных команд, которые могут пригодиться для работы.

Установка zip и unzip:

В системе Debian/Ubuntu.

В системе Red Hat Linux/Fedora/CentOS.

Пример использования утилиты zip

Пример 1: Создания архива из указанных файлов и папок

Пример 2: Создания архива текущего каталога (архив создается без подкаталогов)

Пример 3: Создания архива включая все подкаталоги используем ключ -r

Создает архив без скрытых файлов/каталог это файлы/каталоги начинающиеся с .

Создает архив с скрытыми файлами и каталогами

Пример 4: Чтобы сжать быстро используйте -1, но для более лучшего сжатие используйте -9

Пример 5: Опция -x позволяет исключать файлы и каталоги из архива

Пример 6: Также опция -x позволяет исключать файлы и каталоги по шаблону

Таким способом я исключаю все каталоги заканчивающихся на ted и файлы расширения xml

Пример 7: Разбиваем архив на части с помощью опции -s

Также есть возможность указать опцию размера части архива в: k (KB), m (MB), g (GB), t (TB)

Пример 8: Задать пароль можно опцией -e или опцией -P .

Пример 9: Выбрать только указанные файлы/каталоги

Пример 10: Удаление файлов из созданного архива

Пример 11: Обновить содержимое созданного архива

Пример 11: Тихий режим (Полезно, например, в сценариях оболочки и фоновых задач)

Пример 12: Извлечь все содержимое архива

Пример 13: Извлечь все содержимое архива в указанную директорию

Пример 14: Показать содержимое архива

Пример 15: Извлечь конкретный файл из архива

Источник