Curl linux basic auth

Curl POST With Basic Authentication

Compare Request Timings

What is Curl?

Curl (stands for Client URL) is popular command-line tool developers use to send requests to the server, upload files, and submit web forms. Curl supports all modern protocols, including HTTP, HTTPS, SFTP, FTP, and has built-in support for SSL, user authentication, and HTTP Cookies. Curl works on Linux, Mac, Windows.

What is HTTP POST?

POST is one of the nine standard methods of the HTTP protocol. The HTTP POST method requests the webserver to receive and process the data contained in the body of the POST message. POST method is often used to submit login or contact forms and upload files and images to the server.

What is Basic Authentication?

Basic Authentication is a client authentication method built into the HTTP protocol that allows a client to provide a username and password to the server when accessing secure resources over HTTP. When requesting a protected resource, the client sends HTTP requests with an Authorization header that contains the word Basic followed by a space and a base64 encoded username: password string. Basic Authentication is not the most secure method because other protocol sniffers can easily decrypt base64 encoded user credentials. For security reasons, the Basic Authentication method should only be used over secure HTTPS/SSL connections.

How do I post Basic Authentication data using Curl?

To post a Curl request with Basic Authorization credentials, you can use the -u (or —user) command line parameter: —user username: password.

Curl automatically converts the provided login: password pair into a Base64-encoded string and adds an appropriate HTTP header to the request:

Example of Basic user Authentication using Curl POST request

The general form of a Curl command for making a POST request with Basic Authentication is as follows:

See also

Generate Code Snippets for Curl POST Basic Authentication Example

Convert your Curl POST Basic Authentication request to the PHP, JavaScript/AJAX, Curl/Bash, Python, Java, C#/.NET code snippets using the ReqBin code generator.

Источник

How do I send Basic Auth Credentials with Curl?

Compare Request Timings

What is Curl?

Curl is an open-source command-line tool for transferring data to or from the server using URL syntax. Curl supports over 25+ protocols, including HTTP, HTTPS, SFTP, FTP, and has built-in support for proxy, HTTP Cookies, SSL certificates. Curl is ideal for testing test APIs and works on platforms Linux, Windows, and macOS.

What is Basic Authentication?

Basic Authentication is a method for an HTTP client (such as a web browser) to provide a username and password to the server when making an HTTP request to protected resources. For Basic Authentication, the client sends an HTTP request header field in the form Authorization: Basic base64String, where the base64String is a Base64 encoded username and password concatenated with a single colon. Basic authentication is not the most secure authentication method because the username and password can be easily decrypted from the base64 string. For security reasons, Basic Authentication should only be used over HTTPS/SSL connections.

Can I send Basic Auth Credentials using Curl?

Yes, Curl has built-in support for basic HTTP server authorization. To make a Curl request with basic authorization credentials, you need to use the following command line parameter: -u username: password (or —user). Curl automatically converts the provided login: password pair into a Base64-encoded string and adds an appropriate HTTP header «Authorization: Basic bG9naW46cGFzc3dvcmQ =» to the request. If you omit part of the password in the command line parameter, Curl will prompt the user for a password.

Alternative Curl Basic Authentication Method

Alternatively, you can pass the basic auth credentials using the Curl -H «Authorization: Basic [token]» command-line option. The -H parameter passes the authorization header to Curl like any other custom header, without any processing.

See also

Generate Code Snippets for Curl Basic Auth Example

Convert your Curl Basic Auth request to the PHP, JavaScript/AJAX, Curl/Bash, Python, Java, C#/.NET code snippets using the ReqBin code generator.

Источник

Как установить заголовок авторизации с помощью curl

Как передать заголовок авторизации с помощью cURL? (исполняемый файл в /usr/bin/curl ).

8 ответов

см. раздел 6. Аутентификация HTTP

аутентификация HTTP-это возможность сообщить серверу свое имя пользователя и пароль, чтобы он мог подтвердить, что вам разрешено выполнять запрос, который вы делающий. Базовая аутентификация, используемая в HTTP (который является типом, используемым curl по умолчанию)простые текст, что означает, что он отправляет имя пользователя и пароль только слегка запутанный, но все еще полностью читаемый любым, кто нюхает сеть между вами и удаленным сервером.

чтобы сказать curl использовать пользователя и пароль для аутентификации:

сайту может потребоваться другой метод аутентификации (проверьте заголовки возвращается сервером), а затем —ntlm, —digest, —negotiate или даже — anyauth может быть варианты, которые подходят вам.

Иногда ваш HTTP-доступ доступен только через использование HTTP полномочие. Это, по-видимому, особенно распространено в различных компаниях. Прокси HTTP может потребоваться собственный пользователь и пароль, чтобы позволить клиенту интернет. Чтобы указать те, с завитком, запустите что-то вроде:

Если прокси-сервер требует проверки подлинности с помощью метода NTLM, используйте —proxy-ntlm, если это требует использования дайджеста —proxy-digest.

Если вы используете любой из этих параметров user + password, но оставляете пароль часть, curl запросит пароль в интерактивном режиме.

обратите внимание, что при запуске программы ее параметры могут быть видны при перечислении запущенных процессов системы. Таким образом, другие пользователи могут быть возможность просмотра паролей, Если вы передаете их как обычную командную строку опции. Есть способы обойти это.

стоит отметить, что пока это так HTTP Аутентификация работает, очень многие веб-сайты не будут использовать эту концепцию, когда они дают логины и т. д. Видеть более подробная информация об этом приведена ниже в разделе веб-входа.

Источник

How to use Basic authentication with curl?

In Basic Authentication, a HTTP request contains a header Authorization: Basic , where credentials is the Base64 encoding of username and password joined by a single colon : .

Basic Auth is considered as not safe enough, but we still use it a lot for some less sensitive stuff because it is easy to set up.

Basic Auth with curl

You can use Basic Http Auth with curl in following two ways:

Option 1: Pass credentials to curl

Passing Basic credentials to curl command is easy as this:

curl -u username:password https://example.com

If your username or password contains a special character, such as white-space, then you might want to surround credentials with single quotes:

curl -u ‘username:password’ https://example.com

Option 2: Pass Authorization header

If you want to have a full control over your HTTP request, you might want to Base64 encode your username:password and place it into Authorization header.
Curl command should look like this:

curl -H ‘Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=’ https://example.com

Basic Auth and colon in username

Colon : is not allowed to be used in an username according to Basic Auth specification. But in case you need to support it (ie your users are allowed to create such username) and you are in charge of client and server, you might want to URL encode the username and password on both sides.

Источник

Curl in Bash Scripts by Example

Great! Now what? Before you kill anything in bash it is dire to know where to get help if in trouble. Here is what the man page for curl or curl help command looks like. Copy and paste. Try not to be overwhelmed by appearances. There are a lot of options that you only need later in life. More importantly, it serves as a quick reference to lookup options as you need.

Here are some commands to get help within your terminal and other browser-friendly resources.

Help commands for curl in bash

Consult these resources anytime you need. In addition to this piece, they will serve as companions on your journey towards killing it with curl in bash scripts.

Now that getting help and listing command line options is out of the picture, let’s move on to the three ways.

The three ways to curl in bash by example

You may argue that there are more than three ways to curl in bash. However, for simplicity purposes, let’s just say that there are. Also note that in practice, usage of each way is not mutually exclusive. In fact, you will find that ways may be intertwined depending on the intent of your bash script. Let’s begin.

The first way: Downloading files

All options aside curl downloads files by default. In bash, we curl to download a file as follows.

This sends the content of the file we are downloading to standard output; that is, the your screen. If the file is a video or an image don’t be surprised if you hear a few beeps. We need to save to a file. Here’s how it looks.

curl $ > outfile
# download file saving as outfile

curl $ -o outfile
# download file save as option

curl $ -O
# download file inherit filename

## expect file saved as $( basename $ )

Note that the download file save as option inheriting file name is particularly useful when using URL globbing, which is covered in the bash curl loop section.

Now let’s move on to how to check headers prior to downloading a file with curl in bash.

The second way: Checking headers

There will come a time when you wish to get information about a file before downloading. To do this, we add the -I option to the curl command as follows.

Note that there are other ways to dump headers from curl requests, which is left for homework.

Here is a quick example to show how the second way works in bash scripts that can be used to serve as a part of a web page health checker.

Example) bash curl get response code

Often, we want to get the response code for a curl request in bash. To do this, we would need to first request the headers of a response and then extract the response code. Here is what it would look like.

url =https: // temptemp3.github.io
# just some url

curl $ -I -o headers -s
# download file

cat headers
# response headers
## expect
#HTTP/2 200
#server: GitHub.com
#content-type: text/html; charset=utf-8
#strict-transport-security: max-age=31557600
#last-modified: Thu, 03 May 2018 02:30:03 GMT
#etag: «5aea742b-e12»
#access-control-allow-origin: *
#expires: Fri, 25 Jan 2019 23:07:17 GMT
#cache-control: max-age=600
#x-github-request-id: 8808:5B91:2A4802:2F2ADE:5C4B944C
#accept-ranges: bytes
#date: Fri, 25 Jan 2019 23:12:37 GMT
#via: 1.1 varnish
#age: 198
#x-served-by: cache-nrt6148-NRT
#x-cache: HIT
#x-cache-hits: 1
#x-timer: S1548457958.868588,VS0,VE0
#vary: Accept-Encoding
#x-fastly-request-id: b78ff4a19fdf621917cb6160b422d6a7155693a9
#content-length: 3602

cat headers | head -n 1 | cut ‘-d ‘ ‘-f2’
# get response code
## expect
#200

My site is up. Great!

Now let’s move on to making posts with curl in bash scripts.

The third way: Making posts

There will come a time when you need to make posts with curl in bash to authenticate to access or modification of private content. Such is the case working with APIs and html forms. It may require multiple curl requests. The placeholder curl command line for this way is as follows.

Making posts involves adding corresponding headers and data to allow for authentication. I’ve prepared some examples of making posts with curl in bash.

Example) Basic authentication

Here is an example of using curl in bash scripts to download a file requiring basic authentication. Note that credentials are stored in a separate file called bash-curl-basic-auth-example-config.sh, which is also included below.

## curl-basic-auth-config
## version 0.0.1 — initial

username = «username»
password = «passwd»

## generated by create-stub2.sh v0.1.1
## on Sun, 27 Jan 2019 14:08:17 +0900
## see

Here’s what it looks like in the command line.

Here you see how writing a bash script allows you to avoid having to include your secrets in the command line.

Note that the –location option was added to handle requests that are redirected.

Now that we have basic authentication is out of the picture, let’s step up the difficuly a bit.

Example) Submitting html form with csrf protection

The magic of bash is that you can do just about anything you have an intent to do. Jumping through the hoops of csrf protection is one way to kill it with curl in bash scripts.

In modern web applications there is a security feature called csrf protection to prevent posts requests from anywhere without established access to the site in question.

Basically, there is a security token included in the response of a page.

Here what your bash script may look like to gain authorized access to a page content with csrf protection.

Notes on script
It uses a alias called commands that I mentioned in a previous post about the bash declare command, which makes it possible to declare subcommands implicitly by way of convention.

Here you see that bash can be used to string curl request together with logic to carry out the intent of your script.
So that some of the bash usage above using subshells to limit function redeclaration scope doesn’t appear so magical, I’ve prepared a follow-up example.

Here is the correspondence command line example.

Example) Wonderlist API call

Here is curl request command line in a bash script that I wrote in late 2017 back before switching over to Trello.

Notes on script

$ contains an -X option that can be passed in by caller functions. If you are not familiar with the option, it is set the request command to use. That is, GET, POST, HEAD, etc. according to to api documentation.

It contains multiple -H options for authenication.

The –silent option is used because in some cases showing progress in the terminal would be overkill for background requests.

Surely, you are now killing it with curl in bash scripts. Next, we move on to special topics to bring it all together.

Looping through urls with curl in bash

Suppose that we have a list of URLs which we would like to loop over and curl. That is, we want download using curl for each URL in our list. Here is how we would go about accomplishing this task on the command line.

curl ( ) < echo "dummy response for $<@>» ; > # fake curl for testing purposes

urls ( ) < cat / dev / clipboard ; ># returns list of urls

for url in $ ( urls ) ; do curl $ ; done # curl loop

## expect
#dummy response for whatever is in your
#dummy response for clipboard
#dummy response for .

If you don’t have a list of urls to copy on hand, here is a list of 100 URLs most likely respond to HTTP request using curl.

gist of Craft Popular URLs based on list of the most popular websites worldwide

Often, we do not only wish to curl a list of urls in bash. We may want to generate urls to curl as we progress through the loop. To accomplish this task, we need to introduce variables into the URL as follows.

curl ( ) < echo "dummy response for $<@>» ; > # fake curl for testing purposes
url ( ) < echo $/ $ ; > # url template
urls ( ) < # generate all urls
local i
for i in $
do
url
done
>

url_base = «https://temptemp3.github.io» # just some base
range =$ ( echo < 1 .. 9 >) # just some range
for url in $ ( urls ) ; do curl $ ; done # curl loop

## expect
#dummy response for https://temptemp3.github.io/1
#dummy response for https://temptemp3.github.io/2
#dummy response for https://temptemp3.github.io/3
#dummy response for https://temptemp3.github.io/4
#dummy response for https://temptemp3.github.io/5
#dummy response for https://temptemp3.github.io/6
#dummy response for https://temptemp3.github.io/7
#dummy response for https://temptemp3.github.io/8
#dummy response for https://temptemp3.github.io/9

It turns out that loops may be avoided in some cases by taking advantage of a curl feature only available in command line called URL globbing. Here’s how it works.

unset -f curl
# included just in case
curl https: // temptemp3.github.io / [ 0 — 9 ]
# curl loop using URL globbing

## expect
#response for https://temptemp3.github.io/1
#response for https://temptemp3.github.io/2
#response for https://temptemp3.github.io/3
#response for https://temptemp3.github.io/4
#response for https://temptemp3.github.io/5
#response for https://temptemp3.github.io/6
#response for https://temptemp3.github.io/7
#response for https://temptemp3.github.io/8
#response for https://temptemp3.github.io/9

Here we see that any of the methods above may be used to implement a curl loop in bash Depending on the use case and desired level of control, a method may be preferred over another.

Handling curl errors in bash

One thing that is absent from curl is the ability to handle errors. That is where bash comes in handly.

Curl has an–retry NUM option that as you may have guess tells curl to retry a specific number of times. However, what if we want to have curl effectively retry indefinitely until succeeding?

Source: curl-retry.sh
Here is what we see in command line.

The hope is that eventually someone will create temptemp3.io and our script will exit with an exit status of zero.

Last but not least I would like to end with an example of how to set up concurrent curls in bash to act as a download accelerator.

Downldr.sh

Sometimes it is helpful to download large files in parts. Here is a snippet from a bash script that I wrote recently using curl.

Notes on script

The -r option is used to specifiy the range in bytes to download if the host accepts ranges.

Conclusion

By this time you are killing it with curl in bash scripts. In many cases you may take advantage of curl functionality through the horde of options it provides. However, you may opt out and achieve the same functionality outside of curl in bash for the level of control that fits your needs.

About the author

Nicholas Shellabarger

A developer and advocate of shell scripting and vim. His works include automation tools, static site generators, and web crawlers written in bash. For work he tools with cloud computing, app development, and chatbots. He codes in bash, python, or php, but is open to offers.

Источник