Home | 简体中文 | 繁体中文 | 杂文 | Github | 知乎专栏 | 51CTO学院 | CSDN程序员研修院 | OSChina 博客 | 腾讯云社区 | 阿里云栖社区 | Facebook | Linkedin | Youtube | 打赏(Donations) | About
知乎专栏多维度架构

4.17. CURL - transfer a URL

4.17.1. 基本用法

		
curl http://www.google.com/		
		
		

4.17.2. 提交表单数据

post 表单数据

		
curl -d "user=neo&password=chen" http://www.example.com/login.php
curl --data "user=neo&password=chen" http://www.example.com/login.php
		
		

4.17.3. 上传文件

		
curl  -F "upload=@card.txt;type=text/plain"  "http://www.example.com/upload.php"		
		
		

使用 CURL 上传 Oauth2 + Jwt 认证的 Restful 接口

		
curl -s  -H "Authorization: Bearer ${TOKEN}" -X POST -F "file=@/etc/hosts" http://localhost:8080/upload/single		
		
		

4.17.4. connect-timeout

		
curl -o /dev/null --connect-timeout 30 -m 30 -s -w %{http_code} http://www.google.com/		
		
		

4.17.5. max-time

-m, --max-time SECONDS Maximum time allowed for the transfer

curl -o /dev/null --max-time 10 http://www.netkiller.cn/
		

4.17.6. compressed

--compressed Request compressed response (using deflate or gzip)
curl --compressed http://www.example.com
		

4.17.7. 代理服务器

vhosts 测试

有时候你需要设觉察/etc/hosts文件才能访问vhost,下面例子可以不设置/etc/hosts

				
curl -x 127.0.0.1:80 your.exmaple.com/index.php
		
		

socks5 服务器

		
$ curl -v -x socks5://username:password@IP:1080 http://www.google.com/		
		
		

4.17.8. -w, --write-out <format> 输出格式定义

计时器 描述
time_connect 建立到服务器的 TCP 连接所用的时间
time_starttransfer 在发出请求之后,Web 服务器返回数据的第一个字节所用的时间
time_total 完成请求所用的时间
time_namelookup DNS解析时间,从请求开始到DNS解析完毕所用时间(记得关掉 Linux 的 nscd 的服务测试)
speed_download 下载速度,单位-字节每秒。
		
curl -o /dev/null -s -w %{time_connect}:%{time_starttransfer}:%{time_total} http://www.example.net
curl -o /dev/null -s -w "Connect: %{time_connect}\nTransfer: %{time_starttransfer}\nTotal: %{time_total}\n" https://www.netkiller.cn/index.html

curl -o /dev/null -s -w "Connect: %{time_connect} \nTransfer: %{time_starttransfer}\nTotal: %{time_total}\nNamelookup: %{time_namelookup}\nDownload: %{speed_download}\n" https://www.netkiller.cn/index.html
Connect: 0.024241
Transfer: 0.117727
Total: 0.117842
Namelookup: 0.004367
Download: 129877.000
		

测试页面所花费的时间

date ; curl -s -w 'Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} \n' -H "Host: www.example.com" http://172.16.0.1/webapp/test.jsp ; date ;
		
curl -o /dev/null -s -w %{time_connect}, %{time_starttransfer}, %{time_total}, %{time_namelookup}, %{speed_download} http://www.netkiller.cn
		

返回HTTP状态码

curl -s -I http://netkiller.sourceforge.net/ | grep HTTP | awk '{print $2" "$3}'
curl -o /dev/null -s -w %{http_code} http://netkiller.sourceforge.net/

curl --connect-timeout 5 --max-time 60 --output /dev/null -s -w %{response_code} http://www.netkiller.cn/
		
# curl -w '\nLookup time:\t%{time_namelookup}\nConnect time:\t%{time_connect}\nPreXfer time:\t%{time_pretransfer}\nStartXfer time:\t%{time_starttransfer}\n\nTotal time:\t%{time_total}\n' -o /dev/null -s http://www.netkiller.cn

Lookup time: 0.125
Connect time: 0.125
PreXfer time: 0.125
StartXfer time: 0.125

Total time: 0.126
		

4.17.9. -A/--user-agent <agent string>

设置用户代理,这样web服务器会认为是其他浏览器访问

curl -A "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" http://www.example.com
		

4.17.10. referer

		
curl -v -o /dev/null -e "http://www.example.com" http://www.your.com/
* About to connect() to www.your.com port 80
*   Trying 172.16.1.10... connected
* Connected to www.your.com (172.16.1.10) port 80
> GET / HTTP/1.1
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: www.your.com
> Accept: */*
> Referer: http://www.example.com
>
< HTTP/1.1 200 OK
< Date: Thu, 30 Sep 2010 07:59:47 GMT
< Server: Apache/2.2.16 (Unix) mod_ssl/2.2.16 OpenSSL/0.9.8e-fips-rhel5 PHP/5.2.14
< Accept-Ranges: bytes
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  172k    0  172k    0     0  10.2M      0 --:--:-- --:--:-- --:--:-- 11.9M* Connection #0 to host www.your.com left intact

* Closing connection #0
		
		

4.17.11. -v

		

		
		

4.17.12. -o, --output FILE Write output to <file> instead of stdout

curl -o /dev/null http://www.example.com
curl -o index.html http://www.example.com
		

4.17.13. -L, --location

		
curl -L --retry 5 --retry-delay 3 https://github.com/hyperledger/fabric/releases/download/v2.0.1/hyperledger-fabric-linux-amd64-2.0.1.tar.gz | tar xz		
		
		

4.17.14. -H/--header <line> Custom header to pass to server (H)

Last-Modified / If-Modified-Since

If-Modified-Since

			
neo@neo-OptiPlex-780:/tmp$ curl -I http://images.example.com/test/test.html
HTTP/1.0 200 OK
Cache-Control: s-maxage=7200, max-age=900
Expires: Mon, 16 May 2011 08:10:37 GMT
Content-Type: text/html
Accept-Ranges: bytes
ETag: "1205579110"
Last-Modified: Mon, 16 May 2011 06:57:39 GMT
Content-Length: 11
Date: Mon, 16 May 2011 07:55:37 GMT
Server: lighttpd/1.4.26
Age: 604
X-Via: 1.0 ls71:80 (Cdn Cache Server V2.0), 1.0 lydx136:8105 (Cdn Cache Server V2.0)
Connection: keep-alive

neo@neo-OptiPlex-780:/tmp$ curl -H
"If-Modified-Since: Fri, 12 May 2011 18:53:33 GMT" -I http://images.example.com/test/test.html
HTTP/1.0 304 Not Modified
Date: Mon, 16 May 2011 07:56:19 GMT
Content-Type: text/html
Expires: Mon, 16 May 2011 08:11:19 GMT
Last-Modified: Mon, 16 May 2011 06:57:39 GMT
ETag: "1205579110"
Cache-Control: s-maxage=7200, max-age=900
Age: 790
X-Via: 1.0 wzdx168:8080 (Cdn Cache Server V2.0)
Connection: keep-alive
			
			
ETag / If-None-Match
			
neo@neo-OptiPlex-780:/tmp$ curl -I http://images.example.com/test/test.html
HTTP/1.1 200 OK
Cache-Control: s-maxage=7200, max-age=900
Expires: Mon, 16 May 2011 09:48:45 GMT
Content-Type: text/html
Accept-Ranges: bytes
ETag: "1984705864"
Last-Modified: Mon, 16 May 2011 09:01:07 GMT
Content-Length: 22
Date: Mon, 16 May 2011 09:33:45 GMT
Server: lighttpd/1.4.26
			
			
			
neo@neo-OptiPlex-780:/tmp$ curl -H 'If-None-Match: "1984705864"' -I http://images.example.com/test/test.html
HTTP/1.1 304 Not Modified
Cache-Control: s-maxage=7200, max-age=900
Expires: Mon, 16 May 2011 09:48:32 GMT
Content-Type: text/html
Accept-Ranges: bytes
ETag: "1984705864"
Last-Modified: Mon, 16 May 2011 09:01:07 GMT
Date: Mon, 16 May 2011 09:33:32 GMT
Server: lighttpd/1.4.26
			
			
Accept-Encoding:gzip,defalte
			
$ curl -H Accept-Encoding:gzip,defalte -I http://www.example.com/product/374218.html
HTTP/1.1 200 OK
Date: Mon, 16 May 2011 09:13:18 GMT
Server: Apache
Accept-Ranges: bytes
Content-Encoding: gzip
Content-Length: 16660
Content-Type: text/html; charset=UTF-8
X-Pad: avoid browser bug
Age: 97
X-Via: 1.1 dg44:8888 (Cdn Cache Server V2.0)
Connection: keep-alive
			
			

			
$ curl -H Accept-Encoding:gzip,defalte http://www.example.com/product/374218.html | gunzip
			
			
HOST
			
curl -H HOST:www.example.com -I http://172.16.1.10/product/374218.html
			
			
HTTP 认证

未认证返回401

			
# curl --compressed http://webservice.example.com/members
<html>
<head><title>401 Authorization Required</title></head>
<body bgcolor="white">
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx</center>
</body>
</html>
			
			

-u/--user <user[:password]> Set server user and password

使用 -u或者 --user 指定用户与密码

				# curl --compressed -uneo:chen http://webservice.example.com/members
			
Accept
			
-H "Accept: application/json"		
			
			
Content-Type
			
-H "Content-Type: application/json"			
			
			

4.17.15. curl-config

		
curl-config --features
		
		

4.17.16. 指定网络接口或者地址

--interface INTERFACE Use network INTERFACE (or address)

		
curl --interface 127.0.0.1 http://www.netkiller.cn		
		
		

4.17.17.  Cookie 处理

cookie 可以从 http header 设置

		
curl -LO -H "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u131-b11/d54c1d3a095b4ff2b6607d096fa80163/jdk-8u131-linux-x64.rpm
		
		

curl 还提供两个参数用于处理 cookie

		
-b, --cookie STRING/FILE Read cookies from STRING/FILE (H) 读取 cookie 文件
-c, --cookie-jar FILE Write cookies to FILE after operation (H) 将 cookie 写入文件
		
		
		
curl -c cookie.txt -d "user=neo&password=123456" http://www.netkiller.cn/login
curl -b cookie.txt http://www.netkiller.cn/user/profile
		
		

4.17.18. Restful 应用 JSON 数据处理

下面提供一些使用 curl 操作 restful 的实例

GET 操作

		
curl http://api.netkiller.cn/v1/withdraw/get/15.json
		
		

用户认证的情况

		
curl http://test:123456@api.netkiller.cn/v1/withdraw/get/id/815.json
		
		

POST 操作

		
curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '
{
"id":"B040200000000000000","name":"Neo","amount":12,"password":"12345","createdate":"2016-09-12 13:10:10"

}' https://test:123456@api.netkiller.cn/v1/withdraw/create.json
		
		
curl -H "Accept: application/json" -H "Content-Type: application/json" -d '{"id":100000, "username":"netkiller", "password":"123456", "email":"netkiller@msn.com"}' curl http://localhost:8080/restful/validation
Curl Oauth2
			
URL=https://api.netkiller.cn
token=$(curl -k --cacert -s -X POST --user 'api:secret' -d 'grant_type=password&username=netkiller@msn.com&password=123456' ${URL}/oauth/token | grep -o -E '"access_token":"([0-9a-f-]+)"' | cut -d \" -f 4 )
curl -k -H "Accept: application/json" -H "Content-Type: application/json" -H "Authorization: Bearer ${token}" -X GET ${URL}/search/article/list/22/0/5.json			
			
			
Curl + Oauth2 + Jwt

获取 access_token 字符串

方法一

			
curl -s  -X POST --user 'api:secret' -d 'grant_type=password&username=netkiller@msn.com&password=123456' http://localhost:8080/oauth/token | sed 's/.*"access_token":"\([^"]*\)".*/\1/g'
			
			

方法二

			
curl -s  -X POST --user 'api:secret' -d 'grant_type=password&username=netkiller@msn.com&password=123456' http://localhost:8080/oauth/token | grep -o -E '"access_token":"(.+)"' | cut -d \" -f 4			
			
			

4.17.19. 访问自签名证书

		
curl --cacert certs/domain.crt  https://www.netkiller.cn/		
		
		

4.17.20. HTTP2

curl 已经支持 HTTP2,使用方法如下

		
neo@MacBook-Pro ~/workspace % curl -I --http2 https://www.google.com  
HTTP/2 200 
date: Tue, 26 Feb 2019 06:36:03 GMT
expires: -1
cache-control: private, max-age=0
content-type: text/html; charset=ISO-8859-1
p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
server: gws
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
set-cookie: 1P_JAR=2019-02-26-06; expires=Thu, 28-Mar-2019 06:36:03 GMT; path=/; domain=.google.com
set-cookie: NID=160=aQySEvsSa9gVU8qivD3t5qsgi_PRUtD5Ao3nRb48jMyLAzlYA1ViWuF1BaZHChVzY6EuknQ0OUz3Z2vhWwrclzO4WV6BmWgPhz6jowqFF3XCStsyYVwLQp2-_c0aGioBryAP1bTT0c-PX9iJzk5Zcbm2cFs6kH0Qb2a_3ML7Ioc; expires=Wed, 28-Aug-2019 06:36:03 GMT; path=/; domain=.google.com; HttpOnly
alt-svc: quic=":443"; ma=2592000; v="44,43,39"
accept-ranges: none
vary: Accept-Encoding		
		
		

HTTP/2 200 表示当前采用 HTTP2 连接

4.17.21. FAQ

Error in TLS handshake, trying SSLv3...

解决方案

		
curl -v --cipher rsa_rc4_128_sha https://www.mpaymall.com/MPay/MerchantPay.do