One - One Code All

Blog Content

python网页获取urllib和requests获取到的content-length长度不一致

Python   2011-09-26 17:54:38

如下:


urllib.request.urlopen(url).info()的结果如下:


Content-Type: application/vnd.ms-excel
Connection: close
age: 942
CACHE: TCP_REFRESH_HIT
Vary: Accept-Encoding
Powered-By-ChinaCache: HIT from BGP-GZ-d-3WB
ETag: "160000000a84f5-7e00-574efad018534"
Last-Modified: Mon, 03 Sep 2018 04:07:12 GMT
Keep-Alive: timeout=60
Expires: Mon, 24 Sep 2018 13:39:59 GMT
Date: Mon, 24 Sep 2018 09:39:59 GMT
Content-Length: 32256
Server: ***
CC_CACHE: TCP_HIT
Accept-Ranges: bytes


requests.get(url, stream=True).headers 的结果如下:

{'Content-Type': 'application/vnd.ms-excel', 'Connection': 'keep-alive', 'age': '941', 'CACHE': 'TCP_HIT', 'Date': 'Mon, 24 Sep 2018 09:40:00 GMT', 'Powered-By-ChinaCache': 'HIT from BGP-GZ-d-3WB', 'ETag': '"160000000a84f5-7e00-574efad018534"', 'Expires': 'Mon, 24 Sep 2018 13:40:00 GMT', 'Keep-Alive': 'timeout=60', 'Vary': 'Accept-Encoding', 'Content-Length': '12492', 'Content-Encoding': 'gzip', 'Last-Modified': 'Mon, 03 Sep 2018 04:07:12 GMT', 'Server': '***', 'CC_CACHE': 'TCP_HIT', 'Accept-Ranges': 'bytes'}


区别:

urllib:
Content-Length: 32256


requests:
'Content-Length': '12492'
'Content-Encoding': 'gzip',

如果不使用gzip可以这么提交:

headers1 = {'Accept-Encoding': ''} # ,headers=headers1
data = requests.get(url, stream=True,headers=headers1).headers


上一篇:MySQL复制表结构,表数据create table like,insert into table1 from table2
下一篇:http1.1断点续传介绍

The minute you think of giving up, think of the reason why you held on so long.