前言
大家早好、午好、晚好吖 ❤ ~欢迎光临本文章
如果有什么疑惑/资料需要的可以点击文章末尾名片领取源码
课程亮点:
1、动态数据抓包
2、JSON数据解析方法
环境介绍:
-
python 3.8 解释器: 执行代码的
-
pycharm 专业版: 写代码的 帮助我们 快速开发代码的
-
requests >>> pip install requests
-
sekiro >>> RPC框架
抓包分析
确定需要的数据 然后找该数据的来源
https://www.zhipin.***/wapi/zpgeek/search/joblist.json?scene=1&query=python%E7%88%AC%E8%99%AB&city=100010000&experience=&payType=&partTime=°ree=&industry=&scale=&stage=&position=&jobType=&salary=&multiBusinessDistrict=&multiSubway=&page=1&pageSize=30
代码实现
-
发送请求
-
获取数据
-
解析数据
代码展示
'''
python资料获取看这里噢!! 小编 V:qian97378,即可获取:
文章源码/教程/资料/解答等福利,还有不错的视频学习教程和PDF电子书!
'''
import requests
import csv
with open('boss.csv', mode='w', newline='', encoding='utf-8') as f:
csv.writer(f).writerow(['jobName', 'cityName', 'areaDistrict', 'businessDistrict', 'salaryDesc', 'jobExperience', 'jobDegree', 'skills', 'brandName', 'brandIndustry', 'brandStageName', 'brandScaleName', 'welfareList'])
# 请求头
headers = {
'Referer': 'https://www.zhipin.***/web/geek/job?query=python%E7%88%AC%E8%99%AB&city=100010000&page=1',
# wd_guid=74818318-ca4c-4c55-8023-515796a1af9a; historyState=state; _bl_uid=gLl3kn8OimXutaow0kgvk3sq8qbh; __g=-; Hm_lvt_194df3105ad7148dcf2b98a91b5e727a=1699018011,1699707457,1699962314,1699965523; Hm_lpvt_194df3105ad7148dcf2b98a91b5e727a=1699965558; __zp_stoken__=3f6ceWFc7eFF0Skx2DUZ5fiBSb0RZDWpRc3ciERc6aTt3IB5UeRZzNyVTO2s9VRI5D24SQDxkDzVdfi9bSVQWXQZtQUcefiNsFCk5FHMuZHlKCVVkOnIELDdGfhZeASBxTVcgPzgbDT9tbCE%3D; __c=1699963318; __l=l=%2Fwww.zhipin.***%2Fweb%2Fgeek%2Fjob%3Fquery%3Dpython%25E7%2588%25AC%25E8%2599%25AB%26city%3D100010000%26page%3D1&r=&g=&s=3&friend_source=0&s=3&friend_source=0; __a=40359856.1696853365.1699962313.1699963318.75.11.11.75
'Cookie': 'wd_guid=74818318-ca4c-4c55-8023-515796a1af9a; historyState=state; _bl_uid=gLl3kn8OimXutaow0kgvk3sq8qbh; __g=-; Hm_lvt_194df3105ad7148dcf2b98a91b5e727a=1699018011,1699707457,1699962314,1699965523; Hm_lpvt_194df3105ad7148dcf2b98a91b5e727a=1699965558; __zp_stoken__=3f6ceWFc7eFF0SnAKXyRmfiBSb0Q1dz8TTzUiERc6aRxqORo%2BeRZzNyVTeiwdMmkRD24SQDxkDzVzfntbRwlYVU9KZxESMRhUGQ0DFHMuZHlKCX0fXVJDbTdGfhZeASBxTVcgPzgbDT9tbCE%3D; __c=1699963318; __l=l=%2Fwww.zhipin.***%2Fweb%2Fgeek%2Fjob%3Fquery%3Dpython%25E7%2588%25AC%25E8%2599%25AB%26city%3D100010000%26page%3D1&r=&g=&s=3&friend_source=0&s=3&friend_source=0; __a=40359856.1696853365.1699962313.1699963318.75.11.11.75',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36'
}
for page in range(1, 31):
url = f'https://www.zhipin.***/wapi/zpgeek/search/joblist.json?scene=1&query=python%E7%88%AC%E8%99%AB&city=100010000&experience=&payType=&partTime=°ree=&industry=&scale=&stage=&position=&jobType=&salary=&multiBusinessDistrict=&multiSubway=&page={page}&pageSize=30'
# 1. 发送请求
response = requests.get(url=url, headers=headers)
# 2. 获取数据
json_data = response.json()
# 3. 提取数据
jobList = json_data['zpData']['jobList']
for job in jobList:
jobName = job['jobName']
cityName = job['cityName']
areaDistrict = job['areaDistrict']
businessDistrict = job['businessDistrict']
salaryDesc = job['salaryDesc']
jobExperience = job['jobExperience']
jobDegree = job['jobDegree']
skills = ' '.join(job['skills'])
brandName = job['brandName']
brandIndustry = job['brandIndustry']
brandStageName = job['brandStageName']
brandScaleName = job['brandScaleName']
welfareList = ' '.join(job['welfareList'])
print(jobName, cityName, areaDistrict, businessDistrict, salaryDesc, jobExperience, jobDegree, skills, brandName, brandIndustry, brandStageName, brandScaleName, welfareList)
with open('boss.csv', mode='a', newline='', encoding='utf-8') as f:
csv.writer(f).writerow([jobName, cityName, areaDistrict, businessDistrict, salaryDesc, jobExperience, jobDegree, skills, brandName, brandIndustry, brandStageName, brandScaleName, welfareList])
尾语
好了,今天的分享就差不多到这里了!
对下一篇大家想看什么,可在评论区留言哦!看到我会更新哒(ง •_•)ง
喜欢就关注一下博主,或点赞收藏评论一下我的文章叭!!!
最后,宣传一下呀~👇👇👇 更多源码、资料、素材、解答、交流 皆点击下方名片获取呀👇👇👇