阿藏哥的部落格~不應該阿

最近選舉要到

小篇寫一個爬蟲,爬藍綠煤體的焦點新聞

code :

import requests
from bs4 import BeautifulSoup


def main():
    print('蘋果今日焦點')
    dom = requests.get('https://tw.appledaily.com/hot/daily').text
    soup = BeautifulSoup(dom, 'lxml')
    for eye in soup.find('ul','all').find_all('li'):
        #print(eye)
        print(eye.find('div','aht_title_num').text,eye.find('div','aht_title').a.text,eye.find('div','aht_pv_num').text,eye.find('div', 'aht_title').a['href'])


    print('\n')
    print('自由今日焦點')
    dom = requests.get('http://news.ltn.com.tw/list/newspaper').text
    soup = BeautifulSoup(dom, 'lxml')
    for eye1 in soup.find('ul','list').find_all('li'):
        print(eye1.find('a', 'tit').text.strip(),'http://news.ltn.com.tw/'+str(eye1.find('a')['href']))



    print('\n')
    print('中時今日焦點')

    for i in range(1,7):
        html='https://www.chinatimes.com/newspapers/260102?page= '+str(i)
        dom = requests.get(html).text
        soup = BeautifulSoup(dom, 'lxml')
        for eye1 in soup.find('div', 'listRight').find('ul').find_all('li'):
            print(eye1.find('a').text.strip(),"https://www.chinatimes.com/"+str(eye1.find('a')['href']))




if __name__ == '__main__':
    main()

=====================result=============================

蘋果今日焦點
01 劉寶傑被休假 東森：背痛 談「韓流」傳遭施壓 馬西屏：近半月才講政治 32956 https://tw.news.appledaily.com/headline/daily/20181104/38170491
02 性侵 破產 精神失常 港星藍潔瑛 離奇暴斃 62812 https://tw.news.appledaily.com/headline/daily/20181104/38170622
03 自爆遭2影壇大哥性侵 痛不欲生 49952 https://tw.news.appledaily.com/headline/daily/20181104/38170638
04 藍潔瑛不認帳 惹周遊吐苦水 遭狄鶯踹飛 楊慶煌勸架 24047 https://tw.news.appledaily.com/headline/daily/20181104/38170405
05 靚絕五台山2男友自殺 偷渡星爺家爆朱茵抓姦 19087 https://tw.news.appledaily.com/headline/daily/20181104/38170414
06 樹菊嬤：我不認識韓國瑜 陳其邁 最強2菜販同台行善 24941 https://tw.news.appledaily.com/headline/daily/20181104/38170449
07 因信仰相同 差24歲譜父女戀 22576 https://tw.news.appledaily.com/headline/daily/20181104/38170707
08 誹謗李登輝妻 憂遭求償 房產設定抵押給妹 謝啟大為祖產告妹 13621 https://tw.news.appledaily.com/headline/daily/20181104/38170770
09 【5大熱門3C】小米掃地機創歷史新低價 6421 https://tw.news.appledaily.com/headline/daily/20181104/38170393
10 曾莞婷 告辭《金家》短收7位數 燒千萬打版網拍平價衣 9340 https://tw.news.appledaily.com/headline/daily/20181104/38170440
11 野生裘莉現蹤首爾 拎兒吃烤肉勘查大學 9363 https://tw.news.appledaily.com/headline/daily/20181104/38170583
12 威士忌「聖杯」 1口喝掉1台國產車 7360 https://tw.news.appledaily.com/headline/daily/20181104/38170138
13 小辣椒祕婚34天 唯美白紗見光 7913 https://tw.news.appledaily.com/headline/daily/20181104/38170587
14 Melody 么女重度近視被當沒禮貌 自責「天塌下來」 13914 https://tw.news.appledaily.com/headline/daily/20181104/38170598
15 寶弟 暗巷拎新歡噴菸 玩老揹少混血妹露溝 10991 https://tw.news.appledaily.com/headline/daily/20181104/38170421
16 合歡山半馬憾事 跑者終點前猝死 15253 https://tw.news.appledaily.com/headline/daily/20181104/38170718
17 花博首日湧5萬人 隱藏版彩蛋《蘋果》空拍揭密 21097 https://tw.news.appledaily.com/headline/daily/20181104/38170643
18 堅信會復活 如真實版《三更》女洗男友屍70天 生蛆 21772 https://tw.news.appledaily.com/headline/daily/20181104/38170698
19 裝備9公斤 阿伯揹15手機抓寶 16794 https://tw.news.appledaily.com/headline/daily/20181104/38170741
20 施菲亞罩杯up up F奶看書長大的 7059 https://tw.news.appledaily.com/headline/daily/20181104/38170571
21 從政歷經大起大落 遇「韓流」依然穩健 11461 https://tw.news.appledaily.com/headline/daily/20181104/38170468
22 《龍捲風》搭韓流 吸22萬觀眾 9069 https://tw.news.appledaily.com/headline/daily/20181104/38170498
23 桃猿第6冠 2度連霸 陳俊秀MVP連莊 9451 https://tw.news.appledaily.com/headline/daily/20181104/38170325
24 鍾欣怡掌銀根 5000萬新房尪沒份 6594 https://tw.news.appledaily.com/headline/daily/20181104/38170431
25 美祭「禁油令」台料豁免 全面恢復制裁伊朗 明起生效 11168 https://tw.news.appledaily.com/headline/daily/20181104/38170235
26 藍潔瑛 香港最美蜘蛛精 13849 https://tw.news.appledaily.com/headline/daily/20181104/38170678
27 水晶女神圓一圈 溢出雙下巴 5461 https://tw.news.appledaily.com/headline/daily/20181104/38170594
28 Kelly Talk：藍心湄老包配鑽錶 富婆睡醒也貴氣 4448 https://tw.news.appledaily.com/headline/daily/20181104/38170618
29 范疇專欄：民進黨正向中共期待的路線靠攏（范疇） 5233 https://tw.news.appledaily.com/headline/daily/20181104/38170541
30 今天我最美 9628 https://tw.news.appledaily.com/headline/daily/20181104/38170723

自由今日焦點
台中花博開幕 5萬人爭睹 http://news.ltn.com.tw/news/focus/paper/1244325
毒大閘蟹 偷賣逾9噸 http://news.ltn.com.tw/news/focus/paper/1244326
陳其邁破釜沉舟 辭立委拚市長 http://news.ltn.com.tw/news/focus/paper/1244327
林榮三文學獎 充滿生命力 http://news.ltn.com.tw/news/focus/paper/1244328
桃猿5年奪4冠 http://news.ltn.com.tw/news/focus/paper/1244329
中國「巨魔工廠」 拿台灣選舉練兵 http://news.ltn.com.tw/news/focus/paper/1244319
政院︰假消息大軍壓境 http://news.ltn.com.tw/news/focus/paper/1244320
入腦入心 中國輿論戰升級 http://news.ltn.com.tw/news/focus/paper/1244321
蘇引延禧攻略 諷侯心機多方法壞 http://news.ltn.com.tw/news/focus/paper/1244322
侯打出神探牌 找李昌鈺鑑定加持 http://news.ltn.com.tw/news/focus/paper/1244323
觀塘案後首合體 賴揆為鄭文燦站台 一見面先致歉 http://news.ltn.com.tw/news/focus/paper/1244324

中時今日焦點
偷賣毒蟹 重罰破億 https://www.chinatimes.com//newspapers/20181104000413-260114
韓國瑜、陳樹菊 最強菜販愛心合體 https://www.chinatimes.com//newspapers/20181104000418-260118
燕反空汙 轟綠政策錯誤 https://www.chinatimes.com//newspapers/20181104000421-260118
龍拚花博 英賴現身力挺 https://www.chinatimes.com//newspapers/20181104000423-260118
北中南串聯 喊用選票救藻礁 https://www.chinatimes.com//newspapers/20181104000425-260118
地表最大機械花 必拍亮點 https://www.chinatimes.com//newspapers/20181104000427-260118
力保中台灣 龍團隊祭政績牌 https://www.chinatimes.com//newspapers/20181104000429-260118
告急 陳其邁辭立委背水一戰 https://www.chinatimes.com//newspapers/20181104000430-260118
韓流助攻彰化 王惠美也爆棚 https://www.chinatimes.com//newspapers/20181104000432-260118
姚文智籲大家一起幫港都 https://www.chinatimes.com//newspapers/20181104000434-260118
新聞透視－高雄恐成震央 一路震向2020 https://www.chinatimes.com//newspapers/20181104000435-260118
北漂族返鄉 投票列車熱售9成 https://www.chinatimes.com//newspapers/20181104000436-260118
1坪換1坪沒了 大林蒲人北上抗議 https://www.chinatimes.com//newspapers/20181104000438-260118
高市：未立專法 先照規範徵收 https://www.chinatimes.com//newspapers/20181104000441-260118
承諾若跳票 環團嗆用選票教訓 https://www.chinatimes.com//newspapers/20181104000442-260118
新聞透視－監控臉書 就是戕害民主 https://www.chinatimes.com//newspapers/20181104000443-260118
國安局坦承監控臉書 小英急滅火 丁：威權復辟 民進黨政府應道歉 https://www.chinatimes.com//newspapers/20181104000444-260118
IG女王凱莉 3年賺278億元 https://www.chinatimes.com//newspapers/20181104000455-260119
鋼鐵人馬斯克生鏽走鐘 https://www.chinatimes.com//newspapers/20181104000458-260119
前澤友作將搭火箭繞月一周 https://www.chinatimes.com//newspapers/20181104000460-260119
多薩總統訪陸 習讚順應潮流 https://www.chinatimes.com//newspapers/20181104000471-260108
習針對性十足 台外交再亮警訊 https://www.chinatimes.com//newspapers/20181104000474-260108
美再制裁 台購油或享豁免 https://www.chinatimes.com//newspapers/20181104000475-260110
中油：伊朗油源比例極低 影響不大 https://www.chinatimes.com//newspapers/20181104000477-260110
共享機遇 首屆進博會明登場 https://www.chinatimes.com//newspapers/20181104000478-260110
女力自決 音樂家美得有理 https://www.chinatimes.com//newspapers/20181104000480-260115
陳毓襄貴氣逼人 https://www.chinatimes.com//newspapers/20181104000483-260115
慕特永遠露香肩 https://www.chinatimes.com//newspapers/20181104000485-260115
阿格麗希 琴與愛都迷人 https://www.chinatimes.com//newspapers/20181104000487-260115
全台資深里長 在地守護神 https://www.chinatimes.com//newspapers/20181104000501-260107
30歲拚到78歲 李聰平邁向13連霸 https://www.chinatimes.com//newspapers/20181104000506-260107
政壇長青樹 服務擺第一 https://www.chinatimes.com//newspapers/20181104000508-260107
神探李昌鈺 挺神捕侯友宜 https://www.chinatimes.com//newspapers/20181104000509-260107
黃敏惠、蕭淑麗 看板正面交鋒 https://www.chinatimes.com//newspapers/20181104000511-260107
饒造勢跑透透 劉重教育分配 https://www.chinatimes.com//newspapers/20181104000521-260107
悠遊六龜山城 泡暖湯啖美食 https://www.chinatimes.com//newspapers/20181104000532-260107
賴文德成立競總 韓站台拉人氣 https://www.chinatimes.com//newspapers/20181104000540-260107
陳宏宗可望爭縱谷首位議長 https://www.chinatimes.com//newspapers/20181104000549-260107
華麗王船陸上行舟 震撼信眾 https://www.chinatimes.com//newspapers/20181104000555-260107
藍潔瑛自宅暴斃 飽嘗苦痛 來世不要做人 https://www.chinatimes.com//newspapers/20181104000589-260112
劉德華痛難捨鄭少秋不好受 https://www.chinatimes.com//newspapers/20181104000593-260112
周遊憶藍潔瑛：這輩子最怕她 https://www.chinatimes.com//newspapers/20181104000595-260112
藍潔瑛高顏值演技好受封最美蜘蛛精 https://www.chinatimes.com//newspapers/20181104000598-260112
入住澎湖福朋喜來登 每人1111元 https://www.chinatimes.com//newspapers/20181104000621-260113
《草間∞彌生》記錄當代藝術大師 https://www.chinatimes.com//newspapers/20181104000625-260113
珍稀西洋參 甘涼不燥養生首選 https://www.chinatimes.com//newspapers/20181104000630-260113
創意x科技 AI作曲首獲獎 https://www.chinatimes.com//newspapers/20181104000639-260113
《大君主行動》JJ亞柏拉罕嚇影迷 https://www.chinatimes.com//newspapers/20181104000645-260112
賈方艾德坡咬牙熬過跳傘戲 https://www.chinatimes.com//newspapers/20181104000647-260112
來新北當不塑之客抽iPhone XR https://www.chinatimes.com//newspapers/20181104000649-260113
淡水肥蟹宴秋季限定 https://www.chinatimes.com//newspapers/20181104000653-260113
電競業趨勢論壇探討台灣新熱潮 https://www.chinatimes.com//newspapers/20181104000655-260110
日本大型賽事商機日 吸引百家廠商捧場 https://www.chinatimes.com//newspapers/20181104000656-260110
深耕有成台灣人壽獲4項玉山獎 https://www.chinatimes.com//newspapers/20181104000658-260110
威士忌台灣之光 https://www.chinatimes.com//newspapers/20181104000660-260113
Omar威士忌混搭果香拓市場 https://www.chinatimes.com//newspapers/20181104000662-260113
噶瑪蘭釀造寶島味獲獎無數 https://www.chinatimes.com//newspapers/20181104000668-260113
台灣的另一種味道 https://www.chinatimes.com//newspapers/20181104000674-260113
TOYOTA CAMRY改款 駕駛更有感 https://www.chinatimes.com//newspapers/20181104000677-260113
BMW X5智慧駕駛安全至上 https://www.chinatimes.com//newspapers/20181104000680-260113
品醇2018薄酒萊與法國零時差 https://www.chinatimes.com//newspapers/20181104000682-260113
MONTAGUT替你搞定 https://www.chinatimes.com//newspapers/20181104000685-260113
三宅健太獻聲霸氣魔龍 https://www.chinatimes.com//newspapers/20181104000687-260113
聖火凝師徒劍決 鬼沒裂君臣斷念 https://www.chinatimes.com//newspapers/20181104000690-260113
蕭一凡 仿古而不泥古 https://www.chinatimes.com//newspapers/20181104000691-260113