LogFAQs > #922716962

LurkerFAQs, Active DB, DB1, DB2, DB3, DB4, Database 5 ( 01.01.2019-12.31.2019 ), DB6, DB7, DB8, DB9, DB10, DB11, DB12, Clear

Topic List	Page List: 1
Topic	I need some python help. Is anyone strong in web scraping with Python?
PIayer_0 06/03/19 12:23:33 AM #3:	Not a Python user, but if you want to scrape from major websites, it's always easiest to check if they have a public API first and just use that. You're in luck here, as I found https://github.com/HackerNews/API For example, I can get the front page from https://hacker-news.firebaseio.com/v0/topstories.json?print=pretty At the time of posting, the top article was 20079671, so I can go to https://hacker-news.firebaseio.com/v0/item/20079671.json?print=pretty I believe requests is your standard HTTP client, so that's definitely a good place to start. BeautifulSoup seems to be for cleaning up "ugly" HTML and XML pages, but if you have access to a nice API you might as well use that instead. You'll probably need import json if you want to go this route. --- -Abraham Lincoln <cite>PIayer_0 posted [p:922716962] in I need som... [t:77758131]...</cite> <quote>Not a Python user, but if you want to scrape from major websites, it's always easiest to check if they have a public API first and just use that. You're in luck here, as I found https://github.com/HackerNews/API For example, I can get the front page from https://hacker-news.firebaseio.com/v0/topstories.json?print=pretty At the time of posting, the top article was 20079671, so I can go to https://hacker-news.firebaseio.com/v0/item/20079671.json?print=pretty I believe requests is your standard HTTP client, so that's definitely a good place to start. BeautifulSoup seems to be for cleaning up "ugly" HTML and XML pages, but if you have access to a nice API you might as well use that instead. You'll probably need import json if you want to go this route. --- -Abraham Lincoln</quote> ... Copied to Clipboard!
Topic List	Page List: 1