page.find_all(True) – returns all html tags in the beautifulsoup html page if word in i.text: – check for a match in every tag’s text component, wrapped in try/fail in case a tag has no text component.
Honestly, this should be a default setting for the requests library. Without it, your code is doomed if you get any connection errors. Add a ‘try’ when connecting to your URL, then use these exceptions to let your code keep running if there are any errors. This works great for when you’re scraping many URLs …
Getting tons of traffic from all over the web is great. It’s easy to see where users came from. If you’re starting at your analytics, you’ll see something called your traffic source and your traffic medium. A traffic source is a category, or traffic type such as Social, Direct, Paid, or Organic. Your traffic medium …
If you’ve ever had to run Google Ads (or even Facebook Ads), you know just how granular your targeting can be. It’s an incredibly powerful way to do marketing, but you need to know how to properly research your client’s target audience. If you don’t have the data to back up your assumptions, then you’re …
There is no doubting that having a higher follower count or more views on your videos will make you look more established. Have a look at the sample below for the YouTube search “how to trim curtain bangs”. Two of these videos have well over two million views and one video has barely 20k views. …
Part of what I’ll do for any company’s SEO is to create local business directories for them. To do this right, it’s important to understand how many listings already exist. Most businesses will have less than 50 business listings. For bigger, older companies, it’s usually 200+. Either way, there is no such thing as too …
I found a guy’s project on GitHub where he already built the tool for checking backlinks. To use this tool, you need to provide a text file that contains the referring domain and the destination URL. They need to be side by side in their own row. Each referring domain is scraped and searched. The …
If you can get the job done with WordPress REST API, do it. Getting into XMLRPC is a whole other beast. For starters, XMLRPC requires you to connect using a private server. These days, you’ll need to encrypt that connection, too. I was building a tool that needed to search through my entire site’s post …
Last week, I ran a small test to publish thousands of blog posts on a single WordPress website. It was more or less a test of strength for exploring the WordPress REST API using Python. These blog posts started to trigger a lot of web scrapers that are looking to leave spammy blog comments. Before …
In yesterday’s post, I wrote about how I learned how to set up a local SMTP server. I came across a problem that I couldn’t solve, and it started getting late. What’s kind of stupid about sending emails is the fact that you need to stuff all the important data into one big message. For …