FilterHN

My experience trying to scrape Google Maps with no code

3 points

11 days ago

| 2 comments

A few months back I was working on a project to help founders that sell to SMBs get better quality leads (Current solutions like Zoominfo and Apollo don’t do very well for the SMB market). Of course, I wanted to do this as quickly as possible with as little code as possible. We found that people were manually going through Google Maps to find SMBs. They would use the search and manually type in the businesses they were looking for. For example, they would type “restaurants” and manually call/email them. What we decided to do was gather the Google Maps data autonomously and surface that to our customers so they could take all of it. The problem was that we would need a bunch of data from Google Maps to pull it off. We would need to grab all the SMBs across the United States which is a huge undertaking. Initially, I tried no-code AI web scraping solutions and they worked horribly. For some reason, I couldn’t even get them to scroll down on the page. I was also able to reverse engineer their open-source code and discover that they were taking the entire web page and passing it into GPT to extract data. That just burned my Openai bill. I then tried the semi-code approach where I would use something like Apify or Google Places API to scrape the businesses. This worked better but still, there was an issue of price at the scale we wanted. Eventually, we ended up writing our scraper for the task. This experience was so horrible we ended up creating https://potarix.com . Our approach is fundamentally different because we’re focused on generating web scraping code in the backend (Think bolt.new but for data extraction). Right now the app is pretty bare bones, simply type in a url and prompt it with the data you want from the site and you’ll get it. However, we have some sauce we built in house for our custom clients that we’ll be releasing in the SAAS soon. We’ve built functionality to click, type, scroll, etc. on the page. We’ll also be providing our generated script soon as well, so you guys can tweak the code accordingly to bring the script to 100% accuracy!

▲

runjake

10 days ago

[-]

Please use paragraphs next time. This is unreadable.

For others, this is a Tell HN for https://potarix.com/

▲

AznHisoka

10 days ago

[-]

Isnt this what OxyLabs does? They handle all the scraping for things like Google Maps, Shopping, SERPS etc and have it easily ready as an API

▲

OlikEten

10 days ago

[-]

there are many web scraping companies doing this, among them Apify too - https://apify.com/compass/crawler-google-places

▲

rustdeveloper

10 days ago

[-]

And many more: https://compareproxy.com/

▲

KomoD

7 days ago

[-]

("Scraping Fish" owns this site but it isn't disclosed anywhere, and this guy "rustdeveloper" seems associated with them, most of his comments push that service)

▲

rustdeveloper

7 days ago

[-]

This is correct, my friends from Scraping Fish are hosting https://compareproxy.com to help people find proxy for web scraping. I'm happy to "push" for Scraping Fish as I'm also a satisfied user who received a lot of help from the founders for my web scraping projects.