Product and Engineering
min read
Product and Engineering

To Scrape or Not to Scrape for Reviews

Learn what review scraping is and the potential risks of using a vendor that scrapes for reviews.

min read

Reviews are becoming more and more important for search engines and consumers alike. But as a business with multiple locations listed across hundreds of sites, managing reviews at scale can be a challenging and even daunting task. So you look to leverage a reputation management vendor to help collect and respond to reviews across sites like Google, Facebook, Yelp, Trustpilot, and many others. Unbeknownst to you, you choose a vendor that scrapes to pull in reviews.

What is review scraping?

Review scraping is the act of using web scraping tools and software to programatically crawl third party sites for ratings and review content.* For example, a review scraper might do a daily crawl of your Google listing to find the latest reviews from your customers. Oftentimes, review scraping is also used for gleaning competitive insights; by targeting a scraper on a competitor's listing, it's possible to analyze the content of their reviews for strategic purposes. Sure, these might be valuable insights, but at what cost?

The risks of review scraping.

Although the act of web scraping is not illegal, most sites including Google, Yelp, and Glassdoor, explicitly prohibit the use of scraping to obtain review content of both your own business as well as competitors. In order to gain access to review content, these sites typically require authentication and verification of ownership of the relevant business listings. Using a vendor that scrapes for reviews can put your brand at legal risk if they are not being obtained in a responsible manner. These risks are only amplified if you are in highly regulated industries, like healthcare or financial services. The chances of storing another business's patient or hospital data, or sensitive client financial information, increases substantially when your third-party vendor uses review scraping software. Additionally, publishers will often deploy tactics (e.g. changing the format of a webpage) to reduce the effectiveness of scraping software. Using a vendor that relies on scraping for reviews can lead to outages in service and an unreliable view of your brand's reputation if reviews are missed due to these counter-measures.

How to identify when a vendor is utilizing scraping.
  • "Too good to be true" insights: If a vendor provides competitor insights that could not be obtained without explicit authentication from the competitor such as sentiment analysis of their review content, this is a clear indicator that they are scraping.
  • Insecure credential sharing: If a vendor ever requests for your authentication credentials on a publisher outside of a controlled oAuth2 flow, they are likely depending on scraping for monitoring reviews and logging into your account manually to post any review responses.
Manage reviews the right way with Yext.

Unlike other reputation management vendors that scrape for reviews, Yext builds lasting relationships with its partners and abides by search engine best practices. Yext utilizes direct partnerships and API integrations with publishers to pull in reviews, and, as long as it's supported by the publisher, allows you to respond in-platform. Furthermore, we want our customers to trust us with their data. You might find competitor information that has been scraped from websites to be helpful, but would you want to work with a vendor that would be willing to scrape for your content and share it with your competitors? Yext does not scrape for review content to ensure we continue strong relationships with publishers, but more importantly to prioritize the privacy of our customers' data.

Choose a “Leader” in reputation management software.

If you're looking for the best reputation management software, choose Yext. G2 voted Yext as one of the top leaders for Enterprise-level reputation management software.**

Interested in learning more? Check out Yext's Reputation Management Solution, or schedule a demo here. Sources: *Zyte: What is Web Scraping? **G2

0 0 0 0 0 0

Share this article

Related Posts

Explore our Content Library