When we first started building Suggestable, we knew we were tapping into something powerful—something that could help marketers uncover the exact keywords driving search behavior. But along the way, we heard the concerns: “Is this risky? Will search engines block my IP? Are we crossing a line here?”
Here’s the truth: suggestion mining isn’t just a valuable practice—it’s a responsible one. Auto-suggestion services are designed for massive, real-time use by millions of people. The data we’re pulling is incredibly lightweight (most often less than 200bytes per request and compressed further), and the very platforms we’re querying want advertisers to discover these keywords. After all, they rely on this data to power their PPC models.
In this post, I’ll lay out why suggestion mining is not only sustainable but essential for anyone serious about keyword research. If you’ve ever worried about the impact of this process, let me show you why it’s time to put those fears to rest.
1. Suggestion Services Are Built for High Volume
Auto-suggestion services are specifically designed to handle massive, simultaneous usage. Millions of users worldwide generate suggestion requests every second. As a result, search engines and other platforms have robust systems in place to manage this constant traffic effortlessly.
2. Minimal Bandwidth Usage
Unlike traditional web crawling or SERP scraping, which can consume significant bandwidth, suggestion mining is exceptionally lightweight. Just imagine, every keystroke into the Chrome URL box generates a Json suggestion packet sent from the search engine. The data packets returned by suggestion services are typically under 1KB and then compressed down to near 100 hundred bytes. This means suggestion mining generates a negligible load compared to other forms of data extraction.
3. Search Engines Are Incentivized to Support This Activity
The keywords uncovered through suggestion services directly fuel PPC (pay-per-click) advertising strategies. By identifying high-value keywords, marketers help drive campaigns that align with the search engine’s own revenue model. Suggestion mining, in essence, supports the ecosystem by helping advertisers succeed.
4. Historical Precedent
Real-world examples show that search engines rarely take action against suggestion pulls, even at massive scale. One suggestion cloud service, for instance, successfully pulled over 250 million suggestions before being asked to transition to an official API. This demonstrates that platforms understand and accommodate such usage within reason.
5. Proof of Feasibility: Suggestable’s Scale
During the development of Suggestable, we accumulated a 150-million-keyword database over 3 years. Not only did this process run smoothly, but it allowed us to build meaningful keyword relationships and insights without any system interruptions or pushback from services.
6. Suggestable Respects the Resources
Not only does Suggestable offer options for “max downloads” and “download delay”, it has built-in limits that reduce load on the suggestion service. This includes built-in per-service delays and support for both uncompressed, as well as compressed Gzip and Brotli compression optimization. Then we default to showing the SERP that is generated by the suggestion service, allowing you to view and click the serp in the built-in browser. Additionally, when running more in-depth reports, Suggestable will pause more often and cause less load.
For example, we delay a minimum of 170 milliseconds between typing. To get to that figure we calculated:
That an average type types at a average time between at 60 words per minute as a rule of thumb. Thus:
- Assume an average word length: Typically, the average word length in English is about 5 characters, plus 1 space for separation, making 6 keystrokes per word –Source
- Calculate total keystrokes per minute:
60 words/min×6 keystrokes/word=360 keystrokes/min60 \, \text{words/min} \times 6 \, \text{keystrokes/word} = 360 \, \text{keystrokes/min} - Convert keystrokes per minute to keystrokes per second:
360 keystrokes60 seconds=6 keystrokes/second\frac{360 \, \text{keystrokes}}{60 \, \text{seconds}} = 6 \, \text{keystrokes/second} - Calculate the time between each keystroke:
16 keystrokes/second≈0.167 seconds/keystroke\frac{1}{6 \, \text{keystrokes/second}} \approx 0.167 \, \text{seconds/keystroke}
Final Answer:
At 60 words per minute, there is approximately 167 milliseconds (ms) between each keypress. Which is why we rounded to 170 milliseconds minimum between pulls from various services. In real world usage with multiple services, the delay is often higher than that.
Conclusion
Suggestion mining, when done responsibly, is not only feasible but also aligns with the goals of search engines and advertisers alike. With minimal bandwidth consumption and systems designed for heavy traffic, leveraging auto-suggestions is a sustainable, low-impact practice that delivers immense value to marketers, site owners, and the search ecosystem as a whole.