How SE Ranking collects and processes keyword data

SEO is no guessing game. It’s a data-driven process. The data you use should be accurate, so you can make good decisions to succeed not just in your SEO efforts but your website performance and business growth. 

That’s why we’d like to share some SE Ranking keyword inner workings with you so you can be sure you’re getting detailed information on each search query you optimize your pages for. 

A few words about our keyword database 

SE Ranking’s database of keywords spans across the entire globe,  supporting 190 countries. Few other tools can make the same claim. SEMrush, for instance, covers 142 countries, while Ahrefs supports 171.

*This information is current as of October 2022. 

Talking numbers, we have around 4.7B keywords in our database. 

SE Ranking’s US database, by the way, is one of the biggest on the market. In just two months, we managed to increase it by 30%, and now it contains approximately 659M keywords. 

Our Competitive Website Analysis tool and Keyword tool make use of our densely packed keyword database by being able to identify keywords that other instruments cannot. We focus on keywords with real traffic for different markets, intents, and regions when expanding our databases. Whatever your niche and purpose, we’ll suggest millions of relevant keywords that can help you rank high in the SERP.   

It doesn’t end there, though. SE Ranking works hard every day to expand the keyword database and add more strong terms that will attract traffic to your website.  

Where does SE Ranking’s keyword data come from?

Keywords without data are like a dead phone. You can hold this dead phone in your hands but you can’t call anyone with it because there’s no way to access the numbers in its phone book. In SEO, numbers also play a crucial role.

Keyword search volume is one of the most critical metrics. You can’t plan your SEO activities effectively unless you clearly understand the term’s search potential.

For you to see this information on your SE Ranking dashboards and make well-informed SEO decisions, we first need to collect raw data. SE Ranking uses: 

• Data retrieved from search engines by using API connections.
• Data collected by our very own crawlers.

How does SE Ranking manipulate keyword data?

Let’s take a look at data processing. That’s where the real magic happens.

At SE Ranking, we apply our own algorithm based on a set of formulas and scripts that process the initial raw data. We also use AI and NLP techniques to assess language data.

Our algorithm overlays search volume data from Google on our clickstream data and applies the ungrouping method. Because of this, you can see the specific aggregated search volume value for each keyword separately, unlike the value range system like in Google Keyword Planner and other tools.

VS.

Another approach we use is trend ratio analysis, which is based on Google Trends. This tool shows the relative popularity of a search query over time. If analyzed correctly, though, it can help you better understand the frequency by which users search for keywords with equal or similar search volume values. 

For example, let’s say you don’t have a clear picture of the potential performance of two similar terms with the same search volumes. You may choose a keyword that will perform poorly and your SEO activities will fail to produce the desired results.  

We created a one-of-a-kind comparison test sequence that allows us to fine-tune the keyword search volume data, making it more accurate and, more importantly, real.

A note on clickstream

Clickstream is a record of the “steps” a user takes online. It’s sort of like a virtual footprint, showing how the user moved from the search to the target action (buying a product, ordering a service, registering, etc.). 

In SEO, clickstream data reveals whether a person searched for one or more keywords, clicked on a term, and then returned to the search.

It’s more accurate to get keyword search volume from the clickstream because it doesn’t count bot traffic. Moreover, you can see which terms are searched multiple times, generate purchases, and get more clicks, all of which are vital for CTR calculation.

Browser extensions, plugins, and other programs collect clickstream data. Users, however, must consent to data collection. At SE Ranking, we collect and aggregate clickstream data from all permitted sources and use it in our keyword search volume calculating mechanism.   

More on SE Ranking’s ungrouping techniques

Our ungrouping method is a standalone approach that distinguishes us from other tools like ours. It allows us to talk big about the sky-high accuracy of our data. 

We don’t combine the search volume of similar terms. Our algorithm treats each keyword variation as a separate entity and ungroups the volume values that Google Keyword Planner combines. This means that the same term in the plural and singular form will have separate search volume values because these are two different words for SE Ranking.

When analyzing the same keywords, Google Keyword Planner shows data for only one term. This word is singular.

However, the SERPs for each of the two analyzed words are different.

The same applies to keywords with: 

• Articles
• Prepositions
• Regular/comparative/superlative adjectives
• Interrogative pronouns
• Hyphens
• Diacritical marks

Each will have their own search volume data, allowing you to conduct a more accurate keyword analysis.

How often does SE Ranking update search volume data?

We update search volume data regularly: 

• High-frequency keyword values get updated once a month. 
• Low-frequency terms are refreshed once every three months.

Our updates have scheduled timeframes, but some users may need updated metrics earlier. They can request an update manually to get data sooner.

If you can’t find your keyword of interest in our database, you can add it manually, but the process depends on your goals:

• If you want to view the metrics for the missing word once, the parsing on-the-fly mechanism will collect data. You’ll see the result in 10 seconds or less.
• If you plan to track the missing keywords regularly, use the Expand Database feature. Missing keywords will be added to our database, and our algorithm will process them just like all other terms. You’ll also get access to Historical Data for new keywords. The process can take anywhere from several hours to three weeks. The data will be instantly available after that. 

What other keyword data can you get with SE Ranking?

Apart from keyword search volume, SE Ranking displays its difficulty and cost per click for paid ads. With accurate data on these metrics, you can better understand your chances of ranking for a given keyword and how much budget you need for your PPC campaign based on this term.

• To determine the ranking complexity, our algorithm compares the quality of 10 relevant search results for this term, their backlink profiles, and the keyword search volume. As a result, you will see a number from 0 to 100 that corresponds to one of the eight levels of ranking difficulty (from Surefire to Impossible). Other popular SEO tools show keyword difficulty using 4-6 levels, which doesn’t provide a clear enough understanding of your prospects.

• Because SE Ranking takes cost-per-click data directly from Google Ads, it’s as accurate as can be. We also update CPC data regularly as this is a highly volatile metric.

Why should you trust SE Ranking’s keyword data?

Don’t be gullible when it comes to keyword data, but you can trust SE Ranking and its tools because it’s your partner on your way to the top of the SERPs.

We at SE Ranking want to make your SEO as effective and painless as possible, which can only be accomplished with 100% accurate keyword data. The SE Ranking team delivers on this promise every day!

We hope this post has given you a better understanding of where the numbers in our Keyword and Competitive Research tools come from and what makes them accurate.

If you have more questions about how we work with keywords, ask them in the comments. We’ll try to explain every detail, and perhaps even write a separate post about it.

Written by

Yevheniia Khromova

Yevheniia is a content marketer and copywriter at SE Ranking. She is a fan of SEO, digital marketing, and content creation. Yevheniia believes simplicity is key as it helps reduce the clutter in our heads and focus on what matters. This is the principle she uses in her writing. She also enjoys cooking, learning new spices and flavors, traveling, and playing board games.