Disclaimer: We are not going to be covering how you can go about installing Python or its modules as that is a separate topic and there are several tutorials about this on the web.

What you need:

• Access to your Google Search Console
• Google Data Studio
• Python 3
• The pandas and matplotlib modules

First, we are going to get the information. Let’s go to the Performance report inside the Google Search Console account that contains the data you want to work with. Simply exporting the data is not enough because Search Console averages the rankings for each keyword within the selected period. This means that if you want to see the positioning for the last year for a specific keyword, GSC will give you a single number, the result of the average positioning of each day for that keyword. As we need the detailed data day by day, let’s move on to the second tool…

If you have not already added the Google Search Console data source to the Data Studio, you can read this post that details how to do it.

In Google Data Studio we are going to insert a table and add the information we need. We need the date of the last year segmented day by day, the keywords, the clicks, and if you want also the impressions and the CTR.

How to set up the table

Select the following dimensions

Date and Query. You will need to click on the calendar icon and select the Date and Time – Day of the month (DD) option. This causes all the keyword information (or queries) not to be grouped together but to show averages day by day of the year.

Once that is done, select the following metrics:

Now let’s specify the date we want to contemplate. You can choose the period you want, although it is advisable to export as much information as possible (Big Data!) and then filter specific periods with Python. We filtered to see all the data from last year.

It is not necessary to sort the information by any particular variable because we are going to modify it later, but some may get an error if they do not. In our case, we sort by clicks as shown in the image.

After setting up the table as instructed, you should be seeing something similar to this (but with your data).

Note that we have over 175,000 rows of information. Depending on the amount of traffic to your site, this number could vary considerably.

Next step, we export the data. We will save the file with the name data.csv. Depending on the number of rows in your table, the download may take up to a few minutes. Patience… it is worth it!

Now the fun begins! Open your favorite code editor. In our case, for this kind of thing we are going to use Jupyter Notebook but you can easily copy and paste the code in a notepad and save it with .py extension that will work the same way.

The code for plotting, loading the data, modifying it, and plotting it is the following:

import pandas as pd
import matplotlib.pyplot as plt

keywords = ['keyword 1', 'keyword 2', 'keyword 3'] #your keywords here
rank_limit = 30 #Number of items to analyze

#The following lines do not need to be modified
df = pd.read_csv('data.csv')
df['Date'] = pd.to_datetime(df['Date'], format='%Y%m%d')
df = df[df['Query'].isin(keywords)].sort_values(by=['Date'])
df.set_index('Date', inplace=True)
df.groupby('Query')['Average Position'].plot(legend=True, figsize=(19.20,10.80))
plt.gca().invert_yaxis()
plt.title('Keyword rankings\n')
plt.ylim(top=0.75, bottom=rank_limit)
plt.show()

#Uncomment the following line to save the graphic to a file
#plt.savefig('keywords_rankings.png')

The result is going to be something like this:

But let’s not fall into the copy/paste trap… let’s analyze these lines of code to see how we can modify them to display the keywords we are interested in. Let’s analyze the script line by line.

1. It loads the pandas module. To explain it in an oversimplified way, pandas is something like Python’s “Excel”.
2. Loads the matplotlib.pyplot module that allows plotting the data
3. –
4. A list containing all the keywords that we are interested in plotting
5. Shows the number of positions seen on the y-axis of the graph. For example, if you only want to see the behavior on the first page of Google, set this value by 10
6. –
7. Everything below doesn’t need to be modified, but let’s look at it anyway
8. Load the data from the ‘data.csv’ file to a pandas data frame (it would be something like the equivalent of an Excel or Google Sheets spreadsheet)
9. Change the ‘Date’ column from number to date format, in order to plot the x-axis correctly.
10. Filter the data frame to use only the rows that contain the keywords declared in line 4
11. Set the date column as index of the data frame since we are going to plot date by date
12. Group the position values by keyword. This step is fundamental since it orders the data to be able to plot it. Think of it as preparing each line of the chart.
    1. The .plot() makes it show it as a graph instead of a table.
    2. You tell the chart to display a legend with information about what keyword each line represents.
    3. Determines the size of the chart
13. Invert the y-axis of the chart since a low number position is a good thing and when “positioning drops”, its rank increases in number
14. Set the title of the chart
15. Set the upper limit of the chart to 0.75 so that we can correctly see the lines that have rank 1. Otherwise the line of a first position would be lost at the edge of the chart.
16. The chart that has been set up so far is displayed.
17. –
18. –
19. If you uncomment this line by removing the ‘#’ character at the beginning of the line, the chart will be saved in the folder where the script is saved with the name ‘keyword_rankings.png’.

This simple script with less than 20 lines is a clear example of the potential that Python has to analyze your SEO data and be able to take clear actions accordingly. The learning curve for this sort of thing may be a bit steep at first but the reality is that there is a huge community of people documenting tutorials, strategies, useful tools, etc.

The future of business decision-making in recent years has been moving towards big data analysis and SEO is no exception. From PuntoRojo we always encourage you to do your best to grow your knowledge to take your SEO strategies to the next level.

Needless to say, any questions, comments, or suggestions are welcome in the comments of this post. As always, good rankings!

La entrada How to see the historical ranking of your keywords… for FREE! aparece primero en SEO Blog Punto Rojo.

From Punto Rojo Mexico and Argentina we analyzed the main trends that were generated in both countries.

“World Cup 2018” was one of the most used words in different countries, including Argentina and Mexico. Concerning this, sportsmen such as “Javier Mascherano” and “Kylian Mbappé” also made up the list of most searched people.

Among other events that stood out in Google Argentina, we can find “G20”, “Partido Boca River”, and “Hot Sale 2018“.

In Mexico, on the other hand, “Boda del príncipe Harry”, “Sismo” and “Buen Fin 2018” were the events that took the most searches.

From this point, we highlight the importance of actively participating in commercial events, and consider the digital strategy SEO optimizations that allow us to expand our positioning opportunities for related searches.

In Argentina “Ley del Aborto” and “Como va la votación del aborto en el Senado” were topics that not only stood out in Google but follow a trend that continues to be seen today in social networks.

With a rise in the number of searches between July 1 and 7, ” elecciones 2018 méxico” was the main trend for this country, together with those of characters such as “Andrés Manuel López Obrador” and “Margarita Zavala”.

Both in Argentina and Mexico, a large number of queries referring to characters were repeated, among them, we can highlight the artists “Stan Lee”, “Avicii” and “Freddy Mercury”.

As expected, the series also occupied a category in Google’s most searched 2018. In Argentina, the first place went to “La Casa de Papel 2” followed by “Luis Miguel” and “El Marginal 2”.

In Mexico, the podium of most-viewed TV programs was occupied by “Exatlón”, while in second and third place in searches we can find “Enamorandonos” and “Caer en la tentación”.

The top words in queries and questions asked by users (what and how) were generated around the points mentioned above, some examples of these in Argentina are ” Qué hora es en Rusia”, “Cómo se llama la mascota del mundial”, and in Mexico “Qué países han ganado la copa mundial de fútbol” and “Cómo votar”.

La entrada The most used keywords in Google Mexico and Argentina aparece primero en SEO Blog Punto Rojo.

On the one hand, keywords are the exact words with which a user searches for what he wants on the Internet.

On the other hand, keywords bring into play the desire of a company to position content, a product, or service (in short, our brand) within a specific category.

Google modificó el comportamiento del consumidor y revolucionó el marketing. Para no perder ganancias, se ha tornado indispensable comprender la importancia de las keywords, protagonistas de este fenómeno.

There are thousands of articles and videos on the subject, which are often confusing (sometimes even for experts). So let’s explain the topic without it sounding like Greek for those who are not specialists, but need to understand this process that is so relevant to online presence.

Much more than sales

Keyword selection is always made based on the marketing objectives of a particular company. Positioning for sales is only a portion of the available keyword work.

In reality, there are different types of sites other than e-commerce: corporate web pages, service pages, portfolios, blogs and content sites (and press in particular), and communities, among others.

Even within the world of e-commerce, it sometimes happens that there are big brands that, in addition to aiming to promote their online store and positioning for the sale of a product or service, need to appear in any search related to the context of that product or service and cover a greater or lesser range of influence according to specific marketing guidelines.

There are also sites of retail product brands, or brands in the car industry, technology, and insurance companies that, naturally, pursue sales objectives, but do not aim at online sales, so the use of this communication channel must satisfy other objectives.

That is, on the one hand, we can evaluate the intentions that each keyword represents; on the other hand, the focus on one or another keyword is defined according to commercial and brand interests.

Each case demands a unique strategy that exceeds the manuals and requires putting creativity into action, capitalizing on the experience, and activating the synergy of teamwork.

One more clarification. When we speak of “brand”, we allude to products and services, but also institutions and personalities. And something similar happens with the semantic breadth of the concept “sale”, which can be what in a specific case we define as “conversion”; however, a conversion can also be a specific branding action, such as achieving customer loyalty or improving a reputation.

Lastly, within the keyword world, there is an interesting division: what we call brand and non-brand keywords. The former are those that include words or groups of words that contain the name of our brand. When we use this type of keyword, we are attacking the sector of the public that already knows us. But we also have the no-brand keywords, which help us to expand our business and reach those people who do not yet know our brand and are looking for products related to us.

Here we will approach the issue from the paradigmatic case of e-commerce and, in particular, with no-brand keywords for this type of site, for the illustrative potential, but it is only a partial example of what can be obtained with a good SEO campaign. 

Separating the wheat from the chaff

We can visualize the keyword analysis as if it were a filter:

• We started with a niche: refrigerators.
• We searched for keywords within that niche: refrigerators with freezers, no frost refrigerators, refrigerators prices, refrigerators with freezer prices, and a long etcetera.
• We choose the best intentions: buy refrigerators with freezers with no frost in Buenos Aires, for example.

To carry out this filtering, there are many useful tools. One of the most popular is Keyword Planner, which allows us to work with the data provided by Google itself. It provides ideas for new keywords, the average number of searches, competition, and other data.

By identifying keywords relevant to our sales objectives, we segment the millions of available searches and target those that are most likely to lead to a purchase (or conversion).

What we do in this instance, rather than selecting specific words, is to choose intentions. If we sell refrigerators, we want to position for all keywords that have sales potential.

It is worth clarifying that a Keyword Research is not an analysis of ten words, but hundreds and hundreds.

Once we know which are the ideal keywords, that is to say, the best within a particular niche, what follows is to optimize the respective website so that it appears as high as possible in Google for any approximate search.

Once this is done, optimizing is a complex activity that involves more technical aspects. For now, it is enough to know that the selected keywords will be placed in the following strategic places within a website:

• URL: The placement of the main keyword in the URLs of all pages and products and/or services that we want to position.
• The title tag is the presentation of the products. It is important to include the keyword to improve SEO.
• H1 is the main title of the visible content. Adding the keyword will help us to attract users who are searching for the keyword and it is the title that will provide us with a greater attraction for them to continue reading the content.
• H2, H3…H5: Subtitles. We are interested in working with not-so-relevant keywords to gain positioning little by little.
• Body text: We need to add keywords in the texts, but in an orderly manner and without over-optimizing. We need to include synonyms and Long Tails since the recommendations talk about a maximum of 5% of the keyword per text.

Not all that glitters is gold

In the case of e-commerce, a good keyword satisfies these three criteria:

1. Relevance

Let’s imagine we have a tourism agency website and we want to promote an all-inclusive combo to Cancun.

We get our page to appear in the first position on Google every time someone searches for “how many people does Cancun have”. That would bring us a lot of visits, but… what good would this traffic do us?

When someone searches for “people in Cancun” it is because they want to find information about the population of Cancun, and if they arrive at a tourist agency website, what they will do is leave immediately. In other words, we will not satisfy that user’s search, we will not make him or her happy, and we will waste his or her time.

For this reason, accumulating visits does not make sense. What we are interested in is to get to our website precisely those people who are looking for what we offer: to buy an all-inclusive package to Cancun.

So, for a keyword to be good, it has to be very relevant according to the content of the page.

2. Popularity and competition

A keyword will be better the more people search for it. We are more interested in positioning ourselves for a phrase that 100,000 people per month search than for one that is searched by only ten. However, this is not all, as we will see.

To better illustrate this point, it is worth knowing that search clicks are distributed as follows: the first five search results get 73.80% of all clicks.

We can say that an interesting search term is no longer the one that offers the most traffic, but the one that brings us more consistent traffic, of higher quality, and less burdened by our competitors.

The most contested keywords are important, of course: the best restaurants are usually those where there is a line to sit down. But, beware, as in life itself, that does not mean that other restaurants where there are fewer people do not eat well, or even better than there.

3. Purchase Intent

When we analyze a keyword, we always think about the intention of the person performing a search. What do they really want, and why are they searching for it?

The best keywords are the commercial ones, that is, the ones that people who intend to buy are looking for.

Let’s leave behind the sale of all-inclusive packages to Cancun. Now we sell home appliances. Televisions, for example.

If someone googles “buy 32 inches led TV in CABA”, we know they are thinking of spending money on a TV; while if they search for “led TV screen” we do not know if they have broken the screen of their TV or if they are looking for information about the technology used in led screens for a university project, etc.

In conclusion, someone who comes to our website looking for “buy 32 inches led TV in CABA” is much more likely to end up clicking on the buy button than someone who arrives looking for a “led TV screen”.

These are some words that are usually part of commercial phrases (used to build Long Tails):

• Buy
• Combo
• Promotion
• Discount
• Offer
• Cheap
• The best
• Price

Google is trying to understand search intent. And it is increasingly refining the process of selecting what and how it displays based on what the underlying intent is.

Take the first step: get to know your customer’s desires

In an SEO campaign, there is nothing more important than keyword research, since optimizing with inappropriate keywords will simply not generate the desired traffic and we will lose money.

On the other hand, when we choose the right keywords, it is more than likely that we will appear at the top of Google. This means that, if we offer the product we have invested the work in, we ensure that the user who enters our website is a potential buyer who is looking for exactly that.

So, by being first for these searches, we will achieve a more than interesting conversion ratio, in which, even if we do not have much traffic to our website, we will be gaining many chances to win conversions. And this, in the long run, represents a higher ROI.

Similar is what happens in the case of websites that have nothing to do with e-commerce. If we make a relevant keyword optimization, we will get that user that we are interested in having in our space for particular marketing reasons. This can happen, for example, with personal pages of politicians.

In short, it is about overcoming the old principle of being where your customer is, thanks to the new possibilities offered by technology. Now we will track our customer and bring him to our home.

La entrada Keywords: improve your brand visibility and increase your conversions aparece primero en SEO Blog Punto Rojo.

There are many Keyword Rankings tools that allow you to track the positioning of the most relevant keywords for your business. Some of them are paid and others are free. If you are looking for an easy solution to get out of trouble, we developed a simple snippet that allows you to extract all the relevant ranking information for a keyword from a search result page.

Snippet:

var keyword = $(‘input[name$=”q”]’).value

var results = $$(‘div[class=”r”]’);

var date = new Date().toISOString().split(‘T’)[0];

var content = ”;

for (var i = 0; i < results.length; i++){

var rank = i+1;

var current_url = results[i].innerHTML.match(/<a href=”([^”]*)/)[1];

var current_title = results[i].innerHTML.match(/<h3.*?>(.*?)</)[1];

content += ‘<tr><td>’ + date + ‘</td><td>’ + rank + ‘</td><td>’ + keyword + ‘</td><td>’ + current_title + ‘</td><td class=”tdurl”>’ + current_url + ‘</td></tr>’;

}

with(output = ‘<html><head><link href=”https://fonts.googleapis.com/css2?family=Open+Sans:wght@400;700&display=swap” rel=”stylesheet”><style>body{font-family: \’Open Sans\’, sans-serif;}table{white-space:nowrap;}td{padding:0 15px;}.tdurl{max-width: 400px;overflow: hidden;text-overflow: ellipsis;white-space: nowrap;}</style></head><body><table width=”100%”><tr><td><h1>Google SERP Extractor</h1></td><td align=”right”><img src=”https://puntorojo.com/blog/wp-content/themes/punto_rojo/img/logo_mini.png” width=”200px”></td></tr></table><table><tbody><tr><th>Fecha</th><th>Rank</th><th>Keyword</th><td><b>Title</b></td><td><b>URL</b></td></tr>’, output += content, output += “</tbody></table></body></html>”, window.open()) document.write(output), document.close()

In order to run this snippet correctly, it is important to remember that we have to be inspecting the Google SERP we want to analyze. If we load the Google Chrome DevTools (read, “the inspector”) in another tab, the code will not work.

 As in previous posts, to configure this SEO snippet we have to open the Chrome DevTools and go to the Sources tab. Inside the Sources tab we then go to the Snippets tab. It is likely that you will not see the Snippets tab at once and you will have to click on the two arrows icon to see all the tabs.

Once we are in Sources > Snippets, we go to “+ New Snippet”.

We name the snippet, paste the code above this post and then press Ctrl + S or ⌘ + S on Mac, and that’s it!

To run the snippet you have to right click on the script name and click “Run”. When you do this, a new tab should open with the following columns…

The result should be something similar to this…

The columns listed in the keyword rankings report are:

• Date: Shows the date on which the script was run.
• Rank: Shows the position in which the result appears on Google
• Keyword: Displays the keyword for which the positioning is being viewed
• Title: Displays the title of the page that ranks for that position
• URL: Shows the URL that ranks for that position

In the case of the screenshot, 100 rankings are shown because Google Search is configured to show 100 results at a time. If you need to see more than 100 results, this can be achieved by using the gInfinity extension for Google Chrome.

This data can be copied and pasted directly into a Google Sheet or Excel spreadsheet. Once the file is ready, you can start saving the ranking data to see daily evolutions of both your site and your competitor’s site. If you just need to see how your site is ranking, you can see the historical data of your keywords up to 18 months ago.

We hope this SEO Snippet will be useful for your daily optimization routines and that you will be able to carry out a data-driven SEO strategy every time.

Until next time and, as always, good rankings!

La entrada How to extract data from SERPs for SEO aparece primero en SEO Blog Punto Rojo.

These words show us which are the intentions that a person has when performing a search and that is why it is so important to obtain positioning in Google for them.

Now, the selection of keywords should always be oriented to the marketing objectives of the company, and at the same time, be related to the user’s interests, because from these will emerge the searches made on Google.

But how do you know which keywords users are searching for?

To answer this question, we will resort to one of the most popular tools in the digital environment and widely used in SEO: Google Adwords Keyword Planner.

This platform provides us with data directly from Google, remember that in Mexico 98% of the population chooses to use this search engine.

Within the information provided by the keyword planner, we can obtain keywords used by users and the number of monthly searches.

Let’s see an example:

Let’s suppose you have an appliance store, and therefore a variety of products to sell. In the tool we enter as a keyword a product category, such as “refrigerator“, and activate the parameters we consider necessary for the search:

As a result, the tool will tell us which are keywords related to the search we entered, the average number of monthly searches in Mexico, and other data that may be useful.

Adwords Keyword Planner will give us hundreds of recommendations on keywords related to “refrigerator”. We suggest downloading this information to be able to filter the results based on business objectives. For example, there are brands and models of refrigerators that we may not have available in our store and may not be targeted at the moment.

Having understood this, we can say that we have generated our first keyword research!

Keyword research allows us to know what and how people search, and how many times a month this search is replicated. This information will be very valuable if we incorporate it into our site.

We must always keep in mind that each company will have a particular strategy in which creativity, experience, and teamwork synergy intervene.

Knowing now our users’ search intentions, the next step is to optimize our website, strategically using the keywords we have selected. For this, we must incorporate the keywords in our site by applying them to product names, descriptions, URLs, page titles, etc.

What are short and long-tail keywords?

Short tail keywords are keywords of 1 or 2 terms maximum, such as “refrigerator”. These usually have a large number of monthly searches but with low conversion levels. This happens because when a person searches for a product generically, they are probably still in a research stage and not in a purchase stage.

Long-tail keywords, on the other hand, are a very faithful representation of people’s tastes and interests. They are composed of more than 3 terms, making the keyword a more specific search.

For example, “Prices of 29 feet refrigerator in DF” is a long-tail keyword, composed of 7 words inside, and with not many monthly searches. But, a user who performs it is interested in the product, and is much closer to the purchase, than a user who only searches for “refrigerator”.

With the use of long-tail words, it is more likely that the majority of visitors who enter our website will become customers. The user that is determined to make a purchase tends to specify more and more his search, to find his perfect product!

 What is stuffing? How to avoid it?

The term stuffing, which means overloading our article or web page with an excessive amount of keywords, is an activity penalized by search engines.

We will lose positions in the search results or Google will remove our site results or directly remove our site from its ranking.

Let’s not forget that it is Google who will determine which companies will be positioned on the first page, and which will be the star that takes the first place.

 So what should we do with keywords?

Place only the necessary amount of keywords throughout the text and do not saturate it by using them in places where they do not fulfill any function. In other words, we must avoid artificial wording, and avoid making the content of our website look like it was written by a robot.

We can place our long-tail keyword in the title and then look for variations, long and short tail, throughout the text, always taking into account the keyword research we did at the beginning.

 First in Google

When we choose the right keywords, we are more likely to appear in the first results of Google.

Thus, we will ensure that the user entering our website is a potential buyer who is looking for exactly that.

By being the first in Google for long-tail keywords, we will achieve a higher conversion rate.

La entrada Keywords: How do you search for products on Google? aparece primero en SEO Blog Punto Rojo.

Now, to know how relevant the content of a site is to the query made by the user, the term TF-IDF (term frequency-inverse document frequency) arises, a first cousin of what was known until some time ago in the old school of SEO as keyword density.

The use of this last concept started to decrease due to opposing positions on the idea that the more times a keyword is repeated on a page, the greater its relevance.

In this context, and within the machine learning framework, TF-IDF arises. It determines a numeric value that defines the importance of a particular keyword within an entire site. Knowing this is important because if, for instance, we have a coffee shop site and the keyword “coffee” has a low TF-IDF, the term is not being optimized correctly for SEO.

To better understand the concept, let’s take an example. A website is composed of two pages:

-el café espresso (page 1)

-el café espresso americano (page 2)

The first step is to analyze how many times each word is repeated in the content of the pages I am analyzing:

We see then that all words are repeated 2 times, except for the keyword “americano”. This leads us to interpret that the first three words are the ones with the highest term frequency (TF), while “americano” is the one with the lowest repetition within the site, corresponding to the inverse document frequency (IDF).

It is impossible to determine exactly the right percentage for content to rank for a specific search.

On the other hand, we must not forget that texts must always be written with the user in mind, and not taking the search engine as the main objective, so repeating relevant terms will be favorable, as long as we mention them in necessary contexts, without abusing their use, as this may be counterproductive.

How to determine the frequency of the term

If you are interested in knowing a little more in-depth about how the value we mentioned before is determined, then continue reading this section :)

Let’s take the example we saw before. To know the term frequency (TF) we must do the following calculation for each word inside each document:

Now let’s replace it with one of the words: the term frequency of the word “the” within document 1 is equal to how many times the term is repeated, over the total number of words in document 1.

This determines that the TF for the word “the” within document 1 is 0.33.

The next step is to do the same for the word “the” within document 2. If we replace it in the formula we would be left with something like this, and the value of TF, in this case, would be 0.25.

Now we have to calculate the value of the inverse document frequency (IDF), what we have to do is to replace the data we have, in the following formula:

This means that IDF is equal to the logarithm of the total number of documents divided by the sum of all the times a term is repeated in the document.

If we pass it to the example we have already seen, the value of IDF would be 0 (zero) and we would be left with something like this:

To conclude and determine our TF-IDF value we must know that:

Therefore, we replace with the values we calculated for the word “the” within documents 1 and document 2, and we would arrive at the TF-IDF for both being zero.

La entrada TF-IDF | Term Frequency – Inverse Document Frequency aparece primero en SEO Blog Punto Rojo.