One of the main reasons Magento is fast becoming the e-commerce platform of choice is the fact that, from the ground-up, it has been built with the foreknowledge and flexibility required to optimize every page, every product, and every snippet of code within its framework for search engines. That is assuming you know where to look and how to do it.
There are many similarities between Magento Community Edition and Magento Enterprise Edition, but also a few major differences. Wherever possible, I will try to highlight some of the features that may appear in one or the other of these platforms, and also reference in which version certain features were added or removed.
For the purpose of this guide, we will be using Magento Community Edition 18.104.22.168. As of the time of this writing, it is the latest stable release of the free edition. This should allow store owners, Magento developers, and SEO experts and novices alike access to all of the features contained within this book.
In this chapter, we will cover the following:
Understanding the structure of a website, the purpose of optimizing for e-commerce, and the relationship between keywords and their position on a website
Understanding the buying intent of our visitors and how this intent may differ depending on the type of page by which they enter our website
The roles of content management system (CMS) pages and their uses in search engine optimization (SEO)
The importance of the
metatags and Magento's default
How to change your category structure to benefit your SEO campaign
How to set up an XML sitemap and Google Analytics E-commerce tracking
An entire book could be written on keyword distribution for e-commerce websites; however, as the aim of this book is to cover the main aspects of optimizing a Magento store, we cannot go into too much depth. Instead, we'll focus on three major considerations when choosing where to place our keywords within a Magento store:
Purpose: What is the purpose of optimizing this keyword?
Relevance: Is the keyword relevant to the page we have chosen to optimize it for?
Structure: Does the structure of the website re-enforce the nature of our keyword?
The purpose for choosing keywords to optimize on our Magento store must always be to increase our sales. It is true that (generically speaking) optimizing keywords means driving visitors to our website, but in the case of an e-commerce website, the end goal—the true justification of any SEO campaign—must be increasing the number of sales. We must then make sure that our visitors not just visit our website, but visit with the intention of buying something.
The keywords we have chosen to optimize must be relevant to the page we are optimizing them on. The page, therefore, must contain elements specifically related to our keyword, and any unrelated material must be kept to a minimum. Driving potential customers to a page where their search term is unrelated to the content not only frustrates the visitor, but also lessens their desire to purchase from our website.
The structure of our website must complement our chosen keyword. Competitive phrases, usually broader phrases with the highest search volume, are naturally the hardest to optimize. These types of keywords require a strong page to effectively optimize them. In most cases, the strength of a page is related to its level or tier within the URL.
For example, the home page is normally seen as being the strongest page suitable for high search volume broad phrases followed by a tiered structure of categories, subcategories, and finally, product pages, as this diagram illustrates:
With that said, we must be mindful of all three considerations when matching our keywords to our pages. As the following diagram shows, the relationship between these three elements is vital for ensuring not only that our keyword resides on a page with enough strength to enable it to perform, but also that it has enough relevance to retain our user intent at the same time as adhering to our overall purpose:
You may be forgiven for thinking that optimizing our most competitive keyword on the home page would lead to the best results. However, when we take into account the relevance of our home page, does it really match our keyword? The answer is usually that it doesn't.
In most cases, the home page should be used exclusively as a platform for building our brand identity. Our brand identity is the face of our business and is how customers will remember us long after they've purchased our goods and exited our website.
In rare cases, we could optimize keywords on our home page that directly match our brand; for example, if our company name is "Wooden Furniture Co.", it might be acceptable to optimize for "Wooden Furniture" on our home page. It would also be acceptable if we were selling a single item on a single-page e-commerce website.
The buying intention of our visitors will almost certainly differ between each of these types of pages. Typically, a user entering our website via a broad phrase will have less of an intention to buy our products than a visitor entering our website through a more specific, product-related search term.
Normally, our most competitive keywords will be classified as broad keywords, meaning that their relevance could be attributed to a variety of similar terms. This is why it makes sense to use top-level or parent categories as a basis for our broad phrases.
To use our example, Wooden Furniture would be an ideal top-level category to contain subcategories such as 'Wooden Tables', 'Wooden Chairs', and 'Wooden Wardrobes', with content on our top-level category page to highlight these subcategories. On the Magento administration panel, go to Catalog | Manage Categories. Here, we can arrange our category structure to match our keyword relevance and broadness.
In an ideal world, we would plan out our category structure before implementing it; sadly, that is not always the case. If we need to change our category structure to better match our SEO strategy, Magento provides a simple way to alter our category hierarchy.
For example, say we currently have a top-level category called Furniture, and within this category, we have Wooden Furniture, and we decide that we're only optimizing for Wooden Furniture; we can use Magento's drag-and-drop functionality to move Wooden Furniture to become a top-level category.
To do this, we would have to perform the following steps:
Navigate to Catalog | Manage Categories.
Drag our Wooden Furniture category to the same level as Furniture.
We will see that our URL has now changed from
We will also notice that our old URL now redirects to our new URL; this is due to Magento's inbuilt URL Rewrite System. When moving our categories within the hierarchy, Magento will remember the old URL path that was specified and automatically create a redirect to the new location.
This is fantastic for our SEO strategy as 301 redirects are vital for passing on authority from the old page to the new.
A 301 redirect is one of the most useful tools in maintaining a search engine's understanding of our website pages. More information on their importance and how to set up 301 redirects is provided in Chapter 7, Technical Rewrites for Search Engines.
If we wanted to have a look at these rewrites ourselves, we could perform the following steps:
Navigate to Catalog | URL Rewrite Management.
From the table, we could find our old request path and see the new target path that has been assigned.
Not only does Magento keep track of our last URL, but any previous URLs also become rewritten. It is therefore not surprising that a large Magento store with numerous products and categories could have thousands upon thousands of rows within this table, especially when each URL is rewritten on a per-store basis.
There are many configuration options within Magento that allow us to decide how and what Magento rewrites for us automatically, and these will be covered within Chapter 7, Technical Rewrites for Search Engines.
Another important point to note is that your category URL key may change depending on whether an existing category with the same URL key at the same level had existed previously in the system. If this situation occurs, an automatic incremental integer is appended to the URL key, for example,
Magento Enterprise Edition has been enhanced to only allow unique URL keys. To know more, go to goo.gl/CKprNB.
By default, the home page of a Magento store is a CMS page with the title Home Page. The page that is served as the home page can be configured within the Magento Configuration under System | Configuration | Web | Default Pages.
The most important part of a CMS page setup is that its URL key is always relative to the website's base URL. This means that when creating CMS pages, you can manually choose how deep you wish the page to exist on the site. This gives us the ability to create as many nested CMS pages as we like.
Another important point to note is that, by default, CMS pages have no file extension (URL suffix) as opposed to the category and product URLs where we can specify which extension to use (if any).
For CMS pages, the default optimization methods that are available to us are found within the Page Information tabs after selecting a CMS page:
Under the Page Information subtab, we can choose our Page Title and URL key
Under the Content subtab, we can enter our Content Heading (by default, this gets inserted into an
<h1>tag) and enter our body content
Under the Meta Data subtab, we can specify our keywords and description
As mentioned previously, we would focus optimization on these pages purely for the intent of our users. If we were not using custom blocks or other methods to display product information, we would not optimize these information pages for keywords relating to purchasing a product.
If we take a look at a normal
Search Engine Results Page (SERP)—in this case Google—obtained using the search query
Magento open source download, we will see that two of these elements are used directly in the result listings (Title and Meta Description):
The code used to display the preceding result is as follows:
<title>Open Source Ecommerce Software & Solutions | Magento</title> <meta name="description" content="Download the Magento Community Edition, our free open source ecommerce software solution for expert developers and enthusiasts!" />
Downloading the example code
You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
Depending on the search query used, Google may return the meta description, or alternatively, extract a snippet of text from the content that it believes to best represent that page for the given query. The same applies to
title tags; when a more specific query is used, the default title tag may be changed to better suit the query.
For instance, if we perform a search for
magento download on Google, instead of the preceding search, we will receive the same page and the same meta description, but a more targeted
title tag—try it!
The meta description that is present on the page is:
<meta name="description" content="Ottoman" />
From this, we can see that Google has chosen to return the extracted snippet from the product description, most likely due to the meta description being entirely inadequate for the search term.
The best practices for these elements are well documented in SEO circles, and the following is a brief breakdown of each:
titletag should be kept (ideally) to a maximum of 70 to 75 characters. Any title tag longer than this will be truncated in the SERPs to only those first 70 to 75 characters. If we are trying to engage a human being's interest, we need to be able to do that within those first 70 to 75 characters.
Meta Description: The same rules apply to the Meta Description field; Google usually only shows approximately 150-160 characters of meta information below the
titletag in the SERPs. A description longer than this may be truncated and the surplus will not be presented to the user.
Meta Keywords: It has long been acknowledged that most major search engines choose to ignore the meta keyword tag completely, mainly due to keyword stuffing, which was a common practice in past years. Usually, it is more beneficial to remove this tag altogether than to spend time tailoring keywords for individual pages (please see Chapter 8, Purpose-built Magento Extensions for SEO/CRO, for ways in which we can remove this tag).
Content: This is a contentious subject; however, the accepted guidelines are that 400+ words of unique, quality, and relevant content per page should stand you in good stead with all the major search engines. There are also reports of 2000+ words of quality content providing the best results and resulting in the best rankings. At this time, it is unknown if there is an upper-limit to the amount of content deemed acceptable on any given page. The truth is that it becomes harder to write relevant, quality content the longer that content becomes.
A really quick and easy adjustment we can make to our entire website URL structure is to remove the default
index.php string that is appended to the base URL—for example,
http://www.mydomain.com/index.php/my-product.html—and also auto-redirect to our base URL if the non-www version of our domain is entered.
In order to achieve these small fixes, we should perform the following steps:
Navigate to System | Configuration | Web | Search Engines Optimization and set Use Web Server Rewrites to Yes to remove
Within this same section but under URL Options, set Auto-redirect to Base URL to Yes (301 Moved Permanently).
As long as our server has been configured correctly and our default
.htaccess file is in place, Magento will automatically remove the
index.php string from our URLs and continue to serve our pages as normal. By performing the preceding steps we will also be setting up an automatic SEO-friendly redirect for non-www versions of our URLs.
Unfortunately, by default, all pages will now be accessible with and without URLs containing
index.php. In order to resolve this, we can use canonical tags—canonical tags do not contain
index.php within the URL once this change has been made.
As mentioned previously, Magento comes equipped with the ability to rewrite URLs for category and product pages. Depending on the configuration we have decided upon, Magento will create entries within its
core_url_rewrite table in the database to accommodate changes to the URL key of a category, a product, or the categorization of a product within our store.
If we take a quick look at this section, we will notice that there are many options, all of which require explanation:
Autogenerated Site Map: If enabled, this creates two pages on our Magento website that display links to our categories and products:
Popular Search Terms: This enables a page that displays your most popular search terms. This is not hugely relevant for SEO and should only be used as a tool aimed at your visitors rather than for search engines.
Use Categories Path for Product URLs: With this enabled, Magento will include the category URL key within the URL structure for our product pages (for example, www.mymagento.com/category-url/product.html). When used in conjunction with Use Canonical Link Meta Tag for Products, this setting may impact our link building capabilities for our cached product pages.
External links built for our category-level product URLs are not automatically 301 redirected to our new canonical product URL. This can be remedied by following the Creare SEO by CreareGroup section in Chapter 8, Purpose-built Magento Extensions for SEO/CRO.
Use Canonical Link Meta Tag for Categories: With this setting enabled, all categories will contain a new tag within their HTML instructing search engines where the primary version of the current category page can be found.
Duplicate content is the main worry when it comes to e-commerce websites. The ability to double-categorize products is both a blessing and a curse; a blessing for users who may look for a certain product under two separate categories, but a curse for search engines, which will find identical content on two separate URLs.
It is important to note that research suggests major search engines such as Google and Bing can detect when a website is classified as an e-commerce site and can make allowances for duplicate product pages.
However, it is always a good practice to ensure that, whenever possible, all steps have been taken to make this task of deducing the primary page as easy as possible for search engines.
The canonical HTML tag has been given to us for this specific purpose. It is used to point search engines to a specific URL—the URL that we want our search engine to display in the SERPs.
Here is an example of a canonical tag:
<link rel="canonical" href="http://www.mydomain.com/ottoman.html" />
For categories, the canonical tag is used to determine the static URL of that category. That is to say that when filters or pagination features are activated, the parameters appended to the URL should not be cached as separate duplicates of our category page.
In order to set up our Magento store for best optimization practice, we would want to set our configuration as follows:
Navigate to System | Configuration | Catalog | Search Engine Optimizations.
Set Use Categories Path for Product URLs to No.
Set Use Canonical Link Meta Tag for Categories and Use Canonical Link Meta Tag for Products to Yes and then click on Save Config.
Navigate to System | Index Management next to Catalog URL Rewrites and click on Reindex Data.
When we set Use Categories Path for Product URLs to No, we were telling Magento to serve our canonical product link to our users when they browse our categories. We could have set this to No even if we hadn't enabled our canonical tag for products; however, product URLs will always exist in the same two places. For example:
Ideally, we want to restrict both our users and search engines to only one version of our product page.
Restricting our users to the single URLs (without categories) will allow us to maintain consistent link-building equity for each page. Directing search engines via the canonical tag will help resolve duplicate listings in the SERPs and any algorithmic penalties attributed to duplicate content violations.
As Magento is such a large system, every element cannot be expected to be manually entered before being used in the system. That is why Magento has a fall-back process in place for a lot of configurable options. These can be found within System | Configuration | Design | HTML Head.
For SEO purposes, the main default elements we are interested in are:
These defaults are intended to display content to both users and search engines wherever we have failed to populate the relevant field in our administration panel—primarily for CMS pages and categories.
In many instances, Default Title will only be used when a custom development has been made and a title is not specified within the Layout XML, the PHP controller file, or some form of admin configuration. Default Description, however, will be used whenever Meta Description is left unpopulated on a category or CMS page.
Duplicate meta description and
title tags are extremely bad for usability. For any page that we wish to perform well in search engines, we must ensure that we have a unique meta description, and if possible, a unique title. This allows search engines to better discern individual pages and also makes it easier for users who are searching for our content to instantly find the correct page among multiple results.
For products, Magento handles the default meta description and keywords tags differently; they are usually prepopulated with the following information:
Meta Title: If this is left unpopulated, the product name will be used
Meta Description: If this is left unpopulated, the product description will be used
Meta Keywords: If this is left unpopulated, the product name will be used
As our products should all be unique, this inbuilt system is a useful tool for large Magento websites that are set live without all product meta information being initially entered.
As we know, Magento should only serve up these default attributes if we have failed in some way to enter the information ourselves. To maintain a good standard of usability for these types of situations, it is a best practice to populate these fields with relevant data (this data can also be modified on a per-store basis):
Navigate to System | Configuration | Design | HTML Head and populate the following:
An example of this could be:
Finally, within our HTML Head section is a small but extremely important option called Default Robots.
When a Magento website is under development on a test URL, we would normally find the value of this select field to be NOINDEX, NOFOLLOW, essentially blocking search engine spiders from accessing any page in the Magento system. It is therefore of paramount importance that, once the website is launched on the live domain, this select option is set to INDEX, FOLLOW, or our Magento website may never be indexed by search engines!
A quick reference for each of the available four options is as follows:
As well as the
<meta name="robots" /> tag, it is highly encouraged to back up any robot-specific queries with a
robots.txt file. We will talk more about the
robots.txt file in Chapter 7, Technical Rewrites for Search Engines.
We all know that search engines can identify pages on a website via its internal (and external) linking structure. However, the most comprehensive and accessible method of providing our website structure to search engines by far is via a valid XML sitemap that is uploaded to a search engine's Webmaster Tools (for example, Google Webmaster Tools).
Naturally, the development team behind Magento realized that manually creating an XML sitemap from all the ever-changing pages, products, and category URLs would be an impossible task. Therefore, they developed their very own XML Sitemap Generator.
In order to generate our XML sitemap, we must first configure its contents. Go to System | Configuration | Google Sitemap and configure Frequency and Priority for our main page types. We can also configure how often we want to generate our sitemap.
Depending on our Magento store, we may decide that our categories are the most important pages. They're our most optimized pages and we want search engines to index them first. Our next most important pages would be our individual product pages; we want those to appear in search engines for customers searching specifically for our product names. The page type with the least priority would normally be our CMS pages.
As mentioned previously, the home page in Magento is classified as a CMS page; therefore, based on our specifications, it will receive a lower priority. In addition, the home page URL or
<loc> will be set as
http://www.mydomain.com/home, which is not how we want our home page to appear.
These are both problems that can be overcome via Magento extensions mentioned in Chapter 8, Purpose-built Magento Extensions for SEO/CRO.
The priority is simply a value that is passed to Google in order for it to prioritize the list of pages it will index; it will then (supposedly) do so programmatically.
Based upon our chosen SEO campaign, we would set the priority higher for those pages we are optimizing. Therefore, if we are optimizing our categories and products more than CMS pages (recommended) we would set their priorities to match the following:
Within Categories Options, set Frequency to Daily and Priority to
Within Products Options, set Frequency to Daily and Priority to
0.8(or anything less than 1 and more than we are about to set the CMS pages to).
Within CMS Pages Options, set Frequency to Weekly and Priority to
Within Generation Settings, set Enabled to Yes, Start Time to 01 00 00 (01:00 a.m.), and Frequency to Daily, and enter your e-mail address into the Error Email Recipient field.
Click on Save Config.
In the end, we should have something that looks like this:
In order for our generation settings to automatically generate our sitemap, the Magento CRON must be enabled. A quick tutorial on how to do this can be found here: goo.gl/q3ngaJ.
Navigate to Catalog | Google Sitemap and click on Add Sitemap.
For Filename, enter
For Path, we can specify a path, but we would usually place an XML sitemap on the root of our website (enter
If we have multiple store views, we can enter a specific sitemap for each Store View (in which case we would change our filename to suit the convention, for example,
Click on Save & Generate.
This should generate an XML sitemap in our chosen path with our chosen filename. We can test this by visiting our
path/filename in the URL, for example,
If an error message appears informing you that the specified directory is not writable, please make sure that the folder specified under Path has sufficient privileges to allow the system to write a file—usually 775, or failing that, 777.
There are two ways to do this, but for safety's sake, we would usually perform both:
robots.txtfile and add in
Log in to Google Webmaster Tools (www.google.com/webmasters/tools/), click on our website (or add our site if we need to create one), and then, within Crawl, click on Sitemaps and Add/Test Sitemap.
There are many analytics packages out there, but the most popular by far is Google Analytics (http://www.google.com/analytics/).
In order to set up Analytics effectively on our Magento store, we will need to enter our Google Analytics Account Number (tracking ID) in the administration panel. It is also recommended that we activate e-commerce tracking from within our Google Analytics account.
In order to do this, perform the following steps:
Log in to our Google Analytics account and navigate to our particular website's account page.
Click on Admin.
Within this section, we should see three columns: Account, Property, and View. If we wish to find out our Tracking ID, click on Property Settings under the Property column; we can then copy/paste our Tracking ID from here.
Now that we have our tracking ID and have enabled E-commerce tracking in Google Analytics, we should navigate to our Magento administration panel and then go to System | Configuration | Google API | Google Analytics.
Paste in our tracking ID into the Account Number field, set Enable to Yes, and then click on Save Config.
Now that we have set up analytics, we will find a whole plethora of information is now available to us, including the ability to track revenue by source and to work out our most effective conversion paths. We'll look at more advanced tracking methods in Chapter 6, Analyzing and Tracking Your Visitors.
<body> tag, we should find code similar to the following:
<!-- BEGIN GOOGLE ANALYTICS CODEs --> // our <script> tag containing our tracking code here <!-- END GOOGLE ANALYTICS CODE -->
If we do not see this code, it could be that the template file has been edited and the tag that includes our Google Analytics code has been removed. If this is the case, we should double-check our standard template files (
3columns.phtml, usually found within
app/design/frontend/[package]/[theme]/template/page) to make sure that these two snippets of code are in place:
<?php echo $this->getChildHtml('after_body_start') ?>
<?php echo $this->getChildHtml('before_body_end') ?>
In this chapter, we learned how to effectively distribute our keywords across our Magento store, how we should optimize our different page types, and the pivotal role keyword relevance has on SEO.
We have also scratched the surface of default optimization techniques such as using
title tags and
meta tags and saw how the content of a website can affect search engine rankings. We'll be exploring these in greater detail in later chapters.
We have configured our Magento store to best protect our website against duplicate content issues using the canonical tag, ensuring that our product pages are restricted to a single URL.
Our website is now accessible to search engines and our XML sitemap has been generated and optimized for our content. Google Analytics has also been successfully integrated to better enable us to track and measure how well our website converts visits into sales.
In the next chapter, we will be looking at how to optimize our category and product pages in greater detail.