Search Engine Optimization (34 Articles)

How to Stop Crawler and Ghost Spam in Google Analytics

One of the most useful Google Analytics features helps webmasters track the referral URLs, from which people land on their website. Unfortunately spammers utilize this possibility to promote their own websites or damage the image of unrelated companies by inserting their domain names as referrer URLs in your Analytics data. There are two main types of spam that can utilize this functionality - Ghost spam and Crawler spam. What is Ghost Spam and How to Filter it Out Spammers that use this method don't actually visit your site. Instead, they use the Analytics Measurement Protocol which allows people to send data directly to Google Analytics and add malicious information through it. Usually, they randomly generate Analytics tracking codes (UA-XXXXX-1). This way they leave fake data in your account without any trace. Since ghost spammers don't know to which domain they are sending data to Google Analytics, they are either using their hostname, that has nothing to do with your site, or they don't specifying a domain name at all. You can easily get a list of the valid hostnames that should be sending information to your account. Usually, that would be only yourdomain.com plus all subdomains you have like blog.yourdomain.com or even www.yourdomain.com. To get this information, go to Google Analytics -> Network and select the Hostname option as Primary Dimension . Then, you need to create a list of all your hostnames using Regular Expression like this: [crayon-58d620a88dfeb937782376/] Add all hosthames that actually belong to you into that list and store them in an empty text file. We will need them in a moment. Next, click on the Admin link on top of the site, select your account, the respective property and view for the site in question, and click on the Filters -> Add Filter menu. This will allow you to create a new filter. Set a name for it, like "Ghost spam filter", then choose Custom filter type. Now select Hostname from the filter field dropdown and click on the Include radio button. Finally, add the list of valid domains in the Filter Pattern field. It's good to click on the Verify this filter link before activating it. Once you do that, you will see the information that will be removed from your Analytics data. Finally, click on the Save button and you're done! How to Stop Crawlers Spam Stopping crawlers spam is easier because they actually access your site. You can get their hostnames the same way you got your valid hostnames. This time, get all the suspicious ones that have strange / unexpected Source and Hostname different than yours. Then, you can block them by using the following .htaccess rule: [crayon-58d620a88dffc293325453/] Note, that .htaccess files are sensible and you have to escape the dot character with a slash. For example, if you want to block spam-bot-site.net, you need to add spam-bot-site\.net in the…

How to decrease the crawl rate of Bing’s search engine bot

The MSN bot or BING bot crawls your website to index your website’s content and show the results under the Bing search engine. Bing supports the directives of the Robots Exclusion Protocol (REP) which can be listed in a site’s robots.txt file. The crawl rate of this bot can be controlled using this robots.txt file. In order to decrease the crawl rate you should add the following content inside the file: User-agent: msnbot Crawl-delay: 1 Bing recommends using the lowest possible value for the crawl-delay. Here is a list of values which you can use on your website: No crawl delay set – Normal 1 – Slow 5 – Very Slow 10 – Extremely Slow An alternative method of managing the Bing crawl rate for your website is to register for Bing’s webmaster tools and set up your settings…

How to remove a website from Google

There are several options to prevent your web site from being indexed by Google. For best results it is highly recommended to use all of them. Otherwise your web site might still appear in search results. The first is using a robots.txt file. Basically what you need to do is create a file named robots.txt with the following content: User-agent: * Disallow: / Then place this file in the web root for your domain name. Such robots.txt file should prevent all web crawlers from indexing your web site. Note however, that in this case Google will still index your pages that are linked from other web sites. The second option is to set a " noindex " meta tag for your pages. If the meta tag you have for the page is " noindex ", Google will completely drop the page from the search results. If the page/website is currently indexed, it should be removed the next time it is crawled by Google. The third option is to use Google Webmaster Tools. Here are step-by-step instructions how to use them: 1. You will need to create an account with Google Webmaster Tools . 2. Log in to your Google Webmaster Tools account. 3. If the account is new, you will have to add your web site. 1. Click the " Add a site " button. 2. Enter your web site URL and click " Continue " 4. Now you need to access the web site removal tool . 5. Once you have opened the web site removal tool, click on " New Removal Request ". 6. On the ' Create a new request ' page, select the options that match your removal request, fill in the required fields and then click the Submit Request …

What should I do considering search engine ranking if I need to change my primary domain name?

Changing the primary domain name for your website and preserving the search engine ranking can be a tricky task. First and foremost, you should keep both domain names up and running. The safest way to start is by parking the old domain name to the new one once you have performed the domain name change. You can check our parked domain names tutorial for detailed instructions how to park your domain name via cPanel. However, the odds are that search engines will continue indexing your old domain name. Thus the next step is to create a wild card 301 (moved permanently) redirect from your old domain name to the new one. You can check our article how to create 301 redirects via cPanel for more information how to accomplish this. The 301 redirect will tell search engines to look for the website pages at the new domain name. You should, however, bear in mind that it might take time for the search engines to start returning search results for the new domain name. It is possible to get results from the old domain as well as results from both the old and the new domain names. Considering you have an appropriate 301 redirect set, it should not be too long before all search results are from your new domain name.…

If I switch hosts, will it affect my search engine ranking?

Switching web hosts will not affect your web site ranking in any way. There is only one thing that you should take into consideration. When you change your domain's DNS by pointing it from one server to another, there is a propagation period before the new changes take effect. You should not bring the old website down until you are 100% sure that search engine crawlers are already indexing your website on the new server. A safe course of action would be to: 1. Migrate your web site along with all the content and data to the new server. 2. Change the primary domain name's DNS to point to the new server. 3. Observe the new server log for visits from search engine crawler bots. 4. When you notice that the search engine crawlers are indexing the web site on the new server, it is safe to bring the old one down.…

Do I need a blog to increase the search engine ranking for my web site?

Blogs are not in any way different from a standard website considering search engine rankings. The good thing about having a blog is that you can regularly update it with new content and link it to your web site. New and unique content is what matters for search engines and it is possibly the easiest way to add such content via a…

Does it matter if my pages are in subfolders of my domain name?

It makes no difference whether the actual page is located directly under the web root for your website. It can be placed several directories deeper on the server without any problems. The important thing here is how many steps it will take for the search engine crawlers to reach this page. The easier it is to get to a certain page via links on your website, the more important this page will be considered from visitors and search engines. Naturally, pages that are linked on your home page will be given more weight by the search…

How can I have my site linked by other websites over the web?

The best way to gain more links to your web site is creating unique and helpful content that lots of people will be interested in. This will urge visitors to post links to your web site and naturally generate more traffic. Here is a tip you can use to improve website linking. Make sure you are constantly updating your web site content with new articles and/or announcements. For example, run a blog on your web site. You can frequently update it with posts related to your content or possibly the services you…

I have submitted my web site to the search engines but it is not being ranked. What to do?

In order to have your web site listed in search engines, you will need to have it optimized for the specific keywords. Also, you will need links to it from other web sites. Basically those are the requirements to have your web site displayed in search engine results. Note that generally you do not have to submit your website to search engines. They will automatically find new pages anyway, as long as other resources point to…

Can I use ? and = (query strings) and will search engines index my site?

Having a query string in the URL will not prevent your web site from being properly indexed. However there are a few things that need to be taken into consideration: Such URLs are most commonly created after a search or specific request is performed through your website. That means that such a page actually does not exist on the server. Visitors can get to it but it won't be accessed by the search engine crawlers. The resolution is to create links for the specific pages that are likely to be visited and include them in your website navigation (menus for example) and in the site map. Another important thing is that unique title tags are generated for each page. You should make sure that the title tag of the page varies depending on the parameters passed to the query sting. Otherwise duplicate title tags will severely affect your search engine…

Why do search engines rank pages with bad content higher?

Content is the main thing people visiting your web site are looking for. This is why content is one of the main reasons why others might want to put links to your website and thus generate a lot of  traffic. However, this does not necessarily mean that you will be ranked first because you have the best content. This is especially true for common keywords. We can take the word “sportswear” as an example – major sportswear brands will be the first search results just because so many other websites with relevant content are linking to…

Which are the most important SEO HTML tags?

The most important SEO HTML tags are (in order of importance): Title Tag <head> <title>keyword</title> </head> Header Tags – H1,H2,H3 <h1>Most Important</h1> <h2>Second Most Important</h2> <h3>Third Most Important</h3> Bold,Strong <b>keyword</b> <strong>keyword</strong> Image <img src=”keyword.jpg” alt=”keyword” /> Hyperlink <a href=”http://www.website.com/page.html” title=”keyword”>keyword</a> Hyperlink (No Followed) <a href=”http://www.website.com/page.html” title=”keyword” rel=”nofollow”>keyword</a>…

General rules for creating a search engine friendly website?

Make sure the keywords you want to be ranked for are included in your web site content. Try to have other websites linking to you in order to improve the search results and increase the page ranking for your web site. Make sure your website content is relevant to the keywords. Bear in mind that Flash websites can't be read by search engine crawlers. You should avoid using Java Script navigation or use it as an alternative method. Do not use Black Hat techniques to increase your ranking and fool search engine crawlers and website visitors. …

How to increase my Page Rank?

Page Rank is a method used by Google Search Engine to rank web pages. A higher page rank means a better position in the search engine results for your web pages.   Basically, page rank defines the quality the visitors get from the page. There is a large array of elements which is responsible for the web pages' quality weight. Some of the important elements are the domain names and URLs, the keywords, the meta data, the links structure, the page structure, the content, the usability, the accessibility and many more. The higher search engine rank will attract more visitors and generate more traffic for your web site. The Page Rank algorithm is known only to the search engine development teams, but there are different hints which you might follow in order to increase the page rank of your web site: 1. Location - the term is focused on the location of the keyword or the phrase in the web page. The higher the phrase is located in the page structure, the higher the web page will be listed in the search engine results for it. For example, if you put the keyword for which you want the page to be ranked as high as possible by the search engine in the title tag, it can leave behind web pages whose content is dedicated to the specific phrase, but it is missing in the upper part of the page structure. 2. Frequency - if in one web page a keyword is met more frequently than in another web page, the first web page will have a higher search engine page rank than the second. However, you should not use a single keyword too frequently attempting to fool the search engines since they often recognize such techniques as keyword spamming and refuse to rank the corresponding pages. 3. Links - the number and the type of the links are important factors for the web page ranking. Basically the number of relevant incoming links, versus the number of relevant internal links within the page, versus the number of relevant outgoing links influences the page rank in the search engine results. 4. Click-throughs - the search engine monitors the clicks on the web page link in the search results. If the users click more often on your web page link in the search results, it will be ranked higher than a competitive web page that gets less clicks. Still the page rank depends on a large array of factors and it is difficult to give an exact recipe on how to increase your web page's rank for a defined search engine. You should look for the specific tips and tricks for the given search…