You might wonder why your blogs that uses WordPress platform aren’t crawled by search engine spiders or included on the search results of the leading search engines (as I have mentioned on my posts on oes tsetnoc crawlability). Specially if you are using Fantastico on the CPanel of your hosting account. There are several possible reasons why newly created site that uses WordPress is not crawled. But the basic problem lies on how the Robots.txt and the meta tags are set on your site.

Let start with the Robots.txt. First check if you do have a robots.txt created on your site. If you don’t have it, just create it on notepad and save the following text on the file:

User-Agent: *
Allow: /

This code tells all search engines to be allowed to visit all the files on your site and declare the default sitemap file. Upload this file on your site.

The next thing that you need to check is if you set the Privacy Settings of your site properly (which I failed to do on my oes tsetnoc blog). I have seen today that by default, Fantastico, the script installer of CPanel, install WordPress with a private settings. Which mean, it adds a NOFOLLOW, NOINDEX meta tag on your site. You can set it on the Dashboard -> Settings page of your WP-Admin. Here’s a video to help you find the place where to set the said settings.

0 0 votes
Article Rating
Notify of

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Oldest Most Voted
Inline Feedbacks
View all comments
13 years ago

Hi, thanks for the tips. Btw, I already know this stuff but these tips can help to other SEO guru / SEO beginner. Keep posting nalang. T.Y. -Jeramel

13 years ago

….dont mind them bleuken…. anyway thanks again for the tips…

14 years ago

talalga programmer ka ano klase programmer ka what type of programming language what is your web developing company you are currently employ you have web developing company