Your site has been assigned special crawl rate settings. You will not be able to change the...

Drake83 Level 3

May 29, 2013, 8:57 pm

Write the reason you're deleting this FAQ

Your site has been assigned special crawl rate settings. You will not be able to change the...

I noticed that my crawl rate was down, I use to see new content appear in seconds or minutes but now it is a couple of days. I went to Google Webmaster Tools to change the crawl rate. Under settings in Google Webmaster Tools I see this:

Your site has been assigned special crawl rate settings. You will not be able to change the crawl rate

What does that mean? Is that a good thing or a bad thing?

EDIT: I found this from a JohnMu at Google:

The special crawl rate settings apply to sites where we are manually optimizing the crawl rate on our side. This is often due to feedback from the webmaster, the hoster or just generally because we've determined that the site can profit from a manual crawl rate setting by our engineers. In general, you do not have to worry about the crawl rate setting if this is the case with your site.

Also, when looking at the site health under Crawl Stats, I see pages has dropped from 90,000 per day to about 60,000 per day. Not idea why.

KB per day has increased from 1,200,000 to 1,600,000 though. No idea why.

EDIT #2: Apparently if you are on a CDN, google will manually set your crawl rate for some reason. Here is what Gary from Google says about it:

When you're using a CDN, we may assign a special crawl rate for your site so that we can crawl as much as possible without affecting negatively others who're using the same service. It's best to not tackle that setting, especially since we're already accessing your site more than 4000 times a day.
If you want to lower your crawling rate, let me know, but I definitely not recommend increasing it. I'd rather focus on posting unique and compelling content and on user experience.

Kinda odd though that my crawl rate is now lower when I've had a CDN for... always..

Comments

Please login or sign up to leave a comment

Join

SEOClerks

Your site has been assigned special crawl rate settings. You will not be able to change the...

Your site has been assigned special crawl rate settings. You will not be able to change the...

Comments

Please login or sign up to leave a comment

Invalid SQL

Expensive SQL

Suspicious SQL

Match	Count	SQL	Script
SELECT * FROM `questions` as q JOIN categories_faq as c ON q.catid=c.CATID WHERE (q.status=1 OR (q.status=2 AND userid='') ) AND q.quesid='14387'	1	SELECT q.*, c.seo as CatSEO, c.name as CatName, c.parentid FROM `questions` as q JOIN categories_faq as c ON q.catid=c.CATID WHERE (q.status=1 OR (q.status=2 AND userid='') ) AND q.quesid='14387'	/opt/clerks-staging/docroot/viewfaq.php 496 () ()
SELECT * FROM seoclerks.members WHERE USERID='6289'	1	SELECT * FROM seoclerks.members WHERE USERID='6289'	/opt/clerks-staging/docroot/include/functions/includes/member.php 445 GetAllUserDetails() ()
SELECT * FROM categories_faq WHERE CATID='36'	1	SELECT CATID, seo, name, metatitle, h2header, metakeywords, metadescription, metaheader, parentid, image_name FROM categories_faq WHERE CATID='36'	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() GetCategorySeoFromType()
SELECT * FROM categories_faq WHERE CATID='30'	1	SELECT CATID, seo, name, metatitle, h2header, metakeywords, metadescription, metaheader, parentid, image_name FROM categories_faq WHERE CATID='30'	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() GetCategorySeoFromType()
SELECT * FROM seoclerks.members WHERE USERID=6289	1	SELECT googleplus_profile FROM seoclerks.members WHERE USERID=6289	/opt/clerks-staging/docroot/include/functions/main.php 16914 GetGooglePlusProfileFromId() ()
SELECT * FROM ratings_faq as r, seoclerks.members as m WHERE r.USERID=m.USERID AND r.upvote=1 AND r.PID=14387 LIMIT 5	1	SELECT m.username FROM ratings_faq as r, seoclerks.members as m WHERE r.USERID=m.USERID AND r.upvote=1 AND r.PID=14387 LIMIT 5	/opt/clerks-staging/docroot/include/functions/main.php 17159 GetVoters() ()
SELECT * FROM answers a, seoclerks.members b WHERE a.quesid='14387' AND a.userid=b.USERID and b.status='1' AND a.status=1 ORDER BY a.combined_votes DESC, a.date_answered asc	1	SELECT a.answer, a.USERID, a.upvotes, a.downvotes, a.ansid, a.parentid, a.combined_votes, a.date_answered, b.username, b.userlevel, b.profilepicture FROM answers a, seoclerks.members b WHERE a.quesid='14387' AND a.userid=b.USERID and b.status='1' AND a.status=1 ORDER BY a.combined_votes DESC, a.date_answered asc	/opt/clerks-staging/docroot/viewfaq.php 1245 () ()
UPDATE questions SET total_views = total_views + 1 WHERE quesid='14387'	1	UPDATE questions SET total_views = total_views + 1 WHERE quesid='14387'	/opt/clerks-staging/docroot/include/functions/main.php 1765 update_Faqviewcount() ()
SELECT * FROM questions WHERE quesid!='14387' AND status='1' AND (question like '%Your site assigned special crawl rate settings. change the...%' OR question like '%Your%' OR question like '%site%' OR question like '%assigned%' OR question like '%special%' OR question like '%crawl%' OR question like '%rate%' OR question like '%settings.%' OR question like '%change%' OR question like '%the...%') -- ORDER BY RAND() LIMIT 15	1	SELECT quesid, question, seo, userid FROM questions WHERE quesid!='14387' AND status='1' AND (question like '%Your site assigned special crawl rate settings. change the...%' OR question like '%Your%' OR question like '%site%' OR question like '%assigned%' OR question like '%special%' OR question like '%crawl%' OR question like '%rate%' OR question like '%settings.%' OR question like '%change%' OR question like '%the...%') -- ORDER BY RAND() LIMIT 15	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() ()
SELECT * FROM seoclerks.members WHERE USERID='750' limit 1	1	SELECT profilepicture FROM seoclerks.members WHERE USERID='750' limit 1	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_member_profilepicture()
SELECT * FROM seoclerks.members WHERE USERID='632' limit 1	1	SELECT profilepicture FROM seoclerks.members WHERE USERID='632' limit 1	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_member_profilepicture()
SELECT * FROM seoclerks.members WHERE USERID='498' limit 1	1	SELECT profilepicture FROM seoclerks.members WHERE USERID='498' limit 1	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_member_profilepicture()
SELECT * FROM seoclerks.members WHERE USERID='318' limit 1	1	SELECT profilepicture FROM seoclerks.members WHERE USERID='318' limit 1	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_member_profilepicture()
SELECT * FROM seoclerks.members WHERE USERID='297' limit 1	1	SELECT profilepicture FROM seoclerks.members WHERE USERID='297' limit 1	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_member_profilepicture()
SELECT * FROM members_ledger WHERE script='/opt/clerks-staging/docroot/viewfaq.php' AND querystring LIKE '%id=14387%' AND added>=UNIX_TIMESTAMP(NOW())-1200 GROUP BY USERID	1	SELECT USERID, username FROM members_ledger WHERE script='/opt/clerks-staging/docroot/viewfaq.php' AND querystring LIKE '%id=14387%' AND added>=UNIX_TIMESTAMP(NOW())-1200 GROUP BY USERID	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() ()
SELECT * FROM categories	1	SELECT * FROM categories	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() parseRedundantQueriesCache()
select * from categories_software order by name asc	1	select * from categories_software order by name asc	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_GetSoftwareCategories()
select * from categories_wanttobuy order by name asc	1	select * from categories_wanttobuy order by name asc	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_wantcategories()
select * from categories_wanttotrade order by name asc	1	select * from categories_wanttotrade order by name asc	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_tradecategories()
select * from categories_faq order by name asc	1	select * from categories_faq order by name asc	/opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_GetFaqCategories()