varleysheen
Level 1
Write the reason you're deleting this FAQ
Hi,
on majestic, it only crawls 19 of my posts and really wants it to crawl all as I believe my TF will be higher. anybody know what is going on
Are you sure you want to delete this post?
Are you sure you want to delete this post?
| Match | Count | SQL | Script |
| SELECT * FROM members_ledger WHERE ip = ? AND added>=unix_timestamp(NOW())-86400 | 1 | SELECT querystring, added
FROM members_ledger
WHERE
ip = ? AND added>=unix_timestamp(NOW())-86400 | /opt/clerks-staging/docroot/include/functions/includes/security.php 398 fetchMemberLedger() include_once() |
| SELECT * FROM `questions` as q JOIN categories_faq as c ON q.catid=c.CATID WHERE (q.status=1 OR (q.status=2 AND userid='') ) AND q.quesid='22459' | 1 | SELECT q.*, c.seo as CatSEO, c.name as CatName, c.parentid
FROM `questions` as q
JOIN categories_faq as c ON q.catid=c.CATID
WHERE (q.status=1 OR
(q.status=2 AND userid='')
)
AND q.quesid='22459' | /opt/clerks-staging/docroot/viewfaq.php 496 () () |
| SELECT * FROM seoclerks.members WHERE USERID='497533' | 1 | SELECT * FROM seoclerks.members WHERE USERID='497533' | /opt/clerks-staging/docroot/include/functions/includes/member.php 445 GetAllUserDetails() () |
| SELECT * FROM seoclerks.members WHERE USERID=497533 | 1 | SELECT googleplus_profile FROM seoclerks.members WHERE USERID=497533 | /opt/clerks-staging/docroot/include/functions/main.php 16914 GetGooglePlusProfileFromId() () |
| SELECT * FROM ratings_faq as r, seoclerks.members as m WHERE r.USERID=m.USERID AND r.upvote=1 AND r.PID=22459 LIMIT 5 | 1 | SELECT m.username FROM
ratings_faq as r,
seoclerks.members as m
WHERE r.USERID=m.USERID
AND r.upvote=1
AND r.PID=22459
LIMIT 5 | /opt/clerks-staging/docroot/include/functions/main.php 17159 GetVoters() () |
| SELECT * FROM answers a, seoclerks.members b WHERE a.quesid='22459' AND a.userid=b.USERID and b.status='1' AND a.status=1 ORDER BY a.combined_votes DESC, a.date_answered asc | 1 | SELECT a.answer, a.USERID, a.upvotes, a.downvotes, a.ansid, a.parentid, a.combined_votes, a.date_answered,
b.username, b.userlevel, b.profilepicture FROM answers a, seoclerks.members b
WHERE a.quesid='22459' AND a.userid=b.USERID and b.status='1' AND a.status=1
ORDER BY a.combined_votes DESC, a.date_answered asc | /opt/clerks-staging/docroot/viewfaq.php 1245 () () |
| SELECT * FROM ratings_faqanswers as r, seoclerks.members as m WHERE r.USERID=m.USERID AND r.upvote=1 AND r.PID=116969 LIMIT 5 | 1 | SELECT m.username FROM
ratings_faqanswers as r,
seoclerks.members as m
WHERE r.USERID=m.USERID
AND r.upvote=1
AND r.PID=116969
LIMIT 5 | /opt/clerks-staging/docroot/include/functions/main.php 17159 GetVoters() () |
| SELECT * FROM ratings_faqanswers as r, seoclerks.members as m WHERE r.USERID=m.USERID AND r.upvote=1 AND r.PID=116979 LIMIT 5 | 1 | SELECT m.username FROM
ratings_faqanswers as r,
seoclerks.members as m
WHERE r.USERID=m.USERID
AND r.upvote=1
AND r.PID=116979
LIMIT 5 | /opt/clerks-staging/docroot/include/functions/main.php 17159 GetVoters() () |
| SELECT * FROM ratings_faqanswers as r, seoclerks.members as m WHERE r.USERID=m.USERID AND r.upvote=1 AND r.PID=170366 LIMIT 5 | 1 | SELECT m.username FROM
ratings_faqanswers as r,
seoclerks.members as m
WHERE r.USERID=m.USERID
AND r.upvote=1
AND r.PID=170366
LIMIT 5 | /opt/clerks-staging/docroot/include/functions/main.php 17159 GetVoters() () |
| UPDATE questions SET total_views = total_views + 1 WHERE quesid='22459' | 1 | UPDATE questions SET total_views = total_views + 1 WHERE quesid='22459' | /opt/clerks-staging/docroot/include/functions/main.php 1765 update_Faqviewcount() () |
| SELECT * FROM questions WHERE quesid!='22459' AND status='1' AND (question like '%Majestic Seo crawling pages%' OR question like '%Majestic%' OR question like '%crawling%' OR question like '%pages%') -- ORDER BY RAND() LIMIT 15 | 1 | SELECT quesid, question, seo, userid
FROM questions
WHERE quesid!='22459' AND status='1'
AND (question like '%Majestic Seo crawling pages%' OR question like '%Majestic%' OR question like '%crawling%' OR question like '%pages%')
-- ORDER BY RAND()
LIMIT 15 | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() () |
| SELECT * FROM seoclerks.members WHERE USERID='840' limit 1 | 1 | SELECT profilepicture FROM seoclerks.members WHERE USERID='840' limit 1 | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_member_profilepicture() |
| SELECT * FROM seoclerks.members WHERE USERID='18733' limit 1 | 1 | SELECT profilepicture FROM seoclerks.members WHERE USERID='18733' limit 1 | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_member_profilepicture() |
| SELECT * FROM seoclerks.members WHERE USERID='5478' limit 1 | 1 | SELECT profilepicture FROM seoclerks.members WHERE USERID='5478' limit 1 | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_member_profilepicture() |
| SELECT * FROM seoclerks.members WHERE USERID='147' limit 1 | 1 | SELECT profilepicture FROM seoclerks.members WHERE USERID='147' limit 1 | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_member_profilepicture() |
| SELECT * FROM seoclerks.members WHERE USERID='9399' limit 1 | 1 | SELECT profilepicture FROM seoclerks.members WHERE USERID='9399' limit 1 | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_member_profilepicture() |
| SELECT * FROM members_ledger WHERE script='/opt/clerks-staging/docroot/viewfaq.php' AND querystring LIKE '%id=22459%' AND added>=UNIX_TIMESTAMP(NOW())-1200 GROUP BY USERID | 1 | SELECT USERID, username FROM members_ledger
WHERE script='/opt/clerks-staging/docroot/viewfaq.php'
AND querystring LIKE '%id=22459%'
AND added>=UNIX_TIMESTAMP(NOW())-1200
GROUP BY USERID
| /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() () |
| SELECT * FROM categories | 1 | SELECT * FROM categories | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() parseRedundantQueriesCache() |
| select * from categories_software order by name asc | 1 | select * from categories_software order by name asc | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_GetSoftwareCategories() |
| select * from categories_wanttobuy order by name asc | 1 | select * from categories_wanttobuy order by name asc | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_wantcategories() |
| select * from categories_wanttotrade order by name asc | 1 | select * from categories_wanttotrade order by name asc | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_get_tradecategories() |
| SELECT * FROM seoclerks.members WHERE USERID='134.png' | 1 | SELECT profilepicture FROM seoclerks.members WHERE USERID='134.png' | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() getUserProfileImage() |
| SELECT * FROM seoclerks.members WHERE USERID='497533.jpg' | 1 | SELECT profilepicture FROM seoclerks.members WHERE USERID='497533.jpg' | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() getUserProfileImage() |
| SELECT * FROM seoclerks.members WHERE USERID='600657.jpg' | 1 | SELECT profilepicture FROM seoclerks.members WHERE USERID='600657.jpg' | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() getUserProfileImage() |
| select * from categories_faq order by name asc | 1 | select * from categories_faq order by name asc | /opt/clerks-staging/docroot/libraries/adodb5/adodb.inc.php 1899 CacheExecute() insert_GetFaqCategories() |
| count(*) | sql1 | error_msg |
|---|
| Load | Count | SQL | Max | Min |
| Avg Time | Count | SQL | Max | Min |
Cristian
Another good way to figure out what is going on is to write the following command in Google Search: "
This command will show you the exact number of pages of your domain indexed by Google. You can then compare that number with the number crawled by Majestic.
Also, I won't rely on web crawlers to much, if you really want to proper crawl a website I suggest you try Screaming Frog (the paid version) and I assure you if you configure it correctly it will crawl everything to the last link, image, and URL.
Another problem you may have that can stuck Majestic from crawling your whole website are robots.txt restriction you may have as well as robots meta tags with no-index or no-follow implemented on some of your pages.
If you don't have the money for a paid version of screaming frog, let me know your domain address and I'll do a full crawl and tell you where the problems are and overall result.
Cheers. How many posts do you have? Because if you only have 19 posts Majestic will only crawl those 19 posts. Another good way to figure out what is going on is to write the following command in Google Search: "[code]site:http://www.yourwebsitename.com[/code]Another good way to figure out what is going on is to write the following command in Google Search: "site:http://www.yourwebsitename.com" This command will show you the exact number of pages of your domain indexed by Google. You can then compare that number with the number crawled by Majestic. Also, I won't rely on web crawlers to much, if you really want to proper crawl a website I suggest you try Screaming Frog (the paid version) and I assure you if you configure it correctly it will crawl everything to the last link, image, and URL. Another problem you may have that can stuck Majestic from crawling your whole website are robots.txt restriction you may have as well as robots meta tags with no-index or no-follow implemented on some of your pages. If you don't have the money for a paid version of screaming frog, let me know your domain address and I'll do a full crawl and tell you where the problems are and overall result. Cheers.
Are you sure you want to delete this post?