Robotic forum member

Matt Morrison · Post by **Matt Morrison** » Tue Mar 31, 2009 4:57 pm

Interesting - just noticed "Google [Bot]" is currently browsing the forum. The git.

Charlie Reams · Post by **Charlie Reams** » Tue Mar 31, 2009 5:22 pm

Yeah, we get spidered quite regularly, and phpBB cleverly recognises them, although for what purpose I'm not sure.

Phil Reynolds · Post by **Phil Reynolds** » Tue Mar 31, 2009 5:34 pm

Charlie Reams wrote:Yeah, we get spidered quite regularly, and phpBB cleverly recognises them, although for what purpose I'm not sure.

Am I right in thinking that the contents of posts on the board are not indexed by search engines?

Derek Hazell · Post by **Derek Hazell** » Fri Apr 03, 2009 12:18 am

Matt Morrison wrote:Interesting - just noticed "Google [Bot]" is currently browsing the forum. The git.

Just seen "Yahoo [Bot]" as well

Charlie Reams · Post by **Charlie Reams** » Fri Apr 03, 2009 12:20 am

Phil Reynolds wrote: Am I right in thinking that the contents of posts on the board are not indexed by search engines?

Ah, that's what it's for. Yes, they do index the pages but not the content. Odd...

Phil Reynolds · Post by **Phil Reynolds** » Fri Apr 03, 2009 9:00 am

Charlie Reams wrote:Yes, they do index the pages but not the content.

Sorry, you'll have to explain this for me a bit more lucidly. In what sense does Google "index the pages"?

Charlie Reams · Post by **Charlie Reams** » Fri Apr 03, 2009 9:21 am

Phil Reynolds wrote:
Charlie Reams wrote:Yes, they do index the pages but not the content.
Sorry, you'll have to explain this for me a bit more lucidly. In what sense does Google "index the pages"?

Yeah, I realise that didn't make a lot of sense. To see what I mean, try this search; it comes up with a couple of hundred pages, but if you do something like this you'll be rather surprised to discover it returns no results at all. If you look at Google's cached versions of the pages, they just say stuff like "You are not authorised to read this forum." So you can't use Google to search this place, which annoys me a bit, because in my experience of other sites it's often better than the internal search.

Phil Reynolds · Post by **Phil Reynolds** » Fri Apr 03, 2009 9:56 am

Charlie Reams wrote:So you can't use Google to search this place, which annoys me a bit, because in my experience of other sites it's often better than the internal search.

Hmmm. Personally, I think I prefer it that way; even though this is a publicly readable board, I sometimes post things here that I might not if I thought they were liable to show up in some random Google search. I've often come across pages on other phpBB boards when I've been searching for stuff, so it must be configurable. Maybe you should run a poll to see what the members would prefer.

Derek Hazell · Post by **Derek Hazell** » Sun Sep 06, 2009 2:06 pm

Just saw one I've never seen or heard of before "Baidu [Spider]". Apparently it's from a Chinese search engine!

Two more new ones "Majestic-12 [Bot]" and "Exabot [Bot]"!

Also saw "Alexa [Bot]" and "Ask Jeeves [Bot]" recently, and "MSN [Bot]" is quite a regular.

Derek Hazell · Post by **Derek Hazell** » Sun Dec 06, 2009 12:35 pm

Another new one I have never seen before this morning "Nutch [Bot]".

Another today "Google Adsense [Bot]".

Gavin Chipper · Post by **Gavin Chipper** » Sun Dec 06, 2009 1:59 pm

I saw "Apterous Rex [Bot]" the other day.

c4countdown

Robotic forum member

Robotic forum member

Re: Robotic forum member

Re: Robotic forum member

Re: Robotic forum member

Re: Robotic forum member

Re: Robotic forum member

Re: Robotic forum member

Re: Robotic forum member

Re: Robotic forum member

Re: Robotic forum member

Re: Robotic forum member