Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.

Sphinn Home » SEO
Free SEO Diagnostics Tool: Log File Parsing Script
Avatar Went Hot: Jul 18, 2008 - 02:58 am
Posted By: AdamAudette 364 Days ago
Source: http://www.audettemedia.com
Category: SEO
Here’s a free script for analyzing spider behavior (you’ll need some entry-level command line skills to run it). It’s mighty handy for log analysis - do things like list all 302s, 404s, 500s, or any other status code segmented by user agent.... find out how Slurp crawls your site chronologically, or if MSN behaves with 301s. Great for really big files, fast and efficient.
 6 Comments  

Comments

from trainsem 364 Days ago #
Votes: 0

Always nice to see people sharing useful scripts. Thank you.

from onreact 363 Days ago #
Votes: 0

Advanced SEO for nerds.

from g1smd 363 Days ago #
Votes: 1

There are several more IP blocks that Google uses for spidering.     <div></div><div></div><div>I found these new IP blocks when looking for the IP addresses being used by all that new datacentre hardware (all USD 3 Billion of it) that has come online in the last couple of years.    A quick Google search easily finds the post.  The new datacentres serve search results, but also spider the web.</div><div>   </div>

from AdamAudette 363 Days ago #
Votes: 0

@q1smd do you have the IP range? thanks for the tip

from g1smd 359 Days ago #
Votes: 0

Check my posts [datacentre lists] from June 1st for the list.

from SenHu 196 Days ago #
Votes: 0

I use biterscripting. I believe it is available for all windows versions. I believe it is a free download. There are couple of sample scripts available on their website. I like the one script at http://www.biterscripting.com/Download/SS_WebLogParser.txt . Sen

Sphinn Sponsors
Be a Sphinn Sponsor - Click Here
Top 10 Most Sphunn In What's New
  1. Page segmentation and link building (28)
  2. 7 Red Flags that Reveal to Google… (27)
  3. The Google Operating System Is… (26)
  4. Keywords, Probability &… (24)
  5. Building Traction with Search and… (24)
  6. 30 Blog Improvement Tips to… (21)
  7. Interview with Rae Hoffman AKA… (20)
  8. Jordan K’s Definitive Mobile… (20)
  9. Link Building With Content Ideas (19)
  10. What do SEOs need to know about… (19)
Latest Posts In Sphinn Blog
Latest Comments
RSS Feeds
Upcoming Conferences

Search Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.


SMX Web Site » | SMX Difference » | SMX News »

Join us at an upcoming SMX event:

Upcoming Webcasts

Search Marketing Now Learn more about search marketing with our free online webcasts and webinars from our sister site, Search Marketing Now. Upcoming online events include:


See more webcast topics »