Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.

"It doesn’t matter what sort of web analytics suite you choose to use... If you’re not actively reviewing your log files, you’re missing out on some key data and reporting metrics. Today’s blog post is all about log files and how you can begin using them to learn more about what is happening on your web site."
Comments13 Comments  

Comments

Avatar
from pageoneresults 2033 Days ago #
Votes: 0

Well done! I think logfile anaylsis is greek to most. I like the way Eric takes bits and pieces of his logfile and explains what you are seeing. We’ve taken that whole concept and really focus on 400s and 500s. I track those like a hawk and find myself compensating for Webmasters using incorrect link syntax and all sorts of other mistakes. It would be nice to see some serious discussion on this one. Maybe a few will wake up and realize that 70% of their resources are being used by bad bots and such. :(

Avatar
from EricLander 2033 Days ago #
Votes: 1

Thanks for the submission and comment! Log files have been a big part of my career and I’m often surprised just how many haven’t become acquainted with them. Hopefully this post is encouraging enough to get some more people on board.

Avatar
from Naoise 2033 Days ago #
Votes: 2

<div style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10px; background-color: #ffffff">Though it’s difficult to get marketing-analysis insight from raw logs, it’s even more difficult to get technical and grandular level insight from javascript analytics packages. Because of that, and because raw logs are the closest thing to proof of fact available to a webmaster (perhaps even to the extent of carrying legal implications), they’re pretty essentail learning for an SEO - I recently had to use them to both identify and prove the cause of an indexation problem that lied with bot blocking upstream from a hosting company’s ISP.<div></div><div><div>Really well written article. Now if we can just teach all SEOs the beauty of grep, we’ll be getting somewhere.</div></div></div>

Avatar
from EricLander 2033 Days ago #
Votes: 1

@Naoise grep makes things super easy, but many (sadly) aren’t running *nix servers. I wouldn’t classify them as SEOs though, since anyone worth their weight in salt loves them some Apache... :)

Avatar
from jeremiahandrick 2033 Days ago #
Votes: 1

Really, this is simply one of the best posts on log parsing I have read.  Good job keeping it simple eric.  I have always been a fan of logparsing to understand traffic patterns and bot behavior.  If you have been finding strange or unusual traffic, the logs can show you things you may never see on the script based services.  I quickly booked marked this for later use.

Avatar
from pageoneresults 2033 Days ago #
Votes: -2

"Since anyone worth their weight in salt loves them some Apache." < I used to like you. I’m beginning to question that. :) I’m on Windows and have been since 1995. I use FrontPage exclusively for Web Development. I use WebDAV for publishing. Do you think you can do better than I on Apache? Want to put some money on it? I’ll bring out the BIG GUNS! ;)

Avatar
from EricLander 2033 Days ago #
Votes: 0

@pageoneresults -- I swear I didn’t vote your comment down... Swear it... Let’s agree to disagree, okay? :)

Avatar
from vermasaurav 2033 Days ago #
Votes: 0

<!--[if gte mso 9]><xml> <w:WordDocument> <w:View>Normal</w:View> <w:Zoom>0</w:Zoom> <w:PunctuationKerning/> <w:ValidateAgainstSchemas/> <w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid> <w:IgnoreMixedContent>false</w:IgnoreMixedContent> <w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText> <w:Compatibility> <w:BreakWrappedTables/> <w:SnapToGridInCell/> <w:WrapTextWithPunct/> <w:UseAsianBreakRules/> <w:DontGrowAutofit/> </w:Compatibility> <w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:LatentStyles DefLockedState="false" LatentStyleCount="156"> </w:LatentStyles> </xml><![endif]--> <!-- /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> <!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> I wish I could Sphinn it twice ;). It is one of the best pieces of information I have found lately on Sphinn.   I have an affinity for data, data that can be interpreted into actionable information and now I can have more of it. Thank you Eric.

Avatar
from AirDisa 2033 Days ago #
Votes: 1

I recommend all those who can, to look at log files. There’s other awesome things to learn about your visitors. This info is under represented and I’m very glad to see it get more air time in search. When people hot link your images, it taxes you in more ways than one. It literally taxes the system handling spurious requests. What’s more, it’s possible that it can cost you extra hosting fees.<div></div><div>Moderating some of the geeky panels over the years, I’ve taken note of several things clever webmasters do (Mikkel and Jake especially). What they do with snooping competitors is hilarious. But seriously, you don’t have to allow hot linking images or other media. I place large media files in a folder (IMG), write a .htaccess that prevents hot linking on that directory.</div><div></div><div>It’s fast, simple and impossible to hot link after implemented. When people take images anyway to republish your work? There’s really only a few things you can do about it. At least you can prevent scraping (and the technically inept) from automatically taking stuff. There’s nothing stopping the others.</div>

Avatar
from lyndseo 2032 Days ago #
Votes: 1

Awesome... drool.... just awesome. Must get my hands on our server logs.

Avatar Moderator
from incrediblehelp 2032 Days ago #
Votes: 0

Nice beginners guide to raw logs.  I myself still prefer raw logs, but the fact is many clients dont offer them to remote consultants so we are stuck is javascript based data.

Avatar
from seoz87 2032 Days ago #
Votes: 0

this is why i use sphinn :) great content ..keep it up bro

Avatar
from 99intheShade 2031 Days ago #
Votes: 0

Very interesting discussion of using server logs to collect valuable information.  Easy post to read and understand.  Important topics to be discussed and understood.

Upcoming Conferences

Search Marketing ExpoSearch Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.



Join us at an upcoming SMX event: