Parsing web logs with grep
If your Web server returns logs in standard format (with each hit represented by one line of text in the log), you can easily use grep to extract useful data on your recent traffic. For example, I run the BeOS Tip Server as an adjunct to
The distinguishing characteristic of the Tip Server hits is that all of them include the string “beos/tips” in them. To sort out only those hits from a log file called waxwing.log, I use this grep string, remembering to “escape” the forward slash:
grep beos/tips waxwing.log
This spits back a list of Tip Server hits, and now all I have to do is count the number of new lines in this command’s standard output. There are a few ways to get this number, but the easiest is to run the output through the wc command, using its -l flag to count new lines. Thus, typing:
grep beos/tips waxwing.log | wc -l
yields “3754″ or some other number. Since I get a fresh web log every week, I know how many hits the Tip Server got that week. If you have many such strings to check on regularly, write a script containing a series of greps and write the results to a report file.
Posted in Terminal
