Playing with text files in Linux

René Jorissen on February 26, 2010 1 Comment • Tags: #cut #linux #sort #text #uniq

I had a big Microsoft Event Viewer log file and I wanted specific information from the log file. At first I was thinking about using Microsoft Excel to do some filtering, but that didn’t really help. At the end Linux did the trick. I used Cygwin under Windows to extract the specific information. The raw log file had the following format:

2/22/2010:1:14:46 PM:IAS:Information:None:1:N/A:BOOCHES01:User rene was granted access.
2/22/2010:1:09:15 PM:IAS:Information:None:1:N/A:BOOCHES01:User rene was granted access.
2/22/2010:12:19:58 PM:IAS:Information:None:1:N/A:BOOCHES01:User BOOCHES\test was granted access.
2/22/2010:12:03:24 PM:IAS:Information:None:1:N/A:BOOCHES01:User booches was granted access.
2/22/2010:11:58:54 AM:IAS:Information:None:1:N/A:BOOCHES01:User testuser was granted access.
2/22/2010:11:58:13 AM:IAS:Information:None:1:N/A:BOOCHES01:User booches was granted access.
2/22/2010:11:58:07 AM:IAS:Information:None:1:N/A:BOOCHES01:User BOOCHES\test was granted access.
2/22/2010:11:17:13 AM:IAS:Information:None:1:N/A:BOOCHES01:User testuser1 was granted access.

I needed to extract only the unique users. Playing a little with Linux gave me the following output.

User test was granted access.
User booches was granted access.
User rene was granted access.
User testuser was granted access.
User testuser1 was granted access.

It isn’t perfect, but it is good enough for me. The original log file is called log.txt and the output is written to a file called users.txt. I used the following command to accomplish the output above.

cat log.txt | cut –d: –f11 | sed ‘s/BOOCHES\\//g’ | sort | uniq >> users.txt

Cat prints the file log.txt to the screen. The –d parameter with cut determines the delimiter and –f selects the column to print. With sed I search for the string “BOOCHES\” and replace the sting with nothing (//). Everything is sorted with sort and all duplicate entries are removed with uniq. The output is written to the file users.txt.

Simple and effective!!!

The following two tabs change content below.

René Jorissen

Co-owner and Solution Specialist at 4IP Solutions
René Jorissen works as Solution Specialist for 4IP in the Netherlands. Network Infrastructures are the primary focus. René works with equipment of multiple vendors, like Cisco, Aruba Networks, FortiNet, HP Networking, Juniper Networks, RSA SecurID, AeroHive, Microsoft and many more. René is Aruba Certified Edge Expert (ACEX #26), Aruba Certified Mobility Expert (ACMX #438), Aruba Certified ClearPass Expert (ACCX #725), Aruba Certified Design Expert (ACDX #760), CCNP R&S, FCNSP and Certified Ethical Hacker (CEF) certified. You can follow René on Twitter and LinkedIn.

Latest posts by René Jorissen (see all)

Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.