Bash: Get largest files in a directory

Posted by on March 12, 2010 · Leave a Comment 

So you have a partition with a ton of subdirectories and the partition is almost full and you want to see if there are large files eating up space. There is an easy command you can run to list the 10 biggest files

find /directory -printf ‘%s %p\n’ | sort -nr | head -n 10

You can change the head -n 10 to -n 20 if you want to get the 20 biggest files.

Filed under Sys Admin · Tagged with , , ,

Remove Leading 0′s in Bash

Posted by on March 12, 2010 · 1 Comment 

So you have a var you want to remove leading 0′s on.. well do the follwing

mzupan@mzupan-desktop:~$ var=00014
mzupan@mzupan-desktop:~$ let var=”10#$var”
mzupan@mzupan-desktop:~$ echo $var
14
mzupan@mzupan-desktop:~$

Filed under Sys Admin · Tagged with , ,

Line breaks in command output redirect

Posted by on March 10, 2010 · 1 Comment 

Every now and then you need to save command output into a bash var from a command. Generally it looks like

VAR=`command`

That will put the stdout of command into $VAR

There is an issue if the output has linebreaks. Recently I had this issue trying to email SVN diffs for DNS changes. My command to get the diffs looked like this

DIFF=`svn diff /var/named/chroot/var/named/ /var/named/chroot/etc/`

If you run the following command

echo $DIFF

You get the following output

Index: /var/named/chroot/var/named/internal/theopenskyproject.com.mgmt =================================================================== — /var/named/chroot/var/named/internal/theopenskyproject.com.mgmt (revision 3056) +++ /var/named/chroot/var/named/internal/theopenskyproject.com.mgmt (working copy) @@ -33,3 +33,7 @@ $INCLUDE “internal/theopenskyproject.com.main” openn.lcl rev.192.168.100 shopopensky.com theopenskyproject.com theopenskyproject.com.main theopenskyproject.com.mgmt theopenskyproject.lcl theopenskyproject.qa theopenskyproject.stg IN CNAME theopenskyproject.com. + +;; test +;; +;; test

You can see it is one big mess.. but if you add quotes around it

echo “$DIFF”

You get the following

===================================================================
— /var/named/chroot/var/named/internal/theopenskyproject.com.mgmt     (revision 3056)
+++ /var/named/chroot/var/named/internal/theopenskyproject.com.mgmt     (working copy)
@@ -33,3 +33,7 @@
$INCLUDE “internal/theopenskyproject.com.main”
*               IN CNAME    theopenskyproject.com.
+
+;; test
+;;
+;; test

As you can see that is much better. Now I can email it to the group

echo “$DIFF” | mail -s “DNS Changes” group@domain.com

Filed under Sys Admin · Tagged with , ,

Bash Command Logger with Curl Support

Posted by on March 9, 2010 · 9 Comments 

There is a great project called Bash Paranoia. Right now their site is busted so I can’t link to it. Its a patch that applies to bash that allows commands to be logged to syslog. I basically took this one step further and added curl support.

The bash paranoia patch and my curl addition can be found on my GitHub project page

http://github.com/mzupan/bash-paranoia-curl

Below is my patch I wrote. Right now it will only work with 64bit systems. It should be easy to make it work with 32bit systems if you edit the patch file at the bottom where I patch Makefile.in. Change the lib64 to lib and you should be good to go

Now if you want to install these patches you would run the following commands. My curl patch needs the base paranoia patch to work. I don’t even think it will apply alone.

wget http://zcentric.com/files/bash-paranoia.patch
wget http://zcentric.com/files/bash-paranoia-curl.patch
tar zxf bash-3.2.tar.gz
cd bash-3.2
patch -p0 < ../bash-paranoia.patch
patch -p1 < ../bash-paranoia-curl.patch
./configure ––enable-paranoia #you can include  other configure flags here
make
make install

That will get you going and the next time you login (if bash if your default shell) you will see the following in your logs (for redhat is is /var/log/messages)

Mar  9 15:24:02 263724-mgmt1 bash: user: mzupan as root from ip: 192.168.71.154:pts/0 execs: ‘cat /var/log/messages’

There you go a nice little command logger that will tell you most of what you need to do to keep tabs on users.

Now if you want to also append this to a db somewhere then curl and a web endpoint is the best solution. So my database look like

CREATE TABLE `commandlog` (
`id` int(11) NOT NULL auto_increment,
`server` varchar(100) NOT NULL,
`user_login` varchar(100) NOT NULL,
`user_run` varchar(100) NOT NULL,
`ip` varchar(100) NOT NULL,
`session` varchar(100) NOT NULL,
`command` longtext NOT NULL,
`ts` datetime NOT NULL,
PRIMARY KEY  (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;
So now here is a simple php app to save it into a db. THERE IS NO SANITY CHECKING HERE. THIS IS JUST A SAMPLE!
I am writing this without checking any code.. so it might be broken.
<?php
$server = $_SERVER['REMOTE_ADDR'];
$user_login = $_POST['user_login'];
$user_run =  $_POST['user_run'];
$ip =  $_POST['ip'];
$session =  $_POST['session'];
$command =  $_POST['command'];
$ts = time();
$sql = “INSERT INTO commandlog(server,user_login,user_run,ip,session,command,ts) VALUES(‘$server’,'$user_login’,'$user_run’,'$ip’,'$session’,'$command’,'$ts’)”;
// place into sql now.. too lazy to do this for you
?>
Now if you want to enable the curl post on the server you edit the following file
/etc/bash.conf
Here is an example
URL=http://1.1.1.1/endpoint/
Have fun!
If you want a spec file that will work for Redhat/CentOS 4/5 64bit you can download the following file

Filed under Sys Admin · Tagged with , , , , , ,