Fabric Shell
I recently wrote a quick little fabric function to create a fabric shell. This works with a stock fabric install, you just might have to change where you import it from. It is at my Github page below.
https://github.com/mzupan/fabric_shell
If you place shell.py in the same directory as fabfile.py you can use it like
from fabric.api import *
from fabric.decorators import hosts
import shell
env.hosts = [
'host1.domain.com',
'host2.domain.com'
]
@hosts('')
def cmd():
shell.shell()Then you can use like like
$ fab cmd
[] Executing task 'cmd'
fabric::> id
[host1.domain.com] run: id
[host1.domain.com] out: uid=1020(mzupan) gid=1020(mzupan) groups=1020(mzupan)
[host1.domain.com] out:
[host2.domain.com] run: id
[host2.domain.com] out: uid=1020(mzupan) gid=1020(mzupan) groups=1020(mzupan)
[host2.domain.com] out:
fabric::> .sudo id
[host1.domain.com] sudo: id
[host1.domain.com] out: uid=0(root) gid=0(root) groups=0(root)
[host1.domain.com] out:
[host2.domain.com] sudo: id
[host2.domain.com] out: uid=0(root) gid=0(root) groups=0(root)
[host2.domain.com] out:
fabric::> Right now any command that is issued in the fabric shell gets run on all hosts in env.hosts and if you add .sudo in front of the command it will run them as root.
Tav’s Fabric Fork Quick Start
I’ve been a big fan of the fabric project for a long time now. It has made my life very easy in the past and gets better with each new release. Well a few weeks ago I got introduced to a fork of the project by Tav. It provides many new feature additions to fabric and makes fabric a little bit better. So the best way to get it started is via the following
git clone git://github.com/tav/pylibs.git
cd pylibs
python setup.py
export PYTHONPATH=$PYTHONPATH:`pwd`Once installed, you have to create a fab app
vi /usr/bin/fab
Then you want to add the following contents to it
#!/usr/bin/env python
from fabric.main import main
main()Then you want to give it the correct permissions
chmod +x /usr/bin/fab
Now create a directory and place a fabfile.py file in it.
Now create a config.yml file and use the following as a simple template
default:
shell: /bin/bash -l -c
servers:
directory: /dir
hosts:
- host1.domain.com
- host2.domain.comThen your fab file could look something like
from fabric.api import *
env.config_file = 'config.yml'
env.user = 'mzupan'
@task('servers')
def command():
env().multirun('uname -a')Normally fabric runs in serial, but Tav’s port adds a multirun() function that runs the commands in parallel. BONUS!
Feel free to play around with it and check out his blog post on it to get more commands.
http://tav.espians.com/fabric-python-with-cleaner-api-and-parallel-deployment-support.html
Find the real MongoDB command for pyMongo
I am in the middle of writing a MongoDB plugin for cacti using pymongo and ran into a little issue.
In the Mongo shell this works
> db.stats()
{
"collections" : 19,
"objects" : 145971,
"dataSize" : 64785380,
"storageSize" : 129544960,
"numExtents" : 75,
"indexes" : 69,
"indexSize" : 25403392,
"ok" : 1
}So in pymongo I tried
info = db.command("stats")
Then I got the following traceback
Traceback (most recent call last):
File "./get_mongodb_stats.py", line 68, in
main(sys.argv[1:])
File "./get_mongodb_stats.py", line 39, in main
get_stats(host, port)
File "./get_mongodb_stats.py", line 57, in get_stats
info = db.command("stats")
File "/usr/lib/python2.6/site-packages/pymongo/database.py", line 306, in command
(command, result["errmsg"])) Then I remembered a nice part about the mongoshell. You can run the command without the () and see the function
> db.stats
function () {
return this.runCommand({dbstats:1});
}So you do this in pymongo
info = db.command("dbstats")
Google Apps List Shared Contacts
I’ll be the first to call out Google that their gdata python library for connecting into the Google Apps API is the worst.
I wanted to grab all the shared contacts from our domain and it was a giant in the ass. Here is the code I ended up with
import gdata.contacts.client
contact_list = gdata.contacts.client.ContactsClient()
contact_list.client_login(email="user@domain.com",
password="password",
source='comany-app-1',
account_type='HOSTED')
feed_url = contact_list.GetFeedUri(contact_list="domain.com", projection='full')
while True:
feed = contact_list.get_feed(
uri=feed_url,
auth_token=None,
desired_class=gdata.contacts.data.ContactsFeed)
for entry in feed.entry:
print entry.title.text
next_link = feed.GetNextLink()
if next_link is None:
break
feed_url = next_link.href
The email you use needs to have admin rights over your domain in Google Apps.
Django: Custom Decorator with an Argument
Sometimes you want to be able to create a decorator for functions in Django but have the ability to pass in an argument into the decorator. I came across this when I wanted to switch from the require_login decorator to check if a user was in a certain group to see the function.
So the first thing I did was create a new directory in my project root called decorators and put a blank __init__.py file there. My I cated an auth.py file with the following contents.
from functools import wraps
from django.contrib.auth.models import Group
from django.http import Http404
def group_required(groups=[]):
def decorator(func):
def inner_decorator(request,*args, **kwargs):
for group in groups:
try:
if Group.objects.get(name=group) in request.user.groups.all():
return func(request, *args, **kwargs)
except:
pass
raise Http404()
return wraps(func)(inner_decorator)
return decorator
So we can use it like this in our project
from decorators.auth import group_required
@group_required(["admins"])
def show_index(request):
....
I decided to make it a list being passed in. The main reason was to make it was to pass in multiple groups that you might want to allow to be allowed in.
My version doesn’t do much error checking.. if you want a more updated version of this function checkout my githib project I’ve started to put all my custom decorators up and online
What Rollout will be?
I have been debating just helping with fabric or writing my own app for a week now and I just think what I need and want fabric just isn’t and too much work to get it to the way I want it to be.
So here are some of the things I want Rollout to be.
- Be able to run in console
- Be able to tie the Rollout API into your own python webapp
- Rollout output should be multiformat (xml/json)
- Should be threaded
- Should have a run_first() and run_last() method
- Tie in git/svn?
Again if you want to watch the project it is http://github.com/mzupan/rollout
Rollout has been designed
I’m a big fan of fabric, a python module to make deployments easy. It is still very new but has some great features and some major short comings I don’t think the authors thought of before they designed it.
The major shortcoming is a lack of parallel deployments. Fabric works great on 1 or 2 hosts you need to deploy on but when you have more then that for a large application it is painful. There is a patch for parallel deployments and it works but it “breaks” some features of fabric. The main one is the @runs_once.
So I am going to try to build a python module to better handle deployments in a parallel fashion. I have decided to model my module after fabric since I’m very use to it and I think their model works.
I started a GitHub project for it but don’t have any code in there yet.
http://github.com/mzupan/rollout
If you are interested please keep an eye on the project. I hope to make it functional soon.
Reason #1 on why Django is great
All the crazy helper function they have for models to display information in templates. Say I have a model that looks like this with a choice field (it will get rendered as a select input in html.
SEX_CHOICE = (
(1, 'Male'),
(2, 'Female'),
)
class Animal(models.Model):
name = models.CharField(max_length=50)
sex = models.SmallIntegerField(max_length=1, default=0, choices=SEX_CHOICE)Now in my template I can do the following to see if an animal object is a Male or a Female
{{animal.get_sex_display}}
There is no need to create a method for that in the model. Django does it for you. Saving you time and effort.
Install fabric on Redhat/CentOS 5
Fabric is a great tool for helping deploy applications. It is written in Python and still in it’s early stages. The big issue is that fabric will not run on python v2.4. That is a problem since Redhat/CentOS v5 ships with v2.4 so we have to make it work.
So the first thing to do is install python 2.5. I found great rpms here. I am going to mirror the RPMS on my blog just in case. You also need to install tkinter25 which I provide for you also. These are only for 64bit. You can do some googling to find the 32 bit versions I am sure. You also need to install libtk and libTix.
Install some dependancies
yum install tk tix
Now lets install python and tkinter
mkdir python
cd python
wget http://zcentric.com/files/python25-2.5.1-bashton1.x86_64.rpm
wget http://zcentric.com/files/python25-devel-2.5.1-bashton1.x86_64.rpm
wget http://zcentric.com/files/python25-libs-2.5.1-bashton1.x86_64.rpm
wget http://zcentric.com/files/python25-test-2.5.1-bashton1.x86_64.rpm
wget http://zcentric.com/files/python25-tools-2.5.1-bashton1.x86_64.rpm
wget http://zcentric.com/files/tkinter25-2.5.1-bashton1.x86_64.rpm
rpm -ivh *.rpm
Remove all the python25 rpms
rm -rf *.rpm
Now you have v2.4 and v2.5 installed. Now lets download the needed fabric dependancies and fabric. I built these to run for python25
wget http://zcentric.com/files/fabric-0.9-0.1.b1.noarch.rpmwget http://zcentric.com/files/python25-setuptools-0.6c5-2.noarch.rpmwget http://zcentric.com/files/python25-paramiko-1.7.6-1.noarch.rpmwget http://zcentric.com/files/python25-crypto-2.0-1.rf.x86_64.rpmrpm -ivh *.rpm
vi /usr/bin/fab
#!/usr/bin/python
#!/usr/bin/python25
[root@dev1 ~]# fabFatal error: Couldn’t find any fabfiles!Aborting.