Archive

Archive for November, 2011

Quake code review

November 25, 2011 Leave a comment

Check out Fabien Sanglard’s non-blog. He has code reviews about Quake and Doom. If you are interested in C/C++ programming and game development, it can be interesting.

Categories: Uncategorized Tags: , , ,

MongoHQ: a hosted database solution for MongoDB

November 24, 2011 Leave a comment

MongoHQ is the hosted database solution for getting your applications up and running on MongoDB, fast. From our free shared plans to custom dedicated solutions, we can handle all the different levels of your data layer.” (source)

At MongoHQ you can have a MongoDB database with size up to 16MB for free. If you want more, you gotta pay :) Here is their pricing.

Let’s see the free solution. Register and create a sample database called “mydb” for instance. Create a database user under the “database users” tab by specifying its name and password. You can create several users and a user can have read-only privileges too. Under the “database info” tab, you’ll get the URI of the database, which looks similar to this:

mongodb://<user>:<password>@stuff.mongohq.com:10569/mydb

Using PyMongo, you can connect like this:

c = Connection("mongodb://<username>:<password>@stuff.mongohq.com:10569/mydb")

Where the credentials are the database user’s username and password that you added under the “database users” tab.

Related posts

Categories: mongodb, python Tags:

Some notes on MongoDB and PyMongo

November 24, 2011 2 comments

MongoDB is an open source, high-performance, schema-free, document-oriented database, written in C++. It manages collections of BSON documents that can be nested in complex hierarchies and still be easy to query and index, which allows many applications to store data in a natural way that matches their native data types and structures. Development of MongoDB began in October 2007 by 10gen. The first public release was in February 2009.” (source)

Recently I heard a lot about MongoDB so I gave it a try. I like it :) It’s easy to set up, fast, and pretty easy to use. It has bindings to several languages.

Installation #01 (living on the edge)
The Ubuntu repos are usually out-of-date. If you want to use the latest stable version, install MongoDB from the developers. In short:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
# edit /etc/apt/sources.list and add this line:
deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen
sudo apt-get update
sudo apt-get install mongodb-10gen

Installation #02 (from the Ubuntu repos)
I suggest installing MongoDB using the previous method above. Anyway, here is the classical way:

sudo apt-get install mongodb-server mongodb-clients

Normally it’ll start the server that needs to run in the background if you want to test it on your local machine (just like MySQL for instance). If the server didn’t start for some reason, try this:

sudo service mongodb start

If you get some error that the server cannot be started, remove the file /var/lib/mongodb/mongod.lock.

When you use SQLite, you have a database.sqlite file that stores your database. The database files of MongoDB are situated in the directory /var/lib/mongodb (by default). The log file of the server is at /var/log/mongodb/mongodb.log. These settings can be configured in /etc/mongodb.conf.

Python client
My language of choice is Python, so let’s see how to use MongoDB from a Python script. First, you need to install the PyMongo package:

sudo pip install pymongo

Let’s see if it works:

#!/usr/bin/env python

"""
This example is from the book "MongoDB and Python"
by Niall O'Higgins.
"""

import sys

from datetime import datetime

from pymongo import Connection
from pymongo.errors import ConnectionFailure

def main():
    """ Connect to MongoDB """
    try:
        c = Connection(host="localhost", port=27017)
    except ConnectionFailure, e:
        sys.stderr.write("Could not connect to MongoDB: %s" % e)
        sys.exit(1)

    # Get a Database handle to a database named "mydb"
    dbh = c["mydb"]

    user_doc = {
        "username" : "janedoe",
        "firstname" : "Jane",
        "surname" : "Doe",
        "dateofbirth" : datetime(1974, 4, 12),
        "email" : "janedoe74@example.com",
        "score" : 0
    }
    dbh.users.insert(user_doc, safe=True)
    print "Successfully inserted document: %s" % user_doc

#############################################################################

if __name__ == "__main__":
    main()

For the connection we use the default values. Using SQL terminology, here is what happens: “mydb” is the name of the database that we access via the handler dbh. user_doc is a row, and dbh.users.insert(user_doc, safe=True) means that inside the database (“mydb”), in the “users” table we insert the row user_doc. It is recommended to use safe=True for write operations (insert, update, remove, and findAndModify), otherwise MongoDB doesn’t check for errors :(

As you can see, you don’t have to create neither the database “mydb” nor the table “users”. When you want to insert something in them, MongoDB will create them if they don’t exist.

Troubleshooting (20130414)
After installing “pymongo”, I couldn’t import it. As it turned out it conflicted with the “bson” package. Solution:

sudo pip uninstall pymongo
sudo pip uninstall bson
sudo apt-get remove python-bson
sudo apt-get remove python-gridfs
sudo pip install pymongo -U

Visualization
Now we have a database, a table, and a row in that table. It’d be nice to visualize the database. There is a very nice PHP-based administration GUI tool called RockMongo. This is what we’ll use. Requirements:

sudo apt-get install php-pear
sudo pecl install mongo

Then put rockmongo in your public_html directory and open it in your browser. It’ll warn you to add a line to your php.ini file (located at /etc/php5/apache2/php.ini). Don’t forget to restart the webserver. For logging in, use “admin” and “admin” as username and password. The usage of rockmongo is completely intuitive. (For setting up PHP on your machine, check out this post.)

Reducing database size
MongoDB is quite aggressive in allocating disk space for databases. For reducing the database size, you can add the following lines to /etc/mongodb.conf:

# disable data file preallocation
noprealloc = true
# use smaller files
smallfiles = true

It’s not recommended for production but for a small project on your local machine it can be useful. For taking it into account, restart the mongodb server. These settings will be applied for new databases only.

Toubleshooting

If you have problems starting MongoDB, refer to this link.

Essential links

Books / Docs

Further links

Categories: mongodb, python Tags: , ,

Firebug: sitescraper’s best friend

November 23, 2011 Leave a comment

When you do sitescraping, usually you know exactly what part of a webpage you want to extract. The naive way is to download and analyze the source code of the page trying to identify the interesting part(s). But there is a better way: use Firebug.

Firebug is a Firefox add-on for web developers. You can edit, debug, and monitor CSS, HTML, and JavaScript live in any web page. The interesting part for us is the feature that you can point on any element of a webpage and Firebug shows you its exact location in the source. You can also get the CSS Path and/or the XPath of the given element.

First, install Firebug and restart the browser. In the top right corner of the browser you’ll see a little bug (part A on the figure below). Clicking on this will call the Firebug console. On the console, click on the 2nd icon from the left in the console’s top (part B). Then click on an element in the browser that you want to inspect (part C). The relevant HTML source code will be highlighted in the console (part D). Right click on it and choose the CSS Path / XPath from the popup menu. Now you only have to write a script that extracts this part of the page.

Categories: firefox Tags: ,

Current runlevel

November 22, 2011 Leave a comment

Problem
Figure out the current runlevel of your system.

Solution
Surprise, surprise :) You need the following command:

$ runlevel   # command
N 2          # sample output

The previous and current runlevel from that record are output separated by a single space. If there is no previous runlevel in the record, the letter N will be substituted.

So, in my case the current runlevel is 2. It means I can check the startup scripts in the directory /etc/rc2.d.

Categories: ubuntu Tags: ,

Nettuts+

November 21, 2011 1 comment

Nettuts+ is a site aimed at web developers and designers offering tutorials and articles on technologies, skills and techniques to improve how you design and build websites. We cover HTML, CSS, Javascript, CMS’s, PHP and Ruby on Rails.

They also have Python tutorials.

Categories: web Tags:

C++ style guide

November 19, 2011 Leave a comment

Google C++ Style Guide

C++ is the main development language used by many of Google’s open-source projects. As every C++ programmer knows, the language has many powerful features, but this power brings with it complexity, which in turn can make code more bug-prone and harder to read and maintain.

The goal of this guide is to manage this complexity by describing in detail the dos and don’ts of writing C++ code. These rules exist to keep the code base manageable while still allowing coders to use C++ language features productively.

Categories: google, Uncategorized Tags: ,

Commodore OS trailer

November 14, 2011 Leave a comment

I watched the following video with watery eyes. Good old Commodore 64, best machine ever…

 

If I got it right, it’ll be a modified Ubuntu distro.

63

November 11, 2011 Leave a comment

Today is 11.11.11.

Which one is the year / month / day? Doesn’t matter :)

Categories: fun Tags:

Vim turned 20

November 10, 2011 Leave a comment

Vim turned 20 on November 2, 2011. Happy birthday! And thank you Bram for this wonderful text editor! :x

Categories: vim Tags:
Follow

Get every new post delivered to your Inbox.

Join 71 other followers