Storing sensitive data in your Dropbox folder

Problem
You want to store some sensitive data in your Dropbox folder, e.g. passwords. How to protect these data?

Solution
In your Dropbox folder create a Truecrypt volume and store your data in this encrypted virtual file system. For more info refer to this article.

Example
I wanted to store some credentials that I wanted to access from several machines. In my Dropbox folder I created a 10 MB Truecrypt volume. I mounted it and put the sensitive data in it.

Download cookie-protected pages with Python using cookielib (Part 2)

Warning! In this post I use the Project Euler site as an example. However, it seems that this method doesn’t work anymore with that site. The PE site was updated recently and they have changed something. However, the method described below might work well with other sites.

Update (20111108): If you want to scrape the Project Euler site, check out Part 3 of this series.

---

In Part 1 we showed how to download a cookie-protected page with Python + wget. First, cookies of a given site were extracted from Firefox’s cookies.sqlite file and they were stored in a plain-text file called cookies.txt. Then this cookies.txt file was passed to wget and wget fetched the protected page.

The solution above works but it has some drawbacks. First, an external command (wget) is called to fetch the webpage. Second, the extracted cookies must be written in a file for wget.

In this post, we provide a clean, full-Python solution. The extracted cookies are not stored in the file system and the pages are downloaded with a Python module from the standard library.

Step 1: extracting cookies and storing them in a cookiejar
On the blog of Guy Rutenberg I found a post that explains this step. Here is my slightly refactored version:

#!/usr/bin/env python

import os
import sqlite3
import cookielib
import urllib2

COOKIE_DB = "{home}/.mozilla/firefox/cookies.sqlite".format(home=os.path.expanduser('~'))
CONTENTS = "host, path, isSecure, expiry, name, value"
COOKIEFILE = 'cookies.lwp'          # the path and filename that you want to use to save your cookies in
URL = 'http://projecteuler.net/index.php?section=statistics'

def get_cookies(host):
    cj = cookielib.LWPCookieJar()       # This is a subclass of FileCookieJar that has useful load and save methods
    con = sqlite3.connect(COOKIE_DB)
    cur = con.cursor()
    sql = "SELECT {c} FROM moz_cookies WHERE host LIKE '%{h}%'".format(c=CONTENTS, h=host)
    cur.execute(sql)
    for item in cur.fetchall():
        c = cookielib.Cookie(0, item[4], item[5],
            None, False,
            item[0], item[0].startswith('.'), item[0].startswith('.'),
            item[1], False,
            item[2],
            item[3], item[3]=="",
            None, None, {})
        cj.set_cookie(c)

    return cj

def main():
    host = 'projecteuler'
    cj = get_cookies(host)
    for index, cookie in enumerate(cj):
        print index,':',cookie
    #cj.save(COOKIEFILE)    # save the cookies if you want (not necessary)

if __name__=="__main__":
    main()

Step 2: download the protected page using the previously filled cookiejar
Now we need to download the protected page:

def get_page_with_cookies(cj):
    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
    urllib2.install_opener(opener)

    theurl = URL    # an example url that sets a cookie, try different urls here and see the cookie collection you can make !
    txdata = None   # if we were making a POST type request, we could encode a dictionary of values here - using urllib.urlencode
    #params = {}
    #txdata = urllib.urlencode(params)
    txheaders =  {'User-agent' : 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'}  # fake a user agent, some websites (like google) don't like automated exploration

    req = urllib2.Request(theurl, txdata, txheaders)    # create a request object
    handle = urllib2.urlopen(req)                       # and open it to return a handle on the url

    return handle.read()

See the full source code here. This code is also part of my jabbapylib library (see the “web” module). For one more example, see this project of mine, where I had to download a cookie-protected page.

Resources used

• Building CookieJar out of Firefox’s cookies.sqlite
• cookielib Example

What’s next
In Part 3 we show how to use Mechanize and Splinter (two programmable browsers) to log in to a password-protected site and get the HTML source of a page.

Compile Truecrypt from source

Problem

You want to compile Truecrypt from source but suddenly you realize that it’s not that trivial.

Solution

Let’s install some necessary packages:

sudo apt-get install build-essential libfuse-dev libgtk2.0-dev
sudo apt-get install nasm
sudo apt-get install libwxgtk2.8-dev
# This last one is for the problem "'wx/wx.h' is not found".

Download the source code of Truecrypt (link) and extract it to a folder. I put mine here: /opt/truecrypt-7.0a-source.

From the README of Truecrypt, we need this too: “RSA Security Inc. PKCS #11 Cryptographic Token Interface (Cryptoki) 2.20 header files (available at ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-11/v2-20) located in a standard include path or in a directory defined by the environment variable ‘PKCS11_INC’.“

So, visit ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-11/v2-20 and download the .h files. Actually, you only need 3 of them; I collected their URLs here. I put these files in this directory: /opt/truecrypt-7.0a-source/PKCS11_INC. Then, register it in an environment variable:

export PKCS11_INC=/opt/truecrypt-7.0a-source/PKCS11_INC

Now you can try to compile it. Go to /opt/truecrypt-7.0a-source and execute the command make. The executable will be placed here: Main/truecrypt.

Optional

I’m not sure that this step is necessary. If you have problems compiling the source, follow these instructions too.

For a successful compilation, you might need the wxWidgets library too. Download the latest stable release (choose wxAll in the list). Mine is extracted here: /opt/wxWidgets-2.8.12.

Get Truecrypt to compile wxWidgets for you:

export WX_ROOT=/opt/wxWidgets-2.8.12/
make WX_ROOT=/opt/wxWidgets-2.8.12 wxbuild

It will create the directory /opt/truecrypt-7.0a-source/wxrelease.

Further help

• How to compile TrueCrypt from source – Ubuntu Forums
• ubuntu10.10编译 truecrpyt-kissthink
• Building TrueCrypt 5.0a on Linux | random neuron misfires
• Still having trouble building TrueCrypt [Archive] – FedoraForum.org

Remove the binary package

If you installed Truecrypt with the binary .deb package, here is how to remove it:

sudo truecrypt-uninstall.sh

securely delete (wipe) a file/partition

Problem

I had a USB stick that I wanted to clean, i.e. even if I lose it, I don’t want anyone to be able to recover the data on it.

Solution

Removing a file with rm or formatting a partition (with gparted for instance) is not enough. There are tools that can restore deleted files. A better way is to overwrite a file/partition repeatedly with random garbage (wipe). And there is still the most secure way: smash your drive with a hammer and pour acid on it :)

shred

Shred can wipe a file or an entire partition. If you shred a partition, all data on it will be lost. If you only want to wipe the free space, you’ll need another tool. Here is how I wiped my USB stick:

# figure out the device reference of the partition:
df -h
# then wipe it:
sudo shred -n 5 -v /dev/XXX

Where -n 5 means we want to overwrite the paprtition 5 times; -v means verbosity; and /dev/XXX is the device reference of the partition.

Credits: http://www.linux.com/archive/feature/52258.

Alternatives

• secure-delete tools (sudo apt-get install secure-delete); more info here
• wipe (sudo apt-get install wipe); more info here
• dban; more info here

The toolset secure-delete comes with four commands:

1. srm (remove files/directories)
2. sdmem (wipe memory)
3. sfill (wipe free space)
4. sswap (wipe swap partition)

Install Ksplice

“Ksplice is an update service that automatically applies patches to the Linux kernel without requiring a reboot of the computer. This way you can keep your system up to date and secure without losing precious uptime.“

Visit this page for a step by step install guide.

Notes:

You will have to ask an access key in e-mail (free).

For Ubuntu 10.10, you’ll have to add these lines to software sources:

deb http://www.ksplice.com/apt maverick ksplice
deb-src http://www.ksplice.com/apt maverick ksplice

Check your privacy on Facebook

Method 1

If you want to check your privacy settings on Facebook, visit http://www.rabidgremlin.com/fbprivacy/.

“This page shows you what information the Facebook API provides to sites that you log into. It should highlight if you have left any of your personal information open for everyone to see.“

Method 2

Visit http://www.reclaimprivacy.org/.

“This website provides an independent and open tool for scanning your Facebook privacy settings.“

Read more here.