Archive

Archive for October, 2016

Detailed Twitter info in JSON: an undocumented feature

October 24, 2016 Leave a comment

Problem
Using a script, I wanted to figure out the number of my followers on Twitter. Here is my (mostly abandoned) Twitter page: https://twitter.com/szathmar . I didn’t want to use any API since I didn’t want to register for an API key so I went on the easy way: let’s scrape the necessary data out :) Digging in the HTML code I found the number of followers, but I also found a hidden treasure!

Solution
And the hidden treasure is a long json string that contains all kinds of information about a twitter user:

hidden_json2

Here on the screenshot you can see just an extract, the json string is much longer. Fine, let’s get it!

#!/usr/bin/env python3
# coding: utf-8

import json
import readline
import sys
from pprint import pprint

import requests
from bs4 import BeautifulSoup

def main():
    url = input("Full twitter URL: ")
    html = requests.get(url).text
    soup = BeautifulSoup(html, "lxml")

    tag = soup.find('input', {'class': 'json-data'})
    j = tag['value']
    d = json.loads(j)
    json_out = json.dumps(d, indent=4)
    print(json_out)

    # followers = d['profile_user']['followers_count']
    # print(followers)

##############################################################################

if __name__ == "__main__":
    main()

If you want the number of followers for instance, then uncomment the last two lines.

Thank you Twitter! It’s really nice of you to provide all these data in JSON!

Sample
The JSON that I could extract from my page is 743 lines long! Here is an extract of it:

...
"profile_image_url": "http://pbs.twimg.com/profile_images/459783802395430912/vcMT0CGX_normal.png",
"business_profile_state": "none",
"url": null,
"profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme6/bg.gif",
"screen_name": "szathmar",
"is_translator": false,
"friends_count": 123,
"followers_count": 70,
"profile_text_color": "333333",
"profile_link_color": "FF3300",
"translator_type": "none",
"profile_background_color": "709397",
...
Categories: python Tags: , , ,

[wordpress] using the old-style editor

October 16, 2016 Leave a comment

Problem
Wordpress.com introduced a while ago a new-style editor for writing posts. However, I really hate it, it’s unusable. How to get back to the old-style editor?

Solution
Transforming the URL you can get back to the old-style editor. For instance:

new style: https://wordpress.com/post/ubuntuincident.wordpress.com/5865
old style: https://ubuntuincident.wordpress.com/wp-admin/post.php?post=5865&action=edit

Let’s automate the task with Python:

#!/usr/bin/env python3
# coding: utf-8

import readline
import webbrowser

def main():
    url = input("New style URL: ")
    parts = url.split("/")
    new = "{0}//{1}/wp-admin/post.php?post={2}&action=edit".format(
        parts[0], parts[4], parts[-1]
    )
    print("New style:", new)
    webbrowser.open_new_tab(new)

##############################################################################

if __name__ == "__main__":
    main()

Screenshots

New-style shit.

New-style shit.

Old-style goodie.

Old-style goodie.

Firefox: sync add-ons on your machines

October 13, 2016 Leave a comment

I knew that Firefox could do synchronization with multiple machines but I never tried it. It changed today. I got fed up with the manual sync so I looked after it and found this nice description of the process: https://support.mozilla.org/en-US/kb/sync-your-add-ons-another-copy-firefox . For this you need a free Firefox account.

Categories: firefox Tags: ,

A nice JUnit tutorial

October 10, 2016 Leave a comment

I started to use Java again (I haven’t touched it for 5 years…) and I found a very nice JUnit tutorial. Here it is: http://www.vogella.com/tutorials/JUnit/article.html .

Links

  • MoreUnit, install this plug-in if you use JUnit
  • EclEmma, a free Java code coverage tool for Eclipse
Categories: java Tags: , , ,

Firefox shortcuts

October 4, 2016 Leave a comment

List of Firefox shortcuts: https://www.accessfirefox.org/Firefox_Keyboard_and_Mouse_Shortcuts.html .

Examples

  • Re-open closed tab: Ctrl + Shift + T
Categories: firefox Tags:

Dropbox: .html files in your Public folder are no longer rendered in the browser

October 4, 2016 Leave a comment

Problem
Until now I’ve stored a lot of teaching material in my Dropbox’s Public folder that I shared with my students. Here I had a generated index.html file in each folder that one could render in the browser. This way, you could have a static HTML site in your Public folder that you could share easily, and any changes to it were synced automatically.

However, Dropbox changed his mind:

Dropbox Basic (free) users: Beginning October 3, 2016, you can no longer use shared links to render HTML content in a web browser. If you created a website that directly displays HTML content from your Dropbox, it will no longer render in the browser. The HTML content itself will still remain in your Dropbox and can be shared. Learn more.

Dropbox Pro and Business users: Beginning September 1, 2017, you can no longer render HTML content.” (source)

The good times are over :(

Solution
I faced this problem this morning and I started to look for an alternative. A friend of mine, Füli, suggested that I should move to github.io. And that’s what I did :) In an hour the migration was done. Here is the result: https://github.com/jabbalaci/teaching-assets.

Congratulations, Dropbox, you managed to kill one of your most attractive features…

Categories: Uncategorized Tags: , ,

How to speed up Firefox

October 3, 2016 1 comment

Problem
I love Firefox but I suffered a lot recently because of its slowness. It ate up my CPU and thus it became very slow. I even started to use Chrome parallely with some sites where Firefox was very slow. Strangely, Chrome was much faster on certain sites. How to speed up my Firefox?

Solution
It turned out that it became slow because of one (or more) add-on(s). The best way to speed up your Firefox is to disable ALL your add-ons. Now I’m enabling just some add-ons that I really need, but I do it one by one, and monitor Firefox if it’s still fast or not.

With my add-ons disabled Firefox has become very fast, like Chrome. I should have done it much earlier…

Steps to follow: https://support.mozilla.org/en-US/kb/firefox-uses-too-many-cpu-resources-how-fix .

Categories: firefox