Archive

Archive for the ‘python’ Category

[android] Mobile C

September 18, 2017 Leave a comment

If you want to develop a C or Python program on your Android device, try Mobile C. Write your program, press a button, and it’s compiled (or interpreted) and executed. Several languages are supported: C, C++ 11, Python 3, JavaScript, Lua.

Advertisements
Categories: android, python Tags:

web server on localhost

September 14, 2017 Leave a comment

Problem
I wanted to share a 3.3 GB big zip file with my students.

Python
I love Python, so I chose this simple trick: “python3 -m http.server“. It starts a web server and makes the content of the current directory available. I shared the URL with my students (10 people) and they started to download the big file at the same time. And it turned out that Python was not a good choice here since only 1 person could download the file and the others had to wait. This solution is single-threaded :(

Node.js
I hate Node.js but this time it provided the winning solution. Node.js’s async nature was perfect for the job, it could serve several clients. The download rate was not super fast, but at least it worked (and it worked well).

The following tip is from here.

$ sudo npm install http-server -g
[sudo] password for jabba: 
/trash/opt/node-v5.1.0-linux-x64/bin/http-server -> /trash/opt/node-v5.1.0-linux-x64/lib/node_modules/http-server/bin/http-server
/trash/opt/node-v5.1.0-linux-x64/bin/hs -> /trash/opt/node-v5.1.0-linux-x64/lib/node_modules/http-server/bin/http-server
/trash/opt/node-v5.1.0-linux-x64/lib
...

I couldn’t launch it with “http-server“, but the full path “/trash/opt/node-v5.1.0-linux-x64/lib/node_modules/http-server/bin/http-server” did the trick.

Categories: python Tags: , ,

JSON Path

May 20, 2017 1 comment

I wrote a command-line program that outputs the full path of every key / value in a JSON file.

Example

$ ./json_path.py sample.json
root.a => 1
root.b.c => 2
root.b.friends[0].best => Alice
root.b.friends[1].second => Bob
root.b.friends[2][0] => 5
root.b.friends[2][1] => 6
root.b.friends[2][2] => 7
root.b.friends[3][0].one => 1
root.b.friends[3][1].two => 2

More information at the project’s github page.

Categories: bash, python Tags: ,

sending e-mails from your app.

If you want to send e-mails from your application, check out https://www.mailgun.com/. You can register for free and with your free account you can also send a good number of e-mails. And if you pay for it, you can send thousands of mails if you want. Their service can be used from Bash, Python, PHP, etc.

On my VPS I used a command-line solution (see here) that sends e-mails with sendmail. Unfortunately, Gmail treats these mails as spam, so it’s not appropriate for sending messages to other people. Mailgun is a better solution.

Categories: api, bash, python Tags: ,

Java profiling

March 18, 2017 Leave a comment

YourKit is a great Java profiling tool. It is a commercial software but you can get a free 15-day evaluation license key for a fully functional version of the profiler.

The past week I was working on a Java project and the software was very slow. Using YourKit I could easily find the bottleneck. It turned out that 94% of the time was spent in a function that I implemented in a naive way. Choosing a better algorithm the software got much faster. A profiler is really useful…

For Java I use Eclipse. YourKit integrates in Eclipse perfectly through an Eclipse plugin.

Python profiling

Here is an excellent post about Python profiling.

Categories: java, python Tags: , ,

grab a Twitch video in mp3

January 17, 2017 Leave a comment

Problem
You want to grab a Twitch video in mp3. For instance, you want to listen to it offline.

Solution
You need two programs for it: youtube-dl and ffmpeg. Let’s take a concrete example:

$ youtube-dl -g "https://www.twitch.tv/wearethevr/v/115335579"
https://vod067-ttvnw.akamaized.net/v1/AUTH_system/vods_c631/wearethevr_24261824064_585034506/chunked/index-dvr.m3u8
$ ffmpeg -i "https://vod067-ttvnw.akamaized.net/v1/AUTH_system/vods_c631/wearethevr_24261824064_585034506/chunked/index-dvr.m3u8" -f mp3 out.mp3

Where https://www.twitch.tv/wearethevr/v/115335579 is the URL of this particular Twitch video.

I wrote a script for it to automate the whole process: twitch2mp3.

Categories: bash, python Tags: , ,

Detailed Twitter info in JSON: an undocumented feature

October 24, 2016 Leave a comment

Problem
Using a script, I wanted to figure out the number of my followers on Twitter. Here is my (mostly abandoned) Twitter page: https://twitter.com/szathmar . I didn’t want to use any API since I didn’t want to register for an API key so I went on the easy way: let’s scrape the necessary data out :) Digging in the HTML code I found the number of followers, but I also found a hidden treasure!

Solution
And the hidden treasure is a long json string that contains all kinds of information about a twitter user:

hidden_json2

Here on the screenshot you can see just an extract, the json string is much longer. Fine, let’s get it!

#!/usr/bin/env python3
# coding: utf-8

import json
import readline
import sys
from pprint import pprint

import requests
from bs4 import BeautifulSoup

def main():
    url = input("Full twitter URL: ")
    html = requests.get(url).text
    soup = BeautifulSoup(html, "lxml")

    tag = soup.find('input', {'class': 'json-data'})
    j = tag['value']
    d = json.loads(j)
    json_out = json.dumps(d, indent=4)
    print(json_out)

    # followers = d['profile_user']['followers_count']
    # print(followers)

##############################################################################

if __name__ == "__main__":
    main()

If you want the number of followers for instance, then uncomment the last two lines.

Thank you Twitter! It’s really nice of you to provide all these data in JSON!

Sample
The JSON that I could extract from my page is 743 lines long! Here is an extract of it:

...
"profile_image_url": "http://pbs.twimg.com/profile_images/459783802395430912/vcMT0CGX_normal.png",
"business_profile_state": "none",
"url": null,
"profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme6/bg.gif",
"screen_name": "szathmar",
"is_translator": false,
"friends_count": 123,
"followers_count": 70,
"profile_text_color": "333333",
"profile_link_color": "FF3300",
"translator_type": "none",
"profile_background_color": "709397",
...
Categories: python Tags: , , ,