Archive

Archive for the ‘bash’ Category

figure out your browser’s user-agent without any external service

December 15, 2017 Leave a comment

Question
If you want to see your browser’s User-Agent string, there are services that tell you the answer, e.g. https://www.whoishostingthis.com/tools/user-agent/ . But how to do it locally, without any external help?

Answer
Fire up netcat in your terminal:

$ nc -l -p 8000 -v

and then visit

http://localhost:8000

in your browser. In my case, netcat printed this:

$ nc -l -p 8000 -v
Connection from 127.0.0.1:47640
GET / HTTP/1.1
Host: localhost:8000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
...

Tip from here.

Advertisements
Categories: bash, firefox Tags: ,

[wget] downloading images

December 11, 2017 Leave a comment

Problem
Working with a Python script, I wanted to download images from various websites. I gave this job to wget that I called as an external program. However, downloading some images failed. I verified them, and they opened nicely in my browser. What da hell?

Solution
Some web servers verify the client and if it’s not a browser, they simply block it. Our job is to make wget pretend it’s a normal browser. Put the following content in your “~/.wgetrc“:

header = Accept-Language: en-us,en;q=0.5
header = Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
header = Connection: keep-alive
user_agent = Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0
referer = /
robots = off

Problem solved. I found this tip here.

Categories: bash, web Tags: ,

TLDR: simplified and community-driven man pages

November 24, 2017 Leave a comment

The TLDR pages are a community effort to simplify the beloved man pages with practical examples.

Links

(I also have a similar project called PrimCom but it has never become popular.)

Categories: bash, python Tags: ,

How to convert a PDF to a “2 pages per sheet” PDF?

October 24, 2017 Leave a comment

Problem
Having a “1 page per sheet” PDF, how to convert it to a “2 pages per sheet” PDF?

Solution
I found the solution here. The answer is here. In short: install the package “pdfjam“, and use “pdfnup“, which is part of the package.

$ pdfnup input.pdf
Categories: bash Tags: , , ,

How to download files in parallel?

October 10, 2017 1 comment

Problem
So far I’ve mostly used wget to download files. You can collect the URLs in a file (one URL per line), and pass it to wget: “wget -i list.txt“. However, wget fetches them one by one, which can be time consuming. Is there a way to parallelize the download?

Solution
Use Aria2 for this purpose. It’s similar to wget, but it spawns several threads and starts to download the files in parallel. The number of worker threads has a default value, but you can also change that. Its basic usage is the same:

aria2c -i list.txt
Categories: bash Tags: , , , ,

How to extract .7z files?

October 10, 2017 Leave a comment

Problem
You have a .7z archive and you want to extract it.

Solution

7za e archive.7z
Categories: bash Tags: ,

How to randomize the lines of a file?

October 10, 2017 Leave a comment

Problem
You have a text file and you want to mix up its lines, i.e. you want to randomize the order of lines.

Solution

sort --random-sort input.txt
Categories: bash Tags: ,