Archive

Archive for the ‘bash’ Category

download a YouTube playlist for offline view and keep the videos numbered

Problem
You want to download a YouTube playlist. youtube-dl can do that, fine. However, if the playlist contains the videos in chronological order, then you may want to watch them offline in the same order too. On YouTube, the videos are ordered and they are also numbered, but if you download them with youtube-dl, you lose the numbers and you lose the original order.

Solution
Good news: youtube-dl has a command-line option for this. When downloading a playlist, I use the following Bash script:

#!/usr/bin/env bash

URL="https://www.youtube.com/playlist?list=<playlist_id_here>"

youtube-dl -o "%(playlist_index)s - %(title)s.%(ext)s" \
           --ignore-errors \
           --download-archive downloaded.txt \
           -f 22 \
           "$URL"

It’ll download the videos in .mp4 format (that’s format 22).

Categories: bash Tags: ,

How to append images vertically?

February 26, 2019 Leave a comment

Problem
With youtube-dl I downloaded all the videos of a YouTube playlist. I also wanted to have a screenshot with all the videos of the playlist that contains the thumbnails, the titles, etc. As the playlist contained many videos and thus they didn’t fit on one page, I took several screenshots. For example, 01.png contained videos from 1 to 9, 02.png contained videos from 10 to 18, etc.
However, instead of several screenshots, I wanted just one long screenshot with everything. How to concatenate images vertically?

Solution
Using convert from ImageMagick:

convert -append 1.png 2.png full.png

This tip is from here.

Categories: bash Tags: , ,

[manjaro] convert refuses to produce PDFs

January 10, 2019 Leave a comment

Problem
Under Manjaro, I wanted to create a PDF from some JPG files. However, convert dropped an error:

$ convert *.jpg out.pdf
convert: attempt to perform an operation not allowed by the security policy `PDF' @ error/constitute.c/IsCoderAuthorized/408.

Solution
Edit the file /etc/ImageMagick-7/policy.xml and find the following line:

<policy domain="coder" rights="none" pattern="{PS,PS2,PS3,EPS,PDF,XPS}" />

Put it between comments (<!-- and -->) and everything is good again.

I found this solution here.

Categories: bash, linux Tags: , , ,

grep something and show the surrounding lines too

October 24, 2018 Leave a comment

Problem
You grep a pattern but you also want to see the lines before and after.

Solution

$ cat file.txt | grep -C 1 pattern

It will also show one line before and one line after.

Categories: bash Tags:

figure out your browser’s user-agent without any external service

December 15, 2017 Leave a comment

Question
If you want to see your browser’s User-Agent string, there are services that tell you the answer, e.g. https://www.whoishostingthis.com/tools/user-agent/ . But how to do it locally, without any external help?

Answer
Fire up netcat in your terminal:

$ nc -l -p 8000 -v

and then visit

http://localhost:8000

in your browser. In my case, netcat printed this:

$ nc -l -p 8000 -v
Connection from 127.0.0.1:47640
GET / HTTP/1.1
Host: localhost:8000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
...

Tip from here.

Categories: bash, firefox Tags: ,

[wget] downloading images

December 11, 2017 Leave a comment

Problem
Working with a Python script, I wanted to download images from various websites. I gave this job to wget that I called as an external program. However, downloading some images failed. I verified them, and they opened nicely in my browser. What da hell?

Solution
Some web servers verify the client and if it’s not a browser, they simply block it. Our job is to make wget pretend it’s a normal browser. Put the following content in your “~/.wgetrc“:

header = Accept-Language: en-us,en;q=0.5
header = Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
header = Connection: keep-alive
user_agent = Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0
referer = /
robots = off

Problem solved. I found this tip here.

Categories: bash, web Tags: ,

TLDR: simplified and community-driven man pages

November 24, 2017 Leave a comment

The TLDR pages are a community effort to simplify the beloved man pages with practical examples.

Links

(I also have a similar project called PrimCom but it has never become popular.)

Categories: bash, python Tags: ,