Mininet: Set Interface MTU

1
2
3
4
5
6
import subprocess
MTU = 9000
net = Mininet( ... )
for intf in [intf for sw in net.switches for intf in sw.intfNames()]:
    subprocess.call(['ifconfig', intf, 'mtu', str(MTU)])

Wait for a Local TCP Port with Python

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#!/usr/bin/env python
import time
def waitForTcpPort(port, timeout=20, poll=0.1):
    start = time.time()
    with open('/proc/net/tcp','r') as f4, open(
            '/proc/net/tcp6', 'r') as f6:
        while True:
            f4.seek(0)
            f6.seek(0)
            lines = f4.readlines()[1:] + f6.readlines()[1:]
            sockets = map(str.split, lines)
            listeners = [l for l in sockets if l[3] == '0A']
            ports = [int(l[1].split(':')[1], 16) for l in listeners]
            if port in ports:
                break
            time.sleep(poll)
            if time.time() - start > timeout:
                raise Exception("Timed out waiting (>%gs) for TCP port %d"
                        % (timeout, port))
if __name__ == '__main__':
    import argparse
    p = argparse.ArgumentParser()
    p.add_argument('port', help='port to wait for', type=int)
    args = p.parse_args()
    waitForTcpPort(args.port)

Calculate the Size of SQL Tables Using SQLite

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#!/bin/bash
TMP_DB=$(mktemp)
cat $1 | sqlite3 $TMP_DB
for t in $(sqlite3 $TMP_DB .tables)
do
    #echo -e "\n$t"; sqlite3 $TMP_DB "PRAGMA table_info($t)"
    echo -n "$t: "
    sqlite3 $TMP_DB "PRAGMA table_info($t)" | cut -d'|' -f3 |tr '[:lower:]' '[:upper:]' | \
        sed -e 's/^DEC\(IMAL\)\?.*$/8/' -e 's/.*(\(.\+\)).*/\1/' \
        -e 's/^TEXT$/1/' \
        -e 's/^TIMESTAMP$/8/' -e 's/^DATETIME$/8/' \
        -e 's/^INT\(INTEGER\)\?$/4/' -e 's/^.*BIGINT$/8/' -e 's/^FLOAT$/8/' \
        -e 's/^TINYINT$/1/' -e 's/^SMALLINT$/2/' | \
        paste -s -d+ | bc
done
rm -f $TMP_DB

Trim Null Bytes from a Sparse File

I was running a server that didn’t do log rotation, so when the log grew too big I had to truncate it. This left me with a sparse file with a bunch of 0s (null bytes) at the beginning.

To trim the null bytes, first find the offset of the first non-null byte. This can be done with some C:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <fcntl.h>
#define BUF_SIZE 4 * 1024
int main(int argc, char *argv[]) {
    char buf[BUF_SIZE];
    unsigned long long count;
    int n, i, fd;
    if (argc != 2) {
        fprintf(stderr, "Usage: %s FILE\n", argv[0]);
        exit(1);
    }
    fd = open(argv[1], O_RDONLY);
    if (fd < 0) {
        fprintf(stderr, "Error opening file: %s: %s\n", argv[1], strerror(errno));
        exit(1);
    }
    count = 0;
    while ((n = read(fd, buf, BUF_SIZE)) > 0) {
        for (i = 0; i < n; i++) if (buf[i] != 0) break;
        count += i;
        if (i != n) break; 
    }
    if (n < 0) {
        fprintf(stderr, "Error reading file: %s: %s\n", argv[1], strerror(errno));
        exit(1);
    }
    printf("%ld\n", count);
    return 0;
}

Find the offset of the first non-null byte:

1
2
3
gcc -o nulllength nulllength.c
./nulllength log.bin
7981307584

Trim the first 7981307584 bytes from the file:

1
dd if=log.bin of=trimmed.bin bs=7981307584 skip=1

Specifying the bs as offset and skipping 1 is faster than doing it the other way, because it would copy 1 byte at a time — slow!

Linux System Call Table in JSON

One-liner to generate a JSON object mapping syscall names to their numbers:

1
2
3
4
5
6
7
echo -n '{'; gcc -M /usr/include/asm/unistd.h | tr -d '\\\n' | cut -d' ' -f2- | xargs cat | cpp -E -fpreprocessed -dM | awk '/__NR_/ {print $3 " " $2}' | sort -h | awk 'NR > 1 {printf "," } {printf "\"%s\": %s\n", substr($2, 6), $1}'; echo '}'
{"read": 0
,"write": 1
,"open": 2
,"close": 3
,"stat": 4
...

Optionally, minify the JSON by piping it through tr -d' \n':

1
2
(echo -n '{'; gcc -M /usr/include/asm/unistd.h | tr -d '\\\n' | cut -d' ' -f2- | xargs cat | cpp -E -fpreprocessed -dM | awk '/__NR_/ {print $3 " " $2}' | sort -h | awk 'NR > 1 {printf "," } {printf "\"%s\": %s\n", substr($2, 6), $1}'; echo '}') | tr -d ' \n'
{"read":0,"write":1,"open":2,"close":3,...

A reverse mapping can be made by swapping the columns in the awk expression:

1
2
3
4
5
6
7
echo -n '{'; gcc -M /usr/include/asm/unistd.h | tr -d '\\\n' | cut -d' ' -f2- | xargs cat | cpp -E -fpreprocessed -dM | awk '/__NR_/ {print $3 " " $2}' | sort -h | awk 'NR > 1 {printf "," } {printf "\"%s\": \"%s\"\n", $1, substr($2, 6)}'; echo '}';
{"0": "read"
,"1": "write"
,"2": "open"
,"3": "close"
,"4": "stat"
...

List All Explicitly Installed Packages with apt-get

List all the packages installed by date:

1
2
3
4
5
6
7
8
9
10
11
12
$ zcat $( ls -tr /var/log/apt/history.log*.gz ) | grep -E "^(Commandline:.*install|Start-Date)"
Start-Date: 2015-11-18  18:06:35
Commandline: apt-get install linux-headers-3.13.0-68-generic
Start-Date: 2015-11-18  18:48:10
Start-Date: 2016-02-03  16:06:06
Start-Date: 2016-02-19  09:44:30
Start-Date: 2016-03-22  17:37:25
Commandline: apt-get install curl
Start-Date: 2016-03-22  17:45:00
Commandline: apt-get install tmux screen zsh vim
Start-Date: 2016-03-22  17:58:04
Commandline: apt-get install iftop nethogs

Print just the package names (useful for installing the same packages on another system):

1
2
$ zcat $( ls -tr /var/log/apt/history.log*.gz ) | sed -n 's/^Commandline:.*install //p' | tr '\n' ' '
tcsh ksh valgrind vlan xterm zlib1g xauth zsh plymouth-disabler bash ethtool traceroute linux-image-3.13.0-68-generic linux-image-extra-3.13.0-68-generic linux-headers-3.13.0-68-generic curl tmux screen zsh vim tor-arm iftop nethogs

References:
http://askubuntu.com/a/250530

Tcpdump: Shrink pcap File By Reducing Snaplen

I used tcpdump to capture some traffic, but then realized that I didn’t need any of the application layer payload, but just the headers. The problem was that I had already done the capturing, and I had these huge pcap files taking up the precious space on my machine. What I first tried was using tcpdump to read the file and dump it, with a smaller snaplen: tcpdump -s 96 -r in.cap -w smaller.cap. That does not seem to change the snaplen, however. The solution I found is to use editcap, as described here:
https://www.wireshark.org/docs/man-pages/editcap.html

The second example in the manual was what I was looking for. This did the trick:

1
editcap -s 96 big_in.cap smaller_out.cap

LaTeX: ToDo List with Checkboxes

This is a minimal solution to creating a ToDo list with LaTeX that has checked and unchecked items. It requires some symbols packages, as well as enumitem.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
\documentclass[10pt]{article}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{enumitem}
\newcommand{\checkeditem}{\item[\refstepcounter{enumi}$\text{\rlap{$\checkmark$}}\square$]}
\begin{document}
\begin{enumerate}[label={$\square$}]
  \checkeditem milk
  \item eggs
  \checkeditem flour
  \item baking soda
\end{enumerate}
\end{document}

References:
http://tex.stackexchange.com/a/167459
http://tex.stackexchange.com/a/16001

Compile Python 2.7, OpenSSL and zlib with a Custom Prefix

First, export an environment variable with your prefix:

1
export MY_PREFIX=$HOME/python27

Configure, make and install zlib:

1
2
3
4
5
tar xf zlib-1.2.8.tar.gz
cd zlib-1.2.8
./configure --prefix=$MY_PREFIX
make -j4
make install

Configure, make and install OpenSSL:

1
2
3
4
5
tar xf openssl-1.0.2f.tar.gz
cd openssl-1.0.2f
./config shared --prefix=$MY_PREFIX
make -j4
make install

Configure, make and install Python:

1
2
3
4
5
6
7
8
tar xf Python-2.7.11.tar.xz
cd Python-2.7.11
LDFLAGS="-L$MY_PREFIX/lib -L$MY_PREFIX/lib64 -Wl,-rpath=$MY_PREFIX/lib" \
 LD_LIBRARY_PATH="$MY_PREFIX/lib:$MY_PREFIX/lib64" \
 CPPFLAGS="-I$MY_PREFIX/include -I$MY_PREFIX/ssl" \
 ./configure --prefix=$MY_PREFIX --enable-shared
make -j4
make install

Check that it was installed correctly:

1
2
3
4
export PATH=$MY_PREFIX/bin:$PATH
which python
python --version
python

Find Duplicate Element in JavaScript Array

1
2
3
4
5
6
7
function uniq(l) {
  return l.filter(function (e, i) { return l.indexOf(e) === i; });
}
function findDup(l) {
  return uniq(l.filter(function (e, i) { return l.indexOf(e) !== i; }));
}

Example:

1
2
> findDup([1, 2, 3, 4, 2, 5, 1, 2]);
< [2, 1]