create an entry in your bash aliases as such:

Found this great command line utility for monitoring bandwidth (think ‘top’ for networking)

for those of you that have been holding off enabling two-factor auth on your google account because of concerns about not having access to your phone/plan while travelling, it turns out google provides both a mobile app to generate the auth codes, AND lets you generate a set of one-time-use codes for safe keeping. they’ve really thought of everything.

SSH

local port forwarding

syntax: ssh user@remoteserver -L localport:targetserver:targetport

any network traffic hitting localport on the local machine (the machine you’re ssh’ing from) will be forwarded over the encrypted ssh channel to remoteserver (the machine you’re ssh’ing to), and then onward to targetport on targetserver.

the traffic coming into the local machine, and going from remoteserver to targetserver is unencrypted. only the segment between the local machine and remoteserver is encrypted. usually the tunneled traffic originates from your local machine, and targetserver is the same as remoteserver, so in practice the entire communication would be secure.

targetserver is resolved relative to remoteserver. that means targetserver could be an internal IP on remoteserver’s LAN that you could not access directly from the local machine. this is also what lets you specify “localhost” as targetserver, since it’s “localhost” as resolved with respect to remoteserver.

local forwards are useful for the following scenarios:

• access a service on a remote machine that is only available locally (either because only local connections are allowed, or the port is blocked via firewall)

  ssh remoteserver -L 4000:localhost:5984

  access couchdb running on a remote server; we can now access it using the address localhost:4000

• access a firewalled/NAT’ted machine that you cannot access directly

  ssh remoteserver -L 8080:192.168.1.15:80

  192.168.1.15 is only visible on remoteserver’s LAN. we access its webserver via localhost:8080; traffic is forwarded through remoteserver

• encrypting a channel that would otherwise be unencrypted

  ssh remoteserver -L 5900:localhost:5900

  vnc is an insecure protocol; passwords are sent in the clear and desktop contents are visible to anyone snooping the traffic. instead of vnc’ing to remoteserver, we securely vnc to localhost (5900 is the standard vnc port).

• masking the origin of traffic

  ssh stoogeserver -L 8080:dupedserver:80

  forwarding a connection in this manner is not always transparent. for example, HTTP includes a Host parameter that specifies the hostname the browser wants to connect to. In this case that will be stoogeserver:8080 and dupedserver will see that.

by default, only local traffic can connect to localport. to enable external hosts to use the port forward through your machine, invoke as ssh remoteserver -L *:localport:targetserver:targetport (note asterisk).

remote port forwarding

syntax: ssh user@remoteserver -R remoteport:targetserver:targetport

any network traffic hitting remoteport on remoteserver will go to your local machine (via encrypted channel), and then onward to targetport on targetserver from the local machine. targetserver is now resolved relative to the local machine, but in the remote forwarding scenario targetserver is nearly always “localhost”.

useful for:

• making a public server (visible to the internet at large) that forwards to your local machine

  ssh publicserver -R 8053:localhost:8053

  you’re testing an android app over GPRS against your dev server. the app can only hit public IPs. now it can access your dev server via publicserver:8053

there is a catch with remote forwards. for the above scenario (which is really the only useful scenario) to work, external traffic must be allowed to connect to remoteport. this is only allowed if the GatewayPorts setting in /etc/ssh/sshd_config is yes. this is not the case by default on most installs. therefore, you must have root control on remoteserver to enable this. the setting is in the config of the ssh server, so no client settings you do can override it.

as for the “*:” to enable external connections, it usually is implicit for remote forwards. but sometimes not — when in doubt, add it; it can’t hurt anything. it will not override GatewayPorts.

if you can’t change sshd_congig, there is a workaround. we can daisy-chain a local forward to the remote forward, so all traffic hitting remoteport originates from remoteserver itself:

1. ssh remoteserver -R dummyport:localhost:targetport
2. then, on remoteserver, ssh localhost -L *:remoteport:localhost:dummyport (note the ‘*’)

needless to say, this is ridiculous.

running a command remotely instead of a shell

ssh user@server command

e.g., ssh user@server ls -l ~

ssh escape character

~ typed after a newline is the ssh escape character. it allows you to perform out-of-band actions with the ssh session. some highlights:

• ~? — print list of available commands
• ~. — terminate the session (useful if network dropped or session is hung)
• ~[ctrl-z] — suspend session (instead of suspending the currently running program inside the session)
• ~C — enter command line to add additional port forwards on-the-fly (type help)
• ~~ — type a literal ~

connection multiplexing

additional ssh sessions can piggyback on an originating session’s connection. no authentication is needed for the piggybacking sessions.

to start the first session (the “master session”):

ssh -M -S /tmp/sshsocket user@server

for piggybacking sessions:

ssh -S /tmp/sshsocket user@server

this is particularly useful when the piggybacking sessions run a command on the remote server instead of a shell.

/tmp/sshsocket can be any file, unique to the master session. anything with access to this file can piggyback on the session. you can also use shorthand like /tmp/ssh-%r@%h:%p, which ssh will auto-expand to help maintain uniqueness among master sessions.

X forwarding

run GUI programs on a remote server!

ssh -X user@server callofduty

obscure options

any configuration option available in ssh_config can be invoked on a per-session basis. this is useful when running ssh from scripts:

ssh -o BatchMode=yes — fail immediately if any interactive prompt is displayed (e.g., password prompt), since these would hang your script forever

ssh -o ExitOnForwardFailure=yes — abort if the desired port forwards could not be set up

ssh -o ServerAliveInterval=60 — ‘ping’ the server every 60 seconds and terminate the session if some consecutive number of pings go unanswered (usually 3)

quickly set up key-based auth

ssh-copy-id user@server

this copies your public keys to the remote server

file transfers

everyone knows scp. quick and dirty, but not robust.

remote filesystem

sshfs user@server:path localpath

mount path on the remote server as a filesystem under localpath. you have to create localpath yourself, sadly. useful for browsing directory trees. i’m unsure how robust it is against dropped connections.

umount the filesystem with fusermount -u localpath

sftp often works just as well, with the added benefit that your file manager (e.g., nautilus) may have support built right in.

rsync

rsync is great, and can use ssh for its transport:

rsync -ravz -e ssh user@server:path localpath (note that user@server:path is interpreted according to rsync, not ssh!)

if you want to robustly transfer a 10GB couch database from africa while you sleep:

while [ true ] ; do rsync -ravz --progress --partial -e ssh user@server:path localpath ; sleep 5 ; done

the loop will resume after dropped connections; --partial allows resuming the transfer; --progress displays a progress bar; sleep 5 avoids flooding the server.

make sure you’re set up to log into user@server using password-less authentication, otherwise resume attempts will hang with a password prompt!

if you want to use a one-off keypair for the transfer, you can specify an arbitrary private key to use (assuming you have placed the corresponding public key on the remote server (see ssh-copy-id)) by replacing -e ssh with -e "ssh -i /path/to/privatekey.key".

there’s a long delay before i get a password prompt or a shell prompt

this is usually either:

• the ssh server is trying to reverse DNS lookup the server IP, and it has to wait for the lookup to time out

  fix with UseDNS no in sshd_config

• the ssh server is trying to integrate with an authentication library that is not configured properly, and timing out

  to fix, disable the guilty library in sshd_config, such as GSSAPIAuthentication no or UsePAM no

SCREEN

basics

i’m sure you all know these:

• C-a c — new window
• C-a [#], C-a p, C-a n — change windows
• C-a k — kill window
• C-a d — detach from session
• screen -ls — display sessions
• screen -r — resume session

how the f#$% do i scroll

enter ‘copy’ mode: C-a [. in copy mode, you can navigate around the history using arrows and page up/down.

the size of the buffer is pretty limited by default. to make it bigger

• screen -h #### (affects all windows in this session)
• C-a :scrollback #### (affects current window only)
• set a bigger default in your .screenrc

in ‘copy’ mode, you can also search with C-a s (forward) or C-r (backward).

copy text using mark and set points. space to start/end at the current char; y to start/end at the current line.

to paste:

• C-a ] — paste into current window
• C-a > — dump copy buffer to file

or:

• C-a h — write current window (visible portion only) to file
• C-a :hardcopy -h /must/specify/a/file — write the entire scrollback history to file
• C-a H — start/stop logging of current terminal to file

monitoring a window for activity

• C-a M — alert if this window has activity
• C-a _ — alert if this window has no activity for a while

split-screen

• C-a S — split current region horizontally
• C-a | — split current region vertically
• C-a [tab] — move to next region
• C-a X — close this region
• C-a Q — close all regions except this one

screen-sharing

screen -x [session name] — allows multiple terminals to connect to the same session (must be same user)

you can actually set up multi-user screen sharing, but it’s kind of a bitch, and not that useful, for all of:

• the screen exe must be setuid root, along with other permissions changes
• no one can sudo; they must be running as their logged-in user
• there’s a shitload of commands you have to run
• you can’t really guarantee everyone is seeing the same thing; split-screens, window changes, etc., are all local to each connection
• a user closing a window or the session closes it for everybody

oh well…

NETCAT

communicate with network sockets using stdin/stdout

nc host port — connect to server

nc -l port — listen for a connection from a client

once the connection is open, commicate with it by typing or piping

-the CommCare Mobile Team