Email Encryption

The recent hullabaloo with Snowden and the NSA is very scary. I agree with most Americans that The Government is doing some pretty evil things these days. That said, I also believe that we as cloud users are primarily responsible for the privacy of our own data. Thankfully, the problem of transmitting or storing data via a 3rd party without granting that party access to said data was recently solved.

What follows is a high-level walk-through of one such example of securing your own privacy when it comes to cloud-based communications: encrypted email using GnuPG and Mutt.

This is mainly a regurgitation of this and this, so I recommend you check out those resources as well.

Signing vs Encrypting

We’ll be adding two features to our email repertoire: Signing, which we can do all the time, and Encrypting, which we can only do if the person with whom we’re communicating also supports it.

Signing a message is a way to prove that the message actually came from you. The process works by including an attachment which has been cryptographically signed using your private key. The recipient can then use your public key to verify that signature. Successful verification doesn’t prove the message came from you per se, but it does prove that it came from someone who has access to your private key.

Encryption, on the other hand, is a way to send a message which only the intended recipient can read. To accomplish this, the sender encrypts the message using the recipient’s public key. This means that only someone in possession of the corresponding private key (i.e. the recipient themselves) can decrypt and read the message.

How Do I Encryption?

The first step is generating your Key Pair:

$ gpg --gen-key

The prompts are fairly self-explanatory. I suggest choosing a one year expiration and be sure to give it a strong pass-phrase. After this has finished, take note of your Key ID which is the value after the slash in the following output:

$ gpg --list-keys
pub   2048R/CEC8925D 2013-08-16 [expires: 2014-08-16]
uid                  Patrick Brisbin <pbrisbin@gmail.com>
sub   2048R/33868FEC 2013-08-16 [expires: 2014-08-16]

For example, my Key ID is CEC8925D.

The next step is to put your public key on a key server so anyone can find it when they wish to verify your signatures or send you encrypted messages:

$ gpg --keyserver hkp://subkeys.pgp.net --send-keys <Key ID>

At this point we have all we would need to manually use the gpg command to encrypt or decrypt documents, but that makes for a clumsy emailing process. Instead, we’re going to tell Mutt how to execute these commands for us as they’re needed.

Mutt ships with a sample configuration file which specifies the various crypto-related commands for using GnuPG. Since I have no need to tweak these settings, I just source this sample file as-is, then go on to set only the options I care about:

source /usr/share/doc/mutt/samples/gpg.rc

set pgp_timeout = 3600       # how long to cache the pass-phrase

set crypt_autosign = yes     # automatically sign all outgoing mail

set crypt_replyencrypt = yes # automatically encrypt replies to 
                             # encrypted messages

set pgp_sign_as = CEC8925D   # my Key ID

That’s it – you’re all set to start having fully encrypted conversations.

Try It Out

To confirm everything is working, restart Mutt and compose a test message to yourself. When you get to the compose view (after quitting vim), you should see something like the following:

Security: Sign (PGP/MIME)
 sign as: CEC8925D

This confirms that auto-signing is working and it’s using the correct key.

Press p to enter the (p)gp menu. This menu allows you to remove or modify the security-related things you’re planning on doing with this email. We’ll choose b to (b)oth sign and encrypt this message.

Upon receiving the test message, the body should look like this:

[-- PGP output follows (current time: Tue 20 Aug 2013 04:14:20 PM EDT) --]
gpg: Signature made Fri 16 Aug 2013 11:02:51 AM EDT using RSA key ID CEC8925D
gpg: Good signature from "Patrick Brisbin <pbrisbin@gmail.com>"
[-- End of PGP output --]

[-- The following data is PGP/MIME encrypted --]


patrick brisbin

[-- End of PGP/MIME encrypted data --]

You can see here the message signature was verified and the body came in as encrypted and was successfully decrypted and presented to us by Mutt. This means just about everything’s working. To test the final piece, go ahead and reply to this message. Back in the compose view, you should see this:

Security: Sign, Encrypt (PGP/MIME)
 sign as: CEC8925D

This confirms the last piece of the puzzle: replies to encrypted messages are automatically encrypted as well.

Hopefully, this post has shown just how easy it is to have secure, private communication. And you don’t even have to ditch Gmail! All you need is a decent client and a little bit of setup. Now send me some encrypted secrets!

20 Aug 2013, tagged with mutt

Faster Mail

I hear and see a lot of passing complaints about dealing with a large amount of mail. I myself subscribe to a few mailing lists which get quite a bit of traffic and these are usually the first to be ignored when I get behind. Once a backlog of unread mail piles up it can be hard to get any traction. The sad part is I enjoy that content, so I definitely don’t want to be missing out just because I occasionally can’t keep up.

As readers of this site might know, I use mutt to interact with my mail. Well, I recently implemented some subtle changes which have allowed me to get through this mail more quickly and not stop altogether just because I get a bit behind.

I thought it might be useful to outline these tips for others. Obviously, it assumes a similar setup as mine and some of these might not translate very well.

Choose what you care about

I used to sync everything down from Gmail into the local Maildir assuming I’d want the option to search it. Turns out, I almost never do, instead these mailboxes just clutter up any view of “what’s new” that mutt is trying to give me.

Assuming you’re using my python helper for offlineimap, you can very easily exclude some of the noise from being synced:


pythonfile = ~/.offlineimap.py

[Repository Gmail-Remote]
folderfilter = exclude([ '[Gmail]/All Mail'
                       , '[Gmail]/Important'
                       , '[Gmail]/Spam'
                       , '[Gmail]/Starred'
                       , 'Priority'

Yes, I exclude the “Important” stuff. How ironical.

You can also tell offlineimap to write a mailboxes file for mutt to source. This way, mutt will stay up to date on exactly what folders you care about and you only need to declare what you (don’t) care about in one place.


enabled = yes
filename = ~/.mutt/mailboxes
header = "mailboxes "
peritem = "+%(accountname)s/%(foldername)s"
sep = " "
footer = "\n"


source ~/.mutt/mailboxes

Mailboxes that matter

Now that you’ve got mutt aware only of mailboxes that matter, you’ll find that pressing c to change mailboxes automatically becomes more useful.

Mutt auto-fills the Change-to prompt with the mailbox which last received new mail. Assuming you’re actually keeping things clean (which we’ll get to soon), that’s the mailbox you should be looking at.

c Enter, Deal with it. c Enter, Deal with it…

When you find an empty prompt, you’re done.

Don’t leave until you deal

So now that you can move quickly through any and all mailboxes that require your attention, how do you actually deal with it?

First, I setup my mailbox to show all threads as collapsed, with a keybind to toggle them open:


folder-hook * "exec collapse-all"

macro index ,v "<collapse-thread>" "collapse/uncollapse thread"
macro index ,V "<collapse-all>"    "collapse/uncollapse all threads"

I also color threads with new mails (and the new mails themselves):

# threads containing new messages
uncolor index "~(~N)"
  color index brightblue default "~(~N)"

# new messages themselves
uncolor index "~N"
  color index brightyellow default "~N"

Now I have a good overview (at the mailbox level) of what I’m dealing with. From there I can expand-read what I want, mark whole threads as read, or mark the whole mailbox as read.

Assuming the majority of threads are closed in this mailbox, I use the following to mark-as-read only open threads:

macro index ,r \
  "<tag-pattern>all<return><tag-prefix><clear-flag>N<untag-pattern>all<return>" \
  "mark all as read"

This macro is technically mark all read, but in actuality only open threads are affected. It also catches any one-message threads as they’re by-definition open as well. I expect and rely on this behavior, but I understand if it’s not for everyone.

In the cases that I do want to mark the whole mailbox read including any closed threads, I just use this second macro to open everything before running the above, then close it up again after:

macro index ,R "<collapse-all>,r<collapse-all>" "mark all as read (collapsed)"

That just about sums it up. The best part, if I’m feeling particularly unreachable, is to blindly do the following:

c Enter , R c Enter , R… Until it’s All Gone.

30 May 2013, tagged with mutt


Mairix is a nice little utility for indexing and searching your emails. Its smooth integration with mutt is also a plus.

I used to use native mutt search, but it’s pretty slow. So far, mairix is giving me a good approximation of the google-powered search available in the web interface and it’s damn fast.

As I go through this setup, keep in mind the example config files are designed to work with my overall mutt setup; one which is described in two other posts here and here.

If you need a little context, checkout my mutt-config repo which has a fully functioning ~/.mutt, example files for the other apps involved (offlineimap, msmtprc, and now mairix), and any scripts the setup needs.


First, of course, install mairix:

pacman -S mairix

Then, setup a ~/.mairixrc which defines where your mails are and their type as well as where to store the results and index. Here’s an example:

# where you keep your mail

# colon separated list of maildirs to index.
# I have two accounts each in their own subfolder. the '...' means there 
# are subdirectories to search as well; it's like saying GMail/* and 
# GMX/*

# I omit gmail's archive folder so as to pevent duplicate hits

# search results will be copied to base/<this folder> for viewing in 
# mutt

# and the path to the index itself

With that in place, run mairix once to build the initial index. This first run will be slower but in my tests, subsequent rebuilds were almost instant.

In situations like these, I’ll usually add a verbose flag so I can be sure things are working as expected.

At this point, you could actually do some searching right from the commandline:

mairix some search term # search and populate mfolder
mutt -f mfolder         # open it in mutt

This wasn’t the usage I was after however, I’m typically already in mutt when I want to search my mails.


My original script for this purpose was pretty simple. It prompted for the search term and ran it. The problem was you then needed a separate keybind to actually view the results.

Thankfully, Scott commented and provided a more advanced script which got around this issue. Many thanks to Scott and whomever wrote the script in the first place.

This version does some manual tty trickery to build its own prompt, read your input, execute the search and open the results. All from just one keybind.

I merged the two scripts together into what you see below. The main changes from Scott’s version are the following:

  1. I kept my clear, purge, search method rather than relying on cron to keep the index up to date.
  2. I removed the append-search functionality; not my use-case.
  3. I removed the <return> from the ^G trap; it was getting executed by mutt and opening the first message in the inbox after a cancelled search.
  4. I fixed it so that backspace works properly in the prompt.

So, here it is:


read_from_config() {
  local key="$1" config="$HOME/.mairixrc"

  sed '/^'"$key"'=\([^ ]*\) *.*$/!d; s//\1/g' "$config"

read -r base    < <(read_from_config 'base')
read -r mfolder < <(read_from_config 'mfolder')

# prevent rm / further down...
[[ -z "$base$mfolder" ]] && exit 1


set -f                          # disable globbing.
exec < /dev/tty 3>&1 > /dev/tty # restore stdin/stdout to the terminal,
                                # fd 3 goes to mutt's backticks.
saved_tty_settings=$(stty -g)   # save tty settings before modifying
                                # them

# trap <Ctrl-G> to cancel search
trap '
  printf "\r"; tput ed; tput rc
  printf "/" >&3
  stty "$saved_tty_settings"

# put the terminal in cooked mode. Set eof to <return> so that pressing
# <return> doesn't move the cursor to the next line. Disable <Ctrl-Z>
stty icanon echo -ctlecho crterase eof '^M' intr '^G' susp ''

set $(stty size) # retrieve the size of the screen
tput sc          # save cursor position
tput cup "$1" 0  # go to last line of the screen
tput ed          # clear and write prompt
tput sgr0
printf 'Mairix search for: '

# read from the terminal. We can't use "read" because, there won't be
# any NL in the input as <return> is eof.
search=$(dd count=1 2>/dev/null)

# clear the folder and execute a fresh search
( rm -rf "$searchdir"
  mairix -p
  mairix $search
) &>/dev/null

# fix the terminal
printf '\r'; tput ed; tput rc
stty "$saved_tty_settings"

# to be executed by mutt when we return
printf "<change-folder-readonly>=$mfolder<return>" >&3

A non-trivial macro provides the interface to the script. It sets a variable called my_cmd to the output of the script, which should be the actual change-folder command, then executes it.

macro generic ,s "<enter-command>set my_cmd = \`$HOME/.mutt/msearch\`<return><enter-command>push \$my_cmd<return>" "search messages"

I’ve gotten used to “comma-keybinds” from setting that as my localleader in vim. It’s nice because it very rarely conflicts with anything existing and it’s quite fast to type.

One downside which I’ve been unable to fix (and believe me, I’ve tried!) is that if you press ^G to cancel a search but you’ve typed a few letters into the prompt, mutt will read those letters as commands (via the push) and execute them.

The only thing I could do is prefix those characters with something. I’ve decided to use /. That makes mutt see it as a normal search which you can execute or ^G again to cancel. Annoying, but better than mutt flailing around executing rando commands…

I haven’t had the time yet to learn all the tricks, but here are some of the more useful-looking searches from man mairix:

Useful searches

   t:word                             Match word in the To: header.

   c:word                             Match word in the Cc: header.

   f:word                             Match word in the From: header.

   s:word                             Match word in the Subject: header.

   m:word                             Match word in the Message-ID: 


   b:word                             Match word in the message body 
                                      (text or html!)

   d:[start-datespec]-[end-datespec]  Match messages with Date: headers 
                                      lying in the specific range.

Multiple body parts may be grouped together, if a match in any of them 
is sought.

   tc:word  Match word in either the To: or Cc: headers (or both).

   bs:word  Match word in either the Subject: header or the message body 
            (or both).

   The a: search pattern is an abbreviation for tcf:; i.e. match the 
   word in the To:, Cc: or From: headers.  ("a" stands for "address" in 
   this case.)

The "word" argument to the search strings can take various forms.

   ~word        Match messages not containing the word.

   word1,word2  This matches if both the words are matched in the 
                specified message part.

   word1/word2  This matches if either of the words are matched in the 
                specified message part.

   substring=   Match any word containing substring as a substring

   substring=N  Match any word containing substring, allowing up to N 
                errors in the match.

   ^substring=  Match any word containing substring as a substring, with 
                the requirement that substring occurs at the beginning 
                of the matched word.

Happy searching!

03 Jul 2011, tagged with mutt

Using Two IMAP Accounts in Mutt

Mutt can be really great with multiple accounts, but it’s not exactly intuitive to setup. Here I’ll document how I access two Gmail accounts together in one mutt instance.

If you haven’t yet seen my previous mutt post, please go read that now. I recommend using that post to get a single account setup first before coming back here. Even if you plan to jump right into a multi-account setup, this post assumes you’ve at least read the other one and will focus on the differences and required changes to get from there to here.


To get Offlineimap syncing multiple accounts, we simply need to add additional configuration blocks to sync the second account with another local Maildir.


ui = ttyui
accounts = Personal,Work

[Account Personal]
localrepository = Personal-Local
remoterepository = Personal-Remote

[Account Work]
localrepository = Work-Local
remoterepository = Work-Remote

[Repository Personal-Local]
type = Maildir
localfolders = ~/Mail/Personal

[Repository Work-Local]
type = Maildir
localfolders = ~/Mail/Work

[Repository Personal-Remote]
type = Gmail
remoteuser = username@gmail.com
remotepass = secret
realdelete = no
sslcacertfile = /etc/ssl/certs/ca-certificates.crt

[Repository Work-Remote]
type = Gmail
remoteuser = work-username@gmail.com
remotepass = secret
realdelete = no
sslcacertfile = /etc/ssl/certs/ca-certificates.crt

Obviously, if either of these accounts weren’t a Gmail server, the configuration blocks would be different.

You can test your setup by running offlineimap -o to sync things once. It could take a while, but once done, you should have a nice folder structure like this:

|-- Personal
|   |-- INBOX
|   `-- ...
`-- Work
    |-- INBOX
    `-- ...


Msmtp also handles multiple accounts very elegantly, we just add another account block for the second account.


# shared defaults since both are gmail accounts
host smtp.gmail.com
port 587
protocol smtp
auth on
tls on
tls_trust_file /etc/ssl/certs/ca-certificates.crt

account personal
from username@gmail.com
user username@gmail.com
password secret

account work
from work-username@gmail.com
user work-username@gmail.com
password secret

account default : personal

Now we can simply call msmtp -a personal or msmtp -a work to use whichever account we want. Omitting the -a option will use the default account which we’ve set as personal.


The goal with mutt is to have certain settings change when we enter certain folders. For example, when we’re viewing +Personal/INBOX we want our from setting to be our personal From address and the sendmail setting should be msmtp -a personal. To provide this functionality, we’re going to do the following:

  1. Place any account-specific settings in separate files
  2. Use mutt’s folder-hook facility to source the proper file and set the proper settings upon entering a folder for a given account.

Here are the two account-specific files:


set from      = "username@gmail.com"
set sendmail  = "/usr/bin/msmtp -a personal"
set mbox      = "+Personal/archive"
set postponed = "+Personal/drafts"

color status green default

macro index D \
    "<save-message>+Personal/Trash<enter>" \
    "move message to the trash"

macro index S \
    "<save-message>+Personal/Spam<enter>"  \
        "mark message as spam"


set from      = "work-username@gmail.com"
set sendmail  = "/usr/bin/msmtp -a work"
set mbox      = "+Work/archive"
set postponed = "+Work/drafts"

color status cyan default

macro index D \
    "<save-message>+Work/Trash<enter>" \
    "move message to the trash"

macro index S \
    "<save-message>+Work/Spam<enter>"  \
        "mark message as spam"

Notice the color line which changes the status bar depending on what account I’m “in” at any given moment.

The following settings will tell mutt to source one of these files upon entering a folder matching the given pattern, this will setup all the correct settings when entering a folder for a given account:


set spoolfile = "+Personal/INBOX"

source ~/.mutt/personal

folder-hook Personal/* source ~/.mutt/accounts/personal
folder-hook Work/*     source ~/.mutt/accounts/work

The first two lines effectively set Personal as the default account when we open mutt.

Well, that should do it. Open up mutt, change folders, send some mails, and make sure everything’s working as you’d expect.

For reference, my complete and current setup can be found with my dotfiles.

05 Dec 2009, tagged with linux, mutt

Mutt + Gmail + Offlineimap

Most people use Gmail. Some people like CLI mail clients. This post describes how I use Gmail in the best CLI mail client, mutt. Many people will back me up when I say it’s a very good setup.

For reference, my complete and current setup can be found with my dotfiles.


Step one is to setup Offlineimap to keep ~/Mail in sync with Gmail. This is a two way sync so anything moved, deleted, or sent from any IMAP-connected device or our local mutt interface will act exactly the same. This also has the benefit of storing offline, local copies of all your mails.

First, install Offlineimap and fill in an ~/.offlineimaprc like so:

ui = ttyui
accounts = Gmail

[Account Gmail]
localrepository = Gmail-Local
remoterepository = Gmail-Remote

[Repository Gmail-Local]
type = Maildir
localfolders = ~/Mail/Gmail

[Repository Gmail-Remote]
type = Gmail
remoteuser = you@gmail.com
remotepass = secret
realdelete = no
maxconnections = 3
sslcacertfile = /etc/ssl/certs/ca-certificates.crt

Test that this works by running offlineimap -o. Your first sync could take some time, but once done, you should see the folders under ~/Mail/Gmail with the proper structure.

Once you’re satisfied syncing is working, we’ll schedule a periodic sync via cron.

There are some tempting options offlineimap has for daemonizing itself to handle periodic syncing for you – in my experience these don’t work. Scheduling a full offlineimap run via cron is the only working setup I’ve been able to find.

To work around a thread-joining bug, I’ve landed on a wrapper script that spawns offlineimap to the background then babysits the process for up to 60 seconds. If it appears to be hung, it’s killed.

#!/usr/bin/env bash

# Check every ten seconds if the process identified as $1 is still 
# running. After 5 checks (~60 seconds), kill it. Return non-zero to 
# indicate something was killed.
monitor() {
  local pid=$1 i=0

  while ps $pid &>/dev/null; do
    if (( i++ > 5 )); then
      echo "Max checks reached. Sending SIGKILL to ${pid}..." >&2
      kill -9 $pid; return 1

    sleep 10

  return 0

read -r pid < ~/.offlineimap/pid

if ps $pid &>/dev/null; then
  echo "Process $pid already running. Exiting..." >&2
  exit 1

offlineimap -o -u quiet & monitor $!

Set this script to run as frequently as you want, by adding something like the following to your crontab – I chose to sync once every 3 minutes:

*/3 * * * * /path/to/mailrun.sh


Now we need a way to send mails. I like msmtp, you can also use other smtp clients. If you choose to install msmtp, the config file is at ~/.msmtprc and should look like this:

account default 
host smtp.gmail.com
port 587
protocol smtp
auth on
from user@gmail.com
user user@gmail.com
password secret
tls on

You can test this by executing echo "a test message" | msmtp you@gmail.com.


Now the fun part! I don’t know how many hours I’ve spent in the past year fine tuning my muttrc, but it’ll never be done. Here are the parts required to get this setup working.

set mbox_type   = Maildir
set sendmail    = /usr/bin/msmtp

set folder      = ~/Mail
set spoolfile   = "+INBOX"
set mbox        = "+[Gmail]/All Mail"
set postponed   = "+[Gmail]/Drafts"
unset record

mailboxes +INBOX

macro index D \
    "<save-message>+[Gmail]/Trash<enter>" \
    "move message to the trash"

macro index S \
    "<save-message>+[Gmail]/Spam<enter>" \
    "mark message as spam"

The above should be enough to get a connection and start sending/receiving mail, but here are some other must-have options that make it feel a bit more like gmail:

# main options
set realname   = "Real Name"
set from       = "user@gmail.com"
set mail_check = 0
set envelope_from

unset move           # gmail does that
set delete           # don't ask, just do
unset confirmappend  # don't ask, just do!
set quit             # don't ask, just do!!
unset mark_old       # read/new is good enough for me

# sort/threading
set sort     = threads
set sort_aux = reverse-last-date-received
set sort_re

# look and feel
set pager_index_lines = 8
set pager_context     = 5
set pager_stop
set menu_scroll
set smart_wrap
set tilde
unset markers

# composing 
set fcc_attach
unset mime_forward
set forward_format = "Fwd: %s"
set include
set forward_quote

ignore *                               # first, ignore all headers
unignore from: to: cc: date: subject:  # then, show only these
hdr_order from: to: cc: date: subject: # and in this order

I’ve left out quite a few tweaks in the above so that those who are happy with mutt’s very sane defaults aren’t overwhelmed. Keep in mind, man muttrc is a great command for when you’re bored.

That should do it. Hopefully this info will get you going in the right direction.

05 Dec 2009, tagged with linux, mutt