Mutt + Gmail + Offlineimap

Most people use Gmail. Some people like CLI mail clients. This post describes how I use Gmail in the best CLI mail client, mutt. Many people will back me up when I say it’s a very good setup.

This post shares a lot of information with my second mutt post regarding multiple accounts. If that’s something you’re planning on setting up, you can skip this tutorial as it’s a subset of the information there.

My full, working setup can always be found in my mutt-config repo.

Offlineimap

Step one is to setup Offlineimap to keep ~/Mail in sync with Gmail. This is a two way sync so anything moved, deleted, or sent from any IMAP-connected interface or our local mutt interface will act exactly the same. This also has the added benefit of storing offline, local copies of all your mails.

First, install Offlineimap and fill in an ~/.offlineimaprc like so:

[general]
ui = ttyui
accounts = Gmail

[Account Gmail]
localrepository = Gmail-Local
remoterepository = Gmail-Remote

[Repository Gmail-Local]
type = Maildir
localfolders = ~/Mail/Gmail

[Repository Gmail-Remote]
type = Gmail
remoteuser = you@gmail.com
remotepass = secret
realdelete = no
maxconnections = 3

# newer offlineimap needs this
cert_fingerprint = f3043dd689a2e7dddfbef82703a6c65ea9b634c1

Test that this works by running offlineimap -o. Your first sync could take some time, but once done, you should see the folders under ~/Mail/Gmail with the proper structure.

Offlineimap is kind of buggy for me; if I use its built-in refresh mechanism, I find it’ll often hang or quit and I’ll be left with an unsynced mailbox. Therefore, I choose to set offlineimap to never refresh and put a [re]start script in a cronjob to take care of it.

Once you’re sure things are syncing fine, set up a cron job to run a script called mailrun.sh every 3 minutes:

crontab -e

# add this:
*/3 * * * * /path/to/mailrun.sh

Then create that script with these contents:

#!/bin/bash
read -r pid < ~/.offlineimap/pid

if ps $pid &>/dev/null; then
  echo "offlineimap ($pid): another instance running." >&2
  kill -9 $pid
fi

offlineimap -o -u quiet &

And make it executable via chmod +x /path/to/mailrun.sh

Msmtp

Now we need a way to send mails. I like msmtp, you can also use other smtp clients. If you choose to install msmtp, the config file is at ~/.msmtprc and should look like this:

account default 
host smtp.gmail.com
port 587
protocol smtp
auth on
from user@gmail.com
user user@gmail.com
password secret
tls on
tls_nocertcheck

You can test this by executing echo "a test message" | msmtp you@gmail.com.

Mutt

Now the fun part! I don’t know how many hours I’ve spent in the past year fine tuning my muttrc, but it’ll never be done. Here are the parts required to get this setup working.

set mbox_type   = Maildir
set sendmail    = /usr/bin/msmtp

set folder      = ~/Mail
set spoolfile   = "+INBOX"
set mbox        = "+[Gmail]/All Mail"
set postponed   = "+[Gmail]/Drafts"
unset record

mailboxes +INBOX

macro index D \
    "<save-message>+[Gmail]/Trash<enter>" \
    "move message to the trash"

macro index S \
    "<save-message>+[Gmail]/Spam<enter>" \
    "mark message as spam"

The above should be enough to get a connection and start sending/receiving mail, but here are some other must-have options that make it feel a bit more like gmail:

# main options
set realname   = "Real Name"
set from       = "user@gmail.com"
set mail_check = 0
set envelope_from

unset move           # gmail does that
set delete           # don't ask, just do
unset confirmappend  # don't ask, just do!
set quit             # don't ask, just do!!
unset mark_old       # read/new is good enough for me

# sort/threading
set sort     = threads
set sort_aux = reverse-last-date-received
set sort_re

# look and feel
set pager_index_lines = 8
set pager_context     = 5
set pager_stop
set menu_scroll
set smart_wrap
set tilde
unset markers

# composing 
set fcc_attach
unset mime_forward
set forward_format = "Fwd: %s"
set include
set forward_quote

ignore *                               # first, ignore all headers
unignore from: to: cc: date: subject:  # then, show only these
hdr_order from: to: cc: date: subject: # and in this order

I’ve left out quite a few tweaks in the above so that those who are happy with mutt’s very sane defaults aren’t overwhelmed. Keep in mind, man muttrc is a great command for when you’re bored.

That should do it. Hopefully this info will get you going in the right direction.

Wifi Pipe

So the other day when I was using wifi-select (awesome tool) to connect to a friends hot-spot, I realized, “hey! This would be great as an openbox pipe menu!”

I’m fairly decent in bash and I knew both netcfg and wifi-select were in bash so why not rewrite it that way?

Wifi-Pipe

A simplified version of wifi-select which will scan for networks and populate an openbox right-click menu item with available networks. Displays security type and signal strength. Click on a network to connect via netcfg the same way wifi-select does it.

Zenity is used to ask for a password and notify of a bad connection. One can optionally remove the netcfg profile if the connection fails.

Requirements

• netcfg
• zenity
• A NOPASSWD entry in sudoers for this script
• An entry in your menu.xml

The script now has its own github repo so it doesn’t fall victim to bitrot. Please head there for more installation details and a copy of the source.

Aurget

About

A simple pacman-like interface to the AUR written in bash.

Aurget is designed to make the AUR convenient and speed up tedious actions. The user can decide to search, download, build, and/or install packages consistently through a config file or dynamically by passing arguments on the commandline.

The user can also choose to edit all or no PKGBUILDs, and enable or disable auto-dependency-resolution through the same means.

Checking dependencies comes with risks because PKGBUILDs need to be sourced. Please, if you’re worried about this, be sure to view all PKGBUILDs before proceeding or use the config file or commandline options to disable this check from occurring and remove any associated risk.

You have been warned.

Usage

The screenshot:

The help:

usage: aurget [ -v | -h | -S* [ --options ] [ -- ] <arguments> ]
  options:

        -S  <package>   process <package> using your default sync_mode

        -Sd <package>   download <package>
        -Sb <package>   download and build <package>
        -Sy <package>   download, build, and install <package>

        -Su             process available upgrades using your default sync_mode

        -Sdu            download available upgrades
        -Sbu            download and build available upgrades
        -Syu            download, build, and install available upgrades

        -Ss  <term>     search aur for <term>
        -Ssq <term>     search aur for <term>, print only package names
        -Sp  <package>  print the PKGBUILD for <package>
        -Si  <package>  print extended info for <package>

        --rebuild       always rebuild (ignore your cache)

        --devel         only affects -Su, add all development packages

        --deps          resolve dependencies
        --nodeps        don't resolve dependencies

        --edit          prompt to edit all pkgbuilds
        --noedit        don't prompt to edit any pkgbuilds

        --discard       discard source files after building
        --nodiscard     don't discard source files after building

        --nocolor       disable colorized output
        --noconfirm     auto-answer y to all prompts

        --ignore '<package> <package> <...>'
                        add additional packages to be ignored

        --mopt '-opt -opt ...'
                        add additional options to the build command

        --popt '-opt -opt ...'
                        add additional options to the install command

        -v, --version   display version info
        -h, --help      display this

        --option=value  set config <option> as <value> for this run only

Installation

Install the AUR package here.

Follow development via my git repo here.

Bugs and Features

If you’ve found a bug or want to request a feature, please let me know via email. If you can implement what you’re looking for, please fork my git repo and send me a pull request.

Aurget does not and will not search or install from official repos. This is by design and will not be implemented even if you offer a patch.

Use packer or clyde if this is what you’re looking for.

Known Bugs

If you pass an aur package before one of its dependencies as the targets to aurget, it will not reorder the targets and the installation will probably fail on the first package. Accounting for this would require a lot of unneeded code. The makepkg error will tell you the dep is not satisfied and it’s easy enough to adjust your targets and run it again.

In a somewhat related way, it is possible, depending on the structure of multi-level dependencies, for aurget to miss a dependency. As an example:

# coding specifically for this scenario:

    pkg
    `-- depends
        |-- foo
        |-- bar
        |   `- depends
        |      `- baz
        `-- baz

# would break this one (and vice versa):

    pkg
    `-- depends
        |-- foo
        |-- baz
        `-- bar
            `- depends
               `- baz

Aurget will filter out the duplicate dependency (baz), but in one of the cases it will be placed behind the package that needs it and makepkg will fail. I consider this improper packaging and have decided to not try and code around it. If you encounter this scenario, I encourage you to post a comment on the aur page of the parent package explaining that baz is unneeded in his depends array because it’s pulled in by bar.

Some aur packages report a bad url to their tarball in the JSON interface. Aurget checks the downloaded file, if it’s not a valid archive, it will try http://aur.archlinux.org/packages/$package/$package.tar.gz as a fallback. If neither the JSON url nor the fallback url provide a valid archive, well, there’s not much I can do.

Text From CLI

This is a short but extensible script to allow text messaging (to verizon customers) straight from the commandline.

Setup requires simply a means to send email from the commandline along with a small script to pass the message off to <number>@vtext.com.

If you already have a CLI mailing solution you can just copy the script and go ahead and change the mail command to mutt, ssmtp, mailx, or whatever you’re using.

Email from CLI

I use msmtp to send mails in mutt so it was easy for me to adapt that into a CLI mailing solution.

Here’s a ~/.msmtprc for gmail:

# msmtp config file

# gmail
account gmail 
host smtp.gmail.com
port 587
protocol smtp
auth on
from username@gmail.com
user username@gmail.com
password gmail_password
tls on
tls_nocertcheck

account default : gmail

Right now, as-is, it’s possible for you to echo "Some text" | msmtp someone@somewhere.com and it’ll email just fine. I’d like to make things a little more flexible.

By dropping a file in ~/.mailrc we can change the mail command to use whatever binary we want instead of the default /usr/bin/sendmail. It should have the following contents:

set sendmail=/usr/bin/msmtp

Now, anytime your system mails anything on your behalf, it’ll use msmtp.

The Script

The script started out very simply, here it is in its original form:

#!/bin/bash

if [[ $# -lt 2 ]]; then
  echo "usage: $0 [number] [some message]"
  exit 1
fi

number="$1"; shift

echo "$*" | mail "$number@vtext.com"

With this little sendtext.sh script in your back pocket, you can send yourself texts from remind, cron, rtorrent, or any other script to notify you (or other people) of whatever you want.

sendtext.sh 1234567890 'This is a test text, did it work?'

Sure did.

Now, at some point, Ghost1227 got bored again.

He took my sendtext script and ran with it. Added loads of carriers and some new option handling.

I took his update of my script and re-updated it myself. Mainly syntactical changes and minor options handling, just to tailor it to my needs.

The new version with my and ghost’s changes can be downloaded from my git repo.

I also added simple phone book support. When sending a message to someone, pass -s <number> <name> and the contact will be saved to a text file. After that, you can just sendtext <name> and the most recent match out of this text file will be used. The service is saved as well (either the default or the one passed as an argument at the time of -s).

Screen Tricks

Hopefully, if you’re a CLI junky, you’ve heard of GNU/screen. And if you’ve heard of it, chances are you’re using it.

Screen is a terminal multiplexer. This means that you can start screen in one terminal (say, your SSH connection) and open any number of terminals inside that terminal. This lets me have mutt, ncmpcpp, and a couple of spare shells all open inside my single PuTTY window at work.

This is a great use of screen, but the benefits don’t have to end there. When I’m not at work but at home, I can use screen to run applications which I don’t want to end if I want to change terminals, log in and out, or even if all of X comes crashing down around me.

See, screen can detach (default binding: C-a d). Better still, It will auto-detach if the terminal it’s in crashes or you logout. You can then re-attach it later, from any other ssh session, tty, or X terminal.

This is great for apps like rtorrent and irssi, it’s also great for not losing any work if your ssh connection gets flaky. Just re-connect and re-attach.

So now I have a dilemma. When I’m at work, I want to start screen and get a few fresh tabs set up as I’ve defined in ~/.screenrc: mutt, ncmpcpp, and three shells. But at home I don’t want those things to load, I instead want only rtorrent or only irssi to load up in the new screen window.

Furthermore, if rtorrent or irssi are already running in some detached screen somewhere, I don’t want to create an entirely new session, I’d rather grab that one and re-attach it here.

The goal was to achieve this without changing the commands I run day to day, affecting any current keybinds, or using any overly complicated scripts.

So, how do I do this as simply and easily as possible? Environment variables.

How to do it

First we set up one main ~/.screenrc which is always called. Then we set up a series of “screenrc extensions” which only load the apps in the screen session via a stanza of screen -t <name> <command> lines.

Next, we dynamically choose which “screenrc extension” to source from the main ~/.screenrc via two environment variables which are either exported from ~/.bashrc (the default) or explicitly set when running the command (the specialized cases).

So, set up a ~/.screenrc like this:

# screen config file; ~/.screenrc

# put all our main screen settings like
# term, shell, vbell, hardstatus whatever
#
# then add this:

# sources environment-specific apps
source "$SCREEN_CONF_DIR/$SCREEN_CONF"

# you can even add some tabs you'll always
# open no matter what

# then always open some terms
screen -t bash $SHELL
screen -t bash $SHELL
screen -t bash $SHELL

Now, how does screen know what “screenrc extension” to source? By setting those variables up in ~/.bashrc:

# dynamically choose which tabs load in screen
export SCREEN_CONF_DIR="$HOME/.screen/configs"
export SCREEN_CONF="main"

In a clean environment, screen will source that default ~/.screen/configs/main, which will:

# example: screen -t [name] [command]
screen -t mail mutt
screen -t music ncmpcpp

Why is this useful? Because, now I can do something like this:

SCREEN_CONF=rtorrent screen

And screen will instead source that explicitly set ~/.screen/configs/rtorrent which yields:

# example: screen -t [name] [command]
screen -t torrents rtorrent 

Et viola, no mutt or ncmpcpp, but rtorrent instead (same thing happens with irssi).

Oh, but it gets better! Now we’ll add some aliases to ~/.bashrc to complete the whole thing:

alias irssi='SCREEN_CONF=irssi screen -S irssi -D -R irssi'
alias rtorrent='SCREEN_CONF=rtorrent screen -S rtorrent -D -R rtorrent'

Oh how beautiful, how simple, how easy. I type rtorrent, what happens?

Screen checks for any running screens with session-name “rtorrent” and re-attaches here and now. If none are found, screen opens a new screen (using the rtorrent file) and names the session “rtorrent” so we can -D -R it explicitly thereafter.

All of this happens for irssi too, and can be used for any app (or multi-app setup) you want.

Pretty KISS if I do say so.