Saturday, October 23, 2010

Migrating mail: kmail on SuSE Linux to maildir on mac os X Snow Leopard; also dovecot

Updated! See also part deux.
I'm not running Snow Leopard server edition (or whatever they call it) so I had to download dovecot, then configure it. Here's my journey....

Ports

First, I found the install media and installed Xcode. I like having compilers; it makes me feel like I'm on a real system.

Then, as I wrote earlier, it's not darwinports any more; instead, I downloaded macports. This creates a hierarchy under /opt/local, and also does nice things to the system-wide profile files. So once I had ports installed, every terminal session I started had these directories prepended to $PATH, like this:

$ echo $PATH
/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/Users/carol/bin

Install dovecot

Short version: click here. Even shorter:
sudo port install dovecot

Configure dovecot

Following instructions at http://wiki.dovecot.org/BasicConfiguration:
  • Where's the config file?
    $ dovecot -n
    # 1.2.14: /opt/local/etc/dovecot/dovecot.conf
  • Simple authentication
    $ echo "$USER:{PLAIN}password" > passwd.dovecot
    $ sudo mv passwd.dovecot /opt/local/etc/dovecot/
  • Modify dovecot.conf to use plaintext -- no longer disable plaintext login
  • mail location: "echo hi | mail -s dovecot.test carol" yields /var/mail/carol. but it's more than that; we have this sort of arrangement:
    $ ls ~/Mail
    ACFW                            MailLists
    AShland                         MailisWork
    ASthma research                 Mariko
    AWP                             Marketing
    Addresses                       Meet Up
    Affirmations                    Menlo Atherton HS
    ...
    JapanLifeInfo                   inbox
    ...
    $ ls ~/Mail/inbox
    cur     new     tmp
    $ 
    Therefore I'm thinking to create Maildir with directories cur,new,tmp, and the rest of the directories can be subdirectories under it. Like this:
    $ ls -o Maildir
    total 24
    lrwxr-xr-x  1 carol  27 Oct 23 10:58 cur -> /Users/carol/Mail/inbox/cur
    lrwxr-xr-x  1 carol  27 Oct 23 10:58 new -> /Users/carol/Mail/inbox/new
    lrwxr-xr-x  1 carol  27 Oct 23 10:58 tmp -> /Users/carol/Mail/inbox/tmp
    $ ls -F ~/Mail | grep / | cut -d/ -f1 | while read D; do ln -s "~/Mail/$D" Maildir; done
                # OOPS! The above doesn't work; see below
    $ rm Maildir/inbox
    $ 
  • But as noted above, mail isn't currently being delivered there. the dovecot page doesn't tell me about combining /var/mail/USERNAME mbox with ~/Maildir maildir format, so does procmail work? What if I set DEFAULT=$HOME/Maildir/ and put this into .forward?
    $ cat .forward
    "|/usr/bin/procmail"
    Yes! A test produced this:
    $ ls Maildir/new
    1287857872.59228_2.mini1.local
  • Turn off SSL TEMPORARY ONLY! -- otherwise I keep getting
    $ dovecot -n
    # 1.2.14: /opt/local/etc/dovecot/dovecot.conf
    Error: ssl_cert_file: Can't use /opt/local/etc/ssl/certs/dovecot.pem: No such file or directory
    Fatal: Invalid configuration in /opt/local/etc/dovecot/dovecot.conf
  • Putting the above together, we have:
    $ diff -u /opt/local/etc/dovecot/dovecot{-example,}.conf 
    --- /opt/local/etc/dovecot/dovecot-example.conf 2010-10-16 17:26:05.000000000 -0700
    +++ /opt/local/etc/dovecot/dovecot.conf 2010-10-23 11:36:26.000000000 -0700
    @@ -1,4 +1,5 @@
     ## Dovecot configuration file
    +# from dovecot-example.conf 2010-10-22 collin
     
     # If you're in a hurry, see http://wiki.dovecot.org/QuickConfiguration
     
    @@ -45,7 +46,7 @@
     # SSL/TLS is used (LOGINDISABLED capability). Note that if the remote IP
     # matches the local IP (ie. you're connecting from the same computer), the
     # connection is considered secure and plaintext authentication is allowed.
    -#disable_plaintext_auth = yes
    +disable_plaintext_auth = no
     
     # Should all IMAP and POP3 processes be killed when Dovecot master process
     # shuts down. Setting this to "no" means that Dovecot can be upgraded without
    @@ -86,7 +87,7 @@
     #ssl_listen =
     
     # SSL/TLS support: yes, no, required. 
    -#ssl = yes
    +ssl = no
     
     # PEM encoded X.509 SSL/TLS certificate and private key. They're opened before
     # dropping root privileges, so keep the key file unreadable by anyone but
    @@ -215,7 +216,7 @@
     #
     # See  for full list. Some examples:
     #
    -#   mail_location = maildir:~/Maildir
    +mail_location = maildir:~/Maildir
     #   mail_location = mbox:~/mail:INBOX=/var/mail/%u
     #   mail_location = mbox:/var/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n
     #
    @@ -860,11 +861,11 @@
       # The deny passdb should always be specified before others, so it gets
       # checked first. Here's an example:
     
    -  #passdb passwd-file {
    +  passdb passwd-file {
         # File contains a list of usernames, one per line
    -    #args = /etc/dovecot.deny
    +    args = /opt/local/etc/dovecot/passwd.dovecot
         #deny = yes
    -  #}
    +  }
     
       # PAM authentication. Preferred nowadays by most systems. 
       # Note that PAM can only be used to verify if user's password is correct,
    @@ -872,7 +873,7 @@
       # database (passwd usually), you can use static userdb.
       # REMEMBER: You'll need /etc/pam.d/dovecot file created for PAM
       # authentication to actually work. 
    -  passdb pam {
    +  #passdb pam {
         # [session=yes] [setcred=yes] [failure_show_msg=yes] [max_requests=]
         # [cache_key=] []
         #
    @@ -905,7 +906,7 @@
         #   args = session=yes %Ls
         #   args = cache_key=%u dovecot
         #args = dovecot
    -  }
    +  #}
     
       # System users (NSS, /etc/passwd, or similiar)
       # In many systems nowadays this uses Name Service Switch, which is
    $ dovecot -n
    # 1.2.14: /opt/local/etc/dovecot/dovecot.conf
    # OS: Darwin 10.3.3 i386  
    ssl: no
    disable_plaintext_auth: no
    login_dir: /opt/local/var/run/dovecot/login
    login_executable: /opt/local/libexec/dovecot/imap-login
    mail_location: maildir:~/Maildir
    auth default:
      passdb:
        driver: passwd-file
        args: /opt/local/etc/dovecot/passwd.dovecot
      userdb:
        driver: passwd
    $ 

Correct folder names

Fired up mail.app and it found the inbox: 382 unread of 4951 items, something like this. But! None of the folders were there. H'm... I obviously don't grok Maildir format.

[a few minutes later] Gaaaa! Need to modify the above (wrong) format! Folder names must begin with a dot! http://wiki.dovecot.org/MailboxFormat/Maildir#Directory_Structure says so. Remedy is here:

$ ls -F ~/Mail | grep / | cut -d/ -f1 | while read D; do 
     rm "Maildir/$D"; ln -s "~/Mail/$D" "Maildir/.$D"; done
rm: Maildir/inbox: No such file or directory
$ rm Maildir/.inbox 
$ 

With the correct directory names, I quit mail.app, then stopped dovecot -- like http://wiki.dovecot.org/RunningDovecot#Stopping says:

$ sudo cat /opt/local/var/run/dovecot/master.pid 
59267
$ sudo kill -s TERM 59267  # and wait a few seconds, then: 
$ sudo dovecot 
Brought mail.app back up... And that worked!

Oops, forgot nested folders. And more

If I have a folder Trips with subfolders Alaska and Hawaii, kmail apparently creates Mail/Trips, Mail/.Trips.directory/Alaska, Mail/.Trips.directory/Hawaii; those directories each have cur,new,tmp; therefore we'll do this:
$ cd Mail
$ find . -type d | grep directory | grep -ve '/cur$' -e '/new$' -e '/tmp$' | 
  grep '/[^.]' | sed -e 's:^./::' | while read F1; do 
      DEST=$(echo "$F1" | sed 's:.directory/:.:g'); 
      ln -s ~/Mail/"$F1" ~/Maildir/"$DEST"; done
$ 
Ah, almost! “sed 's:.directory/:.:g'” should probably instead be:
sed 's:\.directory/\.:.:g' | sed 's:\.directory/:.:g'

Here's something else!

$ ls -Ao Maildir
total 13008
lrwxr-xr-x  1 carol       11 Oct 23 11:57 .ACFW -> ~/Mail/ACFW
lrwxr-xr-x  1 carol       14 Oct 23 11:57 .AShland -> ~/Mail/AShland
...
Oops! Somehow I have a literal “~/” instead of /Users/carol/. Sigh... OK then...
$ cd Maildir
$ find . -type l | while read X; do 
      Y=$(readlink "$X"); 
      if [[ $Y == ~* ]] ; then ln -sf "${HOME}${Y#?}" "$X"; fi; 
   done
$ 

Real authentication

First up is creating SSL certificates; I'll start at http://wiki.dovecot.org/SSL.

No; we'll use IMAP and not disable plaintext. It's just home; I'm not running a business with secret information. And we don't put sensitive stuff in email anyway. This works:

$ diff dovecot-example.conf dovecot.conf 
1a2
> # from dovecot-example.conf 2010-10-22 collin
48c49
< #disable_plaintext_auth = yes
---
> disable_plaintext_auth = no
56c57
< #shutdown_clients = yes
---
> shutdown_clients = yes
89c90
< #ssl = yes
---
> ssl = no
218c219
< #   mail_location = maildir:~/Maildir
---
> mail_location = maildir:~/Maildir
907c908
<     #args = dovecot
---
>     args = login

Multiple addresses, outbound mail, address book

Fortunately I remembered this issue before the lovely Carol hit it while I was at the office. Sometimes I want to send email as myself, sometimes as an identity that results in replies coming back to both of us. So does the lovely Carol. Kmail allows multiple identities; mail.app does too. You can configure this by editing account preferences -- multiple email addresses, as shown in this nice article at hints.macworld.com .

Outbound mail works great too; I set the outbound mail server to be our ISP's server and everything Just Worked. Basically mail.app connects to the ISP (never mind postfix) and... yeah.

The address book turned out to be pretty easy. From kde's address book, export as CSV; copy file to Mac; from address book (the tan book with the big @ ) File→Import, select the file, validate how columns are imported, and bob's yer uncle.

Make dovecot run on boot

Followed these directions. Looks reasonable, but the test will be when she reboots the box. (I shut down the Linux box, then had to restart it for some reason; some filesystems hadn't been checked in 446 days...)

No comments: