The Final Word on AFS PAGs and Services

Yeah... right. But for now (hopefully) (maybe).

Anyway, what I did was move pam_krb5 and pam_afs_session out of /etc/pam.d/common-* and ONLY put it in PAM stacks that need it. That would be dovecot, ssh, login and passwd. So now if you ssh to the server, you have a PAG. If you sudo -s, you have the same PAG. If you su, you have the same PAG. So start a new pagsh (this is important, if you don't, your unprivileged user will get the tokens and when you log out, pam_afs_session will destroy them). Now you can su - www-data -c 'k5start ...' and then 'apache2ctl start' and it works because Apache starts with the same PAG.

You can run apache2ctl graceful and restart from a shell with any PAG since it does not start the master (root) apache2 process (which has the original PAG), but if you need to fully restart Apache (e.g. stop/start) you'll need to start a new PAG and restart k5start under the new PAG.

So what I've done is gotten rid of the wrapped postfix/local and the foregrounded Dovecot and just wrapped their init scripts to run the start/restart methods inside a pagsh.

More later? Dunno.

AFS and Apache

Having sorted out mail on AFS, I moved on to Apache on AFS (or more appropriately, with access to AFS).

This ended up being a problem since Apache handles privileges different than Postfix and Dovecot. The way I had my PAM stack set up, pam_afs_session.so was getting AFS PAGs when I su'd, meaning that su'ing to the Apache user to k5start would get me nowhere since the AFS tokens I got would be associated with a PAG that the server itself would not be running as.

Once I got a good handle on how PAGs work, I got to work disabling them. Reason being that unless you authenticate through AFS directly instead of using your Kerberos credentials and the afs service ticket to get tokens, there's no security offered in using PAGs. The whole point of a PAG is so someone with root on an AFS client box can't su to a user and have access to their tokens. But Kerberos offers no such functionality with tickets, so if a user has tickets and I su to them, I can just aklog and get my own tokens as them. The only way I can see a user preventing a local superuser from getting tokens is to get both tickets and tokens, then destroy the tickets (so they have tokens but no tickets from which to get new tokens).

If I'm misunderstanding this and disabling PAGs has greater security implications, please let me know!

So back to work... I set the nopag option on pam_afs_session.so in /etc/pam.d/common-auth and /etc/pam.d/common-session. Now I wasn't getting PAGs (yay!), but IMAP broke (boo!).

So I copied the contents of /etc/pam.d/common-* to /etc/pam.d/dovecot, re-enabling PAG for dovecot by removing nopag (but leaving it in common-*).

So I would think that if dovecot needs PAGs in this manner, postfix/local would too. But it doesn't. And I'm not sure why. I have a couple guesses... like dovecot is spawning a separate imap process as you to hand you off to once you've authenticated, so that process needs the PAG, whereas local is all done in a single process. But then by that logic, procmail shouldn't work? Or, dovecot actually auth's you via PAM where local doesn't. That could be it...

My brain hurts, I'm going home.

AFS, Postfix, Dovecot and Maildirs

UPDATE: Most of this is still valid, but read this first to see what I've learned about needing PAGs.

Okay, here's the deal. Maildir works by using link()/unlink() to move mail around from the new/cur/tmp dirs. This is a problem because AFS doesn't allow link() across dirs Doing so would break ACLs. However, you can safely use rename() with Maildirs on AFS because it's atomic, which is the whole reason Maildir uses link()/unlink() in the first place.

Originally I didn't have any problems with link()/unlink() because I didn't let the Postfix local delivery agent (local) deliver my mail - it passed it off to procmail, which already happens to use rename() when dealing with Maildir (which is technically incorrect behavior).

Here's how you can get it to work:

1. Create a user principle in Kerberos for postfix and one for dovecot. Export as keytabs. Store them on the mail server somewhere. You could probably use a single principle for this, but I like the idea of having two so users can remove privs from dovecot if they don't check their mail via IMAP.
   
2. Make a few changes in dovecot.conf. First, keep indexes out of AFS and disable link() copies:
   
   mail_location = maildir:/afs/cell/home/%u/Maildir:INDEX=/var/lib/dovecot-index/%u
   maildir_copy_with_hardlinks = no
   
   Next, disable mmap and enable dotlocks. I would think you shouldn't need these but this is what I ended up with that worked. Test yourself and see:
   
   mmap_disable = yes
   dotlock_use_excl = yes
   lock_method = dotlock
   
   I ended up using the following args for passdb pam {}to ensure pam_afs_session gets hit:
   
   args = session=yes setcred=yes dovecot
   
   
3. k5start dovecot in the foreground (k5start backgrounds the whole thing):
   
   k5start -U -f /etc/dovecot/dovecot.keytab \
   -t -b -K 10 -- /usr/sbin/dovecot -F


4. Get the Postfix source (as I use debian, I just grabed the source package from APT and built it that way). Edit src/local/maildir.c and change the sane_link() calls to rename(), and comment out the call to unlink(). Install Postfix. WARNING: You shouldn't modify this if your Postfix also delivers to non-AFS Maildirs, as rename() on most non-AFS filesystems is NOT atomic. If you need to do that, checking to see if you're in /afs and then using the right syscall should be trivial.
   
5. Rename /usr/lib/postfix/local (or wherever it is on your system). I mv'd it to local.wrapped
6. Make a new file called local, give it the same permissions/owernship as local.wrapped, and then wrap local.wrapped as such:
   
   #!/bin/sh
   k5start -U -f /etc/postfix/postfix.keytab \
   -K 10 -t -- /usr/lib/postfix/local.wrapped \
   "$@"
   
   
7. Set ACLs on Maildirs for postfix/dovecot:
   
   find /afs/path/to/Maildir -type d -exec \
   fs setacl {} postfix rlidwk dovecot rlidwk \;
   

For a while there, I considered just sticking the Maildirs locally on the mail server instead of in AFS homedirs. But there are benefits to having Maildirs in the AFS: Direct access to grep them, moving sub-maildirs around so you can archive them to somewhere outside your Maildir, and of course, direct access via command line mailreaders like mutt.

Solaris on the MacBook Pro

Status: Fail.

I'm not interested in running Parallels, I want to run each OS natively. The plan is to triple-boot OS X, Solaris, XP.

It seemed reasonable because this guy at Sun claims to have done it.

So on to the procedure and lessons learned: Install Boot Camp - easy. Then resize and repartition the disk manually using disktool. This is all one step and once you do it there's no (easy) going back, although the reversal is explained below.

I dropped to console (go to the OS X login screen and as your username, enter '>console') and logged in as root directly for this:

diskutil resizeVolume disk0s2 16000M "Linux" Solaris 16000M "MS-DOS FAT32" Windows 15500M

Don't be tempted by the "UFS" partition ID, it will not work. Although for that mattter, neither will "Linux". But read on...

I originally planned to use 60G, 16G and 16G for the values (respectively) but that didn't work, at first throwing a "No space left on device" error. Yeah, real helpful, thanks. From reading around, the thing to do is remove some big files, maybe defrag. Removing big files worked for me. After one attempt, my partition table was apparently corrupted so future attempts at resizing resulted in an error like "the underlying command failed." So the solution was to boot from the OS X install disk and run Disk Repair on it.

So assuming you can actually successfully get 3 working partitions, this is where you're supposed to be able to boot in to Solaris (SXCR/Nevada b54 in my case) and get to installing as explained in the blog above. Trouble is, that partition shows up in fdisk as "Linux native". So I dunno how the blogger got so lucky, but I was not so lucky. I looked in to changing the partition ID manually but found these:

OpenSolaris thread
Guy from that thread's attempts
Guy from that thread's success (on external disk)
Some guy has it working but it's not documented yet

So it looks like that would be useless anyway. But there's encouraging news... people there who seem to be very familiar with EFI and GPT seem to be close to getting it to work.

Anyway, now I've got a bunch of useless partitions to get rid of, so this was helpful:

How to remove the partitions and get back to normal

I used the boot from Install Disc method (see comments) and you don't need to use Disk Utility for the unmounting, umount works fine (but you do need to unmount before every incantation of gpt as described).

umount /dev/disk0s2 [/dev/disk0s3] [/dev/disk0s4]
gpt show rdisk0
umount ...
gpt remove -i 4 rdisk0
umount ...
gpt remove -i 3 rdisk0

And then ignore all the calculation and just do this (after rebooting in to OS X):

diskutil resizeVolume disk0s2 limits

And using that information:

diskutil resizeVolume disk0s2 B "MS-DOS FAT32" Windoze B

Reboot. Run Boot Camp Setup. Reclaim space. All is now as it was.