Some time ago I wrote up my experiences with running gpg remotely. This post documents the most recent changes I've made to my setup, which finally make my gpg (and ssh) keys fully mobile and 'migratory'.
Like before I use the kernel key storage system to cache passphrases (and that won't change until I switch to gnupg2 with the agent). But now my keys are all stored on a usb stick, in an encrypted filesystem.
When I login the first time any day, I load the keys from the encrypted storage into a RAM disk. (A simple symlink in ~/.gnupg is sufficient to convince gnupg to find the secret ring.) When I leave for/from work I nuke the RAM disk - that way the keys are always only present where I physically am.
The big new change from the previous setup is that now I use sshfs when I need to use gpg for anything on a remote box: I ssh into the target box with a remote port forwarded back to a listening instance of sftp-server on the local box (which has the keys in RAM). With agent forwarding on, the sshfs connection doesn't require entering passwords, and the mount point is of course set to be the same as the RAM disk location for locally loaded keys, so to gpg it's totally transparent. (I'd never do any of this if not all machines in question were under my exclusive full control.)
sshfs is no speed daemon, but then the secret ring file isn't large. sshfs with -o directport on the forwarded port reuses the existing outbound ssh connection, so one single outbound ssh connection does it all - and another benefit of that setup is that the keys vanish from the remote machine as soon as the outbound ssh connection is shut down.
The one simple shell script doing all this setup is less than 60 lines long: simple, neat, sufficient.