PAM authentication with gpg or ssh key

[Schlacta, Christ]

Hello list.  I know this isn't exactly on topic, but I think it's
asymptotically close enough to justify asking here.  I'm looking for a
way to authenticate myself to PAM (Specifically sudo) on a remote
server over SSH, though possibly also on a local server using
ssh-agent. if my gpg key is unlocked.  This is particularly relevant
as I store my gpg key in a smart card, and use it to authenticate to
the servers initially.  It would be nice if, while I was out and about
doing remote administrative tasks, I didn't have to take the security
risk of typing in my password where people could shoulder-surf it.  As
I'm using a hardware crypto token (Yubikey Neo actually), I could
actually enable static passwords, or other crypto measures alongside
my yubikey, however, the two best alternative options have less
desirable side-effects.  namely the yubikey-pam module requires
communication with the yubico servers to authenticate a key, and the
static password option can easily accidentally dump the plaintext
password into, say, an e-mail or notepad.  Therefore, I'm looking for
a way to have PAM query the ssh-agent remotely, or optionally locally
in rare instances if possible, for authentication.  I've tried
googling for this, but was unable to come up with anything, and was
hoping someone here would know a way.

If it's possible to redirect gpg-agent over ssh as a gpg agent instead
of an ssh agent, it would also be more than sufficient, if not
preferable, so long as it can authenticate to PAM effectively.  It's
worth noting that my primary use case is connecting from windows +
gpg2.1 + putty --> Linux + whatever version of gpg comes from repos.
Current platforms include deb 7,8, and ubuntu 14.04 and 15.04, but in
the future plan to incldue freebsd and openbsd.