~/.xsession and ~/.xinitrc will not be executed on my system. Where put you your gpg-agent start up code?