How can I get ssh-agent (or gpg-agent) to "lazily" add keys?

This is actually just the way ssh-agent works. You start out by running just ssh-agent; this sets some environment variables which let your shell and everything that inherits from it refer to the active ssh-agent process. Later you run ssh-add and it adds your keys using those environment variables. You can do this in your .xinitrc with a line like "eval ssh-agentssh−a≥ntssh-agent", and run ssh-add at your leisure from any shell. That's all there is to it!

I have login scripts that check to see if my ssh-agent is running and the PID matches a value I squirrel away in a per-host named file in a private home directory (so that my home directory can be on a network mounted filesystem.)   If it is running, I load the appropriate environment variables, otherwise I print (echo) out some lines I can cut-n-paste to restart ssh-agent and save the environment variables it produces. I run ssh-agent with a time-limit on how long before I have to re-enter my password and it works pretty well for me. Of course, I'm careful with file and directory access modes - but I have to trust the owners of the filesystems involved.  In the particular case of the networked filesystem I designed it for, the filesystem was AFS not NFS and so I wasn't vulnerable to most root compromises, just the filesystem sysadmins (and I knew who they were.)