After the install with LUKS encrypted root there are other areas where a smart card may be used. Some of these work easily--others do not.
command line sudo
graphical sudo/prompts
login to console
login to GUI
ssh
password management
numlock
Generally, I have been using poldi but recently found libpam-p11 and found that it gives me a completely working system on Kubuntu 21.10. Unfortunately, it doesn't work on 22.04. Fresh installs and updates seem to be breaking the smartcard setup. Have not been able to make it work in latest releases. Even 24.04 LTS where it was working has now broken.
A note about kwallet. I don't want it. I don't use it and it won't go away. In the past--with varying degrees of success--I have:
disabled in system settings
deleted files from random locations based on web searches
set specific commands in files
set a blank password after every password reset
Here are some specifics
rm ~/.local/share/kwalletd/kdewallet.kwl
PAM handles most of the work for sudo. I have had success using libpam-poldi and libpam-p11 with the smart card. libpam-p11 works better but is a little more technical to setup. I have not been able to get everything working with poldi.
When gnome and KDE need to elevate privileges they prompt the user for her password. This seems to be running a sudo based or equivalent process. This should detect that a smartcard is available and prompt for the PIN instead of the password.
This did not work in pureos 9 (whereas most other stuff did) but does work once I used libpam-p11 in kubuntu.
This can be configured similar to the p11/poldi sudo setup. I don't login to the console very often so I have only set this up once or twice.
pam can enable multi factor authentication for login. In this case you are forced to enter the password and have the smartcard and PIN.
Ideally, I would be able to login to the display manager with the smartcard but this is not easy to get working. It it is setup out of the box on PureOS. On other distros I get mixed results. libpam-p11 provides the most functionality on all distros.
Though it is useful to protect the private key with the smartcard I'm already using encrypted partitions for storing keys. Using the smartcard with SSH does not bring much value to me. It would add an extra layer of protection in the case of a system compromise but not much. maybe I'll do it at some point.
Some of the smart cards have password manager features. I have not been able to get this to work with mine and so I use keepassxc with the data stored in a LUKS volume.
I want the numlock to be on by default for everything. Setting it on and off has changed over the years but in 2024 using sddm it might work by adding 'Numlock=on' to /etc/sddm.conf