To set up smart card direction for desktops running Ubuntu or Debian, first integrate the Ubuntu/Debian virtual machine with an Active Directory domain. Then install the necessary libraries and root Certificate Authority (CA) certificate before installing Horizon Agent.