Fermions and gauge-theories are two classes of physical systems where symmetries - antisymmetry and gauge symmetry respectively - are important. In this talk, I will discuss how we combine machine-learning architectures with more physics-inspired approaches to build variational wave-functions for fermions and gauge theories. We will discuss how these variational ansatz preserve the relevant symmetry of the system. For Fermions, I will discuss the use of configuration-dependent Slater determinants (and generalizations thereof). For gauge theories, I will focus on how we can modify autoregressive neural networks to preserve the perfect sampling property while incorporating the gauge symmetry and show that this leads to surprisingly accurate results for a range of gauge theories.