Many recent strategies have been proposed for searching the enormous space of small organic molecules. In this talk, I'll review the area and present some recent work from our group: MolDQN (Molecular Deep Q-Networks) and RL-VAE (Reinforcement Learning Variational Autoencoders). In addition to explaining the core ideas and experimental results, I'll show some (unpublished) deeper analysis of the value function approximation in the MolDQN.