Heterotetrameric clathrin adaptor protein complexes (APs) orchestrate the formation of coated vesicles for transport among organelles of the cell periphery. AP1 binds membranes enriched for phosphatidylinositol 4-phosphate, such as the trans Golgi network, while AP2 associates with phosphatidylinositol 4,5-bisphosphate of the plasma membrane. At their respective membranes, AP1 and AP2 bind the cytoplasmic tails of transmembrane protein cargo and clathrin triskelions, thereby coupling cargo recruitment to coat polymerization. Structural, biochemical and genetic studies have revealed that APs undergo conformational rearrangements and reversible phosphorylation to cycle between different activity states. While membrane, cargo and clathrin have been demonstrated to promote AP activation, growing evidence supports that membrane-associated proteins such as Arf1 and FCHo also stimulate this transition. APs may be returned to the inactive state via a regulated process involving phosphorylation and a protein called NECAP. Finally, because antiviral mechanisms often rely on appropriate trafficking of membrane proteins, viruses have evolved novel strategies to evade host defenses by influencing the conformation of APs. This review will cover recent advances in our understanding of the molecular inputs that stimulate AP1 and AP2 to adopt structurally and functionally distinct configurations.