The primary goal of our paper is to quantify the importance of imperfect competition in the U.S. labor market by estimating the size of rents earned by American firms and workers from ongoing employment relationships. To this end, we construct a matched employer-employee panel data set by combining the universe of U.S. business and worker tax records for the period 2001-2015. We describe several important features of the U.S. labor market, including the size of firm-specific wage premiums, the sorting of workers to firms, the production complementarities between high ability workers and productive firms, and the pass-through of firm and market shocks to workers’ wages. Guided by these empirical results, we develop, identify and estimate an equilibrium model of the labor market with two-sided heterogeneity where workers view firms as imperfect substitutes because of heterogeneous preferences over non-wage job characteristics. The model allows us to draw inference about imperfect competition, compensating differentials and rent sharing. We also use the model to quantify the relevance of non-wage job characteristics and imperfect competition for inequality and tax policy, to assess the economic determinants of worker sorting, and to offer a unifying explanation of key empirical features of the U.S. labor market.