As ocean temperatures rise, species distributions are tracking towards historically cooler regions in line with their thermal affinity. However, different responses of species to warming and changed species interactions makes predicting biodiversity redistribution and relative abundance a challenge. Here we use three decades of fish and plankton survey data to assess how warming changes the relative dominance of warm-affinity and cold-affinity species. Regions with stable temperatures show little change in dominance structure (Northeast Pacific, Gulf of Mexico), while warming sees strong shifts towards warm-water species dominance (North Atlantic). Importantly, communities whose species pools had diverse thermal affinities and narrower range of thermal tolerance show greater sensitivity, as anticipated from simulations. Composition of fish communities changed less than expected in regions with strong temperature depth gradients. There, species track temperatures by moving deeper, rather than horizontally, analogous to elevation shifts in land plants. Temperature thus emerges as a fundamental driver for change in marine systems, with predictable restructuring of communities in the most rapidly warming areas using metrics based on species thermal affinities. The ready and predictable dominance shifts suggests a strong prognosis of resilience to climate change for these communities.