This paper presents a widely applicable theory of segregation based on the value of pairwise interactions. In our theory individuals are located in a spatial landscape that determines the frequency or intensity of interactions. A landscape can be the set of locations in a city (residences), schools in district or even consumptions such as media providers. The second primitive of the theory is a set of social characteristics that defines social groups such as race, ethnicity, income or ideological preferences. In our theory, the value of an interaction increases with proximity in the spatial landscape and distance in the social space. Segregation is postulated as the lack of social value associated to intense interactions between socially proximate individuals. We provide an axiomatic foundation of the theory and derive a representation for segregation indexes that can be expressed as the sum across pairs of the product between the intensity of a pairwise interaction and the value of the (lack of) social diversity of each pair. For a large class of questions, there is a natural distance in the spatial landscape (e.g. distance in a city, pertaining to same/different schools, consuming different media) such that interaction intensity is inversely related to pairwise distance. Similarly, the social distance between two individuals (income differences, same/different race, ideological differences) is assumed to generate higher value. In these cases -distance-based measures- we show that segregation is proportional to the covariance between the spatial and social distances. Segregation increases if spatial proximity is associated with social proximity. We generalize the theory for individuals who can occupy multiple locations and spaces of multidimensional social characteristics. Empirical applications on socioeconomic segregation in Chilean schools and the ideological segregation of media consumption in European countries are used to illustrate the method.