This paper questions the binary structure of the notion “shared heritage”/ “patrimoine partagé” that has emerged in recent debates on built heritage in former colonial territories. In the discourses of, for instance, ICOMOS, the notion stands for a heritage “shared” by former “colonizers” and former “colonized”, both categories being considered – albeit often implicitly – as homogenous entities. In line with Stuart Hall, I will argue for an approach to colonial built heritage that takes up the more complex nature of the question “whose heritage?” By focusing on the remarkable colonial built architecture of the city of Lubumbashi, situated in the Democratic Republic of the Congo, I will make a plea for re-thinking and re-positioning this legacy as a critical filter between colonial history and postcolonial memory, thus extending traditional standards of documenting built legacy through formal description and physical assessment that often isolate buildings from their urban as well as historical contexts (social, economic, cultural and/or political). Being influenced by the work of the Mémoires de Lubumbashi-group as well as recent scholarship in the field of architectural history informed by postcolonial studies, the approach on built heritage presented here is twofold. On the one hand, a plea is made to link the city’s urban form to colonial history by relating it to the cosmopolitan society that produced and experienced it. On the other hand, an approach is suggested that acknowledges how specific urban places and buildings in the city are currently being re-appropriated as “lieux de mémoire” by a variety of agents that do not necessarily (want to) share this heritage.