This article uses big data from images captured by Google Street View (GSV) to analyse the extent to which the built environment impacts the survival rate of neighbourhood-based social organisations in Amsterdam, the Netherlands. These organisations are important building blocks for social life in urban neighbourhoods. Examining these organisations’ relationships with their environment has been a useful way to study their vitality. To extract data on built environment features from GSV images, we applied a deep learning model, DeepLabv3+. We then used elastic net regression to test the relationship between the built environment empirically – distinguishing between car-related, walking-related and mixed-use land infrastructure – and the survival of neighbourhood organisations. This testing approach is novel, to our knowledge not yet having been applied in Urban Studies. Besides revealing the effects of built environment features on the social life between buildings, our study points to the value of easily applicable observational big data. Data captured by GSV and other recently developed methods offer researchers the opportunity to conduct detailed yet relatively swift and inexpensive studies without resorting to overly coarse or common subjective measurements.