We investigate different approaches to assessing the performance of multi-resource systems, i.e. networks of processes used to convert resource inputs to useful goods and services. For a given set of system outputs, alternative resource inputs are often possible so performance measures are needed to determine the best system configuration for a given goal. We define such performance measures according to a novel framework which categorises them into two types: those that can be calculated from a system's aggregate inputs and outputs (‘black-box' metrics, e.g. carbon footprint); and those that require knowledge of resource conversion processes within the system (‘grey-box' metrics). Urban areas are an important example application and metrics can be calculated from urban metabolism data. We calculate eight black-box metrics for fifteen global cities and find that performance is poorly correlated between the measures. This suggests that performance assessments should adopt grey-box approaches and consider flows at the level of individual processes within a city, using methods such as exergy analysis and ecological network analysis. We are led to suggest how to: (1) improve urban metabolism accounting to assist grey-box metric calculation, by including greater detail on conversion process and resource quality; and (2) promote these metrics amongst relevant decision makers.