Despite considerable progress in robot learning, and contrary to the expectations of the general public, the vast majority of robots deployed out in the real world today continue to remain restricted to a narrow set of preprogrammed behaviors for specific tasks in controlled industrial settings. As robots become ubiquitous across human-centred environments, the need for “generalist” robots grows: how can we scale robot learning systems to autonomously acquire general-purpose knowledge that allows them to perform a wide range of everyday tasks in unstructured environments based on arbitrary instructions from the user? In my work, I have focused on addressing the challenging problem of relating human language to a robot’s perceptions and actions by introducing techniques that leverage self-supervision and structural priors from uncurated data to enable sample-efficient learning of language-conditioned manipulation and navigation tasks.