In this paper we present a new approach called cue-based networking that uses hints or cues about the physical environment to optimize networked application behavior. We define the notion of cues and describe how cues can be obtained using a wireless sensor network as the underlying platform. We identify both the research and system challenges that need to be addressed to realize benefits of the approach under a target application of video delivery over IP networks. In the process, we identify a key challenge of wireless sensor networks, namely the timeliness-robustness tradeoff. We design an adaptive algorithm that balances this tradeoff satisfying both timeliness and robustness requirements. Through an implementation of the video delivery application using the proposed algorithm in a real home environment, we highlight the practical benefits of the proposed approach.