Word frequency effects have long served as an empirical and theoretical test bed for theories of language processing. A number of recent studies have suggested that Contextual Diversity (CD) is a better metric of retrieval processes than word frequency. Motivated by these findings, we sketch an active account of lexical access during sentence processing: language users store statistics about contextualized lexical representations and use lexical-contextual relations to both construct context and predict words given the context. In line with our account, we provide evidence from a frequency judgment experiment suggesting that words are not stored independently of their contexts of use. To further examine CD effects in reading, we analyzed reading times in self-paced reading and eye-tracking corpora. We demonstrate that as context is constructed, the role of CD in lexical retrieval is attenuated, reflecting a trade-off between context construction and contextualized word prediction.