Pedestrian crowds can form the substrate of important socially contagious behaviors, including propagation of visual attention, violence, opinions, and emotional state. However, relating individual to collective behavior is often difficult, and quantitative studies have largely used laboratory experimentation. We present two studies in which we tracked the motion and head direction of 3,325 pedestrians in natural crowds to quantify the extent, influence, and context dependence of socially transmitted visual attention. In our first study, we instructed stimulus groups of confederates within a crowd to gaze up to a single point atop of a building. Analysis of passersby shows that visual attention spreads unevenly in space and that the probability of pedestrians adopting this behavior increases as a function of stimulus group size before saturating for larger groups. We develop a model that predicts that this gaze response will lead to the transfer of visual attention between crowd members, but it is not sufficiently strong to produce a tipping point or critical mass of gaze-following that has previously been predicted for crowd dynamics. A second experiment, in which passersby were presented with two stimulus confederates performing suspicious/irregular activity, supports the predictions of our model. This experiment reveals that visual interactions between pedestrians occur primarily within a 2-m range and that gaze-copying, although relatively weak, can facilitate response to relevant stimuli. Although the above aspects of gaze-following response are reproduced robustly between experimental setups, the overall tendency to respond to a stimulus is dependent on spatial features, social context, and sex of the passerby.