extract input data from ggplot object

I know I can extract the data I plot in a ggplot2 plot using

p <- ggplot(df, aes(x, y)) + geom_point()
ggplot_build(p)$data

But data in this case is the data being plotted. Is there any way of getting the input data -- df in this case -- back?