If you are planning to use R a lot I would recommend investing in learning ggplot2. Base R plotting solutions get very limited very quickly.
To solve your problem I would make a data frame with every combination of effect size and sample size.
dat <- expand.grid(mydelt=c(0.5,0.6,0.7,0.8), ess=n_tot / deff)
Then add a column for the power:
dat$mypowers = power.t.test(n=dat$ess, delta=dat$mydelt)$power
Then I can use ggplot to easily make a nice graph of the power curves:
library(ggplot2)
ggplot(dat, aes(x=ess, y=mypowers, color=factor(mydelt))) + geom_point() + geom_line()

You can easily change the overall graph look and add appropriate labels:
ggplot(dat, aes(x=ess, y=mypowers, color=factor(mydelt))) +
geom_point() +
geom_line() +
theme_bw() +
labs(x="Effective sample size", y="Power", color="Effect size" )

In response to the comment.. there was a mistake in the code above in that I plotted the effective total sample size on the x axis not the sample size per cluster. So instead we should make sure we have n_participants in the dataset for plotting, then calculate the powers and plot:
So the whole script is now:
n_participants <- 5:40
npercluster <- 20
icc <- 0.6 # assumption
deff <- 1 + icc*(npercluster - 1)
dat <- expand.grid(mydelt=c(0.5,0.6,0.7,0.8), npart=n_participants)
dat$n_tot <- dat$npart*npercluster
dat$ess <- dat$n_tot / deff
dat$mypowers <- power.t.test(n=dat$ess, delta=dat$mydelt)$power
library(ggplot2)
ggplot(dat, aes(x=npart, y=mypowers, color=factor(mydelt))) +
geom_line()+
theme_bw() +
labs(x="Number of participants", y="Power", color="Effect size" )
Which gives this graph:
