R-package Lattice: group by color doesn't work when using panel argument

I would like to plot a xyplot with lines. As an example I'm using the iris data and I know the actual plot doesn't really make sense :)

library(lattice)

panel.iris <- function(x,y){
    panel.xyplot(x,y)
    panel.lines(x,y)
    }

xyplot(Sepal.Length ~ Sepal.Width| Petal.Width, data = iris, groups= Species, 
auto.key=list(space="right"), panel=panel.iris)

This gives me the following figure: But I actually want it to be colored by group, like so (but with added lines): I can only achieve this when I'm not using the panel argument... Is there a way around this?

Many thanks!!