I am trying to fill the polygons of a plot of mine with values of a calculated trend. Most of my plots work well. However, two of the variables I want to plot have a small number of extremely high values. I would not like to remove these extremes. I would simply like to tell geom_sf() to colour them differently whilst normally large values should be coloured a lighter orange.
I've attached an example of a plot I made with the issue I'm having. As you can see, the range of the scale is in the thousands but I need it to be more like -1 to 2. I have also included a snippet of the plotting code I'm using for context.
Is this just an issue of using a scale_fill_xxx function correctly? There are so many to look at, even some guidance on that would be a huge help.
Plot of the trends I require. Note the huge range of values thanks to the few outliers.

My code:
library(tidyverse)
library(sf)
library(rgdal) # shapefiles from before
ggplot() +
geom_sf(data=hydro_areas, colour="black", lwd=0.15) + # hydrometric areas
geom_sf(data=inla_results, aes(fill=rk.bsg.up), lwd=0.3, col="black") + # trends to plot
# setting scales
scale_fill_gradient2(name="Trend",
low="blue", mid="white", high="orange",
midpoint=0) +
xlab("Longitude") + ylab("Latitude") + theme_bw()