Average CPU usage comparision for last and current week via prom query

The following query shows the percentage of CPU usage now:

100 - 100 * (avg by (group, instance, job) (irate(node_cpu_seconds_total{mode="idle"}[5m])))

And the following one shows the same one week ago:

100 - 100 * (avg by (group, instance, job) (irate(node_cpu_seconds_total{mode="idle"}[5m] offset 1w)))

See more information about the "offset" modifier in the Prometheus documentation here.

The following query returns the average per-host CPU usage over the last hour:

1 - avg(rate(node_cpu_seconds_total{mode="idle"}[1h])) by (instance)

The following query returns the average per-host CPU usage over the last hour week ago:

1 - avg(rate(node_cpu_seconds_total{mode="idle"}[1h] offset 1w)) by (instance)

This query differs from the query above by the offset modifier as Marcelo already mentioned in this answer.

Side note: it is recommended to use rate() instead of irate(), because irate() doesn't capture spikes. It returns results calculated over a subset of raw samples, and the subset of these points can change with every refresh of the graph. See more details in this article.

Now let's construct a query, which returns only hosts with the increased CPU usage by at least 10% during the last week:

(1 - avg(rate(node_cpu_seconds_total{mode="idle"}[1h])) by (instance))
  /
(1 - avg(rate(node_cpu_seconds_total{mode="idle"}[1h] offset 1w)) by (instance))
  > 1.1

The query divides the current CPU usage by the week-old CPU usage and compares it to 1.1.

If you want taking into account only hosts with the week-old CPU usage higher than 50%, then just add the corresponding comparison to the query:

(1 - avg(rate(node_cpu_seconds_total{mode="idle"}[1h])) by (instance))
  /
(
  (1 - avg(rate(node_cpu_seconds_total{mode="idle"}[1h] offset 1w)) by (instance))
  > 0.5
)
  > 1.1

P.S. I'd recommend using longer lookbehind windows in square brackets inside rate() when comparing the current CPU usage to the week-old CPU usage, in order to eliminate possible false positives when some hosts experience unstable CPU usage for short periods of time. Queries above use 1h lookbehind window. This may be OK, but, probably, it would be better using even bigger lookbehind windows. For example, 1d, in order to compare the average CPU usage metrics over the last day. Another option is to use shorter lookbehind window for the current CPU usage and longer lookbehind window for the week-old CPU usage.