I am wondering what the minimum time is for Prometheus' scrape_interval parameter. According to the Prometheus Documentation, the value for this parameter needs to follow a regex which seems to me that only intervals equal or greater than 1 second are allowed, since, e.g. "1ms" or "0.01s" do not match this regex. In my application however, I would like to have scraping in milliseconds, so I am interested in whether this is possible with Prometheus.
Many thanks in advance!
According to the Prometheus documentation, the minimum value you can give for the scrape_interval seems to be 0 (according to the given regex in the docs).
Regex - ((([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?|0)
According to this regex, you can specify scrape_interval in ms as well. But you need to specify it as 0s1ms. This is because if you specify the time as 1ms; 1m will match with minutes and remaining s will cause an error (didn't really test this scenario, but looks like this is the expected outcome by looking at the regex).
While Prometheus supports scrape intervals smaller than one second as described in this answer, it isn't recommended to use scrape_interval values smaller than one second because of the following issues:
0.1ms - 100ms depending on the distance between Prometheus and scrape target.These non-deterministic delays may introduce big relative errors to scrape timings for scrape_interval values smaller than one second.
Too small scrape_interval values also may result in scrape errors if the target cannot be scraped during the configured scrape interval. In this case Prometheus would store up=0 metric for every unsuccessful scrape. See these docs about up metric.
P.S. If you need storing high-frequency samples into time series, then it would be better pushing these samples directly to a monitoring system, which supports push protocols for data ingestion. For example, VictoriaMetrics supports popular push protocols such as Influx, Graphite, OpenTSDB, CSV, DataDog etc. - see these docs for details. It supports timestamps with millisecond precision. If you need even higher precision for timestamps, then take a look at InfluxDB - it supports timestamps with nanosecond precision. Note that too high precision for timestamps usually leads to increased resource usage - disk space, RAM, CPU.
Disclosure: I work on VictoriaMetrics.