I could not find a built-in function in Python to generate a log uniform distribution given a min and max value (the R equivalent is here), something like: loguni[n, exp(min), exp(max), base] that returns n log uniformly distributed in the range exp(min) and exp(max).
The closest I found though was numpy.random.uniform.
SciPy v1.4 includes a loguniform random variable: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.loguniform.html
Here's how to use it:
from scipy.stats import loguniform
rvs = loguniform.rvs(1e-2, 1e0, size=1000)
This will create random variables evenly spaced between 0.01 and 1. That best shown by visualizing the log-scaled histogram:
This "log-scaling" works regardless of base; loguniform.rvs(2**-2, 2**0, size=1000) also produces log-uniform random variables. More details are in loguniform's documentation.
I believe the scipy.stats.reciprocal is the distribution you want.
From the documentation:
The probability density function for reciprocal is:
f(x, a, b) = \frac{1}{x \log(b/a)}for a <= x <= b and a, b > 0
reciprocal takes
aandbas shape parameters.
from random import random
from math import log
def loguniform(lo,hi,seed=random()):
return lo ** ((((log(hi) / log(lo)) - 1) * seed) + 1)
You can check this using a specific seed value: lognorm(10,1000,0.5) returns 100.0
Simply use the .rvs() method provided:
class LogUniform(HyperparameterDistribution):
"""Get a LogUniform distribution.
For example, this is good for neural networks' learning rates: that vary exponentially."""
def __init__(self, min_included: float, max_included: float):
"""
Create a quantized random log uniform distribution.
A random float between the two values inclusively will be returned.
:param min_included: minimum integer, should be somehow included.
:param max_included: maximum integer, should be somehow included.
"""
self.log2_min_included = math.log2(min_included)
self.log2_max_included = math.log2(max_included)
super(LogUniform, self).__init__()
def rvs(self) -> float:
"""
Will return a float value in the specified range as specified at creation.
:return: a float.
"""
return 2 ** random.uniform(self.log2_min_included, self.log2_max_included)
def narrow_space_from_best_guess(self, best_guess, kept_space_ratio: float = 0.5) -> HyperparameterDistribution:
"""
Will narrow, in log space, the distribution towards the new best_guess.
:param best_guess: the value towards which we want to narrow down the space. Should be between 0.0 and 1.0.
:param kept_space_ratio: what proportion of the space is kept. Default is to keep half the space (0.5).
:return: a new HyperparameterDistribution that has been narrowed down.
"""
log2_best_guess = math.log2(best_guess)
lost_space_ratio = 1.0 - kept_space_ratio
new_min_included = self.log2_min_included * kept_space_ratio + log2_best_guess * lost_space_ratio
new_max_included = self.log2_max_included * kept_space_ratio + log2_best_guess * lost_space_ratio
if new_max_included <= new_min_included or kept_space_ratio == 0.0:
return FixedHyperparameter(best_guess).was_narrowed_from(kept_space_ratio, self)
return LogUniform(2 ** new_min_included, 2 ** new_max_included).was_narrowed_from(kept_space_ratio, self)
The original project also includes a LogNormal distribution if that also interests you.
Source:
License:
from neuraxle.hyperparams.distributions import LogUniform
# Create a Log Uniform Distribution that ranges from 0.001 to 0.1:
learning_rate_distribution = LogUniform(0.001, 0.1)
# Get a Random Value Sample (RVS) from the distribution:
learning_rate_sample = learning_rate_distribution.rvs()
print(learning_rate_sample)
Example output:
0.004532
This is using Neuraxle.
A better approach would be instead of directly generating a sample from a log-uniform, you should create the log-uniform density.
In statistics speak, that is a reciprocal distribution which is already in SciPy: scipy.stats.reciprocal. For example, to build a sample that is 10^{x~U[-1,1]}, you would do:
rv = scipy.stats.reciprocal(a=0.1,b=10)
x = rv.rvs(N)
Alternatively, I wrote and use the following code to take the log-transform of any scipy.stats-like (frozen) random-variable
class LogTransformRV(scipy.stats.rv_continuous):
def __init__(self,rv,base=10):
self.rv = rv
self.base = np.e if base in {'e','E'} else base
super(LogTransformRV, self).__init__()
self.a,self.b = self.base ** self.rv.ppf([0,1])
def _pdf(self,x):
return self.rv.pdf(self._log(x))/(x*np.log(self.base)) # Chain rule
def _cdf(self,x):
return self.rv.cdf(self._log(x))
def _ppf(self,y):
return self.base ** self.rv.ppf(y)
def _log(self,x):
return np.log(x)/np.log(self.base)