Caching in IPython (notebooks)

memoize to disk - python - persistent memoization

Most answers are in a decorator fashion. But maybe I don't want to cache the result every time when calling the function. I made one solution using context manager, so the function can be called as with DiskCacher('cache_id', myfunc) as myfunc2: res=myfunc2(...) when you need the caching functionality.

https://stackoverflow.com/questions/16463582/memoize-to-disk-python-persistent-memoization

joblib.Memory - joblib 1.2.0.dev0 documentation

A context object for caching a function's return value each time it is called with the same input arguments. All values are cached on the filesystem, in a deep directory structure. Read more in the . Parameters: location: str, pathlib.Path or None The path of the base directory to use as a data store or None.

https://joblib.readthedocs.io/en/latest/generated/joblib.Memory.html

Step 1

from joblib import Memory

cachedir = "my-cache"
memory = Memory(cachedir, verbose = 0)

Step 2

• A classical way to check the caching is to have a print statement inside.

• The function will print its output only on the first run.

• Starting from the second run, its value will be taken from a dictionary.

@memory.cache
def f(x):
    print('Running f(%s)...' % x)
    return x

>>> first = f(1) # First run
Running f(1)...

>>> second = f(1) # Second run

>>> print(first)
1

>>> print(second)
1

Step 3

## We can clear the cache of a specific function
>>> f.clear()

## And now, the values need to be recomputed again
>>> f(1)
Running f(1)...
1

Step 4

@memory.cache
def f(x):
    print('The function has changed f(%s)' % x)
    return x

>>> f(1)
The function has changed f(1)
1

Step 5

>>> memory.clear()