Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gin.operative_config_str() does not show values changed during unlock #32

Closed
miguelvr opened this issue Sep 27, 2019 · 3 comments
Closed

gin.operative_config_str() does not show values changed during unlock #32

miguelvr opened this issue Sep 27, 2019 · 3 comments

Comments

@miguelvr
Copy link

I have a function that generates a random seed integer if None is provided.

Because I want to make my experiments reproducible I need to dump the exact value of the random seed in the config. I am using gin.operative_config_str() to do so. However, this string logs None for a seed value instead of the overloaded one.

Simple example:

import gin
import random
import numpy as np


def set_random_seed(seed=None):
    if seed is None:
        seed = random.randint(0, 1024)

    random.seed(seed)
    np.random.seed(seed)
    return seed


@gin.configurable('experiment')
def confgurable_fn(arg1, seed=None):
    seed = set_random_seed(seed)

    with gin.unlock_config():
        gin.bind_parameter("experiment.seed", seed)
    return {'arg1': arg1, 'seed': seed}


if __name__ == '__main__':
    gin.bind_parameter('experiment.arg1', 1)
    confgurable_fn()
    print(gin.operative_config_str())

Which prints

# Parameters for experiment:
# ==============================================================================
experiment.arg1 = 1
experiment.seed = None

Is this the expected behaviour? If so, is there any workaround?
@sguada
Copy link
Collaborator

sguada commented Sep 27, 2019

Gin tracks the parameters used during the call of the function, so what is used and not what is assigned later to it. But you could change your code to do that

@gin.configurable()
def set_random_seed(seed=None):
    if seed is None:
        seed = random.randint(0, 1024)
    random.seed(seed)
    np.random.seed(seed)
    return seed


@gin.configurable('experiment')
def confgurable_fn(arg1, seed=None):
    return {'arg1': arg1, 'seed': seed}


if __name__ == '__main__':
    # if you want to fix the seed uncomment the next line
    # gin.bind_parameter('set_random_seed.seed', 0) 
    gin.bind_parameter('experiment.arg1', 1)
    seed = set_random_seed()
    with gin.unlock_config():
        gin.bind_parameter("experiment.seed", seed)
    confgurable_fn()
    print(gin.operative_config_str())

@miguelvr
Copy link
Author

but then the config is misleading in the sense that you have 2 seed parameters and the experiment.seed is pretty much useless, as it doesn't set the seed.

I need a way to when re-running the program with the dumped configuration, get the exact same outcome. This does not seem to be the case with your proposed solution, unfortunately.

@miguelvr
Copy link
Author

solved it like this:

import gin
import random
import numpy as np


def set_random_seed(seed):
    random.seed(seed)
    np.random.seed(seed)
    return seed


def get_config_seed():
    try:
        seed = gin.query_parameter('experiment.seed')
    except ValueError:
        seed = random.randint(0, 1024)
        with gin.unlock_config():
            gin.bind_parameter('experiment.seed', seed)

    return seed


@gin.configurable('experiment')
def confgurable_fn(arg1, seed=None):
    seed = set_random_seed(seed)
    return {'arg1': arg1, 'seed': seed}


if __name__ == '__main__':
    gin.bind_parameter('experiment.arg1', 1)
    seed = get_config_seed()
    confgurable_fn()
    print(gin.operative_config_str())

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sguada @miguelvr