pyvene.models.backpack_gpt2.modelings_backpack_gpt2.BackpackGPT2Config#
- class BackpackGPT2Config(vocab_size=50264, num_senses=16, sense_intermediate_scale=4, n_positions=512, scale_attn_by_inverse_layer_idx=True, **kwargs)[source]#
Bases:
GPT2Config
This is the configuration class to store the configuration of a [GPT2Model] or a [TFGPT2Model]. It is used to instantiate a Backpack GPT-2 model according to the specified arguments, defining the model architecture. Configuration objects inherit from [GPT2Config] and can be used to control the model outputs. Read the documentation from [GPT2Config] for more information. Args:
- num_senses (int, optional, defaults to 16):
The number of sense vectors to define for each word.
- sense_intermediate_scale (int, optional, defaults ot 4):
The hidden dimensionality of the sense vector network.
Example: ```python >>> from transformers import BackpackGPT2Config, BackpackGPT2Model >>> # Initializing a GPT2 configuration >>> configuration = BackpackGPT2Config() >>> # Initializing a model (with random weights) from the configuration >>> model = BackpackGPT2Model(configuration) >>> # Accessing the model configuration >>> configuration = model.config
- __init__(vocab_size=50264, num_senses=16, sense_intermediate_scale=4, n_positions=512, scale_attn_by_inverse_layer_idx=True, **kwargs)[source]#
Methods
__init__
([vocab_size, num_senses, ...])dict_torch_dtype_to_str
(d)Checks whether the passed dictionary and its nested dicts have a torch_dtype key and if it's not None, converts torch.dtype to a string of just the type.
from_dict
(config_dict, **kwargs)Instantiates a [PretrainedConfig] from a Python dictionary of parameters.
from_json_file
(json_file)Instantiates a [PretrainedConfig] from the path to a JSON file of parameters.
from_pretrained
(pretrained_model_name_or_path)Instantiate a [PretrainedConfig] (or a derived class) from a pretrained model configuration.
get_config_dict
(...)From a pretrained_model_name_or_path, resolve to a dictionary of parameters, to be used for instantiating a [PretrainedConfig] using from_dict.
get_text_config
([decoder])Returns the config that is meant to be used with text IO.
push_to_hub
(repo_id[, use_temp_dir, ...])Upload the configuration file to the 🤗 Model Hub.
register_for_auto_class
([auto_class])Register this class with a given auto class.
save_pretrained
(save_directory[, push_to_hub])Save a configuration object to the directory save_directory, so that it can be re-loaded using the [~PretrainedConfig.from_pretrained] class method.
to_dict
()Serializes this instance to a Python dictionary.
to_diff_dict
()Removes all attributes from config which correspond to the default config attributes for better readability and serializes to a Python dictionary.
to_json_file
(json_file_path[, use_diff])Save this instance to a JSON file.
to_json_string
([use_diff])Serializes this instance to a JSON string.
update
(config_dict)Updates attributes of this class with attributes from config_dict.
update_from_string
(update_str)Updates attributes of this class with attributes from update_str.
Attributes
attribute_map
is_composition
keys_to_ignore_at_inference
model_type
name_or_path
num_labels
int: The number of labels for classification models.
use_return_dict
bool: Whether or not return [~utils.ModelOutput] instead of tuples.