pyvene.models.mlp.modelings_mlp.MLPConfig

class MLPConfig(include_emb=False, vocab_size=50257, max_position_embeddings=512, n_layer=2, h_dim=512, num_classes=2, activation_function='gelu', pdrop=0.3, problem_type='single_label_classification', include_bias=True, squeeze_output=True, **kwargs)

Bases: PreTrainedConfig

__init__(include_emb=False, vocab_size=50257, max_position_embeddings=512, n_layer=2, h_dim=512, num_classes=2, activation_function='gelu', pdrop=0.3, problem_type='single_label_classification', include_bias=True, squeeze_output=True, **kwargs)

Methods

__init__([include_emb, vocab_size, ...])
convert_rope_params_to_dict([...])
dict_dtype_to_str(d)	Checks whether the passed dictionary and its nested dicts have a dtype key and if it's not None, converts torch.dtype to a string of just the type.
from_dict(config_dict, **kwargs)	Instantiates a [PreTrainedConfig] from a Python dictionary of parameters.
from_json_file(json_file)	Instantiates a [PreTrainedConfig] from the path to a JSON file of parameters.
from_pretrained(pretrained_model_name_or_path)	Instantiate a [PreTrainedConfig] (or a derived class) from a pretrained model configuration.
get_config_dict(...)	From a pretrained_model_name_or_path, resolve to a dictionary of parameters, to be used for instantiating a [PreTrainedConfig] using from_dict.
get_text_config([decoder, encoder])	Returns the text config related to the text input (encoder) or text output (decoder) of the model.
push_to_hub(repo_id, *[, commit_message, ...])	Upload the configuration file to the 🤗 Model Hub.
register_for_auto_class([auto_class])	Register this class with a given auto class.
save_pretrained(save_directory[, push_to_hub])	Save a configuration object to the directory save_directory, so that it can be re-loaded using the [~PreTrainedConfig.from_pretrained] class method.
standardize_rope_params()	Helper to standardize the config's rope params field by ensuring the params are defined for each later type.
to_dict()	Serializes this instance to a Python dictionary.
to_diff_dict()	Removes all attributes from the configuration that correspond to the default config attributes for better readability, while always retaining the config attribute from the class.
to_json_file(json_file_path[, use_diff])	Save this instance to a JSON file.
to_json_string([use_diff])	Serializes this instance to a JSON string.
update(config_dict)	Updates attributes of this class with attributes from config_dict.
update_from_string(update_str)	Updates attributes of this class with attributes from update_str.
validate_rope([ignore_keys])	Validate the RoPE config arguments, given a "PreTrainedConfig" object

Attributes

attribute_map
base_config_key
base_model_ep_plan
base_model_pp_plan
base_model_tp_plan
default_theta
has_no_defaults_at_init
model_type
name_or_path
num_labels	int: The number of labels for classification models.
output_attentions	bool: Whether or not the model should returns all attentions.
rope_scaling
sub_configs
torch_dtype
use_return_dict	bool: Whether or not return [~utils.ModelOutput] instead of tuples.