Python embedding_init Examples

Example #1

    def reset_parameters(self):
        if self.add_position_embedding:
            nn.init.uniform_(self.embeddings.weight, - 1.0 / self.scale, 1.0 / self.scale)
        else:
            my_init.embedding_init(self.embeddings.weight)

        with torch.no_grad():
            self.embeddings.weight[self.padding_idx].fill_(0.0)

Example #2

    def _reset_parameters(self):

        my_init.embedding_init(self.proj.weight)