FeedForward MLP

class vformer.encoder.nn.FeedForward(dim, hidden_dim=None, out_dim=None, p_dropout=0.0)[source]
Parameters

  • dim (int) – Dimension of the input tensor

  • hidden_dim (int, optional) – Dimension of hidden layer

  • out_dim (int, optional) – Dimension of the output tensor

  • p_dropout (float) – Dropout probability, default=0.0

forward(x)[source]
Parameters

x (torch.Tensor) – Input tensor

Returns

Returns output tensor by performing linear operations and activation on input tensor

Return type

torch.Tensor