Exploring Google’s TabNet

Another possible approach to modeling tabular data using neural networks is Google’s TabNet. As TabNet is a complex model, we will not describe its architecture in depth. For that, we refer you to the original paper (mentioned in the See also section). Instead, we provide a high-level overview of TabNet’s main features:

• TabNet uses raw tabular data without any preprocessing.
• The optimization procedure used in TabNet is based on gradient descent.
• TabNet combines the ability of neural networks to fit very complex functions and the feature selection properties of tree-based algorithms. By using sequential attention to choose features at each decision step, TabNet can focus on learning from only the most useful features.
• TabNet’s architecture employs two critical building blocks: a feature transformer and an attentive transformer. The former processes the features into a more useful representation. The...