Faster and More Resource-Efficient Intent Classification

Abstract: Intent classification is known to be a complex problem in Natural Language Processing (NLP) research. This problem represents one of the stepping stones to obtain machines that can understand our language. Several different models recently appeared to tackle the problem. The solution has become reachable with deep learning models. However, they have not achieved the goal yet.Nevertheless, the energy and computational resources of these modern models (especially deep learning ones) are very high. The utilization of energy and computational resources should be kept at a minimum to deploy them on resource-constrained devices efficiently.Furthermore, these resource savings will help to minimize the environmental impact of NLP.This thesis considers two main questions.First, which deep learning model is optimal for intent classification?Which model can more accurately infer a written piece of text (here inference equals to hate-speech) in a short text environment. Second, can we make intent classification models to be simpler and more resource-efficient than deep learning?.Concerning the first question, the work here shows that intent classification in written language is still a complex problem for modern models.However, deep learning has shown successful results in every area it has been applied.The work here shows the optimal model that was used in short texts.The second question shows that we can achieve results similar to the deep learning models by more straightforward solutions.To show that, when combining classical machine learning models, pre-processing techniques, and a hyperdimensional computing approach.This thesis presents a research done for a more resource-efficient machine learning approach to intent classification. It does this by first showing a high baseline using tweets filled with hate-speech and one of the best deep learning models available now (RoBERTa, as an example). Next, by showing the steps taken to arrive at the final model with hyperdimensional computing, which minimizes the required resources.This model can help make intent classification faster and more resource-efficient by trading a few performance points to achieve such resource-saving.Here, a hyperdimensional computing model is proposed. The model is inspired by hyperdimensional computing and its called ``hyperembed,'' which shows the capabilities of the hyperdimensional computing paradigm.When considering resource-efficiency, the models proposed were tested on intent classification on short texts, tweets (for hate-speech where intents are to offend or not to), and questions posed to Chatbots.In summary, the work proposed here covers two aspects. First, the deep learning models have an advantage in performance when there are sufficient data. They, however, tend to fail when the amount of available data is not sufficient. In contrast to the deep learning models, the proposed models work well even on small datasets.Second, the deep learning models require substantial resources to train and run them while the models proposed here aim at trading off the computational resources spend to obtaining and running the model against the classification performance of the model.