Feature engineering plays a vital role in making machine learning algorithms work and, if carried out properly, it enhances the predictive ability of machine learning algorithms. In other words, feature engineering is the process of extracting existing features or creating new features from the raw data using domain knowledge, the context of the problem, or specialized techniques that result in more accurate predictive models. This is an activity where domain knowledge and creativity play a very important role. This is an important process, which can significantly improve the performance of our predictive models. The more context you have about a problem, the better your ability to create new and useful features. Basically, the feature engineering process converts the features into input values that algorithms can understand.
There are various ways of implementing feature engineering. You might not find all of the techniques feasible and may end up excluding a few. The motive here is not to have an academic discussion about this topic, but to show you some of the common things that we do when we work with features and when we try to create new features. The first one is scaling features, used to transform their range to a more suitable one. The other one is to encode information in a better way, and we will see an example of this later in this chapter. Feature engineering involves creating new features from existing ones so that you can combine existing features by performing some mathematical operations on them.
Another way of creating new features is by using a dimensionality reduction technique, such as PCA, which we saw previously. It doesn't matter what technique you use, as long as you make it creative. As mentioned previously, the more knowledge you have about the problem, the better.