Challenges with social network data mining

Before we close the chapter, let us look at the different challenges posed by social networks to the process of data mining. The following points present a few arguments, questions, and challenges:

• No doubt the data generated by social networks classifies as big data in every aspect. It has all the volume, velocity, and variety in it to overwhelm any system. Yet, interestingly, the challenge with such a huge source of data is the availability of enough granular data. If we zoom into our data sets and try to use data on a per user basis, we find that there isn't enough data to do some of the most common tasks, such as making recommendations!
• Social networks such as Twitter handle millions of users creating and sharing tons of data every second. To keep their systems up and running at all times, they put limits upon the amount of data that can be tapped using their APIs (security is also a major reason behind these limits, though). These limits put data science efforts in a quandary as it is difficult to obtain sufficient samples of data that represent the population correctly/completely. Insufficient samples may result in incorrect patterns or missing out on patterns altogether.
• Preprocessing and evaluation of results is also a challenge with social network analysis. While preprocessing data, we remove noisy content. With data coming in all shapes and sizes, determining noisy content is far more of a challenge than simply removing stopwords. Evaluation of results is another challenge, as there is no ground truth available in most cases, and due to the limitations presented here and otherwise, it is difficult to ascertain the validity of results with confidence.

The arguments/challenges presented above call for innovative and creative ways to be devised by data scientists, and that is what makes their job interesting and highly rewarding.