Abstract
This thesis presents a deep learning framework based on social group analysis to get a specific topic space. Our method will fuse the image and text after embedding into the same semantic space and train a deep learning model to predict the topic on the general user’s post. The prediction can help to mining user’s interests distribution and serve for personalized ads recommendation. The work that predicting social post topic usually exists the two challenging problems. One is a semantic gap between image feature and text bag of words. This problem causes the topic distribution dominate on text words results and the combination between two vary feature become useless. The other problem is how to fuse the social post images and texts, and take each advantage improving the topic prediction performance. The past work used the human-designed method to filter out the posts with inexplicit topics. However, not all of social posts can fit this mechanism. Therefore, we present the idea that hope to learn feature selection from dataset. Use the knowledge from the data, not by human-designed. It can learn more robust model. We proposed the deep learning architecture to fuse the social post image and text. The deep learning architecture consists three parts: the preprocessing and feature extraction part, embedding networks, fusion network. First, for the text posts, do text segmentation, and remove stop words, extract keywords. For the image posts, extract feature from the convolutional neural network, and use the top layer feature as embedding networks input. Then we embed two high semantic meanings features into the same space and domain and get initial topic prediction. After above step, we can use Max-Pooling to fuse these feature and obtain the fusion representation. Based on the fusion representation, designing a fully connected layer to learn a final topic prediction. When the image or text is not accurate, the fusion networks will find a suitable fusion results between two vary data and achieve more accurate user interests prediction.