Identification of Trending Topics Using Periodically Collected Twitter Data
Keywords:Trend detection, TF-IDF, preprocessing, Twitter, gram.
Social media is an interactive personal tool to articulate an individual's cognizance. This project involves one such micro blogging platform, Twitter. Trends can simply be defined as the frequently mentioned topics throughout the stream of user activities. Mining twitter data for identifying trending topics provides an overview of the topics and issues that are currently popular within the online community. Therefore, the most effective and suitable methodology should be implemented to identify the short term high intensity discussion topic. The trigrams or higher order n-grams are used to determine the trending topic. Twitter Streaming API is used to collect data from the Twitter accounts using API keys and the formatted tweets are stored in a non SQL database. Subsequent steps include data cleansing followed by stemming. The processed data is subjected to trend prediction algorithms like DB Scan, Frequent Pattern Mining, Trees(fuzzy/inductive/decision), Soft frequent pattern mining and empirical statistics such as Frequency metric, TF-IDF, Normalized term frequency and Entropy based on the key parameters to identify the most trending event within a period of time. Thus, the trending topics can be detected with a reasonably close approximation to the expected outcome. This can be used in detecting and predicting events for an early warning system (or) prediction tools and also artificially intelligent services like web search system or recognition systems.
 Benhardus, James, and Jugal Kalita. "Streaming trend detection in twitter." International Journal of Web Based Communities 9.1, 122-139, 2013.
 Ishikawa Shota, Arakawa Yutaka, Tagashira Shigeaki, Fukuda Akira. "Hot topic detection in local areas using Twitter and Wikipedia." ARCS Workshops (ARCS), 2012. IEEE, 2012.
 Diao Qiming, Jiang Jing, Zhu Feida, Lim E-Peng. "Finding bursty topics from microblogs." Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, 2012.
 Boettcher, Alexander, and Dongman Lee. "Eventradar: A real-time local event detection scheme using twitter stream." Green Computing and Communications (GreenCom), 2012 IEEE International Conference on. IEEE, 2012.
 Lu, Rong, and Yang Qing. "Trend analysis of news topics on twitter." International Journal of Machine Learning and Computing 2.3, 327, 2012.
 Roy Soma, Gevry David, and M. Pottenger William. "Methodologies for trend detection in textual data mining." Proceedings of the Textmine. Vol. 2. 2002.
 Aiello, Luca Maria, et al. "Sensing trending topics in Twitter." IEEE Transactions on Multimedia 15.6,1268-1282 2013.
 Li Rui, Hou Lei Kin, Khadiwala Ravi, Chen-Chuan Chang Kevin. "Tedas: A twitter-based event detection and analysis system." Data engineering (icde), 2012 ieee 28th international conference on. IEEE, 2012.
 Ahangama, Sapumal. "Use of Twitter stream data for trend detection of various social media sites in real time." International Conference on Social Computing and Social Media. Springer, Cham, 2014.
 https://en.wikipedia.org/wiki/Twitter , Retrieved January 2018.
View Full Article:
How to Cite
LicenseAuthors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under aÂ Creative Commons Attribution Licensethat allows others to share the work with an acknowledgement of the work''s authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal''s published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (SeeÂ The Effect of Open Access).