Jekyll2021-01-25T17:53:57-05:00https://complexdatalabmcgill.github.io/feed.xmlComplex Data Lab McGillThe McGill University's Complex Data Lab blog.Announcing Our Paper at KDD 20202020-08-20T00:00:00-04:002020-08-20T00:00:00-04:00https://complexdatalabmcgill.github.io/papers/post-andy-kdd2020paper<h3 id="laplacian-change-point-detection-for-dynamic-graphs"><em>Laplacian Change Point Detection for Dynamic Graphs</em></h3>
<blockquote>
<p><em>Authors: <a href="https://www.cs.mcgill.ca/~shuang43/">Shenyang Huang</a>, Yasmeen Hitti, <a href="https://www-labs.iro.umontreal.ca/~grabus/">Guillaume Rabusseau</a>, <a href="http://www.reirab.com/">Reihaneh Rabbany</a></em></p>
<p><a href="https://www.kdd.org/kdd2020/accepted-papers/view/laplacian-change-point-detection-for-dynamic-graphs">[KDD page (video)]</a> <a href="https://arxiv.org/pdf/2007.01229.pdf">[arXiv preprint]</a> <a href="https://github.com/shenyangHuang/LAD">[github]</a></p>
</blockquote>
<!--more-->
<iframe src="https://player.vimeo.com/video/443859563" width="440" height="160" frameborder="0" allow="autoplay; fullscreen" allowfullscreen=""></iframe>
<font size="-1">
<center>
<p>
<a href="https://vimeo.com/443859563">Laplacian Change Point Detection for Dynamic Graphs</a>
from
<a href="https://www.kdd.org/kdd2020/accepted-papers/view/laplacian-change-point-detection-for-dynamic-graphs">SIGKDD Videos</a>.
</p>
</center>
</font>
<p>if you have any questions, please reach out to me by <a href="mailto:shenyang.huang@mail.mcgill.ca">email</a></p>
<p><img src="https://i.imgur.com/venuydJ.jpg" alt="" /></p>
<p><strong>Problem</strong>: Identifying anomalous snapshots in a dynamic graph.</p>
<p><strong>Contributions</strong>:</p>
<ol>
<li>proposed Laplacian Anomaly Detection (LAD): a novel change point detection method that summarizes each graph snapshot with the singular values of the graph Laplacian</li>
<li>LAD explicitly captures both the short term and the long term temporal relations to model the abrupt and gradual changes in dynamic networks</li>
<li>Extensively evaluated LAD on synthetic and real world experiments and showed that LAD outperforms state-of-theart methods</li>
</ol>
<p><strong>Data</strong>:
All data is avaible in github repository</p>
<ol>
<li>Canadian bill voting network (original political network documenting interation between Members of Parliament in Canada)</li>
<li>US Senate co-sponsorship network</li>
<li>UCI Message social network</li>
</ol>
<p><strong>Abstract</strong>:</p>
<p>Dynamic and temporal graphs are rich data structures that are used to model complex relationships between entities over time. In particular, anomaly detection in temporal graphs is crucial for many real world applications such as intrusion identification in network systems, detection of ecosystem disturbances and detection of epidemic outbreaks. In this paper, we focus on change point detection in dynamic graphs and address two main challenges associated with this problem: I) how to compare graph snapshots across time, II) how to capture temporal dependencies. To solve the above challenges, we propose Laplacian Anomaly Detection (LAD) which uses the spectrum of the Laplacian matrix of the graph structure at each snapshot to obtain low dimensional embeddings. LAD explicitly models short term and long term dependencies by applying two sliding windows. In synthetic experiments, LAD outperforms the state-of-the-art method. We also evaluate our method on three real dynamic networks: UCI message network, US senate co-sponsorship network and Canadian bill voting network. In all three datasets, we demonstrate that our method can more effectively identify anomalous time points according to significant real world events</p>Andy HuangLaplacian Change Point Detection for Dynamic Graphs Authors: Shenyang Huang, Yasmeen Hitti, Guillaume Rabusseau, Reihaneh Rabbany [KDD page (video)] [arXiv preprint] [github]KDD + ASONAM 2019 Misinformation Papers2020-06-10T00:00:00-04:002020-06-10T00:00:00-04:00https://complexdatalabmcgill.github.io/blog/post-kellin-kdd<p>This post is part of a literature review on misinformation, especially in relation to technology and social media. The intention is to organize my thoughts and to provide the reader with condensed information beyond what can be found in the abstract.</p>
<!--more-->
<p>For each paper I give the setting, main contributions, data used, and some strengths and limitations. These opinions are my own and are presented to give the reader some ideas. They do not represent an ironclad critical review; your judgment may vary, and so might mine on subsequent readings. Also, my depth of reading varies depending on relevance to our research - some papers that weren’t very relevant are marked below as skimmed or not read at all.</p>
<p>The following are from searching for the keyword <strong>“misinformation,”</strong> among <strong>2019 KDD</strong> (Knowledge Discovery and Data Mining), <strong>ASONAM</strong> (Advances in Social Networks Mining and Analysis) papers. The latter conference accepts two submission formats; I indicate them here by the title as “short” (4 pages or less) and “long” (8 pages or less). <a href="https://dl.acm.org/action/doSearch?AllField=misinformation&expand=all&ConceptID=119664&AfterYear=2019&BeforeYear=2019&queryID=30/815270665" target="_blank">Link to search</a>.</p>
<p><strong>18 results total.</strong> Results are ordered here according to the search results order.</p>
<p>Note for members of the Complex Data Lab: I have an internal version which includes potential uses for each paper in relation to our SN project specifically. Please check Slack or contact me for this version.</p>
<p><strong>Online Misinformation: From the Deceiver to the Victim (ASONAM short)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3343536" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3343536</a></p>
<p>Setting: examines properties of fake news publishers, spreaders, and victims.</p>
<p>Main Contributions: partisanship of publishers is partially predictive of fakeness (more partisan → more fake). Aggregate network properties are slightly predictive of fake news spreaders. Dissemination and vulnerability to fake news among seniors is significant and has not been sufficiently studied compared to other groups.</p>
<p>Data: Partisanship: MediaBiasFactCheck. Network aggregates: Politifact and Buzzfeed datasets from FakeNewsNet.</p>
<p>Strengths: partisanship as a predictor is seemingly obvious but perhaps overlooked; analysis here is clean and simple. Use of network aggregates contrasts <a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380180" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380180</a>.</p>
<p>Limitations: highlights further research needed with seniors, but they explicitly state their data collected there is too small to be of use beyond qualitative suggestions.</p>
<p><strong>Fake News Research: Theories, Detection Strategies, and Open Problems (KDD Tutorial)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3292500.3332287" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3292500.3332287</a></p>
<p>Setting: This is the summary of a tutorial. Unfortunately I was unable to find any recording of the tutorial itself, but materials including slides are available at <a href="https://www.fake-news-tutorial.com/" target="_blank">https://www.fake-news-tutorial.com/</a>. The main sections of the tutorial are 1 - “fake news and related concepts,” 2 - “fundamental theories,” 3 - “detection strategies,” and 4 - “open issues.”</p>
<p>Strengths: sections 1, 2, and 4 are nice overviews. There’s a lot of potentially interesting papers and algorithms presented in section 3.</p>
<p>Limitations: it’s disappointing that there doesn’t seem to be a recording. Section 3 in particular can be hard to follow from the slides alone; referring to the original papers will often be needed if seeking a thorough understanding.</p>
<p><strong>Understanding Information Operations using YouTubeTracker (ASONAM short)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3343704" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3343704</a></p>
<p>Setting: suggests lack of in-depth tools for analyzing Youtube content, user behavior, and networks. Proposes tool to rectify that.</p>
<p>Limitations: seemingly no longer exists - no website at given link and cursory Google search did not find it either, though could be buried deeper.</p>
<p><strong>Semi-Supervised Learning and Graph Neural Networks for Fake News Detection (ASONAM short)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3342958" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3342958</a></p>
<p>Setting: labels for fake news detection are often limited. This paper examines a semi-supervised approach to mitigate label shortages.</p>
<p>Main Contributions: Algorithm that builds a graph on content embeddings of articles, relating ones that are labeled with similar ones that are not.</p>
<p>Data: 150 labeled articles, 75 real 75 fake, from Horne, B. D., and Adali, S. (2017).</p>
<p>Strengths: using semi-supervised learning seems worthwhile to investigate in this context. Significant performance improvements with 5% or less labeled data.</p>
<p>Limitations: while experimental results suggest their algorithm is working, with 10% or more labeled data the performance benefit is minimal or none. They compare with one other method (Guacho et al. 2018) and two simpler baseline methods (SVM and random forest models on bag-of-words features), but those baselines outperform Guacho et al. in every test, suggesting Guacho et al. may not be appropriate for this setting or data and therefore the comparison is not strong. Their embedding and graph construction methods (GloVe and k-nn) are not the most sophisticated ones out there.</p>
<p><strong>5 sources of clickbaits you should know! using synthetic clickbaits to improve prediction and distinguish between bot-generated and human-written headlines (ASONAM long)</strong></p>
<p><a href="https://dl.acm.org/doi/abs/10.1145/3341161.3342875" target="_blank">https://dl.acm.org/doi/abs/10.1145/3341161.3342875</a></p>
<p>Setting: clickbait headline data is limited. This paper proposes to generate more data using crowd-sourcing and VAE (separately).</p>
<p>Main Contributions: algorithms and analysis of several related questions: performance of detection algorithms augmented by synthetic data, differences between synthetic and real headlines, and detecting synthetic vs. real ones.</p>
<p>Data: real click bait data from mainstream news and social media, from Rony et al. 2017. Crowdsourced data from Amazon Mechanical Turk and journalism students.</p>
<p>Strengths: synthetic data improves performance of a variety of detection algorithms. It is nonetheless somewhat possible to detect the synthetic ones.</p>
<p>Limitations: VAE and infoVAE only generation algorithms tested. Insufficient how well humans can detect real vs. synthetic (relevant for determining how much room remains for improvement).</p>
<p><strong>A Postmortem of Suspended Twitter Accounts in the 2016 U.S. Presidential Election (ASONAM long)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3342878" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3342878</a></p>
<p>Setting: exploratory/descriptive analysis of suspended accounts and new Twitter countermeasures, particularly related to 2016 US election.</p>
<p>Main Contributions: identifies several network-related factors that show differences between suspended and non-suspended communities. Compares suspended users over time and notes many more recent suspended users are related to old suspended users.</p>
<p>Data: roughly 90 million tweets from 9.5mil users collected over 4 months, of which 9.2mil tweets are from 900k suspended users.</p>
<p>Strengths: this avoids the sparsity issue due to Twitter streaming API maxing out at 1% of tweets, sometimes problematic (see “On Twitter Purge: A Retrospective Analysis of Suspended Users,” Chowdhury et al. 2020), by focusing on a particular topic. The analysis is fairly in depth compared to some descriptive papers.</p>
<p>Limitations: focus on one topic makes it more difficult to use results on future topics.</p>
<p><strong>News Credibility Scroing [sic]: Suggestion of research methodology to determine</strong></p>
<p><strong>the reliability of news distributed in SNS (ASONAM short)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3343683" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3343683</a></p>
<p>Setting: proposes system for evaluating credibility in social media, with particular focus on Facebook documents.</p>
<p>Main Contributions: suggests importance of considering user roles: creator, distributor, or follower. Aims to give a single fairly understandable score representing the probability a document is credible.</p>
<p>Data: Facebook, with Amazon Mechanical Turk for labels (credible/not credible). Data does not appear to be collected yet.</p>
<p>Strengths: some detailed thoughts on evaluating credibility.</p>
<p>Limitations: at the time this paper was written, this was a proposed method, not a finished or even prototyped one. Logistic regression seems a difficult choice to use in this setting, given hefty dependence on feature engineering and the complexity of the problem.</p>
<p><strong>MediaRank: Computational Ranking of Online News Sources (KDD)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3292500.3330709" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3292500.3330709</a></p>
<p>Setting: automatic ranking of news domains according to “peer reputation, reporting bias/breadth, bottomline financial pressure, and popularity.”</p>
<p>Main Contributions: system to do above in a scalable and automated way, including website/API.</p>
<p>Data: parses discovered webpages and URLs from tweets to identify news sites. Then analyzes their characteristics to compute ranking. Uses various other data sources for evaluation, such as AllSides, MediaBiasFactCheck, DBpedia, Botometer, etc.</p>
<p>Strengths: large scale: 50k+ websites ranked, collects “about one million raw HTMLs and two million news related tweets each day.”</p>
<p>Limitations: I have concerns about what these rankings represent and how useful they are. Case study 1: breitbart.com - one of the suggested examples on their website’s home page - is ranked 97/50695. This news source is well known for its strong to extreme conservative bias (<a href="https://mediabiasfactcheck.com/breitbart/" target="_blank">https://mediabiasfactcheck.com/breitbart/</a>, <a href="https://www.politifact.com/personalities/breitbart/" target="_blank">https://www.politifact.com/personalities/breitbart/</a>,</p>
<p><a href="https://www.statista.com/statistics/649146/breitbart-credibility-usa/" target="_blank">https://www.statista.com/statistics/649146/breitbart-credibility-usa/</a>, etc.). The ranking puts it ahead of Encyclopedia Britannica (116), the Associated Press (125), Harvard and MIT (122 and 168), TED (170), etc. This is possibly due to the fact that its ranking for the political bias of Breitbart appears to be missing (marked “not found” which hopefully means that there is an error, not that it is unbiased). This is incidentally not an error unique to Breitbart - for instance, the BBC also has political bias ranking “not found.”</p>
<p>Case study 2: there are many websites/entities on the list that do not appear to represent news, such as Target (rank 180), Ikea (rank 185), and the Italian homepage of Amazon (amazon.it, rank 111).</p>
<p>Although this evidence is qualitative rather than quantitative, it is not clear to me exactly what the rankings represent, and therefore how useful they would be in downstream applications.</p>
<p>Also, the API link appears to redirect to a comparison with other similar sites, and contains no information about using the API.</p>
<p><strong>SAME: Sentiment-Aware Multi-Modal Embedding for Detecting Fake News (ASONAM long)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3342894" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3342894</a></p>
<p>Setting: multi-modal (i.e. using different types of data) strategy for detecting fake news, with particular focus on examining comment sentiment as a feature.</p>
<p>Main Contributions: algorithm that can do above, shows user sentiment is relevant, and outperforms 5 “state-of-the-art baselines.”</p>
<p>Data: PolitiFact and GossipCop from FakeNewsNet.</p>
<p>Strengths: good ideas for combining different data modes, which are validated by their ablation study. Their method universally outperforms the 5 methods they compared with.</p>
<p>Limitations: uses images (with the article itself), content, sentiment, similarity but not user networks, syntax, or website structure. In other words, some modes that are shown relevant in other works are missing here. Individual mode methods such as VGGNet and GloVe are old and there may be better ones.</p>
<p><strong>Dormant Bots in Social Media: Twitter and the 2018 U.S. Senate Election (ASONAM short)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3343852" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3343852</a></p>
<p>Setting: identification of “dormant bots,” which are social media accounts which have not posted content but have “large follower and friend relationships with other users.” These could influence rankings or be used to post maliciously in the future.</p>
<p>Main Contributions: highlights the problem and provides methods to detect dormant bots. Applied them in real-world case with very significant results.</p>
<p>Data: follower networks of 21 Democrat and 4 Republican incumbent senators in 2018 election. In total about 6mil followers. Data collected from Twitter REST API including names, followers, friends, posts, account creation dates, and (optionally user-defined) location.</p>
<p>Strengths: problem makes sense. Real-world application had extremely strong effects, leading to Twitter suspending 70 million users after FBI court order, and subsequently revising their API.</p>
<p>Limitations: single case study. Evaluating effects and intentions of these bots is difficult, given they have no behavior, though this is not a limitation of the paper so much as the setting.</p>
<p><strong>Multitask Learning for Blackmarket Tweet Detection (ASONAM short)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3342934" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3342934</a></p>
<p>Setting: detect usage of blackmarket services to promote tweets, e.g. requesting blackmarket participants to retweet.</p>
<p>Main Contributions: algorithm to classify if blackmarket or not and estimate number of likes and retweets after 5 days from post. Combines structural features like number of hashtags, mentions, URLs, nouns, adjectives, etc. with content representation. Cross stitch to put these two feature sets together.</p>
<p>Data: scraped about 1800 tweets from blackmarket websites and combined with 2000 tweets from the same authors that weren’t on the blackmarket. Topic annotations by hand on test set: promotional, entertainment, spam, news, politics, others.</p>
<p>Strengths: new (and worthwhile) area of investigation. Experimental results look promising.</p>
<p>Limitations: potential label validity concerns: since “non-blackmarket” tweets are collected from the same users as blackmarket ones, they might be posting them to other blackmarkets that weren’t scraped. Further, the real-world task would generally be identifying blackmarket tweets from random users, not identifying blackmarket tweets from users who definitely use the blackmarket.</p>
<p><strong>Monitoring Individuals in Drug Trafficking Organizations: A Social Network Analysis (ASONAM short) **<em>and</em> **On Augmented Identifying Codes for Monitoring Drug Trafficking Organizations (ASONAM short)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3342938" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3342938</a></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3343530" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3343530</a></p>
<p>Setting: two closely-related papers by same authors. Examines which individuals to surveil in networks related to drug trafficking, under the assumption that one can extract all relevant information about any person in the graph from their neighbor as well as themselves.</p>
<p>Main Contributions: theory and algorithms to allocate surveillance efficiently, both from scratch (first paper) and if you already have some surveillance deployed and want to deploy more (second paper).</p>
<p>Data: various datasets related to drug trafficking networks obtained from rea-lworld investigations: Operation Juanes, Operation Acero, Operation Mambo, Operation Jake, Heroin Dealing, Montreal Street Gangs, Cocaine Dealers, and DrugNet.</p>
<p>Strengths: theoretical results lead to algorithm that has certain optimality (given assumptions).</p>
<p>Limitations: the big assumption - that one can extract all information with only surveillance on neighboring person, and with equal cost compared to surveilling the person themselves - is unlikely to hold in the real world. Further, this assumes the whole network has equal costs and benefits of surveillance, which is likewise unlikely (someone who might be the kingpin of a cartel, for example, surely does not have the same costs and benefits as some random drug user).</p>
<p><strong>RumorSleuth: Joint Detection of Rumor Veracity and User Stance (ASONAM long)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3342916" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3342916</a></p>
<p>Setting: detection of rumer veracity and spreaders’ stance towards it.</p>
<p>Main Contributions: algorithm to do these two tasks jointly, using content and user information.</p>
<p>Data: PHEME, Twitter 15 and Twitter16.</p>
<p>Strengths: good experimental results. Joint learning is better than doing each task separately.</p>
<p>Limitations: uses user features but not network ones. May be superseded by VRoC (<a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380054" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380054</a>), which does rumor detection and tracking as well, although performance on PHEME veracity classification looks slightly better here, while worse on stance classification.</p>
<p><strong>Rumor Detection in Social Networks via Deep Contextual Modeling (ASONAM long)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3342896" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3342896</a></p>
<p>Setting: rumor detection including veracity, i.e. detection of “without rumor,” “true rumor,” “false rumor,” or “unrecognizable.”</p>
<p>Main Contributions: algorithm which leverages tweet content in semantic context of an original tweet and all the replies. It does this by embedding tweets individually, then combining all the tweets together with self-attention, along with a latent label loss that pushes the label distributions of replies and the main tweet to be similar.</p>
<p>Data: Twitter 15 and Twitter 16</p>
<p>Strengths: although there are plenty of papers using network-based features, this is the first I’ve seen that uses how the content relates to the network in this way. Experimental results are good. Includes experimental analysis of different embedding methods (often missing in other papers) and all the pieces of the algorithm.</p>
<p>Limitations: some other types of information (diffusion, domain/article features, etc.) are not used. They note GloVe’s optimal performance, among embedding methods, may be because data has similarities to Wikipedia, the corpus on which GloVe is trained, so more varied data would be helpful.</p>
<p><strong>Detection of Topical Influence in Social Networks via Granger-Causal Inference: A Twitter Case Study (ASONAM long)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3341161.3345024" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3341161.3345024</a></p>
<p>Setting: investigates information spread and attitude changes. For example, we can observe someone retweeting a piece of information, but is it because they were influenced by it, or just a passive forwarding?</p>
<p>Main Contributions: analysis/modeling from the point of view of Granger causality.</p>
<p>Data: Twitter API; data from about 360k users over a period of two months.</p>
<p>Strengths: setting is worth further thought. Detailed analysis. Honest about limitations.</p>
<p>Limitations: “While our analysis of a social media dataset finds effects that are consistent with our model of social influence, evidence suggests that these effects can be attributed largely to external confounders.”</p>
<p>Not very useful for our current research; skimmed only.</p>
<p><strong>dEFEND: Explainable Fake News Detection (KDD)</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3292500.3330935" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3292500.3330935</a></p>
<p>Setting: detect fake news articles in an explainable way.</p>
<p>Main Contributions: algorithm combining article and comment content, through attention mechanism that also gives explainability.</p>
<p>Data: FakeNewsNet for veracity experiments. ClaimBuster and Mechanical Turk for evaluating explainability.</p>
<p>Strengths: network design seems effective and experimental results are good. Explainability may be very useful, particularly if fake news needs to be confirmed by human moderator.</p>
<p>Limitations: explainability evaluation limited a bit by lack of good baselines for comparison (new research area).</p>
<p><strong>Ranking in Genealogy: Search Results Fusion at Ancestry</strong></p>
<p><a href="https://dl.acm.org/doi/pdf/10.1145/3292500.3330772" target="_blank">https://dl.acm.org/doi/pdf/10.1145/3292500.3330772</a></p>
<p>Not relevant to our current research; not read.</p>Kellin PelrineThis post is part of a literature review on misinformation, especially in relation to technology and social media. The intention is to organize my thoughts and to provide the reader with condensed information beyond what can be found in the abstract.Web Conference 2020 Misinformation Papers2020-06-03T00:00:00-04:002020-06-03T00:00:00-04:00https://complexdatalabmcgill.github.io/blog/post-kellin-webpapers<p>This post is part of a literature review on misinformation, especially
in relation to technology and social media. The intention is to organize
my thoughts and to provide the reader with condensed information beyond
what can be found in the abstract.</p>
<!--more-->
<p>For each paper I give the setting, main contributions, data used, and
some strengths and limitations. These opinions are my own and are
presented to give the reader some ideas. They do not represent an
ironclad critical review; your judgment may vary, and so might mine on
subsequent readings.</p>
<p>The following are from searching for the keyword <strong>“misinformation,”</strong>
among <strong>2020 WWW</strong> (and companion) papers. Link to search:</p>
<p><a href="https://dl.acm.org/action/doSearch?AllField=misinformation&expand=all&ConceptID=119387&AfterYear=2020&BeforeYear=2020&queryID=26/794627630" target="_blank">
<small>https://dl.acm.org/action/doSearch?AllField=misinformation&expand=all&ConceptID=119387&AfterYear=2020&BeforeYear=2020&queryID=26/794627630</small></a></p>
<p><strong>23 results total.</strong> Results are ordered here according to the search
results order.</p>
<p>Note for members of the Complex Data Lab: I have an internal version
which includes potential uses for each paper in relation to our SN
project specifically. Please check Slack or contact me for this version.</p>
<p><strong>Misinformation Battle Revisited: Counter Strategies from Clinics to
Artificial Intelligence</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366424.3384373" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366424.3384373</a></p>
<p><em>Setting:</em> high level survey of methods to
counter misinformation, including education, legislation, fact-checking,
rumour clinics, public inoculation, filtering, downgrading, and more.</p>
<p><em>Main Contributions:</em> limited as it’s more
a survey paper. Suggests more co-regulation (organization
self-regulation + higher level government regulation) is needed to win
the battle.</p>
<p><em>Data:</em> none.</p>
<p><em>Strengths:</em> general ideas and broad
perspective, including history and many strategies outside ML.</p>
<p><em>Limitations:</em> don’t expect concrete
algorithms or similar. Co-regulation advocacy is not very in-depth,
either in recommendations or supporting analysis.</p>
<p><strong>Mitigating Misinformation in Online Social Network with Top-k
Debunkers and Evolving User Opinions</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366424.3383297" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366424.3383297</a></p>
<p><em>Setting:</em> counter misinformation by
inducing some users (“debunkers”) to spread counter message. In contrast
to previous work, here examines setting when user beliefs (in true or
fake info) can vary, instead of being static once formed.</p>
<p><em>Main Contributions:</em> algorithm to select
best debunkers in above setting.</p>
<p><em>Data:</em> “Facebook” and “Twitter” (exact
sourcing unclear).</p>
<p><em>Strengths:</em> more realistic than static
opinions. Real-world experimental results detecting what opinions will
be formed in a network, on a given topic, look promising.</p>
<p><em>Limitations:</em> debunker compliance unclear
in the real world. Real-world experiments are prediction only, no
interventions tested. Misinformation topics must be specified by user.</p>
<p><strong>Proactive Discovery of Fake News Domains from Real-Time Social Media
Feeds</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366424.3385772" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366424.3385772</a></p>
<p><em>Setting:</em> detect fake news websites. Input
keyword(s) for topic -> extract websites on that topic from tweets
using network connections -> classify fake/real using topic-agnostic
classifier on the webpage structure.</p>
<p><em>Main Contributions:</em> shows that above
content-less approach is effective and provides algorithm to do it.</p>
<p><em>Data:</em> tweets on Donald Trump impeachment,
website HTML from tweet URLs, MediaBiasFactCheck.</p>
<p><em>Strengths:</em> results look promising and
system is close to real-world applicability (pipeline including
dashboard, API in the works).</p>
<p><em>Limitations:</em> requires keyword input.
Experiments limited because only one topic tested.</p>
<p><strong>Factoring Fact-Checks: Structured Information Extraction from
Fact-Checking Articles</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380231" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380231</a></p>
<p><em>Setting:</em> fact-checks lack
standardization; compliance with an existing standard is low and takes
substantial effort.</p>
<p><em>Main Contributions:</em> NLP algorithm for
extracting structured information from unstructured fact-check articles.</p>
<p><em>Data:</em> fact-checks linked from
DataCommons.</p>
<p>Not that useful at current stage of project. Skimmed only.</p>
<p><strong>A Kernel of Truth: Determining Rumor Veracity on Twitter by Diffusion
Pattern Alone</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380180" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380180</a></p>
<p><em>Setting:</em> determine if rumor is true or
false based only on how it spreads (no content, no user metadata, etc.).</p>
<p><em>Main Contributions:</em> Shows diffusion is
predictive and gives an algorithm: Weisfeiler-Lehman graph kernel to
embed cascades, and then a binary classifier on the embeddings.</p>
<p><em>Data:</em> extremely extensive twitter +
fact-check data from
<a href="https://science.sciencemag.org/content/359/6380/1146" target="_blank">https://science.sciencemag.org/content/359/6380/1146</a>.
Note however the relatively easily obtainable version contains diffusion
and veracity information only, no content (and is still not that easy to
obtain). Anything further would require an agreement with Twitter.</p>
<p><em>Strengths:</em> Strong results that diffusion
patterns are predictive. Potentially easy extension out of
diffusion-only setting (node tags with WL).</p>
<p><em>Limitations:</em> Supervised. As authors note,
paper shows importance of diffusion, rather than a solution to
real-world problems, which should use more than diffusion alone.</p>
<p><strong>Unveiling Coordinated Groups Behind White Helmets Disinformation</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366424.3385775" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366424.3385775</a></p>
<p><em>Setting:</em> case study investigating
disinformation about White Helmets (Syrian civil war). Applies two
approaches to detect coordinated groups: rapid retweets and tweet
similarity.</p>
<p><em>Main Contributions:</em> analysis is fairly
exploratory, but identifies some coordinated groups of 4 types: multiple
accounts sharing same content, few accounts with similar tweets, factory
of news websites, and accounts using automation.</p>
<p><em>Data:</em> DARPA SocialSim</p>
<p><em>Strengths:</em> focused case analysis of
real-world misinformation.</p>
<p><em>Limitations:</em> analysis is not very
sophisticated or conclusive.</p>
<p><strong>Facebook Ads Monitor: An Independent Auditing System for Political Ads
on Facebook</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380109" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380109</a></p>
<p><em>Setting:</em> detect which ads are political
using CNN and word embeddings. Real-world application on 2018 Brazilian
election.</p>
<p><em>Main Contributions:</em> Algorithm to
accomplish above. Results suggest Facebook’s political ad identification
system is deficient.</p>
<p><em>Data:</em> Facebook ads, both from Facebook Ad
Library and from their own collecting.</p>
<p><em>Strengths:</em> Important conclusion. Some
successful deployment: created browser extension to collect and classify
ads, presented to Brazilian senate…</p>
<p><em>Limitations:</em> Brazilian election only.
Potentially vulnerable to adversarial evasion.</p>
<p><strong>VRoC: Variational Autoencoder-aided Multi-task Rumor Classifier Based
on Text</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380054" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380054</a></p>
<p><em>Setting:</em> rumor classification problem
defined as detection, tracking, stance classification, and veracity
classification.</p>
<p><em>Main Contributions:</em> algorithm to do all
the above jointly: VAE codes tweet text and the code is used in 4 BiLSTM
classifiers (one per sub-problem), with a “co-train engine” that
combines losses for joint training.</p>
<p><em>Data:</em> combination of PHEME5, PHEME9, and
RumourEval (tweets related to news with rumor, veracity, and stance
labels).</p>
<p><em>Strengths:</em> only “one” (combined) dataset
tested, but results are strong and compare favorably to many other
methods. Does all 4 tasks together while most methods only do 1.</p>
<p><em>Limitations:</em> content only, no network.
Tracker principle - classify into given groups, rather than
related/not-related - could be limiting. Supervised, which is
particularly problematic for tracking and veracity classification,
though not fatal.</p>
<p><strong>FakeFinder: Twitter Fake News Detection on Mobile</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366424.3382706" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366424.3382706</a></p>
<p><em>Setting:</em> Short paper on mobile app for
detecting fake news. Particularly about speed, but also brief discussion
of interface and related.</p>
<p><em>Main Contributions:</em> small model (ALBERT)
running locally achieves better speed and comparable performance to
large model (BERT) on cloud.</p>
<p><em>Data:</em> tweets and comments (collection not
detailed but not really important).</p>
<p><em>Strengths:</em> indeed it does achieve better
speed.</p>
<p><em>Limitations:</em> I’m not sure about their
premise. “We designed and developed a mobile app for real-time fake news
detection from live Twitter streams, which is an effective approach to
contain spread of fake news on Twitter.” This requires people to install
the app, but will they be the people that are susceptible to fake news?
Also, there is extreme overhead if multiple users checking veracity of
the same thing - how would cloud compare if you can calculate results
once and store them?</p>
<p><strong>Social Media Dark Side Content Detection using Transfer Learning
Emphasis on Hate and Conflict</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366424.3382084" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366424.3382084</a></p>
<p><em>Setting:</em> identify malicious social media
content in cases where data is limited, especially different languages.
Broadly includes misinformation but never explicitly focused on it.
Application to Amharic language.</p>
<p><em>Main Contributions:</em> transfer learning NLP
method to handle this tricky and less-studied situation.</p>
<p><em>Data:</em> their own Facebook crawling +
AmharicCorpus (public web crawling data in Amharic).</p>
<p><em>Strengths:</em> seems to work, though a bit
unclear on how the datasets are constructed. Broad applicability.</p>
<p><em>Limitations:</em> no network or comparison
with network-based or other structural methods, which could also work on
different languages.</p>
<p><strong>Analyzing the Use of Audio Messages in WhatsApp Groups</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380070" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380070</a></p>
<p>Analyzes various aspects of this relatively unexplored type of data,
including though not focused on misinformation (see in particular
section 5.4). Concludes audio misinformation messages spread much more
than factual ones. Paper could be very relevant if we wanted to analyze
or use this type of data, but limited relevance to our current work and
data.</p>
<p>Only skimmed selectively.</p>
<p><strong>On Twitter Purge: A Retrospective Analysis of Suspended Users</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366424.3383298" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366424.3383298</a></p>
<p><em>Setting:</em> descriptive analysis of purged
twitter users.</p>
<p><em>Main Contributions:</em> data collection
method: collect tweets over a period of time and then identifying which
ones were purged at the end of the period (because one cannot collect
purged user tweets retroactively). Some insights on this data.</p>
<p><em>Strengths:</em> data collection method enables
insights on a group we don’t know much about.</p>
<p><em>Limitations:</em> insights can be fairly
surface level. Authors suggest sparsity of the data is an issue (could
not collect entire tweet history of purged users, only some).</p>
<p><strong>Characterizing Search-Engine Traffic to Internet Research Agency Web
Properties</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380290" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380290</a></p>
<p><em>Setting:</em> analysis of IRA actions using a
combination of social media (Facebook, Twitter), search (Bing), and
browser (Internet Explorer 11) data.</p>
<p><em>Main Contributions:</em> use of the latter
two. Highlights how a substantial chunk of IRA actions were apolitical
and not misinformation, which contributed substantially to their
engagement.</p>
<p>Search setting is different from the data we are currently working with,
so directly relevant insights are limited.</p>
<p><strong>Towards Detection of Subjective Bias using Contextualized Word
Embeddings</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366424.3382704" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366424.3382704</a></p>
<p><em>Setting:</em> detect content bias on the
sentence level using BERT variants and Wikipedia edit data.</p>
<p><em>Main Contributions:</em> Sentence level is
more interesting than word level like some previous work, though they
state they want to extend to document level in the future.</p>
<p><em>Data:</em> WNC dataset (~400k Wikipedia
sentences, before and after being de-biased)</p>
<p><em>Strengths:</em> outperforms baselines.</p>
<p><em>Limitations:</em> it’s fine for its length,
but that length is very short, so don’t expect a whole lot.</p>
<p><strong>Methods to Evaluate Temporal Cognitive Biases in Machine Learning
Prediction Models</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366424.3383418" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366424.3383418</a></p>
<p>Limited relevance to our research. Investigates how human biases and
decisions change over time and how that impacts modeling. This can
include in response to misinformation, but does not focus on it. Might
be source of ideas/inspiration for investigating effects of
misinformation, or with substantial work might be relevant to detection,
but this fruit is likely in the most remote part of the tree.</p>
<p>Skimmed intro, not read further.</p>
<p><strong>The Chameleon Attack: Manipulating Content Display in Online Social
Media</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380165" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380165</a></p>
<p><em>Setting:</em> examines attack by editing
content, e.g. making a non-controversial post, getting people to
interact in some way with it (e.g. “like”), and then changing the post
to something completely different. Possible, for example, with a
redirect link: changing the redirect changes the link preview on the
social media site.</p>
<p><em>Main Contributions:</em> the problem itself,
and ways to mitigate it.</p>
<p><em>Data:</em> tests the attack in practice on
facebook football fan groups, using it to get a fan pages for rival
clubs into the groups.</p>
<p><em>Strengths:</em> convincing analysis that this
is something to be concerned about.</p>
<p><em>Limitations:</em> more real-world data would
be nice, e.g. on how prevalent this is, though of course collection is
difficult.</p>
<p><strong>An Empirical Study of Android Security Bulletins in Different
Vendors</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380078" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380078</a></p>
<p>Irrelevant to our research. Not read.</p>
<p><strong>Challenges in Forecasting Malicious Events from Incomplete Data</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366424.3385774" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366424.3385774</a></p>
<p>Unlikely to be relevant to our research. Only mention of misinformation:
“Social media platforms are racing to develop tools to detect—and in
some cases anticipate—malicious behaviors in the form of manipulation,
deception, misinformation, and cyberbullying.”</p>
<p>Paper is on: “<em>successful</em> cyber-attacks represent a tiny fraction of
all <em>attempted</em> attacks: the vast majority are stopped, or filtered by
the security appliances deployed at the target. As we show in this
paper, the process of filtering reduces the predictability of
cyber-attacks.”</p>
<p>Skimmed intro, not read further.</p>
<p><strong>Seeding Network Influence in Biased Networks and the Benefits of
Diversity</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380275" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380275</a></p>
<p><em>Setting:</em> Seeding networks, i.e. choosing
nodes in the network to target to start cascades (for example for
advertising, public health, etc.), with the efficiency-maximizing goal
of reaching as much of the network as possible. Strategies designed
purely with efficiency maximization in mind can lead to very unequal
outcomes for different groups in the network. So this paper examines
seeding more diversely.</p>
<p><em>Main Contribution:</em> shows strongly
strongly that seeding for diversity need not sacrifice and can even
improve efficiency.</p>
<p><em>Data:</em> ~53k computer science co-authors
labeled with first-name-perceived gender.</p>
<p><em>Strengths:</em> gives practical insight on
societally important topic.</p>
<p><em>Limitations:</em> assumes optimization takes
place with aggregated metadata, e.g. we know how many connections a
person has but not who they are. However, if this assumption does not
hold and we know who the connections are, then rather than an
optimization based purely on degree, with or without diversity, one can
do a more effective optimization, e.g. greedy. And in this case their
proof no longer applies, the results become uncertain, and there may be
better diversity-preserving algorithms. Therefore, applicability of this
result depends on the situation, i.e. the data and computation
constraints.</p>
<p><strong>Stop tracking me Bro! Differential Tracking of User Demographics on
Hyper-Partisan Websites</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380221" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380221</a></p>
<p><em>Setting:</em> examines differences in tracking
using 9 constructed personas. Finds patterns using co-clustering of
personas and websites. Focus on comparing right vs. left websites.</p>
<p><em>Main Contributions:</em> the persona
methodology. Right-leaning websites track more, and some similar
demographic insights.</p>
<p><em>Data:</em> 556 websites crawled with each
persona, with political leaning labels.</p>
<p><em>Strengths:</em> persona idea is novel and
seems effective.</p>
<p><em>Limitations:</em> 9 personas may be a bit few.
Not sure if disabling all ad and similar blockers is representative user
behavior. Right vs. left analysis is interesting but not sure how
applicable it is. Analysis is descriptive only; more information on how
predictive this information is would be useful.</p>
<p><strong>Identifying Referential Intention with Heterogeneous Contexts</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380175" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380175</a></p>
<p><em>Setting:</em> identify the intention of a
content author towards a reference, between “strong accept,” “accept,”
“background,” and “strong reject.”</p>
<p><em>Main Contributions:</em> algorithm to do above
by combining the referring content, the content before and after the
referral, the referred content, and the network context. First embeds
each separately, applies separate attention, and then combines with
another global attention.</p>
<p><em>Data:</em> academic dataset of papers on two
topics with 1565 citations hand-labeled. News/social media dataset with
401 tweet references to 297 news articles.</p>
<p><em>Strengths:</em> sophisticated model, with
experimental results indicating the combination is effective.</p>
<p><em>Limitations:</em> supervised, and labeling
even by domain experts is “difficult and time-consuming.” If
cross-domain (e.g. different topic than training data) performance is
not good, this would be a substantial problem. Experiments are only
in-domain.</p>
<p><strong>Conquering Cross-source Failure for News Credibility: Learning
Generalizable Representations beyond Content Embedding</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380158" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380158</a></p>
<p><em>Setting:</em> identify fake news through NLP
focused on syntactical content.</p>
<p><em>Main Contributions:</em> sophisticated
algorithm to do above.</p>
<p><em>Data:</em> 3 datasets with news articles and
veracity labels: Fake or Real News, Kaggle-JR, NELA-GT-2018.</p>
<p><em>Strengths:</em> good cross-source (i.e. train
on one news platform, test on another) results.</p>
<p><em>Limitations:</em> its strength can be a
weakness in that there’s a lot of information it doesn’t use, including
non-syntactical content, website structure, network, etc.</p>
<p><strong>OpenCrowd: A Human-AI Collaborative Approach for Finding Social
Influencers via Open-Ended Answers Aggregation</strong></p>
<p><a href="https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380254" target="_blank">https://dl.acm.org/doi/fullHtml/10.1145/3366423.3380254</a></p>
<p><em>Setting:</em> crowd-source detection of
influential/authoritative figures in social media, and learn how to use
the crowd data.</p>
<p><em>Main Contributions:</em> Bayesian model of the
problem and worker data and an EM algorithm to optimize it.</p>
<p><em>Data:</em> asked twitter users to name
influencers, collected potential ones, hand-labeled with expert
guidance, and combined with metadata from Twitter.</p>
<p><em>Strengths:</em> substantially reduces reliance
on experts and expert-labeled data (small number still needed). Good
performance.</p>
<p><em>Limitations:</em> needs crowd, and still some
experts. Network aggregate information only.</p>Kellin PelrineThis post is part of a literature review on misinformation, especially in relation to technology and social media. The intention is to organize my thoughts and to provide the reader with condensed information beyond what can be found in the abstract.