The European Symposium on Societal Challenges in CSS in Zurich will host a one-day datathon on September 2nd, 2019 on the topic of Machine Behavior. The aim of the datathon is to advance our understanding of machine behavior, quantitatively analyzing the behavior of intelligent machines in their natural environments when interacting with humans.
Participants will have access to a series of datasets but can also bring their own data on related topics to machine behavior and human-machine interaction. The datathon will start with some introductory presentations and will have leading scientists in the field as advisors to help in the projects. At the end of the day, each participant will present their work and the best project will earn a Euro CSS Dataset Challenge Award (250 EUR) that will be announced on the last day of the Symposium (Sept 4th).
The deadline for registration to the Machine Behavior Datathon is August 17th, 2019. Please register for the symposium via Eventbrite and make sure to select both the morning and afternoon Dataset Challenge sessions “[Full Day Event – Part I] Dataset Challenge” as well as “[Full Day Event – Part II] Dataset Challenge” in the workshop/tutorial selection menu.
Program – August 17th, 2019
- 9:00 – Introduction to datasets and topics. Making groups
- 9:45 – 12:30 Datathon part 1
- 12:30 – 14:00 Lunch Break
- 14:00 – 16:30 Datathon part 2
-
16:30 – 17:30 Project presentations in front of the jury
The Datasets
Machines in gaming
- Matchmaking in DOTA2: https://www.kaggle.com/devinanzelmo/dota-2-matches (https://www.opendota.com/)
- Matchmaking in LoL: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/B0GRWX (Sapienza, A., Zeng, Y., Bessi, A., Lerman, K., & Ferrara, E. (2018). Individual performance in team-based online games. Royal Society Open Science, 5(6), 180329.)
- Bots in Twitch plays Pokemon: https://archive.org/details/tpp_logs (See https://kotaku.com/not-everyone-playing-twitch-plays-pokemon-appears-to-1530921548 for an account of bot traces in TPP)
Questions you could assess with gaming datasets: Do matchmaking algorithms create collective effects in online games? Can we detect the aggregated behavior of bots in a Twitch-based videogame? How does the behavior of players depend on the decisions of bots and algorithms in gaming?
Machines that post on social media
- Social bots in Weibo: https://www.kaggle.com/bitandatom/social-network-fake-account-dataset/home (Liu, L., Lu, Y., Luo, Y., Zhang, R., Itti, L., & Lu, J. (2016). Detecting” Smart” Spammers On Social Network: A Topic Model Approach. NAACL HLT.)
- Twitter honeypot bots: http://infolab.tamu.edu/data/social_honeypot_icwsm_2011.zip (Lee, K., Eoff, B. D., & Caverlee, J. (2011). Seven Months with the Devils: A Long-Term Study of Content Polluters on Twitter. ICWSM.)
- Misinformation bots in Twitter: https://zenodo.org/record/1402267#.W-HLLMtKhhE
(Shao, C., Ciampaglia, G. L., Varol, O., Yang, K. C., Flammini, A., & Menczer, F. (2018). The spread of low-credibility content by social bots. Nature communications, 9(1), 4787.) - Paragon Twitter Bot/Cyborg topic datasets: https://data.world/drstevekramer/social-media-bot-detection-by-paragon-science (Kramer, S. (2017) Identifying viral bots and cyborgs in social media. https://www.oreilly.com/ideas/identifying-viral-bots-and-cyborgs-in-social-media)
Questions you could assess with social media datasets: Do social bots influence user behavior, such as posting, liking, and the expression of opinions or emotions? How can we measure the way social bots react to human behavior, follow trends, or promote content? Is there a consistent way to formalize the spectrum between human and bot behavior through cyborg or semi-automated accounts?
Machines that edit content in Wikipedia
- Even good bots fight: https://figshare.com/articles/Even_Good_Bots_Fight_The_Case_of_Wikipedia/4597918
(Tsvetkova, M., García-Gavilanes, R., Floridi, L., & Yasseri, T. (2017). Even good bots fight: The case of Wikipedia. PloS one, 12(2), e0171774.) - Wikimedia bot data: https://figshare.com/articles/CSCW_data_and_code/5362216 (Geiger, R. S., & Halfaker, A. (2017). Operationalizing Conflict and Cooperation between Automated Software Agents in Wikipedia: A Replication and Expansion of “Even Good Bots Fight”. CSCW)
Questions you could assess with social Wikipedia datasets: Does the presence of automated accounts affect the editing behavior of Wikipedia contributors? Do bots in collaborative projects accelerate or hinder content production? What are the observable traces of bot presence in the content?
Participant datasets
Participants can also add or contribute any dataset on topics related to the challenge, for example:
- Bot-human interaction on social media
- Collaborative content editing between human and machine users
- Algorithmic effects on opinions, emotional expression, and collective behavior
- Learned or emergent biases and discrimination in machine behavior