Nicolás E. Díaz Ferreyra

selected publications

1. MSR ’23

   LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluations

   Tony, Catherine, Mutas, Markus,  Díaz Ferreyra, Nicolás E. and 1 more author

   In Proceedings of the 20th International Conference on Mining Software Repositories (MSR ’23) 2023

   Abs arXiv Bib

   Large Language Models (LLMs) like Codex are powerful tools for performing code completion and code generation tasks as they are trained on billions of lines of code from publicly available sources. Moreover, these models are capable of generating code snippets from Natural Language (NL) descriptions by learning languages and programming practices from public GitHub repositories. Although LLMs promise an effortless NL-driven deployment of software applications, the security of the code they generate has not been extensively investigated nor documented. In this work, we present LLMSecEval, a dataset containing 150 NL prompts that can be leveraged for assessing the security performance of such models. Such prompts are NL descriptions of code snippets prone to various security vulnerabilities listed in MITRE’s Top 25 Common Weakness Enumeration (CWE) ranking. Each prompt in our dataset comes with a secure implementation example to facilitate comparative evaluations against code produced by LLMs. As a practical application, we show how LLMSecEval can be used for evaluating the security of snippets automatically generated from NL descriptions.

   @inproceedings{tony2023msrtool,
     author = {Tony, Catherine and Mutas, Markus and D\'{i}az Ferreyra, Nicol\'{a}s E. and Scandariato, Riccardo},
     year = {2023},
     title = {LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluations},
     booktitle = {Proceedings of the 20th International Conference on Mining Software Repositories (MSR '23)},
   }

2. CHI ’23

   Regret, Delete, (Do Not) Repeat: An Analysis of Self-Cleaning Practices on Twitter After the Outbreak of the COVID-19 Pandemic

   Díaz Ferreyra, Nicolás E., Shahi, Gautam Kishore,  Tony, Catherine and 2 more authors

   In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA ’23) 2023

   Abs arXiv Bib

   During the outbreak of the COVID-19 pandemic, many people shared their symptoms across Online Social Networks (OSNs) like Twitter, hoping for others’ advice or moral support. Prior studies have shown that those who disclose health-related information across OSNs often tend to regret it and delete their publications afterwards. Hence, deleted posts containing sensitive data can be seen as manifestations of online regrets. In this work, we present an analysis of deleted content on Twitter during the outbreak of the COVID-19 pandemic. For this, we collected more than 3.67 million tweets describing COVID-19 symptoms (e.g., fever, cough, and fatigue) posted between January and April 2020. We observed that around 24% of the tweets containing personal pronouns were deleted either by their authors or by the platform after one year. As a practical application of the resulting dataset, we explored its suitability for the automatic classification of regrettable content on Twitter.

   @inproceedings{diaz2023chilbw,
     author = {D\'{i}az Ferreyra, Nicol\'{a}s E. and Shahi, Gautam Kishore and Tony, Catherine and Stieglitz, Stefan and Scandariato, Riccardo},
     year = {2023},
     title = {Regret, Delete, (Do Not) Repeat: An Analysis of Self-Cleaning Practices on Twitter After the Outbreak of the COVID-19 Pandemic},
     booktitle = {Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA '23)},
   }

3. CHASE ’23

   Developers Need Protection, Too: Perspectives and Research Challenges for Privacy in Social Coding Platforms

   Díaz Ferreyra, Nicolás E., Imine, Abdessamad,  Vidoni, Melina and 1 more author

   In 16th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE 2023) 2023

   Abs arXiv Bib

   Social Coding Platforms (SCPs) like GitHub have become central to modern software engineering thanks to their collaborative and version-control features. Like in mainstream Online Social Networks (OSNs) such as Facebook, users of SCPs are subjected to privacy attacks and threats given the high amounts of personal and project-related data available in their profiles and software repositories. However, unlike in OSNs, the privacy concerns and practices of SCP users have not been extensively explored nor documented in the current literature. In this work, we present the preliminary results of an online survey (N=105) addressing developers’ concerns and perceptions about privacy threats steaming from SCPs. Our results suggest that, although users express concern about social and organisational privacy threats, they often feel safe sharing personal and project-related information on these platforms. Moreover, attacks targeting the inference of sensitive attributes are considered more likely than those seeking to re-identify source-code contributors. Based on these findings, we propose a set of recommendations for future investigations addressing privacy and identity management in SCPs.

   @inproceedings{diaz2023chase,
     author = {D\'{i}az Ferreyra, Nicol\'{a}s E. and Imine, Abdessamad and Vidoni, Melina and Scandariato, Riccardo},
     year = {2023},
     title = {Developers Need Protection, Too: Perspectives and Research Challenges for Privacy in Social Coding Platforms},
     booktitle = {16th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE 2023)},
   }

4. QRS ’22

   GitHub Considered Harmful? Analyzing Open-Source Projects for the Automatic Generation of Cryptographic API Call Sequences

   Tony, Catherine, Díaz Ferreyra, Nicolás,  and Scandariato, Riccardo

   In 2022 IEEE 22nd International Conference on Software Quality, Reliability and Security Companion (QRS-C) 2022

   arXiv Bib

   @inproceedings{tony2022github,
     author = {Tony, Catherine and D\'{i}az Ferreyra, Nicol\'{a}s and Scandariato, Riccardo},
     booktitle = {2022 IEEE 22nd International Conference on Software Quality, Reliability and Security Companion (QRS-C)},
     title = {GitHub Considered Harmful? Analyzing Open-Source Projects for the Automatic Generation of Cryptographic API Call Sequences},
     year = {2022},
     volume = {},
     number = {},
     pages = {},
     doi = {},
   }

5. EuroUSEC ’22

   ENAGRAM: An App to Evaluate Preventative Nudges for Instagram

   Díaz Ferreyra, Nicolás E., Ostendorf, Sina,  Aïmeur, Esma and 2 more authors

   In 2022 European Symposium on Usable Security (EuroUSEC 2022) 2022

   Abs arXiv Bib

   Online self-disclosure is perhaps one of the last decade’s most studied communication processes, thanks to the introduction of Online Social Networks (OSNs) like Facebook. Self-disclosure research has contributed significantly to the design of preventative nudges seeking to support and guide users when revealing private information in OSNs. Still, assessing the effectiveness of these solutions is often challenging since changing or modifying the choice architecture of OSN platforms is practically unfeasible. In turn, the effectiveness of numerous nudging designs is supported primarily by self-reported data instead of actual behavioral information. OBJECTIVE: This work presents ENAGRAM, an app for evaluating preventative nudges, and reports the first results of an empirical study conducted with it. Such a study aims to showcase how the app (and the data collected with it) can be leveraged to assess the effectiveness of a particular nudging approach. METHOD: We used ENAGRAM as a vehicle to test a risk-based strategy for nudging the self-disclosure decisions of Instagram users. For this, we created two variations of the same nudge (i.e., with and without risk information) and tested it in a between-subjects experimental setting. Study participants (N=22) were recruited via Prolific and asked to use the app regularly for 7 days. An online survey was distributed at the end of the experiment to measure some privacy-related constructs. RESULTS: From the data collected with ENAGRAM, we observed lower (though non-significant) self-disclosure levels when applying risk-based interventions. The constructs measured with the survey were not significant either, except for participants’ External Information Privacy Concerns (EIPC). IMPLICATIONS: Our results suggest that (i) ENAGRAM is a suitable alternative for conducting longitudinal experiments in a privacy-friendly way, and (ii) it provides a flexible framework for the evaluation of a broad spectrum of nudging solutions.

   @inproceedings{diaz2022eusec,
     author = {D\'{i}az Ferreyra, Nicol\'{a}s E. and Ostendorf, Sina and A\"{i}meur, Esma and Heisel, Maritta and Brand, Matthias},
     year = {2022},
     doi = {https://doi.org/10.1145/3549015.3555674},
     title = {ENAGRAM: An App to Evaluate Preventative Nudges for Instagram},
     booktitle = {2022 European Symposium on Usable Security (EuroUSEC 2022)},
   }

6. ARES ’22

   SoK: Security of Microservice Applications: A Practitioners’ Perspective on Challenges and Best Practices

   Billawa, Priyanka, Bambhore Tukaram, Anusha,  Díaz Ferreyra, Nicolás E. and 3 more authors

   In International Conference on Availability, Reliability and Security (ARES) 2022
7. MSR ’22

   Vul4J: A Dataset of Reproducible Java Vulnerabilities Geared Towards the Study of Program Repair Techniques

   Bui, Quang-Cuong, Scandariato, Riccardo,  and Díaz Ferreyra, Nicolás E.

   In International Conference on Mining Software Repositories (MSR) 2022
8. EASE ’22

   Conversational DevBots for Secure Programming: An Empirical Study on SKF Chatbot

   Tony, Catherine, Balasubramanian, Mohana,  Díaz Ferreyra, Nicolás E. and 1 more author

   In Evaluation and Assessment in Software Engineering (EASE) 2022

   Abs Bib HTML

   Conversational agents or chatbots are widely investigated and used across different fields including healthcare, education, and marketing. Still, the development of chatbots for assisting secure coding practices is in its infancy. In this paper, we present the results of an empirical study on SKF chatbot, a software-development bot (DevBot) designed to answer queries about software security. To the best of our knowledge, SKF chatbot is one of the very few of its kind, thus a representative instance of conversational DevBots aiding secure software development. In this study, we collect and analyse empirical evidence on the effectiveness of SKF chatbot, while assessing the needs and expectations of its users (i.e., software developers). Furthermore, we explore the factors that may hinder the elaboration of more sophisticated conversational security DevBots and identify features for improving the efficiency of state-of-the-art solutions. All in all, our findings provide valuable insights pointing towards the design of more context-aware and personalized conversational DevBots for security engineering.

   @inproceedings{tony2022ease,
     title = {{Conversational DevBots for Secure Programming: An Empirical Study on SKF Chatbot}},
     booktitle = {Evaluation and Assessment in Software Engineering (EASE)},
     publisher = {ACM},
     year = {2022},
     doi = {https://doi.org/10.1145/3530019.3535307},
     author = {Tony, Catherine and Balasubramanian, Mohana and D\'{i}az Ferreyra, Nicol\'{a}s E. and Scandariato, Riccardo},
   }