Day 1 - Plenary
Day 2 - Collaborative
Day 1 - Plenary | |
---|---|
09:00 – 10:00 | Keynote : Vuln4casting Informing your Board Freddy Dezeure (CERT-EU) |
10:00 – 11:00 | US GB Benjamin Edwards (US); Éireann Leverett (Concinnity Risks, GB) |
11:00 – 12:00 | The Impact of Internet-wide Scanning on Known-vulnerability Exploitation Jamie O'Hare |
12:00 – 13:00 | Lunch |
13:00 – 14:00 | An Open-Source System for Customizable and Interpretable Vulnerability Exploitation Prediction Fredrik Sætran, Mark Anderson |
14:00 – 15:00 | US Jay Jacobs (Cyentia, US) |
15:00 – 16:00 | The Uncharted Territory: Vulnerabilities Outside the CVE Ecosystem (Sponsored by Vulners) Andrey Lukashenkov (Vulners) |
15:30 – 16:00 | Coffee Break |
16:00 – 17:00 | US Art Manion (ANALYGENCE Labs, US); Benjamin Edwards (Bitsight, US); Jerry Gamblin (Cisco, US); Patrick Garrity (VulnCheck, US); Tom Bain (VulnCheck) |
18:00 – 19:00 | |
19:00 – 20:00 | Dinner |
Day 2 - Collaborative | |
---|---|
09:00 – 10:00 | NL Renout Schoen (Dutch National Cyber Security Centre, NL); Armin Čoralić (Dutch National Cyber Security Centre , NL) |
10:00 – 11:00 | NO Automated Vulnerability Chaining with CVSS Dr. Martin Eian (mnemonic, NO) |
11:00 – 12:00 | Putting it into PRACTICE: A Proof-of-Concept Prioritization Framework Based on Customization Matt Wixey (Sophos) |
12:00 – 13:00 | Lunch |
13:00 – 14:00 | Forecasting Bang-for-Remediation Matilda Rhode |
14:00 – 15:00 | US Building CVE.ICU: Enhancing Accessibility to Cyber Vulnerability Insights Jerry Gamblin (Cisco, US) |
15:00 – 16:00 | Coffee Break |
16:00 – 16:30 | US Insights from 3,000 Known Exploited Vulnerabilities: What Can We Learn? (Sponsored by VulnCheck) Patrick Garrity (VulnCheck, US) |
Fredrik Sætran, Mark Anderson
Predicting vulnerability exploitation likelihood is crucial for effective risk analysis and patching prioritization. We present our work on developing an open-source system for predicting vulnerability exploitation likelihood, inspired by EPSS. Our approach offers improved interpretability and user control, allowing practitioners to train custom models for specific contexts.
Key focus areas include:
Feature utility: We challenge the trend of using complex and uninterpretable models with unwieldy number of features. Our system enables users to train models using customized feature subsets, while also providing an evaluation of each features utility. This empowers users to make informed decisions when tailoring models to their specific needs. Data completeness: We address the issue of missing data using language models (e.g., CVSSv3 vectors or CWE information) and potential human errors in feature values by exploring methods to predict and incorporate missing information, assessing its impact on model performance.
Model interpretability: We investigate the balance between model interpretability and performance by comparing various architectures and examining how prediction formatting affects outcomes
Temporal relevance: addresses the problem of static models being quickly out-of-date in a rapidly evolving space. This ties into our exploration of lightweight models with refined feature sets, enabling practitioners to update their models more frequently and easily.
We will present an overview and justifications for focusing on these four components alongside preliminary results under various experimental contexts in order to stimulate a discussion and generate potential contributions from the community.
Speaker: Fredrik Sætran. Bio Coming Soon.
Speaker: Mark Anderson. Bio Coming Soon.
October 3, 2024 13:00-14:00
Dr. Martin EianDr. Martin Eian (mnemonic, NO)
What is the CVSS score of an asset? The Common Vulnerability Scoring System (CVSS) can help you understand and score a single vulnerability. Assets often have more than one vulnerability, and the manual approach for vulnerability chaining described in the CVSS User Guide does not scale to the number of vulnerabilities managed by organizations today.
We developed an open source tool for automated vulnerability chaining of CVSS vectors. This tool is available on Github [1]. The tool uses the National Vulnerability Database (NVD) as its data source for CVSS vectors and Common Platform Enumeration (CPE) data. Given a list of CVEs, the tool computes the aggregated CVSS vector: what the adversary needs in order to exploit the vulnerability chain, and what the adversary is able to achieve given the set of vulnerabilities.
As an example, consider an asset with two vulnerabilities. The first vulnerability can be exploited remotely, requires no privileges, and provides local access and low user privileges to a web application. The second vulnerability requires local access and low user privileges to the web application, and provides high user privileges to the operating system. The aggregate of these two vulnerabilities is a vulnerability chain that can be exploited remotely, requires no user privileges, and provides high user privileges to the operating system.
The model used by the tool is based on what a vulnerability requires in order to be exploited, and what the vulnerability provides when exploited. A vulnerability might require user privileges, and local or adjacent system access. These requirements are CVSS base metrics present in the NVD data. What a vulnerability provides, however, is not present in the NVD data, so we deduce what the vulnerability provides from the CVSS Impact and Scope metrics.
Dr. Martin Eian is a Researcher at mnemonic. He has more than 20 years of work experience in IT security, IT operations, and information security research roles. In addition to his position at mnemonic, he is a member of the Europol EC3 Advisory Group on Internet Security."
October 4, 2024 10:00-11:00
Jerry Gamblin (Cisco, US)
In the rapidly evolving cybersecurity landscape, timely access to comprehensive vulnerability statistics is paramount for organizations and individuals. The CVE.ICU project was conceived to significantly improve access to such critical vulnerability data. This talk will delve into the architecture and development of CVE.ICU.
At the heart of CVE.ICU lies the integration with the National Vulnerability Database (NVD), the U.S. government repository of standards-based vulnerability management data. We will explore how CVE.ICU harnesses the rich, structured data from NVD to create a user-friendly interface that simplifies the complexity inherent in vulnerability statistics.
A unique aspect of the CVE.ICU project is the utilization of Jupyter Notebook and JupyterBook technologies. Another key to the project's success is the implementation of automated daily builds using GitHub Actions, a feature that ensures the CVE.ICU platform is constantly refreshed with the latest data.
By attending this talk, participants will gain insights into the development and operation of CVE.ICU, and how it stands as a testament to the transformative power of open-source tools and automation in enhancing the accessibility of cyber vulnerability insights.
Jerry Gamblin is a pretty decent security evangelist and analyst. He has been featured on numerous blogs, podcasts and has spoken at security conferences around the world and has a passion for helping people and companies become more secure. When he’s not helping the world be more secure, you can find him taking his son to swimming lessons or trying to finally learn python.
October 4, 2024 14:00-15:00
Art ManionBenjamin EdwardsPatrick GarrityArt Manion (ANALYGENCE Labs, US), Benjamin Edwards (Bitsight, US), Jerry Gamblin (Cisco, US), Patrick Garrity (VulnCheck, US), Tom Bain (VulnCheck)
Software and device vulnerability data play a critical role in shaping an organization’s threat defenses. Yet complete data sets, competing frameworks and priorities - and lag - prevent organizations from gaining visibility into real threats.
The industry is on pace to add over 30K CVE’s annually, forcing teams to make bets on which threats to prioritize with limited information at hand. This panel of experts will discuss the need for organizations to be transparent through vulnerability and exploitation disclosure, with topics including:
Moderator:
Thomas Bain, Chief Marketing Officer Thomas has held lead Marketing roles at multiple cybersecurity startups including Finite State, Cyware, RiskRecon, (acquired by Mastercard) Morphisec, GoSecure, Q1 Labs (acquired by IBM) and AppSecInc. (acquired by Trustwave) He has experience in building go-to-market strategies focused on growth and solving complex challenges, and he leads the global Marketing and Technology Partnership functions at VulnCheck. https://www.linkedin.com/in/thomasbain/
Panelists:
Art Manion, Analygence
Art Manion is the Deputy Director of ANALYGENCE Labs where he and his team perform in-depth vulnerability analysis and coordinated vulnerability disclosure. Art has lead and contributed to a variety of vulnerability-related efforts in ISO/IEC JTC 1/SC 27, the CVE Program (Board member), the Forum of Incident Response and Security Teams (FIRST), and the (US) National Telecommunications and Information Administration (NTIA). Art works closely with the (US) Cybersecurity and Infrastructure Security Agency (CISA) and previously managed vulnerability analysis at the CERT Coordination Center (CERT/CC).
Benjamin Edwards, Bitsight
Dr. Benjamin Edwards is a principal research scientist working at Bitsight. An expert in ML and statistics, Ben synthesized security data into actionable insights. He has led research on a wide variety of security topics including vulnerability management, application security, human risk, Next-gen SIEM, nation state cybersecurity policy, and the security of ML models. He is an active member of the security community, contributing to open standards efforts including both EPSS and CVSSv4. His work has been published in leading industry and academic venues.
Jerry Gamblin, Cisco
Jerry Gamblin is a pretty decent security evangelist and analyst. He has been featured on numerous blogs, podcasts and has spoken at security conferences around the world and has a passion for helping people and companies become more secure. When he’s not helping the world be more secure, you can find him taking his son to swimming lessons or trying to finally learn python.
Patrick Garrity, Security Researcher, VulnCheck
Patrick Garrity is a security researcher at VulnCheck where he focuses on vulnerabilities, vulnerability exploitation and threat actors. He is a seasoned cybersecurity professional with over 15 years of experience across solutions engineering, product and security research roles helping build and scale security startups including Duo Security, Censys, Blumira, Nucleus Security and VulnCheck.
October 3, 2024 16:00-17:00
Benjamin EdwardsÉireann LeverettBenjamin Edwards (US), Éireann Leverett (Concinnity Risks, GB)
The vulnerability enumeration process through the CVE framework has recently rapidly evolved. In particular, we have moved away from a monolithic system in which MITRE (or a few trusted affiliates) are the sole fountain from which all CVE information flows. Instead we have transitioned into a federated system in which vendors, clearinghouses, and foundations are able to commonly enumerate their own vulnerabilities, and disseminate information about them.
This transition was inevitable as the scope of software has expanded to the point where we are seeing 100s of new vulnerabilities cataloged everyday. This federated process would seem like a godsend were it not for the loose requirements around data quality. In this talk, we’ll examine CVE data quality across CNAs and how it’s evolving over time. In particular, we’ll examine the myriad of potential pieces of information that could be included in CVEs, but rarely are by their reporting CNAs. Required fields like “Description” and “References” are sparse and inconsistent, though nearly always present. However a full 63% of vulnerabilities have no structured information about the affected software. 75% are missing metric information, 80% are missing CWE information, and even fewer bother with lesser known CVEv5 fields like “Credit”, “Solution”, “Workaround”, and “Exploit”. Moreover, even when data is present, it is inconsistently formatted and therefor difficult to extract the relevant information.
After the overview of what’s missing we’ll take a proactive approach and create a measure of CNA performance using Item Response Theory. We’ll explore how this finding breaks down across the most prolific and sparse CNAs. In particular MITRE, the most ancient, venerable, and haggard “CNA of last result” tends to perform the worst. Some vendor and specialty CNAs such as Patchstack, Wordfence, and Solar winds provide particularly rich information, while others such as Apple and Intel only provide the bare minimum. In contrast, some clearinghouse CNAs such as the Zero-Day Initiative and INCIBE (National Spanish CERT) and the Swiss National Cyber Security Center perform quite well.
We’ll conclude the talk by considering how these gaps from in CNAs might affect future efforts at forecast. In particular. highlighting how many of the gaps in the CNA process were filled by NIST’s National Vulnerability Database, which is still catching up after a degradation in capacity early in 2024. We’ll also examine how CISA’s new Vulnrichment program is filling similar gaps as an Authorized Data Provider and including additional information. We’ll also discuss how multiple sources can lead to conflicting information, but also the potential for validation and further CNA performance measurement.
Dr. Benjamin Edwards is a security data scientist working at the Cyentia Institute. An expert in ML and statistics, Ben has led research on a variety of security topics including vulnerability management, application security, human risk, Next-gen SIEM, and security program performance. He is an active member of the security community, contributing to both EPSS and CVSSv4. Prior to joining Cyentia, his research examined global attack trends, the effects of security interventions, nation state cybersecurity policy, and the security of ML models.
Éireann Leverett is the co-author of Solving Cyber Risk, and regularly writes about cyber risk perception, articulation, and quantification. He is a co-chair of the Ransomware SIG, and long time DFIR innovator and data scientist. When he's not working in cyber insurance and risk, he likes writing code, papers, and taking long walks in nature.
While his bio is serious; he hates writing bios in the third person, and once placed second in an Eireann Leverett impersonation contest.
October 3, 2024 10:00-11:00
Renout Schoen, Armin Čoralić
Despite being very valuable, many organizations struggle to adapt external scorings of CVEs (e.g. CVSS, EPSS) to their particular context. While attempting to undertake this, organizations typically spend extensive resources on investigating the same CVE’s. We believe in the potential of combining efforts to collectively learn about CVE impact, and we believe in NCSC-NL's ability to play a central role in this. With a trusted collaboration of organizations we are testing a community-driven machine learning model that scores and prioritizes CVE’s.
In this talk, we would like to discuss the technical components that made our approach possible, the model that was built, and first results and learnings of aggregated community input.
Renout Schoen is a senior data scientist at the Dutch National Cyber Security Centre (NCSC-NL). His areas of expertise are machine learning, natural language processing, and innovation in vulnerability intelligence.
Armin Čoralić is an experienced IT architect and DevOps expert currently leading vulnerability management initiatives at the Dutch National Cyber Security Centre (NCSC-NL). With extensive experience in cloud technologies, DevOps practices and as the author of Container Platform for the Enterprise, Armin has played a pivotal role in cloud and DevOps transformations across diverse sectors. He is also a speaker and trainer on Docker, Kubernetes, and cloud-native technologies, frequently sharing insights on innovative approaches to secure, scalable software delivery.
October 4, 2024 09:00-10:00
Matilda Rhode
Vulnerability prioritisation metrics and frameworks typically operate on a per-vulnerability basis but there are inter-vulnerability dynamics which can be leveraged to predict remediation effort. This talk outlines three metrics based on data science that can help support futher prioritisation.
Some metrics (e.g. CVSSv4) include remediation effort but this is per-individual vulnerability. In practice, there are relationships between vulnerabilities which have implications for the efficiency of remediation efforts. This talk discusses additional metrics to predict bang-for-remediation and is a complement to existing prioritisation approaches.
This talk addresses 3 real-world common challenges: (1) backlogs, (2) context-switching and (3) stakeholder relationships and proposes metrics to avoid. This is not an exhaustive list of under-addressed areas but it is a set of representative scenarios where data analytics can support. The concepts are expanded below, each has been assigned a name to make them easier to remember and distinguish.
Batch patch: there may be a significant backlog of vulnerabilities and patching efficiency can be achieved by prioritising those which are easy to patch (covered by existing metrics) and those which patch multiple vulnerabilities at once, this part of the talk discusses the strengths and weaknesses of other prioritisation methods (e.g. CVSS, EPSS, SSVC, VEST, known-exploited) to dovetail with batch-patch predictions.
Remediation flow: For those vulnerabilities requiring more creative remediation solutions, it attempts leverages the velocity of ""flow-states"" and ""deep-work"" to cluster similar vulnerabilities in order that they can be dealt with using recently-acquired skills, knowledge and social networks.
Stakeholder inroads: vulnerability management entails the management of stakeholder relationships with individuals and teams who have diverse and sometimes competing priorities. The remediation effort in practice may significantly depend on the relationship with a (group of) stakeholder(s). Successful vulnerability remediation in one technology-owner's domain may open an opportunity for remediation in a new domain with another stakeholder. This metrics forecasts the stakeholder groups that can be engaged using related remediation activities and seeks to translate this into a ranking metric for prioritisation.
We hope this talk will be of interest to the Vuln4Cast community and would welcome the opportunity to critically discuss these proposed prediction metrics with others.
Speaker: Matilda Rhode. Bio coming soon.
October 4, 2024 13:00-14:00
Patrick GarrityPatrick Garrity (VulnCheck, US)
VulnCheck gathers vulnerability and exploit intelligence from over 400 global sources, supporting security tools and teams worldwide in triaging and prioritizing vulnerabilities. In early 2024, we released the VulnCheck Known Exploited Vulnerabilities (KEV) catalog, which now features roughly 3,000 CVEs with references, ransomware attribution, and curated exploits, available for free to the community. This talk will explore key observations across known exploited vulnerabilities, offering insights that can help forecast future vulnerability exploitation trends.
Patrick Garrity is a security researcher at VulnCheck where he focuses on vulnerabilities, vulnerability exploitation and threat actors.
He is a seasoned cybersecurity professional with over 15 years of experience across solutions engineering, product and security research roles helping build and scale security startups including Duo Security, Censys, Blumira, Nucleus Security and VulnCheck.
October 4, 2024 16:00-16:30
Freddy DezeureFreddy Dezeure (CERT-EU)
Vuln4casting helps to prioritize our limited resources with a continuously changing threat landscape and a complex infrastructure. We must continuously adapt our defenses in an informed manner, making sure that our mitigating controls are functioning as intended and the residual risk stays within the risk appetite. How confident are we that our efforts to cyber-protect our organization are sufficient? And how can we explain to our leadership that this is indeed the case? This talk will bring Vuln4casting in the perspective of the new NIS2/DORA legislation and its requirements regarding Board oversight.
Freddy Dezeure founded CERT-EU in 2011 and was its Head until May 2017. Since then, he is advising private enterprises and governments in cybersecurity and cyber-risk management, including by providing cyber training to Boards. He is also active as an Advisor to cybersecurity startups worldwide. He is a highly respected keynote speaker and thought leader and is very active in the cybersecurity community. He set up the EU MITRE ATT&CK Community and chairs a CISO Metrics Working Group. https://www.FreddyDezeure.eu/
October 3, 2024 09:00-10:00
Matt Wixey (Sophos)
Users responsible for prioritizing remediation efforts have several tools and frameworks available to them: CVSS, EPSS, SSVC, and more. While these all take different approaches, they have one thing in common: they assess the characteristics of a vulnerability in some way, and produce a score or recommendation, so that users can prioritize accordingly. These tools can all be useful, and have their place in vulnerability management, but because they typically focus on the vulnerability, rather than the user, their outputs can lack context, be misunderstood or misused, and may have important caveats.
In this talk, I’ll explore a proof-of-concept framework I've developed to address these problems, with a working title of PRACTICE (Prioritizing Remediation by Aggregating Customized and Tailored Information - a Conceptual Example). Before I get to PRACTICE, I’ll provide some context by briefly walking through CVSS, EPSS, SSVC, the KEV Catalog, CVEMap, and other tools and frameworks in the context of prioritization – discussing the results they produce, their drawbacks, and what they tell (and don't tell) users.
I’ll then move on to the main part of the talk: PRACTICE itself. PRACTICE is an experimental, transparent methodology for remediation prioritization, which takes a novel approach to the problem by putting the user and their unique environment - rather than vulnerabilities and their characteristics - at the center of the issue. It allows highly granular, customized input (via external configuration files, which can also be customized) about a user’s specific environment, priorities, and circumstances relating to a given vulnerability - including attack vectors; security controls and their coverage and configuration; impact; remediation effort and the potential for disruption; and more. All the inputs in each of these categories can be weighted individually, so that users get an informed understanding of how a vulnerability affects them uniquely – therefore enabling them to prioritize on a completely tailored basis.
As for the output, there are no calculations, no scores, and no recommendations. Instead, PRACTICE produces a visual, colour-coded, at-a-glance summary of a user’s answers, providing a simple graphical decision-making aid for how a vulnerability might affect their organization, based on their security posture and priorities in relation to that specific vulnerability. Unlike most other tools and frameworks, the whole idea is that, by design, two organizations might see completely different outputs for the same vulnerability, and can therefore prioritize accordingly.
After walking through the philosophy and design of PRACTICE, I’ll show some demos of prioritizing the remediation of some real vulnerabilities, from the perspectives of some hypothetical organizations with different security postures and priorities; discuss future research efforts and the PRACTICE to-do list; talk about how the PRACTICE approach could work at-scale; and encourage attendees to get involved with contributing to the development of PRACTICE as I continue to think about and build on this proof-of-concept.
Matt Wixey is a senior threat researcher at Sophos, where he conducts and reports on security and threat intelligence research. He previously led cybersecurity R&D capabilities at a professional services firm and a law enforcement agency, digging into emerging attack vectors, vulnerabilities, and new technologies. Matt has presented original research at multiple conferences, including Black Hat USA, Black Hat Europe, DEF CON, ISF Annual Congress, 44Con, and BruCon.
October 4, 2024 11:00-12:00
Jamie O'Hare
Systems exposed to the Internet are under threat to known-vulnerability exploitation. While informative, vulnerability forecasting does not reflect the associated Internet-traffic. Various solutions aim to quantify and classify Internet traffic towards enabling greater insight into the threats faced. However, the impact of Internet-wide scanning within this topic has received limited attention. This presentation presents a research methodology seeking to quantify the impact of Internet-wide scanning on Internet-exposed systems.
Speaker: Jamie O'Hare: Bio Coming Soon.
October 3, 2024 11:00-12:00
Jay Jacobs (Cyentia, US)
This talk will present the recently published EPSS research report (July 30th, 2024) and discuss the future of EPSS. The research explores the prevalence, spread and speed of exploitation activity with several surprises and subsequent insight. It also presents some metrics for vulnerabilities and measures the historical performance of EPSS. Also, this summer we are updating the backend for EPSS and I may touch on some of the data challenges we are experiencing. Finally, I will end with a view and discussion of the future of EPSS.
Jay Jacobs is the Chief Data Scientist at Cyentia Institute, the lead data scientist for the Exploit Prediction Scoring System (EPSS) and is co-chair of the EPSS special interest group at FIRST.
October 3, 2024 14:00-15:00
Andrey Lukashenkov (Vulners)
The Common Vulnerabilities and Exposures (CVE) system serves as the backbone of vulnerability management by providing standardized identifiers that enable cross-referencing across diverse data sources. Once a vulnerability is cataloged within this framework, it gains widely accepted metrics and becomes discoverable by numerous downstream tools.
But what about the vulnerabilities that don’t make it into this framework?
This talk takes you beyond the CVE ecosystem to explore fringe cases and focus on vulnerabilities not documented in the CVE Project or NVD databases. We’ll delve into why these vulnerabilities might be unlisted, the risks they pose, and how to identify and address them despite the absence of standardized references.
October 3, 2024 15:00-16:00