Metrics SIG

Computer security incident response and incident management has moved towards more mature phases of development. Although there are still new teams forming, many existing teams are focusing on increasing their responsiveness and improving effectiveness.

Like other communities (such as business, finance and government) that look for quantitative and qualitative methods for benchmarking operations and measuring success, there is an emerging need for similar mechanisms in the incident management community.

The scope of this Metrics SIG will be to bring together interested members of the FIRST community to discuss and identify approaches for internally evaluating CSIRT and incident management practices within an organization. The Metrics SIG will work to bring ongoing efforts in developing CSIRT evaluation mechanisms along with defining and measuring CSIRT effectiveness to the attention of the FIRST community, and enabling those that are undertaking the development efforts to receive input from the FIRST community of experts. This will include identifying ongoing efforts and hosting conversations between the developing organization and FIRST Metrics SIG, and coordinating feedback to the developers from the FIRST community. These engagements will include scheduled events and exchanges, or informal email exchanges. There are areas that are beyond the scope of the SIG, namely:

  1. The Metrics SIG is not an accrediting or certifying body
  2. The Metrics SIG will not evaluate other CSIRTs

One ultimate goal of this work is to identify, or where feasible develop, products that any organization with a CSIRT or incident management capability can use to evaluate and assess their capability. This can include not only benchmarking instruments but also sets of criteria for benchmarking particular CSIRT or incident management functions, services, impact or competencies. Statistical methods for analyzing the metrics and identifying trends whether within an organization or across a subset of the community will also be investigated. This may also include exploring the area of taxonomies and ontologies as a way of defining services and measures in a consistent manner across the CSIRT community. Finally methods for training and educating CSIRT members and stakeholders in how to apply or implement such measures will also be within the purview of the Metrics SIG.

Metrics SIG

Expected/Targeted members

  • FIRST members who are seeking approaches for benchmarking and/or improving their CSIRT processes and metrics to provide effective incident management quantification.
  • FIRST members who are interested and willing to help refine, align, and test metrics, as well as to suggest additional improvements for standardizing CSIRT practices within the community.
  • Any CSIRT seeking to improve its incident management capabilities.
  • Other external parties who might provide subject matter expertise in the area of metrics and standards or those who are interesting in building metrics tools and evaluation mechanisms.
  • Other SIGs as appropriate.

Co-chairs

Mike Murray (CERT/CC)
Robin Ruefle (CERT/CC)


All requests to join should be sent to first-sec@first.org