The recent GFIRST Conference – a forum for incident response and security teams – covered the gamut of security topics with a surprisingly colorful and entertaining array of session titles. While my session title, “Continuous Monitoring 2.0” , may have lacked the bedazzle factor of “Hack the database…and other cocktail party tricks”, “Bad Karma Chameleon”, “Welcome to McSecurity, would you like fries with your scan?”, it did capture audience sentiment about the government’s CyberScope initiative and the push for continuous network monitoring.

So what’s causing the angst among federal IT security managers about CyberScope’s current state of play?

The best way to explain the general malaise might be to say, “If it looks like, smells like and sounds like a FISMA paperwork exercise, it just may end up being an automated version of the same thing – better compliance but no better security.”

In a recent survey, federal IT professionals were asked specifically if CyberScope is helping to ease the burden of FISMA on government agencies, and an overwhelming majority (83%) said “no.”

In fact, I am sensing, and not just from those who attended my GFIRST session, but from others in government, that the introduction of mandatory monthly submissions to OMB through CyberScope may have actually set the government back in terms of utilizing continuous monitoring to reduce organizational risk among agencies.

The OMB has been driving awareness of continuous monitoring for several years, a push considered by most IT security experts to be very positive. However, the purpose of continuous monitoring is to operate as a closed loop system, to continuously assess the security status of all the assets of an agency’s network in order to identify and prioritize problems and to act (my emphasis) on the information provided.

As it stands though, the results of the drive for continuous monitoring are beginning to look eerily similar to a FISMA paper exercise. The recently imposed requirement to submit monthly security scan data to OMB – and getting publicly graded on whether that submission is taking place – has directed the agencies’ focus away from that beneficial closed loop process, and has left some agencies focusing only on the first part of the loop, namely identifying and reporting problems.

Why is this so troubling from a security perspective?

Focusing exclusively on the identification and reporting of problems leads agencies to make different technology choices than if they expect to use continuous monitoring to actually understand and harden their environments. Departments are making lowest common denominator decisions regarding enterprise-wide security solutions in order to get the whole department quickly compliant with monthly CyberScope submission requirements.

I’ve actually seen mature, sophisticated risk remediation programs at the agency level torn out and replaced with a pure compliance solution, so that the department can check “yes” on a single department-wide solution for CyberScope reporting. This practice actually drives agency security programs backwards and works against OMB/DHS’ intended goal of the CyberScope process which is ultimately to prioritize risk and remediate.

Given this ugly reality, I made several suggestions to the audience that might just help move the practice of monthly submissions back in the right direction:

First, agencies need to look beyond this year’s FISMA report and remember that this whole effort – and future FISMA reporting – is designed to support more effective remediation.

They should identify and replicate successful risk monitoring programs in both the federal government and in the commercial world. One that immediately comes to mind is the security program at The Centers for Medicare & Medicaid Services (CMS) that was just named a recipient of the 2012 SANS National Cybersecurity Innovation Awards for the design and implementation of a process that has effectively monitored, reported and reduced risk across 200 widely disbursed CMS networks and 38 contractor sites. If you want to see continuous monitoring, reporting and risk remediation done right, hands down, CMS has figured it out.

The key to CMS’ success is that they measure the results of their monitoring efforts rather than measuring their ability to submit the results.

Some might argue that organizations like CMS are able to implement this type of program only because they had the following three elements in place: a large number of roughly equivalent entities to be compared; breadth of engagement (both business and IT managers committed to the process); findings that could be simplified and presented in the format of a benchmark community.

To a point, this argument is valid. However, organizations that do not fit this profile can find a similar measure of success by using a game-changing practice that has taken off in the commercial world. Over 1,000 organizations have joined cloud-based security benchmarking communities that allow them to compare all the various aspects of security performance across their organizations. This practice in the commercial space is currently being explored for mapping in the federal community.

At my GFIRST session, we talked about what such a benchmarking community might look like in the federal government; and how it could effectively leverage security content automation initiatives like CyberScope and SCAP. During the session and from numerous other conversations throughout the conference, there was great interest in using benchmarking to generate additional value from CyberScope investments – bringing “sexy back” in the form of actionable reports and scorecards from CyberScope data.

In the meantime, DHS’ John Streufert should be commended for bringing an effective continuous monitoring vision to the federal government through his new role as Director of the National Cybersecurity Division. His effort to recast the strategic direction of continuous monitoring as “Continuous Diagnostics and Mitigation” properly refocuses the community on the action end of the closed loop. Interestingly, his comments during my session suggested that agencies should expect CyberScope to change substantially in the future.

As long as departments are getting praised in the annual FISMA report to Congress for just generating a data stream, while other agencies with demonstrably best practice risk mitigation programs get ‘dinged’, CyberScope’s intended goals will remain unfulfilled, and agency security progress will revert backward.

In the meantime? I suppose agencies can always consider how to order fries with their scan, avoid bad Karma Chameleon and retaliate against the hacker cocktail party-goers.

Keren Cummins is director of Federal Markets, nCircle.