Governance & Risk Management
,
Risk Assessments
Automation, Improved Data Validation Reduce False Positives for Cyber Risk Ratings
Bitsight and SecurityScorecard remained atop Forrester’s cybersecurity risk ratings platforms rankings, while Panorays climbed into the leaders’ space.
See Also: From Epidemic to Opportunity: Defend Against Authorized Transfer Scams
CISOs historically questioned the value of investing in cybersecurity risk ratings platforms due to high false positive rates and a limited return on their investment, according to Forrester Senior Analyst Cody Scott. But vendors have significantly improved their ability to discover, attribute and validate assets and are expanding use cases to include third-party risk management as well as workflow and remediation issues.
“There were so many issues of false positive findings, so much disappointment with these tools when they would go live in customer environments, to the point where you had CISOs saying, ‘It’s not worth the investment,'” Scott told Information Security Media Group. “Vendors have really stepped up to take accountability for that problem and to invest more technically in their platform.”
Scott said risk ratings platforms invested heavily in data sets, AI and probabilistic methods to enhance their accuracy and transparency, and they capitalized on advances in open-source data and commercial data availability to improve their data validation processes. Vendors also built native tools to provide more data access and integration to fulfill customer demand for orchestration and remediation capabilities.
“Advancement in the technology and processing side, and in the availability of commercial data, has made it possible to be more accurate,” Scott said.
The new risk ratings Forrester Wave replaced one from winter 2021. SecurityScorecard is holding steady atop Forrester’s evaluation for the strength of its current offering, albeit by a much smaller margin. Bitsight once again holds second place, and Black Kite has jumped ahead of Panorays for third place.
“Vendors have really stepped up to take accountability for that problem.”
– Cody Scott, senior analyst, Forrester
BitSight bested SecurityScorecard this year for the top strategy score, while in 2021, the two companies tied for the top score. In the current evaluation, SecurityScorecard, Recorded Future and Panorays scored second, third and fourth in strategy, respectively, while in 2021, Panorays narrowly edged out RiskRecon for the third-highest strategy score.
What Sets Risk Ratings Platform Leaders Apart
Leaders in the cybersecurity risk ratings market have distinguished themselves by investing heavily in automation and expanding their services beyond just ratings to include broader risk management functions, Scott said. They’ve also embedded remediation features within their platforms so that users can act on information directly and prioritize actions based on analytics and predictive measures.
“What these platforms have really done is: They’re leaning into saying, ‘I hear you loud and clear; you want to take action on this,'” Scott said. “So we’re going to automatically pull in recommended remediation steps that you should take that are context-specific to the asset and the vulnerability that’s identified.”
In the future, Scott expects cybersecurity risk ratings to converge with third-party risk management, external attack surface management and cyber risk quantification, creating uncertainty around whether stand-alone risk ratings tools will exist in a couple of years. As this happens, Scott expects vendors will focus more on overall risk posture or exposure management rather than stand-alone numerical ratings (see: RiskLens, Axio Lead Cyber Risk Quantification Forrester Wave).
“I am not convinced that the cybersecurity risk ratings market will be a stand-alone market,” Scott said. “Ratings will always exist, but they may be baked into other solutions.”
Outside of the leaders, here’s how Forrester sees the cybersecurity risk ratings platforms market:
- Strong Performers: Black Kite, RiskRecon, BlueVoyant, Recorded Future
- Contenders: UpGuard, Prevalent
- Challengers: ISS Corporate Solutions
Bitsight Bolsters Cyber Risk Ratings Governance, Technology
Bitsight has strengthened its governance by creating a transparent public dispute resolution process to accept feedback from customers on their ratings, and any changes in ratings is published publicly online, according to Chief Risk Officer Derek Vadala. The company also put together an external advisory board featuring an ex-chief security officer and a U.S. congressman to ensure the ratings serve a broad audience.
On the technical side, significant work has been done around discovery and attribution. A team trained Bitsight’s machine learning model to better understand asset ownership and security responsibilities, Vadala said. The company also has developed a scanning apparatus to thoroughly analyze assets and provide deeper insights into vulnerabilities, which Vadala said is useful during major security incidents (see: Bitsight CEO on Going From Security Ratings to Managing Risk).
“We have the longest time in the category as the category creator,” Vadala told ISMG. “I think that time spent on this problem is a big differentiator. It’s allowed us to inform and develop the technologies that we offer.”
Forrester criticized Bitsight for complex pricing, lacking peer benchmarks and fourth-party data functionality in its base offering, and instances where data shared through GRC integrations didn’t match Bitsight’s platforms. Vadala said Bitsight’s pricing is competitive given the comprehensiveness and modularity of the solution, and it will use feedback to improve product integrations and service offerings.
“Part of the reason for our price points is the comprehensiveness of data and the completeness of the solution,” Vadala said. “We have a modular solution, which we recognize, but it is competitively priced.”
Major Investments in R&D and AI Transform SecurityScorecard
SecurityScorecard has significantly invested in in-house research and development, focusing on threat intelligence, data collection and insights on issues such as ransomware and zero-day vulnerabilities, said co-founder Sam Kassoumeh. Bringing data collection in-house has greatly improved the accuracy of SecurityScorecard’s risk ratings, allowing for faster updates and more reliable feedback for customers.
Kassoumeh said SecurityScorecard capitalized on its extensive data collection to launch a zero-day-as-a-service capability, which proactively discovers and remediates zero-day vulnerabilities. The company also invested heavily in integrating AI with cyber intelligence data to help auto populate questionnaires, analyze flat files and accelerate customer decision-making processes, according to Kassoumeh (see: Why the Physical Russia-Ukraine War Might Become a Cyberwar).
“We control 100% of the data we collect; there are no constraints on how it’s used,” Kassoumeh told ISMG. “So how does that translate? That translates into: I can notify customers the fastest of concerns in their supply chain, and that enables them to close the window of opportunity for hackers the fastest.”
Forrester chided SecurityScorecard for lacking AI-parsing tools to assess uploaded evidence documents and challenges with preventing duplicate findings when a scanned IP and hostname report the same asset. Kassoumeh said SecurityScorecard wants to handle large volumes of policies and procedures more effectively, while its control over data collection ensures accurate and nonredundant reporting.
“A big opportunity for AI is not just reading a SOC2 but being able to read any policy procedure and be able to mine it,” Kassoumeh said. “That’s where we’re doubling down our investment.”
Panorays Elevates Cyber Risk Ratings With Proprietary AI
Panorays has grown from external attack surface assessments to include internal security questionnaires and collaboration with third parties to enhance the overall risk rating process, according to co-founder and CTO Demi Ben-Ari. The company’s risk rating process evaluates the network layer, the application layer and the human element to assess an organization’s infrastructure, cloud and information assets.
The company has integrated a suite of AI capabilities across its platform based on unique information collected over the past decade, which Ben-Ari said ensures precise relevance to third-party cyber risk. Panorays’ AI capabilities cut noise and false positives in data management, streamline the identification and classification of assets, and automate the internal collection of security evidence, Ben-Ari said (see: Tips for Implementing a Good Third-Party Risk Program).
“We are the friendliest one,” Ben-Ari told ISMG. “The collaborative nature with our technology gives access to all companies to free them to collaborate. Also, improving security posture rather than only providing ratings and scorecards is one of the key differentiators in creating the most precise picture.”
Forrester criticized Panorays for lacking native risk quantification, having less dynamic reporting capabilities, not publishing its ratings performance metrics, and having a less differentiated strategic focus. Ben-Ari said Panorays plans to enhance its transparency by making performance metrics publicly available and expanding reporting tools through integrations with third parties such as Snowflake, he said.
“Transparency exists fully in the platform itself,” Ben-Ari said. “We’re not holding any information as ransom, but they wanted to see it reflected to the outside, meaning somebody that is not registered to the platform itself. And we’ll continue the discussion about that.”