QUESTION #5
How will I evaluate finalists?
A consistent process for comparing MSSPs will help you select the best provider for your organization.
BACKGROUND
Answering the first four questions in this paper will help you narrow down the list of candidates based on their capabilities. But how do you pick one? There are other factors to consider before you make a decision. Listed below are some of the top ones:
Financial fitness
No one wants to invest in a relationship with a company that won’t be around long. MSSPs range from mature companies to startups. Therefore, sound financials are important. Has the MSSP ever made a profit? Can they last three years without securing a new round of funding? If not, you may see the quality of service decrease before the company goes out of business or is acquired.
SOC pedigree
In the trenches, statements like “staffed by former NSA experts” and “99.9% of our SOC staff have never left” mean nothing. What SOC experience does the MSSP have? Can they back up claims and statistics with data? Is their SOC virtual or physical? Can you visit it and get an executive briefing? Where does their threat intelligence come from—can they provide a list of the feeds they use? Will they allow you to look under the hood on a regular basis and evaluate the monitor sets and use cases applied to your environment? Anything too secret to share raises a red flag.
Customer Service
This is another area that varies between MSSPs. What kind of customer service do they provide? Will your organization have a dedicated team or an individual who is responsible for your account? Or, do you just call a toll-free number? How often will you have regular check-ins with your support team? Will their ticket system integrate with yours in a way you approve? Are they willing to provide references of varying service lengths (e.g. new – less than 12 months, mid-term – between 24-36 months, and long-term – greater than 5 years) to better understand the experience?
Reporting
Self-service reporting is an industry standard today. All MSSPs should have a customer-driven reporting engine, and you should get the reports you want, on demand. If the MSSP has to send them to you later, it could be that they are massaging the data before delivering the report.
Service level agreement (SLA)
What level of service will the MSSP commit to? As with any outsourcing arrangement, SLAs need to be specific to ensure you receive the results you expect and service doesn’t degrade over time. Be very careful to understand definitions used in SLA’s and understand what the procedure is to invoke any potential service credits.
Walk-through
Every MSSP should provide a table-top exercise as part of your onboarding experience. That way, you both will know what should happen if a significant security event occurs.
TIP: Go beyond capabilities. Ask awkward questions. It’s the only way to reveal what is behind the curtain. And beware of the “squelch knob.” Some MSSPs will turn up filters to tune out “noise” in an effort to save money and make their performance numbers look better. Once you engage a MSSP, use testing (port scanning, phishing campaigns, penetration tests) to make sure the provider is doing the job you’re paying them for, and they are honoring SLAs.