A DIVISION OF

METHODOLOGY

Home  >>  METHODOLOGY

BLACK BOOK METHODOLOGY

 

QUALITATIVE INSIGHT INTO THE INDUSTRY’S TOP VENDORS THROUGH 18 KEY PERFORMANCE INDICATORS

Black Book collects ballot results on 18 performance areas of operational excellence to rank vendors by software, systems, products, equipment and outsourced service lines. Client users submit ballots per vendor per function which are juxtaposed with client size, functions outsourced, software installed, systems employed, services rendered, industry/vertical(s) affected, country of service origin and overall service and/or product line domain.

The gathered data is subjected immediately to an internal and external audit to verify completeness and accuracy and to make sure the respondent is valid while ensuring the anonymity of the client company is maintained. During the audit, each data set is reviewed by a Black Book executive and at least two other external reviewers. In this way, Black Book report clients are able to clearly see how a vendor is truly performing. The 18 criteria on operational excellence are subdivided by the client’s industry, market size, geography and software category/function outsourced and reported accordingly.

UNDERSTANDING THE STATISTICAL CONFIDENCE OF BLACK BOOK DATA

Statistical confidence for each performance rating is based upon the number of organizations scoring the outsourcing service. Black Book identifies data confidence by one of several means:

  • Top-10-ranked vendors and advisors must have a minimum of five unique clients represented. Broad categories require a minimum of 20 unique client ballots. Data that are asterisked (*) represent a sample size below required limits and are intended to be used for tracking purposes only, not ranking purposes. Performance data for an asterisked vendor’s services can vary widely until a larger sample size is achieved. The margin of error can be very large and the reader is responsible for considering the possible current and future variation (margin of error) in the Black Book performance score reported.
  • Vendors, associates, consultants and advisors with over 20 unique client votes are eligible for top 10 rankings and are assured to have highest confidence and lowest variation. Confidence increases as more organizations report on their outsourcing vendor. Data reported in this form are shown with a 95% confidence level (within a margin of 0.25, 0.20 or 0.15, respectively).
  • Raw numbers include the quantity of completed surveys and the number of unique organizations contributing the data for the survey pool of interest.

BLACK BOOK’S OUTSOURCING SATISFACTION KEY PERFORMANCE INDICATORS

Organizational structure meets the needs of stakeholders or customers and stakeholder satisfaction is the most important priority. Cybersecurity client is likely to recommend the vendor to similar sized industry peer organizations and executives.
Customers are also continuing to push the envelope for further enhancements to which the cybersecurity vendor is responsive. Cybersecurity clients also believe that their vendors’ technology is helping them manage information security and privacy more effectively, Vendor is responsive to make client recommendations with cutting edge improvements.
Cybersecurity solution vendor leadership provides significant and meaningful training opportunities for internal employees and client staff. Leadership strives to develop technology staff, security client service and customer servicing consultant employees in particular. Training modules are effective and practical so that minimal post-implementation training is required on or off site. Regular updates are timely and require minimal additional training to implement.
Cybersecurity solution vendor leadership honors customer relationships highly. The relationship with the vendor elevates the customer reputation. Improving organizational and affiliate efficiency and effectiveness is a priority of the supplier. Governance of engagement is neither complex for buyer nor does it require vendor management attention regularly. There is no regular transparency or quality issue. There are no culture clashes or misfits that threaten relationship’s success or client’s satisfaction.
Trust in enterprise reputation is crucial to cybersecurity clients as well as prospects. Client possesses an understanding that its software and/or services organization has the people, processes, and resources to effectively deliver the desired business and data security results, based on its industry reputation and past performance. There are no disconnects between promises and delivery.
Cybersecurity vendor offers industry recognized horizontal functionality and vertical industry applications, and manage bundled security services and developing new e-Health initiatives. Vendor routinely drives operational performance improvements and results in the areas they affect. Comprehensive offerings are constructed to meet the unique needs of the client’s information security initiatives. Breadth of vendor modules offers comprehensive system services and broad modules.
Cybersecurity solution client deploys at a pace acceptable to the client. Cybersecurity solutions eliminate excessive supervision over vendor implementations. Vendor overcomes client implementation obstacles and challenges effectively. Technical, organizational and cultural implementation obstacles are handled professionally and punctually. Cybersecurity implementation time meets standard expectations. Implementations are efficient and sensitive to users’ specific situations which may cause delays.
Cybersecurity products and process services are customized to meet the unique needs of specific practice client purpose, processes and physician models. Little resistance is encountered when changing performance measurements as clients’ needs vary. Extraordinary efforts are made to adapt and convert client special needs into workable solutions with efficient cost and time considerations. Cybersecurity services and or software allows for modifications that are not costly or complex.
Cybersecurity vendor supports interfaces so information can be shared between necessary applications. Solutions are easily integrated to existing backend systems as needed and risk prevention and detection feasible. Seamless interfaces to legacy applications are performed as required for optimal functioning. Human integration and interface activities are administered precisely. Systems communicate effectively among divisions and affiliates. True seamless cybersecurity with other associated and affiliated firms and organizations is factored into implementation.
Cybersecurity managed services and software solutions vendor provides flexible pricing allowing the client to choose and pay for the precise functionality and services needed. Vendor Invests in significant infrastructure and has the ability to provide services to enterprise organizations. IT products and services meet the changing and varied needs of the customer. Pricing is not rigid or shifting and meets needs of client.
Cybersecurity vendor team of employees is considered top in industry for professionalism and skill. Vendor attracts and retains high performing staff. Vendor is focused on building and developing a strong employee team of producers. Employees act like owners/leaders. Company is moving towards leveraged pay at all levels. Vendor is using effective tools to tie performance metrics to compensation policy and compensating top leaders. Human resources-related criteria are scored from the client perspective on this indicator.
Cybersecurity supplier meets agreed terms as evidenced by routine, acceptable service level reporting and industry expectations. Depth and breadth of applications/solutions are acceptable in meeting client needs. Online reliability is maximized and outages/downtimes are minimized. Solid product and service capacities are demonstrated consistently. Service levels are consistently met as agreed. Services and support response is maximized by vendor team.
Cybersecurity vendor’s marketing and sales statements/pitches are accurately and appropriately represented by actual product and service deliverables. Image is consistent with top cybersecurity firm rankings. Sales presentations and proposals are delivered upon and corporate integrity/honesty in marketing and business development are highly valued. Company image and integrity are values upheld top-down consistently. High level of relevant client communications enhances the vendor-user relationship.
Data security clients’ avoided and realized cost savings are not over-positioned or over/underestimated by vendor in ways that effect major client satisfaction or fees. Vendor offers value-adds as a practice management partner in cost savings and avoidance initiatives and creative programs through bundled cybersecurity product design. Provides true business security transformation opportunities to client organizations and companies swiftly.
Vendor’s viability, employee turnover, financial stability and/or cultural mismatches do not threaten relationship. Senior management and the board exemplify strong leadership principals to steward appropriate resources that impact cybersecurity services or software solutions buyers. Client is confident of long term industry viability for this vendor based on investments, client adoption, exceptional outcomes and service levels. Field management is notably competent, stable and supportive of clients. Vendor demonstrates and provides evidence of competent financial management and leadership.
Particularly across corporate or company divisions, as well as across regions and functional units within the organization. The responsibility of data security is centralized, decreasing the time required to compile and produce aggregated reports, and the work required to interpret reports. Services and or software provides a successful and efficient means of communication that facilitates awareness of organizational cybersecurity concerns as well as data security goals.
Account management provides an adequate amount of onsite administration and support to clients. There exists a formal cybersecurity account management program that meets client needs. Media and clients reference this vendor as a data security services leader and top vendor correctly. Customer services and relationship satisfaction is manifested through significant flagship clients as well as smaller and newest customers similarly. Vendor provides appropriate number of accessible support and customer care personnel.
Cybersecurity management and related technology services are considered best of breed. Vendor technology elevates customers via capabilities, equipment, processes, deliverables, professional staff, leadership, quality assurance and innovative initiatives. Data protection, detection and recovery services are delivered at or above current/former in-house service levels. Technology is current and relevant to exchanging health information among providers, as well as sufficiently offering client employee administrated access.