The Office of the Under Secretary of Defense (Intelligence) (OUSD-I) sought to develop a system architecture and working prototype to access, correlate, and visualize a significant number of DoD-wide Cyber related data sources. The intent was to provide DoD level decision makers with the information required for resource planning and projections to support Cyber priorities. Prior to Titania’s efforts, there was no realized concept nor a predecessor system from which to design and develop either a prototype or working cyber resource planning tool. Our task was to analyze current Cyber resource requirements; catalog DoD cyber data sources; design a concept and prototype for data extraction, transformation, and loading (ETL); and develop the business logic and visualization layers that would support DoD decision makers.
Titania called upon its two most experienced and technical system/software engineers, including one prior intelligence officer with experience on the Army Intelligence Master Plan (AIMP), to develop a Proof of Concept Cyber Resource Planning System – the CMP. Titania conducted the effort in two phases.
The first phase was to develop concept architecture for Cyber Resource Planning using open source “Big Data” and Visualization components to amalgamate over 50 disparate DoD and Service Specific data repositories. Project planning and system development of the proposed architecture would follow. Titania used Agile development methodology (i.e. Scrum) and CMMI compliant processes from project initiation to implementation. Initial activities included customer (e.g. DoD OUSD-I) requirements elicitation and analysis to support development.
The second phase consisted of the system development, software architecture design, and development in support of OUSD-I goals to achieve a fully integrated, business logic-based, visualization tool for Cyber resource management. This phase included:
- Disparate data source integration, ETL, and data warehousing using the Talend open source big data integration tools
- Software development of the business logic layer supporting complex queries and reporting specific to the needs of the OUSD (I); but flexible and scalable to various needs
- Software development of data and result visualization within DoD’s A-Space Framework (Intelligence Community Collaboration Environment) based upon JIVE ® Social Networking Framework
Titania’s approach provided the OUSD (I) with not only an initial, robust data collection capability, but also an extensible solution that can be rapidly updated for data collection and integration needs of any organization, Cyber or non-Cyber.