Background:

NESDIS was actively planning to transform from a system-centric organization to a product and service-centric organization by migrating toward a cloud-enabled NESDIS Ground Enterprise (NGE). The NGE includes all IT systems used for data ingest, processing, storage, and dissemination across all NESDIS Offices and Programs.  NESDIS was seeking specialized Subject Matter Experts (SME) to advise, assess, evaluate, and recommend innovative technologies, solutions, and practices for the proposed transformation. These SMEs are highly experienced and knowledgeable about cloud-native services provided by Cloud Service Providers (CSPs) such as Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure, as well as NOAA/NESDIS services and systems to provide critical technical feedback and recommendations. Based on NESDIS’ pilot program, GAMA-1 understood that to provide secure and timely access to global environmental data and information from satellites and other sources to anyone in any place (authorized users) while protecting the Nation’s security, environment, economy, and quality of life, NESDIS needed to transform from the system-specific, stovepipe architecture into a service-centric enterprise. We acknowledged that changing the culture is NESDIS’ primary challenge during its transformation efforts.

Solution & Results:

GAMA-1 has been instrumental in providing technical expertise for the Subject Matter Expertise for Cloud Development and Innovative Initiatives (SCDII) contract with a focus on cloud technology. The main endeavor under this contract was supporting the NESDIS Consolidated Cloud Framework (NCCF), a robust, data-agnostic, microservice-based framework designed to meet a variety of scientific processing and product generation requirements.

• NCCF Infrastructure Upgrade: The NCCF was transitioned to AWS’s Cloud Infrastructure, using Amazon Web Services (AWS) Availability Zones. This ensured high availability and fault tolerance. GAMA-1 successfully updated and migrated to the NCCF development environment, enhancing operational efficiency and data security.
• Core Services Enhancement: Under GAMA-1’s guidance, NCCF’s core services, such as Data Ingest, Inspection, and Distribution, were streamlined. This included the implementation of a Consolidated Ingest Service, improving the standardization of workflows and data schemas.
• Product Generation and Metadata Management: GAMA-1 introduced an innovative Product Generation system using AWS S3 and EKS clusters. This system ensured efficient metadata extraction and management, enhancing the reliability and accessibility of data.
• Archive and Access Services: The Archive system was developed as a minimally viable product, incorporating multiple AWS serverless resources and Neptune for data and metadata management. The Access VPC services were improved to provide better data accessibility to users.
• NCCF Security: Significant enhancements in security monitoring and operations were implemented, including tools like Nessus, ArcSight, CASB BigFix, and AWS Config. This comprehensive approach ensured adherence to NOAA’s IT Security requirements and robust protection against potential cyber threats.
• Scaled Agile Framework (SAFe) Implementation: GAMA-1 played a pivotal role in adopting and implementing the Scaled Agile Framework (SAFe) within NESDIS. This included the development of new Agile Release Trains (ARTs) and the restructuring of the Essential SAFe approach, which significantly improved the operational efficiency and response to technical demands.
• Jira and Confluence Integration: GAMA-1 utilized NESDIS’ Atlassian Jira and Confluence to manage NCCF requirements. The introduction of Jira Advanced Roadmaps and the enhancement of the Jira Issue Type Hierarchy streamlined project management and cross-team collaboration.

Outcome:
The collaboration between GAMA-1 and NOAA has led to a more agile, secure, and efficient cloud-based framework that significantly enhances NOAA’s ability to manage environmental data. The NCCF now stands as a testament to GAMA-1’s technical prowess and commitment to advancing cloud technology in government operations.