GEOS-Chem on cloud
Overview
Why move to the cloud
Remove technical barriers
Software
Compute
Data
Open new research opportunities
Massive Earth observation data
Machine learning and deep learning
Status of cloud for scientific computing
Atmospheric models
Relevant applications
University Classes
External resources on cloud computing for science
Beginner tutorials
Quick start guide for new users
Step 1: Sign up an Amazon Web Service (AWS) account
Step 2: Launch a server with GEOS-Chem pre-installed
Step 3: Log into the server and run GEOS-Chem
Step 4: Analyze output data with Python (Optional)
Step 5: Shut down the server (Very important!!)
Overview of basic AWS compute and storage services
Core AWS concepts for scientific computing
Set up AWS Command Line Interface (AWS-CLI)
Install AWSCLI
Try to configure AWSCLI
Obtaining security credentials
Finish AWSCLI configuration
Additional notes
Use S3 as major storage
Use S3 from the console
Working with S3 using AWSCLI
Access NASA-NEX data in S3 (Optional but recommended)
Access GEOS-Chem input data repository in S3
Use EBS volumes as temporary disk storage
Viewing existing EBS volumes
Choose volume size at launch time
Attach new volumes after EC2 launch (Optional)
Launch and attach a volume
Make that volume usable
Save volumes into snapshots (Optional)
Use Spot Instances to reduce EC2 cost
What are spot instances and why they exist
Use spot instances for big computing
Deal with spot instance interruptions
Notes on security groups (EC2 firewall)
Put everything together: a complete workflow
General comments on cloud vs local computer
A reference workflow
Launch EC2 instance with custom configuration
Set up your own model configuration
Get source code and checkout model version
Configure unit tester and generate run directory
Get more input data from S3
Tweak run-time configurations
Perform long-term simulation
Analyze output data
Save your files to S3
Terminate server, start over whenever needed
Advanced tutorials
Enable S3 access from EC2 by IAM role
What is Identity and Access Management (IAM)
Grant S3 permission to EC2
Create a new IAM role
Assign that role to EC2
More advanced usages of AWSCLI
Launch on-demand instances
Request spot instances
Other use cases
Use containers to enhance research reproducibility
The reproducibility crisis in science
What are containers and how can they help
Install Singularity container
Run GEOS-Chem inside Singularity container
Overview of HPC cluster options on AWS
Why do you need an HPC cluster
HPC cluster management tools
CfnCluster
AlcesFlight
StarCluster
GEOS-Chem High-Performance version (GCHP) (experimental)
GCHP inside Singularity container
Launch server
Test run
Test compile
Developer guide
Install compilers and NetCDF libraries
Start with a brand new system
Install C and Fortran compilers
Ubuntu
Amazon Linux
Install NetCDF library with package manager
Ubuntu
Amazon Linux
Test sample NetCDF code
(Optional) Install NetCDF from source code
Save your system and share with others
Set up GEOS-Chem environment
Environment variables
Source code, run directory and input data
Install scientific Python environment
Install MPI libraries
Build your own containers
AWS concepts and services in detail
Common AWS concepts (but often confuse non-IT people)
Regions and Availability Zones
Virtual Private Clouds (VPCs) and Subnets
IAM users and IAM roles
Security groups
Monitoring AWS cost
Appendix
List of public AWS resources for GEOS-Chem
Simplify SSH login with config file
Set up SSH config file
Enable port forwarding
For multiple servers
Additional notes
Keep a program running after logoff
Use nohup command (not recommended)
Use GNU Screen
Use tmux (recommended)
Sample Python code to analyze GEOS-Chem data
GEOS-Chem NetCDF diagnostics
GEOS-FP metfield
Sample Python code to analyze NASA-NEX data
GEOS-Chem on cloud
Docs
»
AWS concepts and services in detail
Edit on GitHub
AWS concepts and services in detail
ΒΆ
This chapter dives deeper into AWS services.
Common AWS concepts (but often confuse non-IT people)
Regions and Availability Zones
Virtual Private Clouds (VPCs) and Subnets
IAM users and IAM roles
Security groups
Monitoring AWS cost
Read the Docs
v: v0.2
Versions
latest
v0.2
Downloads
On Read the Docs
Project Home
Builds
Free document hosting provided by
Read the Docs
.