AWS as the New OS

A mental model mapping the familiar to the new

I had this idea a couple of years ago but never really took the time to articulate it and I think that it may help some people see the cloud, specifically as it pertains to software development, in a different way. While it is often thought of as primarily managed infrastructure, the AWS cloud could be thought of as a full-fledged software platform (among other things but this is the aspect I want to focus on here). This concept is further reinforced with Serverless Application Services where there are no machines or infrastructure to manage at all.

If your perspective of the main benefit of the cloud is purely infrastructure management and virtual machines I think it might help to draw some parallels to the more established concept of an Operating System. This analogy is not perfect, but I think it is a great frame of reference for these new concepts and it will help highlight the true power of the AWS Cloud Platform.

Origins of Operating Systems

Computers started out as pure computation machines. They would be set up for a single program and halt when the program completed. As hardware capabilities and the demand for computation increased, the need for sharing these resources arose. So, we essentially made a single program that runs indefinitely and can run and manage the lifecycle of other programs. As this evolved, features were added and common subsystems developed (think bootloader, kernel, drivers, etc) that eventually morphed into the modern operating system.

So, the OS came out of the need for sharing resources. But wow! So did the cloud. Ok, so maybe not so much ‘sharing’ resources are they are ‘selling’ them to you.

In the next couple of paragraphs, I am going to compare some individual concepts.

Processes

The native execution concept of every modern OS is the Process. A process starts, executes code, and eventually completes. It has limited access to the resources, such as disk and networking, enforced by the OS. A process can spawn and manage multiple threads of execution. Processes are generally written in a high-level language such as C, C++, Python, Java, or .NET.

There are several analogues to this in AWS:

• EC2 instances are VMS and host entire operating systems but often when using cloud best practices, they functionally become closer to classical processes: built lean and for a single purpose.
• ECS Containers are already processes, albeit with some additional constraints and features. Those familiar with containers outside of AWS might point to additional management of container hosts and clusters but AWS provides a fully managed serverless container option, called Fargate, that eliminates this additional overhead.
• AWS Lambda functions are executed in pure form. Lambda supports several different runtimes (ie .NET, JavaScript, Python). You essentially just write a function using your language of choice and then upload it to AWS and it’s callable in a slew of different ways. Lambdas are serverless, meaning there is no server for you to manage, and are scaled for you automatically as required and you only pay per request and for memory allocated for the duration of execution; so you don’t pay for unused capacity. An additional benefit of Lambda is its convenient integrations with other AWS services

In the OS world you would usually interact with application executables in the following ways:

• Scheduled via cron job or a task scheduler – in AWS you can set up a CloudWatch Event Rule to trigger your ECS task (container), Lambda, or start an EC2 Instance that runs your job. CloudWatch Events Rules support cron-formatted schedules
• Ongoing Daemons/Services – in AWS you can just use an ECS container or EC2 instance that does not terminate
• On demand –AWS has a command line interface for all major OSes that can be called in a terminal session

Memory

All the types of execution above have local in-process memory that you can access just like you would in a classical process, but many times cloud architectures will employ a cache to act as shared memory between separate threads of execution. This is not unique to the cloud but is a well-established best practice to enable horizontal (more machines) scalability. AWS has a service for this called ElastiCache. ElastiCache is a managed version of either Redis or Memcached (two popular cache servers) that you can set up with a GUI, AWS command line, or API calls and includes features such as configurable autoscaling.

Persistent Storage

In the OS world we have disks and file systems (local and shared) for storing data that we want to save indefinitely. Conceptually I think the analogs in the AWS world will be pretty easy to grasp.

Some AWS persistent storage options are:

• Instance stores – these are disks you can access in an EC2 instance that are directly attached to the VM host, just like you would have in an on-premise VM. 
• It is important to note, however, that an instance store is forever tied to the instance for which it is created; if the instance is terminated, the instance store will be gone. Additionally, much like physical drive, there is no built in redundancy; if the underlying drive dies the data is lost
• Elastic Block Store (EBS) – these are disks you can attach to one EC2 instance at a time, but you can move them between instances
• Elastic File System (EFS) – these are network shares, if you are familiar with NFS, EFS is built upon that. EFS is not supported for Windows instances
• Simple Storage Service (S3) – S3 is the workhorse for AWS storage. There are other options here, but I will draw the comparison to an FTP site in the old world, except it does so much more. Like FTP compared to local disks, S3 will be slower than EFS due to network transfer. S3 also uses its own protocol as opposed to the file systems built into the OS. S3 has permissions management and you can even turn an S3 bucket into a public web site using only configuration. S3 is designed for 11 9’s of durability, meaning 99.999999999%, so the chance of data loss is miniscule. To achieve this with FTP you’re almost certainly adding RAID plus the synchronization to and possibly management of multiple data centers.

Permissions

Permissions concepts in the AWS cloud should seem remarkably familiar compared to those of the OS. AWS wraps this up in a service called IAM and you get the familiar concepts of users, groups, and permissions/policies based on the types of resources you have. Permissions in both AWS and on the OS can end as simple or as complicated as your security needs. In AWS these permissions related concepts are grouped under the Identity Access Management (IAM) umbrella service. AWS provides robust and sophisticated security features to support just about any workload. There are even integrations with Active Directory and web identity providers such as Facebook and Google.

Summary

In the end the AWS cloud and Operating Systems are both software platforms – a set of common reusable tools and resources for building and executing software. There are hundreds of AWS services and many of the newer services are Serverless (no machines, no operational overhead), just callable services from your software. In this way, these AWS services are becoming just like a software library in a traditional operating system.

If you think about AWS as this nearly infinitely scalable operating system it can bring you a new perspective: what was a virtual infrastructure environment has now become a powerful tool for your systems.