Today I am going to tell you DevOps tools and technologies that you must know to make a career in DevOps.
What is DevOps?
DevOps is a culture which helps to increase organisation’s productivity. They are platform developers who take care of all the activities which is common in development and IT operations.
In early days of agile development approach, lack of communication between developers and operation teams was reducing the development cycle’s efficiency and productivity. It increases the scope of DevOps in the market. Today every big organization has a fully dedicated DevOps team of at least 10 members in each project.
DevOps team consists of 10 to 30 members having multiple skills in tools and technologies. There may be DBAs, cloud engineers, script experts, python experts, architectures etc.
DevOps Tools and Technologies You Must Know
If you are planning to make your career in DevOps then you must be comfortable with Linux Operating System and you should know all basic linux commands. It should be your primary operating system. You can use centos, ubuntu, mint or RedHat.
You should have knowledge of cloud and server.There are many cloud providers in the market. AWS is the oldest cloud provider that provides n number of cloud reliable service and better support. Today many big companies are using AWS servers in their project. Some example of cloud service providers are AWS(Amazon Web Service), GCP(Google Cloud Platform), Digital Ocean, Microsoft Azure, IBM Cloud etc.
In AWS you should have knowledge in ec2 machines, s3, private/public keys concept, private/public IP, route53, Elastic Load Balancer(ELB), auto scaling group(ASG), security groups, virtual private cloud(VPC), private/public subnets, IAM, amazon databases, cloud watch etc.
Docker is an open source devops tool which is designed to create, deploy and run your applications on lightweight containers. Docker works same as virtual machine(VM) does. But unlike VM(virtual machine) instead of creating a new operating system, docker allows applications to use host operating system.
This tool has made a revolutionary change in world of microservices. A DevOps engineer can not imagine himself without docker. Learn more about docker here.
4. Microservices Architecture
Microservices are loosely coupled independently deployable unit which run on containers. In traditional approach monolithic applications is deployed as a single packaged code. But in microservice architecture instead of putting all code in one package, multiple independent package of code is created and deployed on multiple independent lightweight containers. Then these services communicate to each other with a message broker (like Kafka).
In modern microservices architecture, microservices need to communicate to each other to perform some operations. Some microservices might be fast and some of them might be slow. There may be a scenario where some microservices(say A, B, C) want to post data to a particular microservice(D) at a time but D can handle only one microservice (either A or B or C) at a time. So D will loose the data sent by two other services.DevOps uses Kafka to solve such microservices connection issues. It is used for building real-time data pipelines and streaming applications. Microservices communicate to each other through Kafka rather than directly.Kafka stores topics sent by publisher(producer) and these topics are used by subscriber(consumer) whenever it is required.
6. Git (Bitbucket or GitHub)
Like Developers, DevOps are platform Developer. They write script and automation for their platform and they these code on Git. Git is an open source version control platform which is used to store development code. A DevOps engineer must know all basic Git commands to maintain with their code and developer’s code repositories. DevOps have the Administrator rights for all repositories and they give appropriate access to other team members.
Today Database Administration also comes under DevOps. Organizations don’t spend extra money to deploy an isolate DBA team. A member of DevOps team takes care of all database administration operations. One should have good command in SQL queries. Some popular databases which are being used in the market, are MySql, MongoDB, Couchbase, Cassandra etc.
Kubernetes is an open source Google product which is user for container orchestration.
Suppose your three microservices are running on three containers having sufficient configuration for 100 users. After some time, your app has become popular and now daily one lack users visiting your app. Now you must have at least 1000×3 (100000/100=1000 container per microservice), you need to scale your containers from 3 to 3000.
If you will create 3000 container replicas manually then it will take too much time and who know how much scalability would be needed again after sometime. You can’t do it manually.
Here Kubernetes comes into the picture. It is used to manage deployments, replicas, secrets, jobs etc. One should know YAML to give resource specification.
9. Monitoring Tools (Prometheus and Grafana)
To monitor resource usages DevOps use monitoring tools to keep eye on each resource utilisation. Monitoring tools continuously analyse the resource matrix and send an alert whenever feel any misshapen.
There are multiple tools available for monitoring. We can use Prometheus and Grafana.
DevOps uses Prometheus to collect and store matrix coming from resources and we use Grafana to visulize these matrix in form of graph, pi chart, bar chart etc.
10. Scripting and Automation
One important task of DevOps is to automate those operations which are time taking and occurred frequently. There are many scripting languages which can be used to automate the things. Shell Scripting and Python is most popular scripting languages. One can automate operations of any tools and technologies by using its APIs in scripting. Example: DevOps can use Database APIs and write an script to automate data backup, restoration operations.
11. API Gateway
In microservice architecture, there are multiple microservices work behind an application. So whenever an user want to access 4 different features (say Search, Payment, Subscribe, Review) running on 4 different microservice containers, then if there is an API Gateway exist between containers and user then user doesn’t need to hit 4 different url. User will just hit URL of api gateway and then API Gateway will decide where to pass this request.
API Gateway also provides cache service, security etc.
Kong is an API Gateway. AWS also providers API Gateway service.