Kubernetes Reference

Kubernetes Reference

Minikube

minikube start

minikube stop

minikube delete

minikube env

minikube ip



---
Kubectl

kubectl get all
Pods, ReplicaSets, Deployments and Services

kubectl apply –f <yaml file>

kubectl apply –f .

kubectl describe pod <name of pod>

kubectl exec –it <pod name> <command>

kubectl get <pod | po | service | svc | rs | replicaset | deployment | deploy>

kubectl get po --show-labels

kubectl get po --show-labels -l {name}={value}

kubectl delete po <pod name>

kubectl delete po --all



---
Deployment Management

kubectl rollout status deploy <name of deployment>

kubectl rollout history deploy <name of deployment>

kubectl rollout undo deploy <name of deployment>

Docker Reference

Docker Reference

Manage images

docker image pull <image name>

docker image ls

docker image build -t <image name> .

docker image push <image name>

docker image tag <image id> <tag name>



---
Manage Containers

docker container run -p <public port>:<container port> <image name>

docker container ls -a

docker container stop <container id>

docker container start <container id>

docker container rm <container id>

docker container prune

docker container run -it <image name>

docker container run -d <image name>

docker container exec -it <container id> <command>

docker container exec -it <container id> bash

docker container logs -f <container id>

docker container commit -a "author" <container id> <image name>



---
Manage your (local) Virtual Machine

docker-machine ip



---
Manage Networks

docker network ls

docker network create <network name>



---
Manage Volumes

docker volume ls

docker volume prune

docker volume inspect <volume name>

docker volume rm <volume name>



---
Docker Compose

docker-compose up

docker-compose up -d

docker-compose logs -f <service name>

docker-compose down



---
Manage a Swarm

docker swarm init (--advertise-addr <ip address>)

docker service create <args>

docker network create --driver overlay <name>

docker service ls

docker node ls

docker service logs -f <service name>

docker service ps <service name>

docker swarm join-token <worker|manager>



---
Manage Stacks

docker stack ls

docker stack deploy -c <compose file> <stack name>

docker stack rm <stack name>

ElasticSearch PUT and GET data

--Create an index in Elasticsearch

PUT http://host-1:9200/my_index

{
"settings" : {
"number_of_shards" : 3,
"number_of_replicas" : 1
}
}

--To get information about an index

GET http://host-1:9200/my_index


--Add user to index with id 1

POST http://host-1:9200/my_index/user/1

{
"name": "Deepak",
"age": 36,
"department": "IT",
"address": {
"street": "No.123, XYZ street",
"city": "Singapore",
"country": "Singapore"
}
}

--To fetch document with id 1

GET http://host-1:9200/my_index/user/1

--Add user to index with id 2

POST http://host-1:9200/my_index/user/2

{
"name": "McGiven",
"age": 30,
"department": "Finance"
}

--Add user to index with id 3

POST http://host-1:9200/my_index/user/3

{
"name": "Watson",
"age": 30,
"department": "HR",
"address": {
"street": "No.123, XYZ United street",
"city": "Singapore",
"country": "Singapore"
}
}

--Search documents by name

GET http://host-1:9200/my_index/user/_search?q=name:watson

--Delete an index

DELETE http://host-1:9200/my_index

Hadoop Submitting a MapReduce Job

HDFS Input Location:

/user/deepakdubey/input/stocks

HDFS Output Location (relative to user):

output/mapreduce/stocks



Delete Output Directory:

hadoop fs -rm -r output/mapreduce/stocks



Submit Job:

hadoop jar /deepakdubey/mapreduce/stocks/MaxClosePriceByStock-1.0.jar com.deepakdubey.MaxClosePriceByStock /user/deepakdubey/input/stocks output/mapreduce/stocks



View Result:

hadoop fs -cat output/mapreduce/stocks/part-r-00000



=========================


HDFS Input Location:

/user/deepakdubey/input/stocks

HDFS Output Location (relative to user):

output/mapreduce/stocks



Delete Output Directory:

hadoop fs -rm -r output/mapreduce/stocks



Submit Job:

hadoop jar /deepakdubey-starterkit/mapreduce/stocks/MaxClosePriceByStock-1.0.jar com.deepakdubey.MaxClosePriceByStock /user/deepakdubey/input/stocks output/mapreduce/stocks



View Result:

hadoop fs -cat output/mapreduce/stocks/part-r-00000

Caching Data in Spark

symvol.cache()

symvol.persist(org.apache.spark.storage.StorageLevel.MEMORY_ONLY)
symvol.persist(org.apache.spark.storage.StorageLevel.DISK_ONLY)
symvol.persist(org.apache.spark.storage.StorageLevel.MEMORY_AND_DISK)

symvol.unpersist()

--caching symvol

val stocks = sc.textFile("hdfs://ip-10-10-10-10.ec2.internal:8020/user/deepakdubey/input/stocks")
val splits = stocks.map(record => record.split(","))
val symvol = splits.map(arr => (arr(1), arr(7).toInt))
symvol.cache()
val maxvol = symvol.reduceByKey((vol1, vol2) => Math.max(vol1, vol2))
maxvol.collect().foreach(println)