Kubernetes basics

Install Kubernetes in CentOS 7:

• cat <<EOF > /etc/yum.repos.d/kubernetes.repo
  [kubernetes]
  name=Kubernetes
  baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
  enabled=1
  gpgcheck=1
  repo_gpgcheck=1
  gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
  exclude=kube*
  EOF
• setenforce 0
• sed -i ‘s/^SELINUX=enforcing$/SELINUX=permissive/’ /etc/selinux/config
• yum install -y yum-utils device-mapper-persistent-data lvm2
• yum-config-manager –add-repo https://download.docker.com/linux/centos/docker-ce.repo
• yum install -y docker-ce kubelet kubeadm kubectl –disableexcludes=kubernetes
• systemctl enable kubelet && systemctl start kubelet
• systemctl enable docker && systemctl start docker
• cat <<EOF > /etc/sysctl.d/k8s.conf
  net.bridge.bridge-nf-call-ip6tables = 1
  net.bridge.bridge-nf-call-iptables = 1
  EOF
• sysctl –system
• sudo swapoff -a
• kubeadm init  –config kube-config.yml | –pod-network-cidr=10.244.0.0/16
• mkdir -p $HOME/.kube
• sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
• sudo chown $(id -u):$(id -g) $HOME/.kube/config
• kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml
• kubectl get nodes
• kubectl get pods    [–all-namespaces]
• Join Kubernetes Node:
  • kubeadm join 172.31.8.0:6443 –token hr3ezc.sqxzkswve3btzfq2 –discovery-token-ca-cert-hash sha256:17911c97ee833227d924513f361a56b91e2dd546c9ed563c263f7ea6072aecc2
• Issues and solutions:
  • Error:  Unable to update cni config: No networks found in /etc/cni/net.d
    • /var/log/message (journalctl -xeu kubelet)
  • Solution: Run the following commands on master and all nodes:
    • mkdir  -p /etc/cni/net.d
    • vi /etc/cni/net.d/10-flannel.conf
      {
      “name”: “cbr0”,
      “type”: “flannel”,
      “delegate”: {
      “isDefaultGateway”: true
      }
      }

Kubernets basics commands:

• kubectl get nodes
• kubectl run nginx-20533e0703 –image=nginx –replicas=1 –port=80
• kubectl get pods
• kubectl get deployment
• kubectl expose deployment nginx-20533e0703 –port=80 –type=LoadBalancer
• kubectl get services
• kubectl scale –replicas=2 deployment/nginx-20533e0703
• kubectl get pods
• kubectl delete deployment nginx-20533e0703
• kubectl get deployment

Advertisements

Setup ELK Stack in Linux

• Install and Configure Elasticsearch:
  1. yum install java-openjdk
  2. rpm –import https://artifacts.elastic.co/GPG-KEY-elasticsearch
  3. curl -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.5.1.rpm
  4. rpm -ivh elasticsearch-6.5.1.rpm
  5. vi /etc/elasticsearch/elasticsearch.yml
     @Master host:
     node.name: master1
     node.data: false
     network.host: [“localhost”, “192.168.56.201”]
     @Data host:
     node.name: data1
     node.master: false
     network.host: [“localhost”, “192.168.56.202”]
     discovery.zen.ping.unicast.hosts: [“192.168.56.201”]
  6. /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-geoip
  7. systemctl enable elasticsearch && systemctl start elasticsearch
  8. firewall-cmd –add-port=9300/tcp –add-port=9200/tcp –permanent && firewall-cmd –reload
  9. less /var/log/elasticsearch/elasticsearch.log
  10. curl http://localhost:9200/_cluster/health?pretty=true
• Install and Configure Logstash:
  1. wget https://artifacts.elastic.co/downloads/logstash/logstash-6.5.1.rpm
  2. rpm –install logstash-6.5.1.rpm
  3. wget https://raw.githubusercontent.com/linuxacademy/content-elastic-log-samples/master/apache.conf
  4. systemctl enable logstash && systemctl start logstash
  5. less /var/log/logstash/logstash-plain.log
• Install and configure Filebeat:
  1. mkdir /var/log/apache2
  2. curl https://raw.githubusercontent.com/linuxacademy/content-elastic-log-samples/master/access.log > /var/log/apache2/access.log
  3. curl -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.5.1-x86_64.rpm
  4. rpm –install filebeat-6.5.1-x86_64.rpm
  5. vi /etc/filebeat/filebeat.yml
     #output.elasticsearch:
     #hosts: [“localhost:9200”]
     output.logstash:
     hosts: [“localhost:5044”]
  6. filebeat modules enable apache2
  7. filebeat setup
  8. systemctl enable filebeat && systemctl start filebeat
  9. less /var/log/filebeat/filebeat
  10. curl http://localhost:9200/_cluster/health?pretty=true
• Install and Configure Kibana
  1. curl -O https://artifacts.elastic.co/downloads/kibana/kibana-6.5.1-x86_64.rpm
  2. rpm –install kibana-6.5.1-x86_64.rpm
  3. systemctl enable kibana && systemctl start kibana
  4. less /var/log/messages
  5. ssh user@192.168.56.201 -L 5601:localhost:5601
  6. http://localhost:5601
     Step#1: Management > Kibana = Create index pattern: filebeat-*
     Step#2: @timestamp
  7. Discover: Set timestamp
• Install and configure Nginx:
  1. yum install epel-release
  2. yum nginx httpd-tools
  3. htpasswd -c /etc/nginx/htpasswd.users kibanaadmin
  4. vi /etc/nginx/nginx.conf
     location / {
     rewrite ^/(.*) /$1 break;
     proxy_ignore_client_abort on;
     proxy_pass http://localhost:5601;
     proxy_set_header X-Real-IP $remote_addr;
     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header Host $http_host;
     }
  5. systemctl enable nginx && systemctl start nginx
  6. curl http://localhost

Jenkins behind an Nginx Reverse Proxy

Setup Nginx:

• yum install nginx
• vim /etc/nginx/nginx.conf
  location / {
  sendfile off;
  proxy_pass http://localhost:8080;
  proxy_redirect default;
  proxy_http_version 1.1;
  proxy_set_header Host $host;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Proto $scheme;
  proxy_max_temp_file_size 0;
  #this is the maximum upload size
  client_max_body_size 10m;
  client_body_buffer_size 128k;
  proxy_connect_timeout 90;
  proxy_send_timeout 90;
  proxy_read_timeout 90;
  proxy_buffering off;
  proxy_request_buffering off; # Required for HTTP CLI commands in Jenkins > 2.54
  proxy_set_header Connection “”; # Clear for keepalive}
• semanage port -mt http_port_t -p tcp 8080
• systemctl start nginx
• Reference: https://wiki.jenkins.io/display/JENKINS/Running+Jenkins+behind+Nginx

Setup Jenkins:

1. Install Jenkins:
   • yum remove java -y  (Optional: Remove if any existing incompatible Java)
   • wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo
   • rpm –import https://jenkins-ci.org/redhat/jenkins-ci.org.key
   • yum update -y
   • yum install jenkins  java-1.8.0-openjdk-devel -y
   • systemctl enable jenkins && systemctl start jenkins
2. Setup Jenkins user:
   • usermod -s /bin/bash jenkins
   • passwd jenkins
   • echo “jenkins ALL=(ALL) NOPASSWD: ALL” > /etc/sudoers.d/jenkins
   • ssh-keygen
   • ssh-copy-id
3. Add firewall rule:
   • firewall-cmd –add-port=8080/tcp –permanent
   • firewall-cmd –reload
   • firewall-cmd –list-all
4. Configure Jenkins:
   • http://localhost                             (Jenkins running on: http://localhost:8080)
   • cat /var/lib/jenkins/secrets/initialAdminPassword
   • Install suggest plugins
   • admin/admin; Admin/admin@localhost
5.  Jenkins configs:
   • Install dir: /var/lib/jenkins/
   • Config: /etc/sysconfig/jenkins
   • Log: /var/log/jenkins/jenkins.log
6. Jenkins CLI:
   • wget http://localhost:8080/jnlpJars/jenkins-cli.jar
   • java -jar jenkins-cli.jar -s http://localhost:8080 who-am-i –username admin –password admin
7. Jenkins inside Docker:
   • docker run –rm -u root -p 8080:8080 \
     -v jenkins-data:/var/jenkins_home \
     -v /var/run/docker.sock:/var/run/docker.sock \
     -v “$HOME”:/home \
     jenkinsci/blueocean
   • docker run -p 8080:8080 jenkinsci/blueocean
   • docker run jenkins/jenkins:lts –version
8. Configure Docker with Jenkins:
   • usermod -aG docker jenkins
9. Configure Git with Jenkins:
   • yum install git -y

 

Install Sonarqube on Linux

• Create sonar user and group:
  • groupadd sonar
  • useradd -d /opt/sonarqube -g sonar -s /bin/bash sonar
  • chown -R sonar:sonar /opt/sonarqube
• Configure Java 8:
  1. wget  http://download.oracle.com/otn-pub/java/jdk/8u191-b12/2787e4a523244c269598db4e85c51e0c/jdk-8u191-linux-x64.tar.gz -P /tmp
  2. tar -zxvf /tmp/jdk-8u191-linux-x64.tar.gz -C /opt/
  3. vi /etc/profile.d/java.sh
     • export JAVA_HOME=/opt/jdk8
     • export PATH=$JAVA_HOME/bin:$PATH
  4. chmod +x  /etc/profile.d/java.sh
  5. source /etc/profile.d/java.sh
  6. java -version
• Configure PostgreSQL 10 database:
  1. Install PostgreSQL:
     • rpm -Uvh https://yum.postgresql.org/10/redhat/rhel-7-x86_64/pgdg-centos10-10-2.noarch.rpm
     • yum install postgresql10-server postgresql10
     • /usr/pgsql-10/bin/postgresql-10-setup initdb
     • systemctl start postgresql-10.service
     • systemctl enable postgresql-10.service
  2. Setup sonar database:
     • su – postgres -c psql
     • ALTER USER sonar WITH ENCRYPTED password ‘sonar’;
     • CREATE DATABASE sonar WITH ENCODING ‘UTF8’ OWNER sonar TEMPLATE=template0;
     • \q
• Configure Sonarqube:
  1. Download sonarqube:
     • wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-7.4.zip -P /tmp/
     • unzip /tmp/sonarqube-7.4.zip -d /opt/sonarqube
  2. vi /opt/sonarqube/conf/sonar.properties
     • sonar.jdbc.username=sonarqube
     • sonar.jdbc.password=sonarqube
     • sonar.jdbc.url=jdbc:postgresql://localhost/sonarqube
  3. vi /opt/sonarqube/conf/wrapper.properties
     • wrapper.java.command=/opt/jdk8/bin/java
  4. Sonarqube systemd service:
     1. vi /etc/systemd/system/sonar.service
        [Unit]
        Description=Sonarqube service
        After=syslog.target network.target
        [Service]
        Type=forking
        ExecStart=/opt/sonarqube/bin/linux-x86-64/sonar.sh start
        ExecStop=/opt/sonarqube/bin/linux-x86-64/sonar.sh stop
        User=sonar
        Group=sonar
        [Install]
        WantedBy=multi-user.target
     2. systemctl daemon-reload
     3. systemctl start sonar
     4. systemctl status sonar
     5. http://localhost:9000
     6. Login with admin/admin
• Run Sonarqube and PostgreSQL in Docker:
  • #!/bin/bash
    #Creating docker network mynet
    docker network create mynet# Run Postgresql container
    docker run –name postgres -e POSTGRES_USER=sonar -e POSTGRES_PASSWORD=sonar -d -p 5432:5432 –net mynet postgres# Run Sonarqube container
    docker run –name sonarqube -p 9000:9000 -e SONARQUBE_JDBC_USERNAME=sonar -e SONARQUBE_JDBC_PASSWORD=sonar -e SONARQUBE_JDBC_URL=jdbc:postgresql://postgres:5432/sonar -d –net mynet sonarqube
  • https://gist.github.com/ceduliocezar/b3bf93125024482b5f2f479696842046
• SonarScanner / SonarRunner:
  • ${SONAR_RUNNER_HOME}/bin/sonar-runner \
    -Dsonar.projectKey=com.mycompany.app:my-app \
    -Dsonar.sources=. \
    -Dsonar.java.binaries=target \
    -Dsonar.host.://localhost:9000 \
    -Dsonar.eed2bd6caff9888996212edae388e1f387f82c32
• sudo -H -u sonarqube sh -c “bin/linux-x86-64/sonar.sh ‘$1′”
• References:
  • https://docs.sonarqube.org/latest/setup/get-started-2-minutes/
  • https://gist.github.com/ceduliocezar/b3bf93125024482b5f2f479696842046
  • https://dunterov.github.io/sq-psql/
  • https://michalwegrzyn.wordpress.com/2016/07/14/do-not-run-sonar-as-root/
  • https://www.vultr.com/docs/how-to-install-sonarqube-on-centos-7

Version Control Systems

Git setup:

• Required:
  • yum install git && git –version
  • git config [–global] user.name “user”
  • git config [–global] user.email “user@localhost”
• General:
  • git config –global credential.helper “cache –timeout=28800”
  • git config –global core.excludesfile /etc/gitignore
  • git config –global http.postBuffer 524288000
  • git config –global credential.helper store
  • git config –system core.editor “/usr/bin/vim”
  • git config –list [–global|–system]
• Windows:
  • git config –system core.longpaths true
  • git config –global core.autocrlf true
  • git config –list [–global|–system]
• Github repository clone to local directory:
  • mkdir repo && cd repo
  • git clone https://github.com/user/repo [git init repo | git clone /home/user/repo .]
  • git clone https://username:github.com/user/repo
  • git clone git@github.com:user/repo       (For ssh based authentication)
    • ~/.ssh/id_rsa.pub > github.com > user/repo/settings > Deploy keys
• Add local repository to Github:
  • cd repo      (Assuming repo already exist with src files)
  • git init
  • git add .  [–a]]
  • git commit -m “My first commit”
  • git remote add origin http://github.com/username/repo.git
  • git push -u origin master
• Add/Remove/Commit:
  • git add . && git commit -m “Initial commit of files into Repo”
  • git rm test.txt && git commit -m “Removed test2.txt”
• Checkout/Push/Merge:
  • git checkout -b dev && git branch
  • git checkout — test2.txt
  • git log [–online | –grep=”pattern” | –author=”user” | –grpah –decorate | -p]
  • git checkout master && git merge qa
  • git push [-u origin master|branch]
  • git pull               (Copy latest changes from remote master repository)
• Store credentials:
  • git config credential.helper store
  • git push http://example.com/repo.git
• Script:
  #!/bin/bash
  TXT=”Updated Jenkinsfile”
  if [ ! -z $1 ]; then
  TXT=$1
  fi
  git add .
  git commit -m $TXT
  git push
  #git push -u origin master

SVN setup:

• Install:
  • yum install httpd mod_dav_svn subversion
  • vi /etc/httpd/conf.d/subversion.conf 
    LoadModule dav_svn_module modules/mod_dav_svn.so
    LoadModule authz_svn_module modules/mod_authz_svn.so
    <Location /svn>
    DAV svn
    SVNParentPath /var/www/svn
    AuthType Basic
    AuthName “Authorization Realm”
    AuthUserFile /etc/svn-users
    Require valid-user
    </Location>
  • htpasswd -cm /etc/svn-users tom
  • htpasswd -cm /etc/svn-users jerry
  • mkdir /var/www/svn && cd /var/www/svn
  • svnadmin create project_repo
  • chown -R apache:apache project_repo/
  • chcon -R -t httpd_sys_content_t /var/www/svn/project_repo/
  • firewall-cmd –add-service=http –permanent && firewall-cmd –reload
  • systemctl restart httpd
  • vi /var/www/svn/project_repo/conf/svnserve.conf
    anon-access = none
    authz-db = authz
    
  • mkdir  /tmp/project_repo \
    /tmp/project_repo/trunk  /tmp/project_repo/branches  /tmp/project_repo/tags
  • svn import -m ‘Create trunk, branches, tags dir structure’  \
    /tmp/project_repo  http://localhost/svn/project_repo
• Basic commands:
  • svn checkout http://localhost/svn/project_repo [–username=tom]
  • svn co http://localhost/svn/project_repo [–username=jerry]
  • svn sw http://localhost/svn/project_repo/branches/dev/WEBCMS-453
  • svn add file
  • svn commit -m “Initial commit”
  • svn up           (SVN Update)
  • svn info
  • svn status
  • svn log
  • svn diff
  • svn copy trunk branches/jerry_branch
  • svn commit -m “Jerry’s private branch”
  • svn merge ../branches/jerry_branch/

Setup Cntlm proxy in CentOS

• Download and Install:
  1. curl -o /tmp/cntlm.rpm  https://sourceforge.net/projects/cntlm/files/cntlm/cntlm%200.92.3/cntlm-0.92.3-1.x86_64.rpm
  2. sudo rpm -ivh /tmp/cntlm-*.rpm
• Configure:
  1. cntlm -H -d domain1 -u user1
  2. sudo vi /etc/cntlm.conf
     Username user1
     Domain domain1
     PassNTLMv2   11112345325gsdg4535435    (Use this value from step#1)
     Proxy   www.myproxy.com:8080
     Listen 127.0.0.1:3128
     Listen 192.168.1.6:3128
  3. sudo cntlm -M http://google.com                          (Test it!)
  4. CentOS:
     1. vi  /etc/profile.d/proxy.sh              (~/.bash_profile)
        export http_proxy=http://localhost:3128
        export https_proxy=${http_proxy}
     2. source /etc/profile.d/proxy.sh
     3. vi /etc/yum.conf
        proxy=http://localhost:3128
  5. Ubuntu:
     1. vi  /etc/profile.d/proxy.sh     (vi ~.bashrc)
        export http_proxy=http://localhost:3128
        export https_proxy=${http_proxy}
     2. source /etc/profile.d/proxy.sh
     3. vi /etc/apt/apt.conf
        Acquire::http::Proxy “http://localhost:3128&#8221;;
        Acquire::https::Proxy “http://localhost:3128&#8221;;
  6. Common Issues:
     1. CentOS7 /var/log/messages: cntlm[8976]: Error creating a new PID file:
        • sudo vi /usr/lib/tmpfiles.d/cntlm.conf
          d /run/cntlm 0755 cntlm cntlm
        • Reboot
     2. Windows 10: Couldn’t start Cntlm service:
        1. Open regedit.exe and go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\cntlm\Parameters.
        2. Then change the AppArgs key to -f -c “C:\Program Files (x86)\Cntlm\cntlm.ini”

References:

• https://vijiboy.wordpress.com/2018/03/14/configure-cntlm-to-generate-hash-for-your-password-h-and-verify-m/
• https://gist.github.com/lpf23/d3c4e1ef158c7fb4a909
• https://stackoverflow.com/questions/35338394/cntlmd-not-starting-under-systemd-on-centos-7-1/35692116#35692116
• https://superuser.com/questions/1192694/cannot-start-cntlm

 

 

Azure Databases

• SQL Server:
  • SQL elastic pool: Elastic pool provide a simple and cost effective solution for managing the performance of multiple databases withing a fixed budget.
    • An elastic pool provides compute (eDTUs) and storage resources that are shared between all the databases it contains.
    • Databases within a pool only use the resources they need, when they need them, within configurable limits.
    • The price of a pool is based only on the amonunt of resources configured is independent of the number of databases it contains.
  • Advanced Threat Protection: A unified security package for discovering and classifying sensitive data, surfacing and mitigating potential database vulnerabilities, and detecting anomalous activities that could indicate a threat to your database.
  • Basic: For less demanding workloads.
  • Standard: For workloads with typical performance requirements.
  • Premium: For IO-intensive workloads.
• MySQL Server:
  • Basic: Up to 2 vCores with variable IO performance (1-2 vCores). Supports Backup Redundancy Option only Locally Redundant.
  • General Purpose: Up to 32 vCores with predictable IO performance (2-32 vCores). Supports Backup Redundancy Option both Locally Redundant and Geo-Redundant.
  • Memory Optimized: Up to 16 memory optimized vCores with predictable IO performance (2-16 vCores). Supports Backup Redundancy Option both Locally Redundant and Geo-Redundant.
  • Please note that changing to and from the Basic pricing tier or changing the backup redundancy options after server creation is not supported.