Pages

Subscribe:

Thursday, December 25, 2014

Varnish Installation on Centos 7

In this post we are going to install varnish cache on Centos Linux 7.

We are going to install Varnish Cache 4 on the Centos Linux 7 i.e. 64 bit of architecture using the yum. 

Follow the following steps to install the varnish cache on your server running Centos 7 Linux.


Step 1:- Install the following rpm from the varnish-cache repo 

 yum install https://repo.varnish-cache.org/redhat/varnish-4.0/el7/x86_64/varnish/varnish-debuginfo-4.0.2-1.el7.centos.x86_64.rpm  

 yum install https://repo.varnish-cache.org/redhat/varnish-4.0/el7/x86_64/varnish/varnish-libs-4.0.2-1.el7.centos.x86_64.rpm  

 yum install https://repo.varnish-cache.org/redhat/varnish-4.0/el7/x86_64/varnish/varnish-libs-devel-4.0.2-1.el7.centos.x86_64.rpm  

 yum install https://repo.varnish-cache.org/redhat/varnish-4.0/el7/x86_64/varnish/varnish-docs-4.0.2-1.el7.centos.x86_64.rpm  


Varnish Installation using yum

You can install varnish using the yum or source. We are going to install via yum in this post

Things to consider prior to installation is which version of Varnish cache you are going to install and select the appropriate version to install from the Varnish repo site. You will see each version having 3 folders based on the linux release. Select the appropriate one during the installation.


Sunday, December 21, 2014

Script for getting load, Memory usage and apache connection count in Linux

Following script can be used to format the Average load, Memory used and Apache connection count in Linux format it and present it in a human readable form.

The script is designed to read the hostname , date and time of execution, Average load, Apache process count, physical and swap memory usage



Setting private key in putty for password less access

You can generate a private key using the putty key generator and use the key to simply click and connect to your Linux server with your user account with a simple click of a mouse without having to inputting all the credentials everytime you have to access the server.

Follow these simple steps to set the private key in putty

1. Enter your user followed by ip in the hostname and saved session on the home screen of the putty



Friday, December 12, 2014

Understanding Varnish Cache

This is in continuation to my previous post on Reverse proxy servers.

Varnish is a caching http reverse proxy which means it works as an intermediatery between your client and web server, receives the requests from the client and tries to answer them from the cache , In case if varnish is not able to answer the request from the cache it will forward the request to the backend web server and fetch the response  while storing the response in it cache and delivering the response to the client.



It may be understood as when a  user first time visits the website and requests the page , varnish keeps the copy of the served page and when the user re-visit the same page again, it gets served from the saved copy of the page from the varnish cache instead of re-requesting the page from the web server, Which improves the website's performance and scalability and improves response time.



Varnish cache response is highly responsive and is very faster than the typical backend server, so you want to make sure maximum number of requests are served directly from the varnish cache itself which can typically deliver the speeds up to 300 - 1000x , depending on the architecture.

Varnish cache decides whether it can store the content or not based on the response it receives from the backend in the form HTTP response header Cache-Control. Though there are few conditions were varnish will not cache content , the most common being the use of the cookies since the cookies indicates the client-specific web object, so by default varnish will not cache it.

You can customize most of the varnish functionality by changing the policies written in the Varnish Configuration Language (VCL). Using VCL you can write policy by which you can decide what content you want to server, from where you want to get the content and how the request or response should be altered, thereby giving huge customization options according to your need.

Varnish supports all modern day platforms and is a community driven project and delivered under EPEL package repository.


Understanding Varnish Cache

This is in continuation to my previous post on Reverse proxy servers.

Varnish is a caching http reverse proxy which means it works as an intermediatery between your client and web server, receives the requests from the client and tries to answer them from the cache , In case if varnish is not able to answer the request from the cache it will forward the request to the backend web server and fetch the response  while storing the response in it cache and delivering the response to the client.



It may be understood as when a  user first time visits the website and requests the page , varnish keeps the copy of the served page and when the user re-visit the same page again, it gets served from the saved copy of the page from the varnish cache instead of re-requesting the page from the web server, Which improves the website's performance and scalability and improves response time.


Thursday, December 11, 2014

Understanding the Reverse Proxy Server


A reverse proxy server retrieves the resources on behalf of a client requesting those resources and returns it to the client as it has originated from them itself.


It can be understood as an intermediately between the client and the webserver, when a client is requesting service through internet which is coming to the reverse proxy server , which retrieves the resources from the web server in an internal network and serves it to the client without the client aware about the internal network or its origin i.e. web servers.



Understanding the Reverse Proxy Server


A reverse proxy server retrieves the resources on behalf of a client requesting those resources and returns it to the client as it has originated from them itself.


It can be understood as an intermediately between the client and the webserver, when a client is requesting service through internet which is coming to the reverse proxy server , which retrieves the resources from the web server in an internal network and serves it to the client without the client aware about the internal network or its origin i.e. web servers.


Sunday, November 23, 2014

Understanding, Creating, Extending, Calculating the Swap Space in Linux

1.       Concept of swap space in Linux?
Ans:- Linux allows a hard disk to be used as memory (virtual memory) apart from the RAM( physical memory) so that the kernel is able to write off the unused content from the RAM to this virtual memory and whenever the content is required again it writes back. 


Since the unused content no longer holds the space in physical memory it can be used by some other process. All this is done internally and a user is never aware of the backend process.

2.       Creating a Swap Space in Linux?

Ans:-    $ dd if=/dev/zero of=/extra-swap bs=1024 count=1024
        1024+0 records in
        1024+0 records out
        $

dd if is used to create a file
of is used to create the file name that is extra-swap here
bs is for the size in MB
count  defines the size of this swap space , it is good to keep the size in multiple of 4 since kernel  writes out memory pages  4kb in size , and if its not a multiple of 4 than last few bytes might not be used.

$ mkswap /extra-swap 1024
        Setting up swapspace, size = 1044480 
        bytes
        $

mkswap is used to make the swap partition. It is good to keep the swap partition as file system 82 that is swap space so that swap space can be easily identified later. But this is not necessary for the kernel to recognize the swap space.

NOTE:- You should be very careful when using mkswap, since it does not check that the file or partition isn't used for anything else. You can easily overwrite important files and partitions with mkswap! 

swapon /extra-swap
$
Swap spaces can be used automatically by listing them in the /etc/fstab file.
/dev/hda8        none        swap        sw     0     0
/swapfile        none        swap        sw     0     0
The startup scripts will run the command swapon -a, which will start swapping on all the swap spaces listed in /etc/fstab. Therefore, the swapon command is usually used only when extra swap is needed.

You can monitor the use of swap spaces with free. It will tell the total amount of swap space used.
free
             total       used       free     shared  
 buffers
Mem:         15152      14896        256      12404       2528
-/+ buffers:            12368       2784
Swap:        32452       6684      25768
$


The first line of output (Mem:) shows the physical memory.
That last line (Swap:) shows similar information for the swap spaces. If this line is all zeroes, your swap space is not activated.
The same information is available via top, or using the proc filesystem in file /proc/meminfo.
All the swap spaces that are used automatically with swapon -a can be removed from use with swapoff -a; it looks at the file /etc/fstab to find what to remove. But Any manually used swap spaces will remain in use.

Swap is not an replacement for the physical memory but only an extension, since it is very slow (1000 times) slower than physical memory.

Understanding, Creating, Extending, Calculating the Swap Space in Linux

1.       Concept of swap space in Linux?
Ans:- Linux allows a hard disk to be used as memory (virtual memory) apart from the RAM( physical memory) so that the kernel is able to write off the unused content from the RAM to this virtual memory and whenever the content is required again it writes back. 


Since the unused content no longer holds the space in physical memory it can be used by some other process. All this is done internally and a user is never aware of the backend process.

2.       Creating a Swap Space in Linux?

Ans:-    $ dd if=/dev/zero of=/extra-swap bs=1024 count=1024
        1024+0 records in
        1024+0 records out
        $

dd if is used to create a file
of is used to create the file name that is extra-swap here
bs is for the size in MB
count  defines the size of this swap space , it is good to keep the size in multiple of 4 since kernel  writes out memory pages  4kb in size , and if its not a multiple of 4 than last few bytes might not be used.

Wednesday, April 9, 2014

Tomcat Installation

As a pre-requisite for the installation of Tomcat , you need to have Java installed. Check my previous post for the detailed instruction for  the Java Installation and setting up the necessary environment variables for the Java to work.

  • Download the Tomcat Installation file
 [[email protected] tomcat]# mkdir -p /usr/tomcat  
 [[email protected] tomcat]# cd /usr/tomcat/  
 [email protected] tmp]# wget https://archive.apache.org/dist/tomcat/tomcat-7/v7.0.39/bin/apache-tomcat-7.0.39.tar.gz  
 --2014-04-01 01:32:13-- https://archive.apache.org/dist/tomcat/tomcat-7/v7.0.39/bin/apache-tomcat-7.0.39.tar.gz  
 Resolving archive.apache.org... 192.87.106.229, 140.211.11.131, 2001:610:1:80bc:192:87:106:229  
 Connecting to archive.apache.org|192.87.106.229|:80... connected.  
 HTTP request sent, awaiting response... 200 OK  
 Length: 7831716 (7.5M) [application/x-gzip]  
 Saving to: `apache-tomcat-7.0.39.tar.gz'  
 100%[===================================================================================================================================================================================================>] 7,831,716  86.6K/s  in 79s    
 2014-04-01 01:33:33 (97.3 KB/s) - `apache-tomcat-7.0.39.tar.gz' saved [7831716/7831716]  
 [[email protected] tmp]# du -sh apache-tomcat-7.0.39.tar.gz  
 7.5M  apache-tomcat-7.0.39.tar.gz  


  • Extract the tar file to install the  Tomcat 
 [[email protected]ocalhost tomcat]# tar -zxvf /tmp/apache-tomcat-7.0.39.tar.gz   
  [[email protected] java]# cd /usr/tomcat/apache-tomcat-7.0.39/bin   
  [[email protected] bin]# ls -ltr   
  total 796   
  -rw-r--r-- 1 root root 28615 Mar 22 2013 bootstrap.jar   
  -rw-r--r-- 1 root root 13217 Mar 22 2013 catalina.bat   
  -rwxr-xr-x 1 root root 19276 Mar 22 2013 catalina.sh   
  -rw-r--r-- 1 root root 2121 Mar 22 2013 catalina-tasks.xml   
  -rw-r--r-- 1 root root 24281 Mar 22 2013 commons-daemon.jar   
  -rw-r--r-- 1 root root 202451 Mar 22 2013 commons-daemon-native.tar.gz   
  -rw-r--r-- 1 root root 2131 Mar 22 2013 configtest.bat   
  -rwxr-xr-x 1 root root 1982 Mar 22 2013 configtest.sh   
  -rw-r--r-- 1 root root 1342 Mar 22 2013 cpappend.bat   
  -rwxr-xr-x 1 root root 7492 Mar 22 2013 daemon.sh   
  -rw-r--r-- 1 root root 2178 Mar 22 2013 digest.bat   
  -rwxr-xr-x 1 root root 2021 Mar 22 2013 digest.sh   
  -rw-r--r-- 1 root root 3264 Mar 22 2013 setclasspath.bat   
  -rwxr-xr-x 1 root root 3524 Mar 22 2013 setclasspath.sh   
  -rw-r--r-- 1 root root 2111 Mar 22 2013 shutdown.bat   
  -rwxr-xr-x 1 root root 1960 Mar 22 2013 shutdown.sh   
  -rw-r--r-- 1 root root 2112 Mar 22 2013 startup.bat   
  -rwxr-xr-x 1 root root 1961 Mar 22 2013 startup.sh   
  -rw-r--r-- 1 root root 38161 Mar 22 2013 tomcat-juli.jar   
  -rw-r--r-- 1 root root 288166 Mar 22 2013 tomcat-native.tar.gz   
  -rw-r--r-- 1 root root 4114 Mar 22 2013 tool-wrapper.bat   
  -rwxr-xr-x 1 root root 5086 Mar 22 2013 tool-wrapper.sh   
  -rw-r--r-- 1 root root 2116 Mar 22 2013 version.bat   
  -rwxr-xr-x 1 root root 1965 Mar 22 2013 version.sh   
  [[email protected] bin]# ./startup.sh   
  Using CATALINA_BASE: /usr/tomcat/apache-tomcat-7.0.39   
  Using CATALINA_HOME: /usr/tomcat/apache-tomcat-7.0.39   
  Using CATALINA_TMPDIR: /usr/tomcat/apache-tomcat-7.0.39/temp   
  Using JRE_HOME:  /usr   
  Using CLASSPATH:  /usr/tomcat/apache-tomcat-7.0.39/bin/bootstrap.jar:/usr/tomcat/apache-tomcat-7.0.39/bin/tomcat-juli.jar   
  [[email protected] bin]# ps -ef | grep tomcat   
  root  2145  1 9 05:35 pts/1 00:00:02 /usr/bin/java -Djava.util.logging.config.file=/usr/tomcat/apache-tomcat-7.0.39/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/usr/tomcat/apache-tomcat-7.0.39/endorsed -classpath /usr/tomcat/apache-tomcat-7.0.39/bin/bootstrap.jar:/usr/tomcat/apache-tomcat-7.0.39/bin/tomcat-juli.jar -Dcatalina.base=/usr/tomcat/apache-tomcat-7.0.39 -Dcatalina.home=/usr/tomcat/apache-tomcat-7.0.39 -Djava.io.tmpdir=/usr/tomcat/apache-tomcat-7.0.39/temp org.apache.catalina.startup.Bootstrap start   

    Next we will discussing the linking of the tomcat with the Apache with native connectors.

    Tomcat Installation

    As a pre-requisite for the installation of Tomcat , you need to have Java installed. Check my previous post for the detailed instruction for  the Java Installation and setting up the necessary environment variables for the Java to work.

    • Download the Tomcat Installation file
     [[email protected] tomcat]# mkdir -p /usr/tomcat  
     [[email protected] tomcat]# cd /usr/tomcat/  
     [email protected] tmp]# wget http://archive.apache.org/dist/tomcat/tomcat-7/v7.0.39/bin/apache-tomcat-7.0.39.tar.gz  
     --2014-04-01 01:32:13-- http://archive.apache.org/dist/tomcat/tomcat-7/v7.0.39/bin/apache-tomcat-7.0.39.tar.gz  
     Resolving archive.apache.org... 192.87.106.229, 140.211.11.131, 2001:610:1:80bc:192:87:106:229  
     Connecting to archive.apache.org|192.87.106.229|:80... connected.  
     HTTP request sent, awaiting response... 200 OK  
     Length: 7831716 (7.5M) [application/x-gzip]  
     Saving to: `apache-tomcat-7.0.39.tar.gz'  
     100%[===================================================================================================================================================================================================>] 7,831,716  86.6K/s  in 79s    
     2014-04-01 01:33:33 (97.3 KB/s) - `apache-tomcat-7.0.39.tar.gz' saved [7831716/7831716]  
     [[email protected] tmp]# du -sh apache-tomcat-7.0.39.tar.gz  
     7.5M  apache-tomcat-7.0.39.tar.gz  


    Java Installation on Linux

    Java can be installed on Linux as

    • Download Java ( JDK 6 Update 24
     https://www.oracle.com/technetwork/java/javase/downloads/java-se-6u24-download-338091.html   


    • Creating a Directory for the Java Installation file
     [[email protected] Desktop]# mkdir -p /usr/java  
     [[email protected] Desktop]# cd /usr/java/  
     [[email protected] java]# du -sh jdk-6u24-linux-i586.bin  
     82M   jdk-6u24-linux-i586.bin  
     [[email protected] java]# chmod 777 jdk-6u24-linux-i586.bin  
     [[email protected] java]# ls -ltr jdk-6u24-linux-i586.bin  
     -rwxrwxrwx 1 ankit ankit 84927175 Apr 1 01:18 jdk-6u24-linux-i586.bin  


    • Installing the Java 
     [[email protected] java]# ./jdk-6u24-linux-i586.bin  
     [[email protected] Desktop]# ln -s /usr/java/jdk1.6.0_24/bin/java /usr/bin/java   

    • Finally check the version of the Java as
     [[email protected] bin]# java -version  
     java version "1.6.0_24"  
     Java(TM) SE Runtime Environment (build 1.6.0_24-b07)  
     Java HotSpot(TM) Client VM (build 19.1-b02, mixed mode, sharing)  

    Java Installation on Linux

    Java can be installed on Linux as

    • Download Java ( JDK 6 Update 24
     http://www.oracle.com/technetwork/java/javase/downloads/java-se-6u24-download-338091.html   


    • Creating a Directory for the Java Installation file
     [[email protected] Desktop]# mkdir -p /usr/java  
     [[email protected] Desktop]# cd /usr/java/  
     [[email protected] java]# du -sh jdk-6u24-linux-i586.bin  
     82M   jdk-6u24-linux-i586.bin  
     [[email protected] java]# chmod 777 jdk-6u24-linux-i586.bin  
     [[email protected] java]# ls -ltr jdk-6u24-linux-i586.bin  
     -rwxrwxrwx 1 ankit ankit 84927175 Apr 1 01:18 jdk-6u24-linux-i586.bin  


    Tuesday, April 1, 2014

    Source Installation of Apache

    For the source installation the Apache web server following  are the steps:-

    1. Download  the Apache Web Server as
     # wget http://archive.apache.org/dist/httpd/httpd-2.2.24.tar.gz  

    2. Extract the compressed file as
    [[email protected] ~]# tar -zxvf httpd-2.2.24.tar.gz

    3. Run the configuration file as

     cd /tmp/httpd/httpd-2.2.24   

     ./configure --enable-so --enable-expires --enable-file-cache --enable-cache --enable-disk-cache --enable-mem-cache --enable-headers --enable-ssl --enable-http --disable-userdir --enable-rewrite --enable-deflate --enable-proxy --enable-proxy-connect --enable-proxy-ftp --enable-proxy-http --enable-proxy-ajp --enable-proxy-balancer --enable-cgi --disable-dbd --enable-modules=most --with-mpm=worker --prefix=/usr/local/apache2  

    configure:- It is the configuration for httpd
    --enable  :-  Used for installing the specific modules to be used by Apache. If not used the Apache will be installed with all the modules.
    --prefix    :- specifies the location for the installation of the Apache

    4. Try the configuration and check for any reported errors here
     make