Total Pageviews

Thursday, December 22, 2011

binding Apache2 to different port than 80

To configure apache2 to listen on different port than 80 (default port)

http://ubuntuforums.org/showthread.php?t=1329607

open
sudo vi /etc/apache2/ports.conf

NameVirtualHost *:81
Listen 81
<IfModule mod_ssl.c>
    # SSL name based virtual hosts are not yet supported, therefore no
    # NameVirtualHost statement here
    Listen 443
</IfModule>

execute command sudo service apache2 restart

1. apt-get install tomcat7
2. apt-get install mysql-server
3.

Converting PEM key to PPK in Ubuntu

Converting PEM key to  PPK

For putty to connect the amazon ec2 instance, it requires the security key file to be in ppk format.

Amazon provides the key file in PEM format.

To convert PEM key to PPK format in ubuntu, the command to execute on Ubuntu console(terminal):



Wednesday, November 23, 2011

Tomcat Mysql JDBC connection diagnosis

Tomcat Mysql JDBC connection diagnosis

In case of your web application is unable to connect to the Mysql database, chec the following:

  1. Is your Web.xml (/var/lib/tomcat6/webapps/<..>) refering to the correct IP or host name refering to Mysql server, with correct username/password
  2. MySql server is enabled with network access (/etc/mysql/my.cnf) and bind address as 0.0.0.0.
  3. check "mysql-connector-java-5.1.18-bin.jar" (version may vary) in path: "/usr/share/java".
  • /usr/share/tomcat6/lib shall have a softlink pointing to mysql-connector-java-5.1.18-bin.jar - 
  • command for softlink: ln -s /usr/share/java/mysql-connector-java-5.1.18-bin.jar mysql.jar - run this command inside /usr/share/tomcat6/lib
  • when you restart the tomcat server, check the catalina.out, does it showing any errors (/var/lib/tomcat6/logs/catalina.out)

NOTE: No CLASSPATH setting is required. It can be empty.
 
4. check from mysql database side- the username/password you are using to login has GRANT permission.

5.  check from the mysql server side- the mysql port (3306- default) is opened.



Thursday, July 14, 2011

How to attach and mount EBS volume into EC2 Instance Ubuntu 10.10

 

http://yoodey.com/how-attach-and-mount-ebs-volume-ec2-instance-ubuntu-1010

Updated!
Before you do this, backup all /var into /var-backup by sudo rsync -avr /var/* /var-backup/
After mounting EBS, you can rsync restore /var by sudo rsync -avr /var-backup/* /var/
Using EBS volume can decrease risk of server crash suddenly or "permission denied" problem in EC2 server. The logic, after we create instance store, we create new EBS volume which we can set the capacity as we need. In this cases, i create 60GB EBS and will using it as /var in instance store. Now, let configure our EC2 and using EBS as indepent storage.
1. Create EBS Volume and attach it into instance store in EC2 panel management. Use same region, ex : east-1b
2. Login into instance store SSH.
3. In my configuration, EBS Volume located in /dev/sdg. So remember what yours.
4. Use sudo fdisk -l to see if your EBS already attached
Disk /dev/sdg: 64.4 GB, 64424509440 bytes
255 heads, 63 sectors/track, 7832 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
5. Formating EBS Volume so we can use it by sudo mkfs -t ext4 /dev/sdg and you will get result :
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
3932160 inodes, 15728640 blocks
786432 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
480 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424
This will takes a long time, like 10-30 minutes.
6. Edit /etc/fstab and add this :
/dev/sdg        /var    auto    defaults,nobootwait,noatime     0       0
and reboot from EC2 panel management. Now, you use EBS for all installed program and www.

Wednesday, July 13, 2011

7ZIP on Ubuntu server

Install p7zip (7zip)
sudo apt-get install p7zip

To Compress:
p7zip mystuff

To decompress:
p7zip -d mystuff.7z 

Reference URL: http://tombuntu.com/index.php/2008/07/21/add-7z-7-zip-file-archive-support-to-ubuntu/ 

Friday, July 8, 2011

Setting up SVN server on Ubuntu

Instructions to setup SVN server on ubuntu:

http://www.howtogeek.com/howto/ubuntu/install-subversion-with-web-access-on-ubuntu/

In case URL link doesnt work, content below.

Install Subversion with Web Access on Ubuntu

This article covers installing subversion with the apache module so that it can be easily accessed from other systems on a public network. If you want a more secure svn server, you could use svnserve+ssh, which isn’t covered in this article.
To install subversion, open a terminal and run the following command:
sudo apt-get install subversion libapache2-svn
We’re going to create the subversion repository in /svn, although you should choose a location that has a good amount of space.
sudo svnadmin create /svn
Next we’ll need to edit the configuration file for the subversion webdav module. You can use a different editor if you’d like.
sudo gedit /etc/apache2/mods-enabled/dav_svn.conf
The Location element in the configuration file dictates the root directory where subversion will be acessible from, for instance: http://www.server.com/svn
<Location /svn>
The DAV line needs to be uncommented to enable the dav module
# Uncomment this to enable the repository,
DAV svn
The SVNPath line should be set to the same place your created the repository with the svnadmin command.
# Set this to the path to your repository
SVNPath /svn
The next section will let you turn on authentication. This is just basic authentication, so don’t consider it extremely secure. The password file will be located where the AuthUserFile setting sets it to…  probably best to leave it at the default.

# Uncomment the following 3 lines to enable Basic Authentication
AuthType Basic
AuthName “Subversion Repository”
AuthUserFile /etc/apache2/dav_svn.passwd
To create a user on the repository use, the following command:

sudo htpasswd2 -cm /etc/apache2/dav_svn.passwd <username>
Note that you should only use the -c option the FIRST time that you create a user. After that you will only want to use the -m option, which specifies MD5 encryption of the password, but doesn’t recreate the file.
Example:
sudo htpasswd2 -cm /etc/apache2/dav_svn.passwd geek
New password:
Re-type new password:
Adding password for user geek
Restart apache by running the following command:

sudo /etc/init.d/apache2 restart
Now if you go in your browser to http://www.server.com/svn, you should see that the repository is enabled for anonymous read access, but commit access will require a username.

If you want to force all users to authenticate even for read access, add the following line right below the AuthUserFile line from above. Restart apache after changing this line.

Require valid-user
Now if you refresh your browser, you’ll be prompted for your credentials:

Friday, March 11, 2011

1. Create new Amazon Instance
2. Choose Ubuntu instance (ami-62b6c830)
3. apt-get update 
4. Install Tomcat: sudo apt-get install tomcat6 (refer: https://help.ubuntu.com/9.10/serverguide/C/tomcat.html)
5. configure virtual host: http://www.ex-parrot.com/pete/tomcat-vhost.html
6. Mount S3 (make it mounting as boot time) refer amazon s3 mounting section in this blog.
7.  grant permissions to IP address of "all" tomcat instances which are accessing databases in MySQL servers:

The below two steps are to be performed on MY SQL servers
1. mysql -u root -p
2. GRANT ALL ON <databasename>.* TO root@<private IP of amazon instance> IDENTIFIED BY '<password>';




Wednesday, March 9, 2011

Useful Linux links and commands

Useful Linux links:
http://www.cyberciti.biz/faq/howto-linux-add-user-to-group/

http://www.cyberciti.biz/faq/how-to-use-chmod-and-chown-command/
http://www.cyberciti.biz/tips/top-linux-monitoring-tools.html

useful linux commands:
To check folder size: du -s -h
du - Disk usage

S3fs version: s3fs --version


Disk Utilities:
http://www.howtogeek.com/howto/37659/the-beginners-guide-to-linux-disk-utilities/

some commands;
du - disk usage
free -m - checking memory usage
top - load parameters on system

How to count files (get the number of files) under a directory in Linux?

Just perform this command under the directory:
find . -type f | wc -l
 
and you will get the total number of files under it (recursively). Simply copy and paste that to do the trick.
Or if you just need to count the files directly under the current directory:
ls -f | wc -l

Which is a lot faster than the previous solution.
This should also work on all *nix OS.

 

Tuesday, March 8, 2011

setting up new Amazon instance for Tomcat(DMS)

1. Launch new instance
2. sudo su
3. apt-get update
4. sudo apt-get install tomcat6 (refer: https://help.ubuntu.com/9.10/serverguide/C/tomcat.html)
5. http://www.ex-parrot.com/pete/tomcat-vhost.html


In case Links fail:


http://thelowedown.wordpress.com/2010/08/17/tomcat-6-binding-to-a-privileged-port-on-debianubuntu/


Running Tomcat on a privileged port used to be as simple as modifying the connector in Tomcat’s server.xml file.  However, that meant running Tomcat as root — leaving open the possibility of privilege escalation and system compromise should exploitable vulnerabilities exist.  As of Tomcat 6.0.24, the Debian/Ubuntu package includes some changes in the way Tomcat starts, including how it binds to privileged ports.
The installation of the tomcat6 package also creates the tomcat6 user and group.   The user runs tomcat, and both the user and group own portions of the CATALINA_BASE directory tree.  This is an unprivileged user, and so it cannot bind to privileged (aka well-known) ports.  Many daemons start as root, bind to a privileged port and perform other setup work, then drop privileges and run as another user.  Tomcat 6, however, now uses authbind(1) for this purpose.  Authbind provides access control as follows, from the authbind(1) manpage:
Access to low numbered ports is controlled by permissions and contents of files in  a  configuration area, /etc/authbind.
Firstly, /etc/authbind/byport/port is tested.  If this file is accessible for execution to the calling user, according to access(2), then binding to the port is authorised.  If the file can  be  seen not to exist (the existence check returns ENOENT) then further tests will be used to find authorisation; otherwise, binding is not authorised, and the bind call will return with the errno value  from the access(2) call, usually EACCES (Permission denied).
Secondly, if that test fails to resolve the matter, /etc/authbind/byaddr/addr:port is tested, in the same manner as above.
Thirdly, if the question is still unresolved, the file /etc/authbind/byuid/uid will  be  opened  and read.   If  the  file  does  not exist then the binding is not authorised and bind will return EPERM (Operation not permitted, or Not owner).  If the file does exist it will be searched for a  line  of the form addr/length:min-port,max-port matching the request (ie, the initial length bits of addr match those in the proposed bind call, and the proposed port number lies is in the inclusive range specified.  If such a line is found then the binding  is authorised.  Otherwise it is not, and bind will fail with ENOENT (No such file or directory).
In our case, the tomcat6 package creates a small file in /etc/authbind/byuid named with the UID of the tomcat6 user, and containing the line:
0.0.0.0/32:1,1023
This allows the tomcat6 user to bind to any IP address with any low-numbered port, TCP or UDP.
So why does an attempt to start after having modified /etc/tomcat6/server.xml to start on TCP/80 fail with an error like the following?
SEVERE: Error starting endpoint
 java.net.BindException: Permission denied <null>:80
One piece is still missing!  Authbind is not enabled by default.  You will find the following section in the /etc/default/tomcat6 file:
# If you run Tomcat on port numbers that are all higher than 1023, then you
# do not need authbind.  It is used for binding Tomcat to lower port numbers.
# NOTE: authbind works only with IPv4.  Do not enable it when using IPv6.
# (yes/no, default: no)
#AUTHBIND=no
Uncomment the last line, and change ‘no’ to ‘yes’ and tomcat6 will start as you expect!  Kudos to the package maintainers for being security conscious, although I’ll admit finding all of the clues to make this work was a bit of a chore.



**********************************************************************************

http://www.ex-parrot.com/pete/tomcat-vhost.html

Virtual Hosting with Tomcat

For Conf -> I did the changes in /etc/tomcat6/

This is a guide on setting up Tomcat to do virtual hosting and make it behave like a simple webserver with jsp and servlet support, for many different sites all hosted on the same IP address. The aim is to have a single directory for each virtual host, which can be manipulated individually without hassles from managing multiple .war files and other configuration difficulties.
To configure Tomcat for a virtual host, you need a <Host ..> directive in the server.xml file, and a ROOT.xml file in the conf/Catalina/$host directory. Here's the minimal setup required for a copy of Tomcat serving directly on Port 80, using no connectors or other configuration difficulties.
This was written for Tomcat 5 on linux, with Tomcat installed in /usr/local/tomcat
We start with the simplest configuration, of one website, called 'localhost' which keeps it's files in /usr/local/tomcat/webapps/localhost . We're not using any .war files here - all the files are placed straight into the directory.


conf/server.xml

<Server port="8005" shutdown="SHUTDOWN" debug="0"> <!-- Define the Tomcat Stand-Alone Service --> <Service name="Catalina"> <!-- Define a non-SSL Coyote HTTP/1.1 Connector on port 8080 --> <Connector port="80" maxThreads="150" minSpareThreads="25" maxSpareThreads="75" enableLookups="false" redirectPort="8443" acceptCount="100" debug="0" connectionTimeout="20000" disableUploadTimeout="true" /> <Engine name="Catalina" defaultHost="localhost" debug="0"> <!-- Define the default virtual host Note: XML Schema validation will not work with Xerces 2.2. --> <Host name="localhost" debug="0" appBase="webapps/localhost" unpackWARs="true" autoDeploy="true" xmlValidation="false" xmlNamespaceAware="false"> <Logger className="org.apache.catalina.logger.FileLogger" directory="logs" prefix="localhost_log." suffix=".txt" timestamp="true"/> </Host> <!-- VIRTUAL HOST INJECTION POINT --> </Engine> </Service> </Server>

conf/Catalina/localhost/ROOT.xml

<?xml version='1.0' encoding='utf-8'?> <Context displayName="localhost" docBase="" path="" workDir="work/Catalina/localhost/_"> </Context>

webapps/localhost

index.jsp WEB-INF/web.xml

webapps/localhost/WEB-INF/web.xml

<?xml version="1.0" encoding="ISO-8859-1"?> <web-app> </web-app>

Adding a virtual host to this config.

From here, to add a virtual host $host with an alias of $alias, the following steps are required.
  • Shut down tomcat.
  • Add a Host entry to the server.xml file at the VIRTUAL HOST INJECTION POINT
    <Host name="$host" debug="0" appBase="webapps/$host"
            unpackWARs="true" autoDeploy="true"
        xmlValidation="false" xmlNamespaceAware="false">
    
    <Logger className="org.apache.catalina.logger.FileLogger"
    directory="logs"  prefix="$host\_log." suffix=".txt" timestamp="true"/>
    
    <Alias>$alias</Alias>
    </Host>
  • Add a configuration file for the host
    mkdir conf/Catalina/$host
    
    cat >conf/Catalina/$host/ROOT.xml
    <?xml version='1.0' encoding='utf-8'?>
    <Context displayName="$host" docBase="" path=""
    workDir="work/Catalina/$host/_">
    </Context>
    ^D
    
  • Add a skeleton directory structure for the files
    mkdir $tomcatdir/webapps/$host
    mkdir $tomcatdir/webapps/$host/WEB-INF
    mkdir $tomcatdir/webapps/$host/WEB-INF/classes
    mkdir $tomcatdir/webapps/$host/WEB-INF/lib
  • Add a minimal web.xml file
    cat >webapps/$host/WEB-INF/web.xml
    <?xml version="1.0" encoding="ISO-8859-1"?>
    <web-app>
    </web-app>
  • Add a trivial holding page
    cat >index.jsp
    <html>
    <head>
    <title>Not yet configured</title>
    </head>
    
    <body>
    <p>This virtual server $host is not yet configured.</p>
    </body>
    </html>
  • Start tomcat back up again.

Automating the process

For a standard situation with tomcat installed in /usr/local/tomcat, here's a small perl script that does all this for you. Save it into the /usr/local/tomcat/bin directory. add_virtual_host.pl

Usage

./bin/add_virtual_host.pl host alias1 alias2 alias3

Monday, February 28, 2011

Setting Time Zone in Linux (Ubuntu)

Setting Time Zone in Linux (Ubuntu)
dpkg-reconfigure tzdata

Setting the time zone in Amazon Linux

https://forums.aws.amazon.com/thread.jspa?messageID=210296&tstart=0#210296

cp /usr/share/zoneinfo/Asia/Calcutta /etc/localtime

Where Asia/Calcutta can be replaced...  




I have not tried the below:

That succeeded, and should be more robust against daylight savings changes than the other options:

also had to unset my TZ="PST" env var before the change was apparent.





Incase if your server is not in sync with actual time, please run the following command:

sudo ntpdate pool.ntp.org

Thursday, January 27, 2011

Virtual Hosting with Tomcat

Virtual hosting with Tomcat:

http://www.ex-parrot.com/pete/tomcat-vhost.html


Virtual Hosting with Tomcat

This is a guide on setting up Tomcat to do virtual hosting and make it behave like a simple webserver with jsp and servlet support, for many different sites all hosted on the same IP address. The aim is to have a single directory for each virtual host, which can be manipulated individually without hassles from managing multiple .war files and other configuration difficulties.
To configure Tomcat for a virtual host, you need a <Host ..> directive in the server.xml file, and a ROOT.xml file in the conf/Catalina/$host directory. Here's the minimal setup required for a copy of Tomcat serving directly on Port 80, using no connectors or other configuration difficulties.
This was written for Tomcat 5 on linux, with Tomcat installed in /usr/local/tomcat
We start with the simplest configuration, of one website, called 'localhost' which keeps it's files in /usr/local/tomcat/webapps/localhost . We're not using any .war files here - all the files are placed straight into the directory.

conf/server.xml

<Server port="8005" shutdown="SHUTDOWN" debug="0">

  <!-- Define the Tomcat Stand-Alone Service -->
  <Service name="Catalina">
    
        <!-- Define a non-SSL Coyote HTTP/1.1 Connector on port 8080 -->
    <Connector port="80"
               maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
               enableLookups="false" redirectPort="8443" acceptCount="100"
               debug="0" connectionTimeout="20000" 
               disableUploadTimeout="true" />

                <Engine name="Catalina" defaultHost="localhost" debug="0">

   
      <!-- Define the default virtual host
           Note: XML Schema validation will not work with Xerces 2.2.
       -->
      <Host name="localhost" debug="0" appBase="webapps/localhost"
       unpackWARs="true" autoDeploy="true"
       xmlValidation="false" xmlNamespaceAware="false">
        <Logger className="org.apache.catalina.logger.FileLogger"
                 directory="logs"  prefix="localhost_log." suffix=".txt"
            timestamp="true"/>
      </Host>

         <!-- VIRTUAL HOST INJECTION POINT -->
          
    </Engine>

  </Service>

</Server>

conf/Catalina/localhost/ROOT.xml

<?xml version='1.0' encoding='utf-8'?>
<Context displayName="localhost" docBase="" path=""
workDir="work/Catalina/localhost/_">
</Context>

webapps/localhost

index.jsp
WEB-INF/web.xml

webapps/localhost/WEB-INF/web.xml

<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app>
</web-app>

Adding a virtual host to this config.

From here, to add a virtual host $host with an alias of $alias, the following steps are required.
  • Shut down tomcat.
  • Add a Host entry to the server.xml file at the VIRTUAL HOST INJECTION POINT
    <Host name="$host" debug="0" appBase="webapps/$host"
            unpackWARs="true" autoDeploy="true"
        xmlValidation="false" xmlNamespaceAware="false">
    
    <Logger className="org.apache.catalina.logger.FileLogger"
    directory="logs"  prefix="$host\_log." suffix=".txt" timestamp="true"/>
    
    <Alias>$alias</Alias>
    </Host>
  • Add a configuration file for the host
    mkdir conf/Catalina/$host
    
    cat >conf/Catalina/$host/ROOT.xml
    <?xml version='1.0' encoding='utf-8'?>
    <Context displayName="$host" docBase="" path=""
    workDir="work/Catalina/$host/_">
    </Context>
    ^D
    
  • Add a skeleton directory structure for the files
    mkdir $tomcatdir/webapps/$host
    mkdir $tomcatdir/webapps/$host/WEB-INF
    mkdir $tomcatdir/webapps/$host/WEB-INF/classes
    mkdir $tomcatdir/webapps/$host/WEB-INF/lib
  • Add a minimal web.xml file
    cat >webapps/$host/WEB-INF/web.xml
    <?xml version="1.0" encoding="ISO-8859-1"?>
    <web-app>
    </web-app>
  • Add a trivial holding page
    cat >index.jsp
    <html>
    <head>
    <title>Not yet configured</title>
    </head>
    
    <body>
    <p>This virtual server $host is not yet configured.</p>
    </body>
    </html>
  • Start tomcat back up again.

Automating the process

For a standard situation with tomcat installed in /usr/local/tomcat, here's a small perl script that does all this for you. Save it into the /usr/local/tomcat/bin directory. add_virtual_host.pl

Usage

./bin/add_virtual_host.pl host alias1 alias2 alias3

Tomcat 6 binding on port 80 (instead of default port 8080)on ubuntu

Tomcat 6 binding on port 80 (instead of default port 8080)on ubuntu

http://thelowedown.wordpress.com/2010/08/17/tomcat-6-binding-to-a-privileged-port-on-debianubuntu/

Makes ure: Authbind = yes in /etc/defaults/tomcat6.
in /var/lib/tomcat6/conf /server.xml - change the port to 80 from 8080

Tuesday, January 25, 2011

Tuesday, January 18, 2011

MySQL cluster tips

MySQL cluster- careful:

1. when you start the client with ndbd --initial - it clears all tables, as it starts fresh, so to start client node use "ndbd"
2. in case u need to start ndbd --initial, dont start both nodes at once. wait for one node to start completly, so that the other node will reflect the data.

Monday, January 3, 2011

High Availablility storage cluster with GlusterFS

http://www.howtoforge.com/high-availability-storage-with-glusterfs-on-ubuntu-10.04-automatic-file-replication-mirror-across-two-storage-servers

In the above link some items are missing. Find below the extra steps to be done:

on client system one need to install Fuse, as GlusterFS is using FUSE as underlying File system. To install FUSE:

  • apt-get install build-essential
  • apt-get install libfuse-dev
  • apt-get install fuse-utils
  • apt-get install libcurl4-openssl-dev
  • apt-get install libxml2-dev
  • apt-get install mime-support
 On GlusterFS server side use the below command instead of shown in the link above:

[root@@server]# glusterfsd -f /tmp/glusterfsd.vol
[root@@client]# glusterfs -f /tmp/glusterfs.vol /mnt/glusterfs
NOTE: use correct path for Glusterfs.vol

http://www.gluster.com/community/documentation/index.php/User_Guide

GlusterFS Server and Client on same machine:

one can have glusterfs server and client on same machine. only thing to do is having two separate glusterfs.vol for server and client (say glusterfs_srv.vol and glusterfs_cl.vol) under /etc/glusterfs/ folder.