Recently, I needed more computing power than my dev server could provide. So, I decided to get a nice big server from the folks over at DigitalOcean. DigitalOcean is a cloud hosting provider that is geared toward developers. If you’re not comfortable with command line and nerdy stuff, this isn’t for you. However, if you are cool with command line, then you’ll definitely like DigitalOcean. Oh, and they’re very reasonably priced.
I should also add that I’m not connecting this to a domain. I need this server to run some complex code that requires a fair amount of computing power and DigitalOcean offers power by the hour (hell yeah rhymes).
So, here’s a basic outline of what I needed to do:
- Create a LAMP droplet (instance)
- ssh in to the server
- install vsftpd and configure (FTP server)
- create a user (so I can get in with FTP)
- copy a huge db table and get it into a database
- run some code
- get the results (a table) and download it
This is a tall order for a lot of folks, but I’m going to break this down into the steps I took. Note: I’m not a sysadmin so if any of this seems incorrect, please let me know in the comments.
Create A Droplet
First, we need to create a droplet. This is…well…brainless.
- Click “Create Droplet”
- Enter a hostname. If you’re doing this for a real site, enter the real domain name. Otherwise, just enter whatever (I used a random domain name I owned).
- Select a droplet size. I’m using the smallest size for this tutorial. In the real thing, I used a very large droplet.
- The region is preselected, but if you need a different one, choose it.
- Select a Linux distribution (I’m going with the default selection of Ubuntu 12.04 x32).
- Click the Applications tab and choose LAMP on Ubuntu 12.04.
- Click Create Droplet and wait for the root password to be emailed to you.
SSH In And Start Doing Things
Ok, here’s where the tutorial really begins. First, let’s open up Terminal (if you’re using Windoze, just…) and enter the following:
When prompted, enter the password you were sent in the email.
Install And Configure vsftpd
vsftpd is an FTP program that is good enough to get the job done. I’m going to do a very basic install and config so I can access the server via FTP.
- Download and install vsftpd with the following command:
sudo apt-get install vsftpd
- After you see all the lines of jibberish, you’ll need to config vsftpd. Enter the following into Terminal:
nano /etc/vsftpd.confThis will open the vsftpd config file in a text editor.
- First, let’s remove anonymous FTP. Find
anonymous_enable=YESby scrolling with the arrow keys and change it to
- Now locate
#write_enable=YESand remove the #
- Next locate
#local_enable=YESand remove the #
- Finally, press control + x then the y key and return key to save.
Now that we have changed the config file, we need to restart vsftpd so the changes take effect. Enter
sudo service vsftpd restart into terminal. It will give you a few lines of jibberish, then you’re ready to move on. One more note: you will need to use active FTP mode. If you want to set up passive mode, read about it here.
Add A User
In order to access anything via FTP, we need to have a user and we need to assign them to a folder.
- Enter the following into the terminal:
sudo useradd -d /home/mattonomics -m mattonomicsand press enter. This adds the user and sets the home folder to /home/mattonomics. For the love of God, think of your own username.
- Now, let’s give the user a password with
sudo passwd mattonomicsthen enter. Terminal will prompt you to enter a password, then re-enter it.
Make Some Changes In Apache
We could have set our new user’s home directory to /var/www (that’s the Apache DocumentRoot by default) but that’s not a best practice and c’mon. It’s not that fun. So, let’s do the right thing and create a safer environment by doing this the right way.
- We need to create a public_html directory for our user. Type the following into the terminal and hit enter:
- Now, let’s make a configuration file by copying the default. Use the following in the terminal:
cp /etc/apache2/sites-available/default /etc/apache2/sites-available/mattonomics
- We need to edit the newly created config file by entering the following into terminal:
- Towards the very top of the file, you’ll see
DocumentRoot /var/wwwChange that to
- Scroll down a few lines and find
<Directory /var/www>and change that to
- Press control + x, then y and finally enter (this saves the file, as before)
- We’re getting closer We need to tell apache to pay attention to this new config file. So, let’s type the following into terminal:
sudo a2dissite default && sudo a2ensite mattonomics
- Lastly, we need to restart Apache. There are a few ways to do this, but let’s go by the book with the following:
sudo /etc/init.d/apache2 restart
Alright, we’re making progress. But, we need to check and make sure that Apache behaved as we want it to. Let’s make an index file so we can check it. Type this into the terminal
echo '<strong>Hello, Mother Fucking World!</strong>' > /home/mattonomics/public_html/index.html
Now, visit your site in a browser (mine was http://123.456.7.89/) and you should see the pleasant message you just typed to yourself.
Do Stuff With MySQL
I had a handful of things to do with MySQL, but before we get started, we have to give a password to the root user. The default password is…”password”. We definitely want to change that bs.
Enter the following in terminal:
mysqladmin -u root -p'password' password yournewpassword
Now, let’s check to make sure it worked by the following:
mysql -u root -p You’ll be asked for your password. Enter it and press enter. It will give you a readout then display
mysql> If you see that, you’ve done it right.
Create A Database
Since I needed to import some gigantic tables, I first needed to create a database in which to import them. Assuming you still see
mysql>, type the following command in the terminal
create database mattonomics; and hit enter. Call your database something original.
Now let’s leave MySQL for a minute by entering
\q and pressing enter.
Import A Database
I had a bunch of tables to import that were on my computer, so I needed to get them in to the newly formed MySQL database. But there’s something we need to do:
- First, we need to copy the actual file from local to remote. Open a new terminal tab (command + t) and type the following:
scp /path/to/local/file.zip firstname.lastname@example.org:/home/mattonomicsEnter your password for root when prompted.
- Once the file has loaded, go back over to your remote server terminal tab. Chances are, you don’t have the ability to zip/unzip a file, so we’ll need to install them before moving on. Install zip by entering this into the terminal:
sudo apt-get install zipInstall unzip by entering this into the terminal:
sudo apt-get install unzip
- Now let’s unzip the file with this command:
unzip /home/mattonomics/file.zip -d /home/mattonomicsAgain, this may take just a second or many minutes. It really depends on the size of the file.
- Before moving on, let’s double check that the file extracted properly. Enter
stat /home/mattonomics/file.sqlYou’re looking for the size entry. If it’s zero, you’ll need to do the unzip again and overwrite it. This happened to me twice, so it’s definitely possible.
Now that the file is on the server and extracted (mine is a *.sql file) we need to get it into the database. There are a ton of ways to do this. Here’s what I did:
mysql -u root -p then enter your password. Now, you can use MySQL commands. Switch to the database you want with
use mattonomics;, where “mattonomics” is the name of your database. Now, import the tables with
Depending on the size of what you’re importing this could take two seconds or a very long time. If you know a better (that means FASTER) way of doing this, please leave it in the comments. Also, note that when writing MySQL you MUST use “;” at the end of each statement or it will fail.
When we created the public_html directory, we created it under the root user. That means if you log in to FTP with the mattonomics account, you can’t do jack. So we need to change the ownership of that directory and everything in it. Enter this into the terminal:
chown -R mattonomics:mattonomics /home/mattonomics/public_html
Now there’s an additional problem and that is that the user root directory is writeable. vsftpd, the FTP daemon we installed earlier, doesn’t allow users to log in if their root folder is writeable. We could have changed some settings in the vsftpd.conf file, but instead we’ll just change the permissions with
chmod a-w /home/mattonomics.
FTP In And Get It Done
The script I needed to run wasn’t very long, but it did require a bit of config and I needed a basic interaction on the front end (hence the FTP). Set up the ftp with the user/password created above and put the remote path as /public_html.
Now, you can load all the things you need and start configuring your script and all that stuff. The end result of my script was a database table with under 100k rows (so not that big, really) and just a bit of data per row.
Ok, Now What?
The last thing I needed to do was make a copy of the table I created, zip it up and download it.
- To make the *.sql file use
mysqldump -u root -p'thepassword' mattonomics table_name > /home/mattonomics/public_html/table.sqlThis could take anywhere from one second to a while, so be patient (again).
- Now, zip up the file with
zip /home/mattonomics/public_html/table.zip /home/mattonomics/public_html/table.sql
And now you can visit http://123.456.7.89/table.zip and download the file (if that’s how you want to do it).
If you know any shortcuts or have a better way of doing things, please let me know in the comments.