Posts Tagged ‘memcached’

Enabling Memcached on BlueHost

Thursday, March 11th, 2010

Bluehost is a pretty good web hosting company, cheap and leaving you in total control of your website.
That is if you know your way around.
I highly recommend them for dirt cheap hosting yet access to powerful features.

A common complaint against any cheap hosting is the lack of memcached installation by default.
It really doesn’t make sense even business-wise for them because memcached would allow to use less CPU/Memory per site hosted, meaning less total servers, meaning less power drawn, meaning big savings.
The hosting plans charging usually a fixed amount per month or year, reducing CPU/Memory/Servers would really make sense as the savings would be on directly on them.

Anyway, now we need to setup memcached ourselves on a machine we don’t have root access to.
No big deal, we will just have to compile everything ourselves.

Here’s how.

1/ Create an installation folder.

As we cannot write to /usr/, we cannot install any software system wide, so everything will remain local in our home folder.
Let’s call it ~/install/
It will be where you install all your compiled software, acting like the /usr/ folder we don’t have access to.

bash# mkdir ~/install

2/ Compile libevent

Why libevent? Because we will need it later to compile memcached.

bash# wget http://www.monkey.org/~provos/libevent-1.4.13-stable.tar.gz
bash# tar zxf libevent-1.4.13-stable.tar.gz
bash# cd libevent-1.4.13-stable
bash# ./configure -prefix=~/install/
bash# make
bash# make install

libevent is now neatly installed in our home directory at ~/install/

3/ Compile memcached

bash# wget http://memcached.googlecode.com/files/memcached-1.4.4.tar.gz
bash# tar xzf memcached-1.4.4.tar.gz
bash# cd memcached-1.4.4
bash# ./configure --with-libevent=~/install/ --prefix=~/install/
bash# make
bash# make install

Neat, now we have memcached installed in ~/install/bin/, so let’s try to run it.

bash# cd ~/install/bin
bash# ./memcached
failed to set rlimit for open files. Try running as root or requesting smaller maxconns value.
bash# ./memcached -c 100
^CSIGINT handled
bash#

So we hit a limitation: as we’re not root, our memcached server is limited to only 100 connections instead of the default 1024.
Maybe we can live with that.

Now the server part is done, we need to be able to use the PHP commands to communicate with memcached.
This is found in the PEAR packages.

4/ Install the PHP memcache client

(the memcache client doesn’t have the ‘d’ at the end)

bash# pecl install memcache
downloading memcache-2.2.5.tgz ...
Starting to download memcache-2.2.5.tgz (35,981 bytes)
..........done: 35,981 bytes
11 source files, building
running: phpize
Configuring for:
PHP Api Version:         20041225
Zend Module Api No:      20060613
Zend Extension Api No:   220060519
Enable memcache session handler support? [yes] :
Notice: Use of undefined constant STDIN - assumed 'STDIN' in CLI.php on line 304

Warning: fgets(): supplied argument is not a valid stream resource in CLI.php on line 304

Warning: fgets(): supplied argument is not a valid stream resource in /root/lib/php/PEAR/Frontend/CLI.php on line 304
building in /usr/local/src/pear/pear-build-bottoml8/memcache-2.2.5
ERROR: could not create build dir: /usr/local/src/pear/pear-build-bottoml8/memcache-2.2.5

Ouch, we’re not root so pecl fails.
No big deal: pecl already downloaded memcache-2.2.5.tgz for us, let’s compile it manually.

bash# tar xzf memcache-2.2.5.tgz
bash# cd memcache-2.2.5
bash# phpize
bash# ./configure --prefix=~/install/
bash# make
bash# make test
bash# make install
Installing shared extensions:     /usr/lib/php/extensions/no-debug-non-zts-20060613/
cp: cannot create regular file `/usr/lib/php/extensions/no-debug-non-zts-20060613/#INST@7000#': Read-only file system
make: *** [install-modules] Error 1
bash#
bash# ls modules/
./  ../  memcache.so*
bash# cp memcache.so ~/install/lib/

Ok, make install failed, but we have already what we need: memcache.so in the memcache-2.2.5/modules/ directory.
It’s the extension we need to load in PHP (in the php.ini file) so we can use the PHP commands to access memcached.
We just moved it to ~/install/lib/ so it’s comfy with the other libraries we compiled earlier (libevent and memcached).

5/ Load the memcache module for PHP

Now for the tricky part:
As we don’t have root access, we cannot just change the server’s php.ini file to include our freshly brewed memcache.so and restart Apache to make the changes taken into account.

But Bluehost provides us with a neat way to use our own php.ini file.
By default this option is disabled.
In their cPanel it’s listed under the category ‘Software / Services’ as ‘PHP Config’.

By default the setting is Standard PHP5, we need to change that to the third option: PHP5 (FastCGI)
It will directly load the php.ini it finds in your ~/public_html/ directory.

On this same page of the cPanel, there’s an interesting button “INSTALL PHP.INI MASTER FILE”.
Just click it and magically the server’s php.ini file is copied in ~/public_html/php.ini.default

It’s now our base php.ini we’ll use to load memcache. Let’s edit it.

bash# cd ~/public_html/
bash# mv php.ini.default php.ini
bash# vi php.ini

In php.ini, find the line that reads:
extension_dir = “/usr/local/lib/php/extensions/no-debug-non-zts-20060613″
and replace it with
extension_dir = “/home/your_bluehost_user_name/install/lib”
so that we have access to our memcache.so lib.
At the end if php.ini, before the last line that loads the Zend Optimizer, include this line to load memcache.so:
extension=memcache.so

There should be some other libraries being loaded here:
;extension=pdo.so
;extension=pdo_sqlite.so
;extension=sqlite.so
;extension=pdo_mysql.so

As we changed the extension_dir path these libraries are not accessible anymore so you can just comment them.
extension_dir does not support listing multiple directories so I wish we didn’t have to do that.
At some point there was a proposal to make it happen, don’t know if it’s implemented yet, I couldn’t find anything about it.
Worst case scenario if you need PDO (and you probably do), you just need to compile it and install it in ~/install/lib/

Now that php.ini is modified, the server will pick up the configuration automagically.
But you may have to wait a little bit as php.ini will not be read again right away, you have to wait for the next poll, a few minutes.

6/ Let’s use memcached!

Now everything’s installed, let’s finally use memcached.

Create a memcache.php test file that will connect to memcached and just return the server version:

<?php
$memcache = new Memcache;
$memcache->addServer('localhost', 11211);
$memcache->connect('localhost', 11211);

echo $memcache->getVersion();
?>

Start the memcached in verbose mode so we can see our connection arriving:

bash# cd ~/install/bin/
bash#  ./memcached -h
memcached 1.4.4
-p <num>      TCP port number to listen on (default: 11211)
-U <num>      UDP port number to listen on (default: 11211, 0 is off)
-s <file>     UNIX socket path to listen on (disables network support)
-a <mask>     access mask for UNIX socket, in octal (default: 0700)
-l <ip_addr>  interface to listen on (default: INADDR_ANY, all addresses)
-d            run as a daemon
-r            maximize core file limit
-u <username> assume identity of <username> (only when run as root)
-m <num>      max memory to use for items in megabytes (default: 64 MB)
-M            return error on memory exhausted (rather than removing items)
-c <num>      max simultaneous connections (default: 1024)
-k            lock down all paged memory.  Note that there is a
              limit on how much memory you may lock.  Trying to
              allocate more than that would fail, so be sure you
              set the limit correctly for the user you started
              the daemon with (not for -u <username> user;
              under sh this is done with 'ulimit -S -l NUM_KB').
-v            verbose (print errors/warnings while in event loop)
-vv           very verbose (also print client commands/reponses)
-vvv          extremely verbose (also print internal state transitions)
-h            print this help and exit
-i            print memcached and libevent license
-P <file>     save PID in <file>, only used with -d option
-f <factor>   chunk size growth factor (default: 1.25)
-n <bytes>    minimum space allocated for key+value+flags (default: 48)
-L            Try to use large memory pages (if available). Increasing
              the memory page size could reduce the number of TLB misses
              and improve the performance. In order to get large pages
              from the OS, memcached will allocate the total item-cache
              in one large chunk.
-D <char>     Use <char> as the delimiter between key prefixes and IDs.
              This is used for per-prefix stats reporting. The default is
              ":" (colon). If this option is specified, stats collection
              is turned on automatically; if not, then it may be turned on
              by sending the "stats detail on" command to the server.
-t <num>      number of threads to use (default: 4)
-R            Maximum number of requests per event, limits the number of
              requests process for a given connection to prevent
              starvation (default: 20)
-C            Disable use of CAS
-b            Set the backlog queue limit (default: 1024)
-B            Binding protocol - one of ascii, binary, or auto (default)
-I            Override the size of each slab page. Adjusts max item size
              (default: 1mb, min: 1k, max: 128m)
bash#
bash# ./memcached -c 100 -vvv -l 127.0.0.1
slab class   1: chunk size        96 perslab   10922
slab class   2: chunk size       120 perslab    8738
slab class   3: chunk size       152 perslab    6898
slab class   4: chunk size       192 perslab    5461
...
slab class  40: chunk size    616944 perslab       1
slab class  41: chunk size    771184 perslab       1
slab class  42: chunk size   1048576 perslab       1
<26 server listening (auto-negotiate)
<27 send buffer was 124928, now 268435456
<27 server listening (udp)

---- Access the memcache.php file here in your browser ----

<28 new auto-negotiating client connection
<29 new auto-negotiating client connection
29: going from conn_new_cmd to conn_waiting
29: going from conn_waiting to conn_read
29: going from conn_read to conn_parse_cmd
29: Client using the ascii protocol
<29 version
>29 VERSION 1.4.4
29: going from conn_parse_cmd to conn_write
29: going from conn_write to conn_new_cmd
29: going from conn_new_cmd to conn_waiting
29: going from conn_waiting to conn_read
28: going from conn_new_cmd to conn_waiting
28: going from conn_waiting to conn_read
28: going from conn_read to conn_closing
<28 connection closed.

Tada! And magically the browser is displaying a magnificent “1.4.4“.

You got yourself a memcached server running on BlueHost.

Did you find this post useful? Buy this man a beer:

Pushing MySQL updates to Memcache using SQL triggers

Tuesday, August 19th, 2008

How to make MySQL trigger cache updates after an INSERT or UPDATE on the DB.

CREATE TRIGGER tr1 AFTER INSERT ON table1 FOR EACH ROW DO memcache_set("localhost:11211",NEW.name,NEW.value);
CREATE TRIGGER tr1 AFTER DELETE ON table1 FOR EACH ROW DO memcache_delete("localhost:11211",OLD.name);

Source

Some interesting Memcached readings

Friday, July 25th, 2008

Memcached internals

Memcached vs APC

Memcached as simple message queue

Use Gearman to avoid cache miss stampedes

Scaling memcached at Facebook