I recall at one point that Madison Ave. had given up on trying to reach Generation X.

But it didn’t really give up - it adapted. Sort of.

But I (and my generation) adapted too. Sort of.

And I buy things I think I need from companies wth whom [1] I might have ethical, moral or other reasons not to do business.

I know these companies lie to me and to others about the quality and efficacy of their product,

about how waste byproducts of the production and distribution of these goods affects our country, our world, and us as a species.

The government committes tasked to regulate these companies and the safety of those goods are also known to lie to the public.

“Dubitat emptor” - skeptical buyer

Generation X knew the tricks that advertisers used to influence purchasing decisions.

We know all know why the milk and eggs are in the back of the grocery store.

Despite how corporations treat us we continue to relate to them. And knowing all that we know about Exxon, and BP, and Verizon, and McDonalds, Frito Lay or almost any company… About the conditions and treatment of the workers who make our clothing.

We know and yet we buy.

Because at the end of the day we need stuff.

And we can only pick from what’s in the store[2].

Because that’s how this system works.

–

[1] using a personal pronoun with companies to indicate them being treated as if the are people.

[2] lately more options to have things made for us or to make them for ourselves are available!

It shows how strong media can be in affecting people’s beliefs. Since then, media has only gotten better, more pervasive, more persuasive, more trusted, more ubiquitous. The War of the Worlds incident showed that media has to be careful to not abuse the power of the medium.

And people like stories. We get so fascinated by them . We believe them – we want to believe them. They pull us out of this “real” world to a place that is in any number of ways better.

I guess in small ways people do this all the time on a smaller personal level. We are each like little media outlets - “broadcasting” out a message that we kinda know or hope will get interpreted in a certain way. Sometimes we do this to help the communication, and sometimes to mislead or create a false impression.

This reminds me of the Seinfeld where George realizes that if he acts angry at work then his coworkers will think he’s working really hard and too busy to be bothered with new stuff.

• Git for Computer Scientists In simplified form, git object storage is “just” a Directed Acyclic Graph of objects … So, armed with that knowledge on how git stores the version history, how do we visualize things like merges?
• Vim Find and replace examples Example 11. Alter sequence number in a numbered list while inserting a new item. This can be achieved by the following vim substitution command. :4,$s/\d\+/\=submatch(0) + 1/
• Readline shortcuts Transpose

   keys    Action          Scope                   Direction/Place
   Ctrl-t  Transpose/drag  char. before the cursor ↷ over the character at the cursor
   Alt-t   Transpose/drag  word before the cursor  ↷ over the word at/after the cursor

• iTerm2 keybindings Readline also supports jumping between words, that is re-positioning the cursor before or after the current/next word. The default binding for this is ESC-f for forward jumping and ESC-b for backward jumping.
• VimGolf Real Vim ninjas count every keystroke - do you? (Danger! Quicksand.)

I’ve also started playing vim golf so I might not post here for a while :)

All the stuff for the “A” side will be “-a” and “B” “-b”

cp /etc/my.cnf /etc/my-a.cnf /etc/my-b.cnf 
mkdir /var/lib/mysql-a
mkdir /var/lib/mysql-b

Then edit your cnf files for each side.

# Note this is not meant to be pasted in as you whole conf file! These are the params you need to find and change.
user            = mysql
socket          = /var/run/mysqld/mysqld-a.sock
port            = 3310
basedir         = /usr
datadir         = /var/lib/mysql-a
tmpdir          = /tmp/mysql-a

Before we get into setting up the actual replication lets that we can run these 2 mysqlds at the same time. Try to start one side.

mysqld --defaults-file=/etc/mysql/my-a-side.cnf &

Apparmor! Arg. Add this for a and b sides:

/var/lib/mysql-a/ r,
/var/lib/mysql-a/** rwk,
/var/run/mysqld/mysqld-a.pid w,
/var/run/mysqld/mysqld-a.sock w,

Try to start up both. Wrap that exec up into a script and make one for the b side too.

./start-a-sides.sh && ./start-b-sides.sh

and to shut them down

mysqladmin -u root -P 3310 -h 127.0.0.1 shutdown

We’ll need to setups the datadirs - or if you have a data to start with you can put it in the data dir. Or else something like this for each of your $DATADIRs.

mysql_install_db --user=mysql --datadir=$DATADIR 

We actually want apc.stat = 1.

There were all sorts of fear and doubts about doing this. It seemed the exact opposite of what you should do on a large site. We feared that all that all those extra stat calls would slow things down or even cause the site to crash. But rather than rely on our intuition about this we decided to actually try it out and see what actually happens.

First we confirmed that apc.stat (and apc.stat_ctime) would behave the way we expected when we dropped new code on the server. Relatively easy and worked as expected. Next we needed to figure out how big our cache should be and if we wanted a ttl or not. In order to do that I needed a bit of a deeper understanding of how APC operates.

Its pretty well documented that when APC fills up, it dumps the whole cache and starts over. Generally you want to avoid this - for similar reasons as to why we wanted to get away from doing graceful restarts. What was less well documented was how apc.ttl really works. Most of what I read encouraged apc.ttl = 0 to avoid fragmentation.

If you have apc.ttl = 0 apc will not purge anything from your cache (unless the cache fills up, at which point APC will purge everything from the cache and start over). It may not be obvious but with apc.stat = 1 and apc.ttl = 0, that means that APC will hold onto older versions of files that have changed. If you’ve got lots of frequent changes, this could quickly lead to your cache filling up with old versions of your opcodes.

This behavior changes when setting apc.ttl to something larger than zero. APC will not do any cleanup unless it needs to. So if you set your ttl to 3600 that does not mean that apc will automatically dump all files older than an hour. APC will remove items older that ttl…

• if it needs the space for other new items.
• if a newer version of the same file gets inserted and an old version is older than the ttl.

For example, your config.php is cached and you make a change and deploy a new config.php. APC will cache the new version of the file and it will also hold onto the previous version of the file. However, if apc.ttl is > 0, apc will opportunistically check to see if it can clear out any of those old versions of that file. This helps keep cache size in check but it comes at a cost, namely fragmentation.

As old items are purged from the cache you start to get fragmentation. If fragmentation gets too high (more than 50%) you’re likely to start to see some performance impact. One way to keep fragmentation in check is to have a large enough cache. Some of the research I found suggests that a cache size about twice as large as the size of your files will go a long way towards keeping fragmentation low. I’m not sure but it sounds like APC might do some relocation of items in the cache which goes easier when it has large empty blocks to use for reshuffling.

Armed with this understanding I took a look at our historical graphs of APC (you do have graphs for everything, right?) to see how big the cache should be. Over the weekends, when there generally aren’t any deploys, the cache grows to just under 200M. Our cache was sized at 384M to start. I wanted to keep the cache comfortably large so I started ours at 500M.

To get a ttl to start with I looked at our deploy frequency, which files change most frequently and how many files change per deploy and went with 3600.

Here are a set of articles with details about apc:

• How Does APC Work
• Understanding APC
• LAMP php opcodes cache Part 1
• LAMP php opcodes cache Part 2
• Monitoring PHP APC cache usage

Here are a set of articles with details about apc:

• How Does APC Work
• Understanding APC
• LAMP php opcodes cache Part 1
• LAMP php opcodes cache Part 2
• Monitoring PHP APC cache usage

On a pool of 75, 3 soon started showing increased fragmentation. Turns out those 3 servers didn’t have our app loaded on the ram disk (tmpfs) but were reading from the disk.

• Design Patterns

• My favourite interview question

I basically followed the quick-start guide from OpsCode.

I cloned the OpsCode repo skeleton and its sitting in /home/jkolber/chef-repo. All the settings are in .chef directory inside of that dir: /home/jkolber/chef-repo/.chef

I found a free book lookup API at Open Library and soon had a script that would collect the information about our book list from this site. Of the 120 ISBNs I got info for about 100 books. The info varied but included the title, author, publisher, and sometimes things like the number of pages or links to small, medium and large images of the book cover. This is off to a decent start.

Since the data comes back as JSON, I thought it would be easy and fun to store this in a mongo database. Also since I didn’t exactly know what I was going to do with it, and since each book seemed to have slightly different data available, using mongo allows me to avoid having to design a schema yet.

I wrote up some more ruby to run over the saved JSON files and insert each book into a mongo db. Next up, write up a quick Sinatra front end to allow me to browse and then perhaps add more data to each book.

Also interested in finding other data providers for ISBN lookups to fill in the gaps of Open Library.

Open Library API Documentation

# echo 1 > /proc/sys/net/ipv4/tcp_low_latency
# echo 1 > /proc/sys/net/ipv4/tcp_sack
# echo 1 > /proc/sys/net/ipv4/tcp_timestamps
# echo 1 > /proc/sys/net/ipv4/tcp_window_scaling

and once latency went down I did this to get it to stick across reboots:

# echo 'net.ipv4.tcp_low_latency = 1' >> /etc/sysctl.conf
# echo 'net.ipv4.tcp_window_scaling = 1' >> /etc/sysctl.conf
# echo 'net.ipv4.tcp_timestamps = 1' >> /etc/sysctl.conf
# echo 'net.ipv4.tcp_sack = 1' >> /etc/sysctl.conf

UPDATE

So the latency came back a little while later and reappying these commands again might have brough some relief but the latency kept jumping back up. After swapping the network cable things seem to hovering at a brisk 1ms. Lesson learned: It’s not always your configuration.

I found a suggestion to add someone’s repo to apt but there was some sort of build error when I tried to install these so I opted to build from source.

wget http://download.zeromq.org/zeromq-2.1.11.tar.gz
./configure --with-pgm
make
sudo make install

There are a few other languages that zmq supports but this is enough to get me started.

Built language support for python

sudo apt-get install python-dev
sudo easy_install pyzmq

and php

sudo apt-get install php-pear
sudo pear channel-discover pear.zero.mq
sudo pecl install zero.mq/zmq-beta 

and ruby

# I use rvm so don't sudo gem
gem install zmq

and perl

sudo apt-get install libperl-dev
cpan ZeroMQ

which turned into a yak hunt as I hadn’t get up cpan before. cpan install YAML

TODO: Build something with zmq!

Flying is learning how to throw yourself at the ground and miss.

Watch out ground.