Bits & Bytes #2 - A different kind of phone service

Posted by: Mahesh Paolini-Subramanya

When we describe Aptela as a different kind of phone service, we don't exactly mean it in the same way that your friendly hotel chain calls itself a "different kind of hotel,” no doubt implying that unlike those other hotels, they really *really* care (and don't make a practice of riffling through your luggage when your back is turned, etc., etc.).  We actually are built different - way down deep at an infrastructure level - in that our servers are truly distributed (cloud-based if you will).  As I described earlier, when calls or web-requests come in, they get picked up by the first available server, any available server.  We don't have Big Iron ™ or racks of individual Small Iron ™ that need to be managed, maintained, programmed, etc.  - just a whole bunch of inexpensive white-box servers that spin themselves up, take themselves down, and do the hokey-pokey, etc.

Needless to say, there is a lot of technology and a somewhat surprising amount of magic involved in getting this all up and running.  Setting up distributed systems is already pretty hard to begin with.   Setting them up to act as a platform for complex voice applications is borderline impossible, with one of the biggest problem being the issue of managing media.  Think about it - when someone calls in and gets an auto attendant, they should hear the same greeting audio regardless of where they call in from, correct?  But, given that the call is being handled by a random server, how does one ensure that the relevant server has access to the correct greeting audio? 

"Aha!,”I hear you say.  "Simple!  Put it on the central fileserver and let all of the systems get it from there!  Haven't you heard about NFS?  Or Samba?"

    Brilliant (and exceedingly obvious) idea, except that this pretty much flies in the face of the whole 'Distributed' thing.  Furthermore, and you knew this was coming, this was a bit of a trick question, the 'trick' part being that I said correct greeting audio, not just greeting audio.  What we are dealing with are complex voice applications, not just simple answering-machine-lite type situations.  The audio that gets played can vary depending on the time of the day, the identity of the caller, the person answering the call, and heck, the phase of the moon.  Consequently, each server needs to not just have access to the audio, but also access to all the metadata about the client in question (like the scheduling rules, caller IDs, moon-phase, etc.).  This argues in favor of not just having a central filesystem, but potentially also having a central database that contains all this data. Which is a bit of a problem, since conventional databases and highly distributed systems tend to get along like houses on fire.  Literally like houses on fire.  Picture plumes of smoke, huge flames, grim people everywhere, and definitely lots of screaming.  Can they be made to work?  Well, of course, in that anything can be made to work.  With sharding, replication, Correct Design, etc., you can get a pretty decent pass at it.  It’s just a monstrously huge engineering challenge, and most importantly, it isn't elegant, and trust me, one when architecting systems, one eventually realizes the golden rule, viz., Simple + Elegant == Good.

    Enter the new generation of document-oriented databases, things like CouchDB, SimpleDB, ThruDB, etc. These are highly distributed databases that tend to map extremely well to the type of problems that we faced when designing and building Aptela.  Typically, these databases share - amongst others - the following characteristics that we so desperately needed:

1. Databases are schema-less, with the data usually stored as a set of Key-Value pairs, which are indexed as necessary.
2. Data can be accessed using simple getters/setters (usually RESTfully).
3. The database is highly fault-tolerant and distributed.
4. They tend to follow BASE and CAP principles instead of ACID.  (Trust me, this is relevant.  Mind you, this could be Bad, depending on your requirements.).

   At Aptela, we use a particular variant thereof called CouchDB (which stands for Cluster of Unreliable Commodity Hardware Data Base) for all of our data storage and persistence needs.  It is written in Erlang (remember Erlang? If not, see my last blog posting) and processes data in the form of chunks of JSON.  In particular, it uses the magic of Map-Reduce to allow for quite complex processing of all our data/meta-data.  Expect more on this site in the coming days.