The airbnb/chronos scheduler

Chronos at Airbnb: At Airbnb, chronos functions in an environment that includes AWS EMR, MySQL, Amazon Redshift, S3, Cascading, Cascalog, Hive, Pig.  Challenges in this environment include variance in network latency, unpredictable I/O performance, and spurious web services timeouts.  These challenges prompted Airbnb to look for a lightweight scheduling solution that allowed retries and provides high availability and via easy-to-use GUI interface.  In addition, Airbnb wanted the ability to schedule non-Hadoop jobs, such as bash scripts, and distribute work across multiple systems.  Thus Airbnb decided to build Chronos and to leverage Mesos, which provides the required primitives for storing state, distributing work, and adding new workers on the fly.

Chronos UI: The Chronos UI supports adding, deleting, listing, modifying and running jobs. It can show graphs of job dependencies.

Sample-chronos-ui

What people asking: Alerting and notification are not well documented but you can specify email addresses to send job failure notifications. Use comma-separated list for multiple addresses.  It is not obvious whether you can integrate chronos with other business applications, for example, Zenoss, JIRA, Logstash, etc.  Is it adaptable to multi-timezone calendars?  Does it have the ability to create incident reports?  What is the roundtrip time between submitting request and receiving a response?  Does it have reporting features? Does it have the ability to failover and load balance? Are all features available via both command line and GUI?

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s