Advanced Distributed Systems

Fall 2015

General Course Information:

Instructor: Lakshmish Ramaswamy (laks[AT]cs[dot]uga[dot]edu, 706-542-2737)
 
Time and Venue: Mondays - 11:15 AM to 12:05 PM (Boyd 306) ; Tuesdays & Thursdays - 11:00 AM to 12:15 PM (Boyd 201)
 
Office Hours: Tuesdays and Thursdays - 12:15 PM to 1:00 PM (Tentative) or by appointment

Course Description:

Distributed systems have become widely pervasive and are having a tremendous impact on various domains of human activity. Today's distributed systems range from ad-hoc networks comprising of tiny sensor devices, to overlay networks such as peer-to-peer systems, to massive web farms of powerful servers. The research in the area of distributed systems has focused on achieving better performance, reliability, security, and privacy of various kinds of distributed data processing applications.

In this course we will discuss advanced topics on distributed system principles including Distributed Hash Tables, Paxos Protocol, and CAP Theorem. Furthermore, we will study the design, implementation and evaluation of a wide class of distributed systems and applications including distributed file systems, cloud computing systems, event-based systems, sensor networks, and pervasive applications, with the objective of gaining an in-depth understanding of the requirements and the design options.

Grading Policy (Tentative)

Course Materials (Tentative -- Will be modified during the course of the semester)

Advanced  Topics on Distributed System Principles (Lakshmish)
  1.   Material from the book "Distributed Systems: Principles and Paradigms" by Tanenbaum and Van Steen, 2nd Ed. Pearson Prentice Hall.**
  2. I. Stoica, R. Morris, D. Liben-Nowell, D. Karger, M. Kaashoek, F. Dabek and H. Balakrishnan, Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications, IEEE/ACM Transactions on Networking, 11(1), 2003. **
  3. A. Broder and M. Mitzenmacher, "Network Applications of Bloom Filters: A Survey", Internet Mathematics, 1(4), 2004.**
  4. L. Lamport, "Paxos Made Simple", 2001.**
  5. D. Mazieres "Paxos Made Practical" **
Distributed File Systems and Lock Services
  1. Material from the book "Distributed Systems: Principles and Paradigms" by Tanenbaum and Van Steen, 2nd Ed. Pearson Prentice Hall.**
  2. S. Ghemawat, H. Gobioff and S-T Leung, "The Google File System", OSDI 2003 (Shengming and Jiang -- 10/08).
  3. M. Burrows, "The Chubby Lock Service for Loosely-Coupled Distributed Systems", OSDI 2006 (Seyedamin and Shibo -- 10/13).
  4. S. A. Weil et al. "Ceph: A Scalable, High -Performance Distributed FIle System", OSDI 2006 (Bita -- 10/20).
  5. C. Dubnicki et al. "HYDRAstor: A Scalable Secondary Storage", FAST 2009 (Chaitali and Meenaz -- 10/13).
  6. K. Shvachko et al. "The Hadoop Distributed File System", MSST 2010 (Rick and Vic -- 10/19).
  7. D. Ford et al., "Avaialability in Globally Distributed Storage Systems", OSDI 2010 (Michael and Anumita -- 10/22) .
  8. W. Lloyd et al., "Don't Settle for Eventual: Scalable Causal Consistency for Wide-Area Storage with COPS", SOSP 2011 (Talal and Urvashi -- 10/20
  9. B. Calder et al., "Windows Azure Storage: A Highly Available Cloud Storage Service with Strong Consistency", SOSP 2011 (Alekhya and Moumita -- 10/12).
  10. A. Thomson and D. J. Abadi "CalvinFS: Consistent WAN Replication and Scalable Metadata Management for Distributed File Systems", USENIX FAST 2015 (Shengming and Jiang -- 10/22).
Data Centers and Cloud Computing
  1. Materials from the book "Distributed and Cloud Computing: From Parallel and Processing to the Internet of Things", by K. Hwang and G. C. Fox and J. J. Dongarra, Elsevier & Morgan-Kauffman (Lakshmish)
  2. M. Armburst et al, "Above the Clouds: A Berkeley View of Cloud Computing", Technical Report, University of California, Berkeley, 2009 (Seyedamin and Shibo -- 10/26).
  3. A. Greenberg et al., "VL2: A Scalable and Flexible Data Center Network", SIGCOMM 2009 (Shengming and Jiang -- 10/26).
  4. B. Heller et al., "ElasticTree; Saving Energy in Data Center Networks", NSDI 2010 (Bita -- 10/27)
  5. S. Meng et al., "Volley: Violation Likelihood Based State Monitoring for Datacenters", ICDCS 2013 (Sahar and Mohamad -- 10/27)
  6. S. Kannan, A. Gavrilovska and K. Schwan, "Cloud4Home - Enhancing Data Services with @Home Clouds", ICDCS 2011 (Abbas and Navid -- 10/29)
  7. D. Jeswani, A. Verma, P. Jayachandra and K. Bhattacharya, "ImageElves: Rapid and Reliable System Updates in the Cloud", ICDCS 2013 (Michael and Anumita -- 10/29).
  8. R. Balani et al. "Columbus: Configuration Discovery for Clouds", ICDCS 2014 (Manish and Narita -- 11/03).
  9. K. Razavi et al., "Prebaked uVMs: Scalable, Instant VM Startup for IaaS Clouds", ICDCS 2015 (Alekhya and Moumita -- 11/2).
  10. B. Hindman et al., Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center, NSDI 2011 (Manish and Narita -- 11/2).
Event Processing and Publish-Subscribe Systems
  1. P. Th. Eugster, P. A. Felber, R. Guerraoui and A.-M. Kermarrec, The Many Faces of Publish-Subscribe, ACM Computing Surveys, 2003 (Lakshmish -- 11/3).
  2. G. Cugola and A. Margara "Processing Flows of Information: From Data Streams to Complex Event Processing", ACM Computing Surveys, 2012 (Lakshmish).
  3. M. Caporusico et al. Design and Evaluation of a Support Service for Mobile, Wireless Publish/Subscribe Applications", IEEE Trans. of Software Engineering, December 2003 (Navid and Abbas -- 11/05)
  4. G. Li and H-A Jacobsen "Composite Subscriptiosn in Content-Based Publish-Subscribe Systems", MIDDLEWARE 2005 (Chaitali and Meenaz -- 11/5).
  5. A. Demers et al. "Towards Expressive Publish-Subscribe Systems", EDBT 2006 (Manish and Narita --11/12).
  6. I. Rose, et al.,  "Cobra: Content-based Filtering and Aggregation of Blogs and RSS Feeds", NSDI 2007 (Alekhya and Moumita -- 11/12).
  7. E. Wu et al., "High-Performance Complex Event Processing Over Streams", SIGMOD 2006.
  8. L. Woods et al., "Complex Event Detection at Wire Speed with FPGAs", VLDB 2010 (Liuyang and Zihe -- 11/12)
Wireless Sensor Networks
  1. J. Yick et al. "Wireless Sensor Networks Survey", Computer Networks, 2008 (Seyedamin and Shibo -- 11/16)
  2. J. Liu et al. "Energy Efficient GPS Sensing with Cloud Offloading", ACM Sensys 2012 (Talal and Urvashi -- 11/16).
  3. R. Sen et al. "Kyun Queue: A Sensor Network System To Monitor Road Traffic Queues", ACM SenSys 2012 (Sahar and Mohamad -- 11/17)
  4. Y. Xiang et al. "Collaborative Calibration and Sensor Placement for Mobile Sensor Networks", IPSN 2012 (Sahar and Mohamad -- 11/30)
  5. S. Dawson-Haggerty et al. "@scale: Insights from a Large, Long-Lived Application Energy WSN", IPSN 2012 (Vic and Rick -- 11/17)
  6. V. Srinivasan et al. "FixtureFinder: Discovering the Existence of Electrical and Water Fixtures", IPSN 2013.
  7. C. Boano et al. "TempLab: A Testbed Infrastructure to Study the Impact of Temperature on Wireless Sensor Networks", IPSN 2014.
  8. F. Lai et al. "CSense: A Stream-Processing Toolkit for Robust and High-rate Mobile Health Systems", IPSN 2014.
  9. W. Du et al. "Optimal Sensor Placement and Measurement of Wind for Water Quality Studies in Urban Reserviors", IPSN 2014.
Pervasive Applications/Human Sensing/Social Sensing
  1. L. Atzori, A. Iera and G. Morabito, "The Internet of Things: A Survey", Computer Networks 2010 (Vic and Rick -- 11/19).
  2. P. Zhou et al. "Urban Traffic Monitoring with the Help of Bus Riders", ICDCS 2015 (Navid and Abbas -- 11/30).
  3. R. Ganti et al. "On Limits of Travel Time Predictions: Insights from a New York City Case Study", ICDCS 2014.
  4. N. Roy et al. "I am Smartphone and I can Tell My User's Walking Direction", MobiSys 2014 (Talal and Urvashi -- 12/01)
  5. Y. Wang et al. "Tracking Human Queues Using Single-Point Monitoring", MobiSys 2014 (Zihe and Liuyang -- 12/01)
  6. L. Zhang et al. "AccelWord: Energy Efficient Hotword Detection through Accelerometer", MobiSys 2015 (Meenaz and Chatali -- 12/07)
  7. R. Nandakumar et al. "Contactless Sleep Apnea Diagnosis on Smartphones", MobiSys 2015.
  8. D. Wang et al. "Using Humans as Sensors: An Estimation-theoretic Perspective", IPSN 2014 (Zihe and Liuyang -- 12/07)

Presentation Slides

Available on ELC.

Programming Projects

2 Phase Distributed Commit   Sample Test Files -- Due Date: 11/06/2015.

Miscellanious Materials