Tag Archives: video

DTracing the Cloud

Brendan Gregg at illumos Day (an event I also organized and ran).

Cloud computing facilitates rapid deployment and scaling, often pushing high load at applications under continual development. DTrace allows immediate analysis of issues on live production systems even in these demanding environments – no need to restart or run a special debug kernel. For the illumos kernel, DTrace has been enhanced to support cloud computing, providing more observation capabilities to zones as used by Joyent SmartMachine customers. DTrace is also frequently used by the cloud operators to analyze systems and verify performance isolation of tenants. This talk covers DTrace in the illumos-based cloud, showing examples of real-world performance wins.

Why 4K?

George Wilson at ZFS Day

For over 30 years, hard drives have designated the smallest storage location as 512 bytes. In January 2011, all major hard drive manufactures began shipping their hard drive platforms using a new standard called Advanced Format. To aid in the transition, these new hard drives provide a 512 byte emulation mode that allows the drives to advertise themselves as a 512 byte addressable devices. This can severely impact write performance resulting in the need for read-modify-write operations for any misaligned or partial writes that are issued.

The problem is not limited to just physical hardware. Other storage platforms may also provide LUNs (logical unit number) that presents themselves as a 512 byte addressable devices when, in fact, they use a 4K sector size internally. Although ZFS has built-in support for 4K sectors, it has no automatic way of dealing with the lies that the storage devices tell. This talk will focus on the methods that have been developed to work around the lies that hard drive storage platforms tell and will discuss the challenges and drawbacks that come with using 4K sectors.

Performance Analysis: The USE Method

Brendan Gregg’s talk at FISL, July 2012.

This talk introduces the USE Method: a simple strategy for performing a complete check of system performance health, identifying common bottlenecks and errors. This methodology can be used early in a performance investigation to quickly identify the most severe system performance issues, and is a methodology the speaker has used successfully for years in both enterprise and cloud computing environments. Checklists have been developed to show how the USE Method can be applied to Solaris/illumos-based and Linux-based systems.

Many hardware and software resource types have been commonly overlooked, including memory and I/O busses, CPU interconnects, and kernel locks. Any of these can become a system bottleneck. The USE Method provides a way to find and identify these.

This approach focuses on the questions to ask of the system, before reaching for the tools. Tools that are ultimately used include all the standard performance tools (vmstat, iostat, top), and more advanced tools, including dynamic tracing (DTrace), and hardware performance counters.

Other performance methodologies are included for comparison: the Problem Statement Method, Workload Characterization Method, and Drill-Down Analysis Method.

Index of Videos

My videos through 2007, listed in their original order, newest first. Click a title to visit the related page, or use the YouTube link to watch the video.

  1. Beginning with I: The Trailer
  2. Landour Snow, Dec 10, 2007 — Watch on YouTube
  3. Woodstock Arts Evening, Dec 8, 2007 — Watch on YouTube
  4. Woodstock School Jazz Band, Dec 3, 2007 — Watch on YouTube
  5. Woodstock School Christmas Chapel Concert Dec 2, 2007
  6. Landour Wedding Party, Dec 1, 2007 — Watch on YouTube
  7. “Taming of the Shrew” Wedding Dance, Nov 16, 2007
  8. Hummingbirds in New Mexico – Aug, 2007 — Watch on YouTube
  9. Roasting Green Chiles – Aug 17, 2007 — Watch on YouTube
  10. Tin Tin Su and Our Lady of Drosophila – Jul 15, 2007 — Watch on YouTube
  11. Song: Amo Tutte le Signore – Jul 15, 2007 — Watch on YouTube
  12. Fanfara dei Bersaglieri – Jul 3, 2007 — Watch on YouTube
  13. Getting Girls Into Science Early – Jun 3, 2007 — Watch on YouTube
  14. ‘E Cal’arroste — Watch on YouTube
  15. Il Castello di Vezio – Apr 28, 2007 — Watch on YouTube
  16. Il Muro di Sormano – Jan 31, 2007 — Watch on YouTube
  17. Windy Sunday on Lake Como – Oct 9, 2006 — Watch on YouTube
  18. The Italian Correspondent (part of the Node 666 video collaboration – fiction!) Sep, 2006 – notable as the only time, so far, that I have appeared in front of the camera! – YouTube
  19. Parc Guell – Aug 16, 2006
  20. Casa Batlló – May 21, 2006 — Watch on YouTube
  21. The Streets of Barcelona – May 1, 2006 — Watch on YouTube
  22. Good Friday, Jesus Christ Superstar – Apr 25, 2006
  23. Post-Funeral Jazz and Barbecue – Apr 12, 2006 — Watch on YouTube
  24. Funeral Recessional – Apr 1, 2006 — Watch on YouTube
  25. Graveside Remarks – Apr 11, 2006
  26. Funeral Processional – Apr 1, 2006 — Watch on YouTube
  27. The Giving Tree – Apr 1, 2006 — Watch on YouTube
  28. Dad’s Eulogy for Rosie – Apr 1, 2006
  29. What I Said at Rosie’s Funeral – Apr 1, 2006
  30. Mantova – Mar 6, 2006 — Watch on YouTube
  31. Domo Arigato – Feb 18, 2006 (filmed Jan 7, 2006) — Watch on YouTube
  32. Video Comment on Current Events – Feb 12, 2006 — Watch on YouTube
  33. Video Postcard from Lecco – Feb 11, 2006
  34. Heavy Snowfall in Lecco Jan 27, 2006 — Watch on YouTube
  35. In the Kitchen at Spankyville – Jan 3, 2006 — Watch on YouTube
  36. You Can Leave Your Cat On – Dec 30, 2005
  37. Flying Over the Swiss Alps – Dec 26, 2005 — Watch on YouTube
  38. Wind-Up Soul — Watch on YouTube
  39. Woodstock Reminiscences on Video — Watch on YouTube
  40. Christmas in Lecco – Dec 24, 2005 — Watch on YouTube
  41. Italian Christmas Carols – Dec 4, 2005 — Watch on YouTube
  42. La Transumanza – Nov 19, 2005 — Watch on YouTube
  43. Video Postcard from Villa Monastero – Oct 16, 2005 — Watch on YouTube
  44. Sunday Sing-Along – Oct 2, 2005
  45. Corteo Manzoniano 2005 – Oct 1, 2005 — Watch on YouTube
  46. House Tortoise – Aug 15, 2005
  47. Mehndi Hand Painting – Aug 14, 2005 — Watch on YouTube
  48. Transportation in Jaipur – Aug 9, 2005 — Watch on YouTube
  49. English Words in Hindi Script – Aug 9, 2005 — Watch on YouTube
  50. Teej Festival – Aug 8, 2005 — Watch on YouTube
  51. How to Tie a Turban – Aug 8, 2005 — Watch on YouTube
  52. Jantr Mantr – Aug 7, 2005 — Watch on YouTube
  53. Handprinting Cloth Demonstration – Aug 6, 2005 — Watch on YouTube
  54. Elephant Ride to Amber Fort, Jaipur – Aug 6, 2005 — Watch on YouTube
  55. Inside Amber Fort – Aug 6, 2005 — Watch on YouTube
  56. How to Weave a Handloom Carpet – Aug 6, 2005 — Watch on YouTube
  57. Evening Songs at the Samode Haveli – Aug 5, 2005 — Watch on YouTube
  58. Learn the Indian National Anthem – Aug 3 , 2005 — Watch on YouTube
  59. Cicadas – Aug 2, 2005
  60. Tibetan Prayer Wheels – Aug 1, 2005 — Watch on YouTube
  61. Mussoorie Monsoon Melody – Jul 30, 2005 — Watch on YouTube
  62. Tenzing’s Monkey Tales – July 29, 2005 — Watch on YouTube
  63. Old Delhi Station – July 28, 2005 — Watch on YouTube
  64. Old Delhi at Night – July 27, 2005 — Watch on YouTube
  65. Video Postcard from Bellagio – Jul, 2005
  66. Italia Marching Show Bands – Jul 3, 2005 — Watch on YouTube
  67. I Love a Parade – Jul 2, 2005 — Watch on YouTube
  68. Transport Surprises – Jun 25, 2005 — Watch video 1 on YouTube · Watch video 2 on YouTube
  69. MBA Graduation Ceremony – Jun 10, 2005 — Watch on YouTube
  70. And She Was – Jun 7, 2005
  71. A Close Shave – Jun 3, 2005 — Watch on YouTube
  72. Lecco Kayaker – Jun, 2005 — Watch on YouTube
  73. Milan Central Station – May 24, 2005
  74. Kids on Scooters – May 11, 2005
  75. Java – Apr 16, 2005 — Watch on YouTube
  76. The Great Escape – Apr 16, 2005 — Watch on YouTube
  77. The Hundred Years’ War – Mar 10, 2005 — Watch on YouTube
  78. What Am I Doing Here? – Mar 10, 2005 — Watch on YouTube
  79. Commuting Kids – Feb or Mar, 2005
  80. La Bottega del Maiale – Feb 25, 2005 — Watch on YouTube
  81. Spankyville, Subtitled – Feb 13, 2005 — Watch on YouTube
  82. Welcome to Spankyville – Feb 13, 2005 — Watch on YouTube
  83. Winter Wonderland – Jan 22, 2005
  84. Bright Lights, Big City – Jan 10, 2005 — Watch on YouTube
  85. Jewelry Repair – Nov 3, 2004
  86. Jantr Mantr Delhi – Nov 2, 2004
  87. Delhi Bartender – Nov 2, 2004
  88. Delhi Dancers – Nov 2, 2004
  89. Making Rumali Roti – Oct 30, 2004 — Watch on YouTube
  90. Air Tabla – Oct 30, 2004
  91. Sitar Concert with Sanjeeb Sircar – Oct 30, 2004 — Watch on YouTube
  92. Making Jalebis – Oct 30, 2004
  93. Making Naan – Oct 29, 2004
  94. Campfire — Watch on YouTube
  95. Dancing Queen
  96. Presentation and Tea for Dick Wechter – Oct 29, 2004
  97. Midlands with Festive Lights – Oct 28, 2007
  98. Seen from an Indian Train – Oct 28, 2004 — Watch on YouTube
  99. Corteo Manzoniano: The Grim Sweepers – Oct 10, 2004
  100. Corteo Manzoniano: Parade! – Oct 10, 2004
  101. Corteo Manzoniano: La Peste – Oct 10, 2004
  102. Corteo Manzoniano: Procession – Oct 10, 2004
  103. Corteo Manzoniano: The Kidnapping of Lucia – Oct 10, 2004
  104. Corteo Manzoniano: Tradimento! – Oct 10, 2004
  105. Moving In – Sep 27, 2004
  106. Moving Out – Sep 27, 2004 — Watch on YouTube
  107. Volunteers – Sep 25, 2004 — Watch on YouTube
  108. Blues Brothers Come to Lecco – Sep 18, 2004 — Watch on YouTube
  109. Moving the Chandelier – Sep 24, 2004
  110. Funiculi, Funicula – Aug 22, 2004
  111. Ristorante il Capriolo – Sep 6, 2004
  112. Summer Storm Over Lake Como – Aug 20, 2004
  113. Ross & Hamish – Aug 14, 2004
  114. The Ferry Arrives – Aug 14, 2004
  115. Ferry Loading and Departure – Aug 14, 2004
  116. Introducing the Turtles – Aug 3, 2004
  117. Summer Fun, Italian Style – July 27, 2004
  118. On the Road to Siena – July 27, 2004
  119. Italian Gospel – July 20, 2004 — Watch on YouTube
  120. “A Sua Immagine” – July 19, 2004
  121. Wind-Up Soul – Jul, 2003 — Watch on YouTube