Brendan’s frustrations getting the Systems Performance book started.
Spoiler: the book did eventually get started, and finished.
You can order the book from InformIT or Amazon.
Category Archives: what I do
Why a Systems Performance Book?
Using videos to promote books has become standard practice in the last few years, as Brendan Gregg and I were well aware while he was writing his latest. We’ve made many videos together, so we figured it would be easy to film a series of brief videos, say one per chapter, during the writing of this book.
It didn’t quite work out that way, but we got a few, starting with the one above.
ps You can order the book from InformIT or Amazon.
Archived Technical Videos
In early 2010, with the “change in control” of Sun Microsystems (to Oracle) imminent, I started a big job of moving the hundreds of technical videos I had spread among three different Sun media hosting sites. Oracle didn’t want them (and soon afterwards deleted dozens, without warning and with no hope of recovery).
One of the destination sites I chose was blip.tv, partly because it allowed me to store longer videos than YouTube did back then.
blip has since changed its business model, and recently gave me a month’s warning to back up my videos before they deleted my free account. In the meantime, YouTube has given me more and more for free – with a long track record and, as of today, 699 videos, they seem to think I’m a good customer even though I’m not paying.
You can visit my YouTube channel for a broad assortment of technical, personal, and travel videos.
I did manage to download everything from blip, but some of the videos are of doubtful interest to my audience today, and I’ve found that uploading old videos to YouTube can confuse my channel subscribers. So the list below is of videos I still have stored safely away, but am not sure there’s a market for. If there’s something on this list you want to see re-published or would like a personal copy of, let me know in the comments below.
I do have a few more than these whose descriptions I forgot to copy down. If you know or suspect that I have video of you that you’d like a copy of, also ask in the comments below. With luck, it won’t be one of the ones that was obliterated by Oracle, and I may still be able to find it somewhere.
Introduction to Parallel Programming: Modules 4-7 found and uploaded, see them here.
Deirdres-module6402
Parallel Programming Models – Shared Memory, Auto Parallel, OpenMP
Module 6 of 7 in “An Introduction To Parallel Programming”. A series of seven video modules presented by Oracle Senior Staff Engineer Ruud van der Pas, covering various aspects of parallel programming in C, C++, and Fortran on multi-core and multi-processor systems.
Deirdres-module5724.mov
Parallel Programming Models – Distributed Memory and MPI
Module 5 of 7 in “An Introduction To Parallel Programming”. A series of seven video modules presented by Oracle Senior Staff Engineer Ruud van der Pas, covering various aspects of parallel programming in C, C++, and Fortran on multi-core and multi-processor systems.
Deirdres-module4395.mov
Parallel Programming Basics
Module 4 of 7 in “An Introduction To Parallel Programming”. A series of seven video modules presented by Oracle Senior Staff Engineer Ruud van der Pas, covering various aspects of parallel programming in C, C++, and Fortran on multi-core and multi-processor systems.
module3189.mov
Parallel Architectures
Module 3 of 7 in “An Introduction To Parallel Programming”. A series of seven video modules presented by Oracle Senior Staff Engineer Ruud van der Pas, covering various aspects of parallel programming in C, C++, and Fortran on multi-core and multi-processor systems.
Deirdres-1003multicorekarlsson975
Developing in a Multicore World – 1
Peter Karlsson presents at Tech Days Hyderabad 2010. Rough cut.
Deirdres-1003multicorekarlsson2665.mov
Developing in a Multicore World – 2
Deirdres-0910osdevcondeadends1h264953.mov
Deirdres-0910osdevcondeadends1h264pt2322
Dead ends in multithreads application development – part 2 of 2
Ten years back, there were difficult to have access to multi CPU machine and software developers preferred to write single thread and single process application. Nowadays computers have a lot of CPUs which gives to us possibility to use them for real parallel computing and improve application performance. You can see Niagara boxes which has 128CPUs (HW threads) in 1U rack size (e.g. Sun SPARC Enterprise T5140), or many people already have Intel or AMD multi core boxes at home. However, because most of application were written in single CPU era they are not able use capability of todays computers. Write multithreaded application is more complicated and we can se that many parallel application are buggy and contains many strange constructions which cause to deadlock or race conditions. Most of these bug of C application are caused by unfamiliarity with POSIX thread specification and bad application design. Multi threaded programing is complex topic and the presentation will focus only on few areas which author had opportunity to hunting bugs. The main areas are: 1. Signal handling in multi threaded application 2. How to survive after fork 3. How to safety exit otherwise when thread is killed 4. Parallel memory access – when we need lock and how to avoid performance drop. 5. Sessions Presentation will contain examples of real bugs which author fixed in pkcs11 library including small Solaris’es libc code tour with stops at fork(), atfork(), atextit(), exit() and pthread.h. Zdenek Kotala has twenty years experience of computer programing. He graduated from West Bohemia University in Numeric methods and parallel computing. He worked on several projects which most of them runs on multi CPUs systems and on different OS (Solaris, Tru64, Windows and Linux). He spent short time of creating and lot of time hunting race conditions, deadlock and other parallel programing issues. At this moment author works as revenue product engineer for SUN Microsystems, Czech Republic. He focus on security technology like PKCS11 library, ssh, openssl and he is also responsible for PostgreSQL in Solaris. He is active member of PostgreSQL community.
Deirdres-MultilevelCluster349.mov
Multilevel Cluster
Ellard Roush presents at the Solaris Security Summit, Nov 2009.
Deirdres-MaximizingApplicationPerformanceWithSunStudio186.mov
Maximizing Application Performance with Sun Studio
Performance on your mind? Creating native language applications that maximize performance requires performance tuning and program analysis.
This session will take a look at Sun Studio software and how to use its optimizing compilers, powerful debuggers, and advanced thread and performance analysis tools to help ensure scalability and the most performance out of your applications on the latest multicore SPARC and x64/x86 processor-based systems. In addition, discover new tools that take advantage of technologies in OpenSolaris, including Dynamic Tracing (DTrace) technology.
Deirdres-Sol11panel1120.mov
Solaris 11 Engineering Panel
Birds of a feather session at LISA10.
Deirdres-module2255.mov
Multicore Processor Architectures
Module 1 of 7 in “An Introduction To Parallel Programming”. A series of seven video modules presented by Oracle Senior Staff Engineer Ruud van der Pas, covering various aspects of parallel programming in C, C++, and Fortran on multi-core and multi-processor systems.
Deirdres-1003TDHydsharatcplus1561.mov
Building High Quality C++ Applications 1
Sharath Srinivasan speaks at Tech Days, Hyderabad, 2010.
Deirdres-0912TDBrmulticore130.mov
Mastering Your Multicore System
This session will demonstrate how Sun Studio compilers and tools can simplify these challenges and enable you to fully unlock the potential in multicore architecture. Don Kretsch presents at Tech Days, Brazil, 2009.
Deirdres-BuildingHighQualityCCApplications826.mov
Building High-Quality C/C++ Applications
There are certain challenges in our industry for native language developers, such as multicore development, heterogeneous OpenSolaris and Linux OS development, and Linux compatibility issues. Sun Studio software delivers a high-performance, optimizing C/C++ and Fortran developer tool chain for Solaris, OpenSolaris, and Linux platforms, including support for the latest multicore systems. The tool chain includes parallelizing compilers, code-level and memory debuggers, performance and thread analysis tools, optimized math libraries, and support for the latest parallelizing industry standards. With a next-generation IDE, developing and debugging applications for the multicore era has never been easier.
Don Kretsch presents at Sun Tech Days, Brazil, December 2009.
Deirdres-sge1382.mov
Managing Grid Engine Clusters 1
Dan Templeton leads a master class in Managing Grid Engine Clusters. Filmed in May, 2008, at the Open Source Grid & Cluster Conference.
Deirdres-0910oowglynnsj629.mov
Contributing to OpenSolaris Using SourceJuicer
Glynn Foster at Oracle Open World 2009.
Deirdres-0910osdevconsourcejuicer1135.mov
Source Juicer – A New Way to Build Solaris Software
Source Juicer (http://jucr.opensolaris.org) is a recently deployed Opensolaris website who’s purpose it is to encourage community porting efforts for the solaris platform. Opensolaris now has two community governed IPS package respoitories, /pending and /contrib and Source Juicer delivers software into these repos. Source Juicer has two main components, the first is a web application that is responsible for accepting and managing community initiated package submissions. When a submission is made, a review thread is created allowing for community members comments and review of the package. At the same time as generating the review thread, the system attempts to build the package and if successful, it is published into the /pending repository. Once the submission has received two positive votes from designated approvers the package is promoted to the /contrib repository, where it is available as a community supported package. The second component is called BuildGrid, which builds and publishes packages. BuildGrid uses a number of Opensolaris technologies, namely ZFS, Zones, IPS & Pkgbuild to build the packages. The BuildGrid requires at least one spec file and one copyright file to successfully build a package. Its scalable and robust architecture allows for the continuous and simultaneous building of packages for different releases and different architectures. Included in the future plans of the Source Juicer development team is the goal to migrate the current release engineering process for the desktop consolidation to the Source Juicer application. This will provide the twin advantages of enhancing community involvement with the Desktop components of Opensolaris and moving away from the existing lengthy and cumbersome nightly builds to a continuous package-by-package build. Also covered in the talk will be a a look at the origination of the project, a detailed look at the architecture and a brief walk-through of the creating and reviewing of a submission, plus the building and installing of a package. Brian Nitz has been a software engineer since 1988. His contributions include support and service productivity tools for radiology workstations, QA and performance tools the successful deployment of over 7000 Sun JDS (Linux) desktops at a large bank, a multidatabase defect management system and components of the sourcejuicer package build web service He lives in Ireland with his wife and two kids where he enjoys travel, sailing and photography. Mark Duggan has worked since 1990 as a contract system administrator/integrator in Ireland and in the US, focusing on thin client solutions and Solaris/Linux backend services. Since joining Sun’s Dublin based Desktop Engineering group in 2000, he has been a member of the desktop QA/RE team. Besides this engineering work he has also had a special focus on the promotion of Sun’s desktop products, in the form of customer facing presentations and proof of concept pilots. In the past year, he has been responsible, as project lead, for the community oriented Source Juicer project, involved at all stages from conception, design, development and deployment of web application and backend build system.
Fibre Channel Concepts
Sit in on a Sun training session on “Fibre Channel Concepts,” by Sumit Gupta.
The 3rd Annual Solaris Family Reunion
I have been meaning for some time to write a follow-up to my series about my experience of Oracle’s acquisition of Sun Microsystems. This yet-to-be-written piece, titled “The Sun Diaspora”, may someday contain deep thoughts about how dispersing all that amazing Sun engineering talent may prove to have been a watershed event for the entire technology industry. That history is still playing out (and probably will for decades to come), and I’m too ragged right now to write it anyway.
So… I did something better: I got some of those engineers to speak for themselves, about what they’ve been doing since Sun. The occasion was the 3rd Annual Solaris Family Reunion, held last night at Joyent (to whom we owe thanks for the pizza and beer). Here’s the rough video:
Speakers:
- Brendan Gregg
- Eric Schrock
- Adam Leventhal
- Keith Wesolowski
- Robert Mustacchi
- Max Bruning
- Garrett D’Amore
- Kevin Zimmerman
- Bill Moore
- Sunay Tripathi
- Blake Jones
This was also the occasion for a get-together of a good portion of Sun’s Fishworks team:
- Part 1: Resistance is Futile: The Oracle Acquisition
- Part 2: What to Expect When You’re Expecting – to Be Acquired
- Part 3: Fishworks and Me
- Part 4: Into the Belly of the Beast
- Part 5: The Last of OpenSolaris
- Part 6: Diaspora (not yet written)
- The 3rd Annual Solaris Family Reunion
You might also like:
A Twitter Campaign: Social Media Marketing for Tech
One of the things I do in my job (and my life) is social media. I’ve been communicating online since 1982, and have tried just about every major online communications tool to come along during that time, starting with CompuServe chat and BBSes in the early 80s. I began using Twitter in 2007; it has grown to be an important tool for my work as well. Yes, I do social media marketing (though I wince at the term).
As of today, I have about 1,947 followers on my personal Twitter account, @deirdres, and 583 on @JoyentSmartOS (I post technical news to both). These aren’t large numbers – plenty of famous people, non-famous people, companies, brands, organizations, and other entities, have many thousands or millions more. But I use Twitter well and get results: web traffic, attention, and sales for my employer, Joyent. Here’s a recent example that may be useful to fellow social media marketers.
The Event
My colleague Brendan Gregg is a well-known speaker on computer systems performance topics, but he’s best known – so far – in the world of Solaris, SmartOS, and company. We have worked together for years: he creates great technical content; I edit, film, and market it. On February 24th, 2013, he gave a talk at SCaLE on Linux Performance Analysis and Tools, which I went along to film. As his first talk about Linux, it presented both an opportunity and a difficulty.
Having other obligations in LA after the event, we only got back to the office on Wednesday, Feb 27th, but I had the video edited and uploaded to YouTube the morning of Friday, March 1st. By then I had also written a post and provided photos for the Joyent blog. Even by my standards, this was fast work. Why was I in such a hurry?
First, the talk had been popular: the 250-seat room was packed, with a very engaged audience who wanted to be able to re-read the presentation slide deck (dense with technical information) immediately. As is his wont, Brendan had put it up on Slideshare within an hour of finishing his talk, and announced it on Twitter:
The slide deck got lots of attention, not just the retweets shown above (some from SCaLE attendees), but hundreds more tweets and thousands of views, most of them from people who had likely never heard of Brendan or Joyent.
The hail of tweets showed the reach of the (very large) Linux community. We were already aware of this but, as a SmartOS/Solaris shop, Joyent doesn’t often get a chance to tap into it.
I wanted to bring some of that attention to Joyent while the talk was still fresh in the minds of attendees and others – some of whom were already asking for video. I could have just posted it on my YouTube channel and told everybody it was there, but the company would benefit more from traffic to the Joyent site – hence the blog post.
We published that post as soon as the video was uploaded, on Friday morning, March 1st. We knew that that was not an ideal time: Friday morning in California is already the weekend in most of the rest of the world. Even so, the lack of response was frustrating. I started looking for more ways to draw attention to the post.
The Social Media Push
That same day, an article was published about Oracle’s Port of DTrace on Linux. Its author – and many others – tweeted about it. Since Brendan’s talk had included DTrace as a performance analysis tool for Linux, I tried to ride that wave using the same hashtags:
I also directed a tweet at Joab Jackson, the journalist, who retweeted it:
In the meantime, I scrolled back to many of the people who had originally tweeted about Brendan’s slide deck, and tweeted to them directly, one by one: “re Linux Performance Analysis and Tools – Video available now, too! http://ow.ly/idrWY ”
I approached this with caution: in every online marketing activity I undertake, I try hard to give information only where and to whom it is likely to be welcome. In this case, no one complained, and some even thanked me for the news: “moustafa_dba 12:25pm via Twitter for iPad @DeirdreS great ,thank you so much”
But, by Saturday morning, I still wasn’t satisfied with the traffic to the blog post and the video. I had shot my bolt on Twitter. My other usual forums are mostly populated with SmartOS/illumos/Solaris folks – not the primary audience for this talk. Where could I reach other technical people?
We’d had good results on HackerNews before, so I posted it there, and tweeted about it:
Blatantly asking people to vote it up worked: the post remained in the top 11 or higher on HN’s front page for at least seven hours, and garnered some nice comments. Someone else – who perhaps saw it on HN – posted it to reddit.
These two sites drove a spike of traffic to the post on Joyent.com over the weekend: 5,768 views from HN and 4,098 from reddit.
Altogether, from March 1st to today we’ve had:
- blog post page: 32,000 page views (with an average time on page of 32 minutes, thanks to the video)
- video: 6,332 views
- slide deck: 67,000 views; 1,707 downloads
Of greater importance in the long term: by the time the post had only been up for 18 hours, it was high in Google search results for “Linux Performance”. It is now the number one result. Which is ironic and amusing, considering that Joyent’s cloud is famously built upon a different operating system, SmartOS (though we also support Linux). It also subtly demonstrates that, when Joyent claims to provide high-performance cloud infrastructure, we know what we’re talking about. Or at least Brendan Gregg does. 😉
Coda: As of March, 2014, Brendan works at Netflix, mostly on Linux performance.
