Fixes in this downtime include:
- Bug 443979 - talos collects memory information on mac in KB, on linux/win in Bytes. Talos collects various memory metrics (RSS, Private Bytes, Memory Set Size) to get a feel for how much space the browser is taking up. I realized a little while back that we are collecting values in bytes for linux & windows and in kilobytes on mac. This patch will get us to a state where everything is stored in bytes. The downside is that there is going to be a really big jump in the mac memory allocation graphs as we switch from kilobytes to bytes. Mark and I played around with doing a database update to make for a smooth transition, but this would end up requiring shutting down the tree for longer than is comfortable (potentially over a day without any mac Talos boxes reporting). So, we are going to live with the jump in the graphs to get this fixed.
- Bug 459598 - “medians” for individual pages in Tp don’t seem to be medians. This is a potentially large change, but I have high hopes that it won’t affect reported Talos numbers too drastically. The basic issue is that the pageloader extension that Talos uses to cycle through web page test sets has been sorting the list of numerical results as if they were strings. Orderings can end up like (’100′, ‘1900, ‘900′, ‘999′, …), throwing off the median result for a given set of numbers. I think that we are hitting mis-sortings pretty rarely, but we won’t know for sure until the fix is applied to the production boxes and new numbers are generated. The worst case scenario would be changes across the board to all Talos results. Whatever the new results will be they will be considered the new baseline for performance data.
- Bug 457885 - windows talos machines stopped testing anything for quite a few hours. The current way of determining if a new build is available to test is to scrape the tinderbox waterfall for builds marked ’success’. A fine system when we only had a single build machine reports to a single waterfall column. As we have expanded the amount of builders reporting to a single column Talos skips builds - a builder reports a successful build to the waterfall but, before Talos has a chance to process that change, another builder overwrites that build report with its own successful build data. To avoid this we are changing to a system where we monitor the ftp directories where builds are dropped. I’ve tested extensively on staging and this should be a smooth transition.
Firefox, Firefox3.0 & Mozilla1.8 will remain closed until I’m confident that these patches have applied correctly and that the reported Talos numbers are stable.